1CARGO-BENCH(1) General Commands Manual CARGO-BENCH(1)
2
6 cargo-bench - Execute benchmarks of a package
7
9 cargo bench [options] [benchname] [-- bench-options]
10
12 Compile and execute benchmarks.
13 The benchmark filtering argument benchname and all the arguments
15 following the two dashes (--) are passed to the benchmark binaries and
16 thus to libtest (rustc's built in unit-test and micro-benchmarking
17 framework). If you are passing arguments to both Cargo and the binary,
18 the ones after -- go to the binary, the ones before go to Cargo. For
19 details about libtest's arguments see the output of cargo bench --
20 --help and check out the rustc book's chapter on how tests work at
21 <https://doc.rust-lang.org/rustc/tests/index.html>.
22 As an example, this will run only the benchmark named foo (and skip
24 other similarly named benchmarks like foobar):
25 cargo bench -- foo --exact
27 Benchmarks are built with the --test option to rustc which creates a
29 special executable by linking your code with libtest. The executable
30 automatically runs all functions annotated with the #[bench] attribute.
31 Cargo passes the --bench flag to the test harness to tell it to run
32 only benchmarks.
33 The libtest harness may be disabled by setting harness = false in the
35 target manifest settings, in which case your code will need to provide
36 its own main function to handle running benchmarks.
37 Note: The #[bench] attribute
39 <https://doc.rust-lang.org/nightly/unstable-book/library-features/test.html>
40 is currently unstable and only available on the nightly channel
41 <https://doc.rust-lang.org/book/appendix-07-nightly-rust.html>.
42 There are some packages available on crates.io
43 <https://crates.io/keywords/benchmark> that may help with
44 running benchmarks on the stable channel, such as Criterion
45 <https://crates.io/crates/criterion>.
46 By default, cargo bench uses the bench profile
48 <https://doc.rust-lang.org/cargo/reference/profiles.html#bench>, which
49 enables optimizations and disables debugging information. If you need
50 to debug a benchmark, you can use the --profile=dev command-line option
51 to switch to the dev profile. You can then run the debug-enabled
52 benchmark within a debugger.
53
55 Benchmark Options
56 --no-run
57 Compile, but don't run benchmarks.
58 --no-fail-fast
60 Run all benchmarks regardless of failure. Without this flag, Cargo
61 will exit after the first executable fails. The Rust test harness
62 will run all benchmarks within the executable to completion, this
63 flag only applies to the executable as a whole.
64 Package Selection
66 By default, when no package selection options are given, the packages
67 selected depend on the selected manifest file (based on the current
68 working directory if --manifest-path is not given). If the manifest is
69 the root of a workspace then the workspaces default members are
70 selected, otherwise only the package defined by the manifest will be
71 selected.
72 The default members of a workspace can be set explicitly with the
74 workspace.default-members key in the root manifest. If this is not set,
75 a virtual workspace will include all workspace members (equivalent to
76 passing --workspace), and a non-virtual workspace will include only the
77 root crate itself.
78 -p spec..., --package spec...
80 Benchmark only the specified packages. See cargo-pkgid(1) for the
81 SPEC format. This flag may be specified multiple times and supports
82 common Unix glob patterns like *, ? and []. However, to avoid your
83 shell accidentally expanding glob patterns before Cargo handles
84 them, you must use single quotes or double quotes around each
85 pattern.
86 --workspace
88 Benchmark all members in the workspace.
89 --all
91 Deprecated alias for --workspace.
92 --exclude SPEC...
94 Exclude the specified packages. Must be used in conjunction with
95 the --workspace flag. This flag may be specified multiple times and
96 supports common Unix glob patterns like *, ? and []. However, to
97 avoid your shell accidentally expanding glob patterns before Cargo
98 handles them, you must use single quotes or double quotes around
99 each pattern.
100 Target Selection
102 When no target selection options are given, cargo bench will build the
103 following targets of the selected packages:
104 • lib — used to link with binaries and benchmarks
106 • bins (only if benchmark targets are built and required features are
108 available)
109 • lib as a benchmark
111 • bins as benchmarks
113 • benchmark targets
115 The default behavior can be changed by setting the bench flag for the
117 target in the manifest settings. Setting examples to bench = true will
118 build and run the example as a benchmark. Setting targets to bench =
119 false will stop them from being benchmarked by default. Target
120 selection options that take a target by name ignore the bench flag and
121 will always benchmark the given target.
122 Binary targets are automatically built if there is an integration test
124 or benchmark being selected to benchmark. This allows an integration
125 test to execute the binary to exercise and test its behavior. The
126 CARGO_BIN_EXE_<name> environment variable
127 <https://doc.rust-lang.org/cargo/reference/environment-variables.html#environment-variables-cargo-sets-for-crates>
128 is set when the integration test is built so that it can use the env
129 macro <https://doc.rust-lang.org/std/macro.env.html> to locate the
130 executable.
131 Passing target selection flags will benchmark only the specified
133 targets.
134 Note that --bin, --example, --test and --bench flags also support
136 common Unix glob patterns like *, ? and []. However, to avoid your
137 shell accidentally expanding glob patterns before Cargo handles them,
138 you must use single quotes or double quotes around each glob pattern.
139 --lib
141 Benchmark the package's library.
142 --bin name...
144 Benchmark the specified binary. This flag may be specified multiple
145 times and supports common Unix glob patterns.
146 --bins
148 Benchmark all binary targets.
149 --example name...
151 Benchmark the specified example. This flag may be specified
152 multiple times and supports common Unix glob patterns.
153 --examples
155 Benchmark all example targets.
156 --test name...
158 Benchmark the specified integration test. This flag may be
159 specified multiple times and supports common Unix glob patterns.
160 --tests
162 Benchmark all targets in test mode that have the test = true
163 manifest flag set. By default this includes the library and
164 binaries built as unittests, and integration tests. Be aware that
165 this will also build any required dependencies, so the lib target
166 may be built twice (once as a unittest, and once as a dependency
167 for binaries, integration tests, etc.). Targets may be enabled or
168 disabled by setting the test flag in the manifest settings for the
169 target.
170 --bench name...
172 Benchmark the specified benchmark. This flag may be specified
173 multiple times and supports common Unix glob patterns.
174 --benches
176 Benchmark all targets in benchmark mode that have the bench = true
177 manifest flag set. By default this includes the library and
178 binaries built as benchmarks, and bench targets. Be aware that this
179 will also build any required dependencies, so the lib target may be
180 built twice (once as a benchmark, and once as a dependency for
181 binaries, benchmarks, etc.). Targets may be enabled or disabled by
182 setting the bench flag in the manifest settings for the target.
183 --all-targets
185 Benchmark all targets. This is equivalent to specifying --lib
186 --bins --tests --benches --examples.
187 Feature Selection
189 The feature flags allow you to control which features are enabled. When
190 no feature options are given, the default feature is activated for
191 every selected package.
192 See the features documentation
194 <https://doc.rust-lang.org/cargo/reference/features.html#command-line-feature-options>
195 for more details.
196 -F features, --features features
198 Space or comma separated list of features to activate. Features of
199 workspace members may be enabled with package-name/feature-name
200 syntax. This flag may be specified multiple times, which enables
201 all specified features.
202 --all-features
204 Activate all available features of all selected packages.
205 --no-default-features
207 Do not activate the default feature of the selected packages.
208 Compilation Options
210 --target triple
211 Benchmark for the given architecture. The default is the host
212 architecture. The general format of the triple is
213 <arch><sub>-<vendor>-<sys>-<abi>. Run rustc --print target-list for
214 a list of supported targets. This flag may be specified multiple
215 times.
216 This may also be specified with the build.target config value
218 <https://doc.rust-lang.org/cargo/reference/config.html>.
219 Note that specifying this flag makes Cargo run in a different mode
221 where the target artifacts are placed in a separate directory. See
222 the build cache
223 <https://doc.rust-lang.org/cargo/guide/build-cache.html>
224 documentation for more details.
225 --profile name
227 Benchmark with the given profile. See the the reference
228 <https://doc.rust-lang.org/cargo/reference/profiles.html> for more
229 details on profiles.
230 --ignore-rust-version
232 Benchmark the target even if the selected Rust compiler is older
233 than the required Rust version as configured in the project's
234 rust-version field.
235 --timings=fmts
237 Output information how long each compilation takes, and track
238 concurrency information over time. Accepts an optional
239 comma-separated list of output formats; --timings without an
240 argument will default to --timings=html. Specifying an output
241 format (rather than the default) is unstable and requires
242 -Zunstable-options. Valid output formats:
243 • html (unstable, requires -Zunstable-options): Write a
245 human-readable file cargo-timing.html to the
246 target/cargo-timings directory with a report of the
247 compilation. Also write a report to the same directory with a
248 timestamp in the filename if you want to look at older runs.
249 HTML output is suitable for human consumption only, and does
250 not provide machine-readable timing data.
251 • json (unstable, requires -Zunstable-options): Emit
253 machine-readable JSON information about timing information.
254 Output Options
256 --target-dir directory
257 Directory for all generated artifacts and intermediate files. May
258 also be specified with the CARGO_TARGET_DIR environment variable,
259 or the build.target-dir config value
260 <https://doc.rust-lang.org/cargo/reference/config.html>. Defaults
261 to target in the root of the workspace.
262 Display Options
264 By default the Rust test harness hides output from benchmark execution
265 to keep results readable. Benchmark output can be recovered (e.g., for
266 debugging) by passing --nocapture to the benchmark binaries:
267 cargo bench -- --nocapture
269 -v, --verbose
271 Use verbose output. May be specified twice for "very verbose"
272 output which includes extra output such as dependency warnings and
273 build script output. May also be specified with the term.verbose
274 config value
275 <https://doc.rust-lang.org/cargo/reference/config.html>.
276 -q, --quiet
278 Do not print cargo log messages. May also be specified with the
279 term.quiet config value
280 <https://doc.rust-lang.org/cargo/reference/config.html>.
281 --color when
283 Control when colored output is used. Valid values:
284 • auto (default): Automatically detect if color support is
286 available on the terminal.
287 • always: Always display colors.
289 • never: Never display colors.
291 May also be specified with the term.color config value
293 <https://doc.rust-lang.org/cargo/reference/config.html>.
294 --message-format fmt
296 The output format for diagnostic messages. Can be specified
297 multiple times and consists of comma-separated values. Valid
298 values:
299 • human (default): Display in a human-readable text format.
301 Conflicts with short and json.
302 • short: Emit shorter, human-readable text messages. Conflicts
304 with human and json.
305 • json: Emit JSON messages to stdout. See the reference
307 <https://doc.rust-lang.org/cargo/reference/external-tools.html#json-messages>
308 for more details. Conflicts with human and short.
309 • json-diagnostic-short: Ensure the rendered field of JSON
311 messages contains the "short" rendering from rustc. Cannot be
312 used with human or short.
313 • json-diagnostic-rendered-ansi: Ensure the rendered field of JSON
315 messages contains embedded ANSI color codes for respecting
316 rustc's default color scheme. Cannot be used with human or
317 short.
318 • json-render-diagnostics: Instruct Cargo to not include rustc
320 diagnostics in JSON messages printed, but instead Cargo itself
321 should render the JSON diagnostics coming from rustc. Cargo's
322 own JSON diagnostics and others coming from rustc are still
323 emitted. Cannot be used with human or short.
324 Manifest Options
326 --manifest-path path
327 Path to the Cargo.toml file. By default, Cargo searches for the
328 Cargo.toml file in the current directory or any parent directory.
329 --frozen, --locked
331 Either of these flags requires that the Cargo.lock file is
332 up-to-date. If the lock file is missing, or it needs to be updated,
333 Cargo will exit with an error. The --frozen flag also prevents
334 Cargo from attempting to access the network to determine if it is
335 out-of-date.
336 These may be used in environments where you want to assert that the
338 Cargo.lock file is up-to-date (such as a CI build) or want to avoid
339 network access.
340 --offline
342 Prevents Cargo from accessing the network for any reason. Without
343 this flag, Cargo will stop with an error if it needs to access the
344 network and the network is not available. With this flag, Cargo
345 will attempt to proceed without the network if possible.
346 Beware that this may result in different dependency resolution than
348 online mode. Cargo will restrict itself to crates that are
349 downloaded locally, even if there might be a newer version as
350 indicated in the local copy of the index. See the cargo-fetch(1)
351 command to download dependencies before going offline.
352 May also be specified with the net.offline config value
354 <https://doc.rust-lang.org/cargo/reference/config.html>.
355 Common Options
357 +toolchain
358 If Cargo has been installed with rustup, and the first argument to
359 cargo begins with +, it will be interpreted as a rustup toolchain
360 name (such as +stable or +nightly). See the rustup documentation
361 <https://rust-lang.github.io/rustup/overrides.html> for more
362 information about how toolchain overrides work.
363 --config KEY=VALUE or PATH
365 Overrides a Cargo configuration value. The argument should be in
366 TOML syntax of KEY=VALUE, or provided as a path to an extra
367 configuration file. This flag may be specified multiple times. See
368 the command-line overrides section
369 <https://doc.rust-lang.org/cargo/reference/config.html#command-line-overrides>
370 for more information.
371 -h, --help
373 Prints help information.
374 -Z flag
376 Unstable (nightly-only) flags to Cargo. Run cargo -Z help for
377 details.
378 Miscellaneous Options
380 The --jobs argument affects the building of the benchmark executable
381 but does not affect how many threads are used when running the
382 benchmarks. The Rust test harness runs benchmarks serially in a single
383 thread.
384 -j N, --jobs N
386 Number of parallel jobs to run. May also be specified with the
387 build.jobs config value
388 <https://doc.rust-lang.org/cargo/reference/config.html>. Defaults
389 to the number of logical CPUs. If negative, it sets the maximum
390 number of parallel jobs to the number of logical CPUs plus provided
391 value. Should not be 0.
392 --keep-going
394 Build as many crates in the dependency graph as possible, rather
395 than aborting the build on the first one that fails to build.
396 Unstable, requires -Zunstable-options.
397
399 See the reference
400 <https://doc.rust-lang.org/cargo/reference/environment-variables.html>
401 for details on environment variables that Cargo reads.
402
404 • 0: Cargo succeeded.
405 • 101: Cargo failed to complete.
407
409 1. Build and execute all the benchmarks of the current package:
410 cargo bench
412 2. Run only a specific benchmark within a specific benchmark target:
414 cargo bench --bench bench_name -- modname::some_benchmark
416
418 cargo(1), cargo-test(1)
419 CARGO-BENCH(1)