1CARGO-BENCH(1) General Commands Manual CARGO-BENCH(1)
2
6 cargo-bench — Execute benchmarks of a package
7
9 cargo bench [options] [benchname] [-- bench-options]
10
12 Compile and execute benchmarks.
13 The benchmark filtering argument benchname and all the arguments
15 following the two dashes (--) are passed to the benchmark binaries and
16 thus to libtest (rustc’s built in unit-test and micro-benchmarking
17 framework). If you are passing arguments to both Cargo and the binary,
18 the ones after -- go to the binary, the ones before go to Cargo. For
19 details about libtest’s arguments see the output of cargo bench --
20 --help and check out the rustc book’s chapter on how tests work at
21 <https://doc.rust-lang.org/rustc/tests/index.html>.
22 As an example, this will run only the benchmark named foo (and skip
24 other similarly named benchmarks like foobar):
25 cargo bench -- foo --exact
27 Benchmarks are built with the --test option to rustc which creates a
29 special executable by linking your code with libtest. The executable
30 automatically runs all functions annotated with the #[bench] attribute.
31 Cargo passes the --bench flag to the test harness to tell it to run
32 only benchmarks.
33 The libtest harness may be disabled by setting harness = false in the
35 target manifest settings, in which case your code will need to provide
36 its own main function to handle running benchmarks.
37 Note: The #[bench] attribute
39 <https://doc.rust-lang.org/nightly/unstable-book/library-features/test.html>
40 is currently unstable and only available on the nightly channel
41 <https://doc.rust-lang.org/book/appendix-07-nightly-rust.html>.
42 There are some packages available on crates.io
43 <https://crates.io/keywords/benchmark> that may help with
44 running benchmarks on the stable channel, such as Criterion
45 <https://crates.io/crates/criterion>.
46 By default, cargo bench uses the bench profile
48 <https://doc.rust-lang.org/cargo/reference/profiles.html#bench>, which
49 enables optimizations and disables debugging information. If you need
50 to debug a benchmark, you can use the --profile=dev command-line option
51 to switch to the dev profile. You can then run the debug-enabled
52 benchmark within a debugger.
53 Working directory of benchmarks
55 The working directory of every benchmark is set to the root directory
56 of the package the benchmark belongs to. Setting the working directory
57 of benchmarks to the package’s root directory makes it possible for
58 benchmarks to reliably access the package’s files using relative paths,
59 regardless from where cargo bench was executed from.
60
62 Benchmark Options
63 --no-run
64 Compile, but don’t run benchmarks.
65 --no-fail-fast
67 Run all benchmarks regardless of failure. Without this flag, Cargo
68 will exit after the first executable fails. The Rust test harness
69 will run all benchmarks within the executable to completion, this
70 flag only applies to the executable as a whole.
71 Package Selection
73 By default, when no package selection options are given, the packages
74 selected depend on the selected manifest file (based on the current
75 working directory if --manifest-path is not given). If the manifest is
76 the root of a workspace then the workspaces default members are
77 selected, otherwise only the package defined by the manifest will be
78 selected.
79 The default members of a workspace can be set explicitly with the
81 workspace.default-members key in the root manifest. If this is not set,
82 a virtual workspace will include all workspace members (equivalent to
83 passing --workspace), and a non-virtual workspace will include only the
84 root crate itself.
85 -p spec…, --package spec…
87 Benchmark only the specified packages. See cargo-pkgid(1) for the
88 SPEC format. This flag may be specified multiple times and supports
89 common Unix glob patterns like *, ? and []. However, to avoid your
90 shell accidentally expanding glob patterns before Cargo handles
91 them, you must use single quotes or double quotes around each
92 pattern.
93 --workspace
95 Benchmark all members in the workspace.
96 --all
98 Deprecated alias for --workspace.
99 --exclude SPEC…
101 Exclude the specified packages. Must be used in conjunction with
102 the --workspace flag. This flag may be specified multiple times and
103 supports common Unix glob patterns like *, ? and []. However, to
104 avoid your shell accidentally expanding glob patterns before Cargo
105 handles them, you must use single quotes or double quotes around
106 each pattern.
107 Target Selection
109 When no target selection options are given, cargo bench will build the
110 following targets of the selected packages:
111 • lib — used to link with binaries and benchmarks
113 • bins (only if benchmark targets are built and required features are
115 available)
116 • lib as a benchmark
118 • bins as benchmarks
120 • benchmark targets
122 The default behavior can be changed by setting the bench flag for the
124 target in the manifest settings. Setting examples to bench = true will
125 build and run the example as a benchmark, replacing the example’s main
126 function with the libtest harness.
127 Setting targets to bench = false will stop them from being bencharmked
129 by default. Target selection options that take a target by name (such
130 as --example foo) ignore the bench flag and will always benchmark the
131 given target.
132 See Configuring a target
134 <https://doc.rust-lang.org/cargo/reference/cargo-targets.html#configuring-a-target>
135 for more information on per-target settings.
136 Binary targets are automatically built if there is an integration test
138 or benchmark being selected to benchmark. This allows an integration
139 test to execute the binary to exercise and test its behavior. The
140 CARGO_BIN_EXE_<name> environment variable
141 <https://doc.rust-lang.org/cargo/reference/environment-variables.html#environment-variables-cargo-sets-for-crates>
142 is set when the integration test is built so that it can use the env
143 macro <https://doc.rust-lang.org/std/macro.env.html> to locate the
144 executable.
145 Passing target selection flags will benchmark only the specified
147 targets.
148 Note that --bin, --example, --test and --bench flags also support
150 common Unix glob patterns like *, ? and []. However, to avoid your
151 shell accidentally expanding glob patterns before Cargo handles them,
152 you must use single quotes or double quotes around each glob pattern.
153 --lib
155 Benchmark the package’s library.
156 --bin name…
158 Benchmark the specified binary. This flag may be specified multiple
159 times and supports common Unix glob patterns.
160 --bins
162 Benchmark all binary targets.
163 --example name…
165 Benchmark the specified example. This flag may be specified
166 multiple times and supports common Unix glob patterns.
167 --examples
169 Benchmark all example targets.
170 --test name…
172 Benchmark the specified integration test. This flag may be
173 specified multiple times and supports common Unix glob patterns.
174 --tests
176 Benchmark all targets in test mode that have the test = true
177 manifest flag set. By default this includes the library and
178 binaries built as unittests, and integration tests. Be aware that
179 this will also build any required dependencies, so the lib target
180 may be built twice (once as a unittest, and once as a dependency
181 for binaries, integration tests, etc.). Targets may be enabled or
182 disabled by setting the test flag in the manifest settings for the
183 target.
184 --bench name…
186 Benchmark the specified benchmark. This flag may be specified
187 multiple times and supports common Unix glob patterns.
188 --benches
190 Benchmark all targets in benchmark mode that have the bench = true
191 manifest flag set. By default this includes the library and
192 binaries built as benchmarks, and bench targets. Be aware that this
193 will also build any required dependencies, so the lib target may be
194 built twice (once as a benchmark, and once as a dependency for
195 binaries, benchmarks, etc.). Targets may be enabled or disabled by
196 setting the bench flag in the manifest settings for the target.
197 --all-targets
199 Benchmark all targets. This is equivalent to specifying --lib
200 --bins --tests --benches --examples.
201 Feature Selection
203 The feature flags allow you to control which features are enabled. When
204 no feature options are given, the default feature is activated for
205 every selected package.
206 See the features documentation
208 <https://doc.rust-lang.org/cargo/reference/features.html#command-line-feature-options>
209 for more details.
210 -F features, --features features
212 Space or comma separated list of features to activate. Features of
213 workspace members may be enabled with package-name/feature-name
214 syntax. This flag may be specified multiple times, which enables
215 all specified features.
216 --all-features
218 Activate all available features of all selected packages.
219 --no-default-features
221 Do not activate the default feature of the selected packages.
222 Compilation Options
224 --target triple
225 Benchmark for the given architecture. The default is the host
226 architecture. The general format of the triple is
227 <arch><sub>-<vendor>-<sys>-<abi>. Run rustc --print target-list for
228 a list of supported targets. This flag may be specified multiple
229 times.
230 This may also be specified with the build.target config value
232 <https://doc.rust-lang.org/cargo/reference/config.html>.
233 Note that specifying this flag makes Cargo run in a different mode
235 where the target artifacts are placed in a separate directory. See
236 the build cache
237 <https://doc.rust-lang.org/cargo/guide/build-cache.html>
238 documentation for more details.
239 --profile name
241 Benchmark with the given profile. See the the reference
242 <https://doc.rust-lang.org/cargo/reference/profiles.html> for more
243 details on profiles.
244 --ignore-rust-version
246 Benchmark the target even if the selected Rust compiler is older
247 than the required Rust version as configured in the project’s
248 rust-version field.
249 --timings=fmts
251 Output information how long each compilation takes, and track
252 concurrency information over time. Accepts an optional
253 comma-separated list of output formats; --timings without an
254 argument will default to --timings=html. Specifying an output
255 format (rather than the default) is unstable and requires
256 -Zunstable-options. Valid output formats:
257 • html (unstable, requires -Zunstable-options): Write a
259 human-readable file cargo-timing.html to the
260 target/cargo-timings directory with a report of the
261 compilation. Also write a report to the same directory with a
262 timestamp in the filename if you want to look at older runs.
263 HTML output is suitable for human consumption only, and does
264 not provide machine-readable timing data.
265 • json (unstable, requires -Zunstable-options): Emit
267 machine-readable JSON information about timing information.
268 Output Options
270 --target-dir directory
271 Directory for all generated artifacts and intermediate files. May
272 also be specified with the CARGO_TARGET_DIR environment variable,
273 or the build.target-dir config value
274 <https://doc.rust-lang.org/cargo/reference/config.html>. Defaults
275 to target in the root of the workspace.
276 Display Options
278 By default the Rust test harness hides output from benchmark execution
279 to keep results readable. Benchmark output can be recovered (e.g., for
280 debugging) by passing --nocapture to the benchmark binaries:
281 cargo bench -- --nocapture
283 -v, --verbose
285 Use verbose output. May be specified twice for “very verbose”
286 output which includes extra output such as dependency warnings and
287 build script output. May also be specified with the term.verbose
288 config value
289 <https://doc.rust-lang.org/cargo/reference/config.html>.
290 -q, --quiet
292 Do not print cargo log messages. May also be specified with the
293 term.quiet config value
294 <https://doc.rust-lang.org/cargo/reference/config.html>.
295 --color when
297 Control when colored output is used. Valid values:
298 • auto (default): Automatically detect if color support is
300 available on the terminal.
301 • always: Always display colors.
303 • never: Never display colors.
305 May also be specified with the term.color config value
307 <https://doc.rust-lang.org/cargo/reference/config.html>.
308 --message-format fmt
310 The output format for diagnostic messages. Can be specified
311 multiple times and consists of comma-separated values. Valid
312 values:
313 • human (default): Display in a human-readable text format.
315 Conflicts with short and json.
316 • short: Emit shorter, human-readable text messages. Conflicts
318 with human and json.
319 • json: Emit JSON messages to stdout. See the reference
321 <https://doc.rust-lang.org/cargo/reference/external-tools.html#json-messages>
322 for more details. Conflicts with human and short.
323 • json-diagnostic-short: Ensure the rendered field of JSON
325 messages contains the “short” rendering from rustc. Cannot be
326 used with human or short.
327 • json-diagnostic-rendered-ansi: Ensure the rendered field of JSON
329 messages contains embedded ANSI color codes for respecting
330 rustc’s default color scheme. Cannot be used with human or
331 short.
332 • json-render-diagnostics: Instruct Cargo to not include rustc
334 diagnostics in JSON messages printed, but instead Cargo itself
335 should render the JSON diagnostics coming from rustc. Cargo’s
336 own JSON diagnostics and others coming from rustc are still
337 emitted. Cannot be used with human or short.
338 Manifest Options
340 --manifest-path path
341 Path to the Cargo.toml file. By default, Cargo searches for the
342 Cargo.toml file in the current directory or any parent directory.
343 --frozen, --locked
345 Either of these flags requires that the Cargo.lock file is
346 up-to-date. If the lock file is missing, or it needs to be updated,
347 Cargo will exit with an error. The --frozen flag also prevents
348 Cargo from attempting to access the network to determine if it is
349 out-of-date.
350 These may be used in environments where you want to assert that the
352 Cargo.lock file is up-to-date (such as a CI build) or want to avoid
353 network access.
354 --offline
356 Prevents Cargo from accessing the network for any reason. Without
357 this flag, Cargo will stop with an error if it needs to access the
358 network and the network is not available. With this flag, Cargo
359 will attempt to proceed without the network if possible.
360 Beware that this may result in different dependency resolution than
362 online mode. Cargo will restrict itself to crates that are
363 downloaded locally, even if there might be a newer version as
364 indicated in the local copy of the index. See the cargo-fetch(1)
365 command to download dependencies before going offline.
366 May also be specified with the net.offline config value
368 <https://doc.rust-lang.org/cargo/reference/config.html>.
369 Common Options
371 +toolchain
372 If Cargo has been installed with rustup, and the first argument to
373 cargo begins with +, it will be interpreted as a rustup toolchain
374 name (such as +stable or +nightly). See the rustup documentation
375 <https://rust-lang.github.io/rustup/overrides.html> for more
376 information about how toolchain overrides work.
377 --config KEY=VALUE or PATH
379 Overrides a Cargo configuration value. The argument should be in
380 TOML syntax of KEY=VALUE, or provided as a path to an extra
381 configuration file. This flag may be specified multiple times. See
382 the command-line overrides section
383 <https://doc.rust-lang.org/cargo/reference/config.html#command-line-overrides>
384 for more information.
385 -C PATH
387 Changes the current working directory before executing any
388 specified operations. This affects things like where cargo looks by
389 default for the project manifest (Cargo.toml), as well as the
390 directories searched for discovering .cargo/config.toml, for
391 example. This option must appear before the command name, for
392 example cargo -C path/to/my-project build.
393 This option is only available on the nightly channel
395 <https://doc.rust-lang.org/book/appendix-07-nightly-rust.html> and
396 requires the -Z unstable-options flag to enable (see #10098
397 <https://github.com/rust-lang/cargo/issues/10098>).
398 -h, --help
400 Prints help information.
401 -Z flag
403 Unstable (nightly-only) flags to Cargo. Run cargo -Z help for
404 details.
405 Miscellaneous Options
407 The --jobs argument affects the building of the benchmark executable
408 but does not affect how many threads are used when running the
409 benchmarks. The Rust test harness runs benchmarks serially in a single
410 thread.
411 -j N, --jobs N
413 Number of parallel jobs to run. May also be specified with the
414 build.jobs config value
415 <https://doc.rust-lang.org/cargo/reference/config.html>. Defaults
416 to the number of logical CPUs. If negative, it sets the maximum
417 number of parallel jobs to the number of logical CPUs plus provided
418 value. If a string default is provided, it sets the value back to
419 defaults. Should not be 0.
420 While cargo bench involves compilation, it does not provide a
422 --keep-going flag. Use --no-fail-fast to run as many benchmarks as
423 possible without stopping at the first failure. To “compile” as many
424 benchmarks as possible, use --benches to build benchmark binaries
425 separately. For example:
426 cargo build --benches --release --keep-going
428 cargo bench --no-fail-fast
429
431 See the reference
432 <https://doc.rust-lang.org/cargo/reference/environment-variables.html>
433 for details on environment variables that Cargo reads.
434
436 • 0: Cargo succeeded.
437 • 101: Cargo failed to complete.
439
441 1. Build and execute all the benchmarks of the current package:
442 cargo bench
444 2. Run only a specific benchmark within a specific benchmark target:
446 cargo bench --bench bench_name -- modname::some_benchmark
448
450 cargo(1), cargo-test(1)
451 CARGO-BENCH(1)