1CARGO-BENCH(1) General Commands Manual CARGO-BENCH(1)
2
6 cargo-bench - Execute benchmarks of a package
7
9 cargo bench [options] [benchname] [-- bench-options]
10
12 Compile and execute benchmarks.
13 The benchmark filtering argument benchname and all the arguments
15 following the two dashes (--) are passed to the benchmark binaries and
16 thus to libtest (rustc's built in unit-test and micro-benchmarking
17 framework). If you are passing arguments to both Cargo and the binary,
18 the ones after -- go to the binary, the ones before go to Cargo. For
19 details about libtest's arguments see the output of cargo bench --
20 --help and check out the rustc book's chapter on how tests work at
21 <https://doc.rust-lang.org/rustc/tests/index.html>.
22 As an example, this will run only the benchmark named foo (and skip
24 other similarly named benchmarks like foobar):
25 cargo bench -- foo --exact
27 Benchmarks are built with the --test option to rustc which creates an
29 executable with a main function that automatically runs all functions
30 annotated with the #[bench] attribute. Cargo passes the --bench flag to
31 the test harness to tell it to run only benchmarks.
32 The libtest harness may be disabled by setting harness = false in the
34 target manifest settings, in which case your code will need to provide
35 its own main function to handle running benchmarks.
36 Note: The #[bench] attribute
38 <https://doc.rust-lang.org/nightly/unstable-book/library-features/test.html>
39 is currently unstable and only available on the nightly channel
40 <https://doc.rust-lang.org/book/appendix-07-nightly-rust.html>.
41 There are some packages available on crates.io
42 <https://crates.io/keywords/benchmark> that may help with
43 running benchmarks on the stable channel, such as Criterion
44 <https://crates.io/crates/criterion>.
45
47 Benchmark Options
48 --no-run
49 Compile, but don't run benchmarks.
50 --no-fail-fast
52 Run all benchmarks regardless of failure. Without this flag, Cargo
53 will exit after the first executable fails. The Rust test harness
54 will run all benchmarks within the executable to completion, this
55 flag only applies to the executable as a whole.
56 Package Selection
58 By default, when no package selection options are given, the packages
59 selected depend on the selected manifest file (based on the current
60 working directory if --manifest-path is not given). If the manifest is
61 the root of a workspace then the workspaces default members are
62 selected, otherwise only the package defined by the manifest will be
63 selected.
64 The default members of a workspace can be set explicitly with the
66 workspace.default-members key in the root manifest. If this is not set,
67 a virtual workspace will include all workspace members (equivalent to
68 passing --workspace), and a non-virtual workspace will include only the
69 root crate itself.
70 -p spec..., --package spec...
72 Benchmark only the specified packages. See cargo-pkgid(1) for the
73 SPEC format. This flag may be specified multiple times and supports
74 common Unix glob patterns like *, ? and []. However, to avoid your
75 shell accidentally expanding glob patterns before Cargo handles
76 them, you must use single quotes or double quotes around each
77 pattern.
78 --workspace
80 Benchmark all members in the workspace.
81 --all
83 Deprecated alias for --workspace.
84 --exclude SPEC...
86 Exclude the specified packages. Must be used in conjunction with
87 the --workspace flag. This flag may be specified multiple times and
88 supports common Unix glob patterns like *, ? and []. However, to
89 avoid your shell accidentally expanding glob patterns before Cargo
90 handles them, you must use single quotes or double quotes around
91 each pattern.
92 Target Selection
94 When no target selection options are given, cargo bench will build the
95 following targets of the selected packages:
96 • lib — used to link with binaries and benchmarks
98 • bins (only if benchmark targets are built and required features are
100 available)
101 • lib as a benchmark
103 • bins as benchmarks
105 • benchmark targets
107 The default behavior can be changed by setting the bench flag for the
109 target in the manifest settings. Setting examples to bench = true will
110 build and run the example as a benchmark. Setting targets to bench =
111 false will stop them from being benchmarked by default. Target
112 selection options that take a target by name ignore the bench flag and
113 will always benchmark the given target.
114 Passing target selection flags will benchmark only the specified
116 targets.
117 Note that --bin, --example, --test and --bench flags also support
119 common Unix glob patterns like *, ? and []. However, to avoid your
120 shell accidentally expanding glob patterns before Cargo handles them,
121 you must use single quotes or double quotes around each glob pattern.
122 --lib
124 Benchmark the package's library.
125 --bin name...
127 Benchmark the specified binary. This flag may be specified multiple
128 times and supports common Unix glob patterns.
129 --bins
131 Benchmark all binary targets.
132 --example name...
134 Benchmark the specified example. This flag may be specified
135 multiple times and supports common Unix glob patterns.
136 --examples
138 Benchmark all example targets.
139 --test name...
141 Benchmark the specified integration test. This flag may be
142 specified multiple times and supports common Unix glob patterns.
143 --tests
145 Benchmark all targets in test mode that have the test = true
146 manifest flag set. By default this includes the library and
147 binaries built as unittests, and integration tests. Be aware that
148 this will also build any required dependencies, so the lib target
149 may be built twice (once as a unittest, and once as a dependency
150 for binaries, integration tests, etc.). Targets may be enabled or
151 disabled by setting the test flag in the manifest settings for the
152 target.
153 --bench name...
155 Benchmark the specified benchmark. This flag may be specified
156 multiple times and supports common Unix glob patterns.
157 --benches
159 Benchmark all targets in benchmark mode that have the bench = true
160 manifest flag set. By default this includes the library and
161 binaries built as benchmarks, and bench targets. Be aware that this
162 will also build any required dependencies, so the lib target may be
163 built twice (once as a benchmark, and once as a dependency for
164 binaries, benchmarks, etc.). Targets may be enabled or disabled by
165 setting the bench flag in the manifest settings for the target.
166 --all-targets
168 Benchmark all targets. This is equivalent to specifying --lib
169 --bins --tests --benches --examples.
170 Feature Selection
172 The feature flags allow you to control which features are enabled. When
173 no feature options are given, the default feature is activated for
174 every selected package.
175 See the features documentation
177 <https://doc.rust-lang.org/cargo/reference/features.html#command-line-feature-options>
178 for more details.
179 --features features
181 Space or comma separated list of features to activate. Features of
182 workspace members may be enabled with package-name/feature-name
183 syntax. This flag may be specified multiple times, which enables
184 all specified features.
185 --all-features
187 Activate all available features of all selected packages.
188 --no-default-features
190 Do not activate the default feature of the selected packages.
191 Compilation Options
193 --target triple
194 Benchmark for the given architecture. The default is the host
195 architecture. The general format of the triple is
196 <arch><sub>-<vendor>-<sys>-<abi>. Run rustc --print target-list for
197 a list of supported targets.
198 This may also be specified with the build.target config value
200 <https://doc.rust-lang.org/cargo/reference/config.html>.
201 Note that specifying this flag makes Cargo run in a different mode
203 where the target artifacts are placed in a separate directory. See
204 the build cache
205 <https://doc.rust-lang.org/cargo/guide/build-cache.html>
206 documentation for more details.
207 --ignore-rust-version
209 Benchmark the target even if the selected Rust compiler is older
210 than the required Rust version as configured in the project's
211 rust-version field.
212 Output Options
214 --target-dir directory
215 Directory for all generated artifacts and intermediate files. May
216 also be specified with the CARGO_TARGET_DIR environment variable,
217 or the build.target-dir config value
218 <https://doc.rust-lang.org/cargo/reference/config.html>. Defaults
219 to target in the root of the workspace.
220 Display Options
222 By default the Rust test harness hides output from benchmark execution
223 to keep results readable. Benchmark output can be recovered (e.g., for
224 debugging) by passing --nocapture to the benchmark binaries:
225 cargo bench -- --nocapture
227 -v, --verbose
229 Use verbose output. May be specified twice for "very verbose"
230 output which includes extra output such as dependency warnings and
231 build script output. May also be specified with the term.verbose
232 config value
233 <https://doc.rust-lang.org/cargo/reference/config.html>.
234 -q, --quiet
236 No output printed to stdout.
237 --color when
239 Control when colored output is used. Valid values:
240 • auto (default): Automatically detect if color support is
242 available on the terminal.
243 • always: Always display colors.
245 • never: Never display colors.
247 May also be specified with the term.color config value
249 <https://doc.rust-lang.org/cargo/reference/config.html>.
250 --message-format fmt
252 The output format for diagnostic messages. Can be specified
253 multiple times and consists of comma-separated values. Valid
254 values:
255 • human (default): Display in a human-readable text format.
257 Conflicts with short and json.
258 • short: Emit shorter, human-readable text messages. Conflicts
260 with human and json.
261 • json: Emit JSON messages to stdout. See the reference
263 <https://doc.rust-lang.org/cargo/reference/external-tools.html#json-messages>
264 for more details. Conflicts with human and short.
265 • json-diagnostic-short: Ensure the rendered field of JSON
267 messages contains the "short" rendering from rustc. Cannot be
268 used with human or short.
269 • json-diagnostic-rendered-ansi: Ensure the rendered field of JSON
271 messages contains embedded ANSI color codes for respecting
272 rustc's default color scheme. Cannot be used with human or
273 short.
274 • json-render-diagnostics: Instruct Cargo to not include rustc
276 diagnostics in in JSON messages printed, but instead Cargo
277 itself should render the JSON diagnostics coming from rustc.
278 Cargo's own JSON diagnostics and others coming from rustc are
279 still emitted. Cannot be used with human or short.
280 Manifest Options
282 --manifest-path path
283 Path to the Cargo.toml file. By default, Cargo searches for the
284 Cargo.toml file in the current directory or any parent directory.
285 --frozen, --locked
287 Either of these flags requires that the Cargo.lock file is
288 up-to-date. If the lock file is missing, or it needs to be updated,
289 Cargo will exit with an error. The --frozen flag also prevents
290 Cargo from attempting to access the network to determine if it is
291 out-of-date.
292 These may be used in environments where you want to assert that the
294 Cargo.lock file is up-to-date (such as a CI build) or want to avoid
295 network access.
296 --offline
298 Prevents Cargo from accessing the network for any reason. Without
299 this flag, Cargo will stop with an error if it needs to access the
300 network and the network is not available. With this flag, Cargo
301 will attempt to proceed without the network if possible.
302 Beware that this may result in different dependency resolution than
304 online mode. Cargo will restrict itself to crates that are
305 downloaded locally, even if there might be a newer version as
306 indicated in the local copy of the index. See the cargo-fetch(1)
307 command to download dependencies before going offline.
308 May also be specified with the net.offline config value
310 <https://doc.rust-lang.org/cargo/reference/config.html>.
311 Common Options
313 +toolchain
314 If Cargo has been installed with rustup, and the first argument to
315 cargo begins with +, it will be interpreted as a rustup toolchain
316 name (such as +stable or +nightly). See the rustup documentation
317 <https://rust-lang.github.io/rustup/overrides.html> for more
318 information about how toolchain overrides work.
319 -h, --help
321 Prints help information.
322 -Z flag
324 Unstable (nightly-only) flags to Cargo. Run cargo -Z help for
325 details.
326 Miscellaneous Options
328 The --jobs argument affects the building of the benchmark executable
329 but does not affect how many threads are used when running the
330 benchmarks. The Rust test harness runs benchmarks serially in a single
331 thread.
332 -j N, --jobs N
334 Number of parallel jobs to run. May also be specified with the
335 build.jobs config value
336 <https://doc.rust-lang.org/cargo/reference/config.html>. Defaults
337 to the number of CPUs.
338
340 Profiles may be used to configure compiler options such as optimization
341 levels and debug settings. See the reference
342 <https://doc.rust-lang.org/cargo/reference/profiles.html> for more
343 details.
344 Benchmarks are always built with the bench profile. Binary and lib
346 targets are built separately as benchmarks with the bench profile.
347 Library targets are built with the release profiles when linked to
348 binaries and benchmarks. Dependencies use the release profile.
349 If you need a debug build of a benchmark, try building it with
351 cargo-build(1) which will use the test profile which is by default
352 unoptimized and includes debug information. You can then run the
353 debug-enabled benchmark manually.
354
356 See the reference
357 <https://doc.rust-lang.org/cargo/reference/environment-variables.html>
358 for details on environment variables that Cargo reads.
359
361 • 0: Cargo succeeded.
362 • 101: Cargo failed to complete.
364
366 1. Build and execute all the benchmarks of the current package:
367 cargo bench
369 2. Run only a specific benchmark within a specific benchmark target:
371 cargo bench --bench bench_name -- modname::some_benchmark
373
375 cargo(1), cargo-test(1)
376 CARGO-BENCH(1)