1CARGO-BENCH(1) General Commands Manual CARGO-BENCH(1)
2
6 cargo-bench - Execute benchmarks of a package
7
9 cargo bench [options] [benchname] [-- bench-options]
10
12 Compile and execute benchmarks.
13 The benchmark filtering argument benchname and all the arguments
15 following the two dashes (--) are passed to the benchmark binaries and
16 thus to libtest (rustc's built in unit-test and micro-benchmarking
17 framework). If you are passing arguments to both Cargo and the binary,
18 the ones after -- go to the binary, the ones before go to Cargo. For
19 details about libtest's arguments see the output of cargo bench --
20 --help. As an example, this will run only the benchmark named foo (and
21 skip other similarly named benchmarks like foobar):
22 cargo bench -- foo --exact
24 Benchmarks are built with the --test option to rustc which creates an
26 executable with a main function that automatically runs all functions
27 annotated with the #[bench] attribute. Cargo passes the --bench flag to
28 the test harness to tell it to run only benchmarks.
29 The libtest harness may be disabled by setting harness = false in the
31 target manifest settings, in which case your code will need to provide
32 its own main function to handle running benchmarks.
33 Note: The #[bench] attribute
35 <https://doc.rust-lang.org/nightly/unstable-book/library-features/test.html>
36 is currently unstable and only available on the nightly channel
37 <https://doc.rust-lang.org/book/appendix-07-nightly-rust.html>.
38 There are some packages available on crates.io
39 <https://crates.io/keywords/benchmark> that may help with
40 running benchmarks on the stable channel, such as Criterion
41 <https://crates.io/crates/criterion>.
42
44 Benchmark Options
45 --no-run
46 Compile, but don't run benchmarks.
47 --no-fail-fast
49 Run all benchmarks regardless of failure. Without this flag, Cargo
50 will exit after the first executable fails. The Rust test harness
51 will run all benchmarks within the executable to completion, this
52 flag only applies to the executable as a whole.
53 Package Selection
55 By default, when no package selection options are given, the packages
56 selected depend on the selected manifest file (based on the current
57 working directory if --manifest-path is not given). If the manifest is
58 the root of a workspace then the workspaces default members are
59 selected, otherwise only the package defined by the manifest will be
60 selected.
61 The default members of a workspace can be set explicitly with the
63 workspace.default-members key in the root manifest. If this is not set,
64 a virtual workspace will include all workspace members (equivalent to
65 passing --workspace), and a non-virtual workspace will include only the
66 root crate itself.
67 -p spec..., --package spec...
69 Benchmark only the specified packages. See cargo-pkgid(1) for the
70 SPEC format. This flag may be specified multiple times and supports
71 common Unix glob patterns like *, ? and []. However, to avoid your
72 shell accidentally expanding glob patterns before Cargo handles
73 them, you must use single quotes or double quotes around each
74 pattern.
75 --workspace
77 Benchmark all members in the workspace.
78 --all
80 Deprecated alias for --workspace.
81 --exclude SPEC...
83 Exclude the specified packages. Must be used in conjunction with
84 the --workspace flag. This flag may be specified multiple times and
85 supports common Unix glob patterns like *, ? and []. However, to
86 avoid your shell accidentally expanding glob patterns before Cargo
87 handles them, you must use single quotes or double quotes around
88 each pattern.
89 Target Selection
91 When no target selection options are given, cargo bench will build the
92 following targets of the selected packages:
93 • lib — used to link with binaries and benchmarks
95 • bins (only if benchmark targets are built and required features are
97 available)
98 • lib as a benchmark
100 • bins as benchmarks
102 • benchmark targets
104 The default behavior can be changed by setting the bench flag for the
106 target in the manifest settings. Setting examples to bench = true will
107 build and run the example as a benchmark. Setting targets to bench =
108 false will stop them from being benchmarked by default. Target
109 selection options that take a target by name ignore the bench flag and
110 will always benchmark the given target.
111 Passing target selection flags will benchmark only the specified
113 targets.
114 Note that --bin, --example, --test and --bench flags also support
116 common Unix glob patterns like *, ? and []. However, to avoid your
117 shell accidentally expanding glob patterns before Cargo handles them,
118 you must use single quotes or double quotes around each glob pattern.
119 --lib
121 Benchmark the package's library.
122 --bin name...
124 Benchmark the specified binary. This flag may be specified multiple
125 times and supports common Unix glob patterns.
126 --bins
128 Benchmark all binary targets.
129 --example name...
131 Benchmark the specified example. This flag may be specified
132 multiple times and supports common Unix glob patterns.
133 --examples
135 Benchmark all example targets.
136 --test name...
138 Benchmark the specified integration test. This flag may be
139 specified multiple times and supports common Unix glob patterns.
140 --tests
142 Benchmark all targets in test mode that have the test = true
143 manifest flag set. By default this includes the library and
144 binaries built as unittests, and integration tests. Be aware that
145 this will also build any required dependencies, so the lib target
146 may be built twice (once as a unittest, and once as a dependency
147 for binaries, integration tests, etc.). Targets may be enabled or
148 disabled by setting the test flag in the manifest settings for the
149 target.
150 --bench name...
152 Benchmark the specified benchmark. This flag may be specified
153 multiple times and supports common Unix glob patterns.
154 --benches
156 Benchmark all targets in benchmark mode that have the bench = true
157 manifest flag set. By default this includes the library and
158 binaries built as benchmarks, and bench targets. Be aware that this
159 will also build any required dependencies, so the lib target may be
160 built twice (once as a benchmark, and once as a dependency for
161 binaries, benchmarks, etc.). Targets may be enabled or disabled by
162 setting the bench flag in the manifest settings for the target.
163 --all-targets
165 Benchmark all targets. This is equivalent to specifying --lib
166 --bins --tests --benches --examples.
167 Feature Selection
169 The feature flags allow you to control which features are enabled. When
170 no feature options are given, the default feature is activated for
171 every selected package.
172 See the features documentation
174 <https://doc.rust-lang.org/cargo/reference/features.html#command-line-feature-options>
175 for more details.
176 --features features
178 Space or comma separated list of features to activate. Features of
179 workspace members may be enabled with package-name/feature-name
180 syntax. This flag may be specified multiple times, which enables
181 all specified features.
182 --all-features
184 Activate all available features of all selected packages.
185 --no-default-features
187 Do not activate the default feature of the selected packages.
188 Compilation Options
190 --target triple
191 Benchmark for the given architecture. The default is the host
192 architecture. The general format of the triple is
193 <arch><sub>-<vendor>-<sys>-<abi>. Run rustc --print target-list for
194 a list of supported targets.
195 This may also be specified with the build.target config value
197 <https://doc.rust-lang.org/cargo/reference/config.html>.
198 Note that specifying this flag makes Cargo run in a different mode
200 where the target artifacts are placed in a separate directory. See
201 the build cache
202 <https://doc.rust-lang.org/cargo/guide/build-cache.html>
203 documentation for more details.
204 Output Options
206 --target-dir directory
207 Directory for all generated artifacts and intermediate files. May
208 also be specified with the CARGO_TARGET_DIR environment variable,
209 or the build.target-dir config value
210 <https://doc.rust-lang.org/cargo/reference/config.html>. Defaults
211 to target in the root of the workspace.
212 Display Options
214 By default the Rust test harness hides output from benchmark execution
215 to keep results readable. Benchmark output can be recovered (e.g., for
216 debugging) by passing --nocapture to the benchmark binaries:
217 cargo bench -- --nocapture
219 -v, --verbose
221 Use verbose output. May be specified twice for "very verbose"
222 output which includes extra output such as dependency warnings and
223 build script output. May also be specified with the term.verbose
224 config value
225 <https://doc.rust-lang.org/cargo/reference/config.html>.
226 -q, --quiet
228 No output printed to stdout.
229 --color when
231 Control when colored output is used. Valid values:
232 • auto (default): Automatically detect if color support is
234 available on the terminal.
235 • always: Always display colors.
237 • never: Never display colors.
239 May also be specified with the term.color config value
241 <https://doc.rust-lang.org/cargo/reference/config.html>.
242 --message-format fmt
244 The output format for diagnostic messages. Can be specified
245 multiple times and consists of comma-separated values. Valid
246 values:
247 • human (default): Display in a human-readable text format.
249 Conflicts with short and json.
250 • short: Emit shorter, human-readable text messages. Conflicts
252 with human and json.
253 • json: Emit JSON messages to stdout. See the reference
255 <https://doc.rust-lang.org/cargo/reference/external-tools.html#json-messages>
256 for more details. Conflicts with human and short.
257 • json-diagnostic-short: Ensure the rendered field of JSON
259 messages contains the "short" rendering from rustc. Cannot be
260 used with human or short.
261 • json-diagnostic-rendered-ansi: Ensure the rendered field of JSON
263 messages contains embedded ANSI color codes for respecting
264 rustc's default color scheme. Cannot be used with human or
265 short.
266 • json-render-diagnostics: Instruct Cargo to not include rustc
268 diagnostics in in JSON messages printed, but instead Cargo
269 itself should render the JSON diagnostics coming from rustc.
270 Cargo's own JSON diagnostics and others coming from rustc are
271 still emitted. Cannot be used with human or short.
272 Manifest Options
274 --manifest-path path
275 Path to the Cargo.toml file. By default, Cargo searches for the
276 Cargo.toml file in the current directory or any parent directory.
277 --frozen, --locked
279 Either of these flags requires that the Cargo.lock file is
280 up-to-date. If the lock file is missing, or it needs to be updated,
281 Cargo will exit with an error. The --frozen flag also prevents
282 Cargo from attempting to access the network to determine if it is
283 out-of-date.
284 These may be used in environments where you want to assert that the
286 Cargo.lock file is up-to-date (such as a CI build) or want to avoid
287 network access.
288 --offline
290 Prevents Cargo from accessing the network for any reason. Without
291 this flag, Cargo will stop with an error if it needs to access the
292 network and the network is not available. With this flag, Cargo
293 will attempt to proceed without the network if possible.
294 Beware that this may result in different dependency resolution than
296 online mode. Cargo will restrict itself to crates that are
297 downloaded locally, even if there might be a newer version as
298 indicated in the local copy of the index. See the cargo-fetch(1)
299 command to download dependencies before going offline.
300 May also be specified with the net.offline config value
302 <https://doc.rust-lang.org/cargo/reference/config.html>.
303 Common Options
305 +toolchain
306 If Cargo has been installed with rustup, and the first argument to
307 cargo begins with +, it will be interpreted as a rustup toolchain
308 name (such as +stable or +nightly). See the rustup documentation
309 <https://rust-lang.github.io/rustup/overrides.html> for more
310 information about how toolchain overrides work.
311 -h, --help
313 Prints help information.
314 -Z flag
316 Unstable (nightly-only) flags to Cargo. Run cargo -Z help for
317 details.
318 Miscellaneous Options
320 The --jobs argument affects the building of the benchmark executable
321 but does not affect how many threads are used when running the
322 benchmarks. The Rust test harness runs benchmarks serially in a single
323 thread.
324 -j N, --jobs N
326 Number of parallel jobs to run. May also be specified with the
327 build.jobs config value
328 <https://doc.rust-lang.org/cargo/reference/config.html>. Defaults
329 to the number of CPUs.
330
332 Profiles may be used to configure compiler options such as optimization
333 levels and debug settings. See the reference
334 <https://doc.rust-lang.org/cargo/reference/profiles.html> for more
335 details.
336 Benchmarks are always built with the bench profile. Binary and lib
338 targets are built separately as benchmarks with the bench profile.
339 Library targets are built with the release profiles when linked to
340 binaries and benchmarks. Dependencies use the release profile.
341 If you need a debug build of a benchmark, try building it with
343 cargo-build(1) which will use the test profile which is by default
344 unoptimized and includes debug information. You can then run the
345 debug-enabled benchmark manually.
346
348 See the reference
349 <https://doc.rust-lang.org/cargo/reference/environment-variables.html>
350 for details on environment variables that Cargo reads.
351
353 • 0: Cargo succeeded.
354 • 101: Cargo failed to complete.
356
358 1. Build and execute all the benchmarks of the current package:
359 cargo bench
361 2. Run only a specific benchmark within a specific benchmark target:
363 cargo bench --bench bench_name -- modname::some_benchmark
365
367 cargo(1), cargo-test(1)
368 CARGO-BENCH(1)