1WULT(1) Generated Python Manual WULT(1)
2
6 wult
7
9 wult [-h] [-q] [-d] [--version] [--force-color] {deploy,scan,start,re‐
10 port,filter,calc} ...
11
13 wult - a tool for measuring C-state latency.
14
17 -h Show this help message and exit.
18 -q Be quiet.
21 -d Print debugging information.
24 --version
27 Print version and exit.
28 --force-color
31 Force coloring of the text output.
32
35 wult deploy
36 Compile and deploy wult helpers and drivers.
37 wult scan
39 Scan for available devices.
40 wult start
42 Start the measurements.
43 wult report
45 Create an HTML report.
46 wult filter
48 Filter datapoints out of a test result.
49 wult calc
51 Calculate summary functions for a wult test result.
52
55 usage: wult deploy [-h] [-q] [-d] [--kernel-src KSRC] [--rebuild-bpf]
56 [--local-build] [--tmpdir-path TMPDIR_PATH] [--keep-tmpdir] [-H HOST‐
57 NAME] [-U USERNAME] [-K PRIVKEY] [-T TIMEOUT] [--skip-drivers]
58 Compile and deploy wult helpers and drivers to the SUT (System Under
60 Test), which can be can be either local or a remote host, depending on
61 the '-H' option. By default, everything is built on the SUT, but the
62 '--local-build' can be used for building on the local system. The driv‐
63 ers are searched for in the following directories (and in the following
64 order) on the local host: ./drivers/idle, $WULT_DATA_PATH/drivers/idle,
65 $HOME/.local/share/wult/drivers/idle, /usr/local/share/wult/driv‐
66 ers/idle, /usr/share/wult/drivers/idle.
67
70 -h Show this help message and exit.
71 -q Be quiet.
74 -d Print debugging information.
77 --kernel-src KSRC
80 Path to the Linux kernel sources to build drivers and eBPF
81 helpers against. The default is '/lib/modules/$(uname -r)/build'
82 on the SUT. If '--local-build' was used, then the path is con‐
83 sidered to be on the local system, rather than the SUT.
84 --rebuild-bpf
87 eBPF helpers sources consist of 2 components: the user-space
88 component and the eBPF component. The user-space component is
89 distributed as a source code, and must be compiled. The eBPF
90 component is distributed as both source code and in binary (com‐
91 piled) form. By default, the eBPF component is not re-compiled.
92 This option is meant to be used by wult developers to re-compile
93 the eBPF component if it was modified.
94 --local-build
97 Build helpers and drivers locally, instead of building on HOST‐
98 NAME (the SUT).
99 --tmpdir-path TMPDIR_PATH
102 When 'wult' is deployed, a random temporary directory is used.
103 Use this option provide a custom path instead. It will be used
104 as a temporary directory on both local and remote hosts. This
105 option is meant for debugging purposes.
106 --keep-tmpdir
109 Do not remove the temporary directories created while deploying
110 'wult'. This option is meant for debugging purposes.
111 -H HOSTNAME, --host HOSTNAME
114 Name of the host to run the command on.
115 -U USERNAME, --username USERNAME
118 Name of the user to use for logging into the remote host over
119 SSH. The default user name is 'root'.
120 -K PRIVKEY, --priv-key PRIVKEY
123 Path to the private SSH key that should be used for logging into
124 the remote host. By default the key is automatically found from
125 standard paths like '~/.ssh'.
126 -T TIMEOUT, --timeout TIMEOUT
129 SSH connect timeout in seconds, default is 8.
130 --skip-drivers
133 Deploy the eBPF helper, but do not deploy the drivers. This is a
134 debug and development option, do not use it for other purposes.
135
138 usage: wult scan [-h] [-q] [-d] [--all] [-H HOSTNAME] [-U USERNAME] [-K
139 PRIVKEY] [-T TIMEOUT]
140 Scan for available devices.
142
145 -h Show this help message and exit.
146 -q Be quiet.
149 -d Print debugging information.
152 --all By default this command prints only the compatible devices which
155 are supported by current wult installation. This option makes
156 this command print about all the compatible devices.
157 -H HOSTNAME, --host HOSTNAME
160 Name of the host to run the command on.
161 -U USERNAME, --username USERNAME
164 Name of the user to use for logging into the remote host over
165 SSH. The default user name is 'root'.
166 -K PRIVKEY, --priv-key PRIVKEY
169 Path to the private SSH key that should be used for logging into
170 the remote host. By default the key is automatically found from
171 standard paths like '~/.ssh'.
172 -T TIMEOUT, --timeout TIMEOUT
175 SSH connect timeout in seconds, default is 8.
176
179 usage: wult start [-h] [-q] [-d] [-H HOSTNAME] [-U USERNAME] [-K
180 PRIVKEY] [-T TIMEOUT] [-c COUNT] [--time-limit LIMIT] [--exclude EX‐
181 CLUDE] [--include INCLUDE] [--keep-filtered] [-o OUTDIR] [--reportid
182 REPORTID] [--stats STATS]
183 [--stats-intervals STATS_INTERVALS] [--list-stats]
184 [-l LDIST] [--cpunum CPUNUM] [--tsc-cal-time TSC_CAL_TIME]
185 [--keep-raw-data] [--no-unload] [--report] [--force]
186 devid
187 Start measuring and recording C-state latency.
189 devid The ID of the device to use for measuring the latency. For exam‐
192 ple, it can be a PCI address of the Intel I210 device, or "tdt"
193 for the TSC deadline timer block of the CPU. Use the 'scan' com‐
194 mand to get supported devices.
195
198 -h Show this help message and exit.
199 -q Be quiet.
202 -d Print debugging information.
205 -H HOSTNAME, --host HOSTNAME
208 Name of the host to run the command on.
209 -U USERNAME, --username USERNAME
212 Name of the user to use for logging into the remote host over
213 SSH. The default user name is 'root'.
214 -K PRIVKEY, --priv-key PRIVKEY
217 Path to the private SSH key that should be used for logging into
218 the remote host. By default the key is automatically found from
219 standard paths like '~/.ssh'.
220 -T TIMEOUT, --timeout TIMEOUT
223 SSH connect timeout in seconds, default is 8.
224 -c COUNT, --datapoints COUNT
227 How many datapoints should the test result include, default is
228 1000000. Note, unless the '--start-over' option is used, the
229 pre-existing datapoints are taken into account. For example, if
230 the test result already has 6000 datapoints and memory.
231 --time-limit LIMIT
234 The measurement time limit, i.e., for how long the SUT should be
235 measured. The default unit is minute, but you can use the fol‐
236 lowing handy specifiers as well: d - days, h - hours, m - min‐
237 utes, s - seconds. For example '1h25m' would be 1 hour and 25
238 minutes, or 10m5s would be 10 minutes and 5 seconds. Value '0'
239 means "no time limit", and this is the default. If this option
240 is used along with the '--datapoints' option, then measurements
241 will stop as when either the time limit is reached, or the re‐
242 quired amount of datapoints is collected.
243 --exclude EXCLUDE
246 Datapoints to exclude: remove all the datapoints satisfying the
247 expression 'EXCLUDE'. Here is an example of an expression:
248 '(WakeLatency < 10000) | (PC6% < 1)'. This filter expression
249 will remove all datapoints with 'WakeLatency' smaller than 10000
250 nanoseconds or package C6 residency smaller than 1%. You can use
251 any metrics in the expression.
252 --include INCLUDE
255 Datapoints to include: remove all datapoints except for those
256 satisfying the expression 'INCLUDE'. In other words, this option
257 is the inverse of '--exclude'. This means, '--include expr' is
258 the same as '--exclude "not (expr)"'.
259 --keep-filtered
262 If the '--exclude' / '--include' options are used, then the dat‐
263 apoints not matching the selector or matching the filter are
264 discarded. This is the default behavior which can be changed
265 with this option. If '--keep-filtered' has been specified, then
266 all datapoints are saved in result. Here is an example. Suppose
267 you want to collect 100000 datapoints where PC6 residency is
268 greater than 0. In this case, you can use these options: -c
269 100000 --exclude="PC6% == 0". The result will contain 100000
270 datapoints, all of them will have non-zero PC6 residency. But
271 what if you do not want to simply discard the other datapoints,
272 because they are also interesting? Well, add the '--keep-fil‐
273 tered' option. The result will contain, say, 150000 datapoints,
274 100000 of which will have non-zero PC6 residency.
275 -o OUTDIR, --outdir OUTDIR
278 Path to the directory to store the results at.
279 --reportid REPORTID
282 Any string which may serve as an identifier of this run. By de‐
283 fault report ID is the current date, prefixed with the remote
284 host name in case the '-H' option was used: [hostname-]YYYYMMDD.
285 For example, "20150323" is a report ID for a run made on March
286 23, 2015. The allowed characters are: ACSII alphanumeric, '-',
287 '.', ',', '_', '~', and ':'.
288 --stats STATS
291 Comma-separated list of statistics to collect. The statistics
292 are collected in parallel with measuring C-state latency. They
293 are stored in the the "stats" sub-directory of the output direc‐
294 tory. By default, only 'turbostat, sysinfo' statistics are col‐
295 lected. Use 'all' to collect all possible statistics. Use
296 '--stats=""' or '--stats="none"' to disable statistics collec‐
297 tion. If you know exactly what statistics you need, specify the
298 comma-separated list of statistics to collect. For example, use
299 'turbostat,acpower' if you need only turbostat and AC power me‐
300 ter statistics. You can also specify the statistics you do not
301 want to be collected by pre-pending the '!' symbol. For example,
302 'all,!turbostat' would mean: collect all the statistics sup‐
303 ported by the SUT, except for 'turbostat'. Use the
304 '--list-stats' option to get more information about available
305 statistics. By default, only 'sysinfo' statistics are collected.
306 --stats-intervals STATS_INTERVALS
309 The intervals for statistics. Statistics collection is based on
310 doing periodic snapshots of data. For example, by default the
311 'acpower' statistics collector reads SUT power consumption for
312 the last second every second, and 'turbostat' default interval
313 is 5 seconds. Use 'acpower:5,turbostat:10' to increase the in‐
314 tervals to 5 and 10 seconds correspondingly. Use the
315 '--list-stats' to get the default interval values.
316 --list-stats
319 Print information about the statistics 'wult' can collect and
320 exit.
321 -l LDIST, --ldist LDIST
324 This tool works by scheduling a delayed event, then sleeping and
325 waiting for it to happen. This step is referred to as a "mea‐
326 surement cycle" and it is usually repeated many times. The
327 launch distance defines how far in the future the delayed event
328 is scheduled. By default this tool randomly selects launch dis‐
329 tance within a range. The default range is [0,4ms], but you can
330 override it with this option. Specify a comma-separated range
331 (e.g '--ldist 10,5000'), or a single value if you want launch
332 distance to be precisely that value all the time. The default
333 unit is microseconds, but you can use the following specifiers
334 as well: ms - milliseconds, us - microseconds, ns - nanoseconds.
335 For example, ' --ldist 10us,5ms' would be a [10,5000] microsec‐
336 onds range. Too small values may cause failures or prevent the
337 SUT from reaching deep C-states. If the range starts with 0, the
338 minimum possible launch distance value allowed by the delayed
339 event source will be used. The optimal launch distance range is
340 system-specific.
341 --cpunum CPUNUM
344 The logical CPU number to measure, default is CPU 0.
345 --tsc-cal-time TSC_CAL_TIME
348 Wult receives raw datapoints from the driver, then processes
349 them, and then saves the processed datapoint in the 'data‐
350 points.csv' file. The processing involves converting TSC cycles
351 to microseconds, so wult needs SUT's TSC rate. TSC rate is cal‐
352 culated from the datapoints, which come with TSC counters and
353 timestamps, so TSC rate can be calculated as "delta TSC / delta
354 timestamp". In other words, wult needs two datapoints to calcu‐
355 late TSC rate. However, the datapoints have to be far enough
356 apart, and this option defines the distance between the data‐
357 points (in seconds). The default distance is 10 seconds, which
358 means that wult will keep collecting and buffering datapoints
359 for 10s without processing them (because processing requires TSC
360 rate to be known). After 10s, wult will start processing all the
361 buffered datapoints, and then the newly collected datapoints.
362 Generally, longer TSC calculation time translates to better ac‐
363 curacy.
364 --keep-raw-data
367 Wult receives raw datapoints from the driver, then processes
368 them, and then saves the processed datapoint in the 'data‐
369 points.csv' file. In order to keep the CSV file smaller, wult
370 keeps only the essential information, and drops the rest. For
371 example, raw timestamps are dropped. With this option, however,
372 wult saves all the raw data to the CSV file, along with the pro‐
373 cessed data.
374 --no-unload
377 This option exists for debugging and troubleshooting purposes.
378 Please, do not use for other reasons. If wult loads kernel mod‐
379 ules, they get unloaded after the measurements are done. But
380 with this option wult will not unload the modules.
381 --report
384 Generate an HTML report for collected results (same as calling
385 'report' command with default arguments).
386 --force
389 By default a network card is not accepted as a measurement de‐
390 vice if it is used by a Linux network interface and the inter‐
391 face is in an active state, such as "up". Use '--force' to dis‐
392 able this safety mechanism. Use it with caution.
393
396 usage: wult report [-h] [-q] [-d] [-o OUTDIR] [--exclude EXCLUDE]
397 [--include INCLUDE] [--even-up-dp-count] [-x XAXES] [-y YAXES] [--hist
398 HIST] [--chist CHIST] [--reportids REPORTIDS] [--report-descr RE‐
399 PORT_DESCR] [--relocatable]
400 [--list-metrics] [--size REPORT_SIZE]
401 respaths [respaths ...]
402 Create an HTML report for one or multiple test results.
404 respaths
407 One or multiple wult test result paths.
408
411 -h Show this help message and exit.
412 -q Be quiet.
415 -d Print debugging information.
418 -o OUTDIR, --outdir OUTDIR
421 Path to the directory to store the report at. By default the re‐
422 port is stored in the 'wult-report-<reportid>' sub-directory of
423 the test result directory. If there are multiple test results,
424 the report is stored in the current directory. The '<reportid>'
425 is report ID of wult test result.
426 --exclude EXCLUDE
429 Datapoints to exclude: remove all the datapoints satisfying the
430 expression 'EXCLUDE'. Here is an example of an expression:
431 '(WakeLatency < 10000) | (PC6% < 1)'. This filter expression
432 will remove all datapoints with 'WakeLatency' smaller than 10000
433 nanoseconds or package C6 residency smaller than 1%. The de‐
434 tailed expression syntax can be found in the documentation for
435 the 'eval()' function of Python 'pandas' module. You can use
436 metrics in the expression, or the special word 'index' for the
437 row number (0-based index) of a datapoint in the results. For
438 example, expression 'index >= 10' will get rid of all datapoints
439 except for the first 10 ones.
440 --include INCLUDE
443 Datapoints to include: remove all datapoints except for those
444 satisfying the expression 'INCLUDE'. In other words, this option
445 is the inverse of '--exclude'. This means, '--include expr' is
446 the same as '--exclude "not (expr)"'.
447 --even-up-dp-count
450 Even up datapoints count before generating the report. This op‐
451 tion is useful when generating a report for many test results (a
452 diff). If the test results contain different count of datapoints
453 (rows count in the CSV file), the resulting histograms may look
454 a little bit misleading. This option evens up datapoints count
455 in the test results. It just finds the test result with the min‐
456 imum count of datapoints and ignores the extra datapoints in the
457 other test results.
458 -x XAXES, --xaxes XAXES
461 A comma-separated list of metrics (or python style regular ex‐
462 pressions matching the names) to use on X-axes of the scatter
463 plot(s), default is 'SilentTime'. Use '--list-metrics' to get
464 the list of the available metrics. Use value 'none' to disable
465 scatter plots.
466 -y YAXES, --yaxes YAXES
469 A comma-separated list of metrics (or python style regular ex‐
470 pressions matching the names) to use on the Y-axes for the scat‐
471 ter plot(s). If multiple metrics are specified for the X- or
472 Y-axes, then the report will include multiple scatter plots for
473 all the X- and Y-axes combinations. The default is '.*Latency'.
474 Use '--list-metrics' to get the list of the available metrics.
475 Use value 'none' to disable scatter plots.
476 --hist HIST
479 A comma-separated list of metrics (or python style regular ex‐
480 pressions matching the names) to add a histogram for, default is
481 '.*Latency'. Use '--list-metrics' to get the list of the avail‐
482 able metrics. Use value 'none' to disable histograms.
483 --chist CHIST
486 A comma-separated list of metrics (or python style regular ex‐
487 pressions matching the names) to add a cumulative distribution
488 for, default is 'None'. Use '--list-metrics' to get the list of
489 the available metrics. Use value 'none' to disable cumulative
490 histograms.
491 --reportids REPORTIDS
494 Every input raw result comes with a report ID. This report ID is
495 basically a short name for the test result, and it used in the
496 HTML report to refer to the test result. However, sometimes it
497 is helpful to temporarily override the report IDs just for the
498 HTML report, and this is what the '--reportids' option does.
499 Please, specify a comma-separated list of report IDs for every
500 input raw test result. The first report ID will be used for the
501 first raw rest result, the second report ID will be used for the
502 second raw test result, and so on. Please, refer to the '--re‐
503 portid' option description in the 'start' command for more in‐
504 formation about the report ID.
505 --report-descr REPORT_DESCR
508 The report description - any text describing this report as
509 whole, or path to a file containing the overall report descrip‐
510 tion. For example, if the report compares platform A and plat‐
511 form B, the description could be something like
512 --relocatable
515 Generate a report which contains a copy of the raw test results.
516 With this option, viewers of the report will also be able to
517 browse raw statistics files which are copied across with the raw
518 test results.
519 --list-metrics
522 Print the list of the available metrics and exit.
523 --size REPORT_SIZE
526 Generate HTML report with a pre-defined set of diagrams and his‐
527 tograms. Possible values: 'small' or 'large'. This option is mu‐
528 tually exclusive with '--xaxes', '--yaxes', '--hist', '--chist'.
529
532 usage: wult filter [-h] [-q] [-d] [--exclude EXCLUDE] [--include IN‐
533 CLUDE] [--exclude-metrics MEXCLUDE] [--include-metrics MINCLUDE] [--hu‐
534 man-readable] [-o OUTDIR] [--list-metrics] [--reportid REPORTID]
535 respath
536 Filter datapoints out of a test result by removing CSV rows and metrics
538 according to specified criteria. The criteria is specified using the
539 row and metric filter and selector options ('--include', '--ex‐
540 clude-metrics', etc). The options may be specified multiple times.
541 respath
544 The wult test result path to filter.
545
548 -h Show this help message and exit.
549 -q Be quiet.
552 -d Print debugging information.
555 --exclude EXCLUDE
558 Datapoints to exclude: remove all the datapoints satisfying the
559 expression 'EXCLUDE'. Here is an example of an expression:
560 '(WakeLatency < 10000) | (PC6% < 1)'. This filter expression
561 will remove all datapoints with 'WakeLatency' smaller than 10000
562 nanoseconds or package C6 residency smaller than 1%. The de‐
563 tailed expression syntax can be found in the documentation for
564 the 'eval()' function of Python 'pandas' module. You can use
565 metrics in the expression, or the special word 'index' for the
566 row number (0-based index) of a datapoint in the results. For
567 example, expression 'index >= 10' will get rid of all datapoints
568 except for the first 10 ones.
569 --include INCLUDE
572 Datapoints to include: remove all datapoints except for those
573 satisfying the expression 'INCLUDE'. In other words, this option
574 is the inverse of '--exclude'. This means, '--include expr' is
575 the same as '--exclude "not (expr)"'.
576 --exclude-metrics MEXCLUDE
579 The metrics to exclude. Expects a comma-separated list of the
580 metrics or python style regular expressions matching the names.
581 For example, the expression 'SilentTime,WarmupDelay,.*Cyc',
582 would remove metrics 'SilentTime', 'WarmupDelay' and all metrics
583 with 'Cyc' in their name. Use '--list-metrics' to get the list
584 of the available metrics.
585 --include-metrics MINCLUDE
588 The metrics to include: remove all metrics except for those
589 specified by this option. The syntax is the same as for '--ex‐
590 clude-metrics'.
591 --human-readable
594 By default the result 'filter' command print the result as a CSV
595 file to the standard output. This option can be used to dump the
596 result in a more human-readable form.
597 -o OUTDIR, --outdir OUTDIR
600 By default the resulting CSV lines are printed to the standard
601 output. But this option can be used to specify the output di‐
602 rectly to store the result at. This will create a filtered ver‐
603 sion of the input test result.
604 --list-metrics
607 Print the list of the available metrics and exit.
608 --reportid REPORTID
611 Report ID of the filtered version of the result (can only be
612 used with '--outdir').
613
616 usage: wult calc [-h] [-q] [-d] [--exclude EXCLUDE] [--include INCLUDE]
617 [--exclude-metrics MEXCLUDE] [--include-metrics MINCLUDE] [-f FUNCS]
618 [--list-funcs] [--list-metrics] [respath]
619 Calculates various summary functions for a wult test result (e.g., the
621 median value for one of the CSV columns).
622 respath
625 The wult test result path to calculate summary functions for.
626
629 -h Show this help message and exit.
630 -q Be quiet.
633 -d Print debugging information.
636 --exclude EXCLUDE
639 Datapoints to exclude: remove all the datapoints satisfying the
640 expression 'EXCLUDE'. Here is an example of an expression:
641 '(WakeLatency < 10000) | (PC6% < 1)'. This filter expression
642 will remove all datapoints with 'WakeLatency' smaller than 10000
643 nanoseconds or package C6 residency smaller than 1%. The de‐
644 tailed expression syntax can be found in the documentation for
645 the 'eval()' function of Python 'pandas' module. You can use
646 metrics in the expression, or the special word 'index' for the
647 row number (0-based index) of a datapoint in the results. For
648 example, expression 'index >= 10' will get rid of all datapoints
649 except for the first 10 ones.
650 --include INCLUDE
653 Datapoints to include: remove all datapoints except for those
654 satisfying the expression 'INCLUDE'. In other words, this option
655 is the inverse of '--exclude'. This means, '--include expr' is
656 the same as '--exclude "not (expr)"'.
657 --exclude-metrics MEXCLUDE
660 The metrics to exclude. Expects a comma-separated list of the
661 metrics or python style regular expressions matching the names.
662 For example, the expression 'SilentTime,WarmupDelay,.*Cyc',
663 would remove metrics 'SilentTime', 'WarmupDelay' and all metrics
664 with 'Cyc' in their name. Use '--list-metrics' to get the list
665 of the available metrics.
666 --include-metrics MINCLUDE
669 The metrics to include: remove all metrics except for those
670 specified by this option. The syntax is the same as for '--ex‐
671 clude-metrics'.
672 -f FUNCS, --funcs FUNCS
675 Comma-separated list of summary functions to calculate. By de‐
676 fault all generally interesting functions are calculated (each
677 metric is associated with a list of functions that make sense
678 for that metric). Use '--list-funcs' to get the list of sup‐
679 ported functions.
680 --list-funcs
683 Print the list of the available summary functions.
684 --list-metrics
687 Print the list of the available metrics and exit.
688
691 Artem Bityutskiy
692 dedekind1@gmail.com
693
696 The latest version of wult may be downloaded from
697 ⟨https://github.com/intel/wult⟩
698wult 2023-08-18 WULT(1)