1NDL(1) Generated Python Manual NDL(1)
2
6 ndl
7
9 ndl [-h] [-q] [-d] [--version] [--force-color] {deploy,scan,start,re‐
10 port,filter,calc} ...
11
13 ndl - a tool for measuring memory access latency observed by a network
14 card.
15
18 -h Show this help message and exit.
19 -q Be quiet.
22 -d Print debugging information.
25 --version
28 Print version and exit.
29 --force-color
32 Force coloring of the text output.
33
36 ndl deploy
37 Compile and deploy ndl helpers and drivers.
38 ndl scan
40 Scan for available devices.
41 ndl start
43 Start the measurements.
44 ndl report
46 Create an HTML report.
47 ndl filter
49 Filter datapoints out of a test result.
50 ndl calc
52 Calculate summary functions for a ndl test result.
53
56 usage: ndl deploy [-h] [-q] [-d] [--kernel-src KSRC] [--local-build]
57 [--tmpdir-path TMPDIR_PATH] [--keep-tmpdir] [-H HOSTNAME] [-U USERNAME]
58 [-K PRIVKEY] [-T TIMEOUT]
59 Compile and deploy ndl helpers and drivers to the SUT (System Under
61 Test), which can be can be either local or a remote host, depending on
62 the '-H' option. By default, everything is built on the SUT, but the
63 '--local-build' can be used for building on the local system. The driv‐
64 ers are searched for in the following directories (and in the following
65 order) on the local host: ./drivers/idle, $WULT_DATA_PATH/drivers/idle,
66 $HOME/.local/share/wult/drivers/idle, /usr/local/share/wult/driv‐
67 ers/idle, /usr/share/wult/drivers/idle. The ndl tool also depends on
68 the following helpers: ndl-helper. These helpers will be compiled on
69 the SUT and deployed to the SUT. The sources of the helpers are
70 searched for in the following paths (and in the following order) on the
71 local host: ./helpers, $WULT_DATA_PATH/helpers, $HOME/.lo‐
72 cal/share/wult/helpers, /usr/local/share/wult/helpers,
73 /usr/share/wult/helpers. By default, helpers are deployed to the path
74 defined by the 'WULT_HELPERSPATH' environment variable. If the variable
75 is not defined, helpers are deployed to '$HOME/.local/bin', where
76 '$HOME' is the home directory of user 'USERNAME' on host 'HOST' (see
77 '--host' and '--username' options).
78
81 -h Show this help message and exit.
82 -q Be quiet.
85 -d Print debugging information.
88 --kernel-src KSRC
91 Path to the Linux kernel sources to build drivers and eBPF
92 helpers against. The default is '/lib/modules/$(uname -r)/build'
93 on the SUT. If '--local-build' was used, then the path is con‐
94 sidered to be on the local system, rather than the SUT.
95 --local-build
98 Build helpers and drivers locally, instead of building on HOST‐
99 NAME (the SUT).
100 --tmpdir-path TMPDIR_PATH
103 When 'ndl' is deployed, a random temporary directory is used.
104 Use this option provide a custom path instead. It will be used
105 as a temporary directory on both local and remote hosts. This
106 option is meant for debugging purposes.
107 --keep-tmpdir
110 Do not remove the temporary directories created while deploying
111 'ndl'. This option is meant for debugging purposes.
112 -H HOSTNAME, --host HOSTNAME
115 Name of the host to run the command on.
116 -U USERNAME, --username USERNAME
119 Name of the user to use for logging into the remote host over
120 SSH. The default user name is 'root'.
121 -K PRIVKEY, --priv-key PRIVKEY
124 Path to the private SSH key that should be used for logging into
125 the remote host. By default the key is automatically found from
126 standard paths like '~/.ssh'.
127 -T TIMEOUT, --timeout TIMEOUT
130 SSH connect timeout in seconds, default is 8.
131
134 usage: ndl scan [-h] [-q] [-d] [--all] [-H HOSTNAME] [-U USERNAME] [-K
135 PRIVKEY] [-T TIMEOUT]
136 Scan for available devices.
138
141 -h Show this help message and exit.
142 -q Be quiet.
145 -d Print debugging information.
148 --all By default this command prints only the compatible devices which
151 are supported by current ndl installation. This option makes
152 this command print about all the compatible devices.
153 -H HOSTNAME, --host HOSTNAME
156 Name of the host to run the command on.
157 -U USERNAME, --username USERNAME
160 Name of the user to use for logging into the remote host over
161 SSH. The default user name is 'root'.
162 -K PRIVKEY, --priv-key PRIVKEY
165 Path to the private SSH key that should be used for logging into
166 the remote host. By default the key is automatically found from
167 standard paths like '~/.ssh'.
168 -T TIMEOUT, --timeout TIMEOUT
171 SSH connect timeout in seconds, default is 8.
172
175 usage: ndl start [-h] [-q] [-d] [-H HOSTNAME] [-U USERNAME] [-K
176 PRIVKEY] [-T TIMEOUT] [-c COUNT] [--time-limit LIMIT] [-o OUTDIR]
177 [--reportid REPORTID] [--stats STATS] [--stats-intervals STATS_INTER‐
178 VALS] [--list-stats] [-l LDIST]
179 [--cpunum CPUNUM] [--exclude EXCLUDE] [--include IN‐
180 CLUDE] [--keep-filtered] [--report] [--force]
181 ifname
182 Start measuring and recording the latency data.
184 ifname The network interface backed by the NIC to use for latency mea‐
187 surements. Today only Intel I210 and I211 NICs are supported.
188 Please, specify NIC's network interface name (e.g., eth0).
189
192 -h Show this help message and exit.
193 -q Be quiet.
196 -d Print debugging information.
199 -H HOSTNAME, --host HOSTNAME
202 Name of the host to run the command on.
203 -U USERNAME, --username USERNAME
206 Name of the user to use for logging into the remote host over
207 SSH. The default user name is 'root'.
208 -K PRIVKEY, --priv-key PRIVKEY
211 Path to the private SSH key that should be used for logging into
212 the remote host. By default the key is automatically found from
213 standard paths like '~/.ssh'.
214 -T TIMEOUT, --timeout TIMEOUT
217 SSH connect timeout in seconds, default is 8.
218 -c COUNT, --datapoints COUNT
221 How many datapoints should the test result include, default is
222 1000000. Note, unless the '--start-over' option is used, the
223 pre-existing datapoints are taken into account. For example, if
224 the test result already has 6000 datapoints and memory.
225 --time-limit LIMIT
228 The measurement time limit, i.e., for how long the SUT should be
229 measured. The default unit is minute, but you can use the fol‐
230 lowing handy specifiers as well: d - days, h - hours, m - min‐
231 utes, s - seconds. For example '1h25m' would be 1 hour and 25
232 minutes, or 10m5s would be 10 minutes and 5 seconds. Value '0'
233 means "no time limit", and this is the default. If this option
234 is used along with the '--datapoints' option, then measurements
235 will stop as when either the time limit is reached, or the re‐
236 quired amount of datapoints is collected.
237 -o OUTDIR, --outdir OUTDIR
240 Path to the directory to store the results at.
241 --reportid REPORTID
244 Any string which may serve as an identifier of this run. By de‐
245 fault report ID is the current date, prefixed with the remote
246 host name in case the '-H' option was used: [hostname-]YYYYMMDD.
247 For example, "20150323" is a report ID for a run made on March
248 23, 2015. The allowed characters are: ACSII alphanumeric, '-',
249 '.', ',', '_', '~', and ':'.
250 --stats STATS
253 Comma-separated list of statistics to collect. The statistics
254 are collected in parallel with measuring C-state latency. They
255 are stored in the the "stats" sub-directory of the output direc‐
256 tory. By default, only 'turbostat, sysinfo' statistics are col‐
257 lected. Use 'all' to collect all possible statistics. Use
258 '--stats=""' or '--stats="none"' to disable statistics collec‐
259 tion. If you know exactly what statistics you need, specify the
260 comma-separated list of statistics to collect. For example, use
261 'turbostat,acpower' if you need only turbostat and AC power me‐
262 ter statistics. You can also specify the statistics you do not
263 want to be collected by pre-pending the '!' symbol. For example,
264 'all,!turbostat' would mean: collect all the statistics sup‐
265 ported by the SUT, except for 'turbostat'. Use the
266 '--list-stats' option to get more information about available
267 statistics. By default, only 'sysinfo' statistics are collected.
268 --stats-intervals STATS_INTERVALS
271 The intervals for statistics. Statistics collection is based on
272 doing periodic snapshots of data. For example, by default the
273 'acpower' statistics collector reads SUT power consumption for
274 the last second every second, and 'turbostat' default interval
275 is 5 seconds. Use 'acpower:5,turbostat:10' to increase the in‐
276 tervals to 5 and 10 seconds correspondingly. Use the
277 '--list-stats' to get the default interval values.
278 --list-stats
281 Print information about the statistics 'ndl' can collect and
282 exit.
283 -l LDIST, --ldist LDIST
286 The launch distance in microseconds. This tool works by schedul‐
287 ing a delayed network packet, then sleeping and waiting for the
288 packet to be sent. This step is referred to as a "measurement
289 cycle" and it is usually repeated many times. The launch dis‐
290 tance defines how far in the future the delayed network packets
291 are scheduled. By default this tool randomly selects launch dis‐
292 tance in range of [5000, 50000] microseconds (same as '--ldist
293 5000,50000'). Specify a comma- separated range or a single value
294 if you want launch distance to be precisely that value all the
295 time. The default unit is microseconds, but you can use the fol‐
296 lowing specifiers as well: ms - milliseconds, us - microseconds,
297 ns - nanoseconds. For example, '--ldist 500us,100ms' would be a
298 [500,100000] microseconds range. Note, too low values may cause
299 failures or prevent the SUT from reaching deep C-states. The op‐
300 timal value is system-specific.
301 --cpunum CPUNUM
304 The CPU number to bind the helper to. The helper will use this
305 CPU to send delayed packets. In normal conditions this means
306 that network packet buffers will be allocated on the NUMA node
307 local to the CPU, but not necessarily local to the network card.
308 Use this option to measure different packet memory locations on
309 a NUMA system. Default is the first CPU local to the NIC.
310 --exclude EXCLUDE
313 Datapoints to exclude: remove all the datapoints satisfying the
314 expression 'EXCLUDE'. Here is an example of an expression:
315 '(WakeLatency < 10000) | (PC6% < 1)'. This filter expression
316 will remove all datapoints with 'WakeLatency' smaller than 10000
317 nanoseconds or package C6 residency smaller than 1%. You can use
318 any metrics in the expression.
319 --include INCLUDE
322 Datapoints to include: remove all datapoints except for those
323 satisfying the expression 'INCLUDE'. In other words, this option
324 is the inverse of '--exclude'. This means, '--include expr' is
325 the same as '--exclude "not (expr)"'.
326 --keep-filtered
329 If the '--exclude' / '--include' options are used, then the dat‐
330 apoints not matching the selector or matching the filter are
331 discarded. This is the default behavior which can be changed
332 with this option. If '--keep-filtered' has been specified, then
333 all datapoints are saved in result. Here is an example. Suppose
334 you want to collect 100000 datapoints where RTD is greater than
335 50 microseconds. In this case, you can use these options: -c
336 100000 --exclude="RTD > 50". The result will contain 100000
337 datapoints, all of them will have RTD bigger than 50 microsec‐
338 onds. But what if you do not want to simply discard the other
339 datapoints, because they are also interesting? Well, add the
340 '--keep-filtered' option. The result will contain, say, 150000
341 datapoints, 100000 of which will have RTD value greater than 50.
342 --report
345 Generate an HTML report for collected results (same as calling
346 'report' command with default arguments).
347 --force
350 By default a network card is not accepted as a measurement de‐
351 vice if it is used by a Linux network interface and the inter‐
352 face is in an active state, such as "up". Use '--force' to dis‐
353 able this safety mechanism. Use it with caution.
354
357 usage: ndl report [-h] [-q] [-d] [-o OUTDIR] [--exclude EXCLUDE] [--in‐
358 clude INCLUDE] [--even-up-dp-count] [-x XAXES] [-y YAXES] [--hist HIST]
359 [--chist CHIST] [--reportids REPORTIDS] [--report-descr REPORT_DESCR]
360 [--relocatable]
361 [--list-metrics]
362 respaths [respaths ...]
363 Create an HTML report for one or multiple test results.
365 respaths
368 One or multiple ndl test result paths.
369
372 -h Show this help message and exit.
373 -q Be quiet.
376 -d Print debugging information.
379 -o OUTDIR, --outdir OUTDIR
382 Path to the directory to store the report at. By default the re‐
383 port is stored in the 'ndl-report-<reportid>' sub-directory of
384 the test result directory. If there are multiple test results,
385 the report is stored in the current directory. The '<reportid>'
386 is report ID of ndl test result.
387 --exclude EXCLUDE
390 Datapoints to exclude: remove all the datapoints satisfying the
391 expression 'EXCLUDE'. Here is an example of an expression:
392 '(WakeLatency < 10000) | (PC6% < 1)'. This filter expression
393 will remove all datapoints with 'WakeLatency' smaller than 10000
394 nanoseconds or package C6 residency smaller than 1%. The de‐
395 tailed expression syntax can be found in the documentation for
396 the 'eval()' function of Python 'pandas' module. You can use
397 metrics in the expression, or the special word 'index' for the
398 row number (0-based index) of a datapoint in the results. For
399 example, expression 'index >= 10' will get rid of all datapoints
400 except for the first 10 ones.
401 --include INCLUDE
404 Datapoints to include: remove all datapoints except for those
405 satisfying the expression 'INCLUDE'. In other words, this option
406 is the inverse of '--exclude'. This means, '--include expr' is
407 the same as '--exclude "not (expr)"'.
408 --even-up-dp-count
411 Even up datapoints count before generating the report. This op‐
412 tion is useful when generating a report for many test results (a
413 diff). If the test results contain different count of datapoints
414 (rows count in the CSV file), the resulting histograms may look
415 a little bit misleading. This option evens up datapoints count
416 in the test results. It just finds the test result with the min‐
417 imum count of datapoints and ignores the extra datapoints in the
418 other test results.
419 -x XAXES, --xaxes XAXES
422 A comma-separated list of metrics (or python style regular ex‐
423 pressions matching the names) to use on X-axes of the scatter
424 plot(s), default is 'LDist'. Use '--list-metrics' to get the
425 list of the available metrics. Use value 'none' to disable scat‐
426 ter plots.
427 -y YAXES, --yaxes YAXES
430 A comma-separated list of metrics (or python style regular ex‐
431 pressions matching the names) to use on the Y-axes for the scat‐
432 ter plot(s). If multiple metrics are specified for the X- or
433 Y-axes, then the report will include multiple scatter plots for
434 all the X- and Y-axes combinations. The default is 'RTD'. Use
435 '--list-metrics' to get the list of the available metrics. Use
436 value 'none' to disable scatter plots.
437 --hist HIST
440 A comma-separated list of metrics (or python style regular ex‐
441 pressions matching the names) to add a histogram for, default is
442 'RTD'. Use '--list-metrics' to get the list of the available
443 metrics. Use value 'none' to disable histograms.
444 --chist CHIST
447 A comma-separated list of metrics (or python style regular ex‐
448 pressions matching the names) to add a cumulative distribution
449 for, default is 'RTD'. Use '--list-metrics' to get the list of
450 the available metrics. Use value 'none' to disable cumulative
451 histograms.
452 --reportids REPORTIDS
455 Every input raw result comes with a report ID. This report ID is
456 basically a short name for the test result, and it used in the
457 HTML report to refer to the test result. However, sometimes it
458 is helpful to temporarily override the report IDs just for the
459 HTML report, and this is what the '--reportids' option does.
460 Please, specify a comma-separated list of report IDs for every
461 input raw test result. The first report ID will be used for the
462 first raw rest result, the second report ID will be used for the
463 second raw test result, and so on. Please, refer to the '--re‐
464 portid' option description in the 'start' command for more in‐
465 formation about the report ID.
466 --report-descr REPORT_DESCR
469 The report description - any text describing this report as
470 whole, or path to a file containing the overall report descrip‐
471 tion. For example, if the report compares platform A and plat‐
472 form B, the description could be something like
473 --relocatable
476 Generate a report which contains a copy of the raw test results.
477 With this option, viewers of the report will also be able to
478 browse raw statistics files which are copied across with the raw
479 test results.
480 --list-metrics
483 Print the list of the available metrics and exit.
484
487 usage: ndl filter [-h] [-q] [-d] [--exclude EXCLUDE] [--include IN‐
488 CLUDE] [--exclude-metrics MEXCLUDE] [--include-metrics MINCLUDE] [--hu‐
489 man-readable] [-o OUTDIR] [--list-metrics] [--reportid REPORTID]
490 respath
491 Filter datapoints out of a test result by removing CSV rows and metrics
493 according to specified criteria. The criteria is specified using the
494 row and metric filter and selector options ('--include', '--ex‐
495 clude-metrics', etc). The options may be specified multiple times.
496 respath
499 The ndl test result path to filter.
500
503 -h Show this help message and exit.
504 -q Be quiet.
507 -d Print debugging information.
510 --exclude EXCLUDE
513 Datapoints to exclude: remove all the datapoints satisfying the
514 expression 'EXCLUDE'. Here is an example of an expression:
515 '(WakeLatency < 10000) | (PC6% < 1)'. This filter expression
516 will remove all datapoints with 'WakeLatency' smaller than 10000
517 nanoseconds or package C6 residency smaller than 1%. The de‐
518 tailed expression syntax can be found in the documentation for
519 the 'eval()' function of Python 'pandas' module. You can use
520 metrics in the expression, or the special word 'index' for the
521 row number (0-based index) of a datapoint in the results. For
522 example, expression 'index >= 10' will get rid of all datapoints
523 except for the first 10 ones.
524 --include INCLUDE
527 Datapoints to include: remove all datapoints except for those
528 satisfying the expression 'INCLUDE'. In other words, this option
529 is the inverse of '--exclude'. This means, '--include expr' is
530 the same as '--exclude "not (expr)"'.
531 --exclude-metrics MEXCLUDE
534 The metrics to exclude. Expects a comma-separated list of the
535 metrics or python style regular expressions matching the names.
536 For example, the expression 'SilentTime,WarmupDelay,.*Cyc',
537 would remove metrics 'SilentTime', 'WarmupDelay' and all metrics
538 with 'Cyc' in their name. Use '--list-metrics' to get the list
539 of the available metrics.
540 --include-metrics MINCLUDE
543 The metrics to include: remove all metrics except for those
544 specified by this option. The syntax is the same as for '--ex‐
545 clude-metrics'.
546 --human-readable
549 By default the result 'filter' command print the result as a CSV
550 file to the standard output. This option can be used to dump the
551 result in a more human-readable form.
552 -o OUTDIR, --outdir OUTDIR
555 By default the resulting CSV lines are printed to the standard
556 output. But this option can be used to specify the output di‐
557 rectly to store the result at. This will create a filtered ver‐
558 sion of the input test result.
559 --list-metrics
562 Print the list of the available metrics and exit.
563 --reportid REPORTID
566 Report ID of the filtered version of the result (can only be
567 used with '--outdir').
568
571 usage: ndl calc [-h] [-q] [-d] [--exclude EXCLUDE] [--include INCLUDE]
572 [--exclude-metrics MEXCLUDE] [--include-metrics MINCLUDE] [-f FUNCS]
573 [--list-funcs] [--list-metrics] [respath]
574 Calculates various summary functions for a ndl test result (e.g., the
576 median value for one of the CSV columns).
577 respath
580 The ndl test result path to calculate summary functions for.
581
584 -h Show this help message and exit.
585 -q Be quiet.
588 -d Print debugging information.
591 --exclude EXCLUDE
594 Datapoints to exclude: remove all the datapoints satisfying the
595 expression 'EXCLUDE'. Here is an example of an expression:
596 '(WakeLatency < 10000) | (PC6% < 1)'. This filter expression
597 will remove all datapoints with 'WakeLatency' smaller than 10000
598 nanoseconds or package C6 residency smaller than 1%. The de‐
599 tailed expression syntax can be found in the documentation for
600 the 'eval()' function of Python 'pandas' module. You can use
601 metrics in the expression, or the special word 'index' for the
602 row number (0-based index) of a datapoint in the results. For
603 example, expression 'index >= 10' will get rid of all datapoints
604 except for the first 10 ones.
605 --include INCLUDE
608 Datapoints to include: remove all datapoints except for those
609 satisfying the expression 'INCLUDE'. In other words, this option
610 is the inverse of '--exclude'. This means, '--include expr' is
611 the same as '--exclude "not (expr)"'.
612 --exclude-metrics MEXCLUDE
615 The metrics to exclude. Expects a comma-separated list of the
616 metrics or python style regular expressions matching the names.
617 For example, the expression 'SilentTime,WarmupDelay,.*Cyc',
618 would remove metrics 'SilentTime', 'WarmupDelay' and all metrics
619 with 'Cyc' in their name. Use '--list-metrics' to get the list
620 of the available metrics.
621 --include-metrics MINCLUDE
624 The metrics to include: remove all metrics except for those
625 specified by this option. The syntax is the same as for '--ex‐
626 clude-metrics'.
627 -f FUNCS, --funcs FUNCS
630 Comma-separated list of summary functions to calculate. By de‐
631 fault all generally interesting functions are calculated (each
632 metric is associated with a list of functions that make sense
633 for that metric). Use '--list-funcs' to get the list of sup‐
634 ported functions.
635 --list-funcs
638 Print the list of the available summary functions.
639 --list-metrics
642 Print the list of the available metrics and exit.
643
646 Artem Bityutskiy
647 dedekind1@gmail.com
648
651 The latest version of ndl may be downloaded from
652 ⟨https://github.com/intel/wult⟩
653ndl 2023-08-18 NDL(1)