1phoronix-test-suite(1) General Commands Manual phoronix-test-suite(1)
2
6 phoronix-test-suite - The Phoronix Test Suite is an extensible open-
7 source platform for performing testing and performance evaluation.
8
10 phoronix-test-suite [options]
11 phoronix-test-suite benchmark [test | suite]
12
14 The **Phoronix Test Suite** is the most comprehensive testing and
15 benchmarking platform available for Linux, Solaris, macOS, Windows, and
16 BSD operating systems. The Phoronix Test Suite allows for carrying out
17 tests in a fully automated manner from test installation to execution
18 and reporting. All tests are meant to be easily reproducible, easy-to-
19 use, and support fully automated execution. The Phoronix Test Suite is
20 open-source under the GNU GPLv3 license and is developed by Phoronix
21 Media in cooperation with partners.
22
24 SYSTEM
25 diagnostics This option will print information that is useful to
26 developers when debugging problems with the Phoronix Test Suite
27 and/or test profiles and test suites.
28 interactive
30 A simple text-driven interactive interface to the Phoronix Test
31 Suite.
32 php-conf
34 This option will print information that is useful to developers
35 when debugging problems with the Phoronix Test Suite and/or test
36 profiles and test suites.
37 shell A simple text-driven shell interface / helper to the Phoronix
39 Test Suite. Ideal for those that may be new to the Phoronix Test
40 Suite
41 system-info
43 Display the installed system hardware and software information
44 as detected by the Phoronix Test Suite Phodevi Library.
45 system-sensors
47 Display the installed system hardware and software sensors in
48 real-time as detected by the Phoronix Test Suite Phodevi
49 Library.
50 TEST INSTALLATION
53 force-install [Test | Suite | OpenBenchmarking ID | Test Result]
54 ... This option will force the installation (or re-installa‐
55 tion) of a test or suite. The arguments and process is similar
56 to the install option but even if the test is installed, the
57 entire installation process will automatically be executed. This
58 option is generally used when debugging a test installation
59 problem or wishing to re-install test(s) due to compiler or
60 other environmental changes.
61 install [Test | Suite | OpenBenchmarking ID | Test Result] ...
63 This option will install the selected test(s) inside the testing
64 environment directory. The install process from downloading of
65 the test files to the installation is fully automated. The
66 install option needs to be supplied with the test name or suite
67 as an argument. Optionally, a OpenBenchmarking.org ID or the
68 name of a saved results file can be supplied as well and the
69 test(s) to install will automatically be extracted from that
70 information. If the test is already installed and was run by the
71 latest version of the installation process, no action will be
72 taken. Multiple arguments can be supplied to install additional
73 tests at the same time.
74 install-dependencies [Test | Suite | OpenBenchmarking ID | Test Result]
76 ...
77 This option will install the external dependencies needed by the
78 selected test(s) using the distribution's package management
79 system. For example, some tests depend upon GCC for compiling
80 code. If GCC is not detected on the system, the Phoronix Test
81 Suite will attempt to install GCC using the distribution's pack‐
82 age management system. If you are running this command as a
83 local user, you may be prompted for the root password while the
84 process is running. For unsupported distributions, the depen‐
85 dency names will be displayed along with common names for the
86 package. The install-dependencies option needs to be supplied
87 with the test name or suite as an argument. When using the
88 install option, the external dependencies are automatically
89 checked.
90 make-download-cache
92 This option will create a download cache for use by the Phoronix
93 Test Suite. The download cache is created of test files already
94 downloaded to the local system. If passing any test/suite names
95 to make-download-cache, the needed files for those test profiles
96 will first be automatically downloaded before creating the
97 cache.
98 remove-installed-test [Test]
100 This option will permanently remove a installed test by the
101 Phoronix Test Suite.
102 TESTING
105 auto-compare This option will autonomously determine the most
106 relevant test(s) to run for any selected sub-system(s). The
107 tests to run are determined via OpenBenchmarking.org integration
108 with the global results pool. Related test results from Open‐
109 Benchmarking.org are also merged to provide a straight-forward
110 and effective means of carrying out a system comparison. If
111 wishing to find comparable results for any particular test pro‐
112 file(s), simply pass the test profile names as additional argu‐
113 ments to this command.
114 benchmark [Test | Suite | OpenBenchmarking ID | Test Result] ...
116 This option will install the selected test(s) (if needed) and
117 will proceed to run the test(s). This option is equivalent to
118 running phoronix-test-suite with the install option followed by
119 the run option. Multiple arguments can be supplied to run addi‐
120 tional tests at the same time and save the results into one
121 file.
122 estimate-run-time [Test | Suite | OpenBenchmarking ID | Test Result]
124 This option will provide estimates for test run-time / length.
125 finish-run [Test Result]
127 This option can be used if a test run had not properly finished
128 running all tests within a saved results file. Using this option
129 when specifying a saved results file where all tests had not
130 completed will attempt to finish testing on the remaining tests
131 where there are missing results.
132 run [Test | Suite | OpenBenchmarking ID | Test Result] ...
134 This option will run the selected test(s).
135 run-random-tests
137 This option will query OpenBenchmarking.org to run random bench‐
138 marks and result comparisons on the system. This test can be
139 used for simply supplying interesting results from your system
140 onto OpenBenchmarking.org, stressing your system with random
141 workloads, seeding new OpenBenchmarking.org results, etc. Basic
142 options are provided at start-up for tuning the randomness of
143 the testing when running this command.
144 run-tests-in-suite
146 This option can be used if you wish to run all of the tests
147 found in a supplied suite, but you wish to re-configure each of
148 the test options rather than using the defaults supplied by the
149 suite.
150 stress-run [Test | Suite | OpenBenchmarking ID | Test Result] ...
152 This option will run the passed tests/suites in the multi-
153 process stress-testing mode. The stress-run mode will not pro‐
154 duce a result file but is rather intended for running multiple
155 test profiles concurrently to stress / burn-in the system. The
156 number of tests to run concurrently can be toggled via the
157 PTS_CONCURRENT_TEST_RUNS environment variable and by default is
158 set to a value of 2.
159 BATCH TESTING
162 batch-benchmark [Test | Suite | OpenBenchmarking ID | Test
163 Result] ... This option and its arguments are equivalent to the
164 benchmark option, but the process will be run in the Phoronix
165 Test Suite batch mode.
166 batch-install [Test | Suite | OpenBenchmarking ID | Test Result] ...
168 If you wish to run the install process in the Phoronix Test
169 Suite batch mode but do not wish to run any tests at this time.
170 Running the install process in the batch mode will use the
171 default values and not prompt the user of any possible options,
172 to ensure the process is fully automated.
173 batch-run [Test | Suite | OpenBenchmarking ID | Test Result] ...
175 This option and its arguments are equivalent to the run option,
176 but the process will be run in the Phoronix Test Suite batch
177 mode.
178 batch-setup
180 This option is used to configure the batch mode options for the
181 Phoronix Test Suite, which is subsequently written to the user
182 configuration file. Among the options are whether to automati‐
183 cally upload the test results to OpenBenchmarking.org and
184 prompting for the saved file name.
185 default-benchmark [Test | Suite | OpenBenchmarking ID | Test Result]
187 ...
188 This option will install the selected test(s) (if needed) and
189 will proceed to run the test(s) in the defaults mode. This
190 option is equivalent to running phoronix-test-suite with the
191 install option followed by the default-run option.
192 default-run [Test | Suite | OpenBenchmarking ID | Test Result] ...
194 This option will run the selected test(s). The name of the test
195 or suite must be supplied or the OpenBenchmarking.org ID or
196 saved local file name. Multiple arguments can be supplied to run
197 additional tests at the same time and save the results in a
198 suite-like fashion. Unlike the normal run option, the default-
199 run will not prompt the user to select from the available test
200 options but will instead use the default options as automati‐
201 cally set by pts-core or the test profile. Use batch-run to
202 automatically test all of the available options.
203 dry-run [Test | Suite | OpenBenchmarking ID | Test Result] ...
205 This option and its arguments pre-set the Phoronix Test Suite
206 batch run mode with enforcing of defaults to not save any
207 results and other behavior intended for a dry/test run. This
208 option is primarily intended for testing/evaluation purposes.
209 internal-run [Test | Suite | OpenBenchmarking ID | Test Result] ...
211 This option and its arguments pre-set the Phoronix Test Suite
212 batch run mode with sane values for carrying out benchmarks in a
213 semi-automated manner and without uploading any of the result
214 data to the public OpenBenchmarking.org.
215 OPENBENCHMARKING.ORG
218 clone-result [OpenBenchmarking ID] ... This option will down‐
219 load a local copy of a file that was saved to OpenBenchmark‐
220 ing.org, as long as a valid public ID is supplied.
221 list-recommended-tests
223 This option will list recommended test profiles for benchmarking
224 sorted by hardware sub-system. The recommended tests are deter‐
225 mined via querying OpenBenchmarking.org and determining the most
226 popular tests for a given environment based upon the number of
227 times a test profile has been downloaded, the number of test
228 results available on OpenBenchmarking.org for a given test pro‐
229 file, the age of the test profile, and other weighted factors.
230 make-openbenchmarking-cache
232 This option will attempt to cache the test profile/suite meta-
233 data from OpenBenchmarking.org for all linked repositories. This
234 is useful if you're going to be running the Phoronix Test Suite
235 / Phoromatic behind a firewall or without any Internet connec‐
236 tion. Those with unrestricted Internet access or not utilizing a
237 large local deployment of the Phoronix Test Suite / Phoromatic
238 shouldn't need to run this command.
239 openbenchmarking-changes
241 This option will list recent changes to test profiles of enabled
242 OpenBenchmarking.org repositories.
243 openbenchmarking-login
245 This option is used for controlling your Phoronix Test Suite
246 client options for OpenBechmarking.org and syncing the client to
247 your account.
248 openbenchmarking-refresh
250 This option is used for refreshing the stored OpenBenchmark‐
251 ing.org repostory information and other data. The Phoronix Test
252 Suite will automatically refresh this data every three days or
253 when other thresholds are exceeded, but this command can be used
254 to manually refresh/updates the data.
255 openbenchmarking-repositories
257 This option will list the OpenBenchmarking.org repositories cur‐
258 rently linked to this Phoronix Test Suite client instance.
259 openbenchmarking-uploads
261 This option will list any recent test result uploads from the
262 system's IP address to OpenBenchmarking.org.
263 recently-added-tests
265 This option will list the most recently added (newest) test pro‐
266 files.
267 upload-result [Test Result]
269 This option is used for uploading a test result to OpenBench‐
270 marking.org.
271 upload-test-profile
273 This option can be used for uploading a test profile to your
274 account on OpenBenchmarking.org. By uploading your test profile
275 to OpenBenchmarking.org, others are then able to browse and
276 access this test suite for easy distribution in a seamless man‐
277 ner by other Phoronix Test Suite clients.
278 upload-test-suite
280 This option can be used for uploading a test suite to your
281 account on OpenBenchmarking.org. By uploading your test suite to
282 OpenBenchmarking.org, others are then able to browse and access
283 this test suite for easy distribution.
284 INFORMATION
287 info [Test | Suite | OpenBenchmarking ID | Test Result] This
288 option will show details about the supplied test, suite, virtual
289 suite, or result file.
290 list-all-tests
292 This option will list all test profiles that are available from
293 the enabled OpenBenchmarking.org repositories. Unlike the other
294 test listing options, list-all-tests will show deprecated tests,
295 potentially broken tests, or other tests not recommended for all
296 environments. The only check in place is ensuring the test pro‐
297 files are at least compatible with the operating system in use.
298 list-available-suites
300 This option will list all test suites that are available from
301 the enabled OpenBenchmarking.org repositories.
302 list-available-tests
304 This option will list all test profiles that are available from
305 the enabled OpenBenchmarking.org repositories where supported on
306 the system and are of a verified state. If the system has no
307 Internet access, it will only list the test profiles where the
308 necesary test assets are available locally on the system or on
309 an available network cache (the same behavior as using the list-
310 cached-tests sub-command), unless using the list-all-tests
311 option to override this behavior.
312 list-available-virtual-suites
314 This option will list all available virtual test suites that can
315 be dynamically created based upon the available tests from
316 enabled OpenBenchmarking.org repositories.
317 list-cached-tests
319 This option will list all test profiles where any needed test
320 profiles are already cached or available from the local system
321 under test. This is primarily useful if testing offline/behind-
322 the-firewall and other use-cases where wanting to rely only upon
323 local data.
324 list-installed-dependencies
326 This option will list all of the packages / external test depen‐
327 dencies that are already installed on the system that the
328 Phoronix Test Suite may potentially depend upon by test pro‐
329 files.
330 list-installed-suites
332 This option will list all suites that are currently installed on
333 the system.
334 list-installed-tests
336 This option will list all test profiles that are currently
337 installed on the system.
338 list-missing-dependencies
340 This option will list all of the packages / external test depen‐
341 dencies that are missing from the system that the Phoronix Test
342 Suite may potentially need by select test profiles.
343 list-not-installed-tests
345 This option will list all test profiles that are supported and
346 available but presently NOT installed on the system.
347 list-possible-dependencies
349 This option will list all of the packages / external test depen‐
350 dencies that are are potentially used by the Phoronix Test
351 Suite.
352 list-saved-results
354 This option will list all of the saved test results found on the
355 system.
356 list-test-usage
358 This option will list various details about installed tests and
359 their usage.
360 list-unsupported-tests
362 This option will list all available test profiles that are
363 available from the enabled OpenBenchmarking.org repositories but
364 are NOT SUPPORTED on the given hardware/software platform. This
365 is mainly a debugging option for those looking for test profiles
366 to potentially port to new platforms, etc.
367 search This option provides command-line searching abilities for test
369 profiles / test suites / test results. The search query can be
370 passed as a parameter otherwise the user is prompted to input
371 their search query..
372 ASSET CREATION
375 build-suite This option will guide the user through the process
376 of generating their own test suite, which they can then run.
377 Optionally, passed as arguments can be the test(s) or suite(s)
378 to add to the suite to be created, instead of being prompted
379 through the process.
380 create-test-profile
382 This option can be used for creating a Phoronix Test Suite test
383 profile by answering questions about the test for constructing
384 the test profile XML meta-data and handling other boiler-plate
385 basics for getting started in developing new tests.
386 debug-benchmark [Test | Suite | OpenBenchmarking ID | Test Result] ...
388 This option is intended for use by test profile writers and is
389 identical to the <em>run</em> option but will yield more infor‐
390 mation during the run process that can be used to debug issues
391 with a test profile or to verify the test profile is functioning
392 correctly.
393 debug-install [Test | Suite | OpenBenchmarking ID | Test Result] ...
395 This option is intended for use by test profile writers and is
396 identical to the install option but will yield more information
397 during the run process that can be used to debug issues with a
398 test profile installer or to verify the test profile is func‐
399 tioning correctly.
400 debug-result-parser [Test | Suite | OpenBenchmarking ID | Test Result]
402 ...
403 This option is intended for use by test profile writers and is
404 used for debugging a result parser. No test execution is done,
405 but there must already be PTS-generated .log files present
406 within the test's installation directory.
407 debug-test-download-links [Test | Suite | OpenBenchmarking ID | Test
409 Result]
410 This option will check all download links within the specified
411 test profile(s) to ensure there are no broken URLs.
412 download-test-files [Test | Suite | OpenBenchmarking ID | Test Result]
414 ...
415 This will download the selected test file(s) to the Phoronix
416 Test Suite download cache but will not install the tests.
417 inspect-test-profile [Test]
419 This option can be used for inspecting a Phoronix Test Suite
420 test profile with providing inside details on test profiles for
421 debugging / evaluation / learning purposes.
422 result-file-to-suite [Test Result]
424 This option will guide the user through the process of generat‐
425 ing their own test suite, which they can then run, that is based
426 upon an existing test results file.
427 validate-result-file
429 This option can be used for validating a Phoronix Test Suite
430 result file as being compliant against the OpenBenchmarking.org
431 specification.
432 validate-test-profile [Test]
434 This option can be used for validating a Phoronix Test Suite
435 test profile as being compliant against the OpenBenchmarking.org
436 specification.
437 validate-test-suite
439 This option can be used for validating a Phoronix Test Suite
440 test suite as being compliant against the OpenBenchmarking.org
441 specification.
442 RESULT MANAGEMENT
445 analyze-all-runs [Test Result] This option will generate a can‐
446 dlestick graph showing the distribution of results from all
447 trial runs. The candlestick graph is similar to the Japanese
448 candlestick charts used by the financial industry, except
449 instead of representing stock data it is numerical result data
450 from all trial runs.0he tip of the upper-wick represents the
451 highest value of the test runs with the tip of the lower-wick
452 representing the lowest value of all test runs. The upper-edge
453 of the candle body represents the first or last run value and
454 the lower-edge represents the first or last run value. Lastly,
455 if the last run value is less than the first run value, the can‐
456 dle body is the same color as the graph background, otherwise
457 the last run value is greater.
458 auto-sort-result-file [Test Result]
460 This option is used if you wish to automatically attempt to sort
461 the results by their result identifier string.
462 compare-results-to-baseline [Test Result] [Test Result]
464 This option will allows you to specify a result as a baseline
465 (first parameter) and a second result file (second parameter)
466 that will offer some analysis for showing how the second result
467 compares to the first in matching tests.
468 edit-result-file [Test Result]
470 This option is used if you wish to edit the title and descrip‐
471 tion of an existing result file.
472 extract-from-result-file [Test Result]
474 This option will extract a single set of test results from a
475 saved results file that contains multiple test results that have
476 been merged. The user is the prompted to specify a new result
477 file name and select which result identifier to extract.
478 merge-results [Test Result] ...
480 This option will manually merge multiple sets of test results
481 generated by the Phoronix Test Suite.
482 refresh-graphs [Test Result]
484 This option will re-render and save all result graphs within a
485 saved file. This option can be used when making modifications to
486 the graphing code or its color/option configuration file and
487 testing the changes.
488 remove-from-result-file [Test Result]
490 This option is used if there is a set of test results you wish
491 to remove/delete from a saved results file. The user must spec‐
492 ify a saved results file and then they will be prompted to
493 select the results identifier associated with the results they
494 wish to remove.
495 remove-result [Test Result]
497 This option will permanently remove the saved file set that is
498 set as the first argument.
499 rename-identifier-in-result-file [Test Result]
501 This option is used if you wish to change the name of the iden‐
502 tifier in a test results file that is shown in the Phoronix Test
503 Suite Results Viewer and the contained graphs.
504 rename-result-file [Test Result]
506 This option is used if you wish to change the name of the saved
507 name of a result file.
508 reorder-result-file [Test Result]
510 This option is used if you wish to manually change the order in
511 which test results are shown in the Phoronix Test Suite Results
512 Viewer and the contained graphs. The user must specify a saved
513 results file and then they will be prompted to select the
514 results identifiers one at a time in the order they would like
515 them to be displayed from left to right.
516 result-file-stats [Test Result]
518 This option is used if you wish to analyze a result file by see‐
519 ing various statistics on the result data for result files con‐
520 taining at least two sets of data.
521 result-file-to-csv [Test Result]
523 This option will read a saved test results file and output the
524 system hardware and software information along with the results
525 to a CSV output. The CSV (Comma Separated Values) output can
526 then be loaded into a spreadsheet for easy viewing.
527 result-file-to-json [Test Result]
529 This option will read a saved test results file and output the
530 basic result information to JSON (JavaScript Object Notation).
531 result-file-to-pdf [Test Result]
533 This option will read a saved test results file and output the
534 system hardware and software information along with the results
535 to a PDF file.
536 result-file-to-text [Test Result]
538 This option will read a saved test results file and output the
539 system hardware and software information to the terminal. The
540 test results are also outputted.
541 show-result [Test Result]
543 Open up the test results in the Phoronix Test Suite Result
544 Viewer or on OpenBenchmarking.org.
545 wins-and-losses [Test Result]
547 This option is used if you wish to analyze a result file to see
548 which runs produced the most wins/losses of those result identi‐
549 fiers in the saved file.
550 OTHER commands This option will display a short list of possible
553 Phoronix Test Suite commands.
554 debug-dependency-handler
556 This option is used for testing the distribution-specific depen‐
557 dency handler for external dependencies.
558 debug-render-test
560 This option is used during the development of the Phoronix Test
561 Suite software for testing of the result and graph rendering
562 code-paths This option will download a large number of reference
563 test results from LinuxBenchmarking.com.
564 debug-self-test
566 This option is used during the development of the Phoronix Test
567 Suite software for testing of internal interfaces, commands, and
568 other common code-paths. The produced numbers should only be
569 comparable for the same version of the Phoronix Test Suite, on
570 the same hardware/software system, conducted on the same day of
571 testing. This isn't intended as any scientific benchmark but
572 simply to stress common PHP code-paths and looking for hot areas
573 to optimize, etc.
574 help This option will display a list of available Phoronix Test Suite
576 commands and possible parameter types.
577 version
579 This option will display the Phoronix Test Suite client version.
580 MODULES
583 auto-load-module This option can be used for easily adding a
584 module to the LoadModules list in the Phoronix Test Suite user
585 configuration file. That list controls what PTS modules are
586 automatically loaded on start-up of the Phoronix Test Suite.
587 list-modules
589 This option will list all of the available Phoronix Test Suite
590 modules on this system.
591 module-info [Phoronix Test Suite Module]
593 This option will show detailed information on a Phoronix Test
594 Suite module such as the version, developer, and a description
595 of its purpose.
596 module-setup [Phoronix Test Suite Module]
598 This option will allow you to configure all available end-user
599 options for a Phoronix Test Suite module. These options are then
600 stored within the user's configuration file. Not all modules may
601 have options that can be configured by the end-user.
602 test-module [Phoronix Test Suite Module]
604 This option can be used for debugging a Phoronix Test Suite mod‐
605 ule.
606 unload-module
608 This option can be used for easily removing a module from the
609 LoadModules list in the Phoronix Test Suite user configuration
610 file. That list controls what modules are automatically loaded
611 on start-up of the Phoronix Test Suite.
612 USER CONFIGURATION
615 enterprise-setup This option can be run by enterprise users
616 immediately after package installation or as part of an in-house
617 setup script. Running this command will ensure the phoronix-
618 test-suite program is never interrupted on new runs to accept
619 user agreement changes and defaults the anonymous usage report‐
620 ing to being disabled and other conservative defaults.
621 network-info
623 This option will print information detected by the Phoronix Test
624 Suite around the system's network configuration.
625 network-setup
627 This option allows the user to configure how the Phoronix Test
628 Suite connects to OpenBenchmarking.org and other web-services.
629 Connecting through an HTTP proxy can be configured through this
630 option.
631 user-config-reset
633 This option can be used for resetting the Phoronix Test Suite
634 user configuration file to its default state.
635 user-config-set
637 This option can be used for setting an XML value in the Phoronix
638 Test Suite user configuration file.
639 WEB / GUI SUPPORT
642 gui Launch the Phoronix Test Suite HTML5 web user-interface in
643 the local GUI mode (no remote web support) and attempt to auto-
644 launch the web-browser. THIS FEATURE IS CURRENTLY EXPERIMENTAL
645 AND NO LONGER ACTIVE DEVELOPMENT. See Phoronix Test Suite Phoro‐
646 matic as an alternative web UI approach.
647 PHOROMATIC
650 n _
651 SEE ALSO
654 Websites:
655 https://www.phoronix-test-suite.com/
656 https://commercial.phoronix-test-suite.com/
657 https://www.openbenchmarking.org/
658 https://www.phoronix.com/
659 https://www.phoronix.com/forums/
660
662 Copyright 2008 - 2019 by Phoronix Media, Michael Larabel.
663