1CTEST(1) CMake CTEST(1)
2
6 ctest - CTest Command-Line Reference
7 Contents
9 · ctest(1)
10 · Synopsis
12 · Description
14 · Options
16 · Label and Subproject Summary
18 · Build and Test Mode
20 · Dashboard Client
22 · Dashboard Client Steps
24 · Dashboard Client Modes
26 · Dashboard Client via CTest Command-Line
28 · Dashboard Client via CTest Script
30 · Dashboard Client Configuration
32 · CTest Start Step
34 · CTest Update Step
36 · CTest Configure Step
38 · CTest Build Step
40 · CTest Test Step
42 · CTest Coverage Step
44 · CTest MemCheck Step
46 · CTest Submit Step
48 · Show as JSON Object Model
50 · Resource Allocation
52 · Resource Specification File
54 · RESOURCE_GROUPS Property
56 · Environment Variables
58 · See Also
60
62 ctest [<options>]
63 ctest --build-and-test <path-to-source> <path-to-build>
64 --build-generator <generator> [<options>...]
65 [--build-options <opts>...] [--test-command <command> [<args>...]]
66 ctest {-D <dashboard> | -M <model> -T <action> | -S <script> | -SP <script>}
67 [-- <dashboard-options>...]
68
70 The ctest executable is the CMake test driver program. CMake-generated
71 build trees created for projects that use the enable_testing() and
72 add_test() commands have testing support. This program will run the
73 tests and report results.
74
76 -C <cfg>, --build-config <cfg>
77 Choose configuration to test.
78 Some CMake-generated build trees can have multiple build config‐
80 urations in the same tree. This option can be used to specify
81 which one should be tested. Example configurations are Debug
82 and Release.
83 --progress
85 Enable short progress output from tests.
86 When the output of ctest is being sent directly to a terminal,
88 the progress through the set of tests is reported by updating
89 the same line rather than printing start and end messages for
90 each test on new lines. This can significantly reduce the ver‐
91 bosity of the test output. Test completion messages are still
92 output on their own line for failed tests and the final test
93 summary will also still be logged.
94 This option can also be enabled by setting the environment vari‐
96 able CTEST_PROGRESS_OUTPUT.
97 -V,--verbose
99 Enable verbose output from tests.
100 Test output is normally suppressed and only summary information
102 is displayed. This option will show all test output.
103 -VV,--extra-verbose
105 Enable more verbose output from tests.
106 Test output is normally suppressed and only summary information
108 is displayed. This option will show even more test output.
109 --debug
111 Displaying more verbose internals of CTest.
112 This feature will result in a large number of output that is
114 mostly useful for debugging dashboard problems.
115 --output-on-failure
117 Output anything outputted by the test program if the test should
118 fail. This option can also be enabled by setting the CTEST_OUT‐
119 PUT_ON_FAILURE environment variable
120 --stop-on-failure
122 Stop running the tests when the first failure happens.
123 -F Enable failover.
125 This option allows CTest to resume a test set execution that was
127 previously interrupted. If no interruption occurred, the -F
128 option will have no effect.
129 -j <jobs>, --parallel <jobs>
131 Run the tests in parallel using the given number of jobs.
132 This option tells CTest to run the tests in parallel using given
134 number of jobs. This option can also be set by setting the
135 CTEST_PARALLEL_LEVEL environment variable.
136 This option can be used with the PROCESSORS test property.
138 See Label and Subproject Summary.
140 --resource-spec-file <file>
142 Run CTest with resource allocation enabled, using the resource
143 specification file specified in <file>.
144 When ctest is run as a Dashboard Client this sets the Resource‐
146 SpecFile option of the CTest Test Step.
147 --test-load <level>
149 While running tests in parallel (e.g. with -j), try not to start
150 tests when they may cause the CPU load to pass above a given
151 threshold.
152 When ctest is run as a Dashboard Client this sets the TestLoad
154 option of the CTest Test Step.
155 -Q,--quiet
157 Make CTest quiet.
158 This option will suppress all the output. The output log file
160 will still be generated if the --output-log is specified.
161 Options such as --verbose, --extra-verbose, and --debug are
162 ignored if --quiet is specified.
163 -O <file>, --output-log <file>
165 Output to log file.
166 This option tells CTest to write all its output to a <file> log
168 file.
169 -N,--show-only[=<format>]
171 Disable actual execution of tests.
172 This option tells CTest to list the tests that would be run but
174 not actually run them. Useful in conjunction with the -R and -E
175 options.
176 <format> can be one of the following values.
178 human Human-friendly output. This is not guaranteed to be
180 stable. This is the default.
181 json-v1
183 Dump the test information in JSON format. See Show as
184 JSON Object Model.
185 -L <regex>, --label-regex <regex>
187 Run tests with labels matching regular expression.
188 This option tells CTest to run only the tests whose labels match
190 the given regular expression.
191 -R <regex>, --tests-regex <regex>
193 Run tests matching regular expression.
194 This option tells CTest to run only the tests whose names match
196 the given regular expression.
197 -E <regex>, --exclude-regex <regex>
199 Exclude tests matching regular expression.
200 This option tells CTest to NOT run the tests whose names match
202 the given regular expression.
203 -LE <regex>, --label-exclude <regex>
205 Exclude tests with labels matching regular expression.
206 This option tells CTest to NOT run the tests whose labels match
208 the given regular expression.
209 -FA <regex>, --fixture-exclude-any <regex>
211 Exclude fixtures matching <regex> from automatically adding any
212 tests to the test set.
213 If a test in the set of tests to be executed requires a particu‐
215 lar fixture, that fixture’s setup and cleanup tests would nor‐
216 mally be added to the test set automatically. This option pre‐
217 vents adding setup or cleanup tests for fixtures matching the
218 <regex>. Note that all other fixture behavior is retained,
219 including test dependencies and skipping tests that have fixture
220 setup tests that fail.
221 -FS <regex>, --fixture-exclude-setup <regex>
223 Same as -FA except only matching setup tests are excluded.
224 -FC <regex>, --fixture-exclude-cleanup <regex>
226 Same as -FA except only matching cleanup tests are excluded.
227 -D <dashboard>, --dashboard <dashboard>
229 Execute dashboard test.
230 This option tells CTest to act as a CDash client and perform a
232 dashboard test. All tests are <Mode><Test>, where <Mode> can be
233 Experimental, Nightly, and Continuous, and <Test> can be Start,
234 Update, Configure, Build, Test, Coverage, and Submit.
235 See Dashboard Client.
237 -D <var>:<type>=<value>
239 Define a variable for script mode.
240 Pass in variable values on the command line. Use in conjunction
242 with -S to pass variable values to a dashboard script. Parsing
243 -D arguments as variable values is only attempted if the value
244 following -D does not match any of the known dashboard types.
245 -M <model>, --test-model <model>
247 Sets the model for a dashboard.
248 This option tells CTest to act as a CDash client where the
250 <model> can be Experimental, Nightly, and Continuous. Combining
251 -M and -T is similar to -D.
252 See Dashboard Client.
254 -T <action>, --test-action <action>
256 Sets the dashboard action to perform.
257 This option tells CTest to act as a CDash client and perform
259 some action such as start, build, test etc. See Dashboard Client
260 Steps for the full list of actions. Combining -M and -T is sim‐
261 ilar to -D.
262 See Dashboard Client.
264 -S <script>, --script <script>
266 Execute a dashboard for a configuration.
267 This option tells CTest to load in a configuration script which
269 sets a number of parameters such as the binary and source direc‐
270 tories. Then CTest will do what is required to create and run a
271 dashboard. This option basically sets up a dashboard and then
272 runs ctest -D with the appropriate options.
273 See Dashboard Client.
275 -SP <script>, --script-new-process <script>
277 Execute a dashboard for a configuration.
278 This option does the same operations as -S but it will do them
280 in a separate process. This is primarily useful in cases where
281 the script may modify the environment and you do not want the
282 modified environment to impact other -S scripts.
283 See Dashboard Client.
285 -I [Start,End,Stride,test#,test#|Test file], --tests-information
287 Run a specific number of tests by number.
288 This option causes CTest to run tests starting at number Start,
290 ending at number End, and incrementing by Stride. Any addi‐
291 tional numbers after Stride are considered individual test num‐
292 bers. Start, End, or Stride can be empty. Optionally a file
293 can be given that contains the same syntax as the command line.
294 -U, --union
296 Take the Union of -I and -R.
297 When both -R and -I are specified by default the intersection of
299 tests are run. By specifying -U the union of tests is run
300 instead.
301 --rerun-failed
303 Run only the tests that failed previously.
304 This option tells CTest to perform only the tests that failed
306 during its previous run. When this option is specified, CTest
307 ignores all other options intended to modify the list of tests
308 to run (-L, -R, -E, -LE, -I, etc). In the event that CTest runs
309 and no tests fail, subsequent calls to CTest with the
310 --rerun-failed option will run the set of tests that most
311 recently failed (if any).
312 --repeat <mode>:<n>
314 Run tests repeatedly based on the given <mode> up to <n> times.
315 The modes are:
316 until-fail
318 Require each test to run <n> times without failing in
319 order to pass. This is useful in finding sporadic fail‐
320 ures in test cases.
321 until-pass
323 Allow each test to run up to <n> times in order to pass.
324 Repeats tests if they fail for any reason. This is use‐
325 ful in tolerating sporadic failures in test cases.
326 after-timeout
328 Allow each test to run up to <n> times in order to pass.
329 Repeats tests only if they timeout. This is useful in
330 tolerating sporadic timeouts in test cases on busy
331 machines.
332 --repeat-until-fail <n>
334 Equivalent to --repeat until-fail:<n>.
335 --max-width <width>
337 Set the max width for a test name to output.
338 Set the maximum width for each test name to show in the output.
340 This allows the user to widen the output to avoid clipping the
341 test name which can be very annoying.
342 --interactive-debug-mode [0|1]
344 Set the interactive mode to 0 or 1.
345 This option causes CTest to run tests in either an interactive
347 mode or a non-interactive mode. On Windows this means that in
348 non-interactive mode, all system debug pop up windows are
349 blocked. In dashboard mode (Experimental, Nightly, Continuous),
350 the default is non-interactive. When just running tests not for
351 a dashboard the default is to allow popups and interactive
352 debugging.
353 --no-label-summary
355 Disable timing summary information for labels.
356 This option tells CTest not to print summary information for
358 each label associated with the tests run. If there are no
359 labels on the tests, nothing extra is printed.
360 See Label and Subproject Summary.
362 --no-subproject-summary
364 Disable timing summary information for subprojects.
365 This option tells CTest not to print summary information for
367 each subproject associated with the tests run. If there are no
368 subprojects on the tests, nothing extra is printed.
369 See Label and Subproject Summary.
371 --build-and-test See Build and Test Mode.
373 --test-output-size-passed <size>
375 Limit the output for passed tests to <size> bytes.
376 --test-output-size-failed <size>
378 Limit the output for failed tests to <size> bytes.
379 --overwrite
381 Overwrite CTest configuration option.
382 By default CTest uses configuration options from configuration
384 file. This option will overwrite the configuration option.
385 --force-new-ctest-process
387 Run child CTest instances as new processes.
388 By default CTest will run child CTest instances within the same
390 process. If this behavior is not desired, this argument will
391 enforce new processes for child CTest processes.
392 --schedule-random
394 Use a random order for scheduling tests.
395 This option will run the tests in a random order. It is com‐
397 monly used to detect implicit dependencies in a test suite.
398 --submit-index
400 Legacy option for old Dart2 dashboard server feature. Do not
401 use.
402 --timeout <seconds>
404 Set the default test timeout.
405 This option effectively sets a timeout on all tests that do not
407 already have a timeout set on them via the TIMEOUT property.
408 --stop-time <time>
410 Set a time at which all tests should stop running.
411 Set a real time of day at which all tests should timeout. Exam‐
413 ple: 7:00:00 -0400. Any time format understood by the curl date
414 parser is accepted. Local time is assumed if no timezone is
415 specified.
416 --print-labels
418 Print all available test labels.
419 This option will not run any tests, it will simply print the
421 list of all labels associated with the test set.
422 --no-tests=<[error|ignore]>
424 Regard no tests found either as error or ignore it.
425 If no tests were found, the default behavior of CTest is to
427 always log an error message but to return an error code in
428 script mode only. This option unifies the behavior of CTest by
429 either returning an error code if no tests were found or by
430 ignoring it.
431 --help,-help,-usage,-h,-H,/?
433 Print usage information and exit.
434 Usage describes the basic command line interface and its
436 options.
437 --version,-version,/V [<f>]
439 Show program name/version banner and exit.
440 If a file is specified, the version is written into it. The
442 help is printed to a named <f>ile if given.
443 --help-full [<f>]
445 Print all help manuals and exit.
446 All manuals are printed in a human-readable text format. The
448 help is printed to a named <f>ile if given.
449 --help-manual <man> [<f>]
451 Print one help manual and exit.
452 The specified manual is printed in a human-readable text format.
454 The help is printed to a named <f>ile if given.
455 --help-manual-list [<f>]
457 List help manuals available and exit.
458 The list contains all manuals for which help may be obtained by
460 using the --help-manual option followed by a manual name. The
461 help is printed to a named <f>ile if given.
462 --help-command <cmd> [<f>]
464 Print help for one command and exit.
465 The cmake-commands(7) manual entry for <cmd> is printed in a
467 human-readable text format. The help is printed to a named
468 <f>ile if given.
469 --help-command-list [<f>]
471 List commands with help available and exit.
472 The list contains all commands for which help may be obtained by
474 using the --help-command option followed by a command name. The
475 help is printed to a named <f>ile if given.
476 --help-commands [<f>]
478 Print cmake-commands manual and exit.
479 The cmake-commands(7) manual is printed in a human-readable text
481 format. The help is printed to a named <f>ile if given.
482 --help-module <mod> [<f>]
484 Print help for one module and exit.
485 The cmake-modules(7) manual entry for <mod> is printed in a
487 human-readable text format. The help is printed to a named
488 <f>ile if given.
489 --help-module-list [<f>]
491 List modules with help available and exit.
492 The list contains all modules for which help may be obtained by
494 using the --help-module option followed by a module name. The
495 help is printed to a named <f>ile if given.
496 --help-modules [<f>]
498 Print cmake-modules manual and exit.
499 The cmake-modules(7) manual is printed in a human-readable text
501 format. The help is printed to a named <f>ile if given.
502 --help-policy <cmp> [<f>]
504 Print help for one policy and exit.
505 The cmake-policies(7) manual entry for <cmp> is printed in a
507 human-readable text format. The help is printed to a named
508 <f>ile if given.
509 --help-policy-list [<f>]
511 List policies with help available and exit.
512 The list contains all policies for which help may be obtained by
514 using the --help-policy option followed by a policy name. The
515 help is printed to a named <f>ile if given.
516 --help-policies [<f>]
518 Print cmake-policies manual and exit.
519 The cmake-policies(7) manual is printed in a human-readable text
521 format. The help is printed to a named <f>ile if given.
522 --help-property <prop> [<f>]
524 Print help for one property and exit.
525 The cmake-properties(7) manual entries for <prop> are printed in
527 a human-readable text format. The help is printed to a named
528 <f>ile if given.
529 --help-property-list [<f>]
531 List properties with help available and exit.
532 The list contains all properties for which help may be obtained
534 by using the --help-property option followed by a property name.
535 The help is printed to a named <f>ile if given.
536 --help-properties [<f>]
538 Print cmake-properties manual and exit.
539 The cmake-properties(7) manual is printed in a human-readable
541 text format. The help is printed to a named <f>ile if given.
542 --help-variable <var> [<f>]
544 Print help for one variable and exit.
545 The cmake-variables(7) manual entry for <var> is printed in a
547 human-readable text format. The help is printed to a named
548 <f>ile if given.
549 --help-variable-list [<f>]
551 List variables with help available and exit.
552 The list contains all variables for which help may be obtained
554 by using the --help-variable option followed by a variable name.
555 The help is printed to a named <f>ile if given.
556 --help-variables [<f>]
558 Print cmake-variables manual and exit.
559 The cmake-variables(7) manual is printed in a human-readable
561 text format. The help is printed to a named <f>ile if given.
562
564 CTest prints timing summary information for each LABEL and subproject
565 associated with the tests run. The label time summary will not include
566 labels that are mapped to subprojects.
567 When the PROCESSORS test property is set, CTest will display a weighted
569 test timing result in label and subproject summaries. The time is
570 reported with sec*proc instead of just sec.
571 The weighted time summary reported for each label or subproject j is
573 computed as:
574 Weighted Time Summary for Label/Subproject j =
576 sum(raw_test_time[j,i] * num_processors[j,i], i=1...num_tests[j])
577 for labels/subprojects j=1...total
579 where:
581 · raw_test_time[j,i]: Wall-clock time for the i test for the j label or
583 subproject
584 · num_processors[j,i]: Value of the CTest PROCESSORS property for the i
586 test for the j label or subproject
587 · num_tests[j]: Number of tests associated with the j label or subpro‐
589 ject
590 · total: Total number of labels or subprojects that have at least one
592 test run
593 Therefore, the weighted time summary for each label or subproject rep‐
595 resents the amount of time that CTest gave to run the tests for each
596 label or subproject and gives a good representation of the total
597 expense of the tests for each label or subproject when compared to
598 other labels or subprojects.
599 For example, if SubprojectA showed 100 sec*proc and SubprojectB showed
601 10 sec*proc, then CTest allocated approximately 10 times the CPU/core
602 time to run the tests for SubprojectA than for SubprojectB (e.g. so if
603 effort is going to be expended to reduce the cost of the test suite for
604 the whole project, then reducing the cost of the test suite for Subpro‐
605 jectA would likely have a larger impact than effort to reduce the cost
606 of the test suite for SubprojectB).
607
609 CTest provides a command-line signature to configure (i.e. run cmake
610 on), build, and/or execute a test:
611 ctest --build-and-test <path-to-source> <path-to-build>
613 --build-generator <generator>
614 [<options>...]
615 [--build-options <opts>...]
616 [--test-command <command> [<args>...]]
617 The configure and test steps are optional. The arguments to this com‐
619 mand line are the source and binary directories. The --build-generator
620 option must be provided to use --build-and-test. If --test-command is
621 specified then that will be run after the build is complete. Other
622 options that affect this mode include:
623 --build-target
625 Specify a specific target to build.
626 If left out the all target is built.
628 --build-nocmake
630 Run the build without running cmake first.
631 Skip the cmake step.
633 --build-run-dir
635 Specify directory to run programs from.
636 Directory where programs will be after it has been compiled.
638 --build-two-config
640 Run CMake twice.
641 --build-exe-dir
643 Specify the directory for the executable.
644 --build-generator
646 Specify the generator to use. See the cmake-generators(7) man‐
647 ual.
648 --build-generator-platform
650 Specify the generator-specific platform.
651 --build-generator-toolset
653 Specify the generator-specific toolset.
654 --build-project
656 Specify the name of the project to build.
657 --build-makeprogram
659 Specify the explicit make program to be used by CMake when con‐
660 figuring and building the project. Only applicable for Make and
661 Ninja based generators.
662 --build-noclean
664 Skip the make clean step.
665 --build-config-sample
667 A sample executable to use to determine the configuration that
668 should be used. e.g. Debug, Release etc.
669 --build-options
671 Additional options for configuring the build (i.e. for CMake,
672 not for the build tool). Note that if this is specified, the
673 --build-options keyword and its arguments must be the last
674 option given on the command line, with the possible exception of
675 --test-command.
676 --test-command
678 The command to run as the test step with the --build-and-test
679 option. All arguments following this keyword will be assumed to
680 be part of the test command line, so it must be the last option
681 given.
682 --test-timeout
684 The time limit in seconds
685
687 CTest can operate as a client for the CDash software quality dashboard
688 application. As a dashboard client, CTest performs a sequence of steps
689 to configure, build, and test software, and then submits the results to
690 a CDash server. The command-line signature used to submit to CDash is:
691 ctest (-D <dashboard> | -M <model> -T <action> | -S <script> | -SP <script>)
693 [-- <dashboard-options>...]
694 Options for Dashboard Client include:
696 --group <group>
698 Specify what group you’d like to submit results to
699 Submit dashboard to specified group instead of default one. By
701 default, the dashboard is submitted to Nightly, Experimental, or
702 Continuous group, but by specifying this option, the group can
703 be arbitrary.
704 This replaces the deprecated option --track. Despite the name
706 change its behavior is unchanged.
707 -A <file>, --add-notes <file>
709 Add a notes file with submission.
710 This option tells CTest to include a notes file when submitting
712 dashboard.
713 --tomorrow-tag
715 Nightly or Experimental starts with next day tag.
716 This is useful if the build will not finish in one day.
718 --extra-submit <file>[;<file>]
720 Submit extra files to the dashboard.
721 This option will submit extra files to the dashboard.
723 --http1.0
725 Submit using HTTP 1.0.
726 This option will force CTest to use HTTP 1.0 to submit files to
728 the dashboard, instead of HTTP 1.1.
729 --no-compress-output
731 Do not compress test output when submitting.
732 This flag will turn off automatic compression of test output.
734 Use this to maintain compatibility with an older version of
735 CDash which doesn’t support compressed test output.
736 Dashboard Client Steps
738 CTest defines an ordered list of testing steps of which some or all may
739 be run as a dashboard client:
740 Start Start a new dashboard submission to be composed of results
742 recorded by the following steps. See the CTest Start Step sec‐
743 tion below.
744 Update Update the source tree from its version control repository.
746 Record the old and new versions and the list of updated source
747 files. See the CTest Update Step section below.
748 Configure
750 Configure the software by running a command in the build tree.
751 Record the configuration output log. See the CTest Configure
752 Step section below.
753 Build Build the software by running a command in the build tree.
755 Record the build output log and detect warnings and errors. See
756 the CTest Build Step section below.
757 Test Test the software by loading a CTestTestfile.cmake from the
759 build tree and executing the defined tests. Record the output
760 and result of each test. See the CTest Test Step section below.
761 Coverage
763 Compute coverage of the source code by running a coverage analy‐
764 sis tool and recording its output. See the CTest Coverage Step
765 section below.
766 MemCheck
768 Run the software test suite through a memory check tool. Record
769 the test output, results, and issues reported by the tool. See
770 the CTest MemCheck Step section below.
771 Submit Submit results recorded from other testing steps to the software
773 quality dashboard server. See the CTest Submit Step section
774 below.
775 Dashboard Client Modes
777 CTest defines three modes of operation as a dashboard client:
778 Nightly
780 This mode is intended to be invoked once per day, typically at
781 night. It enables the Start, Update, Configure, Build, Test,
782 Coverage, and Submit steps by default. Selected steps run even
783 if the Update step reports no changes to the source tree.
784 Continuous
786 This mode is intended to be invoked repeatedly throughout the
787 day. It enables the Start, Update, Configure, Build, Test, Cov‐
788 erage, and Submit steps by default, but exits after the Update
789 step if it reports no changes to the source tree.
790 Experimental
792 This mode is intended to be invoked by a developer to test local
793 changes. It enables the Start, Configure, Build, Test, Cover‐
794 age, and Submit steps by default.
795 Dashboard Client via CTest Command-Line
797 CTest can perform testing on an already-generated build tree. Run the
798 ctest command with the current working directory set to the build tree
799 and use one of these signatures:
800 ctest -D <mode>[<step>]
802 ctest -M <mode> [ -T <step> ]...
803 The <mode> must be one of the above Dashboard Client Modes, and each
805 <step> must be one of the above Dashboard Client Steps.
806 CTest reads the Dashboard Client Configuration settings from a file in
808 the build tree called either CTestConfiguration.ini or DartConfigura‐
809 tion.tcl (the names are historical). The format of the file is:
810 # Lines starting in '#' are comments.
812 # Other non-blank lines are key-value pairs.
813 <setting>: <value>
814 where <setting> is the setting name and <value> is the setting value.
816 In build trees generated by CMake, this configuration file is generated
818 by the CTest module if included by the project. The module uses vari‐
819 ables to obtain a value for each setting as documented with the set‐
820 tings below.
821 Dashboard Client via CTest Script
823 CTest can perform testing driven by a cmake-language(7) script that
824 creates and maintains the source and build tree as well as performing
825 the testing steps. Run the ctest command with the current working
826 directory set outside of any build tree and use one of these signa‐
827 tures:
828 ctest -S <script>
830 ctest -SP <script>
831 The <script> file must call CTest Commands commands to run testing
833 steps explicitly as documented below. The commands obtain Dashboard
834 Client Configuration settings from their arguments or from variables
835 set in the script.
836
838 The Dashboard Client Steps may be configured by named settings as docu‐
839 mented in the following sections.
840 CTest Start Step
842 Start a new dashboard submission to be composed of results recorded by
843 the following steps.
844 In a CTest Script, the ctest_start() command runs this step. Arguments
846 to the command may specify some of the step settings. The command
847 first runs the command-line specified by the CTEST_CHECKOUT_COMMAND
848 variable, if set, to initialize the source directory.
849 Configuration settings include:
851 BuildDirectory
853 The full path to the project build tree.
854 · CTest Script variable: CTEST_BINARY_DIRECTORY
856 · CTest module variable: PROJECT_BINARY_DIR
858 SourceDirectory
860 The full path to the project source tree.
861 · CTest Script variable: CTEST_SOURCE_DIRECTORY
863 · CTest module variable: PROJECT_SOURCE_DIR
865 CTest Update Step
867 In a CTest Script, the ctest_update() command runs this step. Argu‐
868 ments to the command may specify some of the step settings.
869 Configuration settings to specify the version control tool include:
871 BZRCommand
873 bzr command-line tool to use if source tree is managed by
874 Bazaar.
875 · CTest Script variable: CTEST_BZR_COMMAND
877 · CTest module variable: none
879 BZRUpdateOptions
881 Command-line options to the BZRCommand when updating the source.
882 · CTest Script variable: CTEST_BZR_UPDATE_OPTIONS
884 · CTest module variable: none
886 CVSCommand
888 cvs command-line tool to use if source tree is managed by CVS.
889 · CTest Script variable: CTEST_CVS_COMMAND
891 · CTest module variable: CVSCOMMAND
893 CVSUpdateOptions
895 Command-line options to the CVSCommand when updating the source.
896 · CTest Script variable: CTEST_CVS_UPDATE_OPTIONS
898 · CTest module variable: CVS_UPDATE_OPTIONS
900 GITCommand
902 git command-line tool to use if source tree is managed by Git.
903 · CTest Script variable: CTEST_GIT_COMMAND
905 · CTest module variable: GITCOMMAND
907 The source tree is updated by git fetch followed by git reset
909 --hard to the FETCH_HEAD. The result is the same as git pull
910 except that any local modifications are overwritten. Use GITUp‐
911 dateCustom to specify a different approach.
912 GITInitSubmodules
914 If set, CTest will update the repository’s submodules before
915 updating.
916 · CTest Script variable: CTEST_GIT_INIT_SUBMODULES
918 · CTest module variable: CTEST_GIT_INIT_SUBMODULES
920 GITUpdateCustom
922 Specify a custom command line (as a semicolon-separated list) to
923 run in the source tree (Git work tree) to update it instead of
924 running the GITCommand.
925 · CTest Script variable: CTEST_GIT_UPDATE_CUSTOM
927 · CTest module variable: CTEST_GIT_UPDATE_CUSTOM
929 GITUpdateOptions
931 Command-line options to the GITCommand when updating the source.
932 · CTest Script variable: CTEST_GIT_UPDATE_OPTIONS
934 · CTest module variable: GIT_UPDATE_OPTIONS
936 HGCommand
938 hg command-line tool to use if source tree is managed by Mercu‐
939 rial.
940 · CTest Script variable: CTEST_HG_COMMAND
942 · CTest module variable: none
944 HGUpdateOptions
946 Command-line options to the HGCommand when updating the source.
947 · CTest Script variable: CTEST_HG_UPDATE_OPTIONS
949 · CTest module variable: none
951 P4Client
953 Value of the -c option to the P4Command.
954 · CTest Script variable: CTEST_P4_CLIENT
956 · CTest module variable: CTEST_P4_CLIENT
958 P4Command
960 p4 command-line tool to use if source tree is managed by Per‐
961 force.
962 · CTest Script variable: CTEST_P4_COMMAND
964 · CTest module variable: P4COMMAND
966 P4Options
968 Command-line options to the P4Command for all invocations.
969 · CTest Script variable: CTEST_P4_OPTIONS
971 · CTest module variable: CTEST_P4_OPTIONS
973 P4UpdateCustom
975 Specify a custom command line (as a semicolon-separated list) to
976 run in the source tree (Perforce tree) to update it instead of
977 running the P4Command.
978 · CTest Script variable: none
980 · CTest module variable: CTEST_P4_UPDATE_CUSTOM
982 P4UpdateOptions
984 Command-line options to the P4Command when updating the source.
985 · CTest Script variable: CTEST_P4_UPDATE_OPTIONS
987 · CTest module variable: CTEST_P4_UPDATE_OPTIONS
989 SVNCommand
991 svn command-line tool to use if source tree is managed by Sub‐
992 version.
993 · CTest Script variable: CTEST_SVN_COMMAND
995 · CTest module variable: SVNCOMMAND
997 SVNOptions
999 Command-line options to the SVNCommand for all invocations.
1000 · CTest Script variable: CTEST_SVN_OPTIONS
1002 · CTest module variable: CTEST_SVN_OPTIONS
1004 SVNUpdateOptions
1006 Command-line options to the SVNCommand when updating the source.
1007 · CTest Script variable: CTEST_SVN_UPDATE_OPTIONS
1009 · CTest module variable: SVN_UPDATE_OPTIONS
1011 UpdateCommand
1013 Specify the version-control command-line tool to use without
1014 detecting the VCS that manages the source tree.
1015 · CTest Script variable: CTEST_UPDATE_COMMAND
1017 · CTest module variable: <VCS>COMMAND when UPDATE_TYPE is <vcs>,
1019 else UPDATE_COMMAND
1020 UpdateOptions
1022 Command-line options to the UpdateCommand.
1023 · CTest Script variable: CTEST_UPDATE_OPTIONS
1025 · CTest module variable: <VCS>_UPDATE_OPTIONS when UPDATE_TYPE
1027 is <vcs>, else UPDATE_OPTIONS
1028 UpdateType
1030 Specify the version-control system that manages the source tree
1031 if it cannot be detected automatically. The value may be bzr,
1032 cvs, git, hg, p4, or svn.
1033 · CTest Script variable: none, detected from source tree
1035 · CTest module variable: UPDATE_TYPE if set, else
1037 CTEST_UPDATE_TYPE
1038 UpdateVersionOnly
1040 Specify that you want the version control update command to only
1041 discover the current version that is checked out, and not to
1042 update to a different version.
1043 · CTest Script variable: CTEST_UPDATE_VERSION_ONLY
1045 UpdateVersionOverride
1047 Specify the current version of your source tree.
1048 When this variable is set to a non-empty string, CTest will
1050 report the value you specified rather than using the update com‐
1051 mand to discover the current version that is checked out. Use of
1052 this variable supersedes UpdateVersionOnly. Like UpdateVer‐
1053 sionOnly, using this variable tells CTest not to update the
1054 source tree to a different version.
1055 · CTest Script variable: CTEST_UPDATE_VERSION_OVERRIDE
1057 Additional configuration settings include:
1059 NightlyStartTime
1061 In the Nightly dashboard mode, specify the “nightly start time”.
1062 With centralized version control systems (cvs and svn), the
1063 Update step checks out the version of the software as of this
1064 time so that multiple clients choose a common version to test.
1065 This is not well-defined in distributed version-control systems
1066 so the setting is ignored.
1067 · CTest Script variable: CTEST_NIGHTLY_START_TIME
1069 · CTest module variable: NIGHTLY_START_TIME if set, else
1071 CTEST_NIGHTLY_START_TIME
1072 CTest Configure Step
1074 In a CTest Script, the ctest_configure() command runs this step. Argu‐
1075 ments to the command may specify some of the step settings.
1076 Configuration settings include:
1078 ConfigureCommand
1080 Command-line to launch the software configuration process. It
1081 will be executed in the location specified by the BuildDirectory
1082 setting.
1083 · CTest Script variable: CTEST_CONFIGURE_COMMAND
1085 · CTest module variable: CMAKE_COMMAND followed by
1087 PROJECT_SOURCE_DIR
1088 LabelsForSubprojects
1090 Specify a semicolon-separated list of labels that will be
1091 treated as subprojects. This mapping will be passed on to CDash
1092 when configure, test or build results are submitted.
1093 · CTest Script variable: CTEST_LABELS_FOR_SUBPROJECTS
1095 · CTest module variable: CTEST_LABELS_FOR_SUBPROJECTS
1097 See Label and Subproject Summary.
1099 CTest Build Step
1101 In a CTest Script, the ctest_build() command runs this step. Arguments
1102 to the command may specify some of the step settings.
1103 Configuration settings include:
1105 DefaultCTestConfigurationType
1107 When the build system to be launched allows build-time selection
1108 of the configuration (e.g. Debug, Release), this specifies the
1109 default configuration to be built when no -C option is given to
1110 the ctest command. The value will be substituted into the value
1111 of MakeCommand to replace the literal string ${CTEST_CONFIGURA‐
1112 TION_TYPE} if it appears.
1113 · CTest Script variable: CTEST_CONFIGURATION_TYPE
1115 · CTest module variable: DEFAULT_CTEST_CONFIGURATION_TYPE, ini‐
1117 tialized by the CMAKE_CONFIG_TYPE environment variable
1118 LabelsForSubprojects
1120 Specify a semicolon-separated list of labels that will be
1121 treated as subprojects. This mapping will be passed on to CDash
1122 when configure, test or build results are submitted.
1123 · CTest Script variable: CTEST_LABELS_FOR_SUBPROJECTS
1125 · CTest module variable: CTEST_LABELS_FOR_SUBPROJECTS
1127 See Label and Subproject Summary.
1129 MakeCommand
1131 Command-line to launch the software build process. It will be
1132 executed in the location specified by the BuildDirectory set‐
1133 ting.
1134 · CTest Script variable: CTEST_BUILD_COMMAND
1136 · CTest module variable: MAKECOMMAND, initialized by the
1138 build_command() command
1139 UseLaunchers
1141 For build trees generated by CMake using one of the Makefile
1142 Generators or the Ninja generator, specify whether the
1143 CTEST_USE_LAUNCHERS feature is enabled by the CTestUseLaunchers
1144 module (also included by the CTest module). When enabled, the
1145 generated build system wraps each invocation of the compiler,
1146 linker, or custom command line with a “launcher” that communi‐
1147 cates with CTest via environment variables and files to report
1148 granular build warning and error information. Otherwise, CTest
1149 must “scrape” the build output log for diagnostics.
1150 · CTest Script variable: CTEST_USE_LAUNCHERS
1152 · CTest module variable: CTEST_USE_LAUNCHERS
1154 CTest Test Step
1156 In a CTest Script, the ctest_test() command runs this step. Arguments
1157 to the command may specify some of the step settings.
1158 Configuration settings include:
1160 ResourceSpecFile
1162 Specify a resource specification file.
1163 · CTest Script variable: CTEST_RESOURCE_SPEC_FILE
1165 · CTest module variable: CTEST_RESOURCE_SPEC_FILE
1167 See Resource Allocation for more information.
1169 LabelsForSubprojects
1171 Specify a semicolon-separated list of labels that will be
1172 treated as subprojects. This mapping will be passed on to CDash
1173 when configure, test or build results are submitted.
1174 · CTest Script variable: CTEST_LABELS_FOR_SUBPROJECTS
1176 · CTest module variable: CTEST_LABELS_FOR_SUBPROJECTS
1178 See Label and Subproject Summary.
1180 TestLoad
1182 While running tests in parallel (e.g. with -j), try not to start
1183 tests when they may cause the CPU load to pass above a given
1184 threshold.
1185 · CTest Script variable: CTEST_TEST_LOAD
1187 · CTest module variable: CTEST_TEST_LOAD
1189 TimeOut
1191 The default timeout for each test if not specified by the TIME‐
1192 OUT test property.
1193 · CTest Script variable: CTEST_TEST_TIMEOUT
1195 · CTest module variable: DART_TESTING_TIMEOUT
1197 CTest Coverage Step
1199 In a CTest Script, the ctest_coverage() command runs this step. Argu‐
1200 ments to the command may specify some of the step settings.
1201 Configuration settings include:
1203 CoverageCommand
1205 Command-line tool to perform software coverage analysis. It
1206 will be executed in the location specified by the BuildDirectory
1207 setting.
1208 · CTest Script variable: CTEST_COVERAGE_COMMAND
1210 · CTest module variable: COVERAGE_COMMAND
1212 CoverageExtraFlags
1214 Specify command-line options to the CoverageCommand tool.
1215 · CTest Script variable: CTEST_COVERAGE_EXTRA_FLAGS
1217 · CTest module variable: COVERAGE_EXTRA_FLAGS
1219 These options are the first arguments passed to CoverageCommand.
1221 CTest MemCheck Step
1223 In a CTest Script, the ctest_memcheck() command runs this step. Argu‐
1224 ments to the command may specify some of the step settings.
1225 Configuration settings include:
1227 MemoryCheckCommand
1229 Command-line tool to perform dynamic analysis. Test command
1230 lines will be launched through this tool.
1231 · CTest Script variable: CTEST_MEMORYCHECK_COMMAND
1233 · CTest module variable: MEMORYCHECK_COMMAND
1235 MemoryCheckCommandOptions
1237 Specify command-line options to the MemoryCheckCommand tool.
1238 They will be placed prior to the test command line.
1239 · CTest Script variable: CTEST_MEMORYCHECK_COMMAND_OPTIONS
1241 · CTest module variable: MEMORYCHECK_COMMAND_OPTIONS
1243 MemoryCheckType
1245 Specify the type of memory checking to perform.
1246 · CTest Script variable: CTEST_MEMORYCHECK_TYPE
1248 · CTest module variable: MEMORYCHECK_TYPE
1250 MemoryCheckSanitizerOptions
1252 Specify options to sanitizers when running with a sani‐
1253 tize-enabled build.
1254 · CTest Script variable: CTEST_MEMORYCHECK_SANITIZER_OPTIONS
1256 · CTest module variable: MEMORYCHECK_SANITIZER_OPTIONS
1258 MemoryCheckSuppressionFile
1260 Specify a file containing suppression rules for the MemoryCheck‐
1261 Command tool. It will be passed with options appropriate to the
1262 tool.
1263 · CTest Script variable: CTEST_MEMORYCHECK_SUPPRESSIONS_FILE
1265 · CTest module variable: MEMORYCHECK_SUPPRESSIONS_FILE
1267 Additional configuration settings include:
1269 BoundsCheckerCommand
1271 Specify a MemoryCheckCommand that is known to be command-line
1272 compatible with Bounds Checker.
1273 · CTest Script variable: none
1275 · CTest module variable: none
1277 PurifyCommand
1279 Specify a MemoryCheckCommand that is known to be command-line
1280 compatible with Purify.
1281 · CTest Script variable: none
1283 · CTest module variable: PURIFYCOMMAND
1285 ValgrindCommand
1287 Specify a MemoryCheckCommand that is known to be command-line
1288 compatible with Valgrind.
1289 · CTest Script variable: none
1291 · CTest module variable: VALGRIND_COMMAND
1293 ValgrindCommandOptions
1295 Specify command-line options to the ValgrindCommand tool. They
1296 will be placed prior to the test command line.
1297 · CTest Script variable: none
1299 · CTest module variable: VALGRIND_COMMAND_OPTIONS
1301 DrMemoryCommand
1303 Specify a MemoryCheckCommand that is known to be a command-line
1304 compatible with DrMemory.
1305 · CTest Script variable: none
1307 · CTest module variable: DRMEMORY_COMMAND
1309 DrMemoryCommandOptions
1311 Specify command-line options to the DrMemoryCommand tool. They
1312 will be placed prior to the test command line.
1313 · CTest Script variable: none
1315 · CTest module variable: DRMEMORY_COMMAND_OPTIONS
1317 CudaSanitizerCommand
1319 Specify a MemoryCheckCommand that is known to be a command-line
1320 compatible with cuda-memcheck or compute-sanitizer.
1321 · CTest Script variable: none
1323 · CTest module variable: CUDA_SANITIZER_COMMAND
1325 CudaSanitizerCommandOptions
1327 Specify command-line options to the CudaSanitizerCommand tool.
1328 They will be placed prior to the test command line.
1329 · CTest Script variable: none
1331 · CTest module variable: CUDA_SANITIZER_COMMAND_OPTIONS
1333 CTest Submit Step
1335 In a CTest Script, the ctest_submit() command runs this step. Argu‐
1336 ments to the command may specify some of the step settings.
1337 Configuration settings include:
1339 BuildName
1341 Describe the dashboard client platform with a short string.
1342 (Operating system, compiler, etc.)
1343 · CTest Script variable: CTEST_BUILD_NAME
1345 · CTest module variable: BUILDNAME
1347 CDashVersion
1349 Legacy option. Not used.
1350 · CTest Script variable: none, detected from server
1352 · CTest module variable: CTEST_CDASH_VERSION
1354 CTestSubmitRetryCount
1356 Specify a number of attempts to retry submission on network
1357 failure.
1358 · CTest Script variable: none, use the ctest_submit()
1360 RETRY_COUNT option.
1361 · CTest module variable: CTEST_SUBMIT_RETRY_COUNT
1363 CTestSubmitRetryDelay
1365 Specify a delay before retrying submission on network failure.
1366 · CTest Script variable: none, use the ctest_submit()
1368 RETRY_DELAY option.
1369 · CTest module variable: CTEST_SUBMIT_RETRY_DELAY
1371 CurlOptions
1373 Specify a semicolon-separated list of options to control the
1374 Curl library that CTest uses internally to connect to the
1375 server. Possible options are CURLOPT_SSL_VERIFYPEER_OFF and
1376 CURLOPT_SSL_VERIFYHOST_OFF.
1377 · CTest Script variable: CTEST_CURL_OPTIONS
1379 · CTest module variable: CTEST_CURL_OPTIONS
1381 DropLocation
1383 Legacy option. When SubmitURL is not set, it is constructed
1384 from DropMethod, DropSiteUser, DropSitePassword, DropSite, and
1385 DropLocation.
1386 · CTest Script variable: CTEST_DROP_LOCATION
1388 · CTest module variable: DROP_LOCATION if set, else
1390 CTEST_DROP_LOCATION
1391 DropMethod
1393 Legacy option. When SubmitURL is not set, it is constructed
1394 from DropMethod, DropSiteUser, DropSitePassword, DropSite, and
1395 DropLocation.
1396 · CTest Script variable: CTEST_DROP_METHOD
1398 · CTest module variable: DROP_METHOD if set, else
1400 CTEST_DROP_METHOD
1401 DropSite
1403 Legacy option. When SubmitURL is not set, it is constructed
1404 from DropMethod, DropSiteUser, DropSitePassword, DropSite, and
1405 DropLocation.
1406 · CTest Script variable: CTEST_DROP_SITE
1408 · CTest module variable: DROP_SITE if set, else CTEST_DROP_SITE
1410 DropSitePassword
1412 Legacy option. When SubmitURL is not set, it is constructed
1413 from DropMethod, DropSiteUser, DropSitePassword, DropSite, and
1414 DropLocation.
1415 · CTest Script variable: CTEST_DROP_SITE_PASSWORD
1417 · CTest module variable: DROP_SITE_PASSWORD if set, else
1419 CTEST_DROP_SITE_PASWORD
1420 DropSiteUser
1422 Legacy option. When SubmitURL is not set, it is constructed
1423 from DropMethod, DropSiteUser, DropSitePassword, DropSite, and
1424 DropLocation.
1425 · CTest Script variable: CTEST_DROP_SITE_USER
1427 · CTest module variable: DROP_SITE_USER if set, else
1429 CTEST_DROP_SITE_USER
1430 IsCDash
1432 Legacy option. Not used.
1433 · CTest Script variable: CTEST_DROP_SITE_CDASH
1435 · CTest module variable: CTEST_DROP_SITE_CDASH
1437 ScpCommand
1439 Legacy option. Not used.
1440 · CTest Script variable: CTEST_SCP_COMMAND
1442 · CTest module variable: SCPCOMMAND
1444 Site Describe the dashboard client host site with a short string.
1446 (Hostname, domain, etc.)
1447 · CTest Script variable: CTEST_SITE
1449 · CTest module variable: SITE, initialized by the site_name()
1451 command
1452 SubmitURL
1454 The http or https URL of the dashboard server to send the sub‐
1455 mission to.
1456 · CTest Script variable: CTEST_SUBMIT_URL
1458 · CTest module variable: SUBMIT_URL if set, else CTEST_SUB‐
1460 MIT_URL
1461 TriggerSite
1463 Legacy option. Not used.
1464 · CTest Script variable: CTEST_TRIGGER_SITE
1466 · CTest module variable: TRIGGER_SITE if set, else CTEST_TRIG‐
1468 GER_SITE
1469
1471 When the --show-only=json-v1 command line option is given, the test
1472 information is output in JSON format. Version 1.0 of the JSON object
1473 model is defined as follows:
1474 kind The string “ctestInfo”.
1476 version
1478 A JSON object specifying the version components. Its members
1479 are
1480 major A non-negative integer specifying the major version com‐
1482 ponent.
1483 minor A non-negative integer specifying the minor version com‐
1485 ponent.
1486 backtraceGraph
1488 JSON object representing backtrace information with the follow‐
1489 ing members:
1490 commands
1492 List of command names.
1493 files List of file names.
1495 nodes List of node JSON objects with members:
1497 command
1499 Index into the commands member of the backtrace‐
1500 Graph.
1501 file Index into the files member of the backtraceGraph.
1503 line Line number in the file where the backtrace was
1505 added.
1506 parent Index into the nodes member of the backtraceGraph
1508 representing the parent in the graph.
1509 tests A JSON array listing information about each test. Each entry is
1511 a JSON object with members:
1512 name Test name.
1514 config Configuration that the test can run on. Empty string
1516 means any config.
1517 command
1519 List where the first element is the test command and the
1520 remaining elements are the command arguments.
1521 backtrace
1523 Index into the nodes member of the backtraceGraph.
1524 properties
1526 Test properties. Can contain keys for each of the sup‐
1527 ported test properties.
1528
1530 CTest provides a mechanism for tests to specify the resources that they
1531 need in a fine-grained way, and for users to specify the resources
1532 available on the running machine. This allows CTest to internally keep
1533 track of which resources are in use and which are free, scheduling
1534 tests in a way that prevents them from trying to claim resources that
1535 are not available.
1536 When the resource allocation feature is used, CTest will not oversub‐
1538 scribe resources. For example, if a resource has 8 slots, CTest will
1539 not run tests that collectively use more than 8 slots at a time. This
1540 has the effect of limiting how many tests can run at any given time,
1541 even if a high -j argument is used, if those tests all use some slots
1542 from the same resource. In addition, it means that a single test that
1543 uses more of a resource than is available on a machine will not run at
1544 all (and will be reported as Not Run).
1545 A common use case for this feature is for tests that require the use of
1547 a GPU. Multiple tests can simultaneously allocate memory from a GPU,
1548 but if too many tests try to do this at once, some of them will fail to
1549 allocate, resulting in a failed test, even though the test would have
1550 succeeded if it had the memory it needed. By using the resource alloca‐
1551 tion feature, each test can specify how much memory it requires from a
1552 GPU, allowing CTest to schedule tests in a way that running several of
1553 these tests at once does not exhaust the GPU’s memory pool.
1554 Please note that CTest has no concept of what a GPU is or how much mem‐
1556 ory it has, nor does it have any way of communicating with a GPU to
1557 retrieve this information or perform any memory management. CTest sim‐
1558 ply keeps track of a list of abstract resource types, each of which has
1559 a certain number of slots available for tests to use. Each test speci‐
1560 fies the number of slots that it requires from a certain resource, and
1561 CTest then schedules them in a way that prevents the total number of
1562 slots in use from exceeding the listed capacity. When a test is exe‐
1563 cuted, and slots from a resource are allocated to that test, tests may
1564 assume that they have exclusive use of those slots for the duration of
1565 the test’s process.
1566 The CTest resource allocation feature consists of two inputs:
1568 · The resource specification file, described below, which describes the
1570 resources available on the system.
1571 · The RESOURCE_GROUPS property of tests, which describes the resources
1573 required by the test.
1574 When CTest runs a test, the resources allocated to that test are passed
1576 in the form of a set of environment variables as described below. Using
1577 this information to decide which resource to connect to is left to the
1578 test writer.
1579 The RESOURCE_GROUPS property tells CTest what resources a test expects
1581 to use grouped in a way meaningful to the test. The test itself must
1582 read the environment variables to determine which resources have been
1583 allocated to each group. For example, each group may correspond to a
1584 process the test will spawn when executed.
1585 Note that even if a test specifies a RESOURCE_GROUPS property, it is
1587 still possible for that to test to run without any resource allocation
1588 (and without the corresponding environment variables) if the user does
1589 not pass a resource specification file. Passing this file, either
1590 through the --resource-spec-file command-line argument or the
1591 RESOURCE_SPEC_FILE argument to ctest_test(), is what activates the
1592 resource allocation feature. Tests should check the
1593 CTEST_RESOURCE_GROUP_COUNT environment variable to find out whether or
1594 not resource allocation is activated. This variable will always (and
1595 only) be defined if resource allocation is activated. If resource allo‐
1596 cation is not activated, then the CTEST_RESOURCE_GROUP_COUNT variable
1597 will not exist, even if it exists for the parent ctest process. If a
1598 test absolutely must have resource allocation, then it can return a
1599 failing exit code or use the SKIP_RETURN_CODE or SKIP_REGULAR_EXPRES‐
1600 SION properties to indicate a skipped test.
1601 Resource Specification File
1603 The resource specification file is a JSON file which is passed to
1604 CTest, either on the ctest(1) command line as --resource-spec-file, or
1605 as the RESOURCE_SPEC_FILE argument of ctest_test(). If a dashboard
1606 script is used and RESOURCE_SPEC_FILE is not specified, the value of
1607 CTEST_RESOURCE_SPEC_FILE in the dashboard script is used instead. If
1608 --resource-spec-file, RESOURCE_SPEC_FILE, and CTEST_RESOURCE_SPEC_FILE
1609 in the dashboard script are not specified, the value of
1610 CTEST_RESOURCE_SPEC_FILE in the CMake build is used instead. If none of
1611 these are specified, no resource spec file is used.
1612 The resource specification file must be a JSON object. All examples in
1614 this document assume the following resource specification file:
1615 {
1617 "version": {
1618 "major": 1,
1619 "minor": 0
1620 },
1621 "local": [
1622 {
1623 "gpus": [
1624 {
1625 "id": "0",
1626 "slots": 2
1627 },
1628 {
1629 "id": "1",
1630 "slots": 4
1631 },
1632 {
1633 "id": "2",
1634 "slots": 2
1635 },
1636 {
1637 "id": "3"
1638 }
1639 ],
1640 "crypto_chips": [
1641 {
1642 "id": "card0",
1643 "slots": 4
1644 }
1645 ]
1646 }
1647 ]
1648 }
1649 The members are:
1651 version
1653 An object containing a major integer field and a minor integer
1654 field. Currently, the only supported version is major 1, minor
1655 0. Any other value is an error.
1656 local A JSON array of resource sets present on the system. Currently,
1658 this array is restricted to being of size 1.
1659 Each array element is a JSON object with members whose names are
1661 equal to the desired resource types, such as gpus. These names
1662 must start with a lowercase letter or an underscore, and subse‐
1663 quent characters can be a lowercase letter, a digit, or an
1664 underscore. Uppercase letters are not allowed, because certain
1665 platforms have case-insensitive environment variables. See the
1666 Environment Variables section below for more information. It is
1667 recommended that the resource type name be the plural of a noun,
1668 such as gpus or crypto_chips (and not gpu or crypto_chip.)
1669 Please note that the names gpus and crypto_chips are just exam‐
1671 ples, and CTest does not interpret them in any way. You are free
1672 to make up any resource type you want to meet your own require‐
1673 ments.
1674 The value for each resource type is a JSON array consisting of
1676 JSON objects, each of which describe a specific instance of the
1677 specified resource. These objects have the following members:
1678 id A string consisting of an identifier for the resource.
1680 Each character in the identifier can be a lowercase let‐
1681 ter, a digit, or an underscore. Uppercase letters are
1682 not allowed.
1683 Identifiers must be unique within a resource type. How‐
1685 ever, they do not have to be unique across resource
1686 types. For example, it is valid to have a gpus resource
1687 named 0 and a crypto_chips resource named 0, but not two
1688 gpus resources both named 0.
1689 Please note that the IDs 0, 1, 2, 3, and card0 are just
1691 examples, and CTest does not interpret them in any way.
1692 You are free to make up any IDs you want to meet your own
1693 requirements.
1694 slots An optional unsigned number specifying the number of
1696 slots available on the resource. For example, this could
1697 be megabytes of RAM on a GPU, or cryptography units
1698 available on a cryptography chip. If slots is not speci‐
1699 fied, a default value of 1 is assumed.
1700 In the example file above, there are four GPUs with ID’s 0 through 3.
1702 GPU 0 has 2 slots, GPU 1 has 4, GPU 2 has 2, and GPU 3 has a default of
1703 1 slot. There is also one cryptography chip with 4 slots.
1704 RESOURCE_GROUPS Property
1706 See RESOURCE_GROUPS for a description of this property.
1707 Environment Variables
1709 Once CTest has decided which resources to allocate to a test, it passes
1710 this information to the test executable as a series of environment
1711 variables. For each example below, we will assume that the test in
1712 question has a RESOURCE_GROUPS property of
1713 2,gpus:2;gpus:4,gpus:1,crypto_chips:2.
1714 The following variables are passed to the test process:
1716 CTEST_RESOURCE_GROUP_COUNT
1718 The total number of groups specified by the RESOURCE_GROUPS
1719 property. For example:
1720 · CTEST_RESOURCE_GROUP_COUNT=3
1722 This variable will only be defined if ctest(1) has been given a
1724 --resource-spec-file, or if ctest_test() has been given a
1725 RESOURCE_SPEC_FILE. If no resource specification file has been
1726 given, this variable will not be defined.
1727 CTEST_RESOURCE_GROUP_<num>
1729 The list of resource types allocated to each group, with each
1730 item separated by a comma. <num> is a number from zero to
1731 CTEST_RESOURCE_GROUP_COUNT minus one. CTEST_RESOURCE_GROUP_<num>
1732 is defined for each <num> in this range. For example:
1733 · CTEST_RESOURCE_GROUP_0=gpus
1735 · CTEST_RESOURCE_GROUP_1=gpus
1737 · CTEST_RESOURCE_GROUP_2=crypto_chips,gpus
1739 CTEST_RESOURCE_GROUP_<num>_<resource-type>
1741 The list of resource IDs and number of slots from each ID allo‐
1742 cated to each group for a given resource type. This variable
1743 consists of a series of pairs, each pair separated by a semi‐
1744 colon, and with the two items in the pair separated by a comma.
1745 The first item in each pair is id: followed by the ID of a
1746 resource of type <resource-type>, and the second item is slots:
1747 followed by the number of slots from that resource allocated to
1748 the given group. For example:
1749 · CTEST_RESOURCE_GROUP_0_GPUS=id:0,slots:2
1751 · CTEST_RESOURCE_GROUP_1_GPUS=id:2,slots:2
1753 · CTEST_RESOURCE_GROUP_2_GPUS=id:1,slots:4;id:3,slots:1
1755 · CTEST_RESOURCE_GROUP_2_CRYPTO_CHIPS=id:card0,slots:2
1757 In this example, group 0 gets 2 slots from GPU 0, group 1 gets 2
1759 slots from GPU 2, and group 2 gets 4 slots from GPU 1, 1 slot
1760 from GPU 3, and 2 slots from cryptography chip card0.
1761 <num> is a number from zero to CTEST_RESOURCE_GROUP_COUNT minus
1763 one. <resource-type> is the name of a resource type, converted
1764 to uppercase. CTEST_RESOURCE_GROUP_<num>_<resource-type> is
1765 defined for the product of each <num> in the range listed above
1766 and each resource type listed in CTEST_RESOURCE_GROUP_<num>.
1767 Because some platforms have case-insensitive names for environ‐
1769 ment variables, the names of resource types may not clash in a
1770 case-insensitive environment. Because of this, for the sake of
1771 simplicity, all resource types must be listed in all lowercase
1772 in the resource specification file and in the RESOURCE_GROUPS
1773 property, and they are converted to all uppercase in the
1774 CTEST_RESOURCE_GROUP_<num>_<resource-type> environment variable.
1775
1777 The following resources are available to get help using CMake:
1778 Home Page
1780 https://cmake.org
1781 The primary starting point for learning about CMake.
1783 Online Documentation and Community Resources
1785 https://cmake.org/documentation
1786 Links to available documentation and community resources may be
1788 found on this web page.
1789 Discourse Forum
1791 https://discourse.cmake.org
1792 The Discourse Forum hosts discussion and questions about CMake.
1794
1796 2000-2021 Kitware, Inc. and Contributors
17973.19.7 Mar 15, 2021 CTEST(1)