1CTEST(1)                             CMake                            CTEST(1)
2
3
4

NAME

6       ctest - CTest Command-Line Reference
7
8   Contents
9ctest(1)
10
11Synopsis
12
13Description
14
15Options
16
17Label Matching
18
19Label and Subproject Summary
20
21Build and Test Mode
22
23Dashboard Client
24
25Dashboard Client Steps
26
27Dashboard Client Modes
28
29Dashboard Client via CTest Command-Line
30
31Dashboard Client via CTest Script
32
33Dashboard Client Configuration
34
35CTest Start Step
36
37CTest Update Step
38
39CTest Configure Step
40
41CTest Build Step
42
43CTest Test Step
44
45CTest Coverage Step
46
47CTest MemCheck Step
48
49CTest Submit Step
50
51Show as JSON Object Model
52
53Resource Allocation
54
55Resource Specification File
56
57RESOURCE_GROUPS Property
58
59Environment Variables
60
61See Also
62

SYNOPSIS

64          ctest [<options>]
65          ctest --build-and-test <path-to-source> <path-to-build>
66                --build-generator <generator> [<options>...]
67                [--build-options <opts>...] [--test-command <command> [<args>...]]
68          ctest {-D <dashboard> | -M <model> -T <action> | -S <script> | -SP <script>}
69                [-- <dashboard-options>...]
70

DESCRIPTION

72       The ctest executable is the CMake test driver program.  CMake-generated
73       build trees created for projects  that  use  the  enable_testing()  and
74       add_test()  commands  have  testing support.  This program will run the
75       tests and report results.
76

OPTIONS

78       --preset <preset>, --preset=<preset>
79              Use a test preset to specify test options.  The  project  binary
80              directory  is inferred from the configurePreset key. The current
81              working directory must contain CMake preset files.   See  preset
82              for more details.
83
84       --list-presets
85              Lists  the available test presets. The current working directory
86              must contain CMake preset files.
87
88       -C <cfg>, --build-config <cfg>
89              Choose configuration to test.
90
91              Some CMake-generated build trees can have multiple build config‐
92              urations  in  the same tree.  This option can be used to specify
93              which one should be tested.  Example  configurations  are  Debug
94              and Release.
95
96       --progress
97              Enable short progress output from tests.
98
99              When  the  output of ctest is being sent directly to a terminal,
100              the progress through the set of tests is  reported  by  updating
101              the  same  line  rather than printing start and end messages for
102              each test on new lines.  This can significantly reduce the  ver‐
103              bosity  of  the test output.  Test completion messages are still
104              output on their own line for failed tests  and  the  final  test
105              summary will also still be logged.
106
107              This option can also be enabled by setting the environment vari‐
108              able CTEST_PROGRESS_OUTPUT.
109
110       -V,--verbose
111              Enable verbose output from tests.
112
113              Test output is normally suppressed and only summary  information
114              is displayed.  This option will show all test output.
115
116       -VV,--extra-verbose
117              Enable more verbose output from tests.
118
119              Test  output is normally suppressed and only summary information
120              is displayed.  This option will show even more test output.
121
122       --debug
123              Displaying more verbose internals of CTest.
124
125              This feature will result in a large number  of  output  that  is
126              mostly useful for debugging dashboard problems.
127
128       --output-on-failure
129              Output anything outputted by the test program if the test should
130              fail.  This option can also be enabled by setting the CTEST_OUT‐
131              PUT_ON_FAILURE environment variable
132
133       --stop-on-failure
134              Stop running the tests when the first failure happens.
135
136       -F     Enable failover.
137
138              This option allows CTest to resume a test set execution that was
139              previously interrupted.  If no interruption occurred, the -F op‐
140              tion will have no effect.
141
142       -j <jobs>, --parallel <jobs>
143              Run the tests in parallel using the given number of jobs.
144
145              This option tells CTest to run the tests in parallel using given
146              number of jobs. This option can  also  be  set  by  setting  the
147              CTEST_PARALLEL_LEVEL environment variable.
148
149              This option can be used with the PROCESSORS test property.
150
151              See Label and Subproject Summary.
152
153       --resource-spec-file <file>
154              Run  CTest  with resource allocation enabled, using the resource
155              specification file specified in <file>.
156
157              When ctest is run as a Dashboard Client this sets the  Resource‐
158              SpecFile option of the CTest Test Step.
159
160       --test-load <level>
161              While running tests in parallel (e.g. with -j), try not to start
162              tests when they may cause the CPU load to  pass  above  a  given
163              threshold.
164
165              When  ctest  is run as a Dashboard Client this sets the TestLoad
166              option of the CTest Test Step.
167
168       -Q,--quiet
169              Make CTest quiet.
170
171              This option will suppress all the output.  The output  log  file
172              will  still  be generated if the --output-log is specified.  Op‐
173              tions such as --verbose, --extra-verbose, and  --debug  are  ig‐
174              nored if --quiet is specified.
175
176       -O <file>, --output-log <file>
177              Output to log file.
178
179              This  option tells CTest to write all its output to a <file> log
180              file.
181
182       --output-junit <file>
183              Write test results in JUnit format.
184
185              This option tells CTest to write test results to <file> in JUnit
186              XML format. If <file> already exists, it will be overwritten. If
187              using the -S option to run a  dashboard  script,  use  the  OUT‐
188              PUT_JUNIT keyword with the ctest_test() command instead.
189
190       -N,--show-only[=<format>]
191              Disable actual execution of tests.
192
193              This  option tells CTest to list the tests that would be run but
194              not actually run them.  Useful in conjunction with the -R and -E
195              options.
196
197              <format> can be one of the following values.
198
199                 human  Human-friendly  output.   This is not guaranteed to be
200                        stable.  This is the default.
201
202                 json-v1
203                        Dump the test information in JSON format.  See Show as
204                        JSON Object Model.
205
206       -L <regex>, --label-regex <regex>
207              Run  tests  with labels matching regular expression as described
208              under string(REGEX).
209
210              This option tells CTest to run only the tests whose labels match
211              the  given  regular expression.  When more than one -L option is
212              given, a test will  only  be  run  if  each  regular  expression
213              matches  at least one of the test's labels (i.e. the multiple -L
214              labels form an AND relationship).  See Label Matching.
215
216       -R <regex>, --tests-regex <regex>
217              Run tests matching regular expression.
218
219              This option tells CTest to run only the tests whose names  match
220              the given regular expression.
221
222       -E <regex>, --exclude-regex <regex>
223              Exclude tests matching regular expression.
224
225              This  option  tells CTest to NOT run the tests whose names match
226              the given regular expression.
227
228       -LE <regex>, --label-exclude <regex>
229              Exclude tests with labels matching regular expression.
230
231              This option tells CTest to NOT run the tests whose labels  match
232              the  given regular expression.  When more than one -LE option is
233              given, a test will only be excluded if each  regular  expression
234              matches at least one of the test's labels (i.e. the multiple -LE
235              labels form an AND relationship).  See Label Matching.
236
237       -FA <regex>, --fixture-exclude-any <regex>
238              Exclude fixtures matching <regex> from automatically adding  any
239              tests to the test set.
240
241              If a test in the set of tests to be executed requires a particu‐
242              lar fixture, that fixture's setup and cleanup tests  would  nor‐
243              mally  be  added to the test set automatically. This option pre‐
244              vents adding setup or cleanup tests for  fixtures  matching  the
245              <regex>.  Note  that all other fixture behavior is retained, in‐
246              cluding test dependencies and skipping tests that  have  fixture
247              setup tests that fail.
248
249       -FS <regex>, --fixture-exclude-setup <regex>
250              Same as -FA except only matching setup tests are excluded.
251
252       -FC <regex>, --fixture-exclude-cleanup <regex>
253              Same as -FA except only matching cleanup tests are excluded.
254
255       -D <dashboard>, --dashboard <dashboard>
256              Execute dashboard test.
257
258              This  option  tells CTest to act as a CDash client and perform a
259              dashboard test.  All tests are <Mode><Test>, where <Mode> can be
260              Experimental,  Nightly, and Continuous, and <Test> can be Start,
261              Update, Configure, Build, Test, Coverage, and Submit.
262
263              See Dashboard Client.
264
265       -D <var>:<type>=<value>
266              Define a variable for script mode.
267
268              Pass in variable values on the command line.  Use in conjunction
269              with  -S to pass variable values to a dashboard script.  Parsing
270              -D arguments as variable values is only attempted if  the  value
271              following -D does not match any of the known dashboard types.
272
273       -M <model>, --test-model <model>
274              Sets the model for a dashboard.
275
276              This  option  tells  CTest  to  act  as a CDash client where the
277              <model> can be Experimental, Nightly, and Continuous.  Combining
278              -M and -T is similar to -D.
279
280              See Dashboard Client.
281
282       -T <action>, --test-action <action>
283              Sets the dashboard action to perform.
284
285              This  option  tells  CTest  to act as a CDash client and perform
286              some action such as start, build, test etc. See Dashboard Client
287              Steps for the full list of actions.  Combining -M and -T is sim‐
288              ilar to -D.
289
290              See Dashboard Client.
291
292       -S <script>, --script <script>
293              Execute a dashboard for a configuration.
294
295              This option tells CTest to load in a configuration script  which
296              sets a number of parameters such as the binary and source direc‐
297              tories.  Then CTest will do what is required to create and run a
298              dashboard.   This  option basically sets up a dashboard and then
299              runs ctest -D with the appropriate options.
300
301              See Dashboard Client.
302
303       -SP <script>, --script-new-process <script>
304              Execute a dashboard for a configuration.
305
306              This option does the same operations as -S but it will  do  them
307              in  a separate process.  This is primarily useful in cases where
308              the script may modify the environment and you do  not  want  the
309              modified environment to impact other -S scripts.
310
311              See Dashboard Client.
312
313       -I [Start,End,Stride,test#,test#|Test file], --tests-information
314              Run a specific number of tests by number.
315
316              This  option causes CTest to run tests starting at number Start,
317              ending at number End, and incrementing  by  Stride.   Any  addi‐
318              tional  numbers after Stride are considered individual test num‐
319              bers.  Start, End, or Stride can be empty.   Optionally  a  file
320              can be given that contains the same syntax as the command line.
321
322       -U, --union
323              Take the Union of -I and -R.
324
325              When both -R and -I are specified by default the intersection of
326              tests are run.  By specifying -U the union of tests is  run  in‐
327              stead.
328
329       --rerun-failed
330              Run only the tests that failed previously.
331
332              This  option  tells  CTest to perform only the tests that failed
333              during its previous run.  When this option is  specified,  CTest
334              ignores  all  other options intended to modify the list of tests
335              to run (-L, -R, -E, -LE, -I, etc).  In the event that CTest runs
336              and  no  tests  fail,  subsequent  calls to CTest with the --re‐
337              run-failed option will run the set of tests that  most  recently
338              failed (if any).
339
340       --repeat <mode>:<n>
341              Run  tests repeatedly based on the given <mode> up to <n> times.
342              The modes are:
343
344              until-fail
345                     Require each test to run <n> times without failing in or‐
346                     der to pass.  This is useful in finding sporadic failures
347                     in test cases.
348
349              until-pass
350                     Allow each test to run up to <n> times in order to  pass.
351                     Repeats  tests if they fail for any reason.  This is use‐
352                     ful in tolerating sporadic failures in test cases.
353
354              after-timeout
355                     Allow each test to run up to <n> times in order to  pass.
356                     Repeats  tests  only  if they timeout.  This is useful in
357                     tolerating sporadic timeouts in test cases  on  busy  ma‐
358                     chines.
359
360       --repeat-until-fail <n>
361              Equivalent to --repeat until-fail:<n>.
362
363       --max-width <width>
364              Set the max width for a test name to output.
365
366              Set  the maximum width for each test name to show in the output.
367              This allows the user to widen the output to avoid  clipping  the
368              test name which can be very annoying.
369
370       --interactive-debug-mode [0|1]
371              Set the interactive mode to 0 or 1.
372
373              This  option  causes CTest to run tests in either an interactive
374              mode or a non-interactive mode.  In dashboard  mode  (Experimen‐
375              tal,  Nightly,  Continuous), the default is non-interactive.  In
376              non-interactive   mode,   the   environment    variable    DASH‐
377              BOARD_TEST_FROM_CTEST is set.
378
379              Prior  to CMake 3.11, interactive mode on Windows allowed system
380              debug popup windows to appear.  Now, due to CTest's use of libuv
381              to launch test processes, all system debug popup windows are al‐
382              ways blocked.
383
384       --no-label-summary
385              Disable timing summary information for labels.
386
387              This option tells CTest not to  print  summary  information  for
388              each  label  associated with the tests run.  If there are no la‐
389              bels on the tests, nothing extra is printed.
390
391              See Label and Subproject Summary.
392
393       --no-subproject-summary
394              Disable timing summary information for subprojects.
395
396              This option tells CTest not to  print  summary  information  for
397              each  subproject associated with the tests run.  If there are no
398              subprojects on the tests, nothing extra is printed.
399
400              See Label and Subproject Summary.
401
402       --build-and-test See Build and Test Mode.
403
404       --test-dir <dir> Specify the directory in which to look for tests.
405
406       --test-output-size-passed <size>
407              Limit the output for passed tests to <size> bytes.
408
409       --test-output-size-failed <size>
410              Limit the output for failed tests to <size> bytes.
411
412       --overwrite
413              Overwrite CTest configuration option.
414
415              By default CTest uses configuration options  from  configuration
416              file.  This option will overwrite the configuration option.
417
418       --force-new-ctest-process
419              Run child CTest instances as new processes.
420
421              By  default CTest will run child CTest instances within the same
422              process.  If this behavior is not desired,  this  argument  will
423              enforce new processes for child CTest processes.
424
425       --schedule-random
426              Use a random order for scheduling tests.
427
428              This  option  will  run the tests in a random order.  It is com‐
429              monly used to detect implicit dependencies in a test suite.
430
431       --submit-index
432              Legacy option for old Dart2 dashboard server  feature.   Do  not
433              use.
434
435       --timeout <seconds>
436              Set the default test timeout.
437
438              This  option effectively sets a timeout on all tests that do not
439              already have a timeout set on them via the TIMEOUT property.
440
441       --stop-time <time>
442              Set a time at which all tests should stop running.
443
444              Set a real time of day at which all tests should timeout.  Exam‐
445              ple: 7:00:00 -0400.  Any time format understood by the curl date
446              parser is accepted.  Local time is assumed  if  no  timezone  is
447              specified.
448
449       --print-labels
450              Print all available test labels.
451
452              This  option  will  not  run any tests, it will simply print the
453              list of all labels associated with the test set.
454
455       --no-tests=<[error|ignore]>
456              Regard no tests found either as error or ignore it.
457
458              If no tests were found, the default behavior of CTest is to  al‐
459              ways  log an error message but to return an error code in script
460              mode only.  This option unifies the behavior of CTest by  either
461              returning  an  error  code if no tests were found or by ignoring
462              it.
463
464       --help,-help,-usage,-h,-H,/?
465              Print usage information and exit.
466
467              Usage describes the basic command line  interface  and  its  op‐
468              tions.
469
470       --version,-version,/V [<f>]
471              Show program name/version banner and exit.
472
473              If  a  file  is  specified, the version is written into it.  The
474              help is printed to a named <f>ile if given.
475
476       --help-full [<f>]
477              Print all help manuals and exit.
478
479              All manuals are printed in a human-readable  text  format.   The
480              help is printed to a named <f>ile if given.
481
482       --help-manual <man> [<f>]
483              Print one help manual and exit.
484
485              The specified manual is printed in a human-readable text format.
486              The help is printed to a named <f>ile if given.
487
488       --help-manual-list [<f>]
489              List help manuals available and exit.
490
491              The list contains all manuals for which help may be obtained  by
492              using  the  --help-manual option followed by a manual name.  The
493              help is printed to a named <f>ile if given.
494
495       --help-command <cmd> [<f>]
496              Print help for one command and exit.
497
498              The cmake-commands(7) manual entry for <cmd> is printed in a hu‐
499              man-readable text format.  The help is printed to a named <f>ile
500              if given.
501
502       --help-command-list [<f>]
503              List commands with help available and exit.
504
505              The list contains all commands for which help may be obtained by
506              using the --help-command option followed by a command name.  The
507              help is printed to a named <f>ile if given.
508
509       --help-commands [<f>]
510              Print cmake-commands manual and exit.
511
512              The cmake-commands(7) manual is printed in a human-readable text
513              format.  The help is printed to a named <f>ile if given.
514
515       --help-module <mod> [<f>]
516              Print help for one module and exit.
517
518              The  cmake-modules(7) manual entry for <mod> is printed in a hu‐
519              man-readable text format.  The help is printed to a named <f>ile
520              if given.
521
522       --help-module-list [<f>]
523              List modules with help available and exit.
524
525              The  list contains all modules for which help may be obtained by
526              using the --help-module option followed by a module  name.   The
527              help is printed to a named <f>ile if given.
528
529       --help-modules [<f>]
530              Print cmake-modules manual and exit.
531
532              The  cmake-modules(7) manual is printed in a human-readable text
533              format.  The help is printed to a named <f>ile if given.
534
535       --help-policy <cmp> [<f>]
536              Print help for one policy and exit.
537
538              The cmake-policies(7) manual entry for <cmp> is printed in a hu‐
539              man-readable text format.  The help is printed to a named <f>ile
540              if given.
541
542       --help-policy-list [<f>]
543              List policies with help available and exit.
544
545              The list contains all policies for which help may be obtained by
546              using  the  --help-policy option followed by a policy name.  The
547              help is printed to a named <f>ile if given.
548
549       --help-policies [<f>]
550              Print cmake-policies manual and exit.
551
552              The cmake-policies(7) manual is printed in a human-readable text
553              format.  The help is printed to a named <f>ile if given.
554
555       --help-property <prop> [<f>]
556              Print help for one property and exit.
557
558              The cmake-properties(7) manual entries for <prop> are printed in
559              a human-readable text format.  The help is printed  to  a  named
560              <f>ile if given.
561
562       --help-property-list [<f>]
563              List properties with help available and exit.
564
565              The  list contains all properties for which help may be obtained
566              by using the --help-property option followed by a property name.
567              The help is printed to a named <f>ile if given.
568
569       --help-properties [<f>]
570              Print cmake-properties manual and exit.
571
572              The  cmake-properties(7)  manual  is printed in a human-readable
573              text format.  The help is printed to a named <f>ile if given.
574
575       --help-variable <var> [<f>]
576              Print help for one variable and exit.
577
578              The cmake-variables(7) manual entry for <var> is  printed  in  a
579              human-readable  text  format.   The  help  is printed to a named
580              <f>ile if given.
581
582       --help-variable-list [<f>]
583              List variables with help available and exit.
584
585              The list contains all variables for which help may  be  obtained
586              by using the --help-variable option followed by a variable name.
587              The help is printed to a named <f>ile if given.
588
589       --help-variables [<f>]
590              Print cmake-variables manual and exit.
591
592              The cmake-variables(7) manual is  printed  in  a  human-readable
593              text format.  The help is printed to a named <f>ile if given.
594

LABEL MATCHING

596       Tests  may  have  labels attached to them. Tests may be included or ex‐
597       cluded from a test run by filtering on  the  labels.   Each  individual
598       filter  is  a  regular  expression  applied to the labels attached to a
599       test.
600
601       When -L is used, in order for a test to be included in a test run, each
602       regular  expression must match at least one label.  Using more than one
603       -L option means "match all of these".
604
605       The -LE option works just like -L, but excludes tests rather  than  in‐
606       cluding  them. A test is excluded if each regular expression matches at
607       least one label.
608
609       If a test has no labels attached to it, then -L will never include that
610       test,  and  -LE  will  never exclude that test.  As an example of tests
611       with labels, consider five tests, with the following labels:
612
613test1 has labels tuesday and production
614
615test2 has labels tuesday and test
616
617test3 has labels wednesday and production
618
619test4 has label wednesday
620
621test5 has labels friday and test
622
623       Running ctest with -L tuesday -L test will select test2, which has both
624       labels. Running CTest with -L test will select test2 and test5, because
625       both of them have a label that matches that regular expression.
626
627       Because the matching works with regular  expressions,  take  note  that
628       running  CTest  with  -L  es  will match all five tests.  To select the
629       tuesday and wednesday tests together, use a single  regular  expression
630       that matches either of them, like -L "tue|wed".
631

LABEL AND SUBPROJECT SUMMARY

633       CTest  prints  timing summary information for each LABEL and subproject
634       associated with the tests run. The label time summary will not  include
635       labels that are mapped to subprojects.
636
637       New in version 3.22: Labels added dynamically during test execution are
638       also reported in the timing summary.  See Additional Labels.
639
640
641       When the PROCESSORS test property is set, CTest will display a weighted
642       test  timing  result in label and subproject summaries. The time is re‐
643       ported with sec*proc instead of just sec.
644
645       The weighted time summary reported for each label or  subproject  j  is
646       computed as:
647
648          Weighted Time Summary for Label/Subproject j =
649              sum(raw_test_time[j,i] * num_processors[j,i], i=1...num_tests[j])
650
651          for labels/subprojects j=1...total
652
653       where:
654
655raw_test_time[j,i]: Wall-clock time for the i test for the j label or
656         subproject
657
658num_processors[j,i]: Value of the CTest PROCESSORS property for the i
659         test for the j label or subproject
660
661num_tests[j]:  Number of tests associated with the j label or subpro‐
662         ject
663
664total: Total number of labels or subprojects that have at  least  one
665         test run
666
667       Therefore,  the weighted time summary for each label or subproject rep‐
668       resents the amount of time that CTest gave to run the  tests  for  each
669       label  or  subproject  and gives a good representation of the total ex‐
670       pense of the tests for each label or subproject when compared to  other
671       labels or subprojects.
672
673       For  example, if SubprojectA showed 100 sec*proc and SubprojectB showed
674       10 sec*proc, then CTest allocated approximately 10 times  the  CPU/core
675       time  to run the tests for SubprojectA than for SubprojectB (e.g. so if
676       effort is going to be expended to reduce the cost of the test suite for
677       the whole project, then reducing the cost of the test suite for Subpro‐
678       jectA would likely have a larger impact than effort to reduce the  cost
679       of the test suite for SubprojectB).
680

BUILD AND TEST MODE

682       CTest  provides  a  command-line signature to configure (i.e. run cmake
683       on), build, and/or execute a test:
684
685          ctest --build-and-test <path-to-source> <path-to-build>
686                --build-generator <generator>
687                [<options>...]
688                [--build-options <opts>...]
689                [--test-command <command> [<args>...]]
690
691       The configure and test steps are optional. The arguments to  this  com‐
692       mand line are the source and binary directories.  The --build-generator
693       option must be provided to use --build-and-test.  If --test-command  is
694       specified then that will be run after the build is complete.  Other op‐
695       tions that affect this mode include:
696
697       --build-target
698              Specify a specific target to build.
699
700              If left out the all target is built.
701
702       --build-nocmake
703              Run the build without running cmake first.
704
705              Skip the cmake step.
706
707       --build-run-dir
708              Specify directory to run programs from.
709
710              Directory where programs will be after it has been compiled.
711
712       --build-two-config
713              Run CMake twice.
714
715       --build-exe-dir
716              Specify the directory for the executable.
717
718       --build-generator
719              Specify the generator to use. See the  cmake-generators(7)  man‐
720              ual.
721
722       --build-generator-platform
723              Specify the generator-specific platform.
724
725       --build-generator-toolset
726              Specify the generator-specific toolset.
727
728       --build-project
729              Specify the name of the project to build.
730
731       --build-makeprogram
732              Specify  the explicit make program to be used by CMake when con‐
733              figuring and building the project. Only applicable for Make  and
734              Ninja based generators.
735
736       --build-noclean
737              Skip the make clean step.
738
739       --build-config-sample
740              A  sample  executable to use to determine the configuration that
741              should be used.  e.g.  Debug, Release etc.
742
743       --build-options
744              Additional options for configuring the build  (i.e.  for  CMake,
745              not  for  the  build tool).  Note that if this is specified, the
746              --build-options keyword and its arguments must be the  last  op‐
747              tion  given  on the command line, with the possible exception of
748              --test-command.
749
750       --test-command
751              The command to run as the test step  with  the  --build-and-test
752              option.  All arguments following this keyword will be assumed to
753              be part of the test command line, so it must be the last  option
754              given.
755
756       --test-timeout
757              The time limit in seconds
758

DASHBOARD CLIENT

760       CTest  can operate as a client for the CDash software quality dashboard
761       application.  As a dashboard client, CTest performs a sequence of steps
762       to configure, build, and test software, and then submits the results to
763       a CDash server. The command-line signature used to submit to CDash is:
764
765          ctest (-D <dashboard> | -M <model> -T <action> | -S <script> | -SP <script>)
766                [-- <dashboard-options>...]
767
768       Options for Dashboard Client include:
769
770       --group <group>
771              Specify what group you'd like to submit results to
772
773              Submit dashboard to specified group instead of default one.   By
774              default, the dashboard is submitted to Nightly, Experimental, or
775              Continuous group, but by specifying this option, the  group  can
776              be arbitrary.
777
778              This  replaces  the deprecated option --track.  Despite the name
779              change its behavior is unchanged.
780
781       -A <file>, --add-notes <file>
782              Add a notes file with submission.
783
784              This option tells CTest to include a notes file when  submitting
785              dashboard.
786
787       --tomorrow-tag
788              Nightly or Experimental starts with next day tag.
789
790              This is useful if the build will not finish in one day.
791
792       --extra-submit <file>[;<file>]
793              Submit extra files to the dashboard.
794
795              This option will submit extra files to the dashboard.
796
797       --http1.0
798              Submit using HTTP 1.0.
799
800              This  option will force CTest to use HTTP 1.0 to submit files to
801              the dashboard, instead of HTTP 1.1.
802
803       --no-compress-output
804              Do not compress test output when submitting.
805
806              This flag will turn off automatic compression  of  test  output.
807              Use  this  to  maintain  compatibility  with an older version of
808              CDash which doesn't support compressed test output.
809
810   Dashboard Client Steps
811       CTest defines an ordered list of testing steps of which some or all may
812       be run as a dashboard client:
813
814       Start  Start  a  new  dashboard  submission  to  be composed of results
815              recorded by the following steps.  See the CTest Start Step  sec‐
816              tion below.
817
818       Update Update  the  source  tree  from  its version control repository.
819              Record the old and new versions and the list of  updated  source
820              files.  See the CTest Update Step section below.
821
822       Configure
823              Configure  the  software by running a command in the build tree.
824              Record the configuration output log.  See  the  CTest  Configure
825              Step section below.
826
827       Build  Build  the  software  by  running  a  command in the build tree.
828              Record the build output log and detect warnings and errors.  See
829              the CTest Build Step section below.
830
831       Test   Test  the  software  by  loading  a CTestTestfile.cmake from the
832              build tree and executing the defined tests.  Record  the  output
833              and result of each test.  See the CTest Test Step section below.
834
835       Coverage
836              Compute coverage of the source code by running a coverage analy‐
837              sis tool and recording its output.  See the CTest Coverage  Step
838              section below.
839
840       MemCheck
841              Run the software test suite through a memory check tool.  Record
842              the test output, results, and issues reported by the tool.   See
843              the CTest MemCheck Step section below.
844
845       Submit Submit results recorded from other testing steps to the software
846              quality dashboard server.  See the CTest Submit Step section be‐
847              low.
848
849   Dashboard Client Modes
850       CTest defines three modes of operation as a dashboard client:
851
852       Nightly
853              This  mode  is intended to be invoked once per day, typically at
854              night.  It enables the Start, Update,  Configure,  Build,  Test,
855              Coverage,  and Submit steps by default.  Selected steps run even
856              if the Update step reports no changes to the source tree.
857
858       Continuous
859              This mode is intended to be invoked  repeatedly  throughout  the
860              day.  It enables the Start, Update, Configure, Build, Test, Cov‐
861              erage, and Submit steps by default, but exits after  the  Update
862              step if it reports no changes to the source tree.
863
864       Experimental
865              This mode is intended to be invoked by a developer to test local
866              changes.  It enables the Start, Configure, Build,  Test,  Cover‐
867              age, and Submit steps by default.
868
869   Dashboard Client via CTest Command-Line
870       CTest  can perform testing on an already-generated build tree.  Run the
871       ctest command with the current working directory set to the build  tree
872       and use one of these signatures:
873
874          ctest -D <mode>[<step>]
875          ctest -M <mode> [ -T <step> ]...
876
877       The  <mode>  must  be one of the above Dashboard Client Modes, and each
878       <step> must be one of the above Dashboard Client Steps.
879
880       CTest reads the Dashboard Client Configuration settings from a file  in
881       the  build  tree called either CTestConfiguration.ini or DartConfigura‐
882       tion.tcl (the names are historical).  The format of the file is:
883
884          # Lines starting in '#' are comments.
885          # Other non-blank lines are key-value pairs.
886          <setting>: <value>
887
888       where <setting> is the setting name and <value> is the setting value.
889
890       In build trees generated by CMake, this configuration file is generated
891       by  the CTest module if included by the project.  The module uses vari‐
892       ables to obtain a value for each setting as documented  with  the  set‐
893       tings below.
894
895   Dashboard Client via CTest Script
896       CTest  can  perform  testing  driven by a cmake-language(7) script that
897       creates and maintains the source and build tree as well  as  performing
898       the  testing steps.  Run the ctest command with the current working di‐
899       rectory set outside of any build tree and use one of these signatures:
900
901          ctest -S <script>
902          ctest -SP <script>
903
904       The <script> file must call CTest  Commands  commands  to  run  testing
905       steps  explicitly  as  documented below.  The commands obtain Dashboard
906       Client Configuration settings from their arguments  or  from  variables
907       set in the script.
908

DASHBOARD CLIENT CONFIGURATION

910       The Dashboard Client Steps may be configured by named settings as docu‐
911       mented in the following sections.
912
913   CTest Start Step
914       Start a new dashboard submission to be composed of results recorded  by
915       the following steps.
916
917       In a CTest Script, the ctest_start() command runs this step.  Arguments
918       to the command may specify some of  the  step  settings.   The  command
919       first  runs  the  command-line  specified by the CTEST_CHECKOUT_COMMAND
920       variable, if set, to initialize the source directory.
921
922       Configuration settings include:
923
924       BuildDirectory
925              The full path to the project build tree.
926
927CTest Script variable: CTEST_BINARY_DIRECTORY
928
929CTest module variable: PROJECT_BINARY_DIR
930
931       SourceDirectory
932              The full path to the project source tree.
933
934CTest Script variable: CTEST_SOURCE_DIRECTORY
935
936CTest module variable: PROJECT_SOURCE_DIR
937
938   CTest Update Step
939       In a CTest Script, the ctest_update() command runs  this  step.   Argu‐
940       ments to the command may specify some of the step settings.
941
942       Configuration settings to specify the version control tool include:
943
944       BZRCommand
945              bzr  command-line  tool  to  use  if  source  tree is managed by
946              Bazaar.
947
948CTest Script variable: CTEST_BZR_COMMAND
949
950CTest module variable: none
951
952       BZRUpdateOptions
953              Command-line options to the BZRCommand when updating the source.
954
955CTest Script variable: CTEST_BZR_UPDATE_OPTIONS
956
957CTest module variable: none
958
959       CVSCommand
960              cvs command-line tool to use if source tree is managed by CVS.
961
962CTest Script variable: CTEST_CVS_COMMAND
963
964CTest module variable: CVSCOMMAND
965
966       CVSUpdateOptions
967              Command-line options to the CVSCommand when updating the source.
968
969CTest Script variable: CTEST_CVS_UPDATE_OPTIONS
970
971CTest module variable: CVS_UPDATE_OPTIONS
972
973       GITCommand
974              git command-line tool to use if source tree is managed by Git.
975
976CTest Script variable: CTEST_GIT_COMMAND
977
978CTest module variable: GITCOMMAND
979
980              The source tree is updated by git fetch followed  by  git  reset
981              --hard  to  the  FETCH_HEAD.  The result is the same as git pull
982              except that any local modifications are overwritten.  Use GITUp‐
983              dateCustom to specify a different approach.
984
985       GITInitSubmodules
986              If set, CTest will update the repository's submodules before up‐
987              dating.
988
989CTest Script variable: CTEST_GIT_INIT_SUBMODULES
990
991CTest module variable: CTEST_GIT_INIT_SUBMODULES
992
993       GITUpdateCustom
994              Specify a custom command line (as a semicolon-separated list) to
995              run  in  the source tree (Git work tree) to update it instead of
996              running the GITCommand.
997
998CTest Script variable: CTEST_GIT_UPDATE_CUSTOM
999
1000CTest module variable: CTEST_GIT_UPDATE_CUSTOM
1001
1002       GITUpdateOptions
1003              Command-line options to the GITCommand when updating the source.
1004
1005CTest Script variable: CTEST_GIT_UPDATE_OPTIONS
1006
1007CTest module variable: GIT_UPDATE_OPTIONS
1008
1009       HGCommand
1010              hg command-line tool to use if source tree is managed by  Mercu‐
1011              rial.
1012
1013CTest Script variable: CTEST_HG_COMMAND
1014
1015CTest module variable: none
1016
1017       HGUpdateOptions
1018              Command-line options to the HGCommand when updating the source.
1019
1020CTest Script variable: CTEST_HG_UPDATE_OPTIONS
1021
1022CTest module variable: none
1023
1024       P4Client
1025              Value of the -c option to the P4Command.
1026
1027CTest Script variable: CTEST_P4_CLIENT
1028
1029CTest module variable: CTEST_P4_CLIENT
1030
1031       P4Command
1032              p4  command-line  tool  to use if source tree is managed by Per‐
1033              force.
1034
1035CTest Script variable: CTEST_P4_COMMAND
1036
1037CTest module variable: P4COMMAND
1038
1039       P4Options
1040              Command-line options to the P4Command for all invocations.
1041
1042CTest Script variable: CTEST_P4_OPTIONS
1043
1044CTest module variable: CTEST_P4_OPTIONS
1045
1046       P4UpdateCustom
1047              Specify a custom command line (as a semicolon-separated list) to
1048              run  in  the source tree (Perforce tree) to update it instead of
1049              running the P4Command.
1050
1051CTest Script variable: none
1052
1053CTest module variable: CTEST_P4_UPDATE_CUSTOM
1054
1055       P4UpdateOptions
1056              Command-line options to the P4Command when updating the source.
1057
1058CTest Script variable: CTEST_P4_UPDATE_OPTIONS
1059
1060CTest module variable: CTEST_P4_UPDATE_OPTIONS
1061
1062       SVNCommand
1063              svn command-line tool to use if source tree is managed  by  Sub‐
1064              version.
1065
1066CTest Script variable: CTEST_SVN_COMMAND
1067
1068CTest module variable: SVNCOMMAND
1069
1070       SVNOptions
1071              Command-line options to the SVNCommand for all invocations.
1072
1073CTest Script variable: CTEST_SVN_OPTIONS
1074
1075CTest module variable: CTEST_SVN_OPTIONS
1076
1077       SVNUpdateOptions
1078              Command-line options to the SVNCommand when updating the source.
1079
1080CTest Script variable: CTEST_SVN_UPDATE_OPTIONS
1081
1082CTest module variable: SVN_UPDATE_OPTIONS
1083
1084       UpdateCommand
1085              Specify the version-control command-line tool to use without de‐
1086              tecting the VCS that manages the source tree.
1087
1088CTest Script variable: CTEST_UPDATE_COMMAND
1089
1090CTest module variable: <VCS>COMMAND when UPDATE_TYPE is <vcs>,
1091                else UPDATE_COMMAND
1092
1093       UpdateOptions
1094              Command-line options to the UpdateCommand.
1095
1096CTest Script variable: CTEST_UPDATE_OPTIONS
1097
1098CTest  module  variable: <VCS>_UPDATE_OPTIONS when UPDATE_TYPE
1099                is <vcs>, else UPDATE_OPTIONS
1100
1101       UpdateType
1102              Specify the version-control system that manages the source  tree
1103              if  it  cannot be detected automatically.  The value may be bzr,
1104              cvs, git, hg, p4, or svn.
1105
1106CTest Script variable: none, detected from source tree
1107
1108CTest module variable:  UPDATE_TYPE  if  set,  else  CTEST_UP‐
1109                DATE_TYPE
1110
1111       UpdateVersionOnly
1112              Specify that you want the version control update command to only
1113              discover the current version that is checked out, and not to up‐
1114              date to a different version.
1115
1116CTest Script variable: CTEST_UPDATE_VERSION_ONLY
1117
1118       UpdateVersionOverride
1119              Specify the current version of your source tree.
1120
1121              When  this variable is set to a non-empty string, CTest will re‐
1122              port the value you specified rather than using the  update  com‐
1123              mand to discover the current version that is checked out. Use of
1124              this  variable  supersedes  UpdateVersionOnly.  Like  UpdateVer‐
1125              sionOnly,  using  this  variable  tells  CTest not to update the
1126              source tree to a different version.
1127
1128CTest Script variable: CTEST_UPDATE_VERSION_OVERRIDE
1129
1130       Additional configuration settings include:
1131
1132       NightlyStartTime
1133              In the Nightly dashboard mode, specify the "nightly start time".
1134              With  centralized version control systems (cvs and svn), the Up‐
1135              date step checks out the version of the software as of this time
1136              so  that multiple clients choose a common version to test.  This
1137              is not well-defined in distributed  version-control  systems  so
1138              the setting is ignored.
1139
1140CTest Script variable: CTEST_NIGHTLY_START_TIME
1141
1142CTest   module   variable:  NIGHTLY_START_TIME  if  set,  else
1143                CTEST_NIGHTLY_START_TIME
1144
1145   CTest Configure Step
1146       In a CTest Script, the ctest_configure() command runs this step.  Argu‐
1147       ments to the command may specify some of the step settings.
1148
1149       Configuration settings include:
1150
1151       ConfigureCommand
1152              Command-line  to  launch the software configuration process.  It
1153              will be executed in the location specified by the BuildDirectory
1154              setting.
1155
1156CTest Script variable: CTEST_CONFIGURE_COMMAND
1157
1158CTest    module    variable:    CMAKE_COMMAND    followed   by
1159                PROJECT_SOURCE_DIR
1160
1161       LabelsForSubprojects
1162              Specify a  semicolon-separated  list  of  labels  that  will  be
1163              treated  as subprojects. This mapping will be passed on to CDash
1164              when configure, test or build results are submitted.
1165
1166CTest Script variable: CTEST_LABELS_FOR_SUBPROJECTS
1167
1168CTest module variable: CTEST_LABELS_FOR_SUBPROJECTS
1169
1170              See Label and Subproject Summary.
1171
1172   CTest Build Step
1173       In a CTest Script, the ctest_build() command runs this step.  Arguments
1174       to the command may specify some of the step settings.
1175
1176       Configuration settings include:
1177
1178       DefaultCTestConfigurationType
1179              When the build system to be launched allows build-time selection
1180              of the configuration (e.g. Debug, Release), this  specifies  the
1181              default  configuration to be built when no -C option is given to
1182              the ctest command.  The value will be substituted into the value
1183              of  MakeCommand to replace the literal string ${CTEST_CONFIGURA‐
1184              TION_TYPE} if it appears.
1185
1186CTest Script variable: CTEST_CONFIGURATION_TYPE
1187
1188CTest module variable: DEFAULT_CTEST_CONFIGURATION_TYPE,  ini‐
1189                tialized by the CMAKE_CONFIG_TYPE environment variable
1190
1191       LabelsForSubprojects
1192              Specify  a  semicolon-separated  list  of  labels  that  will be
1193              treated as subprojects. This mapping will be passed on to  CDash
1194              when configure, test or build results are submitted.
1195
1196CTest Script variable: CTEST_LABELS_FOR_SUBPROJECTS
1197
1198CTest module variable: CTEST_LABELS_FOR_SUBPROJECTS
1199
1200              See Label and Subproject Summary.
1201
1202       MakeCommand
1203              Command-line  to  launch the software build process.  It will be
1204              executed in the location specified by  the  BuildDirectory  set‐
1205              ting.
1206
1207CTest Script variable: CTEST_BUILD_COMMAND
1208
1209CTest   module   variable:  MAKECOMMAND,  initialized  by  the
1210                build_command() command
1211
1212       UseLaunchers
1213              For build trees generated by CMake using  one  of  the  Makefile
1214              Generators   or   the   Ninja  generator,  specify  whether  the
1215              CTEST_USE_LAUNCHERS feature is enabled by the  CTestUseLaunchers
1216              module  (also  included by the CTest module).  When enabled, the
1217              generated build system wraps each invocation  of  the  compiler,
1218              linker,  or  custom command line with a "launcher" that communi‐
1219              cates with CTest via environment variables and files  to  report
1220              granular  build warning and error information.  Otherwise, CTest
1221              must "scrape" the build output log for diagnostics.
1222
1223CTest Script variable: CTEST_USE_LAUNCHERS
1224
1225CTest module variable: CTEST_USE_LAUNCHERS
1226
1227   CTest Test Step
1228       In a CTest Script, the ctest_test() command runs this step.   Arguments
1229       to the command may specify some of the step settings.
1230
1231       Configuration settings include:
1232
1233       ResourceSpecFile
1234              Specify a resource specification file.
1235
1236CTest Script variable: CTEST_RESOURCE_SPEC_FILE
1237
1238CTest module variable: CTEST_RESOURCE_SPEC_FILE
1239
1240              See Resource Allocation for more information.
1241
1242       LabelsForSubprojects
1243              Specify  a  semicolon-separated  list  of  labels  that  will be
1244              treated as subprojects. This mapping will be passed on to  CDash
1245              when configure, test or build results are submitted.
1246
1247CTest Script variable: CTEST_LABELS_FOR_SUBPROJECTS
1248
1249CTest module variable: CTEST_LABELS_FOR_SUBPROJECTS
1250
1251              See Label and Subproject Summary.
1252
1253       TestLoad
1254              While running tests in parallel (e.g. with -j), try not to start
1255              tests when they may cause the CPU load to  pass  above  a  given
1256              threshold.
1257
1258CTest Script variable: CTEST_TEST_LOAD
1259
1260CTest module variable: CTEST_TEST_LOAD
1261
1262       TimeOut
1263              The  default timeout for each test if not specified by the TIME‐
1264              OUT test property.
1265
1266CTest Script variable: CTEST_TEST_TIMEOUT
1267
1268CTest module variable: DART_TESTING_TIMEOUT
1269
1270       To report extra test values to CDash, see Additional Test Measurements.
1271
1272   CTest Coverage Step
1273       In a CTest Script, the ctest_coverage() command runs this step.   Argu‐
1274       ments to the command may specify some of the step settings.
1275
1276       Configuration settings include:
1277
1278       CoverageCommand
1279              Command-line  tool  to  perform  software coverage analysis.  It
1280              will be executed in the location specified by the BuildDirectory
1281              setting.
1282
1283CTest Script variable: CTEST_COVERAGE_COMMAND
1284
1285CTest module variable: COVERAGE_COMMAND
1286
1287       CoverageExtraFlags
1288              Specify command-line options to the CoverageCommand tool.
1289
1290CTest Script variable: CTEST_COVERAGE_EXTRA_FLAGS
1291
1292CTest module variable: COVERAGE_EXTRA_FLAGS
1293
1294              These options are the first arguments passed to CoverageCommand.
1295
1296   CTest MemCheck Step
1297       In  a CTest Script, the ctest_memcheck() command runs this step.  Argu‐
1298       ments to the command may specify some of the step settings.
1299
1300       Configuration settings include:
1301
1302       MemoryCheckCommand
1303              Command-line tool to perform  dynamic  analysis.   Test  command
1304              lines will be launched through this tool.
1305
1306CTest Script variable: CTEST_MEMORYCHECK_COMMAND
1307
1308CTest module variable: MEMORYCHECK_COMMAND
1309
1310       MemoryCheckCommandOptions
1311              Specify  command-line  options  to  the MemoryCheckCommand tool.
1312              They will be placed prior to the test command line.
1313
1314CTest Script variable: CTEST_MEMORYCHECK_COMMAND_OPTIONS
1315
1316CTest module variable: MEMORYCHECK_COMMAND_OPTIONS
1317
1318       MemoryCheckType
1319              Specify the type of memory checking to perform.
1320
1321CTest Script variable: CTEST_MEMORYCHECK_TYPE
1322
1323CTest module variable: MEMORYCHECK_TYPE
1324
1325       MemoryCheckSanitizerOptions
1326              Specify options to sanitizers when running with  a  sanitize-en‐
1327              abled build.
1328
1329CTest Script variable: CTEST_MEMORYCHECK_SANITIZER_OPTIONS
1330
1331CTest module variable: MEMORYCHECK_SANITIZER_OPTIONS
1332
1333       MemoryCheckSuppressionFile
1334              Specify a file containing suppression rules for the MemoryCheck‐
1335              Command tool.  It will be passed with options appropriate to the
1336              tool.
1337
1338CTest Script variable: CTEST_MEMORYCHECK_SUPPRESSIONS_FILE
1339
1340CTest module variable: MEMORYCHECK_SUPPRESSIONS_FILE
1341
1342       Additional configuration settings include:
1343
1344       BoundsCheckerCommand
1345              Specify  a  MemoryCheckCommand  that is known to be command-line
1346              compatible with Bounds Checker.
1347
1348CTest Script variable: none
1349
1350CTest module variable: none
1351
1352       PurifyCommand
1353              Specify a MemoryCheckCommand that is known  to  be  command-line
1354              compatible with Purify.
1355
1356CTest Script variable: none
1357
1358CTest module variable: PURIFYCOMMAND
1359
1360       ValgrindCommand
1361              Specify  a  MemoryCheckCommand  that is known to be command-line
1362              compatible with Valgrind.
1363
1364CTest Script variable: none
1365
1366CTest module variable: VALGRIND_COMMAND
1367
1368       ValgrindCommandOptions
1369              Specify command-line options to the ValgrindCommand tool.   They
1370              will be placed prior to the test command line.
1371
1372CTest Script variable: none
1373
1374CTest module variable: VALGRIND_COMMAND_OPTIONS
1375
1376       DrMemoryCommand
1377              Specify  a MemoryCheckCommand that is known to be a command-line
1378              compatible with DrMemory.
1379
1380CTest Script variable: none
1381
1382CTest module variable: DRMEMORY_COMMAND
1383
1384       DrMemoryCommandOptions
1385              Specify command-line options to the DrMemoryCommand tool.   They
1386              will be placed prior to the test command line.
1387
1388CTest Script variable: none
1389
1390CTest module variable: DRMEMORY_COMMAND_OPTIONS
1391
1392       CudaSanitizerCommand
1393              Specify  a MemoryCheckCommand that is known to be a command-line
1394              compatible with cuda-memcheck or compute-sanitizer.
1395
1396CTest Script variable: none
1397
1398CTest module variable: CUDA_SANITIZER_COMMAND
1399
1400       CudaSanitizerCommandOptions
1401              Specify command-line options to the  CudaSanitizerCommand  tool.
1402              They will be placed prior to the test command line.
1403
1404CTest Script variable: none
1405
1406CTest module variable: CUDA_SANITIZER_COMMAND_OPTIONS
1407
1408   CTest Submit Step
1409       In  a  CTest  Script, the ctest_submit() command runs this step.  Argu‐
1410       ments to the command may specify some of the step settings.
1411
1412       Configuration settings include:
1413
1414       BuildName
1415              Describe the dashboard client  platform  with  a  short  string.
1416              (Operating system, compiler, etc.)
1417
1418CTest Script variable: CTEST_BUILD_NAME
1419
1420CTest module variable: BUILDNAME
1421
1422       CDashVersion
1423              Legacy option.  Not used.
1424
1425CTest Script variable: none, detected from server
1426
1427CTest module variable: CTEST_CDASH_VERSION
1428
1429       CTestSubmitRetryCount
1430              Specify  a  number  of  attempts  to retry submission on network
1431              failure.
1432
1433CTest  Script   variable:   none,   use   the   ctest_submit()
1434                RETRY_COUNT option.
1435
1436CTest module variable: CTEST_SUBMIT_RETRY_COUNT
1437
1438       CTestSubmitRetryDelay
1439              Specify a delay before retrying submission on network failure.
1440
1441CTest  Script variable: none, use the ctest_submit() RETRY_DE‐
1442                LAY option.
1443
1444CTest module variable: CTEST_SUBMIT_RETRY_DELAY
1445
1446       CurlOptions
1447              Specify a semicolon-separated list of  options  to  control  the
1448              Curl  library  that  CTest  uses  internally  to  connect to the
1449              server.  Possible  options  are  CURLOPT_SSL_VERIFYPEER_OFF  and
1450              CURLOPT_SSL_VERIFYHOST_OFF.
1451
1452CTest Script variable: CTEST_CURL_OPTIONS
1453
1454CTest module variable: CTEST_CURL_OPTIONS
1455
1456       DropLocation
1457              Legacy  option.   When  SubmitURL  is not set, it is constructed
1458              from DropMethod, DropSiteUser, DropSitePassword,  DropSite,  and
1459              DropLocation.
1460
1461CTest Script variable: CTEST_DROP_LOCATION
1462
1463CTest    module   variable:   DROP_LOCATION   if   set,   else
1464                CTEST_DROP_LOCATION
1465
1466       DropMethod
1467              Legacy option.  When SubmitURL is not  set,  it  is  constructed
1468              from  DropMethod,  DropSiteUser, DropSitePassword, DropSite, and
1469              DropLocation.
1470
1471CTest Script variable: CTEST_DROP_METHOD
1472
1473CTest   module   variable:   DROP_METHOD    if    set,    else
1474                CTEST_DROP_METHOD
1475
1476       DropSite
1477              Legacy  option.   When  SubmitURL  is not set, it is constructed
1478              from DropMethod, DropSiteUser, DropSitePassword,  DropSite,  and
1479              DropLocation.
1480
1481CTest Script variable: CTEST_DROP_SITE
1482
1483CTest module variable: DROP_SITE if set, else CTEST_DROP_SITE
1484
1485       DropSitePassword
1486              Legacy  option.   When  SubmitURL  is not set, it is constructed
1487              from DropMethod, DropSiteUser, DropSitePassword,  DropSite,  and
1488              DropLocation.
1489
1490CTest Script variable: CTEST_DROP_SITE_PASSWORD
1491
1492CTest   module   variable:  DROP_SITE_PASSWORD  if  set,  else
1493                CTEST_DROP_SITE_PASWORD
1494
1495       DropSiteUser
1496              Legacy option.  When SubmitURL is not  set,  it  is  constructed
1497              from  DropMethod,  DropSiteUser, DropSitePassword, DropSite, and
1498              DropLocation.
1499
1500CTest Script variable: CTEST_DROP_SITE_USER
1501
1502CTest   module   variable:   DROP_SITE_USER   if   set,   else
1503                CTEST_DROP_SITE_USER
1504
1505       IsCDash
1506              Legacy option.  Not used.
1507
1508CTest Script variable: CTEST_DROP_SITE_CDASH
1509
1510CTest module variable: CTEST_DROP_SITE_CDASH
1511
1512       ScpCommand
1513              Legacy option.  Not used.
1514
1515CTest Script variable: CTEST_SCP_COMMAND
1516
1517CTest module variable: SCPCOMMAND
1518
1519       Site   Describe  the  dashboard  client  host site with a short string.
1520              (Hostname, domain, etc.)
1521
1522CTest Script variable: CTEST_SITE
1523
1524CTest module variable: SITE, initialized  by  the  site_name()
1525                command
1526
1527       SubmitURL
1528              The  http  or https URL of the dashboard server to send the sub‐
1529              mission to.
1530
1531CTest Script variable: CTEST_SUBMIT_URL
1532
1533CTest module variable:  SUBMIT_URL  if  set,  else  CTEST_SUB‐
1534                MIT_URL
1535
1536       TriggerSite
1537              Legacy option.  Not used.
1538
1539CTest Script variable: CTEST_TRIGGER_SITE
1540
1541CTest  module  variable: TRIGGER_SITE if set, else CTEST_TRIG‐
1542                GER_SITE
1543

SHOW AS JSON OBJECT MODEL

1545       When the --show-only=json-v1 command line option is given, the test in‐
1546       formation  is  output  in  JSON format.  Version 1.0 of the JSON object
1547       model is defined as follows:
1548
1549       kind   The string "ctestInfo".
1550
1551       version
1552              A JSON object specifying the version  components.   Its  members
1553              are
1554
1555              major  A  non-negative integer specifying the major version com‐
1556                     ponent.
1557
1558              minor  A non-negative integer specifying the minor version  com‐
1559                     ponent.
1560
1561       backtraceGraph
1562              JSON  object representing backtrace information with the follow‐
1563              ing members:
1564
1565              commands
1566                     List of command names.
1567
1568              files  List of file names.
1569
1570              nodes  List of node JSON objects with members:
1571
1572                     command
1573                            Index into the commands member of  the  backtrace‐
1574                            Graph.
1575
1576                     file   Index into the files member of the backtraceGraph.
1577
1578                     line   Line  number  in  the file where the backtrace was
1579                            added.
1580
1581                     parent Index into the nodes member of the  backtraceGraph
1582                            representing the parent in the graph.
1583
1584       tests  A JSON array listing information about each test.  Each entry is
1585              a JSON object with members:
1586
1587              name   Test name.
1588
1589              config Configuration that the test can  run  on.   Empty  string
1590                     means any config.
1591
1592              command
1593                     List  where the first element is the test command and the
1594                     remaining elements are the command arguments.
1595
1596              backtrace
1597                     Index into the nodes member of the backtraceGraph.
1598
1599              properties
1600                     Test properties.  Can contain keys for each of  the  sup‐
1601                     ported test properties.
1602

RESOURCE ALLOCATION

1604       CTest provides a mechanism for tests to specify the resources that they
1605       need in a fine-grained way, and for  users  to  specify  the  resources
1606       available  on the running machine. This allows CTest to internally keep
1607       track of which resources are in use  and  which  are  free,  scheduling
1608       tests  in  a way that prevents them from trying to claim resources that
1609       are not available.
1610
1611       When the resource allocation feature is used, CTest will  not  oversub‐
1612       scribe  resources.  For  example, if a resource has 8 slots, CTest will
1613       not run tests that collectively use more than 8 slots at a  time.  This
1614       has  the  effect  of limiting how many tests can run at any given time,
1615       even if a high -j argument is used, if those tests all use  some  slots
1616       from  the same resource.  In addition, it means that a single test that
1617       uses more of a resource than is available on a machine will not run  at
1618       all (and will be reported as Not Run).
1619
1620       A common use case for this feature is for tests that require the use of
1621       a GPU.  Multiple tests can simultaneously allocate memory from  a  GPU,
1622       but if too many tests try to do this at once, some of them will fail to
1623       allocate, resulting in a failed test, even though the test  would  have
1624       succeeded if it had the memory it needed. By using the resource alloca‐
1625       tion feature, each test can specify how much memory it requires from  a
1626       GPU,  allowing CTest to schedule tests in a way that running several of
1627       these tests at once does not exhaust the GPU's memory pool.
1628
1629       Please note that CTest has no concept of what a GPU is or how much mem‐
1630       ory it has, nor does it have any way of communicating with a GPU to re‐
1631       trieve this information or perform any memory management. CTest  simply
1632       keeps  track  of a list of abstract resource types, each of which has a
1633       certain number of slots available for tests to use. Each test specifies
1634       the number of slots that it requires from a certain resource, and CTest
1635       then schedules them in a way that prevents the total number of slots in
1636       use  from  exceeding the listed capacity.  When a test is executed, and
1637       slots from a resource are allocated to that test, tests may assume that
1638       they  have  exclusive use of those slots for the duration of the test's
1639       process.
1640
1641       The CTest resource allocation feature consists of two inputs:
1642
1643       • The resource specification file, described below, which describes the
1644         resources available on the system.
1645
1646       • The  RESOURCE_GROUPS property of tests, which describes the resources
1647         required by the test.
1648
1649       When CTest runs a test, the resources allocated to that test are passed
1650       in the form of a set of environment variables as described below. Using
1651       this information to decide which resource to connect to is left to  the
1652       test writer.
1653
1654       The  RESOURCE_GROUPS property tells CTest what resources a test expects
1655       to use grouped in a way meaningful to the test.  The test  itself  must
1656       read  the  environment variables to determine which resources have been
1657       allocated to each group.  For example, each group may correspond  to  a
1658       process the test will spawn when executed.
1659
1660       Note  that  even  if a test specifies a RESOURCE_GROUPS property, it is
1661       still possible for that to test to run without any resource  allocation
1662       (and  without the corresponding environment variables) if the user does
1663       not pass a resource  specification  file.  Passing  this  file,  either
1664       through  the  --resource-spec-file  command-line  argument  or  the RE‐
1665       SOURCE_SPEC_FILE argument to ctest_test(), is what  activates  the  re‐
1666       source   allocation   feature.   Tests   should   check  the  CTEST_RE‐
1667       SOURCE_GROUP_COUNT environment variable to find out whether or not  re‐
1668       source allocation is activated. This variable will always (and only) be
1669       defined if resource allocation is activated. If resource allocation  is
1670       not  activated,  then  the CTEST_RESOURCE_GROUP_COUNT variable will not
1671       exist, even if it exists for the parent ctest process. If a test  abso‐
1672       lutely must have resource allocation, then it can return a failing exit
1673       code or use the SKIP_RETURN_CODE or SKIP_REGULAR_EXPRESSION  properties
1674       to indicate a skipped test.
1675
1676   Resource Specification File
1677       The  resource  specification  file  is  a  JSON file which is passed to
1678       CTest, either on the ctest(1) command line as --resource-spec-file,  or
1679       as  the  RESOURCE_SPEC_FILE  argument  of  ctest_test(). If a dashboard
1680       script is used and RESOURCE_SPEC_FILE is not specified,  the  value  of
1681       CTEST_RESOURCE_SPEC_FILE  in  the dashboard script is used instead.  If
1682       --resource-spec-file, RESOURCE_SPEC_FILE, and  CTEST_RESOURCE_SPEC_FILE
1683       in  the  dashboard  script  are  not  specified, the value of CTEST_RE‐
1684       SOURCE_SPEC_FILE in the CMake build is used instead. If none  of  these
1685       are specified, no resource spec file is used.
1686
1687       The  resource specification file must be a JSON object. All examples in
1688       this document assume the following resource specification file:
1689
1690          {
1691            "version": {
1692              "major": 1,
1693              "minor": 0
1694            },
1695            "local": [
1696              {
1697                "gpus": [
1698                  {
1699                    "id": "0",
1700                    "slots": 2
1701                  },
1702                  {
1703                    "id": "1",
1704                    "slots": 4
1705                  },
1706                  {
1707                    "id": "2",
1708                    "slots": 2
1709                  },
1710                  {
1711                    "id": "3"
1712                  }
1713                ],
1714                "crypto_chips": [
1715                  {
1716                    "id": "card0",
1717                    "slots": 4
1718                  }
1719                ]
1720              }
1721            ]
1722          }
1723
1724       The members are:
1725
1726       version
1727              An object containing a major integer field and a  minor  integer
1728              field.   Currently, the only supported version is major 1, minor
1729              0. Any other value is an error.
1730
1731       local  A JSON array of resource sets present on the system.  Currently,
1732              this array is restricted to being of size 1.
1733
1734              Each array element is a JSON object with members whose names are
1735              equal to the desired resource types, such as gpus.  These  names
1736              must  start with a lowercase letter or an underscore, and subse‐
1737              quent characters can be a lowercase letter, a digit, or  an  un‐
1738              derscore.  Uppercase  letters  are  not allowed, because certain
1739              platforms have case-insensitive environment variables.  See  the
1740              Environment  Variables section below for more information. It is
1741              recommended that the resource type name be the plural of a noun,
1742              such as gpus or crypto_chips (and not gpu or crypto_chip.)
1743
1744              Please  note that the names gpus and crypto_chips are just exam‐
1745              ples, and CTest does not interpret them in any way. You are free
1746              to  make up any resource type you want to meet your own require‐
1747              ments.
1748
1749              The value for each resource type is a JSON array  consisting  of
1750              JSON  objects, each of which describe a specific instance of the
1751              specified resource. These objects have the following members:
1752
1753              id     A string consisting of an identifier  for  the  resource.
1754                     Each  character in the identifier can be a lowercase let‐
1755                     ter, a digit, or an underscore.   Uppercase  letters  are
1756                     not allowed.
1757
1758                     Identifiers  must  be unique within a resource type. How‐
1759                     ever, they do not  have  to  be  unique  across  resource
1760                     types.  For  example, it is valid to have a gpus resource
1761                     named 0 and a crypto_chips resource named 0, but not  two
1762                     gpus resources both named 0.
1763
1764                     Please  note  that the IDs 0, 1, 2, 3, and card0 are just
1765                     examples, and CTest does not interpret them in  any  way.
1766                     You are free to make up any IDs you want to meet your own
1767                     requirements.
1768
1769              slots  An optional unsigned  number  specifying  the  number  of
1770                     slots  available on the resource. For example, this could
1771                     be megabytes of RAM  on  a  GPU,  or  cryptography  units
1772                     available  on a cryptography chip. If slots is not speci‐
1773                     fied, a default value of 1 is assumed.
1774
1775       In the example file above, there are four GPUs with ID's 0  through  3.
1776       GPU 0 has 2 slots, GPU 1 has 4, GPU 2 has 2, and GPU 3 has a default of
1777       1 slot. There is also one cryptography chip with 4 slots.
1778
1779   RESOURCE_GROUPS Property
1780       See RESOURCE_GROUPS for a description of this property.
1781
1782   Environment Variables
1783       Once CTest has decided which resources to allocate to a test, it passes
1784       this  information  to  the  test  executable as a series of environment
1785       variables. For each example below, we will  assume  that  the  test  in
1786       question       has       a       RESOURCE_GROUPS       property      of
1787       2,gpus:2;gpus:4,gpus:1,crypto_chips:2.
1788
1789       The following variables are passed to the test process:
1790
1791       CTEST_RESOURCE_GROUP_COUNT
1792              The total number of  groups  specified  by  the  RESOURCE_GROUPS
1793              property. For example:
1794
1795CTEST_RESOURCE_GROUP_COUNT=3
1796
1797              This  variable will only be defined if ctest(1) has been given a
1798              --resource-spec-file, or if ctest_test() has been  given  a  RE‐
1799              SOURCE_SPEC_FILE.  If  no  resource  specification file has been
1800              given, this variable will not be defined.
1801
1802       CTEST_RESOURCE_GROUP_<num>
1803              The list of resource types allocated to each  group,  with  each
1804              item  separated  by  a  comma.  <num>  is  a number from zero to
1805              CTEST_RESOURCE_GROUP_COUNT minus one. CTEST_RESOURCE_GROUP_<num>
1806              is defined for each <num> in this range. For example:
1807
1808CTEST_RESOURCE_GROUP_0=gpus
1809
1810CTEST_RESOURCE_GROUP_1=gpus
1811
1812CTEST_RESOURCE_GROUP_2=crypto_chips,gpus
1813
1814       CTEST_RESOURCE_GROUP_<num>_<resource-type>
1815              The  list of resource IDs and number of slots from each ID allo‐
1816              cated to each group for a given  resource  type.  This  variable
1817              consists  of  a  series of pairs, each pair separated by a semi‐
1818              colon, and with the two items in the pair separated by a  comma.
1819              The  first  item in each pair is id: followed by the ID of a re‐
1820              source of type <resource-type>, and the second  item  is  slots:
1821              followed  by the number of slots from that resource allocated to
1822              the given group. For example:
1823
1824CTEST_RESOURCE_GROUP_0_GPUS=id:0,slots:2
1825
1826CTEST_RESOURCE_GROUP_1_GPUS=id:2,slots:2
1827
1828CTEST_RESOURCE_GROUP_2_GPUS=id:1,slots:4;id:3,slots:1
1829
1830CTEST_RESOURCE_GROUP_2_CRYPTO_CHIPS=id:card0,slots:2
1831
1832              In this example, group 0 gets 2 slots from GPU 0, group 1 gets 2
1833              slots  from  GPU  2, and group 2 gets 4 slots from GPU 1, 1 slot
1834              from GPU 3, and 2 slots from cryptography chip card0.
1835
1836              <num> is a number from zero to CTEST_RESOURCE_GROUP_COUNT  minus
1837              one.   <resource-type> is the name of a resource type, converted
1838              to uppercase.  CTEST_RESOURCE_GROUP_<num>_<resource-type> is de‐
1839              fined  for  the  product of each <num> in the range listed above
1840              and each resource type listed in CTEST_RESOURCE_GROUP_<num>.
1841
1842              Because some platforms have case-insensitive names for  environ‐
1843              ment  variables,  the names of resource types may not clash in a
1844              case-insensitive environment.  Because of this, for the sake  of
1845              simplicity,  all  resource types must be listed in all lowercase
1846              in the resource specification file and  in  the  RESOURCE_GROUPS
1847              property,  and  they  are  converted  to  all  uppercase  in the
1848              CTEST_RESOURCE_GROUP_<num>_<resource-type> environment variable.
1849

SEE ALSO

1851       The following resources are available to get help using CMake:
1852
1853       Home Page
1854              https://cmake.org
1855
1856              The primary starting point for learning about CMake.
1857
1858       Online Documentation and Community Resources
1859              https://cmake.org/documentation
1860
1861              Links to available documentation and community resources may  be
1862              found on this web page.
1863
1864       Discourse Forum
1865              https://discourse.cmake.org
1866
1867              The Discourse Forum hosts discussion and questions about CMake.
1868
1869       : https://cdash.org
1870
1872       2000-2021 Kitware, Inc. and Contributors
1873
1874
1875
1876
18773.22.0                           Dec 02, 2021                         CTEST(1)
Impressum