1App::Yath::Command::tesUts(e3r)Contributed Perl DocumentAaptpi:o:nYath::Command::test(3)
2
6 App::Yath::Command::test - Run tests
7
9 This yath command (which is also the default command) will run all the
10 test files for the current project. If no test files are specified this
11 command will look for the 't', and 't2' directories, as well as the
12 'test.pl' file.
13 This command is always recursive when given directories.
15 This command will add 'lib', 'blib/arch' and 'blib/lib' to the perl
17 path for you by default (after any -I's). You can specify -l if you
18 just want lib, -b if you just want the blib paths. If you specify both
19 -l and -b both will be added in the order you specify (order relative
20 to any -I options will also be preserved. If you do not specify they
21 will be added in this order: -I's, lib, blib/lib, blib/arch. You can
22 also add --no-lib and --no-blib to avoid both.
23 Any command line argument that is not an option will be treated as a
25 test file or directory of test files to be run.
26 If you wish to specify the ARGV for tests you may append them after
28 '::'. This is mainly useful for Test::Class::Moose and similar tools.
29 EVERY test run will get the same ARGV.
30
32 $ yath [YATH OPTIONS] test [COMMAND OPTIONS]
33 YATH OPTIONS
35 Developer
36 --dev-lib
38 --dev-lib=lib
39 -D
40 -D=lib
41 -Dlib
42 --no-dev-lib
43 Add paths to @INC before loading ANYTHING. This is what you use if
44 you are developing yath or yath plugins to make sure the yath
45 script finds the local code instead of the installed versions of
46 the same code. You can provide an argument (-Dfoo) to provide a
47 custom path, or you can just use -D without and arg to add lib,
48 blib/lib and blib/arch.
49 Can be specified multiple times
51 Environment
53 --persist-dir ARG
55 --persist-dir=ARG
56 --no-persist-dir
57 Where to find persistence files.
58 --persist-file ARG
60 --persist-file=ARG
61 --pfile ARG
62 --pfile=ARG
63 --no-persist-file
64 Where to find the persistence file. The default is
65 /{system-tempdir}/project-yath-persist.json. If no project is
66 specified then it will fall back to the current directory. If the
67 current directory is not writable it will default to
68 /tmp/yath-persist.json which limits you to one persistent runner on
69 your system.
70 --project ARG
72 --project=ARG
73 --project-name ARG
74 --project-name=ARG
75 --no-project
76 This lets you provide a label for your current project/codebase.
77 This is best used in a .yath.rc file. This is necessary for a
78 persistent runner.
79 Finder Options
81 --finder MyFinder
83 --finder +Test2::Harness::Finder::MyFinder
84 --no-finder
85 Specify what Finder subclass to use when searching for
86 files/processing the file list. Use the "+" prefix to specify a
87 fully qualified namespace, otherwise Test2::Harness::Finder::XXX
88 namespace is assumed.
89 Help and Debugging
91 --show-opts
93 --no-show-opts
94 Exit after showing what yath thinks your options mean
95 --version
97 -V
98 --no-version
99 Exit after showing a helpful usage message
100 Plugins
102 --no-scan-plugins
104 --no-no-scan-plugins
105 Normally yath scans for and loads all App::Yath::Plugin::* modules
106 in order to bring in command-line options they may provide. This
107 flag will disable that. This is useful if you have a naughty plugin
108 that it loading other modules when it should not.
109 --plugins PLUGIN
111 --plugins +App::Yath::Plugin::PLUGIN
112 --plugins PLUGIN=arg1,arg2,...
113 --plugin PLUGIN
114 --plugin +App::Yath::Plugin::PLUGIN
115 --plugin PLUGIN=arg1,arg2,...
116 -pPLUGIN
117 --no-plugins
118 Load a yath plugin.
119 Can be specified multiple times
121 COMMAND OPTIONS
123 Display Options
124 --color
126 --no-color
127 Turn color on, default is true if STDOUT is a TTY.
128 --no-wrap
130 --no-no-wrap
131 Do not do fancy text-wrapping, let the terminal handle it
132 --progress
134 --no-progress
135 Toggle progress indicators. On by default if STDOUT is a TTY. You
136 can use --no-progress to disable the 'events seen' counter and
137 buffered event pre-display
138 --quiet
140 -q
141 --no-quiet
142 Be very quiet.
143 Can be specified multiple times
145 --renderers +My::Renderer
147 --renderers Renderer=arg1,arg2,...
148 --renderer +My::Renderer
149 --renderer Renderer=arg1,arg2,...
150 --no-renderers
151 Specify renderers, (Default: "Formatter=Test2"). Use "+" to give a
152 fully qualified module name. Without "+"
153 "Test2::Harness::Renderer::" will be prepended to your argument.
154 Can be specified multiple times. If the same key is listed multiple
156 times the value lists will be appended together.
157 --show-times
159 -T
160 --no-show-times
161 Show the timing data for each job
162 --term-width 80
164 --term-width 200
165 --term-size 80
166 --term-size 200
167 --no-term-width
168 Alternative to setting $TABLE_TERM_SIZE. Setting this will override
169 the terminal width detection to the number of characters specified.
170 --verbose
172 -v
173 --no-verbose
174 Be more verbose
175 Can be specified multiple times
177 Finder Options
179 --default-at-search ARG
181 --default-at-search=ARG
182 --no-default-at-search
183 Specify the default file/dir search when 'AUTHOR_TESTING' is set.
184 Defaults to './xt'. The default AT search is only used if no files
185 were specified at the command line
186 Can be specified multiple times
188 --default-search ARG
190 --default-search=ARG
191 --no-default-search
192 Specify the default file/dir search. defaults to './t', './t2', and
193 'test.pl'. The default search is only used if no files were
194 specified at the command line
195 Can be specified multiple times
197 --durations file.json
199 --durations http://example.com/durations.json
200 --no-durations
201 Point at a json file or url which has a hash of relative test
202 filenames as keys, and 'SHORT', 'MEDIUM', or 'LONG' as values. This
203 will override durations listed in the file headers. An exception
204 will be thrown if the durations file or url does not work.
205 --exclude-file t/nope.t
207 --no-exclude-file
208 Exclude a file from testing
209 Can be specified multiple times
211 --exclude-list file.txt
213 --exclude-list http://example.com/exclusions.txt
214 --no-exclude-list
215 Point at a file or url which has a new line separated list of test
216 file names to exclude from testing. Starting a line with a '#' will
217 comment it out (for compatibility with Test2::Aggregate list
218 files).
219 Can be specified multiple times
221 --exclude-pattern t/nope.t
223 --no-exclude-pattern
224 Exclude a pattern from testing, matched using m/$PATTERN/
225 Can be specified multiple times
227 --extension ARG
229 --extension=ARG
230 --ext ARG
231 --ext=ARG
232 --no-extension
233 Specify valid test filename extensions, default: t and t2
234 Can be specified multiple times
236 --maybe-durations file.json
238 --maybe-durations http://example.com/durations.json
239 --no-maybe-durations
240 Point at a json file or url which has a hash of relative test
241 filenames as keys, and 'SHORT', 'MEDIUM', or 'LONG' as values. This
242 will override durations listed in the file headers. An exception
243 will be thrown if the durations file or url does not work.
244 --no-long
246 --no-no-long
247 Do not run tests that have their duration flag set to 'LONG'
248 --only-long
250 --no-only-long
251 Only run tests that have their duration flag set to 'LONG'
252 --search ARG
254 --search=ARG
255 --no-search
256 List of tests and test directories to use instead of the default
257 search paths. Typically these can simply be listed as command line
258 arguments without the --search prefix.
259 Can be specified multiple times
261 Formatter Options
263 --formatter ARG
265 --formatter=ARG
266 --no-formatter
267 NO DESCRIPTION - FIX ME
268 --qvf
270 --no-qvf
271 [Q]uiet, but [V]erbose on [F]ailure. Hide all output from tests
272 when they pass, except to say they passed. If a test fails then ALL
273 output from the test is verbosely output.
274 --show-job-end
276 --no-show-job-end
277 Show output when a job ends. (Default: on)
278 --show-job-info
280 --no-show-job-info
281 Show the job configuration when a job starts. (Default: off, unless
282 -vv)
283 --show-job-launch
285 --no-show-job-launch
286 Show output for the start of a job. (Default: off unless -v)
287 --show-run-info
289 --no-show-run-info
290 Show the run configuration when a run starts. (Default: off, unless
291 -vv)
292 Help and Debugging
294 --dummy
296 -d
297 --no-dummy
298 Dummy run, do not actually execute anything
299 Can also be set with the following environment variables:
301 "T2_HARNESS_DUMMY"
302 --help
304 -h
305 --no-help
306 exit after showing help information
307 --keep-dirs
309 --keep_dir
310 -k
311 --no-keep-dirs
312 Do not delete directories when done. This is useful if you want to
313 inspect the directories used for various commands.
314 --summary
316 --summary=/path/to/summary.json
317 --no-summary
318 Write out a summary json file, if no path is provided
319 'summary.json' will be used. The .json extension is added
320 automatically if omitted.
321 Logging Options
323 --bzip2
325 --bz2
326 --bzip2_log
327 -B
328 --no-bzip2
329 Use bzip2 compression when writing the log. This option implies -L.
330 The .bz2 prefix is added to log file name for you
331 --gzip
333 --gz
334 --gzip_log
335 -G
336 --no-gzip
337 Use gzip compression when writing the log. This option implies -L.
338 The .gz prefix is added to log file name for you
339 --log
341 -L
342 --no-log
343 Turn on logging
344 --log-dir ARG
346 --log-dir=ARG
347 --no-log-dir
348 Specify a log directory. Will fall back to the system temp dir.
349 --log-file ARG
351 --log-file=ARG
352 -F ARG
353 -F=ARG
354 --no-log-file
355 Specify the name of the log file. This option implies -L.
356 --log-file-format ARG
358 --log-file-format=ARG
359 --lff ARG
360 --lff=ARG
361 --no-log-file-format
362 Specify the format for automatically-generated log files.
363 Overridden by --log-file, if given. This option implies -L
364 (Default: \$YATH_LOG_FILE_FORMAT, if that is set, or else
365 "%!P%Y-%m-%d~%H:%M:%S~%!U~%!p.jsonl"). This is a string in which
366 percent-escape sequences will be replaced as per POSIX::strftime.
367 The following special escape sequences are also replaced: (%!P :
368 Project name followed by a ~, if a project is defined, otherwise
369 empty string) (%!U : the unique test run ID) (%!p : the process ID)
370 (%!S : the number of seconds since local midnight UTC)
371 Can also be set with the following environment variables:
373 "YATH_LOG_FILE_FORMAT", "TEST2_HARNESS_LOG_FORMAT"
374 Notification Options
376 --notify-email foo@example.com
378 --no-notify-email
379 Email the test results to the specified email address(es)
380 Can be specified multiple times
382 --notify-email-fail foo@example.com
384 --no-notify-email-fail
385 Email failing results to the specified email address(es)
386 Can be specified multiple times
388 --notify-email-from foo@example.com
390 --no-notify-email-from
391 If any email is sent, this is who it will be from
392 --notify-email-owner
394 --no-notify-email-owner
395 Email the owner of broken tests files upon failure. Add `# HARNESS-
396 META-OWNER foo@example.com` to the top of a test file to give it an
397 owner
398 --notify-no-batch-email
400 --no-notify-no-batch-email
401 Usually owner failures are sent as a single batch at the end of
402 testing. Toggle this to send failures as they happen.
403 --notify-no-batch-slack
405 --no-notify-no-batch-slack
406 Usually owner failures are sent as a single batch at the end of
407 testing. Toggle this to send failures as they happen.
408 --notify-slack '#foo'
410 --notify-slack '@bar'
411 --no-notify-slack
412 Send results to a slack channel and/or user
413 Can be specified multiple times
415 --notify-slack-fail '#foo'
417 --notify-slack-fail '@bar'
418 --no-notify-slack-fail
419 Send failing results to a slack channel and/or user
420 Can be specified multiple times
422 --notify-slack-owner
424 --no-notify-slack-owner
425 Send slack notifications to the slack channels/users listed in test
426 meta-data when tests fail.
427 --notify-slack-url https://hooks.slack.com/...
429 --no-notify-slack-url
430 Specify an API endpoint for slack webhook integrations
431 --notify-text ARG
433 --notify-text=ARG
434 --message ARG
435 --message=ARG
436 --msg ARG
437 --msg=ARG
438 --no-notify-text
439 Add a custom text snippet to email/slack notifications
440 Run Options
442 --author-testing
444 -A
445 --no-author-testing
446 This will set the AUTHOR_TESTING environment to true
447 --env-var VAR=VAL
449 -EVAR=VAL
450 -E VAR=VAL
451 --no-env-var
452 Set environment variables to set when each test is run.
453 Can be specified multiple times
455 --event-uuids
457 --uuids
458 --no-event-uuids
459 Use Test2::Plugin::UUID inside tests (default: on)
460 --fields name:details
462 --fields JSON_STRING
463 -f name:details
464 -f JSON_STRING
465 --no-fields
466 Add custom data to the harness run
467 Can be specified multiple times
469 --input ARG
471 --input=ARG
472 --no-input
473 Input string to be used as standard input for ALL tests. See also:
474 --input-file
475 --input-file ARG
477 --input-file=ARG
478 --no-input-file
479 Use the specified file as standard input to ALL tests
480 --io-events
482 --no-io-events
483 Use Test2::Plugin::IOEvents inside tests to turn all prints into
484 test2 events (default: off)
485 --link 'https://travis.work/builds/42'
487 --link 'https://jenkins.work/job/42'
488 --link 'https://buildbot.work/builders/foo/builds/42'
489 --no-link
490 Provide one or more links people can follow to see more about this
491 run.
492 Can be specified multiple times
494 --load ARG
496 --load=ARG
497 --load-module ARG
498 --load-module=ARG
499 -m ARG
500 -m=ARG
501 --no-load
502 Load a module in each test (after fork). The "import" method is not
503 called.
504 Can be specified multiple times
506 --load-import Module
508 --load-import Module=import_arg1,arg2,...
509 --loadim Module
510 --loadim Module=import_arg1,arg2,...
511 -M Module
512 -M Module=import_arg1,arg2,...
513 --no-load-import
514 Load a module in each test (after fork). Import is called.
515 Can be specified multiple times. If the same key is listed multiple
517 times the value lists will be appended together.
518 --mem-usage
520 --no-mem-usage
521 Use Test2::Plugin::MemUsage inside tests (default: on)
522 --retry ARG
524 --retry=ARG
525 -r ARG
526 -r=ARG
527 --no-retry
528 Run any jobs that failed a second time. NOTE: --retry=1 means
529 failing tests will be attempted twice!
530 --retry-isolated
532 --retry-iso
533 --no-retry-isolated
534 If true then any job retries will be done in isolation (as though
535 -j1 was set)
536 --run-id
538 --id
539 --no-run-id
540 Set a specific run-id. (Default: a UUID)
541 --test-args ARG
543 --test-args=ARG
544 --no-test-args
545 Arguments to pass in as @ARGV for all tests that are run. These can
546 be provided easier using the '::' argument separator.
547 Can be specified multiple times
549 --stream
551 --no-stream
552 Use the stream formatter (default is on)
553 --tap
555 --TAP
556 ----no-stream
557 --no-tap
558 The TAP format is lossy and clunky. Test2::Harness normally uses a
559 newer streaming format to receive test results. There are
560 old/legacy tests where this causes problems, in which case setting
561 --TAP or --no-stream can help.
562 Runner Options
564 --blib
566 -b
567 --no-blib
568 (Default: include if it exists) Include 'blib/lib' and 'blib/arch'
569 in your module path
570 --cover
572 --cover=-silent,1,+ignore,^t/,+ignore,^t2/,+ignore,^xt,+ignore,^test.pl
573 --no-cover
574 Use Devel::Cover to calculate test coverage. This disables forking.
575 If no args are specified the following are used:
576 -silent,1,+ignore,^t/,+ignore,^t2/,+ignore,^xt,+ignore,^test.pl
577 --cover-files
579 --no-cover-files
580 Use Test2::Plugin::Cover to collect coverage data for what files
581 are touched by what tests. Unlike Devel::Cover this has very little
582 performance impact (About 4% difference)
583 --event-timeout SECONDS
585 --et SECONDS
586 --no-event-timeout
587 Kill test if no output is received within timeout period. (Default:
588 60 seconds). Add the "# HARNESS-NO-TIMEOUT" comment to the top of a
589 test file to disable timeouts on a per-test basis. This prevents a
590 hung test from running forever.
591 --include ARG
593 --include=ARG
594 -I ARG
595 -I=ARG
596 --no-include
597 Add a directory to your include paths
598 Can be specified multiple times
600 --job-count ARG
602 --job-count=ARG
603 --jobs ARG
604 --jobs=ARG
605 -j ARG
606 -j=ARG
607 --no-job-count
608 Set the number of concurrent jobs to run (Default: 1)
609 Can also be set with the following environment variables:
611 "YATH_JOB_COUNT", "T2_HARNESS_JOB_COUNT", "HARNESS_JOB_COUNT"
612 --lib
614 -l
615 --no-lib
616 (Default: include if it exists) Include 'lib' in your module path
617 --post-exit-timeout SECONDS
619 --pet SECONDS
620 --no-post-exit-timeout
621 Stop waiting post-exit after the timeout period. (Default: 15
622 seconds) Some tests fork and allow the parent to exit before
623 writing all their output. If Test2::Harness detects an incomplete
624 plan after the test exits it will monitor for more events until the
625 timeout period. Add the "# HARNESS-NO-TIMEOUT" comment to the top
626 of a test file to disable timeouts on a per-test basis.
627 --preload-threshold ARG
629 --preload-threshold=ARG
630 --Pt ARG
631 --Pt=ARG
632 -W ARG
633 -W=ARG
634 --no-preload-threshold
635 Only do preload if at least N tests are going to be run. In some
636 cases a full preload takes longer than simply running the tests,
637 this lets you specify a minimum number of test jobs that will be
638 run for preload to happen. This has no effect for a persistent
639 runner. The default is 0, and it means always preload.
640 --preloads ARG
642 --preloads=ARG
643 --preload ARG
644 --preload=ARG
645 -P ARG
646 -P=ARG
647 --no-preloads
648 Preload a module before running tests
649 Can be specified multiple times
651 --switch ARG
653 --switch=ARG
654 -S ARG
655 -S=ARG
656 --no-switch
657 Pass the specified switch to perl for each test. This is not
658 compatible with preload.
659 Can be specified multiple times
661 --tlib
663 --no-tlib
664 (Default: off) Include 't/lib' in your module path
665 --unsafe-inc
667 --no-unsafe-inc
668 perl is removing '.' from @INC as a security concern. This option
669 keeps things from breaking for now.
670 Can also be set with the following environment variables:
672 "PERL_USE_UNSAFE_INC"
673 --use-fork
675 --fork
676 --no-use-fork
677 (default: on, except on windows) Normally tests are run by forking,
678 which allows for features like preloading. This will turn off the
679 behavior globally (which is not compatible with preloading). This
680 is slower, it is better to tag misbehaving tests with the '#
681 HARNESS-NO-PRELOAD' comment in their header to disable forking only
682 for those tests.
683 Can also be set with the following environment variables:
685 "!T2_NO_FORK", "T2_HARNESS_FORK", "!T2_HARNESS_NO_FORK",
686 "YATH_FORK", "!YATH_NO_FORK"
687 --use-timeout
689 --timeout
690 --no-use-timeout
691 (default: on) Enable/disable timeouts
692 Workspace Options
694 --clear
696 -C
697 --no-clear
698 Clear the work directory if it is not already empty
699 --tmp-dir ARG
701 --tmp-dir=ARG
702 --tmpdir ARG
703 --tmpdir=ARG
704 -t ARG
705 -t=ARG
706 --no-tmp-dir
707 Use a specific temp directory (Default: use system temp dir)
708 Can also be set with the following environment variables:
710 "T2_HARNESS_TEMP_DIR", "YATH_TEMP_DIR", "TMPDIR", "TEMPDIR",
711 "TMP_DIR", "TEMP_DIR"
712 --workdir ARG
714 --workdir=ARG
715 -w ARG
716 -w=ARG
717 --no-workdir
718 Set the work directory (Default: new temp directory)
719 Can also be set with the following environment variables:
721 "T2_WORKDIR", "YATH_WORKDIR"
722 YathUI Options
724 --yathui-api-key ARG
726 --yathui-api-key=ARG
727 --no-yathui-api-key
728 Yath-UI API key. This is not necessary if your Yath-UI instance is
729 set to single-user
730 --yathui-durations
732 --no-yathui-durations
733 Poll duration data from Yath-UI to help order tests efficiently
734 --yathui-grace
736 --no-yathui-grace
737 If yath cannot connect to yath-ui it normally throws an error, use
738 this to make it fail gracefully. You get a warning, but things keep
739 going.
740 --yathui-long-duration 10
742 --no-yathui-long-duration
743 Minimum duration length (seconds) before a test goes from MEDIUM to
744 LONG
745 --yathui-medium-duration 5
747 --no-yathui-medium-duration
748 Minimum duration length (seconds) before a test goes from SHORT to
749 MEDIUM
750 --yathui-mode summary
752 --yathui-mode qvf
753 --yathui-mode qvfd
754 --yathui-mode complete
755 --no-yathui-mode
756 Set the upload mode (default 'qvfd')
757 --yathui-project ARG
759 --yathui-project=ARG
760 --no-yathui-project
761 The Yath-UI project for your test results
762 --yathui-retry
764 --no-yathui-retry
765 How many times to try an operation before giving up
766 Can be specified multiple times
768 --yathui-upload
770 --no-yathui-upload
771 Upload the log to Yath-UI
772 --yathui-url http://my-yath-ui.com/...
774 --uri http://my-yath-ui.com/...
775 --no-yathui-url
776 Yath-UI url
777
779 The source code repository for Test2-Harness can be found at
780 http://github.com/Test-More/Test2-Harness/.
781
783 Chad Granum <exodist@cpan.org>
784
786 Chad Granum <exodist@cpan.org>
787
789 Copyright 2020 Chad Granum <exodist7@gmail.com>.
790 This program is free software; you can redistribute it and/or modify it
792 under the same terms as Perl itself.
793 See http://dev.perl.org/licenses/
795perl v5.32.0 2020-07-28 App::Yath::Command::test(3)