1App::Yath(3) User Contributed Perl Documentation App::Yath(3)
2
6 App::Yath - Yet Another Test Harness (Test2-Harness) Command Line
7 Interface (CLI)
8
10 This is the primary documentation for "yath", App::Yath,
11 Test2::Harness.
12 The canonical source of up-to-date command options are the help output
14 when using "$ yath help" and "$ yath help COMMAND".
15 This document is mainly an overview of "yath" usage and common recipes.
17 App::Yath is an alternative to App::Prove, and Test2::Harness is an
19 alternative to Test::Harness. It is not designed to replace
20 Test::Harness/prove. Test2::Harness is designed to take full advantage
21 of the rich data Test2 can provide. Test2::Harness is also able to use
22 non-core modules and provide more functionality than prove can achieve
23 with its restrictions.
24
26 Test2::Harness/App::Yath is is focused on unix-like platforms. Most
27 development happens on linux, but bsd, macos, etc should work fine as
28 well.
29 Patches are welcome for any/all platforms, but the primary author (Chad
31 'Exodist' Granum) does not directly develop against non-unix platforms.
32 WINDOWS
34 Currently windows is not supported, and it is known that the package
35 will not install on windows. Patches are be welcome, and it would be
36 great if someone wanted to take on the windows-support role, but it is
37 not a primary goal for the project.
38
40 To use Test2::Harness, you use the "yath" command. Yath will find the
41 tests (or use the ones you specify) and run them. As it runs, it will
42 output diagnostic information such as failures. At the end, yath will
43 print a summary of the test run.
44 "yath" can be thought of as a more powerful alternative to "prove"
46 (Test::Harness)
47
49 These are common recipes for using "yath".
50 RUN PROJECT TESTS
52 $ yath
53 Simply running yath with no arguments means "Run all tests for the
55 current project". Yath will look for tests in "./t", "./t2", and
56 "./test.pl" and run any which are found.
57 Normally this implies the "test" command but will instead imply the
59 "run" command if a persistent test runner is detected.
60 PRELOAD MODULES
62 Yath has the ability to preload modules. Yath normally forks to start
63 new tests, so preloading can reduce the time spent loading modules over
64 and over in each test.
65 Note that some tests may depend on certain modules not being loaded. In
67 these cases you can add the "# HARNESS-NO-PRELOAD" directive to the top
68 of the test files that cannot use preload.
69 SIMPLE PRELOAD
71 Any module can be preloaded:
73 $ yath -PMoose
75 You can preload as many modules as you want:
77 $ yath -PList::Util -PScalar::Util
79 COMPLEX PRELOAD
81 If your preload is a subclass of Test2::Harness::Runner::Preload then
83 more complex preload behavior is possible. See those docs for more
84 info.
85 LOGGING
87 RECORDING A LOG
88 You can turn on logging with a flag. The filename of the log will be
90 printed at the end.
91 $ yath -L
93 ...
94 Wrote log file: test-logs/2017-09-12~22:44:34~1505281474~25709.jsonl
95 The event log can be quite large. It can be compressed with bzip2.
97 $ yath -B
99 ...
100 Wrote log file: test-logs/2017-09-12~22:44:34~1505281474~25709.jsonl.bz2
101 gzip compression is also supported.
103 $ yath -G
105 ...
106 Wrote log file: test-logs/2017-09-12~22:44:34~1505281474~25709.jsonl.gz
107 "-B" and "-G" both imply "-L".
109 REPLAYING FROM A LOG
111 You can replay a test run from a log file:
113 $ yath test-logs/2017-09-12~22:44:34~1505281474~25709.jsonl.bz2
115 This will be significantly faster than the initial run as no tests are
117 actually being executed. All events are simply read from the log, and
118 processed by the harness.
119 You can change display options and limit rendering/processing to
121 specific test jobs from the run:
122 $ yath test-logs/2017-09-12~22:44:34~1505281474~25709.jsonl.bz2 -v [TEST UUID(S)]
124 Note: This is done using the "$ yath replay ..." command. The "replay"
126 command is implied if the first argument is a log file.
127 PER-TEST TIMING DATA
129 The "-T" option will cause each test file to report how long it took to
130 run.
131 $ yath -T
133 ( PASSED ) job 1 t/yath_script.t
135 ( TIME ) job 1 Startup: 0.07692s | Events: 0.01170s | Cleanup: 0.00190s | Total: 0.09052s
136 PERSISTENT RUNNER
138 yath supports starting a yath session that waits for tests to run. This
139 is very useful when combined with preload.
140 STARTING
142 This starts the server. Many options available to the 'test' command
144 will work here but not all. See "$ yath help start" for more info.
145 $ yath start
147 RUNNING
149 This will run tests using the persistent runner. By default, it will
151 search for tests just like the 'test' command. Many options available
152 to the "test" command will work for this as well. See "$ yath help run"
153 for more details.
154 $ yath run
156 STOPPING
158 Stopping a persistent runner is easy.
160 $ yath stop
162 INFORMATIONAL
164 The "which" command will tell you which persistent runner will be used.
166 Yath searches for the persistent runner in the current directory, then
167 searches in parent directories until it either hits the root directory,
168 or finds the persistent runner tracking file.
169 $ yath which
171 The "watch" command will tail the runner's log files.
173 $ yath watch
175 PRELOAD + PERSISTENT RUNNER
177 You can use preloads with the "yath start" command. In this case, yath
179 will track all the modules pulled in during preload. If any of them
180 change, the server will reload itself to bring in the changes. Further,
181 modified modules will be blacklisted so that they are not preloaded on
182 subsequent reloads. This behavior is useful if you are actively working
183 on a module that is normally preloaded.
184 MAKING YOUR PROJECT ALWAYS USE YATH
186 $ yath init
187 The above command will create "test.pl". "test.pl" is automatically run
189 by most build utils, in which case only the exit value matters. The
190 generated "test.pl" will run "yath" and execute all tests in the "./t"
191 and/or "./t2" directories. Tests in "./t" will ALSO be run by prove but
192 tests in "./t2" will only be run by yath.
193 PROJECT-SPECIFIC YATH CONFIG
195 You can write a ".yath.rc" file. The file format is very simple. Create
196 a "[COMMAND]" section to start the configuration for a command and then
197 provide any options normally allowed by it. When "yath" is run inside
198 your project, it will use the config specified in the rc file, unless
199 overridden by command line options.
200 Note: You can also add pre-command options by placing them at the top
202 of your config file BEFORE any "[cmd]" markers.
203 Comments start with a semi-colon.
205 Example .yath.rc:
207 -pFoo ; Load the 'foo' plugin before dealing with commands.
209 [test]
211 -B ;Always write a bzip2-compressed log
212 [start]
214 -PMoose ;Always preload Moose with a persistent runner
215 This file is normally committed into the project's repo.
217 SPECIAL PATH PSEUDO-FUNCTIONS
219 Sometimes you want to specify files relative to the .yath.rc so that
221 the config option works from any subdirectory of the project. Other
222 times you may wish to use a shell expansion. Sometimes you want both!
223 rel(path/to/file)
225 -I rel(path/to/extra_lib)
226 -I=rel(path/to/extra_lib)
227 This will take the path to ".yath.rc" and prefix it to the path
229 inside rel(...). If for example you have "/project/.yath.rc" then
230 the path would become "/project/path/to/extra_lib".
231 glob(path/*/file)
233 --default-search glob(subprojects/*/t)
234 --default-search=glob(subprojects/*/t)
235 This will add a "--default-search $_" for every item found in the
237 glob. This uses the perl builtin function glob() under the hood.
238 relglob(path/*/file)
240 --default-search relglob(subprojects/*/t)
241 --default-search=relglob(subprojects/*/t)
242 Same as glob() except paths are relative to the ".yath.rc" file.
244 PROJECT-SPECIFIC YATH CONFIG USER OVERRIDES
246 You can add a ".yath.user.rc" file. Format is the same as the regular
247 ".yath.rc" file. This file will be read in addition to the regular
248 config file. Directives in this file will come AFTER the directives in
249 the primary config so it may be used to override config.
250 This file should not normally be committed to the project repo.
252 HARNESS DIRECTIVES INSIDE TESTS
254 "yath" will recognise a number of directive comments placed near the
255 top of test files. These directives should be placed after the "#!"
256 line but before any real code.
257 Real code is defined as any line that does not start with use, require,
259 BEGIN, package, or #
260 good example 1
262 #!/usr/bin/perl
263 # HARNESS-NO-FORK
264 ...
266 good example 2
268 #!/usr/bin/perl
269 use strict;
270 use warnings;
271 # HARNESS-NO-FORK
273 ...
275 bad example 1
277 #!/usr/bin/perl
278 # blah
280 # HARNESS-NO-FORK
282 ...
284 bad example 2
286 #!/usr/bin/perl
287 print "hi\n";
289 # HARNESS-NO-FORK
291 ...
293 HARNESS-NO-PRELOAD
295 #!/usr/bin/perl
297 # HARNESS-NO-PRELOAD
298 Use this if your test will fail when modules are preloaded. This will
300 tell yath to start a new perl process to run the script instead of
301 forking with preloaded modules.
302 Currently this implies HARNESS-NO-FORK, but that may not always be the
304 case.
305 HARNESS-NO-FORK
307 #!/usr/bin/perl
309 # HARNESS-NO-FORK
310 Use this if your test file cannot run in a forked process, but instead
312 must be run directly with a new perl process.
313 This implies HARNESS-NO-PRELOAD.
315 HARNESS-NO-STREAM
317 "yath" usually uses the Test2::Formatter::Stream formatter instead of
319 TAP. Some tests depend on using a TAP formatter. This option will make
320 "yath" use Test2::Formatter::TAP or Test::Builder::Formatter.
321 HARNESS-NO-IO-EVENTS
323 "yath" can be configured to use the Test2::Plugin::IOEvents plugin.
325 This plugin replaces STDERR and STDOUT in your test with tied handles
326 that fire off proper Test2::Event's when they are printed to. Most of
327 the time this is not an issue, but any fancy tests or modules which do
328 anything with STDERR or STDOUT other than print may have really messy
329 errors.
330 Note: This plugin is disabled by default, so you only need this
332 directive if you enable it globally but need to turn it back off for
333 select tests.
334 HARNESS-NO-TIMEOUT
336 "yath" will usually kill a test if no events occur within a timeout
338 (default 60 seconds). You can add this directive to tests that are
339 expected to trip the timeout, but should be allowed to continue.
340 NOTE: you usually are doing the wrong thing if you need to set this.
342 See: "HARNESS-TIMEOUT-EVENT".
343 HARNESS-TIMEOUT-EVENT 60
345 "yath" can be told to alter the default event timeout from 60 seconds
347 to another value. This is the recommended alternative to HARNESS-NO-
348 TIMEOUT
349 HARNESS-TIMEOUT-POSTEXIT 15
351 "yath" can be told to alter the default POSTEXIT timeout from 15
353 seconds to another value.
354 Sometimes a test will fork producing output in the child while the
356 parent is allowed to exit. In these cases we cannot rely on the
357 original process exit to tell us when a test is complete. In cases
358 where we have an exit, and partial output (assertions with no final
359 plan, or a plan that has not been completed) we wait for a timeout
360 period to see if any additional events come into
361 HARNESS-DURATION-LONG
363 This lets you tell "yath" that the test file is long-running. This is
365 primarily used when concurrency is turned on in order to run longer
366 tests earlier, and concurrently with shorter ones. There is also a
367 "yath" option to skip all long tests.
368 This duration is set automatically if HARNESS-NO-TIMEOUT is set.
370 HARNESS-DURATION-MEDIUM
372 This lets you tell "yath" that the test is medium.
374 This is the default duration.
376 HARNESS-DURATION-SHORT
378 This lets you tell "yath" That the test is short.
380 HARNESS-CATEGORY-ISOLATION
382 This lets you tell "yath" that the test cannot be run concurrently with
384 other tests. Yath will hold off and run these tests one at a time after
385 all other tests.
386 HARNESS-CATEGORY-IMMISCIBLE
388 This lets you tell "yath" that the test cannot be run concurrently with
390 other tests of this class. This is helpful when you have multiple tests
391 which would otherwise have to be run sequentially at the end of the
392 run.
393 Yath prioritizes running these tests above HARNESS-CATEGORY-LONG.
395 HARNESS-CATEGORY-GENERAL
397 This is the default category.
399 HARNESS-CONFLICTS-XXX
401 This lets you tell "yath" that no other test of type XXX can be run at
403 the same time as this one. You are able to set multiple conflict types
404 and "yath" will honor them.
405 XXX can be replaced with any type of your choosing.
407 NOTE: This directive does not alter the category of your test. You are
409 free to mark the test with LONG or MEDIUM in addition to this marker.
410 HARNESS-JOB-SLOTS 2
412 HARNESS-JOB-SLOTS 1 10
414 Specify a range of job slots needed for the test to run. If set to a
416 single value then the test will only run if it can have the specified
417 number of slots. If given a range the test will require at least the
418 lower number of slots, and use up to the maximum number of slots.
419 Example with multiple lines.
421 #!/usr/bin/perl
422 # DASH and space are split the same way.
423 # HARNESS-CONFLICTS-DAEMON
424 # HARNESS-CONFLICTS MYSQL
425 ...
427 Or on a single line.
429 #!/usr/bin/perl
430 # HARNESS-CONFLICTS DAEMON MYSQL
431 ...
433 HARNESS-RETRY-n
435 This lets you specify a number (minimum n=1) of retries on test failure
437 for a specific test. HARNESS-RETRY-1 means a failing test will be run
438 twice and is equivalent to HARNESS-RETRY.
439 HARNESS-NO-RETRY
441 Use this to avoid this test being retried regardless of your retry
443 settings.
444
446 This section documents the App::Yath module itself.
447 SYNOPSIS
449 In practice you should never need to write your own yath script, or
450 construct an App::Yath instance, or even access themain instance when
451 yath is running. However some aspects of doing so are documented here
452 for completeness.
453 A minimum yath script looks like this:
455 BEGIN {
457 package App::Yath:Script;
458 require Time::HiRes;
460 require App::Yath;
461 require Test2::Harness::Settings;
462 my $settings = Test2::Harness::Settings->new(
464 harness => {
465 orig_argv => [@ARGV],
466 orig_inc => [@INC],
467 script => __FILE__,
468 start => Time::HiRes::time(),
469 version => $App::Yath::VERSION,
470 },
471 );
472 my $app = App::Yath->new(
474 argv => \@ARGV,
475 config => {},
476 settings => $settings,
477 );
478 $app->generate_run_sub('App::Yath::Script::run');
480 }
481 exit(App::Yath::Script::run());
483 It is important that most logic live in a BEGIN block. This is so that
485 goto::file can be used post-fork to execute a test script.
486 The actual yath script is significantly more complicated with the
488 following behaviors:
489 pre-process essential arguments such as -D and no-scan-plugins
491 re-exec with a different yath script if in developer mode and a local
492 copy is found
493 Parse the yath-rc config files
494 gather and store essential startup information
495 METHODS
497 App::Yath does not provide many methods to use externally.
498 $app->generate_run_sub($symbol_name)
500 This tells App::Yath to generate a subroutine at the specified
501 symbol name which can be run and be expected to return an exit
502 value.
503 $lib_path = $app->app_path()
505 Get the include directory App::Yath was loaded from.
506
508 The source code repository for Test2-Harness can be found at
509 http://github.com/Test-More/Test2-Harness/.
510
512 Chad Granum <exodist@cpan.org>
513
515 Chad Granum <exodist@cpan.org>
516
518 Copyright 2020 Chad Granum <exodist7@gmail.com>.
519 This program is free software; you can redistribute it and/or modify it
521 under the same terms as Perl itself.
522 See http://dev.perl.org/licenses/
524perl v5.36.1 2023-10-04 App::Yath(3)