1App::Yath(3)          User Contributed Perl Documentation         App::Yath(3)
2
3
4

NAME

6       App::Yath - Yet Another Test Harness (Test2-Harness) Command Line
7       Interface (CLI)
8

DESCRIPTION

10       This is the primary documentation for "yath", App::Yath,
11       Test2::Harness.
12
13       The canonical source of up-to-date command options are the help output
14       when using "$ yath help" and "$ yath help COMMAND".
15
16       This document is mainly an overview of "yath" usage and common recipes.
17
18       App::Yath is an alternative to App::Prove, and Test2::Harness is an
19       alternative to Test::Harness. It is not designed to replace
20       Test::Harness/prove. Test2::Harness is designed to take full advantage
21       of the rich data Test2 can provide. Test2::Harness is also able to use
22       non-core modules and provide more functionality than prove can achieve
23       with its restrictions.
24

PLATFORM SUPPORT

26       Test2::Harness/App::Yath is is focused on unix-like platforms. Most
27       development happens on linux, but bsd, macos, etc should work fine as
28       well.
29
30       Patches are welcome for any/all platforms, but the primary author (Chad
31       'Exodist' Granum) does not directly develop against non-unix platforms.
32
33   WINDOWS
34       Currently windows is not supported, and it is known that the package
35       will not install on windows. Patches are be welcome, and it would be
36       great if someone wanted to take on the windows-support role, but it is
37       not a primary goal for the project.
38

OVERVIEW

40       To use Test2::Harness, you use the "yath" command. Yath will find the
41       tests (or use the ones you specify) and run them. As it runs, it will
42       output diagnostic information such as failures. At the end, yath will
43       print a summary of the test run.
44
45       "yath" can be thought of as a more powerful alternative to "prove"
46       (Test::Harness)
47

RECIPES

49       These are common recipes for using "yath".
50
51   RUN PROJECT TESTS
52           $ yath
53
54       Simply running yath with no arguments means "Run all tests for the
55       current project". Yath will look for tests in "./t", "./t2", and
56       "./test.pl" and run any which are found.
57
58       Normally this implies the "test" command but will instead imply the
59       "run" command if a persistent test runner is detected.
60
61   PRELOAD MODULES
62       Yath has the ability to preload modules. Yath normally forks to start
63       new tests, so preloading can reduce the time spent loading modules over
64       and over in each test.
65
66       Note that some tests may depend on certain modules not being loaded. In
67       these cases you can add the "# HARNESS-NO-PRELOAD" directive to the top
68       of the test files that cannot use preload.
69
70       SIMPLE PRELOAD
71
72       Any module can be preloaded:
73
74           $ yath -PMoose
75
76       You can preload as many modules as you want:
77
78           $ yath -PList::Util -PScalar::Util
79
80       COMPLEX PRELOAD
81
82       If your preload is a subclass of Test2::Harness::Runner::Preload then
83       more complex preload behavior is possible. See those docs for more
84       info.
85
86   LOGGING
87       RECORDING A LOG
88
89       You can turn on logging with a flag. The filename of the log will be
90       printed at the end.
91
92           $ yath -L
93           ...
94           Wrote log file: test-logs/2017-09-12~22:44:34~1505281474~25709.jsonl
95
96       The event log can be quite large. It can be compressed with bzip2.
97
98           $ yath -B
99           ...
100           Wrote log file: test-logs/2017-09-12~22:44:34~1505281474~25709.jsonl.bz2
101
102       gzip compression is also supported.
103
104           $ yath -G
105           ...
106           Wrote log file: test-logs/2017-09-12~22:44:34~1505281474~25709.jsonl.gz
107
108       "-B" and "-G" both imply "-L".
109
110       REPLAYING FROM A LOG
111
112       You can replay a test run from a log file:
113
114           $ yath test-logs/2017-09-12~22:44:34~1505281474~25709.jsonl.bz2
115
116       This will be significantly faster than the initial run as no tests are
117       actually being executed. All events are simply read from the log, and
118       processed by the harness.
119
120       You can change display options and limit rendering/processing to
121       specific test jobs from the run:
122
123           $ yath test-logs/2017-09-12~22:44:34~1505281474~25709.jsonl.bz2 -v [TEST UUID(S)]
124
125       Note: This is done using the "$ yath replay ..." command. The "replay"
126       command is implied if the first argument is a log file.
127
128   PER-TEST TIMING DATA
129       The "-T" option will cause each test file to report how long it took to
130       run.
131
132           $ yath -T
133
134           ( PASSED )  job  1    t/yath_script.t
135           (  TIME  )  job  1    Startup: 0.07692s | Events: 0.01170s | Cleanup: 0.00190s | Total: 0.09052s
136
137   PERSISTENT RUNNER
138       yath supports starting a yath session that waits for tests to run. This
139       is very useful when combined with preload.
140
141       STARTING
142
143       This starts the server. Many options available to the 'test' command
144       will work here but not all. See "$ yath help start" for more info.
145
146           $ yath start
147
148       RUNNING
149
150       This will run tests using the persistent runner. By default, it will
151       search for tests just like the 'test' command. Many options available
152       to the "test" command will work for this as well. See "$ yath help run"
153       for more details.
154
155           $ yath run
156
157       STOPPING
158
159       Stopping a persistent runner is easy.
160
161           $ yath stop
162
163       INFORMATIONAL
164
165       The "which" command will tell you which persistent runner will be used.
166       Yath searches for the persistent runner in the current directory, then
167       searches in parent directories until it either hits the root directory,
168       or finds the persistent runner tracking file.
169
170           $ yath which
171
172       The "watch" command will tail the runner's log files.
173
174           $ yath watch
175
176       PRELOAD + PERSISTENT RUNNER
177
178       You can use preloads with the "yath start" command. In this case, yath
179       will track all the modules pulled in during preload. If any of them
180       change, the server will reload itself to bring in the changes. Further,
181       modified modules will be blacklisted so that they are not preloaded on
182       subsequent reloads. This behavior is useful if you are actively working
183       on a module that is normally preloaded.
184
185   MAKING YOUR PROJECT ALWAYS USE YATH
186           $ yath init
187
188       The above command will create "test.pl". "test.pl" is automatically run
189       by most build utils, in which case only the exit value matters. The
190       generated "test.pl" will run "yath" and execute all tests in the "./t"
191       and/or "./t2" directories. Tests in "./t" will ALSO be run by prove but
192       tests in "./t2" will only be run by yath.
193
194   PROJECT-SPECIFIC YATH CONFIG
195       You can write a ".yath.rc" file. The file format is very simple. Create
196       a "[COMMAND]" section to start the configuration for a command and then
197       provide any options normally allowed by it. When "yath" is run inside
198       your project, it will use the config specified in the rc file, unless
199       overridden by command line options.
200
201       Note: You can also add pre-command options by placing them at the top
202       of your config file BEFORE any "[cmd]" markers.
203
204       Comments start with a semi-colon.
205
206       Example .yath.rc:
207
208           -pFoo ; Load the 'foo' plugin before dealing with commands.
209
210           [test]
211           -B ;Always write a bzip2-compressed log
212
213           [start]
214           -PMoose ;Always preload Moose with a persistent runner
215
216       This file is normally committed into the project's repo.
217
218       SPECIAL PATH PSEUDO-FUNCTIONS
219
220       Sometimes you want to specify files relative to the .yath.rc so that
221       the config option works from any subdirectory of the project. Other
222       times you may wish to use a shell expansion. Sometimes you want both!
223
224       rel(path/to/file)
225               -I rel(path/to/extra_lib)
226               -I=rel(path/to/extra_lib)
227
228           This will take the path to ".yath.rc" and prefix it to the path
229           inside rel(...). If for example you have "/project/.yath.rc" then
230           the path would become "/project/path/to/extra_lib".
231
232       glob(path/*/file)
233               --default-search glob(subprojects/*/t)
234               --default-search=glob(subprojects/*/t)
235
236           This will add a "--default-search $_" for every item found in the
237           glob. This uses the perl builtin function glob() under the hood.
238
239       relglob(path/*/file)
240               --default-search relglob(subprojects/*/t)
241               --default-search=relglob(subprojects/*/t)
242
243           Same as glob() except paths are relative to the ".yath.rc" file.
244
245   PROJECT-SPECIFIC YATH CONFIG USER OVERRIDES
246       You can add a ".yath.user.rc" file. Format is the same as the regular
247       ".yath.rc" file. This file will be read in addition to the regular
248       config file. Directives in this file will come AFTER the directives in
249       the primary config so it may be used to override config.
250
251       This file should not normally be committed to the project repo.
252
253   HARNESS DIRECTIVES INSIDE TESTS
254       "yath" will recognise a number of directive comments placed near the
255       top of test files. These directives should be placed after the "#!"
256       line but before any real code.
257
258       Real code is defined as any line that does not start with use, require,
259       BEGIN, package, or #
260
261       good example 1
262               #!/usr/bin/perl
263               # HARNESS-NO-FORK
264
265               ...
266
267       good example 2
268               #!/usr/bin/perl
269               use strict;
270               use warnings;
271
272               # HARNESS-NO-FORK
273
274               ...
275
276       bad example 1
277               #!/usr/bin/perl
278
279               # blah
280
281               # HARNESS-NO-FORK
282
283               ...
284
285       bad example 2
286               #!/usr/bin/perl
287
288               print "hi\n";
289
290               # HARNESS-NO-FORK
291
292               ...
293
294       HARNESS-NO-PRELOAD
295
296           #!/usr/bin/perl
297           # HARNESS-NO-PRELOAD
298
299       Use this if your test will fail when modules are preloaded. This will
300       tell yath to start a new perl process to run the script instead of
301       forking with preloaded modules.
302
303       Currently this implies HARNESS-NO-FORK, but that may not always be the
304       case.
305
306       HARNESS-NO-FORK
307
308           #!/usr/bin/perl
309           # HARNESS-NO-FORK
310
311       Use this if your test file cannot run in a forked process, but instead
312       must be run directly with a new perl process.
313
314       This implies HARNESS-NO-PRELOAD.
315
316       HARNESS-NO-STREAM
317
318       "yath" usually uses the Test2::Formatter::Stream formatter instead of
319       TAP.  Some tests depend on using a TAP formatter. This option will make
320       "yath" use Test2::Formatter::TAP or Test::Builder::Formatter.
321
322       HARNESS-NO-IO-EVENTS
323
324       "yath" can be configured to use the Test2::Plugin::IOEvents plugin.
325       This plugin replaces STDERR and STDOUT in your test with tied handles
326       that fire off proper Test2::Event's when they are printed to. Most of
327       the time this is not an issue, but any fancy tests or modules which do
328       anything with STDERR or STDOUT other than print may have really messy
329       errors.
330
331       Note: This plugin is disabled by default, so you only need this
332       directive if you enable it globally but need to turn it back off for
333       select tests.
334
335       HARNESS-NO-TIMEOUT
336
337       "yath" will usually kill a test if no events occur within a timeout
338       (default 60 seconds). You can add this directive to tests that are
339       expected to trip the timeout, but should be allowed to continue.
340
341       NOTE: you usually are doing the wrong thing if you need to set this.
342       See: "HARNESS-TIMEOUT-EVENT".
343
344       HARNESS-TIMEOUT-EVENT 60
345
346       "yath" can be told to alter the default event timeout from 60 seconds
347       to another value. This is the recommended alternative to HARNESS-NO-
348       TIMEOUT
349
350       HARNESS-TIMEOUT-POSTEXIT 15
351
352       "yath" can be told to alter the default POSTEXIT timeout from 15
353       seconds to another value.
354
355       Sometimes a test will fork producing output in the child while the
356       parent is allowed to exit. In these cases we cannot rely on the
357       original process exit to tell us when a test is complete. In cases
358       where we have an exit, and partial output (assertions with no final
359       plan, or a plan that has not been completed) we wait for a timeout
360       period to see if any additional events come into
361
362       HARNESS-DURATION-LONG
363
364       This lets you tell "yath" that the test file is long-running. This is
365       primarily used when concurrency is turned on in order to run longer
366       tests earlier, and concurrently with shorter ones. There is also a
367       "yath" option to skip all long tests.
368
369       This duration is set automatically if HARNESS-NO-TIMEOUT is set.
370
371       HARNESS-DURATION-MEDIUM
372
373       This lets you tell "yath" that the test is medium.
374
375       This is the default duration.
376
377       HARNESS-DURATION-SHORT
378
379       This lets you tell "yath" That the test is short.
380
381       HARNESS-CATEGORY-ISOLATION
382
383       This lets you tell "yath" that the test cannot be run concurrently with
384       other tests. Yath will hold off and run these tests one at a time after
385       all other tests.
386
387       HARNESS-CATEGORY-IMMISCIBLE
388
389       This lets you tell "yath" that the test cannot be run concurrently with
390       other tests of this class. This is helpful when you have multiple tests
391       which would otherwise have to be run sequentially at the end of the
392       run.
393
394       Yath prioritizes running these tests above HARNESS-CATEGORY-LONG.
395
396       HARNESS-CATEGORY-GENERAL
397
398       This is the default category.
399
400       HARNESS-CONFLICTS-XXX
401
402       This lets you tell "yath" that no other test of type XXX can be run at
403       the same time as this one. You are able to set multiple conflict types
404       and "yath" will honor them.
405
406       XXX can be replaced with any type of your choosing.
407
408       NOTE: This directive does not alter the category of your test. You are
409       free to mark the test with LONG or MEDIUM in addition to this marker.
410
411       HARNESS-JOB-SLOTS 2
412
413       HARNESS-JOB-SLOTS 1 10
414
415       Specify a range of job slots needed for the test to run. If set to a
416       single value then the test will only run if it can have the specified
417       number of slots.  If given a range the test will require at least the
418       lower number of slots, and use up to the maximum number of slots.
419
420       Example with multiple lines.
421               #!/usr/bin/perl
422               # DASH and space are split the same way.
423               # HARNESS-CONFLICTS-DAEMON
424               # HARNESS-CONFLICTS  MYSQL
425
426               ...
427
428       Or on a single line.
429               #!/usr/bin/perl
430               # HARNESS-CONFLICTS DAEMON MYSQL
431
432               ...
433
434       HARNESS-RETRY-n
435
436       This lets you specify a number (minimum n=1) of retries on test failure
437       for a specific test. HARNESS-RETRY-1 means a failing test will be run
438       twice and is equivalent to HARNESS-RETRY.
439
440       HARNESS-NO-RETRY
441
442       Use this to avoid this test being retried regardless of your retry
443       settings.
444

MODULE DOCS

446       This section documents the App::Yath module itself.
447
448   SYNOPSIS
449       In practice you should never need to write your own yath script, or
450       construct an App::Yath instance, or even access themain instance when
451       yath is running.  However some aspects of doing so are documented here
452       for completeness.
453
454       A minimum yath script looks like this:
455
456           BEGIN {
457               package App::Yath:Script;
458
459               require Time::HiRes;
460               require App::Yath;
461               require Test2::Harness::Settings;
462
463               my $settings = Test2::Harness::Settings->new(
464                   harness => {
465                       orig_argv       => [@ARGV],
466                       orig_inc        => [@INC],
467                       script          => __FILE__,
468                       start           => Time::HiRes::time(),
469                       version         => $App::Yath::VERSION,
470                   },
471               );
472
473               my $app = App::Yath->new(
474                   argv    => \@ARGV,
475                   config  => {},
476                   settings => $settings,
477               );
478
479               $app->generate_run_sub('App::Yath::Script::run');
480           }
481
482           exit(App::Yath::Script::run());
483
484       It is important that most logic live in a BEGIN block. This is so that
485       goto::file can be used post-fork to execute a test script.
486
487       The actual yath script is significantly more complicated with the
488       following behaviors:
489
490       pre-process essential arguments such as -D and no-scan-plugins
491       re-exec with a different yath script if in developer mode and a local
492       copy is found
493       Parse the yath-rc config files
494       gather and store essential startup information
495
496   METHODS
497       App::Yath does not provide many methods to use externally.
498
499       $app->generate_run_sub($symbol_name)
500           This tells App::Yath to generate a subroutine at the specified
501           symbol name which can be run and be expected to return an exit
502           value.
503
504       $lib_path = $app->app_path()
505           Get the include directory App::Yath was loaded from.
506

SOURCE

508       The source code repository for Test2-Harness can be found at
509       http://github.com/Test-More/Test2-Harness/.
510

MAINTAINERS

512       Chad Granum <exodist@cpan.org>
513

AUTHORS

515       Chad Granum <exodist@cpan.org>
516
518       Copyright 2020 Chad Granum <exodist7@gmail.com>.
519
520       This program is free software; you can redistribute it and/or modify it
521       under the same terms as Perl itself.
522
523       See http://dev.perl.org/licenses/
524
525
526
527perl v5.36.1                      2023-10-04                      App::Yath(3)
Impressum