1App::Yath(3)          User Contributed Perl Documentation         App::Yath(3)
2
3
4

NAME

6       App::Yath - Yet Another Test Harness (Test2-Harness) Command Line
7       Interface (CLI)
8

DESCRIPTION

10       This is the primary documentation for "yath", App::Yath,
11       Test2::Harness.
12
13       The canonical source of up-to-date command options are the help output
14       when using "$ yath help" and "$ yath help COMMAND".
15
16       This document is mainly an overview of "yath" usage and common recipes.
17
18       App::Yath is an alternative to App::Prove, and Test2::Harness is an
19       alternative to Test::Harness. It is not designed to replace
20       Test::Harness/prove. Test2::Harness is designed to take full advantage
21       of the rich data Test2 can provide. Test2::Harness is also able to use
22       non-core modules and provide more functionality than prove can achieve
23       with its restrictions.
24

PLATFORM SUPPORT

26       Test2::Harness/App::Yath is is focused on unix-like platforms. Most
27       development happens on linux, but bsd, macos, etc should work fine as
28       well.
29
30       Patches are welcome for any/all platforms, but the primary author (Chad
31       'Exodist' Granum) does not directly develop against non-unix platforms.
32
33   WINDOWS
34       Currently windows is not supported, and it is known that the package
35       will not install on windows. Patches are be welcome, and it would be
36       great if someone wanted to take on the windows-support role, but it is
37       not a primary goal for the project.
38

OVERVIEW

40       To use Test2::Harness, you use the "yath" command. Yath will find the
41       tests (or use the ones you specify) and run them. As it runs, it will
42       output diagnostic information such as failures. At the end, yath will
43       print a summary of the test run.
44
45       "yath" can be thought of as a more powerful alternative to "prove"
46       (Test::Harness)
47

RECIPES

49       These are common recipes for using "yath".
50
51   RUN PROJECT TESTS
52           $ yath
53
54       Simply running yath with no arguments means "Run all tests for the
55       current project". Yath will look for tests in "./t", "./t2", and
56       "./test.pl" and run any which are found.
57
58       Normally this implies the "test" command but will instead imply the
59       "run" command if a persistent test runner is detected.
60
61   PRELOAD MODULES
62       Yath has the ability to preload modules. Yath normally forks to start
63       new tests, so preloading can reduce the time spent loading modules over
64       and over in each test.
65
66       Note that some tests may depend on certain modules not being loaded. In
67       these cases you can add the "# HARNESS-NO-PRELOAD" directive to the top
68       of the test files that cannot use preload.
69
70       SIMPLE PRELOAD
71
72       Any module can be preloaded:
73
74           $ yath -PMoose
75
76       You can preload as many modules as you want:
77
78           $ yath -PList::Util -PScalar::Util
79
80       COMPLEX PRELOAD
81
82       If your preload is a subclass of Test2::Harness::Runner::Preload then
83       more complex preload behavior is possible. See those docs for more
84       info.
85
86   LOGGING
87       RECORDING A LOG
88
89       You can turn on logging with a flag. The filename of the log will be
90       printed at the end.
91
92           $ yath -L
93           ...
94           Wrote log file: test-logs/2017-09-12~22:44:34~1505281474~25709.jsonl
95
96       The event log can be quite large. It can be compressed with bzip2.
97
98           $ yath -B
99           ...
100           Wrote log file: test-logs/2017-09-12~22:44:34~1505281474~25709.jsonl.bz2
101
102       gzip compression is also supported.
103
104           $ yath -G
105           ...
106           Wrote log file: test-logs/2017-09-12~22:44:34~1505281474~25709.jsonl.gz
107
108       "-B" and "-G" both imply "-L".
109
110       REPLAYING FROM A LOG
111
112       You can replay a test run from a log file:
113
114           $ yath test-logs/2017-09-12~22:44:34~1505281474~25709.jsonl.bz2
115
116       This will be significantly faster than the initial run as no tests are
117       actually being executed. All events are simply read from the log, and
118       processed by the harness.
119
120       You can change display options and limit rendering/processing to
121       specific test jobs from the run:
122
123           $ yath test-logs/2017-09-12~22:44:34~1505281474~25709.jsonl.bz2 -v [TEST UUID(S)]
124
125       Note: This is done using the "$ yath replay ..." command. The "replay"
126       command is implied if the first argument is a log file.
127
128   PER-TEST TIMING DATA
129       The "-T" option will cause each test file to report how long it took to
130       run.
131
132           $ yath -T
133
134           ( PASSED )  job  1    t/yath_script.t
135           (  TIME  )  job  1    Startup: 0.07692s | Events: 0.01170s | Cleanup: 0.00190s | Total: 0.09052s
136
137   PERSISTENT RUNNER
138       yath supports starting a yath session that waits for tests to run. This
139       is very useful when combined with preload.
140
141       STARTING
142
143       This starts the server. Many options available to the 'test' command
144       will work here but not all. See "$ yath help start" for more info.
145
146           $ yath start
147
148       RUNNING
149
150       This will run tests using the persistent runner. By default, it will
151       search for tests just like the 'test' command. Many options available
152       to the "test" command will work for this as well. See "$ yath help run"
153       for more details.
154
155           $ yath run
156
157       STOPPING
158
159       Stopping a persistent runner is easy.
160
161           $ yath stop
162
163       INFORMATIONAL
164
165       The "which" command will tell you which persistent runner will be used.
166       Yath searches for the persistent runner in the current directory, then
167       searches in parent directories until it either hits the root directory,
168       or finds the persistent runner tracking file.
169
170           $ yath which
171
172       The "watch" command will tail the runner's log files.
173
174           $ yath watch
175
176       PRELOAD + PERSISTENT RUNNER
177
178       You can use preloads with the "yath start" command. In this case, yath
179       will track all the modules pulled in during preload. If any of them
180       change, the server will reload itself to bring in the changes. Further,
181       modified modules will be blacklisted so that they are not preloaded on
182       subsequent reloads. This behavior is useful if you are actively working
183       on a module that is normally preloaded.
184
185   MAKING YOUR PROJECT ALWAYS USE YATH
186           $ yath init
187
188       The above command will create "test.pl". "test.pl" is automatically run
189       by most build utils, in which case only the exit value matters. The
190       generated "test.pl" will run "yath" and execute all tests in the "./t"
191       and/or "./t2" directories. Tests in "./t" will ALSO be run by prove but
192       tests in "./t2" will only be run by yath.
193
194   PROJECT-SPECIFIC YATH CONFIG
195       You can write a ".yath.rc" file. The file format is very simple. Create
196       a "[COMMAND]" section to start the configuration for a command and then
197       provide any options normally allowed by it. When "yath" is run inside
198       your project, it will use the config specified in the rc file, unless
199       overridden by command line options.
200
201       Note: You can also add pre-command options by placing them at the top
202       of your config file BEFORE any "[cmd]" markers.
203
204       Comments start with a semi-colon.
205
206       Example .yath.rc:
207
208           -pFoo ; Load the 'foo' plugin before dealing with commands.
209
210           [test]
211           -B ;Always write a bzip2-compressed log
212
213           [start]
214           -PMoose ;Always preload Moose with a persistent runner
215
216       This file is normally committed into the project's repo.
217
218   PROJECT-SPECIFIC YATH CONFIG USER OVERRIDES
219       You can add a ".yath.user.rc" file. Format is the same as the regular
220       ".yath.rc" file. This file will be read in addition to the regular
221       config file. Directives in this file will come AFTER the directives in
222       the primary config so it may be used to override config.
223
224       This file should not normally be committed to the project repo.
225
226   HARNESS DIRECTIVES INSIDE TESTS
227       "yath" will recognise a number of directive comments placed near the
228       top of test files. These directives should be placed after the "#!"
229       line but before any real code.
230
231       Real code is defined as any line that does not start with use, require,
232       BEGIN, package, or #
233
234       good example 1
235               #!/usr/bin/perl
236               # HARNESS-NO-FORK
237
238               ...
239
240       good example 2
241               #!/usr/bin/perl
242               use strict;
243               use warnings;
244
245               # HARNESS-NO-FORK
246
247               ...
248
249       bad example 1
250               #!/usr/bin/perl
251
252               # blah
253
254               # HARNESS-NO-FORK
255
256               ...
257
258       bad example 2
259               #!/usr/bin/perl
260
261               print "hi\n";
262
263               # HARNESS-NO-FORK
264
265               ...
266
267       HARNESS-NO-PRELOAD
268
269           #!/usr/bin/perl
270           # HARNESS-NO-PRELOAD
271
272       Use this if your test will fail when modules are preloaded. This will
273       tell yath to start a new perl process to run the script instead of
274       forking with preloaded modules.
275
276       Currently this implies HARNESS-NO-FORK, but that may not always be the
277       case.
278
279       HARNESS-NO-FORK
280
281           #!/usr/bin/perl
282           # HARNESS-NO-FORK
283
284       Use this if your test file cannot run in a forked process, but instead
285       must be run directly with a new perl process.
286
287       This implies HARNESS-NO-PRELOAD.
288
289       HARNESS-NO-STREAM
290
291       "yath" usually uses the Test2::Formatter::Stream formatter instead of
292       TAP.  Some tests depend on using a TAP formatter. This option will make
293       "yath" use Test2::Formatter::TAP or Test::Builder::Formatter.
294
295       HARNESS-NO-IO-EVENTS
296
297       "yath" usually uses the Test2::Plugin::IOEvents plugin. This plugin
298       replaces STDERR and STDOUT in your test with tied handles that fire off
299       proper Test2::Event's when they are printed to. Most of the time this
300       is not an issue, but any fancy tests or modules which do anything with
301       STDERR or STDOUT other than print may have really messy errors.
302
303       This directive will disable the plugin on a per-test basis.
304       Alternatively you can use the "--no-io-events" option when running yath
305       to disable it globally for your test suite.
306
307       HARNESS-NO-TIMEOUT
308
309       "yath" will usually kill a test if no events occur within a timeout
310       (default 60 seconds). You can add this directive to tests that are
311       expected to trip the timeout, but should be allowed to continue.
312
313       NOTE: you usually are doing the wrong thing if you need to set this.
314       See: "HARNESS-TIMEOUT-EVENT".
315
316       HARNESS-TIMEOUT-EVENT 60
317
318       "yath" can be told to alter the default event timeout from 60 seconds
319       to another value. This is the recommended alternative to HARNESS-NO-
320       TIMEOUT
321
322       HARNESS-TIMEOUT-POSTEXIT 15
323
324       "yath" can be told to alter the default POSTEXIT timeout from 15
325       seconds to another value.
326
327       Sometimes a test will fork producing output in the child while the
328       parent is allowed to exit. In these cases we cannot rely on the
329       original process exit to tell us when a test is complete. In cases
330       where we have an exit, and partial output (assertions with no final
331       plan, or a plan that has not been completed) we wait for a timeout
332       period to see if any additional events come into
333
334       HARNESS-DURATION-LONG
335
336       This lets you tell "yath" that the test file is long-running. This is
337       primarily used when concurrency is turned on in order to run longer
338       tests earlier, and concurrently with shorter ones. There is also a
339       "yath" option to skip all long tests.
340
341       This duration is set automatically if HARNESS-NO-TIMEOUT is set.
342
343       HARNESS-DURATION-MEDIUM
344
345       This lets you tell "yath" that the test is medium.
346
347       This is the default duration.
348
349       HARNESS-DURATION-SHORT
350
351       This lets you tell "yath" That the test is short.
352
353       HARNESS-CATEGORY-ISOLATION
354
355       This lets you tell "yath" that the test cannot be run concurrently with
356       other tests. Yath will hold off and run these tests one at a time after
357       all other tests.
358
359       HARNESS-CATEGORY-IMMISCIBLE
360
361       This lets you tell "yath" that the test cannot be run concurrently with
362       other tests of this class. This is helpful when you have multiple tests
363       which would otherwise have to be run sequentially at the end of the
364       run.
365
366       Yath prioritizes running these tests above HARNESS-CATEGORY-LONG.
367
368       HARNESS-CATEGORY-GENERAL
369
370       This is the default category.
371
372       HARNESS-CONFLICTS-XXX
373
374       This lets you tell "yath" that no other test of type XXX can be run at
375       the same time as this one. You are able to set multiple conflict types
376       and "yath" will honor them.
377
378       XXX can be replaced with any type of your choosing.
379
380       NOTE: This directive does not alter the category of your test. You are
381       free to mark the test with LONG or MEDIUM in addition to this marker.
382
383       Example with multiple lines.
384               #!/usr/bin/perl
385               # DASH and space are split the same way.
386               # HARNESS-CONFLICTS-DAEMON
387               # HARNESS-CONFLICTS  MYSQL
388
389               ...
390
391       Or on a single line.
392               #!/usr/bin/perl
393               # HARNESS-CONFLICTS DAEMON MYSQL
394
395               ...
396
397       HARNESS-RETRY-n
398
399       This lets you specify a number (minimum n=1) of retries on test failure
400       for a specific test. HARNESS-RETRY-1 means a failing test will be run
401       twice and is equivalent to HARNESS-RETRY.
402
403       HARNESS-NO-RETRY
404
405       Use this to avoid this test being retried regardless of your retry
406       settings.
407

MODULE DOCS

409       This section documents the App::Yath module itself.
410
411   SYNOPSIS
412       In practice you should never need to write your own yath script, or
413       construct an App::Yath instance, or even access themain instance when
414       yath is running.  However some aspects of doing so are documented here
415       for completeness.
416
417       A minimum yath script looks like this:
418
419           BEGIN {
420               package App::Yath:Script;
421
422               require Time::HiRes;
423               require App::Yath;
424               require Test2::Harness::Settings;
425
426               my $settings = Test2::Harness::Settings->new(
427                   harness => {
428                       orig_argv       => [@ARGV],
429                       orig_inc        => [@INC],
430                       script          => __FILE__,
431                       start           => Time::HiRes::time(),
432                       version         => $App::Yath::VERSION,
433                   },
434               );
435
436               my $app = App::Yath->new(
437                   argv    => \@ARGV,
438                   config  => {},
439                   settings => $settings,
440               );
441
442               $app->generate_run_sub('App::Yath::Script::run');
443           }
444
445           exit(App::Yath::Script::run());
446
447       It is important that most logic live in a BEGIN block. This is so that
448       goto::file can be used post-fork to execute a test script.
449
450       The actual yath script is significantly more complicated with the
451       following behaviors:
452
453       pre-process essential arguments such as -D and no-scan-plugins
454       re-exec with a different yath script if in developer mode and a local
455       copy is found
456       Parse the yath-rc config files
457       gather and store essential startup information
458
459   METHODS
460       App::Yath does not provide many methods to use externally.
461
462       $app->generate_run_sub($symbol_name)
463           This tells App::Yath to generate a subroutine at the specified
464           symbol name which can be run and be expected to return an exit
465           value.
466
467       $lib_path = $app->app_path()
468           Get the include directory App::Yath was loaded from.
469

SOURCE

471       The source code repository for Test2-Harness can be found at
472       http://github.com/Test-More/Test2-Harness/.
473

MAINTAINERS

475       Chad Granum <exodist@cpan.org>
476

AUTHORS

478       Chad Granum <exodist@cpan.org>
479

COPYRIGHT

481       Copyright 2020 Chad Granum <exodist7@gmail.com>.
482
483       This program is free software; you can redistribute it and/or modify it
484       under the same terms as Perl itself.
485
486       See http://dev.perl.org/licenses/
487
488
489
490perl v5.32.0                      2020-07-28                      App::Yath(3)