Test::Run::Core_GplArt(3pm)

1Test::Run::Core_GplArt(U3s)er Contributed Perl DocumentatTieosnt::Run::Core_GplArt(3)
2
3
4

NAME

6       Test::Run::Core_GplArt - GPL/Artistic-licensed code of Test::Run::Core.
7

SYNOPSIS

9         use Test::Run::Obj;
10
11         my $tester = Test::Run::Obj->new({'test_files' => \@test_files});
12         $tester->runtests();
13

DESCRIPTION

15       STOP! If all you want to do is write a test script, consider using
16       Test::Simple.  Test::Run::Core is the module that reads the output from
17       Test::Simple, Test::More and other modules based on Test::Builder.  You
18       don't need to know about Test::Run::Core to use those modules.
19
20       Test::Run::Core runs tests and expects output from the test in a
21       certain format.  That format is called TAP, the Test Anything Protocol.
22       It is defined in Test::Harness::TAP.
23
24       "$tester-"runtests()> runs all the testscripts named as arguments and
25       checks standard output for the expected strings in TAP format.
26
27       Test::Run::Obj is an applicative derived class of Test::Run::Core that
28       provides a programmer API for running and analyzing the output of TAP
29       files. For calling from the command line, look at Test::Run::CmdLine.
30
31   Taint mode
32       Test::Run will honor the "-T" or "-t" in the #! line on your test
33       files.  So if you begin a test with:
34
35           #!perl -T
36
37       the test will be run with taint mode on.
38
39   Failure
40       When tests fail, analyze the summary report:
41
42         t/base..............ok
43         t/nonumbers.........ok
44         t/ok................ok
45         t/test-harness......ok
46         t/waterloo..........dubious
47                 Test returned status 3 (wstat 768, 0x300)
48         DIED. FAILED tests 1, 3, 5, 7, 9, 11, 13, 15, 17, 19
49                 Failed 10/20 tests, 50.00% okay
50         Failed Test  Stat Wstat Total Fail  Failed  List of Failed
51         -----------------------------------------------------------------------
52         t/waterloo.t    3   768    20   10  50.00%  1 3 5 7 9 11 13 15 17 19
53         Failed 1/5 test scripts, 80.00% okay. 10/44 subtests failed, 77.27% okay.
54
55       Everything passed but t/waterloo.t.  It failed 10 of 20 tests and
56       exited with non-zero status indicating something dubious happened.
57
58       The columns in the summary report mean:
59
60       Failed Test
61           The test file which failed.
62
63       Stat
64           If the test exited with non-zero, this is its exit status.
65
66       Wstat
67           The wait status of the test.
68
69       Total
70           Total number of tests expected to run.
71
72       Fail
73           Number which failed, either from "not ok" or because they never
74           ran.
75
76       Failed
77           Percentage of the total tests which failed.
78
79       List of Failed
80           A list of the tests which failed.  Successive failures may be
81           abbreviated (ie. 15-20 to indicate that tests 15, 16, 17, 18, 19
82           and 20 failed).
83
84   Functions
85       Test::Run currently only has one interface function, here it is.
86
87       runtests
88             my $allok = $self->runtests();
89
90           This runs all the given @test_files and divines whether they passed
91           or failed based on their output to STDOUT (details above).  It
92           prints out each individual test which failed along with a summary
93           report and a how long it all took.
94
95           It returns true if everything was ok.  Otherwise it will "die()"
96           with one of the messages in the DIAGNOSTICS section.
97

EXPORT

99       None.
100

DIAGNOSTICS

102       "All tests successful.\nFiles=%d,  Tests=%d, %s"
103           If all tests are successful some statistics about the performance
104           are printed.
105
106       "FAILED tests %s\n\tFailed %d/%d tests, %.2f%% okay."
107           For any single script that has failing subtests statistics like the
108           above are printed.
109
110       "Test returned status %d (wstat %d)"
111           Scripts that return a non-zero exit status, both "$? >> 8" and $?
112           are printed in a message similar to the above.
113
114       "Failed 1 test, %.2f%% okay. %s"
115       "Failed %d/%d tests, %.2f%% okay. %s"
116           If not all tests were successful, the script dies with one of the
117           above messages.
118
119       "FAILED--Further testing stopped: %s"
120           If a single subtest decides that further testing will not make
121           sense, the script dies with this message.
122

ENVIRONMENT VARIABLES THAT TEST::HARNESS SETS

124       Test::Run sets these before executing the individual tests.
125
126       "HARNESS_ACTIVE"
127           This is set to a true value.  It allows the tests to determine if
128           they are being executed through the harness or by any other means.
129
130       "HARNESS_VERSION"
131           This is the version of Test::Run.
132

EXAMPLE

134       TODO: FIXME
135
136       Here's how Test::Run tests itself
137
138         $ cd ~/src/devel/Test-Harness
139         $ perl -Mblib -e 'use Test::Run qw(&runtests $verbose);
140           $verbose=0; runtests @ARGV;' t/*.t
141         Using /home/schwern/src/devel/Test-Harness/blib
142         t/base..............ok
143         t/nonumbers.........ok
144         t/ok................ok
145         t/test-harness......ok
146         All tests successful.
147         Files=4, Tests=24, 2 wallclock secs ( 0.61 cusr + 0.41 csys = 1.02 CPU)
148

TODO

156       Provide a way of running tests quietly (ie. no printing) for automated
157       validation of tests.  This will probably take the form of a version of
158       runtests() which rather than printing its output returns raw data on
159       the state of the tests.  (Partially done in Test::Run::Straps)
160
161       Document the format.
162
163       Fix HARNESS_COMPILE_TEST without breaking its core usage.
164
165       Figure a way to report test names in the failure summary.
166
167       Rework the test summary so long test names are not truncated as badly.
168       (Partially done with new skip test styles)
169
170       Add option for coverage analysis.
171
172       Trap STDERR.
173
174       Implement Straps total_results()
175
176       Remember exit code
177
178       Completely redo the print summary code.
179
180       Implement Straps callbacks.  (experimentally implemented)
181
182       Straps->analyze_file() not taint clean, don't know if it can be
183
184       Fix that damned VMS nit.
185
186       HARNESS_TODOFAIL to display TODO failures
187
188       Add a test for verbose.
189
190       Change internal list of test results to a hash.
191
192       Fix stats display when there's an overrun.
193
194       Fix so perls with spaces in the filename work.
195
196       Keeping whittling away at _run_all_tests()
197
198       Clean up how the summary is printed.  Get rid of those damned formats.
199

BUGS

201       HARNESS_COMPILE_TEST currently assumes it's run from the Perl source
202       directory.
203
204       Please use the CPAN bug ticketing system at <http://rt.cpan.org/>.  You
205       can also mail bugs, fixes and enhancements to "<bug-test-harness" at
206       "rt.cpan.org>".
207

COPYRIGHT

209       Copyright 2002-2005 by Michael G Schwern "<schwern at pobox.com>", Andy
210       Lester "<andy at petdance.com>".
211
212       This program is free software; you can redistribute it and/or modify it
213       under the same terms as Perl itself.
214
215       See <http://www.perl.com/perl/misc/Artistic.html>.
216
217
218
219perl v5.32.0                      2020-07-28         Test::Run::Core_GplArt(3)