1TAP::Harness::ReportByDUessecrriCpotnitorni(b3u)ted PerlTADPo:c:uHmaernnteastsi:o:nReportByDescription(3)
2
6 TAP::Harness::ReportByDescription - Report TAP output by test file
7 description rather than test file name
8
10 0.01
11
13 use TAP::Harness::ReportByDescription;
14 my $harness = TAP::Harness::ReportByDescription->new();
15 $harness->aggregate_tests($aggregator, @tests);
16
18 This package subclasses TAP::Harness for the purpose of enabling a user
19 to report the TAP output for a test file by a user-provided description
20 rather than by the name of the test file itself.
21 Why would you want to do this? Three reasons come to mind.
23 • One Master Summary Rather Than Summaries for Individual
25 Subharnesses
26 Suppose that you had a make testing target that is in essence
28 nothing more than a sequential run of several smaller testing
29 targets, each of which is a separate invocation of a test harness.
30 make fulltest : test_prep \
32 compilers \
33 src \
34 oo \
35 codingstd \
36 examples
37 Other things being equal, you would get a Summary at the end of
39 each of the five targets or subharnesses. Under some
40 circumstances, you might prefer to get a single master Summary at
41 the end of the overall program.
42 • Multiple Runs of Same Tests in Different Environments
44 Suppose that you had a set of tests that you wanted to run several
46 times, each time in a slightly different environment. You could
47 write a program which executes multiple runs, writing a summary
48 after each run and then modifying the environment for the next run.
49 perl t/harness --gc-debug --runcore=bounds
51 perl t/harness --gc-debug --runcore=fast
52 perl t/harness --gc-debug --run-pbc
53 As the TAP output flowed by, you would see three instances of each
55 test:
56 t/pmc/arrayiterator.t ............................ ok
58 # ...
59 t/pmc/arrayiterator.t ............................ ok
60 # ...
61 t/pmc/arrayiterator.t ............................ ok
62 ... but you would not be able to tell from the test file's report
64 itself which harness it was a part of. Under certain circumstances
65 it would be nice to be able to differentiate among the different
66 runs:
67 bounds__t/pmc/arrayiterator.t .................... ok
69 # ...
70 fast__t/pmc/arrayiterator.t ...................... ok
71 # ...
72 pbc__t/pmc/arrayiterator.t ....................... ok
73 Here you're providing a description of each run of each test which
75 provides an observer with more information.
76 • Preparation of a Test Harness Archive
78 The ability to provide a specific description for a different run
80 of the same test becomes crucial when preparing a test harness
81 archive. Currently, CPAN distribution Test::Harness::Archive
82 stores the TAP for a particular test file in a file with the name
83 of the test file itself. If you do multiple runs of the same file
84 in different environments, a later run of a test will overwrite the
85 TAP file from an earlier run. You would therefore only be able to
86 include the TAP from the last subharness in an archive. That would
87 impede you from sharing the full results of testing via a smoke-
88 test aggregator such as Smolder.
89 In short, we need (a) a way to run multiple harnesses as if they were
91 one, (b) run the same tests through multiple harnesses and be able to
92 quickly identify which harness we were running it through, and (c)
93 store multiple versions of a file's TAP output in a test harness
94 archive.
95 Need (a) can actually be fulfilled with existing
97 TAP::Parser::Aggregator functionality. Let's build on that to meet
98 needs (b) and (c). To do that we need one package to subclass
99 TAP::Harness and one to subclass TAP::Harness::Archive.
100 TAP::Harness::ReportByDescription and
101 TAP::Harness::Archive::MultiplesHarnesses are these packages.
102
104 "new()"
105 Inherited from TAP::Harness.
106 "aggregate_tests()"
108 Replicated, along with methods called internally from this method, from
109 TAP::Harness. The only change occurs in an internal method
110 "_get_parser_args()", which now assigns the individual test's filename
111 to one variable and a user-provided description to a second variable.
112 my $test_prog = $job->filename;
114 my $spool_prog = $job->description;
115 It is the latter variable which will appear on the console and in a
117 test archive. Since this occurs within an internal method, the user
118 need make no change in how "aggregate_tests()" is called.
119
121 See "TAP::Harness::Archive::MultipleHarnesses::runtests()".
122
124 99% of the code in this module comes from TAP::Harness, written by Andy
125 Armstrong and generations of Perl QA hackers. Documentation and the
126 one small code tweak needed were written by James E Keenan.
127
129 This is free software and is released under the same terms as Perl
130 itself.
131perl v5.32.1 2021-01-2T7AP::Harness::ReportByDescription(3)