1Test2::Plugin::Cover(3)User Contributed Perl DocumentatioTnest2::Plugin::Cover(3)
2
6 Test2::Plugin::Cover - Fast and Minimal file coverage info.
7
9 This plugin will collect minimal file coverage info, and will do so
10 with minimal performance impact.
11 Every time a subroutine is called this tool will do its best to find
13 the filename the subroutine was defined in, and add it to a list. Also,
14 anytime you attempt to open a file with "open()" or "sysopen()" the
15 file will be added to the list. This list will be attached to a test2
16 event just before the test exits. In most formaters the event will only
17 show up as a comment on STDOUT " # This test covered N source files. ".
18 However tools such as Test2::Harness::UI can make full use of the
19 coverage information contained in the event.
20 NOTE: SYSOPEN HOOK DISABLED
22 The sysopen hook is currently disabled because of an unknown segv error
23 on some platforms. I am not certain if it will be enabled again. calls
24 to subs, and calls to open are still hooked.
25
27 This tool is not intended to record comprehensive coverage information,
28 if you want that use Devel::Cover.
29 This tool is intended to obtain and maintain lists of files that were
31 opened, or which define subs which were executed by any given test.
32 This information is useful if you want to determine what test files to
33 run after any given code change.
34 The collected coverage data is contained in test2 events, if you use
36 Test2::Harness aka "yath" then this data can be logged and consumed by
37 other tools such as Test2::Harness::UI.
38
40 Unlike tools that need to record comprehensive coverage (Devel::Cover),
41 This module is only concerned about what files you open, or defined
42 subs executed directly or indirectly by a given test file. As a result
43 this module can get away with a tiny bit of XS code that only fires
44 when a subroutine is called. Most coverage tools fire off XS for every
45 statement.
46
48 This tool uses XS to inject a little bit of C code that runs every time
49 a subroutine is called, or every time "open()" or "sysopen()" is
50 called. This C code obtains the next op that will be run and tries to
51 pull the filename from it. "eval", XS, Moose, and other magic can
52 sometimes mask the filename, this module only makes a minimal attempt
53 to find the filename in these cases.
54 Originally this module only collected the filenames touched by a test.
56 Now in addition to that data it can give you seperate lists of files
57 where subs were called, and files that were touched via open().
58 Additionally the sub list includes the info about what subs were
59 called. In all of these cases it is also possible to know what
60 secgtions of your test called the subs or opened the files.
61 REAL EXAMPLES
63 The following data was gathered using prove to run the full Moose test
64 suite:
65 # Prove on its own
67 Files=478, Tests=17326, 64 wallclock secs ( 1.62 usr 0.46 sys + 57.27 cusr 4.92 csys = 64.27 CPU)
68 # Prove with Test2::Plugin::Cover (no coverage event)
70 Files=478, Tests=17326, 67 wallclock secs ( 1.61 usr 0.46 sys + 60.98 cusr 5.31 csys = 68.36 CPU)
71 # Prove with Devel::Cover
73 Files=478, Tests=17324, 963 wallclock secs ( 2.39 usr 0.58 sys + 929.12 cusr 31.98 csys = 964.07 CPU)
74 no coverage event - No report was generated. This was done to only
76 measure the effect of the XS that adds the data collection overhead,
77 and not the cost of the perl code that generates the report event at
78 the end of every test.
79 The Moose test suite was also run using Test2::Harness aka "yath"
81 # Without Test2::Plugin::Cover
83 Wall Time: 62.51 seconds CPU Time: 69.13 seconds (usr: 1.84s | sys: 0.08s | cusr: 60.77s | csys: 6.44s)
84 # With Test2::Plugin::Cover (no coverage event)
86 Wall Time: 75.46 seconds CPU Time: 82.00 seconds (usr: 1.96s | sys: 0.05s | cusr: 72.64s | csys: 7.35s)
87 As you can see, there is a performance hit, but it is fairly small,
89 specially compared to Devel::Cover. This is not to say anything bad
90 about Devel::Cover which is amazing, but a bad choice for the use case
91 Test2::Plugin::Cover was written to address.
92
94 INLINE
95 use Test2::Plugin::Cover;
96 ...
98 # Arrayref of files covered so far
100 my $covered_files = Test2::Plugin::Cover->files;
101 COMMAND LINE
103 You can tell prove to use the module this way:
104 HARNESS_PERL_SWITCHES=-MTest2::Plugin::Cover prove ...
106 For yath:
108 yath test --cover-files ...
110 SUPPRESS REPORT
112 You can suppess the final report (only collect data, do not send the
113 Test2 event)
114 CLI:
116 HARNESS_PERL_SWITCHES=-MTest2::Plugin::Cover=no_event,1 prove ...
118 INLINE:
120 use Test2::Plugin::Cover no_event => 1;
122
124 If you use a system like Test::Class, Test::Class::Moose, or
125 Test2::Tools::Spec then you divide your tests into subtests (or
126 similar). In these cases it would be nice to track what subtest (or
127 equivelent) touched what files.
128 There are 3 methods telated to this, "set_from()", "get_from()", and
130 "clear_from()" which you can use to manage this meta-data:
131 subtest foo => sub {
133 # Note, this is a simple string, but the 'from' data can also be a data
134 # structure.
135 Test2::Plugin::Cover->set_from("foo");
136 # subroutine() from Some.pm will be recorded as having been called by 'foo'.
138 Some::subroutine();
139 Test2::Plugin::Cover->clear_from();
141 };
142 Doing this manually for all blocks is not ideal, ideally you would hook
144 your tool, such as Test::Class to call "set_from()" and "clear_from()"
145 for you. Adding such a hook is left as an exercide to the reader, and
146 if you make one for a popular tool please upload it to cpan and add a
147 ticket or send an email for me to link to it here.
148 Once you have these hooks in place the data will not only show files
150 and subs that were called, but what called them.
151 Please see the "set_from()" documentation for details on values.
153
155 $class->touch_source_file($file)
156 $class->touch_source_file($file, $sub)
157 $class->touch_source_file($file, \@subs)
158 $class->touch_source_file($file, $subs, $from)
159 This can be used to manually add coverage data. The first argument
160 is the source file to be "touched" by coverage. The second argument
161 is optional, and may be either a subroutine name, or an arrayref of
162 subroutine names. The third argument is optional and can be used to
163 override the default "from" value, which is normally determined for
164 you automatically.
165 If no subroutines are specified it will default to using '*', which
167 means the entire file is considered to be touched.
168 $class->touch_data_file($file)
170 $class->touch_data_file($file, $from)
171 This can be used to manually add coverage data. The first argument
172 is the file to be "touched" by coverage data. Optionally you can
173 override the 'from' value which is normally determined
174 automatically.
175 This is the same as calling "$class->touch_source_file($file,
177 '<>')".
178 $class->enable()
180 $class->disable()
181 $bool = $class->enabled()
182 Toggle or check enabled status. When disabled no coverage is
183 recorded.
184 $class->reload()
186 Reset filter if $0 or __FILE__ have changed. This is advanced
187 usage, you will probably never need this.
188 $val = $class->get_from()
190 Get the current 'from' value. The default is '*' when nothing has
191 set a from value.
192 $class->set_from($val)
194 Set a 'from' value. This can be anything, a string, a hashref, etc.
195 Be advised though that it will usually be serialized to JSON, so
196 make sure anything you put in it will be serializable as json.
197 $class->clear_from()
199 Resets the clear value to '*'
200 $bool = $class->was_from_modified()
202 This will return true if anything has called "set_from()" or
203 "set_from_manager". This can be reset back to false using
204 "reset_from()", which also clears the 'from' and 'from_manager'
205 values.
206 $class->set_from_manager($module)
208 This should be set to a module that implements the following
209 method:
210 sub test_parameters {
212 my $class = shift;
213 my ($test_file, \@from_values) = @_;
214 ...
216 return {
218 # If true - run the test
219 # If false - skip the test
220 # If not present or undef - run the test
221 run => $bool,
222 # The following are optional
224 argv => [ ... ],
225 env => { ... },
226 stdin => "...",
227 };
228 # OR
230 # If true - run the test
231 # If false - skip the test
232 # If undef or empty list - run the test
233 return $bool;
234 }
235 This will be used by Test2::Harness to determine what data needs to
237 be passed to a test given a set of 'from' values to instruct the
238 test to run the necessary parts/subtests/groups/methods/etc.
239 The 'argv' data will be prepended befor any other arguments
241 provided to the test.
242 The 'env' hashref will be merged with any other env vars needed,
244 with these taking priority.
245 The 'stdin' string will be used as STDIN for the test.
247 $arrayref = $class->files()
249 $arrayref = $class->files(root => $path)
250 This will return an arrayref of all files touched so far.
251 The list of files will be sorted alphabetically, and duplicates
253 will be removed.
254 If a root path is provided it MUST be a Path::Tiny instance. This
256 path will be used to filter out any files not under the root
257 directory.
258 $event = $class->report(%options)
260 This will send a Test2 event containing coverage information. It
261 will also return the event.
262 Options:
264 root => Path::Tiny->new("...")
266 Normally this is set to the current directory at module load-
267 time. This is used to filter out any source files that do not
268 live under the current directory. This MUST be a Path::Tiny
269 instance, passing a string will not work.
270 verbose => $BOOL
272 If this is set to true then the comment stating how many source
273 files were touched will be printed as a diagnostics message
274 instead so that it shows up without a verbose harness.
275 ctx => DO NOT USE
277 This is used ONLY when the Test2::API is doing its final book-
278 keeping. Most users will never want to use this.
279 $class->reset_coverage()
281 This will completely clear all coverage data so far.
282 $class->reset_from()
284 This will clear the 'from' value, as well as reset the
285 'was_from_modified' state to false.
286 $class->full_reset()
288 Calls both "reset_coverage()" and "reset_from()".
289 $file_or_undef = $class->filter($file)
291 $file_or_undef = $class->filter($file, root => Path::Tiny->new('...'))
292 This method is used as a callback when getting the final list of
293 covered source files. The default implementation removes any files
294 that are not under the current directory which lets you focus on
295 files in the distribution you are testing. You may return a
296 modified filename if you wish to normalize it here, the default
297 implementation will turn it into a relative path.
298 If you provide a custom "root" parameter, it MUST be a Path::Tiny
300 instance, passing a string will not work.
301 A custom filter callback should look something like this:
303 sub {
305 my $class = shift;
306 my ($file, %params) = @_;
307 # clean_filename() does not exist, it is just an example
309 $file = clean_filename($file, %params);
310 # should_show() does not exist, it is just an example
312 return $file if should_show(%params);
313 # Return undef or an empty list if you do NOT want to show the file.
315 return;
316 }
317 Please take a look at the source to see what and how "filter()" is
319 implemented if you want all the details on how it works.
320 $file_or_undef = $class->extract($file)
322 $file_or_undef = $class->extract($file, %params)
323 This method is responsible for extracting a sensible filename from
324 whatever the XS found. Some magic such as "eval" or Moose can set
325 the "filename" to strings like '(eval 123)' or 'foo bar (defined at
326 FILE line LINE)' or even nonsensical strings, or text with no
327 filenames.
328 If a sensible file name can be extracted it will be returned,
330 otherwise undef (or an empty list) is returned.
331 The default implementation does not use any parameters, but they
333 are passed in for custom implementations to use.
334 A custom extract callback should look something like this:
336 sub {
338 my $class = shift;
339 my ($file, %params) = @_;
340 # It is a valid file
342 return $file if -e $file;
343 # Do not use this, just an example
345 return $1 if $file =~ m/($VALID_FILE_REGEX)/;
346 # Cannot find a file here
348 return;
349 }
350
352 Devel::Cover is by far the best and most complete coverage tool for
353 perl. If you need comprehensive coverage use Devel::Cover.
354 Test2::Plugin::Cover is only better for a limited use case.
355
357 The source code repository for Test2-Plugin-Cover can be found at
358 https://github.com/Test-More/Test2-Plugin-Cover.
359
361 Chad Granum <exodist@cpan.org>
362
364 Chad Granum <exodist@cpan.org>
365
367 Copyright 2020 Chad Granum <exodist@cpan.org>.
368 This program is free software; you can redistribute it and/or modify it
370 under the same terms as Perl itself.
371 See http://dev.perl.org/licenses/
373perl v5.34.0 2021-11-05 Test2::Plugin::Cover(3)