1Test2::Plugin::Cover(3)User Contributed Perl DocumentatioTnest2::Plugin::Cover(3)
2
3
4

NAME

6       Test2::Plugin::Cover - Fast and Minimal file coverage info.
7

DESCRIPTION

9       This plugin will collect minimal file coverage info, and will do so
10       with minimal performance impact.
11
12       Every time a subroutine is called this tool will do its best to find
13       the filename the subroutine was defined in, and add it to a list. Also,
14       anytime you attempt to open a file with "open()" or "sysopen()" the
15       file will be added to the list. This list will be attached to a test2
16       event just before the test exits. In most formaters the event will only
17       show up as a comment on STDOUT " # This test covered N source files. ".
18       However tools such as Test2::Harness::UI can make full use of the
19       coverage information contained in the event.
20
21   NOTE: SYSOPEN HOOK DISABLED
22       The sysopen hook is currently disabled because of an unknown segv error
23       on some platforms. I am not certain if it will be enabled again. calls
24       to subs, and calls to open are still hooked.
25

INTENDED USE CASE

27       This tool is not intended to record comprehensive coverage information,
28       if you want that use Devel::Cover.
29
30       This tool is intended to obtain and maintain lists of files that were
31       opened, or which define subs which were executed by any given test.
32       This information is useful if you want to determine what test files to
33       run after any given code change.
34
35       The collected coverage data is contained in test2 events, if you use
36       Test2::Harness aka "yath" then this data can be logged and consumed by
37       other tools such as Test2::Harness::UI.
38

PERFORMANCE

40       Unlike tools that need to record comprehensive coverage (Devel::Cover),
41       This module is only concerned about what files you open, or defined
42       subs executed directly or indirectly by a given test file. As a result
43       this module can get away with a tiny bit of XS code that only fires
44       when a subroutine is called.  Most coverage tools fire off XS for every
45       statement.
46

LIMITATIONS

48       This tool uses XS to inject a little bit of C code that runs every time
49       a subroutine is called, or every time "open()" or "sysopen()" is
50       called. This C code obtains the next op that will be run and tries to
51       pull the filename from it. "eval", XS, Moose, and other magic can
52       sometimes mask the filename, this module only makes a minimal attempt
53       to find the filename in these cases.
54
55       Originally this module only collected the filenames touched by a test.
56       Now in addition to that data it can give you seperate lists of files
57       where subs were called, and files that were touched via open().
58       Additionally the sub list includes the info about what subs were
59       called. In all of these cases it is also possible to know what
60       secgtions of your test called the subs or opened the files.
61
62   REAL EXAMPLES
63       The following data was gathered using prove to run the full Moose test
64       suite:
65
66           # Prove on its own
67           Files=478, Tests=17326, 64 wallclock secs ( 1.62 usr  0.46 sys + 57.27 cusr  4.92 csys = 64.27 CPU)
68
69           # Prove with Test2::Plugin::Cover (no coverage event)
70           Files=478, Tests=17326, 67 wallclock secs ( 1.61 usr  0.46 sys + 60.98 cusr  5.31 csys = 68.36 CPU)
71
72           # Prove with Devel::Cover
73           Files=478, Tests=17324, 963 wallclock secs ( 2.39 usr  0.58 sys + 929.12 cusr 31.98 csys = 964.07 CPU)
74
75       no coverage event - No report was generated. This was done to only
76       measure the effect of the XS that adds the data collection overhead,
77       and not the cost of the perl code that generates the report event at
78       the end of every test.
79
80       The Moose test suite was also run using Test2::Harness aka "yath"
81
82           # Without Test2::Plugin::Cover
83           Wall Time: 62.51 seconds CPU Time: 69.13 seconds (usr: 1.84s | sys: 0.08s | cusr: 60.77s | csys: 6.44s)
84
85           # With Test2::Plugin::Cover (no coverage event)
86           Wall Time: 75.46 seconds CPU Time: 82.00 seconds (usr: 1.96s | sys: 0.05s | cusr: 72.64s | csys: 7.35s)
87
88       As you can see, there is a performance hit, but it is fairly small,
89       specially compared to Devel::Cover. This is not to say anything bad
90       about Devel::Cover which is amazing, but a bad choice for the use case
91       Test2::Plugin::Cover was written to address.
92

SYNOPSIS

94   INLINE
95           use Test2::Plugin::Cover;
96
97           ...
98
99           # Arrayref of files covered so far
100           my $covered_files = Test2::Plugin::Cover->files;
101
102   COMMAND LINE
103       You can tell prove to use the module this way:
104
105           HARNESS_PERL_SWITCHES=-MTest2::Plugin::Cover prove ...
106
107       For yath:
108
109           yath test --cover-files ...
110
111   SUPPRESS REPORT
112       You can suppess the final report (only collect data, do not send the
113       Test2 event)
114
115       CLI:
116
117           HARNESS_PERL_SWITCHES=-MTest2::Plugin::Cover=no_event,1 prove ...
118
119       INLINE:
120
121           use Test2::Plugin::Cover no_event => 1;
122

KNOWING WHAT CALLED WHAT

124       If you use a system like Test::Class, Test::Class::Moose, or
125       Test2::Tools::Spec then you divide your tests into subtests (or
126       similar). In these cases it would be nice to track what subtest (or
127       equivelent) touched what files.
128
129       There are 3 methods telated to this, "set_from()", "get_from()", and
130       "clear_from()" which you can use to manage this meta-data:
131
132           subtest foo => sub {
133               # Note, this is a simple string, but the 'from' data can also be a data
134               # structure.
135               Test2::Plugin::Cover->set_from("foo");
136
137               # subroutine() from Some.pm will be recorded as having been called by 'foo'.
138               Some::subroutine();
139
140               Test2::Plugin::Cover->clear_from();
141           };
142
143       Doing this manually for all blocks is not ideal, ideally you would hook
144       your tool, such as Test::Class to call "set_from()" and "clear_from()"
145       for you.  Adding such a hook is left as an exercide to the reader, and
146       if you make one for a popular tool please upload it to cpan and add a
147       ticket or send an email for me to link to it here.
148
149       Once you have these hooks in place the data will not only show files
150       and subs that were called, but what called them.
151
152       Please see the "set_from()" documentation for details on values.
153

CLASS METHODS

155       $class->touch_source_file($file)
156       $class->touch_source_file($file, $sub)
157       $class->touch_source_file($file, \@subs)
158       $class->touch_source_file($file, $subs, $from)
159           This can be used to manually add coverage data. The first argument
160           is the source file to be "touched" by coverage. The second argument
161           is optional, and may be either a subroutine name, or an arrayref of
162           subroutine names. The third argument is optional and can be used to
163           override the default "from" value, which is normally determined for
164           you automatically.
165
166           If no subroutines are specified it will default to using '*', which
167           means the entire file is considered to be touched.
168
169       $class->touch_data_file($file)
170       $class->touch_data_file($file, $from)
171           This can be used to manually add coverage data. The first argument
172           is the file to be "touched" by coverage data. Optionally you can
173           override the 'from' value which is normally determined
174           automatically.
175
176           This is the same as calling "$class->touch_source_file($file,
177           '<>')".
178
179       $class->enable()
180       $class->disable()
181       $bool = $class->enabled()
182           Toggle or check enabled status. When disabled no coverage is
183           recorded.
184
185       $class->reload()
186           Reset filter if $0 or __FILE__ have changed. This is advanced
187           usage, you will probably never need this.
188
189       $val = $class->get_from()
190           Get the current 'from' value. The default is '*' when nothing has
191           set a from value.
192
193       $class->set_from($val)
194           Set a 'from' value. This can be anything, a string, a hashref, etc.
195           Be advised though that it will usually be serialized to JSON, so
196           make sure anything you put in it will be serializable as json.
197
198       $class->clear_from()
199           Resets the clear value to '*'
200
201       $bool = $class->was_from_modified()
202           This will return true if anything has called "set_from()" or
203           "set_from_manager". This can be reset back to false using
204           "reset_from()", which also clears the 'from' and 'from_manager'
205           values.
206
207       $class->set_from_manager($module)
208           This should be set to a module that implements the following
209           method:
210
211               sub test_parameters {
212                   my $class = shift;
213                   my ($test_file, \@from_values) = @_;
214
215                   ...
216
217                   return {
218                       # If true - run the test
219                       # If false - skip the test
220                       # If not present or undef - run the test
221                       run => $bool,
222
223                       # The following are optional
224                       argv  => [ ... ],
225                       env   => { ... },
226                       stdin => "...",
227                   };
228
229                   # OR
230                   # If true - run the test
231                   # If false - skip the test
232                   # If undef or empty list - run the test
233                   return $bool;
234               }
235
236           This will be used by Test2::Harness to determine what data needs to
237           be passed to a test given a set of 'from' values to instruct the
238           test to run the necessary parts/subtests/groups/methods/etc.
239
240           The 'argv' data will be prepended befor any other arguments
241           provided to the test.
242
243           The 'env' hashref will be merged with any other env vars needed,
244           with these taking priority.
245
246           The 'stdin' string will be used as STDIN for the test.
247
248       $arrayref = $class->files()
249       $arrayref = $class->files(root => $path)
250           This will return an arrayref of all files touched so far.
251
252           The list of files will be sorted alphabetically, and duplicates
253           will be removed.
254
255           If a root path is provided it MUST be a Path::Tiny instance. This
256           path will be used to filter out any files not under the root
257           directory.
258
259       $event = $class->report(%options)
260           This will send a Test2 event containing coverage information. It
261           will also return the event.
262
263           Options:
264
265           root => Path::Tiny->new("...")
266               Normally this is set to the current directory at module load-
267               time. This is used to filter out any source files that do not
268               live under the current directory.  This MUST be a Path::Tiny
269               instance, passing a string will not work.
270
271           verbose => $BOOL
272               If this is set to true then the comment stating how many source
273               files were touched will be printed as a diagnostics message
274               instead so that it shows up without a verbose harness.
275
276           ctx => DO NOT USE
277               This is used ONLY when the Test2::API is doing its final book-
278               keeping. Most users will never want to use this.
279
280       $class->reset_coverage()
281           This will completely clear all coverage data so far.
282
283       $class->reset_from()
284           This will clear the 'from' value, as well as reset the
285           'was_from_modified' state to false.
286
287       $class->full_reset()
288           Calls both "reset_coverage()" and "reset_from()".
289
290       $file_or_undef = $class->filter($file)
291       $file_or_undef = $class->filter($file, root => Path::Tiny->new('...'))
292           This method is used as a callback when getting the final list of
293           covered source files. The default implementation removes any files
294           that are not under the current directory which lets you focus on
295           files in the distribution you are testing. You may return a
296           modified filename if you wish to normalize it here, the default
297           implementation will turn it into a relative path.
298
299           If you provide a custom "root" parameter, it MUST be a Path::Tiny
300           instance, passing a string will not work.
301
302           A custom filter callback should look something like this:
303
304               sub {
305                   my $class = shift;
306                   my ($file, %params) = @_;
307
308                   # clean_filename() does not exist, it is just an example
309                   $file = clean_filename($file, %params);
310
311                   # should_show() does not exist, it is just an example
312                   return $file if should_show(%params);
313
314                   # Return undef or an empty list if you do NOT want to show the file.
315                   return;
316               }
317
318           Please take a look at the source to see what and how "filter()" is
319           implemented if you want all the details on how it works.
320
321       $file_or_undef = $class->extract($file)
322       $file_or_undef = $class->extract($file, %params)
323           This method is responsible for extracting a sensible filename from
324           whatever the XS found. Some magic such as "eval" or Moose can set
325           the "filename" to strings like '(eval 123)' or 'foo bar (defined at
326           FILE line LINE)' or even nonsensical strings, or text with no
327           filenames.
328
329           If a sensible file name can be extracted it will be returned,
330           otherwise undef (or an empty list) is returned.
331
332           The default implementation does not use any parameters, but they
333           are passed in for custom implementations to use.
334
335           A custom extract callback should look something like this:
336
337               sub {
338                   my $class = shift;
339                   my ($file, %params) = @_;
340
341                   # It is a valid file
342                   return $file if -e $file;
343
344                   # Do not use this, just an example
345                   return $1 if $file =~ m/($VALID_FILE_REGEX)/;
346
347                   # Cannot find a file here
348                   return;
349               }
350

SEE ALSO

352       Devel::Cover is by far the best and most complete coverage tool for
353       perl. If you need comprehensive coverage use Devel::Cover.
354       Test2::Plugin::Cover is only better for a limited use case.
355

SOURCE

357       The source code repository for Test2-Plugin-Cover can be found at
358       https://github.com/Test-More/Test2-Plugin-Cover.
359

MAINTAINERS

361       Chad Granum <exodist@cpan.org>
362

AUTHORS

364       Chad Granum <exodist@cpan.org>
365
367       Copyright 2020 Chad Granum <exodist@cpan.org>.
368
369       This program is free software; you can redistribute it and/or modify it
370       under the same terms as Perl itself.
371
372       See http://dev.perl.org/licenses/
373
374
375
376perl v5.34.0                      2021-11-05           Test2::Plugin::Cover(3)
Impressum