1Devel::Cover(3) User Contributed Perl Documentation Devel::Cover(3)
2
6 Devel::Cover - Code coverage metrics for Perl
7
9 version 1.36
10
12 To get coverage for an uninstalled module:
13 cover -test
15 or
17 cover -delete
19 HARNESS_PERL_SWITCHES=-MDevel::Cover make test
20 cover
21 To get coverage for an uninstalled module which uses Module::Build
23 (0.26 or later):
24 ./Build testcover
26 If the module does not use the t/*.t framework:
28 PERL5OPT=-MDevel::Cover make test
30 If you want to get coverage for a program:
32 perl -MDevel::Cover yourprog args
34 cover
35 To alter default values:
37 perl -MDevel::Cover=-db,cover_db,-coverage,statement,time yourprog args
39
41 This module provides code coverage metrics for Perl. Code coverage
42 metrics describe how thoroughly tests exercise code. By using
43 Devel::Cover you can discover areas of code not exercised by your tests
44 and determine which tests to create to increase coverage. Code
45 coverage can be considered an indirect measure of quality.
46 Although it is still being developed, Devel::Cover is now quite stable
48 and provides many of the features to be expected in a useful coverage
49 tool.
50 Statement, branch, condition, subroutine, and pod coverage information
52 is reported. Statement and subroutine coverage data should be
53 accurate. Branch and condition coverage data should be mostly accurate
54 too, although not always what one might initially expect. Pod coverage
55 comes from Pod::Coverage. If Pod::Coverage::CountParents is available
56 it will be used instead. Coverage data for other criteria are not yet
57 collected.
58 The cover program can be used to generate coverage reports.
60 Devel::Cover ships with a number of reports including various types of
61 HTML output, textual reports, a report to display missing coverage in
62 the same format as compilation errors and a report to display coverage
63 information within the Vim editor.
64 It is possible to add annotations to reports, for example you can add a
66 column to an HTML report showing who last changed a line, as determined
67 by git blame. Some annotation modules are shipped with Devel::Cover
68 and you can easily create your own.
69 The gcov2perl program can be used to convert gcov files to
71 "Devel::Cover" databases. This allows you to display your C or XS code
72 coverage together with your Perl coverage, or to use any of the
73 Devel::Cover reports to display your C coverage data.
74 Code coverage data are collected by replacing perl ops with functions
76 which count how many times the ops are executed. These data are then
77 mapped back to reality using the B compiler modules. There is also a
78 statement profiling facility which should not be relied on. For proper
79 profiling use Devel::NYTProf. Previous versions of Devel::Cover
80 collected coverage data by replacing perl's runops function. It is
81 still possible to switch to that mode of operation, but this now gets
82 little testing and will probably be removed soon. You probably don't
83 care about any of this.
84 The most appropriate mailing list on which to discuss this module would
86 be perl-qa. See <http://lists.perl.org/list/perl-qa.html>.
87 The Devel::Cover repository can be found at
89 <http://github.com/pjcj/Devel--Cover>. This is also where problems
90 should be reported.
91
93 REQUIREMENTS
94 • Perl 5.10.0 or greater.
95 The latest version of Devel::Cover on which Perl 5.8 was supported
97 was 1.23. Perl versions 5.6.1 and 5.6.2 were not supported after
98 version 1.22. Perl versions 5.6.0 and earlier were never
99 supported. Using Devel::Cover with Perl 5.8.7 was always
100 problematic and frequently lead to crashes.
101 Different versions of perl may give slightly different results due
103 to changes in the op tree.
104 • The ability to compile XS extensions.
106 This means a working C compiler and make program at least. If you
108 built perl from source you will have these already and they will be
109 used automatically. If your perl was built in some other way, for
110 example you may have installed it using your Operating System's
111 packaging mechanism, you will need to ensure that the appropriate
112 tools are installed.
113 • Storable and Digest::MD5
115 Both are in the core in Perl 5.8.0 and above.
117 REQUIRED MODULES
119 • B::Debug
120 This was core before Perl 5.30.0.
122 OPTIONAL MODULES
124 • Template, and either PPI::HTML or Perl::Tidy
125 Needed if you want syntax highlighted HTML reports.
127 • Pod::Coverage (0.06 or above) or Pod::Coverage::CountParents
129 One is needed if you want Pod coverage. If
131 Pod::Coverage::CountParents is installed, it is preferred.
132 • Test::More
134 Required if you want to run Devel::Cover's own tests.
136 • Test::Differences
138 Needed if the tests fail and you would like nice output telling you
140 why.
141 • Template and Parallel::Iterator
143 Needed if you want to run cpancover.
145 • JSON::MaybeXS
147 JSON is used to store the coverage database if it is available.
149 JSON::MaybeXS will select the best JSON backend installed.
150 Use with mod_perl
152 By adding "use Devel::Cover;" to your mod_perl startup script, you
153 should be able to collect coverage information when running under
154 mod_perl. You can also add any options you need at this point. I
155 would suggest adding this as early as possible in your startup script
156 in order to collect as much coverage information as possible.
157 Alternatively, add -MDevel::Cover to the parameters for mod_perl. In
159 this example, Devel::Cover will be operating in silent mode.
160 PerlSwitches -MDevel::Cover=-silent,1
162
164 -blib - "use blib" and ignore files matching \bt/ (default true
165 if blib directory exists, false otherwise)
166 -coverage criterion - Turn on coverage for the specified criterion. Criteria
167 include statement, branch, condition, path, subroutine,
168 pod, time, all and none (default all available)
169 -db cover_db - Store results in coverage db (default ./cover_db)
170 -dir path - Directory in which coverage will be collected (default
171 cwd)
172 -ignore RE - Set regular expressions for files to ignore (default
173 "/Devel/Cover\b")
174 +ignore RE - Append to regular expressions of files to ignore
175 -inc path - Set prefixes of files to include (default @INC)
176 +inc path - Append to prefixes of files to include
177 -loose_perms val - Use loose permissions on all files and directories in
178 the coverage db so that code changing EUID can still
179 write coverage information (default off)
180 -merge val - Merge databases, for multiple test benches (default on)
181 -select RE - Set regular expressions of files to select (default none)
182 +select RE - Append to regular expressions of files to select
183 -silent val - Don't print informational messages (default off)
184 -subs_only val - Only cover code in subroutine bodies (default off)
185 -replace_ops val - Use op replacing rather than runops (default on)
186 -summary val - Print summary information if val is true (default on)
187 More on Coverage Options
189 You can specify options to some coverage criteria. At the moment only
190 pod coverage takes any options. These are the parameters which are
191 passed into the Pod::Coverage constructor. The extra options are
192 separated by dashes, and you may specify as many as you wish. For
193 example, to specify that all subroutines containing xx are private,
194 call Devel::Cover with the option -coverage,pod-also_private-xx.
195 Or, to ignore all files in "t/lib" as well as files ending in "Foo.pm":
197 cover -test -silent -ignore ^t/lib/,Foo.pm$
199 Note that "-ignore" replaces any default ignore regexes. To preserve
201 any ignore regexes which have already been set, use "+ignore":
202 cover -test -silent +ignore ^t/lib/,Foo.pm$
204
206 You may select the files for which you want to collect coverage data
207 using the select, ignore and inc options. The system uses the
208 following procedure to decide whether a file will be included in
209 coverage reports:
210 • If the file matches a RE given as a select option, it will be
212 included
213 • Otherwise, if it matches a RE given as an ignore option, it won't
215 be included
216 • Otherwise, if it is in one of the inc directories, it won't be
218 included
219 • Otherwise, it will be included
221 You may add to the REs to select by using +select, or you may reset the
223 selections using -select. The same principle applies to the REs to
224 ignore.
225 The inc directories are initially populated with the contents of perl's
227 @INC array. You may reset these directories using -inc, or add to them
228 using +inc.
229 Although these options take regular expressions, you should not enclose
231 the RE within // or any other quoting characters.
232 The options -coverage, [+-]select, [+-]ignore and [+-]inc can be
234 specified multiple times, but they can also take multiple comma
235 separated arguments. In any case you should not add a space after the
236 comma, unless you want the argument to start with that literal space.
237
239 Sometimes you have code which is uncoverable for some reason. Perhaps
240 it is an else clause that cannot be reached, or a check for an error
241 condition that should never happen. You can tell Devel::Cover that
242 certain criteria are uncoverable and then they are not counted as
243 errors when they are not exercised. In fact, they are counted as
244 errors if they are exercised.
245 This feature should only be used as something of a last resort.
247 Ideally you would find some way of exercising all your code. But if
248 you have analysed your code and determined that you are not going to be
249 able to exercise it, it may be better to record that fact in some
250 formal fashion and stop Devel::Cover complaining about it, so that real
251 problems are not lost in the noise.
252 If you have uncoverable criteria I suggest not using the default HTML
254 report (with uses html_minimal at the moment) because this sometimes
255 shows uncoverable points as uncovered. Instead, you should use the
256 html_basic report for HTML output which should behave correctly in this
257 regard.
258 There are two ways to specify a construct as uncoverable, one invasive
260 and one non-invasive.
261 Invasive specification
263 You can use special comments in your code to specify uncoverable
264 criteria. Comments are of the form:
265 # uncoverable <criterion> [details]
267 The keyword "uncoverable" must be the first text in the comment. It
269 should be followed by the name of the coverage criterion which is
270 uncoverable. There may then be further information depending on the
271 nature of the uncoverable construct.
272 Statements
274 The "uncoverable" comment should appear on either the same line as the
276 statement, or on the line before it:
277 $impossible++; # uncoverable statement
279 # uncoverable statement
280 it_has_all_gone_horribly_wrong();
281 If there are multiple statements (or any other criterion) on a line you
283 can specify which statement is uncoverable by using the "count"
284 attribute, count:n, which indicates that the uncoverable statement is
285 the nth statement on the line.
286 # uncoverable statement count:1
288 # uncoverable statement count:2
289 cannot_run_this(); or_this();
290 Branches
292 The "uncoverable" comment should specify whether the "true" or "false"
294 branch is uncoverable.
295 # uncoverable branch true
297 if (pi == 3)
298 Both branches may be uncoverable:
300 # uncoverable branch true
302 # uncoverable branch false
303 if (impossible_thing_happened_one_way()) {
304 handle_it_one_way(); # uncoverable statement
305 } else {
306 handle_it_another_way(); # uncoverable statement
307 }
308 If there is an elsif in the branch then it can be addressed as the
310 second branch on the line by using the "count" attribute. Further
311 elsifs are the third and fourth "count" value, and so on:
312 # uncoverable branch false count:2
314 if ($thing == 1) {
315 handle_thing_being_one();
316 } elsif ($thing == 2) {
317 handle_thing_being_tow();
318 } else {
319 die "thing can only be one or two, not $thing"; # uncoverable statement
320 }
321 Conditions
323 Because of the way in which Perl short-circuits boolean operations,
325 there are three ways in which such conditionals can be uncoverable. In
326 the case of " $x && $y" for example, the left operator may never be
327 true, the right operator may never be true, and the whole operation may
328 never be false. These conditions may be modelled thus:
329 # uncoverable branch true
331 # uncoverable condition left
332 # uncoverable condition false
333 if ($x && !$y) {
334 $x++; # uncoverable statement
335 }
336 # uncoverable branch true
338 # uncoverable condition right
339 # uncoverable condition false
340 if (!$x && $y) {
341 }
342 "Or" conditionals are handled in a similar fashion (TODO - provide some
344 examples) but "xor" conditionals are not properly handled yet.
345 As for branches, the "count" value may be used for either conditions in
347 elsif conditionals, or for complex conditions.
348 Subroutines
350 A subroutine should be marked as uncoverable at the point where the
352 first statement is marked as uncoverable. Ideally all other criteria
353 in the subroutine would be marked as uncoverable automatically, but
354 that isn't the case at the moment.
355 sub z {
357 # uncoverable subroutine
358 $y++; # uncoverable statement
359 }
360 Non-invasive specification
362 If you can't, or don't want to add coverage comments to your code, you
363 can specify the uncoverable information in a separate file. By default
364 the files PWD/.uncoverable and HOME/.uncoverable are checked. If you
365 use the -uncoverable_file parameter then the file you provide is
366 checked as well as those two files.
367 The interface to managing this file is the cover program, and the
369 options are:
370 -uncoverable_file
372 -add_uncoverable_point
373 -delete_uncoverable_point **UNIMPLEMENTED**
374 -clean_uncoverable_points **UNIMPLEMENTED**
375 The parameter for -add_uncoverable_point is a string composed of up to
377 seven space separated elements: "$file $criterion $line $count $type
378 $class $note".
379 The contents of the uncoverable file is the same, with one point per
381 line.
382
384 User variables
385 The -silent option is turned on when Devel::Cover is invoked via
386 $HARNESS_PERL_SWITCHES or $PERL5OPT. Devel::Cover tries to do the
387 right thing when $MOD_PERL is set. $DEVEL_COVER_OPTIONS is appended to
388 any options passed into Devel::Cover.
389 Note that when Devel::Cover is invoked via an environment variable, any
391 modules specified on the command line, such as via the -Mmodule option,
392 will not be covered. This is because the environment variables are
393 processed after the command line and any code to be covered must appear
394 after Devel::Cover has been loaded. To work around this, Devel::Cover
395 can also be specified on the command line.
396 Developer variables
398 When running Devel::Cover's own test suite, $DEVEL_COVER_DEBUG turns on
399 debugging information, $DEVEL_COVER_GOLDEN_VERSION overrides
400 Devel::Cover's own idea of which golden results it should test against,
401 and $DEVEL_COVER_NO_COVERAGE runs the tests without collecting
402 coverage. $DEVEL_COVER_DB_FORMAT may be set to "Sereal", "JSON" or
403 "Storable" to override the default choice of DB format (Sereal, then
404 JSON if either are available, otherwise Storable).
405 $DEVEL_COVER_IO_OPTIONS provides fine-grained control over the DB
406 format. For example, setting it to "pretty" when the format is JSON
407 will store the DB in a readable JSON format. $DEVEL_COVER_CPUS
408 overrides the automated detection of the number of CPUs to use in
409 parallel testing.
410
412 Some code and ideas cribbed from:
413 • Devel::OpProf
415 • B::Concise
417 • B::Deparse
419
421 • Devel::Cover::Tutorial
422 • B
424 • Pod::Coverage
426
428 There are things that Devel::Cover can't cover.
429 Absence of shared dependencies
431 Perl keeps track of which modules have been loaded (to avoid reloading
432 them). Because of this, it isn't possible to get coverage for a path
433 where a runtime import fails if the module being imported is one that
434 Devel::Cover uses internally. For example, suppose your program has
435 this function:
436 sub foo {
438 eval { require Storable };
439 if ($@) {
440 carp "Can't find Storable";
441 return;
442 }
443 # ...
444 }
445 You might write a test for the failure mode as
447 BEGIN { @INC = () }
449 foo();
450 # check for error message
451 Because Devel::Cover uses Storable internally, the import will succeed
453 (and the test will fail) under a coverage run.
454 Modules used by Devel::Cover while gathering coverage:
456 • B
458 • B::Debug
460 • B::Deparse
462 • Carp
464 • Cwd
466 • Digest::MD5
468 • File::Path
470 • File::Spec
472 • Storable or JSON::MaybeXS (and its backend) or Sereal
474 Redefined subroutines
476 If you redefine a subroutine you may find that the original subroutine
477 is not reported on. This is because I haven't yet found a way to
478 locate the original CV. Hints, tips or patches to resolve this will be
479 gladly accepted.
480 The module Test::TestCoverage uses this technique and so should not be
482 used in conjunction with Devel::Cover.
483
485 Almost certainly.
486 See the BUGS file, the TODO file and the bug trackers at
488 <https://github.com/pjcj/Devel--Cover/issues?sort=created&direction=desc&state=open>
489 and <https://rt.cpan.org/Public/Dist/Display.html?Name=Devel-Cover>
490 Please report new bugs on Github.
492
494 Copyright 2001-2019, Paul Johnson (paul@pjcj.net)
495 This software is free. It is licensed under the same terms as Perl
497 itself.
498 The latest version of this software should be available on CPAN and
500 from my homepage: http://www.pjcj.net/.
501perl v5.34.0 2022-01-21 Devel::Cover(3)