1Devel::Cover(3) User Contributed Perl Documentation Devel::Cover(3)
2
6 Devel::Cover - Code coverage metrics for Perl
7
9 version 1.31
10
12 To get coverage for an uninstalled module:
13 cover -test
15 or
17 cover -delete
19 HARNESS_PERL_SWITCHES=-MDevel::Cover make test
20 cover
21 To get coverage for an uninstalled module which uses Module::Build
23 (0.26 or later):
24 ./Build testcover
26 If the module does not use the t/*.t framework:
28 PERL5OPT=-MDevel::Cover make test
30 If you want to get coverage for a program:
32 perl -MDevel::Cover yourprog args
34 cover
35 To alter default values:
37 perl -MDevel::Cover=-db,cover_db,-coverage,statement,time yourprog args
39
41 This module provides code coverage metrics for Perl. Code coverage
42 metrics describe how thoroughly tests exercise code. By using
43 Devel::Cover you can discover areas of code not exercised by your tests
44 and determine which tests to create to increase coverage. Code
45 coverage can be considered an indirect measure of quality.
46 Although it is still being developed, Devel::Cover is now quite stable
48 and provides many of the features to be expected in a useful coverage
49 tool.
50 Statement, branch, condition, subroutine, and pod coverage information
52 is reported. Statement and subroutine coverage data should be
53 accurate. Branch and condition coverage data should be mostly accurate
54 too, although not always what one might initially expect. Pod coverage
55 comes from Pod::Coverage. If Pod::Coverage::CountParents is available
56 it will be used instead. Coverage data for other criteria are not yet
57 collected.
58 The cover program can be used to generate coverage reports.
60 Devel::Cover ships with a number of reports including various types of
61 HTML output, textual reports, a report to display missing coverage in
62 the same format as compilation errors and a report to display coverage
63 information within the Vim editor.
64 It is possible to add annotations to reports, for example you can add a
66 column to an HTML report showing who last changed a line, as determined
67 by git blame. Some annotation modules are shipped with Devel::Cover
68 and you can easily create your own.
69 The gcov2perl program can be used to convert gcov files to
71 "Devel::Cover" databases. This allows you to display your C or XS code
72 coverage together with your Perl coverage, or to use any of the
73 Devel::Cover reports to display your C coverage data.
74 Code coverage data are collected by replacing perl ops with functions
76 which count how many times the ops are executed. These data are then
77 mapped back to reality using the B compiler modules. There is also a
78 statement profiling facility which should not be relied on. For proper
79 profiling use Devel::NYTProf. Previous versions of Devel::Cover
80 collected coverage data by replacing perl's runops function. It is
81 still possible to switch to that mode of operation, but this now gets
82 little testing and will probably be removed soon. You probably don't
83 care about any of this.
84 The most appropriate mailing list on which to discuss this module would
86 be perl-qa. See <http://lists.perl.org/list/perl-qa.html>.
87 The Devel::Cover repository can be found at
89 <http://github.com/pjcj/Devel--Cover>. This is also where problems
90 should be reported.
91
93 REQUIREMENTS
94 · Perl 5.10.0 or greater.
95 Perl versions 5.6.1, 5.6.2 and 5.8.x may work to an extent but are
97 unsupported. Perl 5.8.7 has problems and may crash.
98 If you want to use an unsupported version you will need to set the
100 environment variable $DEVEL_COVER_UNSUPPORTED. Unsupported
101 versions are also untested. I will consider patches for
102 unsupported versions only if they do not compromise the code. This
103 is a vague, nebulous concept that I will decide on if and when
104 necessary.
105 If you are using an unsupported version, please let me know. I
107 don't want to know if you are just testing Devel::Cover, only if
108 you are seriously using it to do code coverage analysis of real
109 code. If I get no reports of such usage then I will remove support
110 and delete the workarounds for versions of perl below 5.10.0. I
111 may do that anyway.
112 Different versions of perl may give slightly different results due
114 to changes in the op tree.
115 · The ability to compile XS extensions.
117 This means a working C compiler and make program at least. If you
119 built perl from source you will have these already and they will be
120 used automatically. If your perl was built in some other way, for
121 example you may have installed it using your Operating System's
122 packaging mechanism, you will need to ensure that the appropriate
123 tools are installed.
124 · Storable and Digest::MD5
126 Both are in the core in Perl 5.8.0 and above.
128 OPTIONAL MODULES
130 · Template, and either PPI::HTML or Perl::Tidy
131 Needed if you want syntax highlighted HTML reports.
133 · Pod::Coverage (0.06 or above) or Pod::Coverage::CountParents
135 One is needed if you want Pod coverage. If
137 Pod::Coverage::CountParents is installed, it is preferred.
138 · Test::More
140 Required if you want to run Devel::Cover's own tests.
142 · Test::Differences
144 Needed if the tests fail and you would like nice output telling you
146 why.
147 · Template and Parallel::Iterator
149 Needed if you want to run cpancover.
151 · JSON::MaybeXS
153 JSON is used to store the coverage database if it is available.
155 JSON::MaybeXS will select the best JSON backend installed.
156 Use with mod_perl
158 By adding "use Devel::Cover;" to your mod_perl startup script, you
159 should be able to collect coverage information when running under
160 mod_perl. You can also add any options you need at this point. I
161 would suggest adding this as early as possible in your startup script
162 in order to collect as much coverage information as possible.
163 Alternatively, add -MDevel::Cover to the parameters for mod_perl. In
165 this example, Devel::Cover will be operating in silent mode.
166 PerlSwitches -MDevel::Cover=-silent,1
168
170 -blib - "use blib" and ignore files matching \bt/ (default true
171 if blib directory exists, false otherwise)
172 -coverage criterion - Turn on coverage for the specified criterion. Criteria
173 include statement, branch, condition, path, subroutine,
174 pod, time, all and none (default all available)
175 -db cover_db - Store results in coverage db (default ./cover_db)
176 -dir path - Directory in which coverage will be collected (default
177 cwd)
178 -ignore RE - Set regular expressions for files to ignore (default
179 "/Devel/Cover\b")
180 +ignore RE - Append to regular expressions of files to ignore
181 -inc path - Set prefixes of files to include (default @INC)
182 +inc path - Append to prefixes of files regular expressionsto include
183 -loose_perms val - Use loose permissions on all files and directories in
184 the coverage db so that code changing EUID can still
185 write coverage information (default off)
186 -merge val - Merge databases, for multiple test benches (default on)
187 -select RE - Set regular expressions of files to select (default none)
188 +select RE - Append to regular expressions of files to select
189 -silent val - Don't print informational messages (default off)
190 -subs_only val - Only cover code in subroutine bodies (default off)
191 -replace_ops val - Use op replacing rather than runops (default on)
192 -summary val - Print summary information if val is true (default on)
193 More on Coverage Options
195 You can specify options to some coverage criteria. At the moment only
196 pod coverage takes any options. These are the parameters which are
197 passed into the Pod::Coverage constructor. The extra options are
198 separated by dashes, and you may specify as many as you wish. For
199 example, to specify that all subroutines containing xx are private,
200 call Devel::Cover with the option -coverage,pod-also_private-xx.
201 Or, to ignore all files in "t/lib" as well as files ending in "Foo.pm":
203 cover -test -silent -ignore ^t/lib/,Foo.pm$
205 Note that "-ignore" replaces any default ignore regexes. To preserve
207 any ignore regexes which have already been set, use "+ignore":
208 cover -test -silent +ignore ^t/lib/,Foo.pm$
210
212 You may select the files for which you want to collect coverage data
213 using the select, ignore and inc options. The system uses the
214 following procedure to decide whether a file will be included in
215 coverage reports:
216 · If the file matches a RE given as a select option, it will be
218 included
219 · Otherwise, if it matches a RE given as an ignore option, it won't
221 be included
222 · Otherwise, if it is in one of the inc directories, it won't be
224 included
225 · Otherwise, it will be included
227 You may add to the REs to select by using +select, or you may reset the
229 selections using -select. The same principle applies to the REs to
230 ignore.
231 The inc directories are initially populated with the contents of perl's
233 @INC array. You may reset these directories using -inc, or add to them
234 using +inc.
235 Although these options take regular expressions, you should not enclose
237 the RE within // or any other quoting characters.
238 The options -coverage, [+-]select, [+-]ignore and [+-]inc can be
240 specified multiple times, but they can also take multiple comma
241 separated arguments. In any case you should not add a space after the
242 comma, unless you want the argument to start with that literal space.
243
245 Sometimes you have code which is uncoverable for some reason. Perhaps
246 it is an else clause that cannot be reached, or a check for an error
247 condition that should never happen. You can tell Devel::Cover that
248 certain criteria are uncoverable and then they are not counted as
249 errors when they are not exercised. In fact, they are counted as
250 errors if they are exercised.
251 This feature should only be used as something of a last resort.
253 Ideally you would find some way of exercising all your code. But if
254 you have analysed your code and determined that you are not going to be
255 able to exercise it, it may be better to record that fact in some
256 formal fashion and stop Devel::Cover complaining about it, so that real
257 problems are not lost in the noise.
258 If you have uncoverable criteria I suggest not using the default HTML
260 report (with uses html_minimal at the moment) because this sometimes
261 shows uncoverable points as uncovered. Instead, you should use the
262 html_basic report for HTML output which should behave correctly in this
263 regard.
264 There are two ways to specify a construct as uncoverable, one invasive
266 and one non-invasive.
267 Invasive specification
269 You can use special comments in your code to specify uncoverable
270 criteria. Comments are of the form:
271 # uncoverable <criterion> [details]
273 The keyword "uncoverable" must be the first text in the comment. It
275 should be followed by the name of the coverage criterion which is
276 uncoverable. There may then be further information depending on the
277 nature of the uncoverable construct.
278 Statements
280 The "uncoverable" comment should appear on either the same line as the
282 statement, or on the line before it:
283 $impossible++; # uncoverable statement
285 # uncoverable statement
286 it_has_all_gone_horribly_wrong();
287 If there are multiple statements (or any other criterion) on a line you
289 can specify which statement is uncoverable by using the "count"
290 attribute, count:n, which indicates that the uncoverable statement is
291 the nth statement on the line.
292 # uncoverable statement count:1
294 # uncoverable statement count:2
295 cannot_run_this(); or_this();
296 Branches
298 The "uncoverable" comment should specify whether the "true" or "false"
300 branch is uncoverable.
301 # uncoverable branch true
303 if (pi == 3)
304 Both branches may be uncoverable:
306 # uncoverable branch true
308 # uncoverable branch false
309 if (impossible_thing_happened_one_way()) {
310 handle_it_one_way(); # uncoverable statement
311 } else {
312 handle_it_another_way(); # uncoverable statement
313 }
314 If there is an elsif in the branch then it can be addressed as the
316 second branch on the line by using the "count" attribute. Further
317 elsifs are the third and fourth "count" value, and so on:
318 # uncoverable branch false count:2 if ($thing == 1) {
320 handle_thing_being_one(); } elsif ($thing == 2) {
321 handle_thing_being_tow(); } else {
322 die "thing can only be one or two, not $thing"; # uncoverable
323 statement }
324 Conditions
326 Because of the way in which Perl short-circuits boolean operations,
328 there are three ways in which such conditionals can be uncoverable. In
329 the case of " $x && $y" for example, the left operator may never be
330 true, the right operator may never be true, and the whole operation may
331 never be false. These conditions may be modelled thus:
332 # uncoverable branch true
334 # uncoverable condition left
335 # uncoverable condition false
336 if ($x && !$y) {
337 $x++; # uncoverable statement
338 }
339 # uncoverable branch true
341 # uncoverable condition right
342 # uncoverable condition false
343 if (!$x && $y) {
344 }
345 "Or" conditionals are handled in a similar fashion (TODO - provide some
347 examples) but "xor" conditionals are not properly handled yet.
348 As for branches, the "count" value may be used for either conditions in
350 elsif conditionals, or for complex conditions.
351 Subroutines
353 A subroutine should be marked as uncoverable at the point where the
355 first statement is marked as uncoverable. Ideally all other criteria
356 in the subroutine would be marked as uncoverable automatically, but
357 that isn't the case at the moment.
358 sub z {
360 # uncoverable subroutine
361 $y++; # uncoverable statement
362 }
363 Non-invasive specification
365 If you can't, or don't want to add coverage comments to your code, you
366 can specify the uncoverable information in a separate file. By default
367 the files PWD/.uncoverable and HOME/.uncoverable are checked. If you
368 use the -uncoverable_file parameter then the file you provide is
369 checked as well as those two files.
370 The interface to managing this file is the cover program, and the
372 options are:
373 -uncoverable_file
375 -add_uncoverable_point
376 -delete_uncoverable_point **UNIMPLEMENTED**
377 -clean_uncoverable_points **UNIMPLEMENTED**
378 The parameter for -add_uncoverable_point is a string composed of up to
380 seven space separated elements: "$file $criterion $line $count $type
381 $class $note".
382 The contents of the uncoverable file is the same, with one point per
384 line.
385
387 User variables
388 The -silent option is turned on when Devel::Cover is invoked via
389 $HARNESS_PERL_SWITCHES or $PERL5OPT. Devel::Cover tries to do the
390 right thing when $MOD_PERL is set. $DEVEL_COVER_OPTIONS is appended to
391 any options passed into Devel::Cover.
392 Note that when Devel::Cover is invoked via an environment variable, any
394 modules specified on the command line, such as via the -Mmodule option,
395 will not be covered. This is because the environment variables are
396 processed after the command line and any code to be covered must appear
397 after Devel::Cover has been loaded. To work around this, Devel::Cover
398 can also be specified on the command line.
399 Developer variables
401 When running Devel::Cover's own test suite, $DEVEL_COVER_DEBUG turns on
402 debugging information, $DEVEL_COVER_GOLDEN_VERSION overrides
403 Devel::Cover's own idea of which golden results it should test against,
404 and $DEVEL_COVER_NO_COVERAGE runs the tests without collecting
405 coverage. $DEVEL_COVER_DB_FORMAT may be set to "Sereal", "JSON" or
406 "Storable" to override the default choice of DB format (Sereal, then
407 JSON if either are available, otherwise Storable).
408 $DEVEL_COVER_IO_OPTIONS provides fine-grained control over the DB
409 format. For example, setting it to "pretty" when the format is JSON
410 will store the DB in a readable JSON format. $DEVEL_COVER_CPUS
411 overrides the automated detection of the number of CPUs to use in
412 parallel testing.
413
415 Some code and ideas cribbed from:
416 · Devel::OpProf
418 · B::Concise
420 · B::Deparse
422
424 · Devel::Cover::Tutorial
425 · B
427 · Pod::Coverage
429
431 There are things that Devel::Cover can't cover.
432 Absence of shared dependencies
434 Perl keeps track of which modules have been loaded (to avoid reloading
435 them). Because of this, it isn't possible to get coverage for a path
436 where a runtime import fails if the module being imported is one that
437 Devel::Cover uses internally. For example, suppose your program has
438 this function:
439 sub foo {
441 eval { require Storable };
442 if ($@) {
443 carp "Can't find Storable";
444 return;
445 }
446 # ...
447 }
448 You might write a test for the failure mode as
450 BEGIN { @INC = () }
452 foo();
453 # check for error message
454 Because Devel::Cover uses Storable internally, the import will succeed
456 (and the test will fail) under a coverage run.
457 Modules used by Devel::Cover while gathering coverage:
459 · B
461 · B::Debug
463 · B::Deparse
465 · Carp
467 · Cwd
469 · Digest::MD5
471 · File::Path
473 · File::Spec
475 · Storable or JSON::MaybeXS (and its backend) or Sereal
477 Redefined subroutines
479 If you redefine a subroutine you may find that the original subroutine
480 is not reported on. This is because I haven't yet found a way to
481 locate the original CV. Hints, tips or patches to resolve this will be
482 gladly accepted.
483 The module Test::TestCoverage uses this technique and so should not be
485 used in conjunction with Devel::Cover.
486
488 Almost certainly.
489 See the BUGS file, the TODO file and the bug trackers at
491 <https://github.com/pjcj/Devel--Cover/issues?sort=created&direction=desc&state=open>
492 and <https://rt.cpan.org/Public/Dist/Display.html?Name=Devel-Cover>
493 Please report new bugs on Github.
495
497 Copyright 2001-2018, Paul Johnson (paul@pjcj.net)
498 This software is free. It is licensed under the same terms as Perl
500 itself.
501 The latest version of this software should be available on CPAN and
503 from my homepage: http://www.pjcj.net/.
504perl v5.28.1 2018-09-18 Devel::Cover(3)