1Devel::Cover(3) User Contributed Perl Documentation Devel::Cover(3)
2
6 Devel::Cover - Code coverage metrics for Perl
7
9 version 1.40
10
12 To get coverage for an uninstalled module:
13 cover -test
15 or
17 cover -delete
19 HARNESS_PERL_SWITCHES=-MDevel::Cover make test
20 cover
21 or if you are using dzil (Dist::Zilla) and have installed
23 Dist::Zilla::App::Command::cover:
24 dzil cover
26 To get coverage for an uninstalled module which uses Module::Build
28 (0.26 or later):
29 ./Build testcover
31 If the module does not use the t/*.t framework:
33 PERL5OPT=-MDevel::Cover make test
35 If you want to get coverage for a program:
37 perl -MDevel::Cover yourprog args
39 cover
40 To alter default values:
42 perl -MDevel::Cover=-db,cover_db,-coverage,statement,time yourprog args
44
46 This module provides code coverage metrics for Perl. Code coverage
47 metrics describe how thoroughly tests exercise code. By using
48 Devel::Cover you can discover areas of code not exercised by your tests
49 and determine which tests to create to increase coverage. Code
50 coverage can be considered an indirect measure of quality.
51 Devel::Cover is now quite stable and provides many of the features to
53 be expected in a useful coverage tool.
54 Statement, branch, condition, subroutine, and pod coverage information
56 is reported. Statement and subroutine coverage data should be
57 accurate. Branch and condition coverage data should be mostly accurate
58 too, although not always what one might initially expect. Pod coverage
59 comes from Pod::Coverage. If Pod::Coverage::CountParents is available
60 it will be used instead.
61 The cover program can be used to generate coverage reports.
63 Devel::Cover ships with a number of reports including various types of
64 HTML output, textual reports, a report to display missing coverage in
65 the same format as compilation errors and a report to display coverage
66 information within the Vim editor.
67 It is possible to add annotations to reports, for example you can add a
69 column to an HTML report showing who last changed a line, as determined
70 by git blame. Some annotation modules are shipped with Devel::Cover
71 and you can easily create your own.
72 The gcov2perl program can be used to convert gcov files to
74 "Devel::Cover" databases. This allows you to display your C or XS code
75 coverage together with your Perl coverage, or to use any of the
76 Devel::Cover reports to display your C coverage data.
77 Code coverage data are collected by replacing perl ops with functions
79 which count how many times the ops are executed. These data are then
80 mapped back to reality using the B compiler modules. There is also a
81 statement profiling facility which should not be relied on. For proper
82 profiling use Devel::NYTProf. Previous versions of Devel::Cover
83 collected coverage data by replacing perl's runops function. It is
84 still possible to switch to that mode of operation, but this now gets
85 little testing and will probably be removed soon. You probably don't
86 care about any of this.
87 The most appropriate mailing list on which to discuss this module would
89 be perl-qa. See <http://lists.perl.org/list/perl-qa.html>.
90 The Devel::Cover repository can be found at
92 <http://github.com/pjcj/Devel--Cover>. This is also where problems
93 should be reported.
94
96 REQUIREMENTS
97 • Perl 5.12.0 or greater.
98 The latest version of Devel::Cover on which Perl 5.10 was supported
100 was 1.38. The latest version of Devel::Cover on which Perl 5.8 was
101 supported was 1.23. Perl versions 5.6.1 and 5.6.2 were not
102 supported after version 1.22. Perl versions 5.6.0 and earlier were
103 never supported. Using Devel::Cover with Perl 5.8.7 was always
104 problematic and frequently lead to crashes.
105 Different versions of perl may give slightly different results due
107 to changes in the op tree.
108 • The ability to compile XS extensions.
110 This means a working C compiler and make program at least. If you
112 built perl from source you will have these already and they will be
113 used automatically. If your perl was built in some other way, for
114 example you may have installed it using your Operating System's
115 packaging mechanism, you will need to ensure that the appropriate
116 tools are installed.
117 • Storable and Digest::MD5
119 Both are in the core in Perl 5.8.0 and above.
121 OPTIONAL MODULES
123 • Template, and either PPI::HTML or Perl::Tidy
124 Needed if you want syntax highlighted HTML reports.
126 • Pod::Coverage (0.06 or above) or Pod::Coverage::CountParents
128 One is needed if you want Pod coverage. If
130 Pod::Coverage::CountParents is installed, it is preferred.
131 • Test::More
133 Required if you want to run Devel::Cover's own tests.
135 • Test::Differences
137 Needed if the tests fail and you would like nice output telling you
139 why.
140 • Template and Parallel::Iterator
142 Needed if you want to run cpancover.
144 • JSON::MaybeXS
146 JSON is used to store the coverage database if it is available.
148 JSON::MaybeXS will select the best JSON backend installed.
149 Use with mod_perl
151 By adding "use Devel::Cover;" to your mod_perl startup script, you
152 should be able to collect coverage information when running under
153 mod_perl. You can also add any options you need at this point. I
154 would suggest adding this as early as possible in your startup script
155 in order to collect as much coverage information as possible.
156 Alternatively, add -MDevel::Cover to the parameters for mod_perl. In
158 this example, Devel::Cover will be operating in silent mode.
159 PerlSwitches -MDevel::Cover=-silent,1
161
163 -blib - "use blib" and ignore files matching \bt/ (default true
164 if blib directory exists, false otherwise)
165 -coverage criterion - Turn on coverage for the specified criterion. Criteria
166 include statement, branch, condition, path, subroutine,
167 pod, time, all and none (default all available)
168 -db cover_db - Store results in coverage db (default ./cover_db)
169 -dir path - Directory in which coverage will be collected (default
170 cwd)
171 -ignore RE - Set regular expressions for files to ignore (default
172 "/Devel/Cover\b")
173 +ignore RE - Append to regular expressions of files to ignore
174 -inc path - Set prefixes of files to include (default @INC)
175 +inc path - Append to prefixes of files to include
176 -loose_perms val - Use loose permissions on all files and directories in
177 the coverage db so that code changing EUID can still
178 write coverage information (default off)
179 -merge val - Merge databases, for multiple test benches (default on)
180 -select RE - Set regular expressions of files to select (default none)
181 +select RE - Append to regular expressions of files to select
182 -silent val - Don't print informational messages (default off)
183 -subs_only val - Only cover code in subroutine bodies (default off)
184 -replace_ops val - Use op replacing rather than runops (default on)
185 -summary val - Print summary information if val is true (default on)
186 More on Coverage Options
188 You can specify options to some coverage criteria. At the moment only
189 pod coverage takes any options. These are the parameters which are
190 passed into the Pod::Coverage constructor. The extra options are
191 separated by dashes, and you may specify as many as you wish. For
192 example, to specify that all subroutines containing xx are private,
193 call Devel::Cover with the option -coverage,pod-also_private-xx.
194 Or, to ignore all files in "t/lib" as well as files ending in "Foo.pm":
196 cover -test -silent -ignore ^t/lib/,Foo.pm$
198 Note that "-ignore" replaces any default ignore regexes. To preserve
200 any ignore regexes which have already been set, use "+ignore":
201 cover -test -silent +ignore ^t/lib/,Foo.pm$
203
205 You may select the files for which you want to collect coverage data
206 using the select, ignore and inc options. The system uses the
207 following procedure to decide whether a file will be included in
208 coverage reports:
209 • If the file matches a RE given as a select option, it will be
211 included
212 • Otherwise, if it matches a RE given as an ignore option, it won't
214 be included
215 • Otherwise, if it is in one of the inc directories, it won't be
217 included
218 • Otherwise, it will be included
220 You may add to the REs to select by using +select, or you may reset the
222 selections using -select. The same principle applies to the REs to
223 ignore.
224 The inc directories are initially populated with the contents of perl's
226 @INC array. You may reset these directories using -inc, or add to them
227 using +inc.
228 Although these options take regular expressions, you should not enclose
230 the RE within // or any other quoting characters.
231 The options -coverage, [+-]select, [+-]ignore and [+-]inc can be
233 specified multiple times, but they can also take multiple comma
234 separated arguments. In any case you should not add a space after the
235 comma, unless you want the argument to start with that literal space.
236
238 Sometimes you have code which is uncoverable for some reason. Perhaps
239 it is an else clause that cannot be reached, or a check for an error
240 condition that should never happen. You can tell Devel::Cover that
241 certain criteria are uncoverable and then they are not counted as
242 errors when they are not exercised. In fact, they are counted as
243 errors if they are exercised.
244 This feature should only be used as something of a last resort.
246 Ideally you would find some way of exercising all your code. But if
247 you have analysed your code and determined that you are not going to be
248 able to exercise it, it may be better to record that fact in some
249 formal fashion and stop Devel::Cover complaining about it, so that real
250 problems are not lost in the noise.
251 If you have uncoverable criteria I suggest not using the default HTML
253 report (with uses html_minimal at the moment) because this sometimes
254 shows uncoverable points as uncovered. Instead, you should use the
255 html_basic report for HTML output which should behave correctly in this
256 regard.
257 There are two ways to specify a construct as uncoverable, one invasive
259 and one non-invasive.
260 Invasive specification
262 You can use special comments in your code to specify uncoverable
263 criteria. Comments are of the form:
264 # uncoverable <criterion> [details]
266 The keyword "uncoverable" must be the first text in the comment. It
268 should be followed by the name of the coverage criterion which is
269 uncoverable. There may then be further information depending on the
270 nature of the uncoverable construct.
271 In all cases as class attribute may be included in details. At present
273 a single class attribute is recognised: ignore_covered_err. Normally,
274 an error is flagged if code marked as uncoverable is covered. When the
275 ignore_covered_err attribute is specified then such errors will not be
276 flagged. This is a more precise method to flag such exceptions than
277 the global -ignore_covered_err flag to the cover program.
278 There is also a note attribute which can also be included in details.
280 This should be the final attribute and will consude all the remaining
281 text. Currently this attribute is not used, but it is intented as a
282 form of documentation for the uncoverable data.
283 Example:
285 # uncoverable branch true count:1..3 class:ignore_covered_err note:error chk
287 Statements
289 The "uncoverable" comment should appear on either the same line as the
291 statement, or on the line before it:
292 $impossible++; # uncoverable statement
294 # uncoverable statement
295 it_has_all_gone_horribly_wrong();
296 If there are multiple statements (or any other criterion) on a line you
298 can specify which statement is uncoverable by using the "count"
299 attribute, count:n, which indicates that the uncoverable statement is
300 the nth statement on the line.
301 # uncoverable statement count:1
303 # uncoverable statement count:2
304 cannot_run_this(); or_this();
305 Branches
307 The "uncoverable" comment should specify whether the "true" or "false"
309 branch is uncoverable.
310 # uncoverable branch true
312 if (pi == 3)
313 Both branches may be uncoverable:
315 # uncoverable branch true
317 # uncoverable branch false
318 if (impossible_thing_happened_one_way()) {
319 handle_it_one_way(); # uncoverable statement
320 } else {
321 handle_it_another_way(); # uncoverable statement
322 }
323 If there is an elsif in the branch then it can be addressed as the
325 second branch on the line by using the "count" attribute. Further
326 elsifs are the third and fourth "count" value, and so on:
327 # uncoverable branch false count:2
329 if ($thing == 1) {
330 handle_thing_being_one();
331 } elsif ($thing == 2) {
332 handle_thing_being_tow();
333 } else {
334 die "thing can only be one or two, not $thing"; # uncoverable statement
335 }
336 Conditions
338 Because of the way in which Perl short-circuits boolean operations,
340 there are three ways in which such conditionals can be uncoverable. In
341 the case of " $x && $y" for example, the left operator may never be
342 true, the right operator may never be true, and the whole operation may
343 never be false. These conditions may be modelled thus:
344 # uncoverable branch true
346 # uncoverable condition left
347 # uncoverable condition false
348 if ($x && !$y) {
349 $x++; # uncoverable statement
350 }
351 # uncoverable branch true
353 # uncoverable condition right
354 # uncoverable condition false
355 if (!$x && $y) {
356 }
357 "Or" conditionals are handled in a similar fashion (TODO - provide some
359 examples) but "xor" conditionals are not properly handled yet.
360 As for branches, the "count" value may be used for either conditions in
362 elsif conditionals, or for complex conditions.
363 Subroutines
365 A subroutine should be marked as uncoverable at the point where the
367 first statement is marked as uncoverable. Ideally all other criteria
368 in the subroutine would be marked as uncoverable automatically, but
369 that isn't the case at the moment.
370 sub z {
372 # uncoverable subroutine
373 $y++; # uncoverable statement
374 }
375 Non-invasive specification
377 If you can't, or don't want to add coverage comments to your code, you
378 can specify the uncoverable information in a separate file. By default
379 the files PWD/.uncoverable and HOME/.uncoverable are checked. If you
380 use the -uncoverable_file parameter then the file you provide is
381 checked as well as those two files.
382 The interface to managing this file is the cover program, and the
384 options are:
385 -uncoverable_file
387 -add_uncoverable_point
388 -delete_uncoverable_point **UNIMPLEMENTED**
389 -clean_uncoverable_points **UNIMPLEMENTED**
390 The parameter for -add_uncoverable_point is a string composed of up to
392 seven space separated elements: "$file $criterion $line $count $type
393 $class $note".
394 The contents of the uncoverable file is the same, with one point per
396 line.
397
399 User variables
400 The -silent option is turned on when Devel::Cover is invoked via
401 $HARNESS_PERL_SWITCHES or $PERL5OPT. Devel::Cover tries to do the
402 right thing when $MOD_PERL is set. $DEVEL_COVER_OPTIONS is appended to
403 any options passed into Devel::Cover.
404 Note that when Devel::Cover is invoked via an environment variable, any
406 modules specified on the command line, such as via the -Mmodule option,
407 will not be covered. This is because the environment variables are
408 processed after the command line and any code to be covered must appear
409 after Devel::Cover has been loaded. To work around this, Devel::Cover
410 can also be specified on the command line.
411 Developer variables
413 When running Devel::Cover's own test suite, $DEVEL_COVER_DEBUG turns on
414 debugging information, $DEVEL_COVER_GOLDEN_VERSION overrides
415 Devel::Cover's own idea of which golden results it should test against,
416 and $DEVEL_COVER_NO_COVERAGE runs the tests without collecting
417 coverage. $DEVEL_COVER_DB_FORMAT may be set to "Sereal", "JSON" or
418 "Storable" to override the default choice of DB format (Sereal, then
419 JSON if either are available, otherwise Storable).
420 $DEVEL_COVER_IO_OPTIONS provides fine-grained control over the DB
421 format. For example, setting it to "pretty" when the format is JSON
422 will store the DB in a readable JSON format. $DEVEL_COVER_CPUS
423 overrides the automated detection of the number of CPUs to use in
424 parallel testing.
425
427 Some code and ideas cribbed from:
428 • Devel::OpProf
430 • B::Concise
432 • B::Deparse
434
436 • Devel::Cover::Tutorial
437 • B
439 • Pod::Coverage
441
443 There are things that Devel::Cover can't cover.
444 Absence of shared dependencies
446 Perl keeps track of which modules have been loaded (to avoid reloading
447 them). Because of this, it isn't possible to get coverage for a path
448 where a runtime import fails if the module being imported is one that
449 Devel::Cover uses internally. For example, suppose your program has
450 this function:
451 sub foo {
453 eval { require Storable };
454 if ($@) {
455 carp "Can't find Storable";
456 return;
457 }
458 # ...
459 }
460 You might write a test for the failure mode as
462 BEGIN { @INC = () }
464 foo();
465 # check for error message
466 Because Devel::Cover uses Storable internally, the import will succeed
468 (and the test will fail) under a coverage run.
469 Modules used by Devel::Cover while gathering coverage:
471 • B
473 • B::Deparse
475 • Carp
477 • Cwd
479 • Digest::MD5
481 • File::Path
483 • File::Spec
485 • Storable or JSON::MaybeXS (and its backend) or Sereal
487 Redefined subroutines
489 If you redefine a subroutine you may find that the original subroutine
490 is not reported on. This is because I haven't yet found a way to
491 locate the original CV. Hints, tips or patches to resolve this will be
492 gladly accepted.
493 The module Test::TestCoverage uses this technique and so should not be
495 used in conjunction with Devel::Cover.
496
498 Almost certainly.
499 See the BUGS file, the TODO file and the bug trackers at
501 <https://github.com/pjcj/Devel--Cover/issues?sort=created&direction=desc&state=open>
502 and <https://rt.cpan.org/Public/Dist/Display.html?Name=Devel-Cover>
503 Please report new bugs on Github.
505
507 Copyright 2001-2023, Paul Johnson (paul@pjcj.net)
508 This software is free. It is licensed under the same terms as Perl
510 itself.
511 The latest version of this software should be available on CPAN and
513 from my homepage: http://www.pjcj.net/.
514perl v5.38.2 2023-11-30 Devel::Cover(3)