1Devel::Cover::Tutorial(U3s)er Contributed Perl DocumentatDieovnel::Cover::Tutorial(3)
2
3
4

NAME

6       Devel::Cover::Tutorial - An introduction to code coverage
7

VERSION

9       version 1.33
10

TUTORIAL

12       Here's part of a message I sent to perl-qa about code coverage metrics.
13
14   1.0 Introduction
15       It is wise to remember the following quote from Dijkstra, who said:
16
17         Testing never proves the absence of faults, it only shows their presence.
18
19       In particular, code coverage is just one weapon in the software
20       engineer's testing arsenal.
21
22       Any discussion of code coverage metrics is hampered by the fact that
23       many authors use different terms to describe the same kind of coverage.
24       Here, I shall provide only a brief introduction to some of the most
25       common metrics.
26
27   2.0 Metrics
28   2.1 Statement coverage
29       This is the most basic form of code coverage.  A statement is covered
30       if it is executed.  Note that statement != line of code.  Multiple
31       statements on a single line can confuse issues - the reporting if
32       nothing else.
33
34       Where there are sequences of statements without branches it is not
35       necessary to count the execution of every statement, just one will
36       suffice, but people often like the count of every line to be reported,
37       especially in summary statistics.  However it is not clear to me that
38       this is actually useful.
39
40       This type of coverage is fairly weak in that even with 100% statement
41       coverage there may still be serious problems in a program which could
42       be discovered through other types of metric.
43
44       It can be quite difficult to achieve 100% statement coverage.  There
45       may be sections of code designed to deal with error conditions, or
46       rarely occurring events such as a signal received during a certain
47       section of code.  There may also be code that should never be executed:
48
49         if ($param > 20) {
50           die "This should never happen!";
51         }
52
53       It can be useful to mark such code in some way and flag an error if it
54       is executed.
55
56       Statement coverage, or something very similar, can be called statement
57       execution, line, block, basic block or segment coverage.  I tend to
58       favour block coverage which does not attempt to extend its results to
59       each statement.
60
61   2.2 Branch coverage
62       The goal of branch coverage is to ensure that whenever a program can
63       jump, it jumps to all possible destinations.  The most simple example
64       is a complete if statement:
65
66         if ($x) {
67           print "a";
68         } else {
69           print "b";
70         }
71
72       In such a simple example statement coverage is as powerful, but branch
73       coverage should also allow for the case where the else part is missing:
74
75         if ($x) {
76           print "a";
77         }
78
79       Full coverage is only achieved here if $x is true on one occasion and
80       false on another.
81
82       100% branch coverage implies 100% statement coverage.
83
84       Branch coverage is also called decision or all edges coverage.
85
86   2.3 Path coverage
87       There are classes of errors that branch coverage cannot detect, such
88       as:
89
90         $h = undef;
91         if ($x) {
92           $h = { a => 1 };
93         } if ($y) {
94           print $h->{a};
95         }
96
97       100% branch coverage can be achieved by setting ($x, $y) to (1, 1) and
98       then to (0, 0).  But if we have (0, 1) then things go bang.
99
100       The purpose of path coverage is to ensure that all paths through the
101       program are taken.  In any reasonably sized program there will be an
102       enormous number of paths through the program and so in practice the
103       paths can be limited to a single subroutine, if the subroutine is not
104       too big, or simply to two consecutive branches.
105
106       In the above example there are four paths which correspond to the truth
107       table for $x and $y.  To achieve 100% path coverage they must all be
108       taken.  Note that missing elses count as paths.
109
110       In some cases it may be impossible to achieve 100% path coverage:
111
112         a if $x;
113         b;
114         c if $x;
115
116       50% path coverage is the best you can get here.
117
118       Loops also contribute to paths, and pose their own problems which I'll
119       ignore for now.
120
121       100% path coverage implies 100% branch coverage.
122
123       Path coverage and some of its close cousins, are also known as
124       predicate, basis path and LCSAJ (Linear Code Sequence and Jump)
125       coverage.
126
127   2.4 Expression coverage
128       When a boolean expression is evaluated it can be useful to ensure that
129       all the terms in the expression are exercised.  For example:
130
131         a if $x || $y
132
133       The expression should be exercised with ($x, $y) set to (0, 0)
134       (required for branch coverage), (0, 1) and (1, 0) (to ensure that $x
135       and $y are independent) and possibly with (1, 1).
136
137       Expression coverage gets complicated, and difficult to achieve, as the
138       expression gets complicated.
139
140       Expressions which are not directly a part of a branching construct
141       should also be covered:
142
143         $z = $x || $y;
144         a if $z;
145
146       Expression coverage is also known as condition, condition-decision and
147       multiple decision coverage.
148
149   3.0 Other considerations
150       In order to get people to actually use code coverage it needs to be
151       simple to use.  It should also be simple to understand the results and
152       to rectify any problems thrown up.  Finally, if the overhead is too
153       great it won't get used either.
154
155       So there's a basic tutorial on code coverage, or at least my version of
156       it.  Typing a few of these terms into google will probably provide a
157       basis for future research.
158

LICENCE

160       Copyright 2001-2019, Paul Johnson (paul@pjcj.net)
161
162       This software is free.  It is licensed under the same terms as Perl
163       itself.
164
165       The latest version of this software should be available from my
166       homepage: http://www.pjcj.net
167
168
169
170perl v5.30.1                      2020-01-29         Devel::Cover::Tutorial(3)
Impressum