1Test::LectroTest::PropeUrsteyr(3C)ontributed Perl DocumeTnetsatt:i:oLnectroTest::Property(3)
2
6 Test::LectroTest::Property - Properties that make testable claims about
7 your software
8
10 version 0.5001
11
13 use MyModule; # provides my_function_to_test
14 use Test::LectroTest::Generator qw( :common );
16 use Test::LectroTest::Property qw( Test );
17 use Test::LectroTest::TestRunner;
18 my $prop_non_neg = Property {
20 ##[ x <- Int, y <- Int ]##
21 $tcon->label("negative") if $x < 0;
22 $tcon->label("odd") if $x % 2;
23 $tcon->retry if $y == 0; # 0 can't be used in test
24 my_function_to_test( $x, $y ) >= 0;
25 }, name => "my_function_to_test output is non-negative";
26 my $runner = Test::LectroTest::TestRunner->new();
28 $runner->run_suite(
29 $prop_non_neg,
30 # ... more properties here ...
31 );
32
34 STOP! If you're just looking for an easy way to write and run unit
35 tests, see Test::LectroTest first. Once you're comfortable with what
36 is presented there and ready to delve into the full offerings of
37 properties, this is the document for you.
38 This module allows you to define Properties that can be checked
40 automatically by Test::LectroTest. A Property is a specification of
41 your software's required behavior over a given set of conditions. The
42 set of conditions is given by a generator-binding specification. The
43 required behavior is defined implicitly by a block of code that tests
44 your software for a given set of generated conditions; if your software
45 matches the expected behavor, the block of code returns true;
46 otherwise, false.
47 This documentation serves as reference documentation for LectroTest
49 Properties. If you don't understand the basics of Properties yet, see
50 "OVERVIEW" in Test::LectroTest::Tutorial before continuing.
51 Two ways to create Properties
53 There are two ways to create a property:
54 1. Use the "Property" function to promote a block of code that
56 contains both a generator-binding specification and a behavior test
57 into a Test::LectroTest::Property object. This is the preferred
58 method. Example:
59 my $prop1 = Property {
61 ##[ x <- Int ]##
62 thing_to_test($x) >= 0;
63 }, name => "thing_to_test is non-negative";
64 2. Use the "new" method of Test::LectroTest::Property and provide it
66 with the necessary ingredients via named parameters:
67 my $prop2 = Test::LectroTest::Property->new(
69 inputs => [ x => Int ],
70 test => sub { my ($tcon,$x) = @_;
71 thing_to_test($x) >= 0 },
72 name => "thing_to_test is non-negative"
73 );
74 Both are equivalent, but the first is concise, easier to read, and lets
76 LectroTest do some of the heavy lifting for you. The second is
77 probably better, however, if you are constructing property
78 specifications programmatically.
79 Generator-binding specification
81 The generator-binding specification declares that certain variables are
82 to be bound to certain kinds of random-value generators during the
83 tests of your software's behavior. The number and kind of generators
84 define the "condition space" that is examined during property checks.
85 If you use the "Property" function to create your properties, your
87 generator-binding specification must come first in your code block, and
88 you must use the following syntax:
89 ##[ var1 <- gen1, var2 <- gen2, ... ]##
91 Comments are not allowed within the specification, but you may break it
93 across multiple lines:
94 ##[ var1 <- gen1,
96 var2 <- gen2, ...
97 ]##
98 or
100 ##[
102 var1 <- gen1,
103 var2 <- gen2, ...
104 ]##
105 Further, for better integration with syntax-highlighting IDEs, the
107 terminating "]##" delimiter may be preceded by a hash symbol "#" and
108 optional whitespace to make it appear like a comment:
109 ##[
111 var1 <- gen1,
112 var2 <- gen2, ...
113 # ]##
114 On the other hand, if you use "Test::LectroTest::Property->new()" to
116 create your objects, the generator-binding specification takes the form
117 of an array reference containing variable-generator pairs that is
118 passed to "new()" via the parameter named "inputs":
119 inputs => [ var1 => gen1, var2 => gen2, ... ]
121 Normal Perl syntax applies here.
123 Specifying multiple sets of generator bindings
125 Sometimes you may want to repeat a property check with multiple sets of
126 generator bindings. This can happen, for instance, when your condition
127 space is vast and you want to ensure that a particular portion of it
128 receives focused coverage while still sampling the overall space. For
129 times like this, you can list multiple sets of bindings within the
130 "##[" and "]##" delimiters, like so:
131 ##[ var1 <- gen1A, ... ],
133 [ var1 <- gen1B, ... ],
134 ... more sets of bindings ...
135 [ var1 <- gen1N, ... ]##
136 Note that only the first and last set need the special delimiters.
138 The equivalent when using "new()" is as follows:
140 inputs => [ [ var1 => gen1A, ... ],
142 [ var1 => gen1B, ... ],
143 ...
144 [ var1 => gen1N, ... ] ]
145 Regardless of how you declare the sets of bindings, each set must
147 provide bindings for the exact same set of variables. (The generators,
148 of course, can be different.) For example, this kind of thing is
149 illegal:
150 ##[ x <- Int ], [ y <- Int ]##
152 The above is illegal because both sets of bindings must use x or both
154 must use y; they can't each use a different variable.
155 ##[ x <- Int ],
157 [ x <- Int, y <- Float ]##
158 The above is illegal because the second set has an extra variable that
160 isn't present in the first. Both sets must use exactly the same
161 variables. None of the variables may be extra, none may be missing,
162 and all must be named identically across the sets of bindings.
163 Behavior test
165 The behavior test is a subroutine that accepts a test-controller object
166 and a given set of input conditions, tests your software's observed
167 behavior against the required behavior with respect to the input
168 conditions, and returns true or false to indicate acceptance or
169 rejection. If you are using the "Property" function to create your
170 property objects, lexically bound variables are created and loaded with
171 values automatically, per your input-generator specification, so you
172 can just go ahead and use the variables immediately:
173 my $prop = Property {
175 ##[ i <- Int, delta <- Float(range=>[0,1]) ]##
176 my $lo_val = my_thing_to_test($i);
177 my $hi_val = my_thing_to_test($i + $delta);
178 $lo_val == $hi_val;
179 }, name => "my_thing_to_test ignores fractions" ;
180 On the other hand, if you are using
182 "Test::LectroTest::Property->new()", you must declare and initialize
183 these variables manually from Perl's @_ variable in lexicographically
184 increasing order after receiving $tcon, the test controller object.
185 (This inconvenience, by the way, is why the former method is
186 preferred.) The hard way:
187 my $prop = Test::LectroTest::Property->new(
189 inputs => [ i => Int, delta => Float(range=>[0,1]) ],
190 test => sub {
191 my ($tcon, $delta, $i) = @_;
192 my $lo_val = my_thing_to_test($i);
193 my $hi_val = my_thing_to_test($i + $delta);
194 $lo_val == $hi_val
195 },
196 name => "my_thing_to_test ignores fractions"
197 ) ;
198 Control logic, retries, and labeling
200 Inside the behavior test, you have access to a special variable $tcon
201 that allows you to interact with the test controller. Through $tcon
202 you can do the following:
203 · retry the current trial with different inputs (if you don't like
205 the inputs you were given at first)
206 · add labels to the current trial for reporting purposes
208 · attach notes and variable dumps to the current trial for diagnostic
210 purposes, should the trial fail
211 (For the full details of what you can do with $tcon see the
213 "testcontroller" section of Test::LectroTest::TestRunner.)
214 For example, let's say that we have written a function "my_sqrt" that
216 returns the square root of its input. In order to check whether our
217 implementation fulfills the mathematical definition of square root, we
218 might specify the following property:
219 my $epsilon = 0.000_001;
221 Property {
223 ##[ x <- Float ]##
224 return $tcon->retry if $x < 0;
225 $tcon->label("less than one") if $x < 1;
226 my $sx = my_sqrt( $x );
227 abs($sx * $sx - $x) < $epsilon;
228 }, name => "my_sqrt satisfies defn of square root";
229 Because we don't want to deal with imaginary numbers, our square-root
231 function is defined only over non-negative numbers. To make sure we
232 don't accidentally check our property "at" a negative number, we use
233 the following line to re-start the trial with a different input should
234 the input we are given at first be negative:
235 return $tcon->retry if $x < 0;
237 An interesting fact is that for all values x between zero and one, the
239 square root of x is larger than x itself. Perhaps our implementation
240 treats such values as a special case. In order to be confident that we
241 are checking this case, we added the following line:
242 $tcon->label("less than one") if $x < 1;
244 In the property-check output, we can see what percentage of the trials
246 checked this case:
247 1..1
249 ok 1 - 'my_sqrt satisfies defn of square root' (1000 attempts)
250 # 1% less than one
251 Trivial cases
253 Random-input generators may create some inputs that are trivial and
254 don't provide much testing value. To make it easy to label such cases,
255 you can use the following from within your behavior tests:
256 $tcon->trivial if ... ;
258 The above is exactly equivalent to the following:
260 $tcon->label("trivial") if ... ;
262
264 Test::LectroTest::Generator describes the many generators and generator
265 combinators that you can use to define the test or condition spaces
266 that you want LectroTest to search for bugs.
267 Test::LectroTest::TestRunner describes the objects that check your
269 properties and tells you how to turn their control knobs. You'll want
270 to look here if you're interested in customizing the testing procedure.
271
273 The special syntax used to specify generator bindings relies upon a
274 source filter (see Filter::Util::Call). If you don't want to use the
275 syntax, you can disable the filter like so:
276 use Test::LectroTest::Property qw( NO_FILTER );
278
280 Tom Moertel (tom@moertel.com)
281
283 The LectroTest project was inspired by Haskell's QuickCheck module by
284 Koen Claessen and John Hughes:
285 http://www.cs.chalmers.se/~rjmh/QuickCheck/.
286
288 Copyright (c) 2004-13 by Thomas G Moertel. All rights reserved.
289 This program is free software; you can redistribute it and/or modify it
291 under the same terms as Perl itself.
292perl v5.30.0 2019-07-26 Test::LectroTest::Property(3)