1Test::LectroTest::TutorUisaelr(3C)ontributed Perl DocumeTnetsatt:i:oLnectroTest::Tutorial(3)
2
6 Test::LectroTest::Tutorial - How to use LectroTest to test your
7 software
8
10 version 0.5001
11
13 LectroTest is an automated, specification-based testing system. To use
14 it, declare properties that specify the expected behavior of your
15 software. Then invoke LectroTest to test whether those properties
16 hold.
17 LectroTest does this by running repeated random trials against your
19 software. If LectroTest finds that a property doesn't hold, it emits
20 the counterexample that "broke" your software. You can then plug the
21 counterexample into your software to debug the problem. (It's also a
22 good idea to add the counterexample to your list of regression tests.)
23
25 Think of your software's behavior as a haystack that you're searching
26 for needles. Each error is a needle. You want to find the needles and
27 remove of them. LectroTest will search the haystack for you -- it's
28 nice that way -- but first you must tell it about the shape of the
29 haystack and how to recognize a needle when it sees one.
30 The Haystack
32 The shape of the haystack is defined by a set of "generator bindings,"
33 in which variables are bound to the output of value generators:
34 x <- Int, c <- Char( charset=>"A-Z" )
36 The above can be read, "For all integers x and all characters c in the
38 range A through Z." The idea is that each unique instance of the pair
39 (x, c) specifies a point in the haystack that we can search for
40 needles.
41 The Needle Recognizer
43 The "needle recognizer" is defined by a snippet of code that uses the
44 bound variables to inspect a given point in the haystack. It returns a
45 "thumbs up" (true) if the point is free of needles or a "thumbs down"
46 (false) if it finds a needle:
47 the_thing_we_are_testing($x, $c) >= 0;
49 The above asserts for each point in the haystack that the output of the
51 function "the_thing_we_are_testing" must be non-negative.
52 Put them together to make a Property
54 The generator bindings and needle recognizer are combined to make a
55 property:
56 Property {
58 ##[ x <- Int, c <- Char( charset=>"A-Z" ) ]##
59 the_thing_we_are_testing($x, $c) >= 0;
60 }, name => "the_thing_we_are_testing(...) is non-negative";
61 You'll note that we also added a meaningful name. Although not
63 strictly required, it's an excellent practice that makes life easier.
64 (You'll also note that we placed the generator bindings inside of the
65 magic delimiters "##[ ]##". This tells Perl that our bindings are
66 bindings and not regular Perl code.)
67 We can read the above property like so: "For all integers x and all
69 characters c in the range A through Z, we assert that
70 "the_thing_we_are_testing" is non-negative."
71 Testing whether your Properties hold
73 After you define properties for your software, just add them to a small
74 Perl program that uses the Test::LectroTest module:
75 # MyProperties.l.t
77 use MyModule; # provides the_thing_we_are_testing
79 use Test::LectroTest;
80 Property {
82 ##[ x <- Int, c <- Char( charset=>"A-Z" ) ]##
83 the_thing_we_are_testing($x, $c) >= 0;
84 }, name => "the_thing_we_are_testing(...) is non-negative";
85 Then you can test your properties simply by running the program:
87 $ perl MyProperties.l.t
89 If your properties check out, you'll see something like this:
91 1..1
93 ok 1 - 'the_thing_we_are_testing(...) is non-negative' (1000 attempts)
94 If something goes wrong, however, LectroTest will tell you where it
96 happened:
97 1..1
99 not ok 1 - 'the_thing_we_are_testing(...) is non-negative' \
100 falsified in 23 attempts
101 # Counterexample:
102 # $x = 4
103 # $c = "R"
104 What this says is that at the point (x=4, c="R") in the haystack, there
106 is a needle (i.e., your property doesn't hold). With this information,
107 you can examine your code to determine the cause of the error.
108
110 Now that we have big-picture understanding of "LectroTesting," let's
111 try a few examples together.
112 [TODO: write the step-by-step tutorial examples. For now, take a look
114 at the slides from my LectroTest talk for two such examples. The
115 slides are available at the LectroTest Home.]
116
118 Test::LectroTest gives a quick overview of automatic, specification-
119 based testing with LectroTest.
120 Test::LectroTest::Property explains in detail what you can put inside
122 of your property specifications.
123 Test::LectroTest::Generator describes the many generators and generator
125 combinators that you can use to define the shapes of the haystacks you
126 encounter during your testing adventures.
127 Test::LectroTest::TestRunner describes the objects that check your
129 properties and tells you how to turn their control knobs. You'll want
130 to look here if you're interested in customizing the testing procedure.
131
133 Tom Moertel (tom@moertel.com)
134
136 The LectroTest project was inspired by Haskell's QuickCheck module by
137 Koen Claessen and John Hughes:
138 http://www.cs.chalmers.se/~rjmh/QuickCheck/.
139
141 Copyright (c) 2004-13 by Thomas G Moertel. All rights reserved.
142 This program is free software; you can redistribute it and/or modify it
144 under the same terms as Perl itself.
145perl v5.32.1 2021-01-27 Test::LectroTest::Tutorial(3)