1Test::Tutorial(3pm) Perl Programmers Reference Guide Test::Tutorial(3pm)
2
6 Test::Tutorial - A tutorial about writing really basic tests
7
9 AHHHHHHH!!!! NOT TESTING! Anything but testing! Beat me, whip me,
10 send me to Detroit, but don't make me write tests!
11 *sob*
13 Besides, I don't know how to write the damned things.
15 Is this you? Is writing tests right up there with writing documenta‐
17 tion and having your fingernails pulled out? Did you open up a test
18 and read
19 ######## We start with some black magic
21 and decide that's quite enough for you?
23 It's ok. That's all gone now. We've done all the black magic for you.
25 And here are the tricks...
26 Nuts and bolts of testing.
28 Here's the most basic test program.
30 #!/usr/bin/perl -w
32 print "1..1\n";
34 print 1 + 1 == 2 ? "ok 1\n" : "not ok 1\n";
36 since 1 + 1 is 2, it prints:
38 1..1
40 ok 1
41 What this says is: 1..1 "I'm going to run one test." [1] "ok 1" "The
43 first test passed". And that's about all magic there is to testing.
44 Your basic unit of testing is the ok. For each thing you test, an "ok"
45 is printed. Simple. Test::Harness interprets your test results to
46 determine if you succeeded or failed (more on that later).
47 Writing all these print statements rapidly gets tedious. Fortunately,
49 there's Test::Simple. It has one function, "ok()".
50 #!/usr/bin/perl -w
52 use Test::Simple tests => 1;
54 ok( 1 + 1 == 2 );
56 and that does the same thing as the code above. "ok()" is the backbone
58 of Perl testing, and we'll be using it instead of roll-your-own from
59 here on. If "ok()" gets a true value, the test passes. False, it
60 fails.
61 #!/usr/bin/perl -w
63 use Test::Simple tests => 2;
65 ok( 1 + 1 == 2 );
66 ok( 2 + 2 == 5 );
67 from that comes
69 1..2
71 ok 1
72 not ok 2
73 # Failed test (test.pl at line 5)
74 # Looks like you failed 1 tests of 2.
75 1..2 "I'm going to run two tests." This number is used to ensure your
77 test program ran all the way through and didn't die or skip some tests.
78 "ok 1" "The first test passed." "not ok 2" "The second test failed".
79 Test::Simple helpfully prints out some extra commentary about your
80 tests.
81 It's not scary. Come, hold my hand. We're going to give an example of
83 testing a module. For our example, we'll be testing a date library,
84 Date::ICal. It's on CPAN, so download a copy and follow along. [2]
85 Where to start?
87 This is the hardest part of testing, where do you start? People often
89 get overwhelmed at the apparent enormity of the task of testing a whole
90 module. Best place to start is at the beginning. Date::ICal is an
91 object-oriented module, and that means you start by making an object.
92 So we test "new()".
93 #!/usr/bin/perl -w
95 use Test::Simple tests => 2;
97 use Date::ICal;
99 my $ical = Date::ICal->new; # create an object
101 ok( defined $ical ); # check that we got something
102 ok( $ical->isa('Date::ICal') ); # and it's the right class
103 run that and you should get:
105 1..2
107 ok 1
108 ok 2
109 congratulations, you've written your first useful test.
111 Names
113 That output isn't terribly descriptive, is it? When you have two tests
115 you can figure out which one is #2, but what if you have 102?
116 Each test can be given a little descriptive name as the second argument
118 to "ok()".
119 use Test::Simple tests => 2;
121 ok( defined $ical, 'new() returned something' );
123 ok( $ical->isa('Date::ICal'), " and it's the right class" );
124 So now you'd see...
126 1..2
128 ok 1 - new() returned something
129 ok 2 - and it's the right class
130 Test the manual
132 Simplest way to build up a decent testing suite is to just test what
134 the manual says it does. [3] Let's pull something out of the "SYNOPSIS"
135 in Date::ICal and test that all its bits work.
136 #!/usr/bin/perl -w
138 use Test::Simple tests => 8;
140 use Date::ICal;
142 $ical = Date::ICal->new( year => 1964, month => 10, day => 16,
144 hour => 16, min => 12, sec => 47,
145 tz => '0530' );
146 ok( defined $ical, 'new() returned something' );
148 ok( $ical->isa('Date::ICal'), " and it's the right class" );
149 ok( $ical->sec == 47, ' sec()' );
150 ok( $ical->min == 12, ' min()' );
151 ok( $ical->hour == 16, ' hour()' );
152 ok( $ical->day == 17, ' day()' );
153 ok( $ical->month == 10, ' month()' );
154 ok( $ical->year == 1964, ' year()' );
155 run that and you get:
157 1..8
159 ok 1 - new() returned something
160 ok 2 - and it's the right class
161 ok 3 - sec()
162 ok 4 - min()
163 ok 5 - hour()
164 not ok 6 - day()
165 # Failed test (- at line 16)
166 ok 7 - month()
167 ok 8 - year()
168 # Looks like you failed 1 tests of 8.
169 Whoops, a failure! [4] Test::Simple helpfully lets us know on what line
171 the failure occurred, but not much else. We were supposed to get 17,
172 but we didn't. What did we get?? Dunno. We'll have to re-run the
173 test in the debugger or throw in some print statements to find out.
174 Instead, we'll switch from Test::Simple to Test::More. Test::More does
176 everything Test::Simple does, and more! In fact, Test::More does
177 things exactly the way Test::Simple does. You can literally swap
178 Test::Simple out and put Test::More in its place. That's just what
179 we're going to do.
180 Test::More does more than Test::Simple. The most important difference
182 at this point is it provides more informative ways to say "ok".
183 Although you can write almost any test with a generic "ok()", it can't
184 tell you what went wrong. Instead, we'll use the "is()" function,
185 which lets us declare that something is supposed to be the same as
186 something else:
187 #!/usr/bin/perl -w
189 use Test::More tests => 8;
191 use Date::ICal;
193 $ical = Date::ICal->new( year => 1964, month => 10, day => 16,
195 hour => 16, min => 12, sec => 47,
196 tz => '0530' );
197 ok( defined $ical, 'new() returned something' );
199 ok( $ical->isa('Date::ICal'), " and it's the right class" );
200 is( $ical->sec, 47, ' sec()' );
201 is( $ical->min, 12, ' min()' );
202 is( $ical->hour, 16, ' hour()' );
203 is( $ical->day, 17, ' day()' );
204 is( $ical->month, 10, ' month()' );
205 is( $ical->year, 1964, ' year()' );
206 "Is "$ical->sec" 47?" "Is "$ical->min" 12?" With "is()" in place, you
208 get some more information
209 1..8
211 ok 1 - new() returned something
212 ok 2 - and it's the right class
213 ok 3 - sec()
214 ok 4 - min()
215 ok 5 - hour()
216 not ok 6 - day()
217 # Failed test (- at line 16)
218 # got: '16'
219 # expected: '17'
220 ok 7 - month()
221 ok 8 - year()
222 # Looks like you failed 1 tests of 8.
223 letting us know that "$ical->day" returned 16, but we expected 17. A
225 quick check shows that the code is working fine, we made a mistake when
226 writing up the tests. Just change it to:
227 is( $ical->day, 16, ' day()' );
229 and everything works.
231 So any time you're doing a "this equals that" sort of test, use "is()".
233 It even works on arrays. The test is always in scalar context, so you
234 can test how many elements are in a list this way. [5]
235 is( @foo, 5, 'foo has 5 elements' );
237 Sometimes the tests are wrong
239 Which brings us to a very important lesson. Code has bugs. Tests are
241 code. Ergo, tests have bugs. A failing test could mean a bug in the
242 code, but don't discount the possibility that the test is wrong.
243 On the flip side, don't be tempted to prematurely declare a test incor‐
245 rect just because you're having trouble finding the bug. Invalidating
246 a test isn't something to be taken lightly, and don't use it as a cop
247 out to avoid work.
248 Testing lots of values
250 We're going to be wanting to test a lot of dates here, trying to trick
252 the code with lots of different edge cases. Does it work before 1970?
253 After 2038? Before 1904? Do years after 10,000 give it trouble? Does
254 it get leap years right? We could keep repeating the code above, or we
255 could set up a little try/expect loop.
256 use Test::More tests => 32;
258 use Date::ICal;
259 my %ICal_Dates = (
261 # An ICal string And the year, month, date
262 # hour, minute and second we expect.
263 '19971024T120000' => # from the docs.
264 [ 1997, 10, 24, 12, 0, 0 ],
265 '20390123T232832' => # after the Unix epoch
266 [ 2039, 1, 23, 23, 28, 32 ],
267 '19671225T000000' => # before the Unix epoch
268 [ 1967, 12, 25, 0, 0, 0 ],
269 '18990505T232323' => # before the MacOS epoch
270 [ 1899, 5, 5, 23, 23, 23 ],
271 );
272 while( my($ical_str, $expect) = each %ICal_Dates ) {
274 my $ical = Date::ICal->new( ical => $ical_str );
275 ok( defined $ical, "new(ical => '$ical_str')" );
277 ok( $ical->isa('Date::ICal'), " and it's the right class" );
278 is( $ical->year, $expect->[0], ' year()' );
280 is( $ical->month, $expect->[1], ' month()' );
281 is( $ical->day, $expect->[2], ' day()' );
282 is( $ical->hour, $expect->[3], ' hour()' );
283 is( $ical->min, $expect->[4], ' min()' );
284 is( $ical->sec, $expect->[5], ' sec()' );
285 }
286 So now we can test bunches of dates by just adding them to %ICal_Dates.
288 Now that it's less work to test with more dates, you'll be inclined to
289 just throw more in as you think of them. Only problem is, every time
290 we add to that we have to keep adjusting the "use Test::More tests =>
291 ##" line. That can rapidly get annoying. There's two ways to make
292 this work better.
293 First, we can calculate the plan dynamically using the "plan()" func‐
295 tion.
296 use Test::More;
298 use Date::ICal;
299 my %ICal_Dates = (
301 ...same as before...
302 );
303 # For each key in the hash we're running 8 tests.
305 plan tests => keys %ICal_Dates * 8;
306 Or to be even more flexible, we use "no_plan". This means we're just
308 running some tests, don't know how many. [6]
309 use Test::More 'no_plan'; # instead of tests => 32
311 now we can just add tests and not have to do all sorts of math to fig‐
313 ure out how many we're running.
314 Informative names
316 Take a look at this line here
318 ok( defined $ical, "new(ical => '$ical_str')" );
320 we've added more detail about what we're testing and the ICal string
322 itself we're trying out to the name. So you get results like:
323 ok 25 - new(ical => '19971024T120000')
325 ok 26 - and it's the right class
326 ok 27 - year()
327 ok 28 - month()
328 ok 29 - day()
329 ok 30 - hour()
330 ok 31 - min()
331 ok 32 - sec()
332 if something in there fails, you'll know which one it was and that will
334 make tracking down the problem easier. So try to put a bit of debug‐
335 ging information into the test names.
336 Describe what the tests test, to make debugging a failed test easier
338 for you or for the next person who runs your test.
339 Skipping tests
341 Poking around in the existing Date::ICal tests, I found this in
343 t/01sanity.t [7]
344 #!/usr/bin/perl -w
346 use Test::More tests => 7;
348 use Date::ICal;
349 # Make sure epoch time is being handled sanely.
351 my $t1 = Date::ICal->new( epoch => 0 );
352 is( $t1->epoch, 0, "Epoch time of 0" );
353 # XXX This will only work on unix systems.
355 is( $t1->ical, '19700101Z', " epoch to ical" );
356 is( $t1->year, 1970, " year()" );
358 is( $t1->month, 1, " month()" );
359 is( $t1->day, 1, " day()" );
360 # like the tests above, but starting with ical instead of epoch
362 my $t2 = Date::ICal->new( ical => '19700101Z' );
363 is( $t2->ical, '19700101Z', "Start of epoch in ICal notation" );
364 is( $t2->epoch, 0, " and back to ICal" );
366 The beginning of the epoch is different on most non-Unix operating sys‐
368 tems [8]. Even though Perl smooths out the differences for the most
369 part, certain ports do it differently. MacPerl is one off the top of
370 my head. [9] We know this will never work on MacOS. So rather than
371 just putting a comment in the test, we can explicitly say it's never
372 going to work and skip the test.
373 use Test::More tests => 7;
375 use Date::ICal;
376 # Make sure epoch time is being handled sanely.
378 my $t1 = Date::ICal->new( epoch => 0 );
379 is( $t1->epoch, 0, "Epoch time of 0" );
380 SKIP: {
382 skip('epoch to ICal not working on MacOS', 6)
383 if $^O eq 'MacOS';
384 is( $t1->ical, '19700101Z', " epoch to ical" );
386 is( $t1->year, 1970, " year()" );
388 is( $t1->month, 1, " month()" );
389 is( $t1->day, 1, " day()" );
390 # like the tests above, but starting with ical instead of epoch
392 my $t2 = Date::ICal->new( ical => '19700101Z' );
393 is( $t2->ical, '19700101Z', "Start of epoch in ICal notation" );
394 is( $t2->epoch, 0, " and back to ICal" );
396 }
397 A little bit of magic happens here. When running on anything but Mac‐
399 OS, all the tests run normally. But when on MacOS, "skip()" causes the
400 entire contents of the SKIP block to be jumped over. It's never run.
401 Instead, it prints special output that tells Test::Harness that the
402 tests have been skipped.
403 1..7
405 ok 1 - Epoch time of 0
406 ok 2 # skip epoch to ICal not working on MacOS
407 ok 3 # skip epoch to ICal not working on MacOS
408 ok 4 # skip epoch to ICal not working on MacOS
409 ok 5 # skip epoch to ICal not working on MacOS
410 ok 6 # skip epoch to ICal not working on MacOS
411 ok 7 # skip epoch to ICal not working on MacOS
412 This means your tests won't fail on MacOS. This means less emails from
414 MacPerl users telling you about failing tests that you know will never
415 work. You've got to be careful with skip tests. These are for tests
416 which don't work and never will. It is not for skipping genuine bugs
417 (we'll get to that in a moment).
418 The tests are wholly and completely skipped. [10] This will work.
420 SKIP: {
422 skip("I don't wanna die!");
423 die, die, die, die, die;
425 }
426 Todo tests
428 Thumbing through the Date::ICal man page, I came across this:
430 ical
432 $ical_string = $ical->ical;
434 Retrieves, or sets, the date on the object, using any
436 valid ICal date/time string.
437 "Retrieves or sets". Hmmm, didn't see a test for using "ical()" to set
439 the date in the Date::ICal test suite. So I'll write one.
440 use Test::More tests => 1;
442 use Date::ICal;
443 my $ical = Date::ICal->new;
445 $ical->ical('20201231Z');
446 is( $ical->ical, '20201231Z', 'Setting via ical()' );
447 run that and I get
449 1..1
451 not ok 1 - Setting via ical()
452 # Failed test (- at line 6)
453 # got: '20010814T233649Z'
454 # expected: '20201231Z'
455 # Looks like you failed 1 tests of 1.
456 Whoops! Looks like it's unimplemented. Let's assume we don't have the
458 time to fix this. [11] Normally, you'd just comment out the test and
459 put a note in a todo list somewhere. Instead, we're going to explic‐
460 itly state "this test will fail" by wrapping it in a "TODO" block.
461 use Test::More tests => 1;
463 TODO: {
465 local $TODO = 'ical($ical) not yet implemented';
466 my $ical = Date::ICal->new;
468 $ical->ical('20201231Z');
469 is( $ical->ical, '20201231Z', 'Setting via ical()' );
471 }
472 Now when you run, it's a little different:
474 1..1
476 not ok 1 - Setting via ical() # TODO ical($ical) not yet implemented
477 # got: '20010822T201551Z'
478 # expected: '20201231Z'
479 Test::More doesn't say "Looks like you failed 1 tests of 1". That '#
481 TODO' tells Test::Harness "this is supposed to fail" and it treats a
482 failure as a successful test. So you can write tests even before
483 you've fixed the underlying code.
484 If a TODO test passes, Test::Harness will report it "UNEXPECTEDLY SUC‐
486 CEEDED". When that happens, you simply remove the TODO block with
487 "local $TODO" and turn it into a real test.
488 Testing with taint mode.
490 Taint mode is a funny thing. It's the globalest of all global fea‐
492 tures. Once you turn it on, it affects all code in your program and
493 all modules used (and all the modules they use). If a single piece of
494 code isn't taint clean, the whole thing explodes. With that in mind,
495 it's very important to ensure your module works under taint mode.
496 It's very simple to have your tests run under taint mode. Just throw a
498 "-T" into the "#!" line. Test::Harness will read the switches in "#!"
499 and use them to run your tests.
500 #!/usr/bin/perl -Tw
502 ...test normally here...
504 So when you say "make test" it will be run with taint mode and warnings
506 on.
507
509 1 The first number doesn't really mean anything, but it has to be 1.
510 It's the second number that's important.
511 2 For those following along at home, I'm using version 1.31. It has
513 some bugs, which is good -- we'll uncover them with our tests.
514 3 You can actually take this one step further and test the manual
516 itself. Have a look at Test::Inline (formerly Pod::Tests).
517 4 Yes, there's a mistake in the test suite. What! Me, contrived?
519 5 We'll get to testing the contents of lists later.
521 6 But what happens if your test program dies halfway through?! Since
523 we didn't say how many tests we're going to run, how can we know it
524 failed? No problem, Test::More employs some magic to catch that
525 death and turn the test into a failure, even if every test passed
526 up to that point.
527 7 I cleaned it up a little.
529 8 Most Operating Systems record time as the number of seconds since a
531 certain date. This date is the beginning of the epoch. Unix's
532 starts at midnight January 1st, 1970 GMT.
533 9 MacOS's epoch is midnight January 1st, 1904. VMS's is midnight,
535 November 17th, 1858, but vmsperl emulates the Unix epoch so it's
536 not a problem.
537 10 As long as the code inside the SKIP block at least compiles.
539 Please don't ask how. No, it's not a filter.
540 11 Do NOT be tempted to use TODO tests as a way to avoid fixing simple
542 bugs!
543
545 Michael G Schwern <schwern@pobox.com> and the perl-qa dancers!
546
548 Copyright 2001 by Michael G Schwern <schwern@pobox.com>.
549 This documentation is free; you can redistribute it and/or modify it
551 under the same terms as Perl itself.
552 Irrespective of its distribution, all code examples in these files are
554 hereby placed into the public domain. You are permitted and encouraged
555 to use this code in your own programs for fun or for profit as you see
556 fit. A simple comment in the code giving credit would be courteous but
557 is not required.
558perl v5.8.8 2001-09-21 Test::Tutorial(3pm)