1Test::Tutorial(3) User Contributed Perl Documentation Test::Tutorial(3)
2
6 Test::Tutorial - A tutorial about writing really basic tests
7
9 AHHHHHHH!!!! NOT TESTING! Anything but testing! Beat me, whip me,
10 send me to Detroit, but don't make me write tests!
11 *sob*
13 Besides, I don't know how to write the damned things.
15 Is this you? Is writing tests right up there with writing
17 documentation and having your fingernails pulled out? Did you open up
18 a test and read
19 ######## We start with some black magic
21 and decide that's quite enough for you?
23 It's ok. That's all gone now. We've done all the black magic for you.
25 And here are the tricks...
26 Nuts and bolts of testing.
28 Here's the most basic test program.
29 #!/usr/bin/perl -w
31 print "1..1\n";
33 print 1 + 1 == 2 ? "ok 1\n" : "not ok 1\n";
35 Because 1 + 1 is 2, it prints:
37 1..1
39 ok 1
40 What this says is: 1..1 "I'm going to run one test." [1] "ok 1" "The
42 first test passed". And that's about all magic there is to testing.
43 Your basic unit of testing is the ok. For each thing you test, an "ok"
44 is printed. Simple. Test::Harness interprets your test results to
45 determine if you succeeded or failed (more on that later).
46 Writing all these print statements rapidly gets tedious. Fortunately,
48 there's Test::Simple. It has one function, "ok()".
49 #!/usr/bin/perl -w
51 use Test::Simple tests => 1;
53 ok( 1 + 1 == 2 );
55 That does the same thing as the previous code. "ok()" is the backbone
57 of Perl testing, and we'll be using it instead of roll-your-own from
58 here on. If "ok()" gets a true value, the test passes. False, it
59 fails.
60 #!/usr/bin/perl -w
62 use Test::Simple tests => 2;
64 ok( 1 + 1 == 2 );
65 ok( 2 + 2 == 5 );
66 From that comes:
68 1..2
70 ok 1
71 not ok 2
72 # Failed test (test.pl at line 5)
73 # Looks like you failed 1 tests of 2.
74 1..2 "I'm going to run two tests." This number is a plan. It helps to
76 ensure your test program ran all the way through and didn't die or skip
77 some tests. "ok 1" "The first test passed." "not ok 2" "The second
78 test failed". Test::Simple helpfully prints out some extra commentary
79 about your tests.
80 It's not scary. Come, hold my hand. We're going to give an example of
82 testing a module. For our example, we'll be testing a date library,
83 Date::ICal. It's on CPAN, so download a copy and follow along. [2]
84 Where to start?
86 This is the hardest part of testing, where do you start? People often
87 get overwhelmed at the apparent enormity of the task of testing a whole
88 module. The best place to start is at the beginning. Date::ICal is an
89 object-oriented module, and that means you start by making an object.
90 Test "new()".
91 #!/usr/bin/perl -w
93 # assume these two lines are in all subsequent examples
95 use strict;
96 use warnings;
97 use Test::Simple tests => 2;
99 use Date::ICal;
101 my $ical = Date::ICal->new; # create an object
103 ok( defined $ical ); # check that we got something
104 ok( $ical->isa('Date::ICal') ); # and it's the right class
105 Run that and you should get:
107 1..2
109 ok 1
110 ok 2
111 Congratulations! You've written your first useful test.
113 Names
115 That output isn't terribly descriptive, is it? When you have two tests
116 you can figure out which one is #2, but what if you have 102 tests?
117 Each test can be given a little descriptive name as the second argument
119 to "ok()".
120 use Test::Simple tests => 2;
122 ok( defined $ical, 'new() returned something' );
124 ok( $ical->isa('Date::ICal'), " and it's the right class" );
125 Now you'll see:
127 1..2
129 ok 1 - new() returned something
130 ok 2 - and it's the right class
131 Test the manual
133 The simplest way to build up a decent testing suite is to just test
134 what the manual says it does. [3] Let's pull something out of the
135 "SYNOPSIS" in Date::ICal and test that all its bits work.
136 #!/usr/bin/perl -w
138 use Test::Simple tests => 8;
140 use Date::ICal;
142 $ical = Date::ICal->new( year => 1964, month => 10, day => 16,
144 hour => 16, min => 12, sec => 47,
145 tz => '0530' );
146 ok( defined $ical, 'new() returned something' );
148 ok( $ical->isa('Date::ICal'), " and it's the right class" );
149 ok( $ical->sec == 47, ' sec()' );
150 ok( $ical->min == 12, ' min()' );
151 ok( $ical->hour == 16, ' hour()' );
152 ok( $ical->day == 17, ' day()' );
153 ok( $ical->month == 10, ' month()' );
154 ok( $ical->year == 1964, ' year()' );
155 Run that and you get:
157 1..8
159 ok 1 - new() returned something
160 ok 2 - and it's the right class
161 ok 3 - sec()
162 ok 4 - min()
163 ok 5 - hour()
164 not ok 6 - day()
165 # Failed test (- at line 16)
166 ok 7 - month()
167 ok 8 - year()
168 # Looks like you failed 1 tests of 8.
169 Whoops, a failure! [4] Test::Simple helpfully lets us know on what line
171 the failure occurred, but not much else. We were supposed to get 17,
172 but we didn't. What did we get?? Dunno. You could re-run the test in
173 the debugger or throw in some print statements to find out.
174 Instead, switch from Test::Simple to Test::More. Test::More does
176 everything Test::Simple does, and more! In fact, Test::More does
177 things exactly the way Test::Simple does. You can literally swap
178 Test::Simple out and put Test::More in its place. That's just what
179 we're going to do.
180 Test::More does more than Test::Simple. The most important difference
182 at this point is it provides more informative ways to say "ok".
183 Although you can write almost any test with a generic "ok()", it can't
184 tell you what went wrong. The "is()" function lets us declare that
185 something is supposed to be the same as something else:
186 use Test::More tests => 8;
188 use Date::ICal;
190 $ical = Date::ICal->new( year => 1964, month => 10, day => 16,
192 hour => 16, min => 12, sec => 47,
193 tz => '0530' );
194 ok( defined $ical, 'new() returned something' );
196 ok( $ical->isa('Date::ICal'), " and it's the right class" );
197 is( $ical->sec, 47, ' sec()' );
198 is( $ical->min, 12, ' min()' );
199 is( $ical->hour, 16, ' hour()' );
200 is( $ical->day, 17, ' day()' );
201 is( $ical->month, 10, ' month()' );
202 is( $ical->year, 1964, ' year()' );
203 "Is "$ical->sec" 47?" "Is "$ical->min" 12?" With "is()" in place, you
205 get more information:
206 1..8
208 ok 1 - new() returned something
209 ok 2 - and it's the right class
210 ok 3 - sec()
211 ok 4 - min()
212 ok 5 - hour()
213 not ok 6 - day()
214 # Failed test (- at line 16)
215 # got: '16'
216 # expected: '17'
217 ok 7 - month()
218 ok 8 - year()
219 # Looks like you failed 1 tests of 8.
220 Aha. "$ical->day" returned 16, but we expected 17. A quick check shows
222 that the code is working fine, we made a mistake when writing the
223 tests. Change it to:
224 is( $ical->day, 16, ' day()' );
226 ... and everything works.
228 Any time you're doing a "this equals that" sort of test, use "is()".
230 It even works on arrays. The test is always in scalar context, so you
231 can test how many elements are in an array this way. [5]
232 is( @foo, 5, 'foo has 5 elements' );
234 Sometimes the tests are wrong
236 This brings up a very important lesson. Code has bugs. Tests are
237 code. Ergo, tests have bugs. A failing test could mean a bug in the
238 code, but don't discount the possibility that the test is wrong.
239 On the flip side, don't be tempted to prematurely declare a test
241 incorrect just because you're having trouble finding the bug.
242 Invalidating a test isn't something to be taken lightly, and don't use
243 it as a cop out to avoid work.
244 Testing lots of values
246 We're going to be wanting to test a lot of dates here, trying to trick
247 the code with lots of different edge cases. Does it work before 1970?
248 After 2038? Before 1904? Do years after 10,000 give it trouble? Does
249 it get leap years right? We could keep repeating the code above, or we
250 could set up a little try/expect loop.
251 use Test::More tests => 32;
253 use Date::ICal;
254 my %ICal_Dates = (
256 # An ICal string And the year, month, day
257 # hour, minute and second we expect.
258 '19971024T120000' => # from the docs.
259 [ 1997, 10, 24, 12, 0, 0 ],
260 '20390123T232832' => # after the Unix epoch
261 [ 2039, 1, 23, 23, 28, 32 ],
262 '19671225T000000' => # before the Unix epoch
263 [ 1967, 12, 25, 0, 0, 0 ],
264 '18990505T232323' => # before the MacOS epoch
265 [ 1899, 5, 5, 23, 23, 23 ],
266 );
267 while( my($ical_str, $expect) = each %ICal_Dates ) {
270 my $ical = Date::ICal->new( ical => $ical_str );
271 ok( defined $ical, "new(ical => '$ical_str')" );
273 ok( $ical->isa('Date::ICal'), " and it's the right class" );
274 is( $ical->year, $expect->[0], ' year()' );
276 is( $ical->month, $expect->[1], ' month()' );
277 is( $ical->day, $expect->[2], ' day()' );
278 is( $ical->hour, $expect->[3], ' hour()' );
279 is( $ical->min, $expect->[4], ' min()' );
280 is( $ical->sec, $expect->[5], ' sec()' );
281 }
282 Now we can test bunches of dates by just adding them to %ICal_Dates.
284 Now that it's less work to test with more dates, you'll be inclined to
285 just throw more in as you think of them. Only problem is, every time
286 we add to that we have to keep adjusting the "use Test::More tests =>
287 ##" line. That can rapidly get annoying. There are ways to make this
288 work better.
289 First, we can calculate the plan dynamically using the "plan()"
291 function.
292 use Test::More;
294 use Date::ICal;
295 my %ICal_Dates = (
297 ...same as before...
298 );
299 # For each key in the hash we're running 8 tests.
301 plan tests => keys(%ICal_Dates) * 8;
302 ...and then your tests...
304 To be even more flexible, use "done_testing". This means we're just
306 running some tests, don't know how many. [6]
307 use Test::More; # instead of tests => 32
309 ... # tests here
311 done_testing(); # reached the end safely
313 If you don't specify a plan, Test::More expects to see "done_testing()"
315 before your program exits. It will warn you if you forget it. You can
316 give "done_testing()" an optional number of tests you expected to run,
317 and if the number ran differs, Test::More will give you another kind of
318 warning.
319 Informative names
321 Take a look at the line:
322 ok( defined $ical, "new(ical => '$ical_str')" );
324 We've added more detail about what we're testing and the ICal string
326 itself we're trying out to the name. So you get results like:
327 ok 25 - new(ical => '19971024T120000')
329 ok 26 - and it's the right class
330 ok 27 - year()
331 ok 28 - month()
332 ok 29 - day()
333 ok 30 - hour()
334 ok 31 - min()
335 ok 32 - sec()
336 If something in there fails, you'll know which one it was and that will
338 make tracking down the problem easier. Try to put a bit of debugging
339 information into the test names.
340 Describe what the tests test, to make debugging a failed test easier
342 for you or for the next person who runs your test.
343 Skipping tests
345 Poking around in the existing Date::ICal tests, I found this in
346 t/01sanity.t [7]
347 #!/usr/bin/perl -w
349 use Test::More tests => 7;
351 use Date::ICal;
352 # Make sure epoch time is being handled sanely.
354 my $t1 = Date::ICal->new( epoch => 0 );
355 is( $t1->epoch, 0, "Epoch time of 0" );
356 # XXX This will only work on unix systems.
358 is( $t1->ical, '19700101Z', " epoch to ical" );
359 is( $t1->year, 1970, " year()" );
361 is( $t1->month, 1, " month()" );
362 is( $t1->day, 1, " day()" );
363 # like the tests above, but starting with ical instead of epoch
365 my $t2 = Date::ICal->new( ical => '19700101Z' );
366 is( $t2->ical, '19700101Z', "Start of epoch in ICal notation" );
367 is( $t2->epoch, 0, " and back to ICal" );
369 The beginning of the epoch is different on most non-Unix operating
371 systems [8]. Even though Perl smooths out the differences for the most
372 part, certain ports do it differently. MacPerl is one off the top of
373 my head. [9] Rather than putting a comment in the test and hoping
374 someone will read the test while debugging the failure, we can
375 explicitly say it's never going to work and skip the test.
376 use Test::More tests => 7;
378 use Date::ICal;
379 # Make sure epoch time is being handled sanely.
381 my $t1 = Date::ICal->new( epoch => 0 );
382 is( $t1->epoch, 0, "Epoch time of 0" );
383 SKIP: {
385 skip('epoch to ICal not working on Mac OS', 6)
386 if $^O eq 'MacOS';
387 is( $t1->ical, '19700101Z', " epoch to ical" );
389 is( $t1->year, 1970, " year()" );
391 is( $t1->month, 1, " month()" );
392 is( $t1->day, 1, " day()" );
393 # like the tests above, but starting with ical instead of epoch
395 my $t2 = Date::ICal->new( ical => '19700101Z' );
396 is( $t2->ical, '19700101Z', "Start of epoch in ICal notation" );
397 is( $t2->epoch, 0, " and back to ICal" );
399 }
400 A little bit of magic happens here. When running on anything but
402 MacOS, all the tests run normally. But when on MacOS, "skip()" causes
403 the entire contents of the SKIP block to be jumped over. It never
404 runs. Instead, "skip()" prints special output that tells Test::Harness
405 that the tests have been skipped.
406 1..7
408 ok 1 - Epoch time of 0
409 ok 2 # skip epoch to ICal not working on MacOS
410 ok 3 # skip epoch to ICal not working on MacOS
411 ok 4 # skip epoch to ICal not working on MacOS
412 ok 5 # skip epoch to ICal not working on MacOS
413 ok 6 # skip epoch to ICal not working on MacOS
414 ok 7 # skip epoch to ICal not working on MacOS
415 This means your tests won't fail on MacOS. This means fewer emails
417 from MacPerl users telling you about failing tests that you know will
418 never work. You've got to be careful with skip tests. These are for
419 tests which don't work and never will. It is not for skipping genuine
420 bugs (we'll get to that in a moment).
421 The tests are wholly and completely skipped. [10] This will work.
423 SKIP: {
425 skip("I don't wanna die!");
426 die, die, die, die, die;
428 }
429 Todo tests
431 While thumbing through the Date::ICal man page, I came across this:
432 ical
434 $ical_string = $ical->ical;
436 Retrieves, or sets, the date on the object, using any
438 valid ICal date/time string.
439 "Retrieves or sets". Hmmm. I didn't see a test for using "ical()" to
441 set the date in the Date::ICal test suite. So I wrote one:
442 use Test::More tests => 1;
444 use Date::ICal;
445 my $ical = Date::ICal->new;
447 $ical->ical('20201231Z');
448 is( $ical->ical, '20201231Z', 'Setting via ical()' );
449 Run that. I saw:
451 1..1
453 not ok 1 - Setting via ical()
454 # Failed test (- at line 6)
455 # got: '20010814T233649Z'
456 # expected: '20201231Z'
457 # Looks like you failed 1 tests of 1.
458 Whoops! Looks like it's unimplemented. Assume you don't have the time
460 to fix this. [11] Normally, you'd just comment out the test and put a
461 note in a todo list somewhere. Instead, explicitly state "this test
462 will fail" by wrapping it in a "TODO" block:
463 use Test::More tests => 1;
465 TODO: {
467 local $TODO = 'ical($ical) not yet implemented';
468 my $ical = Date::ICal->new;
470 $ical->ical('20201231Z');
471 is( $ical->ical, '20201231Z', 'Setting via ical()' );
473 }
474 Now when you run, it's a little different:
476 1..1
478 not ok 1 - Setting via ical() # TODO ical($ical) not yet implemented
479 # got: '20010822T201551Z'
480 # expected: '20201231Z'
481 Test::More doesn't say "Looks like you failed 1 tests of 1". That '#
483 TODO' tells Test::Harness "this is supposed to fail" and it treats a
484 failure as a successful test. You can write tests even before you've
485 fixed the underlying code.
486 If a TODO test passes, Test::Harness will report it "UNEXPECTEDLY
488 SUCCEEDED". When that happens, remove the TODO block with "local
489 $TODO" and turn it into a real test.
490 Testing with taint mode.
492 Taint mode is a funny thing. It's the globalest of all global
493 features. Once you turn it on, it affects all code in your program and
494 all modules used (and all the modules they use). If a single piece of
495 code isn't taint clean, the whole thing explodes. With that in mind,
496 it's very important to ensure your module works under taint mode.
497 It's very simple to have your tests run under taint mode. Just throw a
499 "-T" into the "#!" line. Test::Harness will read the switches in "#!"
500 and use them to run your tests.
501 #!/usr/bin/perl -Tw
503 ...test normally here...
505 When you say "make test" it will run with taint mode on.
507
509 1. The first number doesn't really mean anything, but it has to be 1.
510 It's the second number that's important.
511 2. For those following along at home, I'm using version 1.31. It has
513 some bugs, which is good -- we'll uncover them with our tests.
514 3. You can actually take this one step further and test the manual
516 itself. Have a look at Test::Inline (formerly Pod::Tests).
517 4. Yes, there's a mistake in the test suite. What! Me, contrived?
519 5. We'll get to testing the contents of lists later.
521 6. But what happens if your test program dies halfway through?! Since
523 we didn't say how many tests we're going to run, how can we know it
524 failed? No problem, Test::More employs some magic to catch that
525 death and turn the test into a failure, even if every test passed
526 up to that point.
527 7. I cleaned it up a little.
529 8. Most Operating Systems record time as the number of seconds since a
531 certain date. This date is the beginning of the epoch. Unix's
532 starts at midnight January 1st, 1970 GMT.
533 9. MacOS's epoch is midnight January 1st, 1904. VMS's is midnight,
535 November 17th, 1858, but vmsperl emulates the Unix epoch so it's
536 not a problem.
537 10. As long as the code inside the SKIP block at least compiles.
539 Please don't ask how. No, it's not a filter.
540 11. Do NOT be tempted to use TODO tests as a way to avoid fixing simple
542 bugs!
543
545 Michael G Schwern <schwern@pobox.com> and the perl-qa dancers!
546
548 Chad Granum <exodist@cpan.org>
549
551 Copyright 2001 by Michael G Schwern <schwern@pobox.com>.
552 This documentation is free; you can redistribute it and/or modify it
554 under the same terms as Perl itself.
555 Irrespective of its distribution, all code examples in these files are
557 hereby placed into the public domain. You are permitted and encouraged
558 to use this code in your own programs for fun or for profit as you see
559 fit. A simple comment in the code giving credit would be courteous but
560 is not required.
561perl v5.32.0 2020-10-15 Test::Tutorial(3)