1Test::Tutorial(3) User Contributed Perl Documentation Test::Tutorial(3)
2
6 Test::Tutorial - A tutorial about writing really basic tests
7
9 AHHHHHHH!!!! NOT TESTING! Anything but testing! Beat me, whip me,
10 send me to Detroit, but don't make me write tests!
11 *sob*
13 Besides, I don't know how to write the damned things.
15 Is this you? Is writing tests right up there with writing
17 documentation and having your fingernails pulled out? Did you open up
18 a test and read
19 ######## We start with some black magic
21 and decide that's quite enough for you?
23 It's ok. That's all gone now. We've done all the black magic for you.
25 And here are the tricks...
26 Nuts and bolts of testing.
28 Here's the most basic test program.
29 #!/usr/bin/perl -w
31 print "1..1\n";
33 print 1 + 1 == 2 ? "ok 1\n" : "not ok 1\n";
35 since 1 + 1 is 2, it prints:
37 1..1
39 ok 1
40 What this says is: 1..1 "I'm going to run one test." [1] "ok 1" "The
42 first test passed". And that's about all magic there is to testing.
43 Your basic unit of testing is the ok. For each thing you test, an "ok"
44 is printed. Simple. Test::Harness interprets your test results to
45 determine if you succeeded or failed (more on that later).
46 Writing all these print statements rapidly gets tedious. Fortunately,
48 there's Test::Simple. It has one function, "ok()".
49 #!/usr/bin/perl -w
51 use Test::Simple tests => 1;
53 ok( 1 + 1 == 2 );
55 and that does the same thing as the code above. "ok()" is the backbone
57 of Perl testing, and we'll be using it instead of roll-your-own from
58 here on. If "ok()" gets a true value, the test passes. False, it
59 fails.
60 #!/usr/bin/perl -w
62 use Test::Simple tests => 2;
64 ok( 1 + 1 == 2 );
65 ok( 2 + 2 == 5 );
66 from that comes
68 1..2
70 ok 1
71 not ok 2
72 # Failed test (test.pl at line 5)
73 # Looks like you failed 1 tests of 2.
74 1..2 "I'm going to run two tests." This number is used to ensure your
76 test program ran all the way through and didn't die or skip some tests.
77 "ok 1" "The first test passed." "not ok 2" "The second test failed".
78 Test::Simple helpfully prints out some extra commentary about your
79 tests.
80 It's not scary. Come, hold my hand. We're going to give an example of
82 testing a module. For our example, we'll be testing a date library,
83 Date::ICal. It's on CPAN, so download a copy and follow along. [2]
84 Where to start?
86 This is the hardest part of testing, where do you start? People often
87 get overwhelmed at the apparent enormity of the task of testing a whole
88 module. Best place to start is at the beginning. Date::ICal is an
89 object-oriented module, and that means you start by making an object.
90 So we test "new()".
91 #!/usr/bin/perl -w
93 use Test::Simple tests => 2;
95 use Date::ICal;
97 my $ical = Date::ICal->new; # create an object
99 ok( defined $ical ); # check that we got something
100 ok( $ical->isa('Date::ICal') ); # and it's the right class
101 run that and you should get:
103 1..2
105 ok 1
106 ok 2
107 congratulations, you've written your first useful test.
109 Names
111 That output isn't terribly descriptive, is it? When you have two tests
112 you can figure out which one is #2, but what if you have 102?
113 Each test can be given a little descriptive name as the second argument
115 to "ok()".
116 use Test::Simple tests => 2;
118 ok( defined $ical, 'new() returned something' );
120 ok( $ical->isa('Date::ICal'), " and it's the right class" );
121 So now you'd see...
123 1..2
125 ok 1 - new() returned something
126 ok 2 - and it's the right class
127 Test the manual
129 Simplest way to build up a decent testing suite is to just test what
130 the manual says it does. [3] Let's pull something out of the "SYNOPSIS"
131 in Date::ICal and test that all its bits work.
132 #!/usr/bin/perl -w
134 use Test::Simple tests => 8;
136 use Date::ICal;
138 $ical = Date::ICal->new( year => 1964, month => 10, day => 16,
140 hour => 16, min => 12, sec => 47,
141 tz => '0530' );
142 ok( defined $ical, 'new() returned something' );
144 ok( $ical->isa('Date::ICal'), " and it's the right class" );
145 ok( $ical->sec == 47, ' sec()' );
146 ok( $ical->min == 12, ' min()' );
147 ok( $ical->hour == 16, ' hour()' );
148 ok( $ical->day == 17, ' day()' );
149 ok( $ical->month == 10, ' month()' );
150 ok( $ical->year == 1964, ' year()' );
151 run that and you get:
153 1..8
155 ok 1 - new() returned something
156 ok 2 - and it's the right class
157 ok 3 - sec()
158 ok 4 - min()
159 ok 5 - hour()
160 not ok 6 - day()
161 # Failed test (- at line 16)
162 ok 7 - month()
163 ok 8 - year()
164 # Looks like you failed 1 tests of 8.
165 Whoops, a failure! [4] Test::Simple helpfully lets us know on what line
167 the failure occurred, but not much else. We were supposed to get 17,
168 but we didn't. What did we get?? Dunno. We'll have to re-run the
169 test in the debugger or throw in some print statements to find out.
170 Instead, we'll switch from Test::Simple to Test::More. Test::More does
172 everything Test::Simple does, and more! In fact, Test::More does
173 things exactly the way Test::Simple does. You can literally swap
174 Test::Simple out and put Test::More in its place. That's just what
175 we're going to do.
176 Test::More does more than Test::Simple. The most important difference
178 at this point is it provides more informative ways to say "ok".
179 Although you can write almost any test with a generic "ok()", it can't
180 tell you what went wrong. Instead, we'll use the "is()" function,
181 which lets us declare that something is supposed to be the same as
182 something else:
183 #!/usr/bin/perl -w
185 use Test::More tests => 8;
187 use Date::ICal;
189 $ical = Date::ICal->new( year => 1964, month => 10, day => 16,
191 hour => 16, min => 12, sec => 47,
192 tz => '0530' );
193 ok( defined $ical, 'new() returned something' );
195 ok( $ical->isa('Date::ICal'), " and it's the right class" );
196 is( $ical->sec, 47, ' sec()' );
197 is( $ical->min, 12, ' min()' );
198 is( $ical->hour, 16, ' hour()' );
199 is( $ical->day, 17, ' day()' );
200 is( $ical->month, 10, ' month()' );
201 is( $ical->year, 1964, ' year()' );
202 "Is "$ical->sec" 47?" "Is "$ical->min" 12?" With "is()" in place, you
204 get some more information
205 1..8
207 ok 1 - new() returned something
208 ok 2 - and it's the right class
209 ok 3 - sec()
210 ok 4 - min()
211 ok 5 - hour()
212 not ok 6 - day()
213 # Failed test (- at line 16)
214 # got: '16'
215 # expected: '17'
216 ok 7 - month()
217 ok 8 - year()
218 # Looks like you failed 1 tests of 8.
219 letting us know that "$ical->day" returned 16, but we expected 17. A
221 quick check shows that the code is working fine, we made a mistake when
222 writing up the tests. Just change it to:
223 is( $ical->day, 16, ' day()' );
225 and everything works.
227 So any time you're doing a "this equals that" sort of test, use "is()".
229 It even works on arrays. The test is always in scalar context, so you
230 can test how many elements are in a list this way. [5]
231 is( @foo, 5, 'foo has 5 elements' );
233 Sometimes the tests are wrong
235 Which brings us to a very important lesson. Code has bugs. Tests are
236 code. Ergo, tests have bugs. A failing test could mean a bug in the
237 code, but don't discount the possibility that the test is wrong.
238 On the flip side, don't be tempted to prematurely declare a test
240 incorrect just because you're having trouble finding the bug.
241 Invalidating a test isn't something to be taken lightly, and don't use
242 it as a cop out to avoid work.
243 Testing lots of values
245 We're going to be wanting to test a lot of dates here, trying to trick
246 the code with lots of different edge cases. Does it work before 1970?
247 After 2038? Before 1904? Do years after 10,000 give it trouble? Does
248 it get leap years right? We could keep repeating the code above, or we
249 could set up a little try/expect loop.
250 use Test::More tests => 32;
252 use Date::ICal;
253 my %ICal_Dates = (
255 # An ICal string And the year, month, day
256 # hour, minute and second we expect.
257 '19971024T120000' => # from the docs.
258 [ 1997, 10, 24, 12, 0, 0 ],
259 '20390123T232832' => # after the Unix epoch
260 [ 2039, 1, 23, 23, 28, 32 ],
261 '19671225T000000' => # before the Unix epoch
262 [ 1967, 12, 25, 0, 0, 0 ],
263 '18990505T232323' => # before the MacOS epoch
264 [ 1899, 5, 5, 23, 23, 23 ],
265 );
266 while( my($ical_str, $expect) = each %ICal_Dates ) {
269 my $ical = Date::ICal->new( ical => $ical_str );
270 ok( defined $ical, "new(ical => '$ical_str')" );
272 ok( $ical->isa('Date::ICal'), " and it's the right class" );
273 is( $ical->year, $expect->[0], ' year()' );
275 is( $ical->month, $expect->[1], ' month()' );
276 is( $ical->day, $expect->[2], ' day()' );
277 is( $ical->hour, $expect->[3], ' hour()' );
278 is( $ical->min, $expect->[4], ' min()' );
279 is( $ical->sec, $expect->[5], ' sec()' );
280 }
281 So now we can test bunches of dates by just adding them to %ICal_Dates.
283 Now that it's less work to test with more dates, you'll be inclined to
284 just throw more in as you think of them. Only problem is, every time
285 we add to that we have to keep adjusting the "use Test::More tests =>
286 ##" line. That can rapidly get annoying. There's two ways to make
287 this work better.
288 First, we can calculate the plan dynamically using the "plan()"
290 function.
291 use Test::More;
293 use Date::ICal;
294 my %ICal_Dates = (
296 ...same as before...
297 );
298 # For each key in the hash we're running 8 tests.
300 plan tests => keys(%ICal_Dates) * 8;
301 ...and then your tests...
303 Or to be even more flexible, we use "no_plan". This means we're just
305 running some tests, don't know how many. [6]
306 use Test::More 'no_plan'; # instead of tests => 32
308 now we can just add tests and not have to do all sorts of math to
310 figure out how many we're running.
311 Informative names
313 Take a look at this line here
314 ok( defined $ical, "new(ical => '$ical_str')" );
316 we've added more detail about what we're testing and the ICal string
318 itself we're trying out to the name. So you get results like:
319 ok 25 - new(ical => '19971024T120000')
321 ok 26 - and it's the right class
322 ok 27 - year()
323 ok 28 - month()
324 ok 29 - day()
325 ok 30 - hour()
326 ok 31 - min()
327 ok 32 - sec()
328 if something in there fails, you'll know which one it was and that will
330 make tracking down the problem easier. So try to put a bit of
331 debugging information into the test names.
332 Describe what the tests test, to make debugging a failed test easier
334 for you or for the next person who runs your test.
335 Skipping tests
337 Poking around in the existing Date::ICal tests, I found this in
338 t/01sanity.t [7]
339 #!/usr/bin/perl -w
341 use Test::More tests => 7;
343 use Date::ICal;
344 # Make sure epoch time is being handled sanely.
346 my $t1 = Date::ICal->new( epoch => 0 );
347 is( $t1->epoch, 0, "Epoch time of 0" );
348 # XXX This will only work on unix systems.
350 is( $t1->ical, '19700101Z', " epoch to ical" );
351 is( $t1->year, 1970, " year()" );
353 is( $t1->month, 1, " month()" );
354 is( $t1->day, 1, " day()" );
355 # like the tests above, but starting with ical instead of epoch
357 my $t2 = Date::ICal->new( ical => '19700101Z' );
358 is( $t2->ical, '19700101Z', "Start of epoch in ICal notation" );
359 is( $t2->epoch, 0, " and back to ICal" );
361 The beginning of the epoch is different on most non-Unix operating
363 systems [8]. Even though Perl smooths out the differences for the most
364 part, certain ports do it differently. MacPerl is one off the top of
365 my head. [9] So rather than just putting a comment in the test, we can
366 explicitly say it's never going to work and skip the test.
367 use Test::More tests => 7;
369 use Date::ICal;
370 # Make sure epoch time is being handled sanely.
372 my $t1 = Date::ICal->new( epoch => 0 );
373 is( $t1->epoch, 0, "Epoch time of 0" );
374 SKIP: {
376 skip('epoch to ICal not working on MacOS', 6)
377 if $^O eq 'MacOS';
378 is( $t1->ical, '19700101Z', " epoch to ical" );
380 is( $t1->year, 1970, " year()" );
382 is( $t1->month, 1, " month()" );
383 is( $t1->day, 1, " day()" );
384 # like the tests above, but starting with ical instead of epoch
386 my $t2 = Date::ICal->new( ical => '19700101Z' );
387 is( $t2->ical, '19700101Z', "Start of epoch in ICal notation" );
388 is( $t2->epoch, 0, " and back to ICal" );
390 }
391 A little bit of magic happens here. When running on anything but
393 MacOS, all the tests run normally. But when on MacOS, "skip()" causes
394 the entire contents of the SKIP block to be jumped over. It's never
395 run. Instead, it prints special output that tells Test::Harness that
396 the tests have been skipped.
397 1..7
399 ok 1 - Epoch time of 0
400 ok 2 # skip epoch to ICal not working on MacOS
401 ok 3 # skip epoch to ICal not working on MacOS
402 ok 4 # skip epoch to ICal not working on MacOS
403 ok 5 # skip epoch to ICal not working on MacOS
404 ok 6 # skip epoch to ICal not working on MacOS
405 ok 7 # skip epoch to ICal not working on MacOS
406 This means your tests won't fail on MacOS. This means less emails from
408 MacPerl users telling you about failing tests that you know will never
409 work. You've got to be careful with skip tests. These are for tests
410 which don't work and never will. It is not for skipping genuine bugs
411 (we'll get to that in a moment).
412 The tests are wholly and completely skipped. [10] This will work.
414 SKIP: {
416 skip("I don't wanna die!");
417 die, die, die, die, die;
419 }
420 Todo tests
422 Thumbing through the Date::ICal man page, I came across this:
423 ical
425 $ical_string = $ical->ical;
427 Retrieves, or sets, the date on the object, using any
429 valid ICal date/time string.
430 "Retrieves or sets". Hmmm, didn't see a test for using "ical()" to set
432 the date in the Date::ICal test suite. So I'll write one.
433 use Test::More tests => 1;
435 use Date::ICal;
436 my $ical = Date::ICal->new;
438 $ical->ical('20201231Z');
439 is( $ical->ical, '20201231Z', 'Setting via ical()' );
440 run that and I get
442 1..1
444 not ok 1 - Setting via ical()
445 # Failed test (- at line 6)
446 # got: '20010814T233649Z'
447 # expected: '20201231Z'
448 # Looks like you failed 1 tests of 1.
449 Whoops! Looks like it's unimplemented. Let's assume we don't have the
451 time to fix this. [11] Normally, you'd just comment out the test and
452 put a note in a todo list somewhere. Instead, we're going to
453 explicitly state "this test will fail" by wrapping it in a "TODO"
454 block.
455 use Test::More tests => 1;
457 TODO: {
459 local $TODO = 'ical($ical) not yet implemented';
460 my $ical = Date::ICal->new;
462 $ical->ical('20201231Z');
463 is( $ical->ical, '20201231Z', 'Setting via ical()' );
465 }
466 Now when you run, it's a little different:
468 1..1
470 not ok 1 - Setting via ical() # TODO ical($ical) not yet implemented
471 # got: '20010822T201551Z'
472 # expected: '20201231Z'
473 Test::More doesn't say "Looks like you failed 1 tests of 1". That '#
475 TODO' tells Test::Harness "this is supposed to fail" and it treats a
476 failure as a successful test. So you can write tests even before
477 you've fixed the underlying code.
478 If a TODO test passes, Test::Harness will report it "UNEXPECTEDLY
480 SUCCEEDED". When that happens, you simply remove the TODO block with
481 "local $TODO" and turn it into a real test.
482 Testing with taint mode.
484 Taint mode is a funny thing. It's the globalest of all global
485 features. Once you turn it on, it affects all code in your program and
486 all modules used (and all the modules they use). If a single piece of
487 code isn't taint clean, the whole thing explodes. With that in mind,
488 it's very important to ensure your module works under taint mode.
489 It's very simple to have your tests run under taint mode. Just throw a
491 "-T" into the "#!" line. Test::Harness will read the switches in "#!"
492 and use them to run your tests.
493 #!/usr/bin/perl -Tw
495 ...test normally here...
497 So when you say "make test" it will be run with taint mode and warnings
499 on.
500
502 1. The first number doesn't really mean anything, but it has to be 1.
503 It's the second number that's important.
504 2. For those following along at home, I'm using version 1.31. It has
506 some bugs, which is good -- we'll uncover them with our tests.
507 3. You can actually take this one step further and test the manual
509 itself. Have a look at Test::Inline (formerly Pod::Tests).
510 4. Yes, there's a mistake in the test suite. What! Me, contrived?
512 5. We'll get to testing the contents of lists later.
514 6. But what happens if your test program dies halfway through?! Since
516 we didn't say how many tests we're going to run, how can we know it
517 failed? No problem, Test::More employs some magic to catch that
518 death and turn the test into a failure, even if every test passed
519 up to that point.
520 7. I cleaned it up a little.
522 8. Most Operating Systems record time as the number of seconds since a
524 certain date. This date is the beginning of the epoch. Unix's
525 starts at midnight January 1st, 1970 GMT.
526 9. MacOS's epoch is midnight January 1st, 1904. VMS's is midnight,
528 November 17th, 1858, but vmsperl emulates the Unix epoch so it's
529 not a problem.
530 10. As long as the code inside the SKIP block at least compiles.
532 Please don't ask how. No, it's not a filter.
533 11. Do NOT be tempted to use TODO tests as a way to avoid fixing simple
535 bugs!
536
538 Michael G Schwern <schwern@pobox.com> and the perl-qa dancers!
539
541 Copyright 2001 by Michael G Schwern <schwern@pobox.com>.
542 This documentation is free; you can redistribute it and/or modify it
544 under the same terms as Perl itself.
545 Irrespective of its distribution, all code examples in these files are
547 hereby placed into the public domain. You are permitted and encouraged
548 to use this code in your own programs for fun or for profit as you see
549 fit. A simple comment in the code giving credit would be courteous but
550 is not required.
551perl v5.16.3 2011-02-23 Test::Tutorial(3)