1Test::Tutorial(3) User Contributed Perl Documentation Test::Tutorial(3)
2
6 Test::Tutorial - A tutorial about writing really basic tests
7
9 AHHHHHHH!!!! NOT TESTING! Anything but testing! Beat me, whip me,
10 send me to Detroit, but don't make me write tests!
11 *sob*
13 Besides, I don't know how to write the damned things.
15 Is this you? Is writing tests right up there with writing
17 documentation and having your fingernails pulled out? Did you open up
18 a test and read
19 ######## We start with some black magic
21 and decide that's quite enough for you?
23 It's ok. That's all gone now. We've done all the black magic for you.
25 And here are the tricks...
26 Nuts and bolts of testing.
28 Here's the most basic test program.
29 #!/usr/bin/perl -w
31 print "1..1\n";
33 print 1 + 1 == 2 ? "ok 1\n" : "not ok 1\n";
35 Because 1 + 1 is 2, it prints:
37 1..1
39 ok 1
40 What this says is: 1..1 "I'm going to run one test." [1] "ok 1" "The
42 first test passed". And that's about all magic there is to testing.
43 Your basic unit of testing is the ok. For each thing you test, an "ok"
44 is printed. Simple. Test::Harness interprets your test results to
45 determine if you succeeded or failed (more on that later).
46 Writing all these print statements rapidly gets tedious. Fortunately,
48 there's Test::Simple. It has one function, ok().
49 #!/usr/bin/perl -w
51 use Test::Simple tests => 1;
53 ok( 1 + 1 == 2 );
55 That does the same thing as the previous code. ok() is the backbone of
57 Perl testing, and we'll be using it instead of roll-your-own from here
58 on. If ok() gets a true value, the test passes. False, it fails.
59 #!/usr/bin/perl -w
61 use Test::Simple tests => 2;
63 ok( 1 + 1 == 2 );
64 ok( 2 + 2 == 5 );
65 From that comes:
67 1..2
69 ok 1
70 not ok 2
71 # Failed test (test.pl at line 5)
72 # Looks like you failed 1 tests of 2.
73 1..2 "I'm going to run two tests." This number is a plan. It helps to
75 ensure your test program ran all the way through and didn't die or skip
76 some tests. "ok 1" "The first test passed." "not ok 2" "The second
77 test failed". Test::Simple helpfully prints out some extra commentary
78 about your tests.
79 It's not scary. Come, hold my hand. We're going to give an example of
81 testing a module. For our example, we'll be testing a date library,
82 Date::ICal. It's on CPAN, so download a copy and follow along. [2]
83 Where to start?
85 This is the hardest part of testing, where do you start? People often
86 get overwhelmed at the apparent enormity of the task of testing a whole
87 module. The best place to start is at the beginning. Date::ICal is an
88 object-oriented module, and that means you start by making an object.
89 Test new().
90 #!/usr/bin/perl -w
92 # assume these two lines are in all subsequent examples
94 use strict;
95 use warnings;
96 use Test::Simple tests => 2;
98 use Date::ICal;
100 my $ical = Date::ICal->new; # create an object
102 ok( defined $ical ); # check that we got something
103 ok( $ical->isa('Date::ICal') ); # and it's the right class
104 Run that and you should get:
106 1..2
108 ok 1
109 ok 2
110 Congratulations! You've written your first useful test.
112 Names
114 That output isn't terribly descriptive, is it? When you have two tests
115 you can figure out which one is #2, but what if you have 102 tests?
116 Each test can be given a little descriptive name as the second argument
118 to ok().
119 use Test::Simple tests => 2;
121 ok( defined $ical, 'new() returned something' );
123 ok( $ical->isa('Date::ICal'), " and it's the right class" );
124 Now you'll see:
126 1..2
128 ok 1 - new() returned something
129 ok 2 - and it's the right class
130 Test the manual
132 The simplest way to build up a decent testing suite is to just test
133 what the manual says it does. [3] Let's pull something out of the
134 "SYNOPSIS" in Date::ICal and test that all its bits work.
135 #!/usr/bin/perl -w
137 use Test::Simple tests => 8;
139 use Date::ICal;
141 $ical = Date::ICal->new( year => 1964, month => 10, day => 16,
143 hour => 16, min => 12, sec => 47,
144 tz => '0530' );
145 ok( defined $ical, 'new() returned something' );
147 ok( $ical->isa('Date::ICal'), " and it's the right class" );
148 ok( $ical->sec == 47, ' sec()' );
149 ok( $ical->min == 12, ' min()' );
150 ok( $ical->hour == 16, ' hour()' );
151 ok( $ical->day == 17, ' day()' );
152 ok( $ical->month == 10, ' month()' );
153 ok( $ical->year == 1964, ' year()' );
154 Run that and you get:
156 1..8
158 ok 1 - new() returned something
159 ok 2 - and it's the right class
160 ok 3 - sec()
161 ok 4 - min()
162 ok 5 - hour()
163 not ok 6 - day()
164 # Failed test (- at line 16)
165 ok 7 - month()
166 ok 8 - year()
167 # Looks like you failed 1 tests of 8.
168 Whoops, a failure! [4] Test::Simple helpfully lets us know on what line
170 the failure occurred, but not much else. We were supposed to get 17,
171 but we didn't. What did we get?? Dunno. You could re-run the test in
172 the debugger or throw in some print statements to find out.
173 Instead, switch from Test::Simple to Test::More. Test::More does
175 everything Test::Simple does, and more! In fact, Test::More does
176 things exactly the way Test::Simple does. You can literally swap
177 Test::Simple out and put Test::More in its place. That's just what
178 we're going to do.
179 Test::More does more than Test::Simple. The most important difference
181 at this point is it provides more informative ways to say "ok".
182 Although you can write almost any test with a generic ok(), it can't
183 tell you what went wrong. The is() function lets us declare that
184 something is supposed to be the same as something else:
185 use Test::More tests => 8;
187 use Date::ICal;
189 $ical = Date::ICal->new( year => 1964, month => 10, day => 16,
191 hour => 16, min => 12, sec => 47,
192 tz => '0530' );
193 ok( defined $ical, 'new() returned something' );
195 ok( $ical->isa('Date::ICal'), " and it's the right class" );
196 is( $ical->sec, 47, ' sec()' );
197 is( $ical->min, 12, ' min()' );
198 is( $ical->hour, 16, ' hour()' );
199 is( $ical->day, 17, ' day()' );
200 is( $ical->month, 10, ' month()' );
201 is( $ical->year, 1964, ' year()' );
202 "Is "$ical->sec" 47?" "Is "$ical->min" 12?" With is() in place, you
204 get more information:
205 1..8
207 ok 1 - new() returned something
208 ok 2 - and it's the right class
209 ok 3 - sec()
210 ok 4 - min()
211 ok 5 - hour()
212 not ok 6 - day()
213 # Failed test (- at line 16)
214 # got: '16'
215 # expected: '17'
216 ok 7 - month()
217 ok 8 - year()
218 # Looks like you failed 1 tests of 8.
219 Aha. "$ical->day" returned 16, but we expected 17. A quick check shows
221 that the code is working fine, we made a mistake when writing the
222 tests. Change it to:
223 is( $ical->day, 16, ' day()' );
225 ... and everything works.
227 Any time you're doing a "this equals that" sort of test, use is(). It
229 even works on arrays. The test is always in scalar context, so you can
230 test how many elements are in an array this way. [5]
231 is( @foo, 5, 'foo has 5 elements' );
233 Sometimes the tests are wrong
235 This brings up a very important lesson. Code has bugs. Tests are
236 code. Ergo, tests have bugs. A failing test could mean a bug in the
237 code, but don't discount the possibility that the test is wrong.
238 On the flip side, don't be tempted to prematurely declare a test
240 incorrect just because you're having trouble finding the bug.
241 Invalidating a test isn't something to be taken lightly, and don't use
242 it as a cop out to avoid work.
243 Testing lots of values
245 We're going to be wanting to test a lot of dates here, trying to trick
246 the code with lots of different edge cases. Does it work before 1970?
247 After 2038? Before 1904? Do years after 10,000 give it trouble? Does
248 it get leap years right? We could keep repeating the code above, or we
249 could set up a little try/expect loop.
250 use Test::More tests => 32;
252 use Date::ICal;
253 my %ICal_Dates = (
255 # An ICal string And the year, month, day
256 # hour, minute and second we expect.
257 '19971024T120000' => # from the docs.
258 [ 1997, 10, 24, 12, 0, 0 ],
259 '20390123T232832' => # after the Unix epoch
260 [ 2039, 1, 23, 23, 28, 32 ],
261 '19671225T000000' => # before the Unix epoch
262 [ 1967, 12, 25, 0, 0, 0 ],
263 '18990505T232323' => # before the MacOS epoch
264 [ 1899, 5, 5, 23, 23, 23 ],
265 );
266 while( my($ical_str, $expect) = each %ICal_Dates ) {
269 my $ical = Date::ICal->new( ical => $ical_str );
270 ok( defined $ical, "new(ical => '$ical_str')" );
272 ok( $ical->isa('Date::ICal'), " and it's the right class" );
273 is( $ical->year, $expect->[0], ' year()' );
275 is( $ical->month, $expect->[1], ' month()' );
276 is( $ical->day, $expect->[2], ' day()' );
277 is( $ical->hour, $expect->[3], ' hour()' );
278 is( $ical->min, $expect->[4], ' min()' );
279 is( $ical->sec, $expect->[5], ' sec()' );
280 }
281 Now we can test bunches of dates by just adding them to %ICal_Dates.
283 Now that it's less work to test with more dates, you'll be inclined to
284 just throw more in as you think of them. Only problem is, every time
285 we add to that we have to keep adjusting the "use Test::More tests =>
286 ##" line. That can rapidly get annoying. There are ways to make this
287 work better.
288 First, we can calculate the plan dynamically using the plan() function.
290 use Test::More;
292 use Date::ICal;
293 my %ICal_Dates = (
295 ...same as before...
296 );
297 # For each key in the hash we're running 8 tests.
299 plan tests => keys(%ICal_Dates) * 8;
300 ...and then your tests...
302 To be even more flexible, use "done_testing". This means we're just
304 running some tests, don't know how many. [6]
305 use Test::More; # instead of tests => 32
307 ... # tests here
309 done_testing(); # reached the end safely
311 If you don't specify a plan, Test::More expects to see done_testing()
313 before your program exits. It will warn you if you forget it. You can
314 give done_testing() an optional number of tests you expected to run,
315 and if the number ran differs, Test::More will give you another kind of
316 warning.
317 Informative names
319 Take a look at the line:
320 ok( defined $ical, "new(ical => '$ical_str')" );
322 We've added more detail about what we're testing and the ICal string
324 itself we're trying out to the name. So you get results like:
325 ok 25 - new(ical => '19971024T120000')
327 ok 26 - and it's the right class
328 ok 27 - year()
329 ok 28 - month()
330 ok 29 - day()
331 ok 30 - hour()
332 ok 31 - min()
333 ok 32 - sec()
334 If something in there fails, you'll know which one it was and that will
336 make tracking down the problem easier. Try to put a bit of debugging
337 information into the test names.
338 Describe what the tests test, to make debugging a failed test easier
340 for you or for the next person who runs your test.
341 Skipping tests
343 Poking around in the existing Date::ICal tests, I found this in
344 t/01sanity.t [7]
345 #!/usr/bin/perl -w
347 use Test::More tests => 7;
349 use Date::ICal;
350 # Make sure epoch time is being handled sanely.
352 my $t1 = Date::ICal->new( epoch => 0 );
353 is( $t1->epoch, 0, "Epoch time of 0" );
354 # XXX This will only work on unix systems.
356 is( $t1->ical, '19700101Z', " epoch to ical" );
357 is( $t1->year, 1970, " year()" );
359 is( $t1->month, 1, " month()" );
360 is( $t1->day, 1, " day()" );
361 # like the tests above, but starting with ical instead of epoch
363 my $t2 = Date::ICal->new( ical => '19700101Z' );
364 is( $t2->ical, '19700101Z', "Start of epoch in ICal notation" );
365 is( $t2->epoch, 0, " and back to ICal" );
367 The beginning of the epoch is different on most non-Unix operating
369 systems [8]. Even though Perl smooths out the differences for the most
370 part, certain ports do it differently. MacPerl is one off the top of
371 my head. [9] Rather than putting a comment in the test and hoping
372 someone will read the test while debugging the failure, we can
373 explicitly say it's never going to work and skip the test.
374 use Test::More tests => 7;
376 use Date::ICal;
377 # Make sure epoch time is being handled sanely.
379 my $t1 = Date::ICal->new( epoch => 0 );
380 is( $t1->epoch, 0, "Epoch time of 0" );
381 SKIP: {
383 skip('epoch to ICal not working on Mac OS', 6)
384 if $^O eq 'MacOS';
385 is( $t1->ical, '19700101Z', " epoch to ical" );
387 is( $t1->year, 1970, " year()" );
389 is( $t1->month, 1, " month()" );
390 is( $t1->day, 1, " day()" );
391 # like the tests above, but starting with ical instead of epoch
393 my $t2 = Date::ICal->new( ical => '19700101Z' );
394 is( $t2->ical, '19700101Z', "Start of epoch in ICal notation" );
395 is( $t2->epoch, 0, " and back to ICal" );
397 }
398 A little bit of magic happens here. When running on anything but
400 MacOS, all the tests run normally. But when on MacOS, skip() causes
401 the entire contents of the SKIP block to be jumped over. It never
402 runs. Instead, skip() prints special output that tells Test::Harness
403 that the tests have been skipped.
404 1..7
406 ok 1 - Epoch time of 0
407 ok 2 # skip epoch to ICal not working on MacOS
408 ok 3 # skip epoch to ICal not working on MacOS
409 ok 4 # skip epoch to ICal not working on MacOS
410 ok 5 # skip epoch to ICal not working on MacOS
411 ok 6 # skip epoch to ICal not working on MacOS
412 ok 7 # skip epoch to ICal not working on MacOS
413 This means your tests won't fail on MacOS. This means fewer emails
415 from MacPerl users telling you about failing tests that you know will
416 never work. You've got to be careful with skip tests. These are for
417 tests which don't work and never will. It is not for skipping genuine
418 bugs (we'll get to that in a moment).
419 The tests are wholly and completely skipped. [10] This will work.
421 SKIP: {
423 skip("I don't wanna die!");
424 die, die, die, die, die;
426 }
427 Todo tests
429 While thumbing through the Date::ICal man page, I came across this:
430 ical
432 $ical_string = $ical->ical;
434 Retrieves, or sets, the date on the object, using any
436 valid ICal date/time string.
437 "Retrieves or sets". Hmmm. I didn't see a test for using ical() to set
439 the date in the Date::ICal test suite. So I wrote one:
440 use Test::More tests => 1;
442 use Date::ICal;
443 my $ical = Date::ICal->new;
445 $ical->ical('20201231Z');
446 is( $ical->ical, '20201231Z', 'Setting via ical()' );
447 Run that. I saw:
449 1..1
451 not ok 1 - Setting via ical()
452 # Failed test (- at line 6)
453 # got: '20010814T233649Z'
454 # expected: '20201231Z'
455 # Looks like you failed 1 tests of 1.
456 Whoops! Looks like it's unimplemented. Assume you don't have the time
458 to fix this. [11] Normally, you'd just comment out the test and put a
459 note in a todo list somewhere. Instead, explicitly state "this test
460 will fail" by wrapping it in a "TODO" block:
461 use Test::More tests => 1;
463 TODO: {
465 local $TODO = 'ical($ical) not yet implemented';
466 my $ical = Date::ICal->new;
468 $ical->ical('20201231Z');
469 is( $ical->ical, '20201231Z', 'Setting via ical()' );
471 }
472 Now when you run, it's a little different:
474 1..1
476 not ok 1 - Setting via ical() # TODO ical($ical) not yet implemented
477 # got: '20010822T201551Z'
478 # expected: '20201231Z'
479 Test::More doesn't say "Looks like you failed 1 tests of 1". That '#
481 TODO' tells Test::Harness "this is supposed to fail" and it treats a
482 failure as a successful test. You can write tests even before you've
483 fixed the underlying code.
484 If a TODO test passes, Test::Harness will report it "UNEXPECTEDLY
486 SUCCEEDED". When that happens, remove the TODO block with "local
487 $TODO" and turn it into a real test.
488 Testing with taint mode.
490 Taint mode is a funny thing. It's the globalest of all global
491 features. Once you turn it on, it affects all code in your program and
492 all modules used (and all the modules they use). If a single piece of
493 code isn't taint clean, the whole thing explodes. With that in mind,
494 it's very important to ensure your module works under taint mode.
495 It's very simple to have your tests run under taint mode. Just throw a
497 "-T" into the "#!" line. Test::Harness will read the switches in "#!"
498 and use them to run your tests.
499 #!/usr/bin/perl -Tw
501 ...test normally here...
503 When you say "make test" it will run with taint mode on.
505
507 1. The first number doesn't really mean anything, but it has to be 1.
508 It's the second number that's important.
509 2. For those following along at home, I'm using version 1.31. It has
511 some bugs, which is good -- we'll uncover them with our tests.
512 3. You can actually take this one step further and test the manual
514 itself. Have a look at Test::Inline (formerly Pod::Tests).
515 4. Yes, there's a mistake in the test suite. What! Me, contrived?
517 5. We'll get to testing the contents of lists later.
519 6. But what happens if your test program dies halfway through?! Since
521 we didn't say how many tests we're going to run, how can we know it
522 failed? No problem, Test::More employs some magic to catch that
523 death and turn the test into a failure, even if every test passed
524 up to that point.
525 7. I cleaned it up a little.
527 8. Most Operating Systems record time as the number of seconds since a
529 certain date. This date is the beginning of the epoch. Unix's
530 starts at midnight January 1st, 1970 GMT.
531 9. MacOS's epoch is midnight January 1st, 1904. VMS's is midnight,
533 November 17th, 1858, but vmsperl emulates the Unix epoch so it's
534 not a problem.
535 10. As long as the code inside the SKIP block at least compiles.
537 Please don't ask how. No, it's not a filter.
538 11. Do NOT be tempted to use TODO tests as a way to avoid fixing simple
540 bugs!
541
543 Michael G Schwern <schwern@pobox.com> and the perl-qa dancers!
544
546 Chad Granum <exodist@cpan.org>
547
549 Copyright 2001 by Michael G Schwern <schwern@pobox.com>.
550 This documentation is free; you can redistribute it and/or modify it
552 under the same terms as Perl itself.
553 Irrespective of its distribution, all code examples in these files are
555 hereby placed into the public domain. You are permitted and encouraged
556 to use this code in your own programs for fun or for profit as you see
557 fit. A simple comment in the code giving credit would be courteous but
558 is not required.
559perl v5.36.0 2023-03-15 Test::Tutorial(3)