1Test::Builder(3pm) Perl Programmers Reference Guide Test::Builder(3pm)
2
6 Test::Builder - Backend for building test libraries
7
9 package My::Test::Module;
10 use base 'Test::Builder::Module';
11 my $CLASS = __PACKAGE__;
13 sub ok {
15 my($test, $name) = @_;
16 my $tb = $CLASS->builder;
17 $tb->ok($test, $name);
19 }
20
22 Test::Simple and Test::More have proven to be popular testing modules,
23 but they're not always flexible enough. Test::Builder provides the a
24 building block upon which to write your own test libraries which can
25 work together.
26 Construction
28 new
29 my $Test = Test::Builder->new;
30 Returns a Test::Builder object representing the current state of
32 the test.
33 Since you only run one test per program "new" always returns the
35 same Test::Builder object. No matter how many times you call
36 "new()", you're getting the same object. This is called a
37 singleton. This is done so that multiple modules share such global
38 information as the test counter and where test output is going.
39 If you want a completely new Test::Builder object different from
41 the singleton, use "create".
42 create
44 my $Test = Test::Builder->create;
45 Ok, so there can be more than one Test::Builder object and this is
47 how you get it. You might use this instead of "new()" if you're
48 testing a Test::Builder based module, but otherwise you probably
49 want "new".
50 NOTE: the implementation is not complete. "level", for example, is
52 still shared amongst all Test::Builder objects, even ones created
53 using this method. Also, the method name may change in the future.
54 reset
56 $Test->reset;
57 Reinitializes the Test::Builder singleton to its original state.
59 Mostly useful for tests run in persistent environments where the
60 same test might be run multiple times in the same process.
61 Setting up tests
63 These methods are for setting up tests and declaring how many there
64 are. You usually only want to call one of these methods.
65 plan
67 $Test->plan('no_plan');
68 $Test->plan( skip_all => $reason );
69 $Test->plan( tests => $num_tests );
70 A convenient way to set up your tests. Call this and Test::Builder
72 will print the appropriate headers and take the appropriate
73 actions.
74 If you call "plan()", don't call any of the other methods below.
76 expected_tests
78 my $max = $Test->expected_tests;
79 $Test->expected_tests($max);
80 Gets/sets the number of tests we expect this test to run and prints
82 out the appropriate headers.
83 no_plan
85 $Test->no_plan;
86 Declares that this test will run an indeterminate number of tests.
88 done_testing
90 $Test->done_testing();
91 $Test->done_testing($num_tests);
92 Declares that you are done testing, no more tests will be run after
94 this point.
95 If a plan has not yet been output, it will do so.
97 $num_tests is the number of tests you planned to run. If a
99 numbered plan was already declared, and if this contradicts, a
100 failing test will be run to reflect the planning mistake. If
101 "no_plan" was declared, this will override.
102 If "done_testing()" is called twice, the second call will issue a
104 failing test.
105 If $num_tests is omitted, the number of tests run will be used,
107 like no_plan.
108 "done_testing()" is, in effect, used when you'd want to use
110 "no_plan", but safer. You'd use it like so:
111 $Test->ok($a == $b);
113 $Test->done_testing();
114 Or to plan a variable number of tests:
116 for my $test (@tests) {
118 $Test->ok($test);
119 }
120 $Test->done_testing(@tests);
121 has_plan
123 $plan = $Test->has_plan
124 Find out whether a plan has been defined. $plan is either "undef"
126 (no plan has been set), "no_plan" (indeterminate # of tests) or an
127 integer (the number of expected tests).
128 skip_all
130 $Test->skip_all;
131 $Test->skip_all($reason);
132 Skips all the tests, using the given $reason. Exits immediately
134 with 0.
135 exported_to
137 my $pack = $Test->exported_to;
138 $Test->exported_to($pack);
139 Tells Test::Builder what package you exported your functions to.
141 This method isn't terribly useful since modules which share the
143 same Test::Builder object might get exported to different packages
144 and only the last one will be honored.
145 Running tests
147 These actually run the tests, analogous to the functions in Test::More.
148 They all return true if the test passed, false if the test failed.
150 $name is always optional.
152 ok
154 $Test->ok($test, $name);
155 Your basic test. Pass if $test is true, fail if $test is false.
157 Just like Test::Simple's "ok()".
158 is_eq
160 $Test->is_eq($got, $expected, $name);
161 Like Test::More's "is()". Checks if "$got eq $expected". This is
163 the string version.
164 is_num
166 $Test->is_num($got, $expected, $name);
167 Like Test::More's "is()". Checks if "$got == $expected". This is
169 the numeric version.
170 isnt_eq
172 $Test->isnt_eq($got, $dont_expect, $name);
173 Like Test::More's "isnt()". Checks if "$got ne $dont_expect".
175 This is the string version.
176 isnt_num
178 $Test->isnt_num($got, $dont_expect, $name);
179 Like Test::More's "isnt()". Checks if "$got ne $dont_expect".
181 This is the numeric version.
182 like
184 $Test->like($this, qr/$regex/, $name);
185 $Test->like($this, '/$regex/', $name);
186 Like Test::More's "like()". Checks if $this matches the given
188 $regex.
189 You'll want to avoid "qr//" if you want your tests to work before
191 5.005.
192 unlike
194 $Test->unlike($this, qr/$regex/, $name);
195 $Test->unlike($this, '/$regex/', $name);
196 Like Test::More's "unlike()". Checks if $this does not match the
198 given $regex.
199 cmp_ok
201 $Test->cmp_ok($this, $type, $that, $name);
202 Works just like Test::More's "cmp_ok()".
204 $Test->cmp_ok($big_num, '!=', $other_big_num);
206 Other Testing Methods
208 These are methods which are used in the course of writing a test but
209 are not themselves tests.
210 BAIL_OUT
212 $Test->BAIL_OUT($reason);
213 Indicates to the Test::Harness that things are going so badly all
215 testing should terminate. This includes running any additional
216 test scripts.
217 It will exit with 255.
219 skip
221 $Test->skip;
222 $Test->skip($why);
223 Skips the current test, reporting $why.
225 todo_skip
227 $Test->todo_skip;
228 $Test->todo_skip($why);
229 Like "skip()", only it will declare the test as failing and TODO.
231 Similar to
232 print "not ok $tnum # TODO $why\n";
234 Test building utility methods
236 These methods are useful when writing your own test methods.
237 maybe_regex
239 $Test->maybe_regex(qr/$regex/);
240 $Test->maybe_regex('/$regex/');
241 Convenience method for building testing functions that take regular
243 expressions as arguments, but need to work before perl 5.005.
244 Takes a quoted regular expression produced by "qr//", or a string
246 representing a regular expression.
247 Returns a Perl value which may be used instead of the corresponding
249 regular expression, or "undef" if its argument is not recognised.
250 For example, a version of "like()", sans the useful diagnostic
252 messages, could be written as:
253 sub laconic_like {
255 my ($self, $this, $regex, $name) = @_;
256 my $usable_regex = $self->maybe_regex($regex);
257 die "expecting regex, found '$regex'\n"
258 unless $usable_regex;
259 $self->ok($this =~ m/$usable_regex/, $name);
260 }
261 is_fh
263 my $is_fh = $Test->is_fh($thing);
264 Determines if the given $thing can be used as a filehandle.
266 Test style
268 level
269 $Test->level($how_high);
270 How far up the call stack should $Test look when reporting where
272 the test failed.
273 Defaults to 1.
275 Setting $Test::Builder::Level overrides. This is typically useful
277 localized:
278 sub my_ok {
280 my $test = shift;
281 local $Test::Builder::Level = $Test::Builder::Level + 1;
283 $TB->ok($test);
284 }
285 To be polite to other functions wrapping your own you usually want
287 to increment $Level rather than set it to a constant.
288 use_numbers
290 $Test->use_numbers($on_or_off);
291 Whether or not the test should output numbers. That is, this if
293 true:
294 ok 1
296 ok 2
297 ok 3
298 or this if false
300 ok
302 ok
303 ok
304 Most useful when you can't depend on the test output order, such as
306 when threads or forking is involved.
307 Defaults to on.
309 no_diag
311 $Test->no_diag($no_diag);
312 If set true no diagnostics will be printed. This includes calls to
314 "diag()".
315 no_ending
317 $Test->no_ending($no_ending);
318 Normally, Test::Builder does some extra diagnostics when the test
320 ends. It also changes the exit code as described below.
321 If this is true, none of that will be done.
323 no_header
325 $Test->no_header($no_header);
326 If set to true, no "1..N" header will be printed.
328 Output
330 Controlling where the test output goes.
331 It's ok for your test to change where STDOUT and STDERR point to,
333 Test::Builder's default output settings will not be affected.
334 diag
336 $Test->diag(@msgs);
337 Prints out the given @msgs. Like "print", arguments are simply
339 appended together.
340 Normally, it uses the "failure_output()" handle, but if this is for
342 a TODO test, the "todo_output()" handle is used.
343 Output will be indented and marked with a # so as not to interfere
345 with test output. A newline will be put on the end if there isn't
346 one already.
347 We encourage using this rather than calling print directly.
349 Returns false. Why? Because "diag()" is often used in conjunction
351 with a failing test ("ok() || diag()") it "passes through" the
352 failure.
353 return ok(...) || diag(...);
355 note
357 $Test->note(@msgs);
358 Like "diag()", but it prints to the "output()" handle so it will
360 not normally be seen by the user except in verbose mode.
361 explain
363 my @dump = $Test->explain(@msgs);
364 Will dump the contents of any references in a human readable
366 format. Handy for things like...
367 is_deeply($have, $want) || diag explain $have;
369 or
371 is_deeply($have, $want) || note explain $have;
373 output
375 failure_output
376 todo_output
377 my $filehandle = $Test->output;
378 $Test->output($filehandle);
379 $Test->output($filename);
380 $Test->output(\$scalar);
381 These methods control where Test::Builder will print its output.
383 They take either an open $filehandle, a $filename to open and write
384 to or a $scalar reference to append to. It will always return a
385 $filehandle.
386 output is where normal "ok/not ok" test output goes.
388 Defaults to STDOUT.
390 failure_output is where diagnostic output on test failures and
392 "diag()" goes. It is normally not read by Test::Harness and
393 instead is displayed to the user.
394 Defaults to STDERR.
396 "todo_output" is used instead of "failure_output()" for the
398 diagnostics of a failing TODO test. These will not be seen by the
399 user.
400 Defaults to STDOUT.
402 reset_outputs
404 $tb->reset_outputs;
405 Resets all the output filehandles back to their defaults.
407 carp
409 $tb->carp(@message);
410 Warns with @message but the message will appear to come from the
412 point where the original test function was called ("$tb->caller").
413 croak
415 $tb->croak(@message);
416 Dies with @message but the message will appear to come from the
418 point where the original test function was called ("$tb->caller").
419 Test Status and Info
421 current_test
422 my $curr_test = $Test->current_test;
423 $Test->current_test($num);
424 Gets/sets the current test number we're on. You usually shouldn't
426 have to set this.
427 If set forward, the details of the missing tests are filled in as
429 'unknown'. if set backward, the details of the intervening tests
430 are deleted. You can erase history if you really want to.
431 summary
433 my @tests = $Test->summary;
434 A simple summary of the tests so far. True for pass, false for
436 fail. This is a logical pass/fail, so todos are passes.
437 Of course, test #1 is $tests[0], etc...
439 details
441 my @tests = $Test->details;
442 Like "summary()", but with a lot more detail.
444 $tests[$test_num - 1] =
446 { 'ok' => is the test considered a pass?
447 actual_ok => did it literally say 'ok'?
448 name => name of the test (if any)
449 type => type of test (if any, see below).
450 reason => reason for the above (if any)
451 };
452 'ok' is true if Test::Harness will consider the test to be a pass.
454 'actual_ok' is a reflection of whether or not the test literally
456 printed 'ok' or 'not ok'. This is for examining the result of
457 'todo' tests.
458 'name' is the name of the test.
460 'type' indicates if it was a special test. Normal tests have a
462 type of ''. Type can be one of the following:
463 skip see skip()
465 todo see todo()
466 todo_skip see todo_skip()
467 unknown see below
468 Sometimes the Test::Builder test counter is incremented without it
470 printing any test output, for example, when "current_test()" is
471 changed. In these cases, Test::Builder doesn't know the result of
472 the test, so its type is 'unknown'. These details for these tests
473 are filled in. They are considered ok, but the name and actual_ok
474 is left "undef".
475 For example "not ok 23 - hole count # TODO insufficient donuts"
477 would result in this structure:
478 $tests[22] = # 23 - 1, since arrays start from 0.
480 { ok => 1, # logically, the test passed since its todo
481 actual_ok => 0, # in absolute terms, it failed
482 name => 'hole count',
483 type => 'todo',
484 reason => 'insufficient donuts'
485 };
486 todo
488 my $todo_reason = $Test->todo;
489 my $todo_reason = $Test->todo($pack);
490 If the current tests are considered "TODO" it will return the
492 reason, if any. This reason can come from a $TODO variable or the
493 last call to "todo_start()".
494 Since a TODO test does not need a reason, this function can return
496 an empty string even when inside a TODO block. Use
497 "$Test->in_todo" to determine if you are currently inside a TODO
498 block.
499 "todo()" is about finding the right package to look for $TODO in.
501 It's pretty good at guessing the right package to look at. It
502 first looks for the caller based on "$Level + 1", since "todo()" is
503 usually called inside a test function. As a last resort it will
504 use "exported_to()".
505 Sometimes there is some confusion about where todo() should be
507 looking for the $TODO variable. If you want to be sure, tell it
508 explicitly what $pack to use.
509 find_TODO
511 my $todo_reason = $Test->find_TODO();
512 my $todo_reason = $Test->find_TODO($pack):
513 Like "todo()" but only returns the value of $TODO ignoring
515 "todo_start()".
516 in_todo
518 my $in_todo = $Test->in_todo;
519 Returns true if the test is currently inside a TODO block.
521 todo_start
523 $Test->todo_start();
524 $Test->todo_start($message);
525 This method allows you declare all subsequent tests as TODO tests,
527 up until the "todo_end" method has been called.
528 The "TODO:" and $TODO syntax is generally pretty good about
530 figuring out whether or not we're in a TODO test. However, often
531 we find that this is not possible to determine (such as when we
532 want to use $TODO but the tests are being executed in other
533 packages which can't be inferred beforehand).
534 Note that you can use this to nest "todo" tests
536 $Test->todo_start('working on this');
538 # lots of code
539 $Test->todo_start('working on that');
540 # more code
541 $Test->todo_end;
542 $Test->todo_end;
543 This is generally not recommended, but large testing systems often
545 have weird internal needs.
546 We've tried to make this also work with the TODO: syntax, but it's
548 not guaranteed and its use is also discouraged:
549 TODO: {
551 local $TODO = 'We have work to do!';
552 $Test->todo_start('working on this');
553 # lots of code
554 $Test->todo_start('working on that');
555 # more code
556 $Test->todo_end;
557 $Test->todo_end;
558 }
559 Pick one style or another of "TODO" to be on the safe side.
561 "todo_end"
563 $Test->todo_end;
564 Stops running tests as "TODO" tests. This method is fatal if
566 called without a preceding "todo_start" method call.
567 caller
569 my $package = $Test->caller;
570 my($pack, $file, $line) = $Test->caller;
571 my($pack, $file, $line) = $Test->caller($height);
572 Like the normal "caller()", except it reports according to your
574 "level()".
575 $height will be added to the "level()".
577 If "caller()" winds up off the top of the stack it report the
579 highest context.
580
582 If all your tests passed, Test::Builder will exit with zero (which is
583 normal). If anything failed it will exit with how many failed. If you
584 run less (or more) tests than you planned, the missing (or extras) will
585 be considered failures. If no tests were ever run Test::Builder will
586 throw a warning and exit with 255. If the test died, even after having
587 successfully completed all its tests, it will still be considered a
588 failure and will exit with 255.
589 So the exit codes are...
591 0 all tests successful
593 255 test died or all passed but wrong # of tests run
594 any other number how many failed (including missing or extras)
595 If you fail more than 254 tests, it will be reported as 254.
597
599 In perl 5.8.1 and later, Test::Builder is thread-safe. The test number
600 is shared amongst all threads. This means if one thread sets the test
601 number using "current_test()" they will all be effected.
602 While versions earlier than 5.8.1 had threads they contain too many
604 bugs to support.
605 Test::Builder is only thread-aware if threads.pm is loaded before
607 Test::Builder.
608
610 An informative hash, accessable via "<details()">, is stored for each
611 test you perform. So memory usage will scale linearly with each test
612 run. Although this is not a problem for most test suites, it can become
613 an issue if you do large (hundred thousands to million) combinatorics
614 tests in the same run.
615 In such cases, you are advised to either split the test file into
617 smaller ones, or use a reverse approach, doing "normal" (code) compares
618 and triggering fail() should anything go unexpected.
619 Future versions of Test::Builder will have a way to turn history off.
621
623 CPAN can provide the best examples. Test::Simple, Test::More,
624 Test::Exception and Test::Differences all use Test::Builder.
625
627 Test::Simple, Test::More, Test::Harness
628
630 Original code by chromatic, maintained by Michael G Schwern
631 <schwern@pobox.com>
632
634 Copyright 2002-2008 by chromatic <chromatic@wgz.org> and
635 Michael G Schwern <schwern@pobox.com>.
636 This program is free software; you can redistribute it and/or modify it
638 under the same terms as Perl itself.
639 See http://www.perl.com/perl/misc/Artistic.html
641perl v5.10.1 2009-07-07 Test::Builder(3pm)