1Test::Builder(3) User Contributed Perl Documentation Test::Builder(3)
2
6 Test::Builder - Backend for building test libraries
7
9 package My::Test::Module;
10 use base 'Test::Builder::Module';
11 my $CLASS = __PACKAGE__;
13 sub ok {
15 my($test, $name) = @_;
16 my $tb = $CLASS->builder;
17 $tb->ok($test, $name);
19 }
20
22 Test::Simple and Test::More have proven to be popular testing modules,
23 but they're not always flexible enough. Test::Builder provides a
24 building block upon which to write your own test libraries which can
25 work together.
26 Construction
28 new
29 my $Test = Test::Builder->new;
30 Returns a Test::Builder object representing the current state of
32 the test.
33 Since you only run one test per program "new" always returns the
35 same Test::Builder object. No matter how many times you call
36 "new()", you're getting the same object. This is called a
37 singleton. This is done so that multiple modules share such global
38 information as the test counter and where test output is going.
39 If you want a completely new Test::Builder object different from
41 the singleton, use "create".
42 create
44 my $Test = Test::Builder->create;
45 Ok, so there can be more than one Test::Builder object and this is
47 how you get it. You might use this instead of "new()" if you're
48 testing a Test::Builder based module, but otherwise you probably
49 want "new".
50 NOTE: the implementation is not complete. "level", for example, is
52 still shared amongst all Test::Builder objects, even ones created
53 using this method. Also, the method name may change in the future.
54 child
56 my $child = $builder->child($name_of_child);
57 $child->plan( tests => 4 );
58 $child->ok(some_code());
59 ...
60 $child->finalize;
61 Returns a new instance of "Test::Builder". Any output from this
63 child will be indented four spaces more than the parent's
64 indentation. When done, the "finalize" method must be called
65 explicitly.
66 Trying to create a new child with a previous child still active
68 (i.e., "finalize" not called) will "croak".
69 Trying to run a test when you have an open child will also "croak"
71 and cause the test suite to fail.
72 subtest
74 $builder->subtest($name, \&subtests);
75 See documentation of "subtest" in Test::More.
77 finalize
79 my $ok = $child->finalize;
80 When your child is done running tests, you must call "finalize" to
82 clean up and tell the parent your pass/fail status.
83 Calling finalize on a child with open children will "croak".
85 If the child falls out of scope before "finalize" is called, a
87 failure diagnostic will be issued and the child is considered to
88 have failed.
89 No attempt to call methods on a child after "finalize" is called is
91 guaranteed to succeed.
92 Calling this on the root builder is a no-op.
94 parent
96 if ( my $parent = $builder->parent ) {
97 ...
98 }
99 Returns the parent "Test::Builder" instance, if any. Only used
101 with child builders for nested TAP.
102 name
104 diag $builder->name;
105 Returns the name of the current builder. Top level builders
107 default to $0 (the name of the executable). Child builders are
108 named via the "child" method. If no name is supplied, will be
109 named "Child of $parent->name".
110 reset
112 $Test->reset;
113 Reinitializes the Test::Builder singleton to its original state.
115 Mostly useful for tests run in persistent environments where the
116 same test might be run multiple times in the same process.
117 Setting up tests
119 These methods are for setting up tests and declaring how many there
120 are. You usually only want to call one of these methods.
121 plan
123 $Test->plan('no_plan');
124 $Test->plan( skip_all => $reason );
125 $Test->plan( tests => $num_tests );
126 A convenient way to set up your tests. Call this and Test::Builder
128 will print the appropriate headers and take the appropriate
129 actions.
130 If you call "plan()", don't call any of the other methods below.
132 If a child calls "skip_all" in the plan, a
134 "Test::Builder::Exception" is thrown. Trap this error, call
135 "finalize()" and don't run any more tests on the child.
136 my $child = $Test->child('some child');
138 eval { $child->plan( $condition ? ( skip_all => $reason ) : ( tests => 3 ) ) };
139 if ( eval { $@->isa('Test::Builder::Exception') } ) {
140 $child->finalize;
141 return;
142 }
143 # run your tests
144 expected_tests
146 my $max = $Test->expected_tests;
147 $Test->expected_tests($max);
148 Gets/sets the number of tests we expect this test to run and prints
150 out the appropriate headers.
151 no_plan
153 $Test->no_plan;
154 Declares that this test will run an indeterminate number of tests.
156 done_testing
158 $Test->done_testing();
159 $Test->done_testing($num_tests);
160 Declares that you are done testing, no more tests will be run after
162 this point.
163 If a plan has not yet been output, it will do so.
165 $num_tests is the number of tests you planned to run. If a
167 numbered plan was already declared, and if this contradicts, a
168 failing test will be run to reflect the planning mistake. If
169 "no_plan" was declared, this will override.
170 If "done_testing()" is called twice, the second call will issue a
172 failing test.
173 If $num_tests is omitted, the number of tests run will be used,
175 like no_plan.
176 "done_testing()" is, in effect, used when you'd want to use
178 "no_plan", but safer. You'd use it like so:
179 $Test->ok($a == $b);
181 $Test->done_testing();
182 Or to plan a variable number of tests:
184 for my $test (@tests) {
186 $Test->ok($test);
187 }
188 $Test->done_testing(@tests);
189 has_plan
191 $plan = $Test->has_plan
192 Find out whether a plan has been defined. $plan is either "undef"
194 (no plan has been set), "no_plan" (indeterminate # of tests) or an
195 integer (the number of expected tests).
196 skip_all
198 $Test->skip_all;
199 $Test->skip_all($reason);
200 Skips all the tests, using the given $reason. Exits immediately
202 with 0.
203 exported_to
205 my $pack = $Test->exported_to;
206 $Test->exported_to($pack);
207 Tells Test::Builder what package you exported your functions to.
209 This method isn't terribly useful since modules which share the
211 same Test::Builder object might get exported to different packages
212 and only the last one will be honored.
213 Running tests
215 These actually run the tests, analogous to the functions in Test::More.
216 They all return true if the test passed, false if the test failed.
218 $name is always optional.
220 ok
222 $Test->ok($test, $name);
223 Your basic test. Pass if $test is true, fail if $test is false.
225 Just like Test::Simple's "ok()".
226 is_eq
228 $Test->is_eq($got, $expected, $name);
229 Like Test::More's "is()". Checks if "$got eq $expected". This is
231 the string version.
232 "undef" only ever matches another "undef".
234 is_num
236 $Test->is_num($got, $expected, $name);
237 Like Test::More's "is()". Checks if "$got == $expected". This is
239 the numeric version.
240 "undef" only ever matches another "undef".
242 isnt_eq
244 $Test->isnt_eq($got, $dont_expect, $name);
245 Like Test::More's "isnt()". Checks if "$got ne $dont_expect".
247 This is the string version.
248 isnt_num
250 $Test->isnt_num($got, $dont_expect, $name);
251 Like Test::More's "isnt()". Checks if "$got ne $dont_expect".
253 This is the numeric version.
254 like
256 $Test->like($this, qr/$regex/, $name);
257 $Test->like($this, '/$regex/', $name);
258 Like Test::More's "like()". Checks if $this matches the given
260 $regex.
261 unlike
263 $Test->unlike($this, qr/$regex/, $name);
264 $Test->unlike($this, '/$regex/', $name);
265 Like Test::More's "unlike()". Checks if $this does not match the
267 given $regex.
268 cmp_ok
270 $Test->cmp_ok($this, $type, $that, $name);
271 Works just like Test::More's "cmp_ok()".
273 $Test->cmp_ok($big_num, '!=', $other_big_num);
275 Other Testing Methods
277 These are methods which are used in the course of writing a test but
278 are not themselves tests.
279 BAIL_OUT
281 $Test->BAIL_OUT($reason);
282 Indicates to the Test::Harness that things are going so badly all
284 testing should terminate. This includes running any additional
285 test scripts.
286 It will exit with 255.
288 skip
290 $Test->skip;
291 $Test->skip($why);
292 Skips the current test, reporting $why.
294 todo_skip
296 $Test->todo_skip;
297 $Test->todo_skip($why);
298 Like "skip()", only it will declare the test as failing and TODO.
300 Similar to
301 print "not ok $tnum # TODO $why\n";
303 Test building utility methods
305 These methods are useful when writing your own test methods.
306 maybe_regex
308 $Test->maybe_regex(qr/$regex/);
309 $Test->maybe_regex('/$regex/');
310 This method used to be useful back when Test::Builder worked on
312 Perls before 5.6 which didn't have qr//. Now its pretty useless.
313 Convenience method for building testing functions that take regular
315 expressions as arguments.
316 Takes a quoted regular expression produced by "qr//", or a string
318 representing a regular expression.
319 Returns a Perl value which may be used instead of the corresponding
321 regular expression, or "undef" if its argument is not recognised.
322 For example, a version of "like()", sans the useful diagnostic
324 messages, could be written as:
325 sub laconic_like {
327 my ($self, $this, $regex, $name) = @_;
328 my $usable_regex = $self->maybe_regex($regex);
329 die "expecting regex, found '$regex'\n"
330 unless $usable_regex;
331 $self->ok($this =~ m/$usable_regex/, $name);
332 }
333 is_fh
335 my $is_fh = $Test->is_fh($thing);
336 Determines if the given $thing can be used as a filehandle.
338 Test style
340 level
341 $Test->level($how_high);
342 How far up the call stack should $Test look when reporting where
344 the test failed.
345 Defaults to 1.
347 Setting $Test::Builder::Level overrides. This is typically useful
349 localized:
350 sub my_ok {
352 my $test = shift;
353 local $Test::Builder::Level = $Test::Builder::Level + 1;
355 $TB->ok($test);
356 }
357 To be polite to other functions wrapping your own you usually want
359 to increment $Level rather than set it to a constant.
360 use_numbers
362 $Test->use_numbers($on_or_off);
363 Whether or not the test should output numbers. That is, this if
365 true:
366 ok 1
368 ok 2
369 ok 3
370 or this if false
372 ok
374 ok
375 ok
376 Most useful when you can't depend on the test output order, such as
378 when threads or forking is involved.
379 Defaults to on.
381 no_diag
383 $Test->no_diag($no_diag);
384 If set true no diagnostics will be printed. This includes calls to
386 "diag()".
387 no_ending
389 $Test->no_ending($no_ending);
390 Normally, Test::Builder does some extra diagnostics when the test
392 ends. It also changes the exit code as described below.
393 If this is true, none of that will be done.
395 no_header
397 $Test->no_header($no_header);
398 If set to true, no "1..N" header will be printed.
400 Output
402 Controlling where the test output goes.
403 It's ok for your test to change where STDOUT and STDERR point to,
405 Test::Builder's default output settings will not be affected.
406 diag
408 $Test->diag(@msgs);
409 Prints out the given @msgs. Like "print", arguments are simply
411 appended together.
412 Normally, it uses the "failure_output()" handle, but if this is for
414 a TODO test, the "todo_output()" handle is used.
415 Output will be indented and marked with a # so as not to interfere
417 with test output. A newline will be put on the end if there isn't
418 one already.
419 We encourage using this rather than calling print directly.
421 Returns false. Why? Because "diag()" is often used in conjunction
423 with a failing test ("ok() || diag()") it "passes through" the
424 failure.
425 return ok(...) || diag(...);
427 note
429 $Test->note(@msgs);
430 Like "diag()", but it prints to the "output()" handle so it will
432 not normally be seen by the user except in verbose mode.
433 explain
435 my @dump = $Test->explain(@msgs);
436 Will dump the contents of any references in a human readable
438 format. Handy for things like...
439 is_deeply($have, $want) || diag explain $have;
441 or
443 is_deeply($have, $want) || note explain $have;
445 output
447 failure_output
448 todo_output
449 my $filehandle = $Test->output;
450 $Test->output($filehandle);
451 $Test->output($filename);
452 $Test->output(\$scalar);
453 These methods control where Test::Builder will print its output.
455 They take either an open $filehandle, a $filename to open and write
456 to or a $scalar reference to append to. It will always return a
457 $filehandle.
458 output is where normal "ok/not ok" test output goes.
460 Defaults to STDOUT.
462 failure_output is where diagnostic output on test failures and
464 "diag()" goes. It is normally not read by Test::Harness and
465 instead is displayed to the user.
466 Defaults to STDERR.
468 "todo_output" is used instead of "failure_output()" for the
470 diagnostics of a failing TODO test. These will not be seen by the
471 user.
472 Defaults to STDOUT.
474 reset_outputs
476 $tb->reset_outputs;
477 Resets all the output filehandles back to their defaults.
479 carp
481 $tb->carp(@message);
482 Warns with @message but the message will appear to come from the
484 point where the original test function was called ("$tb->caller").
485 croak
487 $tb->croak(@message);
488 Dies with @message but the message will appear to come from the
490 point where the original test function was called ("$tb->caller").
491 Test Status and Info
493 current_test
494 my $curr_test = $Test->current_test;
495 $Test->current_test($num);
496 Gets/sets the current test number we're on. You usually shouldn't
498 have to set this.
499 If set forward, the details of the missing tests are filled in as
501 'unknown'. if set backward, the details of the intervening tests
502 are deleted. You can erase history if you really want to.
503 is_passing
505 my $ok = $builder->is_passing;
506 Indicates if the test suite is currently passing.
508 More formally, it will be false if anything has happened which
510 makes it impossible for the test suite to pass. True otherwise.
511 For example, if no tests have run "is_passing()" will be true
513 because even though a suite with no tests is a failure you can add
514 a passing test to it and start passing.
515 Don't think about it too much.
517 summary
519 my @tests = $Test->summary;
520 A simple summary of the tests so far. True for pass, false for
522 fail. This is a logical pass/fail, so todos are passes.
523 Of course, test #1 is $tests[0], etc...
525 details
527 my @tests = $Test->details;
528 Like "summary()", but with a lot more detail.
530 $tests[$test_num - 1] =
532 { 'ok' => is the test considered a pass?
533 actual_ok => did it literally say 'ok'?
534 name => name of the test (if any)
535 type => type of test (if any, see below).
536 reason => reason for the above (if any)
537 };
538 'ok' is true if Test::Harness will consider the test to be a pass.
540 'actual_ok' is a reflection of whether or not the test literally
542 printed 'ok' or 'not ok'. This is for examining the result of
543 'todo' tests.
544 'name' is the name of the test.
546 'type' indicates if it was a special test. Normal tests have a
548 type of ''. Type can be one of the following:
549 skip see skip()
551 todo see todo()
552 todo_skip see todo_skip()
553 unknown see below
554 Sometimes the Test::Builder test counter is incremented without it
556 printing any test output, for example, when "current_test()" is
557 changed. In these cases, Test::Builder doesn't know the result of
558 the test, so its type is 'unknown'. These details for these tests
559 are filled in. They are considered ok, but the name and actual_ok
560 is left "undef".
561 For example "not ok 23 - hole count # TODO insufficient donuts"
563 would result in this structure:
564 $tests[22] = # 23 - 1, since arrays start from 0.
566 { ok => 1, # logically, the test passed since its todo
567 actual_ok => 0, # in absolute terms, it failed
568 name => 'hole count',
569 type => 'todo',
570 reason => 'insufficient donuts'
571 };
572 todo
574 my $todo_reason = $Test->todo;
575 my $todo_reason = $Test->todo($pack);
576 If the current tests are considered "TODO" it will return the
578 reason, if any. This reason can come from a $TODO variable or the
579 last call to "todo_start()".
580 Since a TODO test does not need a reason, this function can return
582 an empty string even when inside a TODO block. Use
583 "$Test->in_todo" to determine if you are currently inside a TODO
584 block.
585 "todo()" is about finding the right package to look for $TODO in.
587 It's pretty good at guessing the right package to look at. It
588 first looks for the caller based on "$Level + 1", since "todo()" is
589 usually called inside a test function. As a last resort it will
590 use "exported_to()".
591 Sometimes there is some confusion about where todo() should be
593 looking for the $TODO variable. If you want to be sure, tell it
594 explicitly what $pack to use.
595 find_TODO
597 my $todo_reason = $Test->find_TODO();
598 my $todo_reason = $Test->find_TODO($pack);
599 Like "todo()" but only returns the value of $TODO ignoring
601 "todo_start()".
602 Can also be used to set $TODO to a new value while returning the
604 old value:
605 my $old_reason = $Test->find_TODO($pack, 1, $new_reason);
607 in_todo
609 my $in_todo = $Test->in_todo;
610 Returns true if the test is currently inside a TODO block.
612 todo_start
614 $Test->todo_start();
615 $Test->todo_start($message);
616 This method allows you declare all subsequent tests as TODO tests,
618 up until the "todo_end" method has been called.
619 The "TODO:" and $TODO syntax is generally pretty good about
621 figuring out whether or not we're in a TODO test. However, often
622 we find that this is not possible to determine (such as when we
623 want to use $TODO but the tests are being executed in other
624 packages which can't be inferred beforehand).
625 Note that you can use this to nest "todo" tests
627 $Test->todo_start('working on this');
629 # lots of code
630 $Test->todo_start('working on that');
631 # more code
632 $Test->todo_end;
633 $Test->todo_end;
634 This is generally not recommended, but large testing systems often
636 have weird internal needs.
637 We've tried to make this also work with the TODO: syntax, but it's
639 not guaranteed and its use is also discouraged:
640 TODO: {
642 local $TODO = 'We have work to do!';
643 $Test->todo_start('working on this');
644 # lots of code
645 $Test->todo_start('working on that');
646 # more code
647 $Test->todo_end;
648 $Test->todo_end;
649 }
650 Pick one style or another of "TODO" to be on the safe side.
652 "todo_end"
654 $Test->todo_end;
655 Stops running tests as "TODO" tests. This method is fatal if
657 called without a preceding "todo_start" method call.
658 caller
660 my $package = $Test->caller;
661 my($pack, $file, $line) = $Test->caller;
662 my($pack, $file, $line) = $Test->caller($height);
663 Like the normal "caller()", except it reports according to your
665 "level()".
666 $height will be added to the "level()".
668 If "caller()" winds up off the top of the stack it report the
670 highest context.
671
673 If all your tests passed, Test::Builder will exit with zero (which is
674 normal). If anything failed it will exit with how many failed. If you
675 run less (or more) tests than you planned, the missing (or extras) will
676 be considered failures. If no tests were ever run Test::Builder will
677 throw a warning and exit with 255. If the test died, even after having
678 successfully completed all its tests, it will still be considered a
679 failure and will exit with 255.
680 So the exit codes are...
682 0 all tests successful
684 255 test died or all passed but wrong # of tests run
685 any other number how many failed (including missing or extras)
686 If you fail more than 254 tests, it will be reported as 254.
688
690 In perl 5.8.1 and later, Test::Builder is thread-safe. The test number
691 is shared amongst all threads. This means if one thread sets the test
692 number using "current_test()" they will all be effected.
693 While versions earlier than 5.8.1 had threads they contain too many
695 bugs to support.
696 Test::Builder is only thread-aware if threads.pm is loaded before
698 Test::Builder.
699
701 An informative hash, accessible via "<details()">, is stored for each
702 test you perform. So memory usage will scale linearly with each test
703 run. Although this is not a problem for most test suites, it can become
704 an issue if you do large (hundred thousands to million) combinatorics
705 tests in the same run.
706 In such cases, you are advised to either split the test file into
708 smaller ones, or use a reverse approach, doing "normal" (code) compares
709 and triggering fail() should anything go unexpected.
710 Future versions of Test::Builder will have a way to turn history off.
712
714 CPAN can provide the best examples. Test::Simple, Test::More,
715 Test::Exception and Test::Differences all use Test::Builder.
716
718 Test::Simple, Test::More, Test::Harness
719
721 Original code by chromatic, maintained by Michael G Schwern
722 <schwern@pobox.com>
723
725 Copyright 2002-2008 by chromatic <chromatic@wgz.org> and
726 Michael G Schwern <schwern@pobox.com>.
727 This program is free software; you can redistribute it and/or modify it
729 under the same terms as Perl itself.
730 See http://www.perl.com/perl/misc/Artistic.html
732perl v5.12.3 2011-02-23 Test::Builder(3)