1Test::Builder(3) User Contributed Perl Documentation Test::Builder(3)
2
6 Test::Builder - Backend for building test libraries
7
9 package My::Test::Module;
10 use base 'Test::Builder::Module';
11 my $CLASS = __PACKAGE__;
13 sub ok {
15 my($test, $name) = @_;
16 my $tb = $CLASS->builder;
17 $tb->ok($test, $name);
19 }
20
22 Test::Simple and Test::More have proven to be popular testing modules,
23 but they're not always flexible enough. Test::Builder provides a
24 building block upon which to write your own test libraries which can
25 work together.
26 Construction
28 new
29 my $Test = Test::Builder->new;
30 Returns a Test::Builder object representing the current state of
32 the test.
33 Since you only run one test per program "new" always returns the
35 same Test::Builder object. No matter how many times you call
36 "new()", you're getting the same object. This is called a
37 singleton. This is done so that multiple modules share such global
38 information as the test counter and where test output is going.
39 If you want a completely new Test::Builder object different from
41 the singleton, use "create".
42 create
44 my $Test = Test::Builder->create;
45 Ok, so there can be more than one Test::Builder object and this is
47 how you get it. You might use this instead of "new()" if you're
48 testing a Test::Builder based module, but otherwise you probably
49 want "new".
50 NOTE: the implementation is not complete. "level", for example, is
52 still shared by all Test::Builder objects, even ones created using
53 this method. Also, the method name may change in the future.
54 subtest
56 $builder->subtest($name, \&subtests, @args);
57 See documentation of "subtest" in Test::More.
59 "subtest" also, and optionally, accepts arguments which will be
61 passed to the subtests reference.
62 name
64 diag $builder->name;
65 Returns the name of the current builder. Top level builders
67 default to $0 (the name of the executable). Child builders are
68 named via the "child" method. If no name is supplied, will be
69 named "Child of $parent->name".
70 reset
72 $Test->reset;
73 Reinitializes the Test::Builder singleton to its original state.
75 Mostly useful for tests run in persistent environments where the
76 same test might be run multiple times in the same process.
77 Setting up tests
79 These methods are for setting up tests and declaring how many there
80 are. You usually only want to call one of these methods.
81 plan
83 $Test->plan('no_plan');
84 $Test->plan( skip_all => $reason );
85 $Test->plan( tests => $num_tests );
86 A convenient way to set up your tests. Call this and Test::Builder
88 will print the appropriate headers and take the appropriate
89 actions.
90 If you call "plan()", don't call any of the other methods below.
92 expected_tests
94 my $max = $Test->expected_tests;
95 $Test->expected_tests($max);
96 Gets/sets the number of tests we expect this test to run and prints
98 out the appropriate headers.
99 no_plan
101 $Test->no_plan;
102 Declares that this test will run an indeterminate number of tests.
104 done_testing
106 $Test->done_testing();
107 $Test->done_testing($num_tests);
108 Declares that you are done testing, no more tests will be run after
110 this point.
111 If a plan has not yet been output, it will do so.
113 $num_tests is the number of tests you planned to run. If a
115 numbered plan was already declared, and if this contradicts, a
116 failing test will be run to reflect the planning mistake. If
117 "no_plan" was declared, this will override.
118 If "done_testing()" is called twice, the second call will issue a
120 failing test.
121 If $num_tests is omitted, the number of tests run will be used,
123 like no_plan.
124 "done_testing()" is, in effect, used when you'd want to use
126 "no_plan", but safer. You'd use it like so:
127 $Test->ok($a == $b);
129 $Test->done_testing();
130 Or to plan a variable number of tests:
132 for my $test (@tests) {
134 $Test->ok($test);
135 }
136 $Test->done_testing(scalar @tests);
137 has_plan
139 $plan = $Test->has_plan
140 Find out whether a plan has been defined. $plan is either "undef"
142 (no plan has been set), "no_plan" (indeterminate # of tests) or an
143 integer (the number of expected tests).
144 skip_all
146 $Test->skip_all;
147 $Test->skip_all($reason);
148 Skips all the tests, using the given $reason. Exits immediately
150 with 0.
151 exported_to
153 my $pack = $Test->exported_to;
154 $Test->exported_to($pack);
155 Tells Test::Builder what package you exported your functions to.
157 This method isn't terribly useful since modules which share the
159 same Test::Builder object might get exported to different packages
160 and only the last one will be honored.
161 Running tests
163 These actually run the tests, analogous to the functions in Test::More.
164 They all return true if the test passed, false if the test failed.
166 $name is always optional.
168 ok
170 $Test->ok($test, $name);
171 Your basic test. Pass if $test is true, fail if $test is false.
173 Just like Test::Simple's "ok()".
174 is_eq
176 $Test->is_eq($got, $expected, $name);
177 Like Test::More's "is()". Checks if "$got eq $expected". This is
179 the string version.
180 "undef" only ever matches another "undef".
182 is_num
184 $Test->is_num($got, $expected, $name);
185 Like Test::More's "is()". Checks if "$got == $expected". This is
187 the numeric version.
188 "undef" only ever matches another "undef".
190 isnt_eq
192 $Test->isnt_eq($got, $dont_expect, $name);
193 Like Test::More's "isnt()". Checks if "$got ne $dont_expect".
195 This is the string version.
196 isnt_num
198 $Test->isnt_num($got, $dont_expect, $name);
199 Like Test::More's "isnt()". Checks if "$got ne $dont_expect".
201 This is the numeric version.
202 like
204 $Test->like($thing, qr/$regex/, $name);
205 $Test->like($thing, '/$regex/', $name);
206 Like Test::More's "like()". Checks if $thing matches the given
208 $regex.
209 unlike
211 $Test->unlike($thing, qr/$regex/, $name);
212 $Test->unlike($thing, '/$regex/', $name);
213 Like Test::More's "unlike()". Checks if $thing does not match the
215 given $regex.
216 cmp_ok
218 $Test->cmp_ok($thing, $type, $that, $name);
219 Works just like Test::More's "cmp_ok()".
221 $Test->cmp_ok($big_num, '!=', $other_big_num);
223 Other Testing Methods
225 These are methods which are used in the course of writing a test but
226 are not themselves tests.
227 BAIL_OUT
229 $Test->BAIL_OUT($reason);
230 Indicates to the Test::Harness that things are going so badly all
232 testing should terminate. This includes running any additional
233 test scripts.
234 It will exit with 255.
236 skip
238 $Test->skip;
239 $Test->skip($why);
240 Skips the current test, reporting $why.
242 todo_skip
244 $Test->todo_skip;
245 $Test->todo_skip($why);
246 Like "skip()", only it will declare the test as failing and TODO.
248 Similar to
249 print "not ok $tnum # TODO $why\n";
251 Test building utility methods
253 These methods are useful when writing your own test methods.
254 maybe_regex
256 $Test->maybe_regex(qr/$regex/);
257 $Test->maybe_regex('/$regex/');
258 This method used to be useful back when Test::Builder worked on
260 Perls before 5.6 which didn't have qr//. Now its pretty useless.
261 Convenience method for building testing functions that take regular
263 expressions as arguments.
264 Takes a quoted regular expression produced by "qr//", or a string
266 representing a regular expression.
267 Returns a Perl value which may be used instead of the corresponding
269 regular expression, or "undef" if its argument is not recognized.
270 For example, a version of "like()", sans the useful diagnostic
272 messages, could be written as:
273 sub laconic_like {
275 my ($self, $thing, $regex, $name) = @_;
276 my $usable_regex = $self->maybe_regex($regex);
277 die "expecting regex, found '$regex'\n"
278 unless $usable_regex;
279 $self->ok($thing =~ m/$usable_regex/, $name);
280 }
281 is_fh
283 my $is_fh = $Test->is_fh($thing);
284 Determines if the given $thing can be used as a filehandle.
286 Test style
288 level
289 $Test->level($how_high);
290 How far up the call stack should $Test look when reporting where
292 the test failed.
293 Defaults to 1.
295 Setting $Test::Builder::Level overrides. This is typically useful
297 localized:
298 sub my_ok {
300 my $test = shift;
301 local $Test::Builder::Level = $Test::Builder::Level + 1;
303 $TB->ok($test);
304 }
305 To be polite to other functions wrapping your own you usually want
307 to increment $Level rather than set it to a constant.
308 use_numbers
310 $Test->use_numbers($on_or_off);
311 Whether or not the test should output numbers. That is, this if
313 true:
314 ok 1
316 ok 2
317 ok 3
318 or this if false
320 ok
322 ok
323 ok
324 Most useful when you can't depend on the test output order, such as
326 when threads or forking is involved.
327 Defaults to on.
329 no_diag
331 $Test->no_diag($no_diag);
332 If set true no diagnostics will be printed. This includes calls to
334 "diag()".
335 no_ending
337 $Test->no_ending($no_ending);
338 Normally, Test::Builder does some extra diagnostics when the test
340 ends. It also changes the exit code as described below.
341 If this is true, none of that will be done.
343 no_header
345 $Test->no_header($no_header);
346 If set to true, no "1..N" header will be printed.
348 Output
350 Controlling where the test output goes.
351 It's ok for your test to change where STDOUT and STDERR point to,
353 Test::Builder's default output settings will not be affected.
354 diag
356 $Test->diag(@msgs);
357 Prints out the given @msgs. Like "print", arguments are simply
359 appended together.
360 Normally, it uses the "failure_output()" handle, but if this is for
362 a TODO test, the "todo_output()" handle is used.
363 Output will be indented and marked with a # so as not to interfere
365 with test output. A newline will be put on the end if there isn't
366 one already.
367 We encourage using this rather than calling print directly.
369 Returns false. Why? Because "diag()" is often used in conjunction
371 with a failing test ("ok() || diag()") it "passes through" the
372 failure.
373 return ok(...) || diag(...);
375 note
377 $Test->note(@msgs);
378 Like "diag()", but it prints to the "output()" handle so it will
380 not normally be seen by the user except in verbose mode.
381 explain
383 my @dump = $Test->explain(@msgs);
384 Will dump the contents of any references in a human readable
386 format. Handy for things like...
387 is_deeply($have, $want) || diag explain $have;
389 or
391 is_deeply($have, $want) || note explain $have;
393 output
395 failure_output
396 todo_output
397 my $filehandle = $Test->output;
398 $Test->output($filehandle);
399 $Test->output($filename);
400 $Test->output(\$scalar);
401 These methods control where Test::Builder will print its output.
403 They take either an open $filehandle, a $filename to open and write
404 to or a $scalar reference to append to. It will always return a
405 $filehandle.
406 output is where normal "ok/not ok" test output goes.
408 Defaults to STDOUT.
410 failure_output is where diagnostic output on test failures and
412 "diag()" goes. It is normally not read by Test::Harness and
413 instead is displayed to the user.
414 Defaults to STDERR.
416 "todo_output" is used instead of "failure_output()" for the
418 diagnostics of a failing TODO test. These will not be seen by the
419 user.
420 Defaults to STDOUT.
422 reset_outputs
424 $tb->reset_outputs;
425 Resets all the output filehandles back to their defaults.
427 carp
429 $tb->carp(@message);
430 Warns with @message but the message will appear to come from the
432 point where the original test function was called ("$tb->caller").
433 croak
435 $tb->croak(@message);
436 Dies with @message but the message will appear to come from the
438 point where the original test function was called ("$tb->caller").
439 Test Status and Info
441 no_log_results
442 This will turn off result long-term storage. Calling this method
443 will make "details" and "summary" useless. You may want to use this
444 if you are running enough tests to fill up all available memory.
445 Test::Builder->new->no_log_results();
447 There is no way to turn it back on.
449 current_test
451 my $curr_test = $Test->current_test;
452 $Test->current_test($num);
453 Gets/sets the current test number we're on. You usually shouldn't
455 have to set this.
456 If set forward, the details of the missing tests are filled in as
458 'unknown'. if set backward, the details of the intervening tests
459 are deleted. You can erase history if you really want to.
460 is_passing
462 my $ok = $builder->is_passing;
463 Indicates if the test suite is currently passing.
465 More formally, it will be false if anything has happened which
467 makes it impossible for the test suite to pass. True otherwise.
468 For example, if no tests have run "is_passing()" will be true
470 because even though a suite with no tests is a failure you can add
471 a passing test to it and start passing.
472 Don't think about it too much.
474 summary
476 my @tests = $Test->summary;
477 A simple summary of the tests so far. True for pass, false for
479 fail. This is a logical pass/fail, so todos are passes.
480 Of course, test #1 is $tests[0], etc...
482 details
484 my @tests = $Test->details;
485 Like "summary()", but with a lot more detail.
487 $tests[$test_num - 1] =
489 { 'ok' => is the test considered a pass?
490 actual_ok => did it literally say 'ok'?
491 name => name of the test (if any)
492 type => type of test (if any, see below).
493 reason => reason for the above (if any)
494 };
495 'ok' is true if Test::Harness will consider the test to be a pass.
497 'actual_ok' is a reflection of whether or not the test literally
499 printed 'ok' or 'not ok'. This is for examining the result of
500 'todo' tests.
501 'name' is the name of the test.
503 'type' indicates if it was a special test. Normal tests have a
505 type of ''. Type can be one of the following:
506 skip see skip()
508 todo see todo()
509 todo_skip see todo_skip()
510 unknown see below
511 Sometimes the Test::Builder test counter is incremented without it
513 printing any test output, for example, when "current_test()" is
514 changed. In these cases, Test::Builder doesn't know the result of
515 the test, so its type is 'unknown'. These details for these tests
516 are filled in. They are considered ok, but the name and actual_ok
517 is left "undef".
518 For example "not ok 23 - hole count # TODO insufficient donuts"
520 would result in this structure:
521 $tests[22] = # 23 - 1, since arrays start from 0.
523 { ok => 1, # logically, the test passed since its todo
524 actual_ok => 0, # in absolute terms, it failed
525 name => 'hole count',
526 type => 'todo',
527 reason => 'insufficient donuts'
528 };
529 todo
531 my $todo_reason = $Test->todo;
532 my $todo_reason = $Test->todo($pack);
533 If the current tests are considered "TODO" it will return the
535 reason, if any. This reason can come from a $TODO variable or the
536 last call to "todo_start()".
537 Since a TODO test does not need a reason, this function can return
539 an empty string even when inside a TODO block. Use
540 "$Test->in_todo" to determine if you are currently inside a TODO
541 block.
542 "todo()" is about finding the right package to look for $TODO in.
544 It's pretty good at guessing the right package to look at. It
545 first looks for the caller based on "$Level + 1", since "todo()" is
546 usually called inside a test function. As a last resort it will
547 use "exported_to()".
548 Sometimes there is some confusion about where "todo()" should be
550 looking for the $TODO variable. If you want to be sure, tell it
551 explicitly what $pack to use.
552 find_TODO
554 my $todo_reason = $Test->find_TODO();
555 my $todo_reason = $Test->find_TODO($pack);
556 Like "todo()" but only returns the value of $TODO ignoring
558 "todo_start()".
559 Can also be used to set $TODO to a new value while returning the
561 old value:
562 my $old_reason = $Test->find_TODO($pack, 1, $new_reason);
564 in_todo
566 my $in_todo = $Test->in_todo;
567 Returns true if the test is currently inside a TODO block.
569 todo_start
571 $Test->todo_start();
572 $Test->todo_start($message);
573 This method allows you declare all subsequent tests as TODO tests,
575 up until the "todo_end" method has been called.
576 The "TODO:" and $TODO syntax is generally pretty good about
578 figuring out whether or not we're in a TODO test. However, often
579 we find that this is not possible to determine (such as when we
580 want to use $TODO but the tests are being executed in other
581 packages which can't be inferred beforehand).
582 Note that you can use this to nest "todo" tests
584 $Test->todo_start('working on this');
586 # lots of code
587 $Test->todo_start('working on that');
588 # more code
589 $Test->todo_end;
590 $Test->todo_end;
591 This is generally not recommended, but large testing systems often
593 have weird internal needs.
594 We've tried to make this also work with the TODO: syntax, but it's
596 not guaranteed and its use is also discouraged:
597 TODO: {
599 local $TODO = 'We have work to do!';
600 $Test->todo_start('working on this');
601 # lots of code
602 $Test->todo_start('working on that');
603 # more code
604 $Test->todo_end;
605 $Test->todo_end;
606 }
607 Pick one style or another of "TODO" to be on the safe side.
609 "todo_end"
611 $Test->todo_end;
612 Stops running tests as "TODO" tests. This method is fatal if
614 called without a preceding "todo_start" method call.
615 caller
617 my $package = $Test->caller;
618 my($pack, $file, $line) = $Test->caller;
619 my($pack, $file, $line) = $Test->caller($height);
620 Like the normal "caller()", except it reports according to your
622 "level()".
623 $height will be added to the "level()".
625 If "caller()" winds up off the top of the stack it report the
627 highest context.
628
630 If all your tests passed, Test::Builder will exit with zero (which is
631 normal). If anything failed it will exit with how many failed. If you
632 run less (or more) tests than you planned, the missing (or extras) will
633 be considered failures. If no tests were ever run Test::Builder will
634 throw a warning and exit with 255. If the test died, even after having
635 successfully completed all its tests, it will still be considered a
636 failure and will exit with 255.
637 So the exit codes are...
639 0 all tests successful
641 255 test died or all passed but wrong # of tests run
642 any other number how many failed (including missing or extras)
643 If you fail more than 254 tests, it will be reported as 254.
645
647 In perl 5.8.1 and later, Test::Builder is thread-safe. The test number
648 is shared by all threads. This means if one thread sets the test
649 number using "current_test()" they will all be effected.
650 While versions earlier than 5.8.1 had threads they contain too many
652 bugs to support.
653 Test::Builder is only thread-aware if threads.pm is loaded before
655 Test::Builder.
656 You can directly disable thread support with one of the following:
658 $ENV{T2_NO_IPC} = 1
660 or
662 no Test2::IPC;
664 or
666 Test2::API::test2_ipc_disable()
668
670 An informative hash, accessible via "details()", is stored for each
671 test you perform. So memory usage will scale linearly with each test
672 run. Although this is not a problem for most test suites, it can become
673 an issue if you do large (hundred thousands to million) combinatorics
674 tests in the same run.
675 In such cases, you are advised to either split the test file into
677 smaller ones, or use a reverse approach, doing "normal" (code) compares
678 and triggering "fail()" should anything go unexpected.
679 Future versions of Test::Builder will have a way to turn history off.
681
683 CPAN can provide the best examples. Test::Simple, Test::More,
684 Test::Exception and Test::Differences all use Test::Builder.
685
687 INTERNALS
688 Test2, Test2::API
689 LEGACY
691 Test::Simple, Test::More
692 EXTERNAL
694 Test::Harness
695
697 Original code by chromatic, maintained by Michael G Schwern
698 <schwern@pobox.com>
699
701 Chad Granum <exodist@cpan.org>
702
704 Copyright 2002-2008 by chromatic <chromatic@wgz.org> and
705 Michael G Schwern <schwern@pobox.com>.
706 This program is free software; you can redistribute it and/or modify it
708 under the same terms as Perl itself.
709 See http://www.perl.com/perl/misc/Artistic.html
711perl v5.32.0 2020-10-15 Test::Builder(3)