1Test::Builder(3)      User Contributed Perl Documentation     Test::Builder(3)
2
3
4

NAME

6       Test::Builder - Backend for building test libraries
7

SYNOPSIS

9         package My::Test::Module;
10         use base 'Test::Builder::Module';
11
12         my $CLASS = __PACKAGE__;
13
14         sub ok {
15             my($test, $name) = @_;
16             my $tb = $CLASS->builder;
17
18             $tb->ok($test, $name);
19         }
20

DESCRIPTION

22       Test::Simple and Test::More have proven to be popular testing modules,
23       but they're not always flexible enough.  Test::Builder provides a
24       building block upon which to write your own test libraries which can
25       work together.
26
27   Construction
28       new
29             my $Test = Test::Builder->new;
30
31           Returns a Test::Builder object representing the current state of
32           the test.
33
34           Since you only run one test per program "new" always returns the
35           same Test::Builder object.  No matter how many times you call
36           "new()", you're getting the same object.  This is called a
37           singleton.  This is done so that multiple modules share such global
38           information as the test counter and where test output is going.
39
40           If you want a completely new Test::Builder object different from
41           the singleton, use "create".
42
43       create
44             my $Test = Test::Builder->create;
45
46           Ok, so there can be more than one Test::Builder object and this is
47           how you get it.  You might use this instead of "new()" if you're
48           testing a Test::Builder based module, but otherwise you probably
49           want "new".
50
51           NOTE: the implementation is not complete.  "level", for example, is
52           still shared by all Test::Builder objects, even ones created using
53           this method.  Also, the method name may change in the future.
54
55       subtest
56               $builder->subtest($name, \&subtests, @args);
57
58           See documentation of "subtest" in Test::More.
59
60           "subtest" also, and optionally, accepts arguments which will be
61           passed to the subtests reference.
62
63       name
64            diag $builder->name;
65
66           Returns the name of the current builder.  Top level builders
67           default to $0 (the name of the executable).  Child builders are
68           named via the "child" method.  If no name is supplied, will be
69           named "Child of $parent->name".
70
71       reset
72             $Test->reset;
73
74           Reinitializes the Test::Builder singleton to its original state.
75           Mostly useful for tests run in persistent environments where the
76           same test might be run multiple times in the same process.
77
78   Setting up tests
79       These methods are for setting up tests and declaring how many there
80       are.  You usually only want to call one of these methods.
81
82       plan
83             $Test->plan('no_plan');
84             $Test->plan( skip_all => $reason );
85             $Test->plan( tests => $num_tests );
86
87           A convenient way to set up your tests.  Call this and Test::Builder
88           will print the appropriate headers and take the appropriate
89           actions.
90
91           If you call "plan()", don't call any of the other methods below.
92
93       expected_tests
94               my $max = $Test->expected_tests;
95               $Test->expected_tests($max);
96
97           Gets/sets the number of tests we expect this test to run and prints
98           out the appropriate headers.
99
100       no_plan
101             $Test->no_plan;
102
103           Declares that this test will run an indeterminate number of tests.
104
105       done_testing
106             $Test->done_testing();
107             $Test->done_testing($num_tests);
108
109           Declares that you are done testing, no more tests will be run after
110           this point.
111
112           If a plan has not yet been output, it will do so.
113
114           $num_tests is the number of tests you planned to run.  If a
115           numbered plan was already declared, and if this contradicts, a
116           failing test will be run to reflect the planning mistake.  If
117           "no_plan" was declared, this will override.
118
119           If "done_testing()" is called twice, the second call will issue a
120           failing test.
121
122           If $num_tests is omitted, the number of tests run will be used,
123           like no_plan.
124
125           "done_testing()" is, in effect, used when you'd want to use
126           "no_plan", but safer. You'd use it like so:
127
128               $Test->ok($a == $b);
129               $Test->done_testing();
130
131           Or to plan a variable number of tests:
132
133               for my $test (@tests) {
134                   $Test->ok($test);
135               }
136               $Test->done_testing(scalar @tests);
137
138       has_plan
139             $plan = $Test->has_plan
140
141           Find out whether a plan has been defined. $plan is either "undef"
142           (no plan has been set), "no_plan" (indeterminate # of tests) or an
143           integer (the number of expected tests).
144
145       skip_all
146             $Test->skip_all;
147             $Test->skip_all($reason);
148
149           Skips all the tests, using the given $reason.  Exits immediately
150           with 0.
151
152       exported_to
153             my $pack = $Test->exported_to;
154             $Test->exported_to($pack);
155
156           Tells Test::Builder what package you exported your functions to.
157
158           This method isn't terribly useful since modules which share the
159           same Test::Builder object might get exported to different packages
160           and only the last one will be honored.
161
162   Running tests
163       These actually run the tests, analogous to the functions in Test::More.
164
165       They all return true if the test passed, false if the test failed.
166
167       $name is always optional.
168
169       ok
170             $Test->ok($test, $name);
171
172           Your basic test.  Pass if $test is true, fail if $test is false.
173           Just like Test::Simple's "ok()".
174
175       is_eq
176             $Test->is_eq($got, $expected, $name);
177
178           Like Test::More's "is()".  Checks if "$got eq $expected".  This is
179           the string version.
180
181           "undef" only ever matches another "undef".
182
183       is_num
184             $Test->is_num($got, $expected, $name);
185
186           Like Test::More's "is()".  Checks if "$got == $expected".  This is
187           the numeric version.
188
189           "undef" only ever matches another "undef".
190
191       isnt_eq
192             $Test->isnt_eq($got, $dont_expect, $name);
193
194           Like Test::More's "isnt()".  Checks if "$got ne $dont_expect".
195           This is the string version.
196
197       isnt_num
198             $Test->isnt_num($got, $dont_expect, $name);
199
200           Like Test::More's "isnt()".  Checks if "$got ne $dont_expect".
201           This is the numeric version.
202
203       like
204             $Test->like($thing, qr/$regex/, $name);
205             $Test->like($thing, '/$regex/', $name);
206
207           Like Test::More's "like()".  Checks if $thing matches the given
208           $regex.
209
210       unlike
211             $Test->unlike($thing, qr/$regex/, $name);
212             $Test->unlike($thing, '/$regex/', $name);
213
214           Like Test::More's "unlike()".  Checks if $thing does not match the
215           given $regex.
216
217       cmp_ok
218             $Test->cmp_ok($thing, $type, $that, $name);
219
220           Works just like Test::More's "cmp_ok()".
221
222               $Test->cmp_ok($big_num, '!=', $other_big_num);
223
224   Other Testing Methods
225       These are methods which are used in the course of writing a test but
226       are not themselves tests.
227
228       BAIL_OUT
229               $Test->BAIL_OUT($reason);
230
231           Indicates to the Test::Harness that things are going so badly all
232           testing should terminate.  This includes running any additional
233           test scripts.
234
235           It will exit with 255.
236
237       skip
238               $Test->skip;
239               $Test->skip($why);
240
241           Skips the current test, reporting $why.
242
243       todo_skip
244             $Test->todo_skip;
245             $Test->todo_skip($why);
246
247           Like "skip()", only it will declare the test as failing and TODO.
248           Similar to
249
250               print "not ok $tnum # TODO $why\n";
251
252   Test building utility methods
253       These methods are useful when writing your own test methods.
254
255       maybe_regex
256             $Test->maybe_regex(qr/$regex/);
257             $Test->maybe_regex('/$regex/');
258
259           This method used to be useful back when Test::Builder worked on
260           Perls before 5.6 which didn't have qr//.  Now its pretty useless.
261
262           Convenience method for building testing functions that take regular
263           expressions as arguments.
264
265           Takes a quoted regular expression produced by "qr//", or a string
266           representing a regular expression.
267
268           Returns a Perl value which may be used instead of the corresponding
269           regular expression, or "undef" if its argument is not recognized.
270
271           For example, a version of "like()", sans the useful diagnostic
272           messages, could be written as:
273
274             sub laconic_like {
275                 my ($self, $thing, $regex, $name) = @_;
276                 my $usable_regex = $self->maybe_regex($regex);
277                 die "expecting regex, found '$regex'\n"
278                     unless $usable_regex;
279                 $self->ok($thing =~ m/$usable_regex/, $name);
280             }
281
282       is_fh
283               my $is_fh = $Test->is_fh($thing);
284
285           Determines if the given $thing can be used as a filehandle.
286
287   Test style
288       level
289               $Test->level($how_high);
290
291           How far up the call stack should $Test look when reporting where
292           the test failed.
293
294           Defaults to 1.
295
296           Setting $Test::Builder::Level overrides.  This is typically useful
297           localized:
298
299               sub my_ok {
300                   my $test = shift;
301
302                   local $Test::Builder::Level = $Test::Builder::Level + 1;
303                   $TB->ok($test);
304               }
305
306           To be polite to other functions wrapping your own you usually want
307           to increment $Level rather than set it to a constant.
308
309       use_numbers
310               $Test->use_numbers($on_or_off);
311
312           Whether or not the test should output numbers.  That is, this if
313           true:
314
315             ok 1
316             ok 2
317             ok 3
318
319           or this if false
320
321             ok
322             ok
323             ok
324
325           Most useful when you can't depend on the test output order, such as
326           when threads or forking is involved.
327
328           Defaults to on.
329
330       no_diag
331               $Test->no_diag($no_diag);
332
333           If set true no diagnostics will be printed.  This includes calls to
334           "diag()".
335
336       no_ending
337               $Test->no_ending($no_ending);
338
339           Normally, Test::Builder does some extra diagnostics when the test
340           ends.  It also changes the exit code as described below.
341
342           If this is true, none of that will be done.
343
344       no_header
345               $Test->no_header($no_header);
346
347           If set to true, no "1..N" header will be printed.
348
349   Output
350       Controlling where the test output goes.
351
352       It's ok for your test to change where STDOUT and STDERR point to,
353       Test::Builder's default output settings will not be affected.
354
355       diag
356               $Test->diag(@msgs);
357
358           Prints out the given @msgs.  Like "print", arguments are simply
359           appended together.
360
361           Normally, it uses the "failure_output()" handle, but if this is for
362           a TODO test, the "todo_output()" handle is used.
363
364           Output will be indented and marked with a # so as not to interfere
365           with test output.  A newline will be put on the end if there isn't
366           one already.
367
368           We encourage using this rather than calling print directly.
369
370           Returns false.  Why?  Because "diag()" is often used in conjunction
371           with a failing test ("ok() || diag()") it "passes through" the
372           failure.
373
374               return ok(...) || diag(...);
375
376       note
377               $Test->note(@msgs);
378
379           Like "diag()", but it prints to the "output()" handle so it will
380           not normally be seen by the user except in verbose mode.
381
382       explain
383               my @dump = $Test->explain(@msgs);
384
385           Will dump the contents of any references in a human readable
386           format.  Handy for things like...
387
388               is_deeply($have, $want) || diag explain $have;
389
390           or
391
392               is_deeply($have, $want) || note explain $have;
393
394       output
395       failure_output
396       todo_output
397               my $filehandle = $Test->output;
398               $Test->output($filehandle);
399               $Test->output($filename);
400               $Test->output(\$scalar);
401
402           These methods control where Test::Builder will print its output.
403           They take either an open $filehandle, a $filename to open and write
404           to or a $scalar reference to append to.  It will always return a
405           $filehandle.
406
407           output is where normal "ok/not ok" test output goes.
408
409           Defaults to STDOUT.
410
411           failure_output is where diagnostic output on test failures and
412           "diag()" goes.  It is normally not read by Test::Harness and
413           instead is displayed to the user.
414
415           Defaults to STDERR.
416
417           "todo_output" is used instead of "failure_output()" for the
418           diagnostics of a failing TODO test.  These will not be seen by the
419           user.
420
421           Defaults to STDOUT.
422
423       reset_outputs
424             $tb->reset_outputs;
425
426           Resets all the output filehandles back to their defaults.
427
428       carp
429             $tb->carp(@message);
430
431           Warns with @message but the message will appear to come from the
432           point where the original test function was called ("$tb->caller").
433
434       croak
435             $tb->croak(@message);
436
437           Dies with @message but the message will appear to come from the
438           point where the original test function was called ("$tb->caller").
439
440   Test Status and Info
441       no_log_results
442           This will turn off result long-term storage. Calling this method
443           will make "details" and "summary" useless. You may want to use this
444           if you are running enough tests to fill up all available memory.
445
446               Test::Builder->new->no_log_results();
447
448           There is no way to turn it back on.
449
450       current_test
451               my $curr_test = $Test->current_test;
452               $Test->current_test($num);
453
454           Gets/sets the current test number we're on.  You usually shouldn't
455           have to set this.
456
457           If set forward, the details of the missing tests are filled in as
458           'unknown'.  if set backward, the details of the intervening tests
459           are deleted.  You can erase history if you really want to.
460
461       is_passing
462              my $ok = $builder->is_passing;
463
464           Indicates if the test suite is currently passing.
465
466           More formally, it will be false if anything has happened which
467           makes it impossible for the test suite to pass.  True otherwise.
468
469           For example, if no tests have run "is_passing()" will be true
470           because even though a suite with no tests is a failure you can add
471           a passing test to it and start passing.
472
473           Don't think about it too much.
474
475       summary
476               my @tests = $Test->summary;
477
478           A simple summary of the tests so far.  True for pass, false for
479           fail.  This is a logical pass/fail, so todos are passes.
480
481           Of course, test #1 is $tests[0], etc...
482
483       details
484               my @tests = $Test->details;
485
486           Like "summary()", but with a lot more detail.
487
488               $tests[$test_num - 1] =
489                       { 'ok'       => is the test considered a pass?
490                         actual_ok  => did it literally say 'ok'?
491                         name       => name of the test (if any)
492                         type       => type of test (if any, see below).
493                         reason     => reason for the above (if any)
494                       };
495
496           'ok' is true if Test::Harness will consider the test to be a pass.
497
498           'actual_ok' is a reflection of whether or not the test literally
499           printed 'ok' or 'not ok'.  This is for examining the result of
500           'todo' tests.
501
502           'name' is the name of the test.
503
504           'type' indicates if it was a special test.  Normal tests have a
505           type of ''.  Type can be one of the following:
506
507               skip        see skip()
508               todo        see todo()
509               todo_skip   see todo_skip()
510               unknown     see below
511
512           Sometimes the Test::Builder test counter is incremented without it
513           printing any test output, for example, when "current_test()" is
514           changed.  In these cases, Test::Builder doesn't know the result of
515           the test, so its type is 'unknown'.  These details for these tests
516           are filled in.  They are considered ok, but the name and actual_ok
517           is left "undef".
518
519           For example "not ok 23 - hole count # TODO insufficient donuts"
520           would result in this structure:
521
522               $tests[22] =    # 23 - 1, since arrays start from 0.
523                 { ok        => 1,   # logically, the test passed since its todo
524                   actual_ok => 0,   # in absolute terms, it failed
525                   name      => 'hole count',
526                   type      => 'todo',
527                   reason    => 'insufficient donuts'
528                 };
529
530       todo
531               my $todo_reason = $Test->todo;
532               my $todo_reason = $Test->todo($pack);
533
534           If the current tests are considered "TODO" it will return the
535           reason, if any.  This reason can come from a $TODO variable or the
536           last call to "todo_start()".
537
538           Since a TODO test does not need a reason, this function can return
539           an empty string even when inside a TODO block.  Use
540           "$Test->in_todo" to determine if you are currently inside a TODO
541           block.
542
543           "todo()" is about finding the right package to look for $TODO in.
544           It's pretty good at guessing the right package to look at.  It
545           first looks for the caller based on "$Level + 1", since "todo()" is
546           usually called inside a test function.  As a last resort it will
547           use "exported_to()".
548
549           Sometimes there is some confusion about where "todo()" should be
550           looking for the $TODO variable.  If you want to be sure, tell it
551           explicitly what $pack to use.
552
553       find_TODO
554               my $todo_reason = $Test->find_TODO();
555               my $todo_reason = $Test->find_TODO($pack);
556
557           Like "todo()" but only returns the value of $TODO ignoring
558           "todo_start()".
559
560           Can also be used to set $TODO to a new value while returning the
561           old value:
562
563               my $old_reason = $Test->find_TODO($pack, 1, $new_reason);
564
565       in_todo
566               my $in_todo = $Test->in_todo;
567
568           Returns true if the test is currently inside a TODO block.
569
570       todo_start
571               $Test->todo_start();
572               $Test->todo_start($message);
573
574           This method allows you declare all subsequent tests as TODO tests,
575           up until the "todo_end" method has been called.
576
577           The "TODO:" and $TODO syntax is generally pretty good about
578           figuring out whether or not we're in a TODO test.  However, often
579           we find that this is not possible to determine (such as when we
580           want to use $TODO but the tests are being executed in other
581           packages which can't be inferred beforehand).
582
583           Note that you can use this to nest "todo" tests
584
585            $Test->todo_start('working on this');
586            # lots of code
587            $Test->todo_start('working on that');
588            # more code
589            $Test->todo_end;
590            $Test->todo_end;
591
592           This is generally not recommended, but large testing systems often
593           have weird internal needs.
594
595           We've tried to make this also work with the TODO: syntax, but it's
596           not guaranteed and its use is also discouraged:
597
598            TODO: {
599                local $TODO = 'We have work to do!';
600                $Test->todo_start('working on this');
601                # lots of code
602                $Test->todo_start('working on that');
603                # more code
604                $Test->todo_end;
605                $Test->todo_end;
606            }
607
608           Pick one style or another of "TODO" to be on the safe side.
609
610       "todo_end"
611            $Test->todo_end;
612
613           Stops running tests as "TODO" tests.  This method is fatal if
614           called without a preceding "todo_start" method call.
615
616       caller
617               my $package = $Test->caller;
618               my($pack, $file, $line) = $Test->caller;
619               my($pack, $file, $line) = $Test->caller($height);
620
621           Like the normal "caller()", except it reports according to your
622           "level()".
623
624           $height will be added to the "level()".
625
626           If "caller()" winds up off the top of the stack it report the
627           highest context.
628

EXIT CODES

630       If all your tests passed, Test::Builder will exit with zero (which is
631       normal).  If anything failed it will exit with how many failed.  If you
632       run less (or more) tests than you planned, the missing (or extras) will
633       be considered failures.  If no tests were ever run Test::Builder will
634       throw a warning and exit with 255.  If the test died, even after having
635       successfully completed all its tests, it will still be considered a
636       failure and will exit with 255.
637
638       So the exit codes are...
639
640           0                   all tests successful
641           255                 test died or all passed but wrong # of tests run
642           any other number    how many failed (including missing or extras)
643
644       If you fail more than 254 tests, it will be reported as 254.
645

THREADS

647       In perl 5.8.1 and later, Test::Builder is thread-safe.  The test number
648       is shared by all threads.  This means if one thread sets the test
649       number using "current_test()" they will all be effected.
650
651       While versions earlier than 5.8.1 had threads they contain too many
652       bugs to support.
653
654       Test::Builder is only thread-aware if threads.pm is loaded before
655       Test::Builder.
656
657       You can directly disable thread support with one of the following:
658
659           $ENV{T2_NO_IPC} = 1
660
661       or
662
663           no Test2::IPC;
664
665       or
666
667           Test2::API::test2_ipc_disable()
668

MEMORY

670       An informative hash, accessible via "details()", is stored for each
671       test you perform.  So memory usage will scale linearly with each test
672       run. Although this is not a problem for most test suites, it can become
673       an issue if you do large (hundred thousands to million) combinatorics
674       tests in the same run.
675
676       In such cases, you are advised to either split the test file into
677       smaller ones, or use a reverse approach, doing "normal" (code) compares
678       and triggering "fail()" should anything go unexpected.
679
680       Future versions of Test::Builder will have a way to turn history off.
681

EXAMPLES

683       CPAN can provide the best examples.  Test::Simple, Test::More,
684       Test::Exception and Test::Differences all use Test::Builder.
685

SEE ALSO

687   INTERNALS
688       Test2, Test2::API
689
690   LEGACY
691       Test::Simple, Test::More
692
693   EXTERNAL
694       Test::Harness
695

AUTHORS

697       Original code by chromatic, maintained by Michael G Schwern
698       <schwern@pobox.com>
699

MAINTAINERS

701       Chad Granum <exodist@cpan.org>
702
704       Copyright 2002-2008 by chromatic <chromatic@wgz.org> and
705                              Michael G Schwern <schwern@pobox.com>.
706
707       This program is free software; you can redistribute it and/or modify it
708       under the same terms as Perl itself.
709
710       See http://www.perl.com/perl/misc/Artistic.html
711
712
713
714perl v5.30.0                      2019-09-06                  Test::Builder(3)
Impressum