1Test2::API(3) User Contributed Perl Documentation Test2::API(3)
2
6 Test2::API - Primary interface for writing Test2 based testing tools.
7
9 The internals of this package are subject to change at any time! The
10 public methods provided will not change in backwards-incompatible ways
11 (once there is a stable release), but the underlying implementation
12 details might. Do not break encapsulation here!
13 Currently the implementation is to create a single instance of the
15 Test2::API::Instance Object. All class methods defer to the single
16 instance. There is no public access to the singleton, and that is
17 intentional. The class methods provided by this package provide the
18 only functionality publicly exposed.
19 This is done primarily to avoid the problems Test::Builder had by
21 exposing its singleton. We do not want anyone to replace this
22 singleton, rebless it, or directly muck with its internals. If you need
23 to do something and cannot because of the restrictions placed here,
24 then please report it as an issue. If possible, we will create a way
25 for you to implement your functionality without exposing things that
26 should not be exposed.
27
29 This package exports all the functions necessary to write and/or verify
30 testing tools. Using these building blocks you can begin writing test
31 tools very quickly. You are also provided with tools that help you to
32 test the tools you write.
33
35 WRITING A TOOL
36 The context() method is your primary interface into the Test2
37 framework.
38 package My::Ok;
40 use Test2::API qw/context/;
41 our @EXPORT = qw/my_ok/;
43 use base 'Exporter';
44 # Just like ok() from Test::More
46 sub my_ok($;$) {
47 my ($bool, $name) = @_;
48 my $ctx = context(); # Get a context
49 $ctx->ok($bool, $name);
50 $ctx->release; # Release the context
51 return $bool;
52 }
53 See Test2::API::Context for a list of methods available on the context
55 object.
56 TESTING YOUR TOOLS
58 The "intercept { ... }" tool lets you temporarily intercept all events
59 generated by the test system:
60 use Test2::API qw/intercept/;
62 use My::Ok qw/my_ok/;
64 my $events = intercept {
66 # These events are not displayed
67 my_ok(1, "pass");
68 my_ok(0, "fail");
69 };
70 As of version 1.302178 this now returns an arrayref that is also an
72 instance of Test2::API::InterceptResult. See the
73 Test2::API::InterceptResult documentation for details on how to best
74 use it.
75 OTHER API FUNCTIONS
77 use Test2::API qw{
78 test2_init_done
79 test2_stack
80 test2_set_is_end
81 test2_get_is_end
82 test2_ipc
83 test2_formatter_set
84 test2_formatter
85 test2_is_testing_done
86 };
87 my $init = test2_init_done();
89 my $stack = test2_stack();
90 my $ipc = test2_ipc();
91 test2_formatter_set($FORMATTER)
93 my $formatter = test2_formatter();
94 ... And others ...
96
98 All exports are optional. You must specify subs to import.
99 use Test2::API qw/context intercept run_subtest/;
101 This is the list of exports that are most commonly needed. If you are
103 simply writing a tool, then this is probably all you need. If you need
104 something and you cannot find it here, then you can also look at "OTHER
105 API EXPORTS".
106 These exports lack the 'test2_' prefix because of how important/common
108 they are. Exports in the "OTHER API EXPORTS" section have the 'test2_'
109 prefix to ensure they stand out.
110 context(...)
112 Usage:
113 $ctx = context()
115 $ctx = context(%params)
116 The context() function will always return the current context. If there
118 is already a context active, it will be returned. If there is not an
119 active context, one will be generated. When a context is generated it
120 will default to using the file and line number where the currently
121 running sub was called from.
122 Please see "CRITICAL DETAILS" in Test2::API::Context for important
124 rules about what you can and cannot do with a context once it is
125 obtained.
126 Note This function will throw an exception if you ignore the context
128 object it returns.
129 Note On perls 5.14+ a depth check is used to insure there are no
131 context leaks. This cannot be safely done on older perls due to
132 <https://rt.perl.org/Public/Bug/Display.html?id=127774> You can
133 forcefully enable it either by setting "$ENV{T2_CHECK_DEPTH} = 1" or
134 "$Test2::API::DO_DEPTH_CHECK = 1" BEFORE loading Test2::API.
135 OPTIONAL PARAMETERS
137 All parameters to "context" are optional.
139 level => $int
141 If you must obtain a context in a sub deeper than your entry point
142 you can use this to tell it how many EXTRA stack frames to look
143 back. If this option is not provided the default of 0 is used.
144 sub third_party_tool {
146 my $sub = shift;
147 ... # Does not obtain a context
148 $sub->();
149 ...
150 }
151 third_party_tool(sub {
153 my $ctx = context(level => 1);
154 ...
155 $ctx->release;
156 });
157 wrapped => $int
159 Use this if you need to write your own tool that wraps a call to
160 context() with the intent that it should return a context object.
161 sub my_context {
163 my %params = ( wrapped => 0, @_ );
164 $params{wrapped}++;
165 my $ctx = context(%params);
166 ...
167 return $ctx;
168 }
169 sub my_tool {
171 my $ctx = my_context();
172 ...
173 $ctx->release;
174 }
175 If you do not do this, then tools you call that also check for a
177 context will notice that the context they grabbed was created at
178 the same stack depth, which will trigger protective measures that
179 warn you and destroy the existing context.
180 stack => $stack
182 Normally context() looks at the global hub stack. If you are
183 maintaining your own Test2::API::Stack instance you may pass it in
184 to be used instead of the global one.
185 hub => $hub
187 Use this parameter if you want to obtain the context for a specific
188 hub instead of whatever one happens to be at the top of the stack.
189 on_init => sub { ... }
191 This lets you provide a callback sub that will be called ONLY if
192 your call to context() generated a new context. The callback WILL
193 NOT be called if context() is returning an existing context. The
194 only argument passed into the callback will be the context object
195 itself.
196 sub foo {
198 my $ctx = context(on_init => sub { 'will run' });
199 my $inner = sub {
201 # This callback is not run since we are getting the existing
202 # context from our parent sub.
203 my $ctx = context(on_init => sub { 'will NOT run' });
204 $ctx->release;
205 }
206 $inner->();
207 $ctx->release;
209 }
210 on_release => sub { ... }
212 This lets you provide a callback sub that will be called when the
213 context instance is released. This callback will be added to the
214 returned context even if an existing context is returned. If
215 multiple calls to context add callbacks, then all will be called in
216 reverse order when the context is finally released.
217 sub foo {
219 my $ctx = context(on_release => sub { 'will run second' });
220 my $inner = sub {
222 my $ctx = context(on_release => sub { 'will run first' });
223 # Neither callback runs on this release
225 $ctx->release;
226 }
227 $inner->();
228 # Both callbacks run here.
230 $ctx->release;
231 }
232 release($;$)
234 Usage:
235 release $ctx;
237 release $ctx, ...;
238 This is intended as a shortcut that lets you release your context and
240 return a value in one statement. This function will get your context,
241 and an optional return value. It will release your context, then return
242 your value. Scalar context is always assumed.
243 sub tool {
245 my $ctx = context();
246 ...
247 return release $ctx, 1;
249 }
250 This tool is most useful when you want to return the value you get from
252 calling a function that needs to see the current context:
253 my $ctx = context();
255 my $out = some_tool(...);
256 $ctx->release;
257 return $out;
258 We can combine the last 3 lines of the above like so:
260 my $ctx = context();
262 release $ctx, some_tool(...);
263 context_do(&;@)
265 Usage:
266 sub my_tool {
268 context_do {
269 my $ctx = shift;
270 my (@args) = @_;
272 $ctx->ok(1, "pass");
274 ...
276 # No need to call $ctx->release, done for you on scope exit.
278 } @_;
279 }
280 Using this inside your test tool takes care of a lot of boilerplate for
282 you. It will ensure a context is acquired. It will capture and rethrow
283 any exception. It will insure the context is released when you are
284 done. It preserves the subroutine call context (array, scalar, void).
285 This is the safest way to write a test tool. The only two downsides to
287 this are a slight performance decrease, and some extra indentation in
288 your source. If the indentation is a problem for you then you can take
289 a peek at the next section.
290 no_context(&;$)
292 Usage:
293 no_context { ... };
295 no_context { ... } $hid;
296 sub my_tool(&) {
297 my $code = shift;
298 my $ctx = context();
299 ...
300 no_context {
302 # Things in here will not see our current context, they get a new
303 # one.
304 $code->();
306 };
307 ...
309 $ctx->release;
310 };
311 This tool will hide a context for the provided block of code. This
313 means any tools run inside the block will get a completely new context
314 if they acquire one. The new context will be inherited by tools nested
315 below the one that acquired it.
316 This will normally hide the current context for the top hub. If you
318 need to hide the context for a different hub you can pass in the
319 optional $hid parameter.
320 intercept(&)
322 Usage:
323 my $events = intercept {
325 ok(1, "pass");
326 ok(0, "fail");
327 ...
328 };
329 This function takes a codeblock as its only argument, and it has a
331 prototype. It will execute the codeblock, intercepting any generated
332 events in the process. It will return an array reference with all the
333 generated event objects. All events should be subclasses of
334 Test2::Event.
335 As of version 1.302178 the events array that is returned is blssed as
337 an Test2::API::InterceptResult instance. Test2::API::InterceptResult
338 Provides a helpful interface for filtering and/or inspecting the events
339 list overall, or individual events within the list.
340 This is intended to help you test your test code. This is not intended
342 for people simply writing tests.
343 run_subtest(...)
345 Usage:
346 run_subtest($NAME, \&CODE, $BUFFERED, @ARGS)
348 # or
350 run_subtest($NAME, \&CODE, \%PARAMS, @ARGS)
352 This will run the provided codeblock with the args in @args. This
354 codeblock will be run as a subtest. A subtest is an isolated test state
355 that is condensed into a single Test2::Event::Subtest event, which
356 contains all events generated inside the subtest.
357 ARGUMENTS:
359 $NAME
361 The name of the subtest.
362 \&CODE
364 The code to run inside the subtest.
365 $BUFFERED or \%PARAMS
367 If this is a simple scalar then it will be treated as a boolean for
368 the 'buffered' setting. If this is a hash reference then it will be
369 used as a parameters hash. The param hash will be used for hub
370 construction (with the specified keys removed).
371 Keys that are removed and used by run_subtest:
373 'buffered' => $bool
375 Toggle buffered status.
376 'inherit_trace' => $bool
378 Normally the subtest hub is pushed and the sub is allowed to
379 generate its own root context for the hub. When this setting is
380 turned on a root context will be created for the hub that
381 shares the same trace as the current context.
382 Set this to true if your tool is producing subtests without
384 user-specified subs.
385 'no_fork' => $bool
387 Defaults to off. Normally forking inside a subtest will
388 actually fork the subtest, resulting in 2 final subtest events.
389 This parameter will turn off that behavior, only the original
390 process/thread will return a final subtest event.
391 @ARGS
393 Any extra arguments you want passed into the subtest code.
394 BUFFERED VS UNBUFFERED (OR STREAMED)
396 Normally all events inside and outside a subtest are sent to the
398 formatter immediately by the hub. Sometimes it is desirable to hold off
399 sending events within a subtest until the subtest is complete. This
400 usually depends on the formatter being used.
401 Things not effected by this flag
403 In both cases events are generated and stored in an array. This
404 array is eventually used to populate the "subevents" attribute on
405 the Test2::Event::Subtest event that is generated at the end of the
406 subtest. This flag has no effect on this part, it always happens.
407 At the end of the subtest, the final Test2::Event::Subtest event is
409 sent to the formatter.
410 Things that are effected by this flag
412 The "buffered" attribute of the Test2::Event::Subtest event will be
413 set to the value of this flag. This means any formatter, listener,
414 etc which looks at the event will know if it was buffered.
415 Things that are formatter dependant
417 Events within a buffered subtest may or may not be sent to the
418 formatter as they happen. If a formatter fails to specify then the
419 default is to NOT SEND the events as they are generated, instead
420 the formatter can pull them from the "subevents" attribute.
421 A formatter can specify by implementing the hide_buffered() method.
423 If this method returns true then events generated inside a buffered
424 subtest will not be sent independently of the final subtest event.
425 An example of how this is used is the Test2::Formatter::TAP formatter.
427 For unbuffered subtests the events are rendered as they are generated.
428 At the end of the subtest, the final subtest event is rendered, but the
429 "subevents" attribute is ignored. For buffered subtests the opposite
430 occurs, the events are NOT rendered as they are generated, instead the
431 "subevents" attribute is used to render them all at once. This is
432 useful when running subtests tests in parallel, since without it the
433 output from subtests would be interleaved together.
434
436 Exports in this section are not commonly needed. These all have the
437 'test2_' prefix to help ensure they stand out. You should look at the
438 "MAIN API EXPORTS" section before looking here. This section is one
439 where "Great power comes with great responsibility". It is possible to
440 break things badly if you are not careful with these.
441 All exports are optional. You need to list which ones you want at
443 import time:
444 use Test2::API qw/test2_init_done .../;
446 STATUS AND INITIALIZATION STATE
448 These provide access to internal state and object instances.
449 $bool = test2_init_done()
451 This will return true if the stack and IPC instances have already
452 been initialized. It will return false if they have not. Init
453 happens as late as possible. It happens as soon as a tool requests
454 the IPC instance, the formatter, or the stack.
455 $bool = test2_load_done()
457 This will simply return the boolean value of the loaded flag. If
458 Test2 has finished loading this will be true, otherwise false.
459 Loading is considered complete the first time a tool requests a
460 context.
461 test2_set_is_end()
463 test2_set_is_end($bool)
464 This is used to toggle Test2's belief that the END phase has
465 already started. With no arguments this will set it to true. With
466 arguments it will set it to the first argument's value.
467 This is used to prevent the use of caller() in END blocks which can
469 cause segfaults. This is only necessary in some persistent
470 environments that may have multiple END phases.
471 $bool = test2_get_is_end()
473 Check if Test2 believes it is the END phase.
474 $stack = test2_stack()
476 This will return the global Test2::API::Stack instance. If this has
477 not yet been initialized it will be initialized now.
478 $bool = test2_is_testing_done()
480 This will return true if testing is complete and no other events
481 should be sent. This is useful in things like warning handlers
482 where you might want to turn warnings into events, but need them to
483 start acting like normal warnings when testing is done.
484 $SIG{__WARN__} = sub {
486 my ($warning) = @_;
487 if (test2_is_testing_done()) {
489 warn @_;
490 }
491 else {
492 my $ctx = context();
493 ...
494 $ctx->release
495 }
496 }
497 test2_ipc_disable
499 Disable IPC.
500 $bool = test2_ipc_diabled
502 Check if IPC is disabled.
503 test2_ipc_wait_enable()
505 test2_ipc_wait_disable()
506 $bool = test2_ipc_wait_enabled()
507 These can be used to turn IPC waiting on and off, or check the
508 current value of the flag.
509 Waiting is turned on by default. Waiting will cause the parent
511 process/thread to wait until all child processes and threads are
512 finished before exiting. You will almost never want to turn this
513 off.
514 $bool = test2_no_wait()
516 test2_no_wait($bool)
517 DISCOURAGED: This is a confusing interface, it is better to use
518 test2_ipc_wait_enable(), test2_ipc_wait_disable() and
519 test2_ipc_wait_enabled().
520 This can be used to get/set the no_wait status. Waiting is turned
522 on by default. Waiting will cause the parent process/thread to wait
523 until all child processes and threads are finished before exiting.
524 You will almost never want to turn this off.
525 $fh = test2_stdout()
527 $fh = test2_stderr()
528 These functions return the filehandles that test output should be
529 written to. They are primarily useful when writing a custom
530 formatter and code that turns events into actual output (TAP,
531 etc.). They will return a dupe of the original filehandles that
532 formatted output can be sent to regardless of whatever state the
533 currently running test may have left STDOUT and STDERR in.
534 test2_reset_io()
536 Re-dupe the internal filehandles returned by test2_stdout() and
537 test2_stderr() from the current STDOUT and STDERR. You shouldn't
538 need to do this except in very peculiar situations (for example,
539 you're testing a new formatter and you need control over where the
540 formatter is sending its output.)
541 BEHAVIOR HOOKS
543 These are hooks that allow you to add custom behavior to actions taken
544 by Test2 and tools built on top of it.
545 test2_add_callback_exit(sub { ... })
547 This can be used to add a callback that is called after all testing
548 is done. This is too late to add additional results, the main use
549 of this callback is to set the exit code.
550 test2_add_callback_exit(
552 sub {
553 my ($context, $exit, \$new_exit) = @_;
554 ...
555 }
556 );
557 The $context passed in will be an instance of Test2::API::Context.
559 The $exit argument will be the original exit code before anything
560 modified it. $$new_exit is a reference to the new exit code. You
561 may modify this to change the exit code. Please note that
562 $$new_exit may already be different from $exit
563 test2_add_callback_post_load(sub { ... })
565 Add a callback that will be called when Test2 is finished loading.
566 This means the callback will be run once, the first time a context
567 is obtained. If Test2 has already finished loading then the
568 callback will be run immediately.
569 test2_add_callback_testing_done(sub { ... })
571 This adds your coderef as a follow-up to the root hub after Test2
572 is finished loading.
573 This is essentially a helper to do the following:
575 test2_add_callback_post_load(sub {
577 my $stack = test2_stack();
578 $stack->top; # Insure we have a hub
579 my ($hub) = Test2::API::test2_stack->all;
580 $hub->set_active(1);
582 $hub->follow_up(sub { ... }); # <-- Your coderef here
584 });
585 test2_add_callback_context_acquire(sub { ... })
587 Add a callback that will be called every time someone tries to
588 acquire a context. This will be called on EVERY call to context().
589 It gets a single argument, a reference to the hash of parameters
590 being used the construct the context. This is your chance to change
591 the parameters by directly altering the hash.
592 test2_add_callback_context_acquire(sub {
594 my $params = shift;
595 $params->{level}++;
596 });
597 This is a very scary API function. Please do not use this unless
599 you need to. This is here for Test::Builder and backwards
600 compatibility. This has you directly manipulate the hash instead of
601 returning a new one for performance reasons.
602 test2_add_callback_context_init(sub { ... })
604 Add a callback that will be called every time a new context is
605 created. The callback will receive the newly created context as its
606 only argument.
607 test2_add_callback_context_release(sub { ... })
609 Add a callback that will be called every time a context is
610 released. The callback will receive the released context as its
611 only argument.
612 test2_add_callback_pre_subtest(sub { ... })
614 Add a callback that will be called every time a subtest is going to
615 be run. The callback will receive the subtest name, coderef, and
616 any arguments.
617 @list = test2_list_context_acquire_callbacks()
619 Return all the context acquire callback references.
620 @list = test2_list_context_init_callbacks()
622 Returns all the context init callback references.
623 @list = test2_list_context_release_callbacks()
625 Returns all the context release callback references.
626 @list = test2_list_exit_callbacks()
628 Returns all the exit callback references.
629 @list = test2_list_post_load_callbacks()
631 Returns all the post load callback references.
632 @list = test2_list_pre_subtest_callbacks()
634 Returns all the pre-subtest callback references.
635 test2_add_uuid_via(sub { ... })
637 $sub = test2_add_uuid_via()
638 This allows you to provide a UUID generator. If provided UUIDs will
639 be attached to all events, hubs, and contexts. This is useful for
640 storing, tracking, and linking these objects.
641 The sub you provide should always return a unique identifier. Most
643 things will expect a proper UUID string, however nothing in
644 Test2::API enforces this.
645 The sub will receive exactly 1 argument, the type of thing being
647 tagged 'context', 'hub', or 'event'. In the future additional
648 things may be tagged, in which case new strings will be passed in.
649 These are purely informative, you can (and usually should) ignore
650 them.
651 IPC AND CONCURRENCY
653 These let you access, or specify, the IPC system internals.
654 $bool = test2_has_ipc()
656 Check if IPC is enabled.
657 $ipc = test2_ipc()
659 This will return the global Test2::IPC::Driver instance. If this
660 has not yet been initialized it will be initialized now.
661 test2_ipc_add_driver($DRIVER)
663 Add an IPC driver to the list. This will add the driver to the
664 start of the list.
665 @drivers = test2_ipc_drivers()
667 Get the list of IPC drivers.
668 $bool = test2_ipc_polling()
670 Check if polling is enabled.
671 test2_ipc_enable_polling()
673 Turn on polling. This will cull events from other processes and
674 threads every time a context is created.
675 test2_ipc_disable_polling()
677 Turn off IPC polling.
678 test2_ipc_enable_shm()
680 Legacy, this is currently a no-op that returns 0;
681 test2_ipc_set_pending($uniq_val)
683 Tell other processes and events that an event is pending. $uniq_val
684 should be a unique value no other thread/process will generate.
685 Note: After calling this test2_ipc_get_pending() will return 1.
687 This is intentional, and not avoidable.
688 $pending = test2_ipc_get_pending()
690 This returns -1 if there is no way to check (assume yes)
691 This returns 0 if there are (most likely) no pending events.
693 This returns 1 if there are (likely) pending events. Upon return it
695 will reset, nothing else will be able to see that there were
696 pending events.
697 $timeout = test2_ipc_get_timeout()
699 test2_ipc_set_timeout($timeout)
700 Get/Set the timeout value for the IPC system. This timeout is how
701 long the IPC system will wait for child processes and threads to
702 finish before aborting.
703 The default value is 30 seconds.
705 MANAGING FORMATTERS
707 These let you access, or specify, the formatters that can/should be
708 used.
709 $formatter = test2_formatter
711 This will return the global formatter class. This is not an
712 instance. By default the formatter is set to Test2::Formatter::TAP.
713 You can override this default using the "T2_FORMATTER" environment
715 variable.
716 Normally 'Test2::Formatter::' is prefixed to the value in the
718 environment variable:
719 $ T2_FORMATTER='TAP' perl test.t # Use the Test2::Formatter::TAP formatter
721 $ T2_FORMATTER='Foo' perl test.t # Use the Test2::Formatter::Foo formatter
722 If you want to specify a full module name you use the '+' prefix:
724 $ T2_FORMATTER='+Foo::Bar' perl test.t # Use the Foo::Bar formatter
726 test2_formatter_set($class_or_instance)
728 Set the global formatter class. This can only be set once. Note:
729 This will override anything specified in the 'T2_FORMATTER'
730 environment variable.
731 @formatters = test2_formatters()
733 Get a list of all loaded formatters.
734 test2_formatter_add($class_or_instance)
736 Add a formatter to the list. Last formatter added is used at
737 initialization. If this is called after initialization a warning
738 will be issued.
739
741 See the "/Examples/" directory included in this distribution.
742
744 Test2::API::Context - Detailed documentation of the context object.
745 Test2::IPC - The IPC system used for threading/fork support.
747 Test2::Formatter - Formatters such as TAP live here.
749 Test2::Event - Events live in this namespace.
751 Test2::Hub - All events eventually funnel through a hub. Custom hubs
753 are how intercept() and run_subtest() are implemented.
754
756 This package has an END block. This END block is responsible for
757 setting the exit code based on the test results. This end block also
758 calls the callbacks that can be added to this package.
759
761 The source code repository for Test2 can be found at
762 http://github.com/Test-More/test-more/.
763
765 Chad Granum <exodist@cpan.org>
766
768 Chad Granum <exodist@cpan.org>
769
771 Copyright 2020 Chad Granum <exodist@cpan.org>.
772 This program is free software; you can redistribute it and/or modify it
774 under the same terms as Perl itself.
775 See http://dev.perl.org/licenses/
777perl v5.36.0 2023-03-15 Test2::API(3)