1Test2::API(3) User Contributed Perl Documentation Test2::API(3)
2
6 Test2::API - Primary interface for writing Test2 based testing tools.
7
9 The internals of this package are subject to change at any time! The
10 public methods provided will not change in backwards-incompatible ways
11 (once there is a stable release), but the underlying implementation
12 details might. Do not break encapsulation here!
13 Currently the implementation is to create a single instance of the
15 Test2::API::Instance Object. All class methods defer to the single
16 instance. There is no public access to the singleton, and that is
17 intentional. The class methods provided by this package provide the
18 only functionality publicly exposed.
19 This is done primarily to avoid the problems Test::Builder had by
21 exposing its singleton. We do not want anyone to replace this
22 singleton, rebless it, or directly muck with its internals. If you need
23 to do something and cannot because of the restrictions placed here,
24 then please report it as an issue. If possible, we will create a way
25 for you to implement your functionality without exposing things that
26 should not be exposed.
27
29 This package exports all the functions necessary to write and/or verify
30 testing tools. Using these building blocks you can begin writing test
31 tools very quickly. You are also provided with tools that help you to
32 test the tools you write.
33
35 WRITING A TOOL
36 The "context()" method is your primary interface into the Test2
37 framework.
38 package My::Ok;
40 use Test2::API qw/context/;
41 our @EXPORT = qw/my_ok/;
43 use base 'Exporter';
44 # Just like ok() from Test::More
46 sub my_ok($;$) {
47 my ($bool, $name) = @_;
48 my $ctx = context(); # Get a context
49 $ctx->ok($bool, $name);
50 $ctx->release; # Release the context
51 return $bool;
52 }
53 See Test2::API::Context for a list of methods available on the context
55 object.
56 TESTING YOUR TOOLS
58 The "intercept { ... }" tool lets you temporarily intercept all events
59 generated by the test system:
60 use Test2::API qw/intercept/;
62 use My::Ok qw/my_ok/;
64 my $events = intercept {
66 # These events are not displayed
67 my_ok(1, "pass");
68 my_ok(0, "fail");
69 };
70 my_ok(@$events == 2, "got 2 events, the pass and the fail");
72 my_ok($events->[0]->pass, "first event passed");
73 my_ok(!$events->[1]->pass, "second event failed");
74 DEEP EVENT INTERCEPTION
76 Normally "intercept { ... }" only intercepts events sent to the main
78 hub (as added by intercept itself). Nested hubs, such as those created
79 by subtests, will not be intercepted. This is normally what you will
80 still see the nested events by inspecting the subtest event. However
81 there are times where you want to verify each event as it is sent, in
82 that case use "intercept_deep { ... }".
83 my $events = intercept_Deep {
85 buffered_subtest foo => sub {
86 ok(1, "pass");
87 };
88 };
89 $events in this case will contain 3 items:
91 The event from "ok(1, "pass")"
93 The plan event for the subtest
94 The subtest event itself, with the first 2 events nested inside it as
95 children.
96 This lets you see the order in which the events were sent, unlike
98 "intercept { ... }" which only lets you see events as the main hub sees
99 them.
100 OTHER API FUNCTIONS
102 use Test2::API qw{
103 test2_init_done
104 test2_stack
105 test2_set_is_end
106 test2_get_is_end
107 test2_ipc
108 test2_formatter_set
109 test2_formatter
110 };
111 my $init = test2_init_done();
113 my $stack = test2_stack();
114 my $ipc = test2_ipc();
115 test2_formatter_set($FORMATTER)
117 my $formatter = test2_formatter();
118 ... And others ...
120
122 All exports are optional. You must specify subs to import.
123 use Test2::API qw/context intercept run_subtest/;
125 This is the list of exports that are most commonly needed. If you are
127 simply writing a tool, then this is probably all you need. If you need
128 something and you cannot find it here, then you can also look at "OTHER
129 API EXPORTS".
130 These exports lack the 'test2_' prefix because of how important/common
132 they are. Exports in the "OTHER API EXPORTS" section have the 'test2_'
133 prefix to ensure they stand out.
134 context(...)
136 Usage:
137 $ctx = context()
139 $ctx = context(%params)
140 The "context()" function will always return the current context. If
142 there is already a context active, it will be returned. If there is not
143 an active context, one will be generated. When a context is generated
144 it will default to using the file and line number where the currently
145 running sub was called from.
146 Please see "CRITICAL DETAILS" in Test2::API::Context for important
148 rules about what you can and cannot do with a context once it is
149 obtained.
150 Note This function will throw an exception if you ignore the context
152 object it returns.
153 Note On perls 5.14+ a depth check is used to insure there are no
155 context leaks. This cannot be safely done on older perls due to
156 <https://rt.perl.org/Public/Bug/Display.html?id=127774> You can
157 forcefully enable it either by setting "$ENV{T2_CHECK_DEPTH} = 1" or
158 "$Test2::API::DO_DEPTH_CHECK = 1" BEFORE loading Test2::API.
159 OPTIONAL PARAMETERS
161 All parameters to "context" are optional.
163 level => $int
165 If you must obtain a context in a sub deeper than your entry point
166 you can use this to tell it how many EXTRA stack frames to look
167 back. If this option is not provided the default of 0 is used.
168 sub third_party_tool {
170 my $sub = shift;
171 ... # Does not obtain a context
172 $sub->();
173 ...
174 }
175 third_party_tool(sub {
177 my $ctx = context(level => 1);
178 ...
179 $ctx->release;
180 });
181 wrapped => $int
183 Use this if you need to write your own tool that wraps a call to
184 "context()" with the intent that it should return a context object.
185 sub my_context {
187 my %params = ( wrapped => 0, @_ );
188 $params{wrapped}++;
189 my $ctx = context(%params);
190 ...
191 return $ctx;
192 }
193 sub my_tool {
195 my $ctx = my_context();
196 ...
197 $ctx->release;
198 }
199 If you do not do this, then tools you call that also check for a
201 context will notice that the context they grabbed was created at
202 the same stack depth, which will trigger protective measures that
203 warn you and destroy the existing context.
204 stack => $stack
206 Normally "context()" looks at the global hub stack. If you are
207 maintaining your own Test2::API::Stack instance you may pass it in
208 to be used instead of the global one.
209 hub => $hub
211 Use this parameter if you want to obtain the context for a specific
212 hub instead of whatever one happens to be at the top of the stack.
213 on_init => sub { ... }
215 This lets you provide a callback sub that will be called ONLY if
216 your call to "context()" generated a new context. The callback WILL
217 NOT be called if "context()" is returning an existing context. The
218 only argument passed into the callback will be the context object
219 itself.
220 sub foo {
222 my $ctx = context(on_init => sub { 'will run' });
223 my $inner = sub {
225 # This callback is not run since we are getting the existing
226 # context from our parent sub.
227 my $ctx = context(on_init => sub { 'will NOT run' });
228 $ctx->release;
229 }
230 $inner->();
231 $ctx->release;
233 }
234 on_release => sub { ... }
236 This lets you provide a callback sub that will be called when the
237 context instance is released. This callback will be added to the
238 returned context even if an existing context is returned. If
239 multiple calls to context add callbacks, then all will be called in
240 reverse order when the context is finally released.
241 sub foo {
243 my $ctx = context(on_release => sub { 'will run second' });
244 my $inner = sub {
246 my $ctx = context(on_release => sub { 'will run first' });
247 # Neither callback runs on this release
249 $ctx->release;
250 }
251 $inner->();
252 # Both callbacks run here.
254 $ctx->release;
255 }
256 release($;$)
258 Usage:
259 release $ctx;
261 release $ctx, ...;
262 This is intended as a shortcut that lets you release your context and
264 return a value in one statement. This function will get your context,
265 and an optional return value. It will release your context, then return
266 your value. Scalar context is always assumed.
267 sub tool {
269 my $ctx = context();
270 ...
271 return release $ctx, 1;
273 }
274 This tool is most useful when you want to return the value you get from
276 calling a function that needs to see the current context:
277 my $ctx = context();
279 my $out = some_tool(...);
280 $ctx->release;
281 return $out;
282 We can combine the last 3 lines of the above like so:
284 my $ctx = context();
286 release $ctx, some_tool(...);
287 context_do(&;@)
289 Usage:
290 sub my_tool {
292 context_do {
293 my $ctx = shift;
294 my (@args) = @_;
296 $ctx->ok(1, "pass");
298 ...
300 # No need to call $ctx->release, done for you on scope exit.
302 } @_;
303 }
304 Using this inside your test tool takes care of a lot of boilerplate for
306 you. It will ensure a context is acquired. It will capture and rethrow
307 any exception. It will insure the context is released when you are
308 done. It preserves the subroutine call context (array, scalar, void).
309 This is the safest way to write a test tool. The only two downsides to
311 this are a slight performance decrease, and some extra indentation in
312 your source. If the indentation is a problem for you then you can take
313 a peek at the next section.
314 no_context(&;$)
316 Usage:
317 no_context { ... };
319 no_context { ... } $hid;
320 sub my_tool(&) {
321 my $code = shift;
322 my $ctx = context();
323 ...
324 no_context {
326 # Things in here will not see our current context, they get a new
327 # one.
328 $code->();
330 };
331 ...
333 $ctx->release;
334 };
335 This tool will hide a context for the provided block of code. This
337 means any tools run inside the block will get a completely new context
338 if they acquire one. The new context will be inherited by tools nested
339 below the one that acquired it.
340 This will normally hide the current context for the top hub. If you
342 need to hide the context for a different hub you can pass in the
343 optional $hid parameter.
344 intercept(&)
346 Usage:
347 my $events = intercept {
349 ok(1, "pass");
350 ok(0, "fail");
351 ...
352 };
353 This function takes a codeblock as its only argument, and it has a
355 prototype. It will execute the codeblock, intercepting any generated
356 events in the process. It will return an array reference with all the
357 generated event objects. All events should be subclasses of
358 Test2::Event.
359 This is a very low-level subtest tool. This is useful for writing tools
361 which produce subtests. This is not intended for people simply writing
362 tests.
363 run_subtest(...)
365 Usage:
366 run_subtest($NAME, \&CODE, $BUFFERED, @ARGS)
368 # or
370 run_subtest($NAME, \&CODE, \%PARAMS, @ARGS)
372 This will run the provided codeblock with the args in @args. This
374 codeblock will be run as a subtest. A subtest is an isolated test state
375 that is condensed into a single Test2::Event::Subtest event, which
376 contains all events generated inside the subtest.
377 ARGUMENTS:
379 $NAME
381 The name of the subtest.
382 \&CODE
384 The code to run inside the subtest.
385 $BUFFERED or \%PARAMS
387 If this is a simple scalar then it will be treated as a boolean for
388 the 'buffered' setting. If this is a hash reference then it will be
389 used as a parameters hash. The param hash will be used for hub
390 construction (with the specified keys removed).
391 Keys that are removed and used by run_subtest:
393 'buffered' => $bool
395 Toggle buffered status.
396 'inherit_trace' => $bool
398 Normally the subtest hub is pushed and the sub is allowed to
399 generate its own root context for the hub. When this setting is
400 turned on a root context will be created for the hub that
401 shares the same trace as the current context.
402 Set this to true if your tool is producing subtests without
404 user-specified subs.
405 'no_fork' => $bool
407 Defaults to off. Normally forking inside a subtest will
408 actually fork the subtest, resulting in 2 final subtest events.
409 This parameter will turn off that behavior, only the original
410 process/thread will return a final subtest event.
411 @ARGS
413 Any extra arguments you want passed into the subtest code.
414 BUFFERED VS UNBUFFERED (OR STREAMED)
416 Normally all events inside and outside a subtest are sent to the
418 formatter immediately by the hub. Sometimes it is desirable to hold off
419 sending events within a subtest until the subtest is complete. This
420 usually depends on the formatter being used.
421 Things not effected by this flag
423 In both cases events are generated and stored in an array. This
424 array is eventually used to populate the "subevents" attribute on
425 the Test2::Event::Subtest event that is generated at the end of the
426 subtest. This flag has no effect on this part, it always happens.
427 At the end of the subtest, the final Test2::Event::Subtest event is
429 sent to the formatter.
430 Things that are effected by this flag
432 The "buffered" attribute of the Test2::Event::Subtest event will be
433 set to the value of this flag. This means any formatter, listener,
434 etc which looks at the event will know if it was buffered.
435 Things that are formatter dependant
437 Events within a buffered subtest may or may not be sent to the
438 formatter as they happen. If a formatter fails to specify then the
439 default is to NOT SEND the events as they are generated, instead
440 the formatter can pull them from the "subevents" attribute.
441 A formatter can specify by implementing the "hide_buffered()"
443 method. If this method returns true then events generated inside a
444 buffered subtest will not be sent independently of the final
445 subtest event.
446 An example of how this is used is the Test2::Formatter::TAP formatter.
448 For unbuffered subtests the events are rendered as they are generated.
449 At the end of the subtest, the final subtest event is rendered, but the
450 "subevents" attribute is ignored. For buffered subtests the opposite
451 occurs, the events are NOT rendered as they are generated, instead the
452 "subevents" attribute is used to render them all at once. This is
453 useful when running subtests tests in parallel, since without it the
454 output from subtests would be interleaved together.
455
457 Exports in this section are not commonly needed. These all have the
458 'test2_' prefix to help ensure they stand out. You should look at the
459 "MAIN API EXPORTS" section before looking here. This section is one
460 where "Great power comes with great responsibility". It is possible to
461 break things badly if you are not careful with these.
462 All exports are optional. You need to list which ones you want at
464 import time:
465 use Test2::API qw/test2_init_done .../;
467 STATUS AND INITIALIZATION STATE
469 These provide access to internal state and object instances.
470 $bool = test2_init_done()
472 This will return true if the stack and IPC instances have already
473 been initialized. It will return false if they have not. Init
474 happens as late as possible. It happens as soon as a tool requests
475 the IPC instance, the formatter, or the stack.
476 $bool = test2_load_done()
478 This will simply return the boolean value of the loaded flag. If
479 Test2 has finished loading this will be true, otherwise false.
480 Loading is considered complete the first time a tool requests a
481 context.
482 test2_set_is_end()
484 test2_set_is_end($bool)
485 This is used to toggle Test2's belief that the END phase has
486 already started. With no arguments this will set it to true. With
487 arguments it will set it to the first argument's value.
488 This is used to prevent the use of "caller()" in END blocks which
490 can cause segfaults. This is only necessary in some persistent
491 environments that may have multiple END phases.
492 $bool = test2_get_is_end()
494 Check if Test2 believes it is the END phase.
495 $stack = test2_stack()
497 This will return the global Test2::API::Stack instance. If this has
498 not yet been initialized it will be initialized now.
499 test2_ipc_disable
501 Disable IPC.
502 $bool = test2_ipc_diabled
504 Check if IPC is disabled.
505 test2_ipc_wait_enable()
507 test2_ipc_wait_disable()
508 $bool = test2_ipc_wait_enabled()
509 These can be used to turn IPC waiting on and off, or check the
510 current value of the flag.
511 Waiting is turned on by default. Waiting will cause the parent
513 process/thread to wait until all child processes and threads are
514 finished before exiting. You will almost never want to turn this
515 off.
516 $bool = test2_no_wait()
518 test2_no_wait($bool)
519 DISCOURAGED: This is a confusing interface, it is better to use
520 "test2_ipc_wait_enable()", "test2_ipc_wait_disable()" and
521 "test2_ipc_wait_enabled()".
522 This can be used to get/set the no_wait status. Waiting is turned
524 on by default. Waiting will cause the parent process/thread to wait
525 until all child processes and threads are finished before exiting.
526 You will almost never want to turn this off.
527 $fh = test2_stdout()
529 $fh = test2_stderr()
530 These functions return the filehandles that test output should be
531 written to. They are primarily useful when writing a custom
532 formatter and code that turns events into actual output (TAP, etc.)
533 They will return a dupe of the original filehandles that formatted
534 output can be sent to regardless of whatever state the currently
535 running test may have left STDOUT and STDERR in.
536 test2_reset_io()
538 Re-dupe the internal filehandles returned by "test2_stdout()" and
539 "test2_stderr()" from the current STDOUT and STDERR. You shouldn't
540 need to do this except in very peculiar situations (for example,
541 you're testing a new formatter and you need control over where the
542 formatter is sending its output.)
543 BEHAVIOR HOOKS
545 These are hooks that allow you to add custom behavior to actions taken
546 by Test2 and tools built on top of it.
547 test2_add_callback_exit(sub { ... })
549 This can be used to add a callback that is called after all testing
550 is done. This is too late to add additional results, the main use
551 of this callback is to set the exit code.
552 test2_add_callback_exit(
554 sub {
555 my ($context, $exit, \$new_exit) = @_;
556 ...
557 }
558 );
559 The $context passed in will be an instance of Test2::API::Context.
561 The $exit argument will be the original exit code before anything
562 modified it. $$new_exit is a reference to the new exit code. You
563 may modify this to change the exit code. Please note that
564 $$new_exit may already be different from $exit
565 test2_add_callback_post_load(sub { ... })
567 Add a callback that will be called when Test2 is finished loading.
568 This means the callback will be run once, the first time a context
569 is obtained. If Test2 has already finished loading then the
570 callback will be run immediately.
571 test2_add_callback_testing_done(sub { ... })
573 This adds your coderef as a follow-up to the root hub after Test2
574 is finished loading.
575 This is essentially a helper to do the following:
577 test2_add_callback_post_load(sub {
579 my $stack = test2_stack();
580 $stack->top; # Insure we have a hub
581 my ($hub) = Test2::API::test2_stack->all;
582 $hub->set_active(1);
584 $hub->follow_up(sub { ... }); # <-- Your coderef here
586 });
587 test2_add_callback_context_acquire(sub { ... })
589 Add a callback that will be called every time someone tries to
590 acquire a context. This will be called on EVERY call to
591 "context()". It gets a single argument, a reference to the hash of
592 parameters being used the construct the context. This is your
593 chance to change the parameters by directly altering the hash.
594 test2_add_callback_context_acquire(sub {
596 my $params = shift;
597 $params->{level}++;
598 });
599 This is a very scary API function. Please do not use this unless
601 you need to. This is here for Test::Builder and backwards
602 compatibility. This has you directly manipulate the hash instead of
603 returning a new one for performance reasons.
604 test2_add_callback_context_init(sub { ... })
606 Add a callback that will be called every time a new context is
607 created. The callback will receive the newly created context as its
608 only argument.
609 test2_add_callback_context_release(sub { ... })
611 Add a callback that will be called every time a context is
612 released. The callback will receive the released context as its
613 only argument.
614 test2_add_callback_pre_subtest(sub { ... })
616 Add a callback that will be called every time a subtest is going to
617 be run. The callback will receive the subtest name, coderef, and
618 any arguments.
619 @list = test2_list_context_acquire_callbacks()
621 Return all the context acquire callback references.
622 @list = test2_list_context_init_callbacks()
624 Returns all the context init callback references.
625 @list = test2_list_context_release_callbacks()
627 Returns all the context release callback references.
628 @list = test2_list_exit_callbacks()
630 Returns all the exit callback references.
631 @list = test2_list_post_load_callbacks()
633 Returns all the post load callback references.
634 @list = test2_list_pre_subtest_callbacks()
636 Returns all the pre-subtest callback references.
637 test2_add_uuid_via(sub { ... })
639 $sub = test2_add_uuid_via()
640 This allows you to provide a UUID generator. If provided UUIDs will
641 be attached to all events, hubs, and contexts. This is useful for
642 storing, tracking, and linking these objects.
643 The sub you provide should always return a unique identifier. Most
645 things will expect a proper UUID string, however nothing in
646 Test2::API enforces this.
647 The sub will receive exactly 1 argument, the type of thing being
649 tagged 'context', 'hub', or 'event'. In the future additional
650 things may be tagged, in which case new strings will be passed in.
651 These are purely informative, you can (and usually should) ignore
652 them.
653 IPC AND CONCURRENCY
655 These let you access, or specify, the IPC system internals.
656 $bool = test2_has_ipc()
658 Check if IPC is enabled.
659 $ipc = test2_ipc()
661 This will return the global Test2::IPC::Driver instance. If this
662 has not yet been initialized it will be initialized now.
663 test2_ipc_add_driver($DRIVER)
665 Add an IPC driver to the list. This will add the driver to the
666 start of the list.
667 @drivers = test2_ipc_drivers()
669 Get the list of IPC drivers.
670 $bool = test2_ipc_polling()
672 Check if polling is enabled.
673 test2_ipc_enable_polling()
675 Turn on polling. This will cull events from other processes and
676 threads every time a context is created.
677 test2_ipc_disable_polling()
679 Turn off IPC polling.
680 test2_ipc_enable_shm()
682 Turn on IPC SHM. Only some IPC drivers use this, and most will turn
683 it on themselves.
684 test2_ipc_set_pending($uniq_val)
686 Tell other processes and events that an event is pending. $uniq_val
687 should be a unique value no other thread/process will generate.
688 Note: After calling this "test2_ipc_get_pending()" will return 1.
690 This is intentional, and not avoidable.
691 $pending = test2_ipc_get_pending()
693 This returns -1 if there is no way to check (assume yes)
694 This returns 0 if there are (most likely) no pending events.
696 This returns 1 if there are (likely) pending events. Upon return it
698 will reset, nothing else will be able to see that there were
699 pending events.
700 $timeout = test2_ipc_get_timeout()
702 test2_ipc_set_timeout($timeout)
703 Get/Set the timeout value for the IPC system. This timeout is how
704 long the IPC system will wait for child processes and threads to
705 finish before aborting.
706 The default value is 30 seconds.
708 MANAGING FORMATTERS
710 These let you access, or specify, the formatters that can/should be
711 used.
712 $formatter = test2_formatter
714 This will return the global formatter class. This is not an
715 instance. By default the formatter is set to Test2::Formatter::TAP.
716 You can override this default using the "T2_FORMATTER" environment
718 variable.
719 Normally 'Test2::Formatter::' is prefixed to the value in the
721 environment variable:
722 $ T2_FORMATTER='TAP' perl test.t # Use the Test2::Formatter::TAP formatter
724 $ T2_FORMATTER='Foo' perl test.t # Use the Test2::Formatter::Foo formatter
725 If you want to specify a full module name you use the '+' prefix:
727 $ T2_FORMATTER='+Foo::Bar' perl test.t # Use the Foo::Bar formatter
729 test2_formatter_set($class_or_instance)
731 Set the global formatter class. This can only be set once. Note:
732 This will override anything specified in the 'T2_FORMATTER'
733 environment variable.
734 @formatters = test2_formatters()
736 Get a list of all loaded formatters.
737 test2_formatter_add($class_or_instance)
739 Add a formatter to the list. Last formatter added is used at
740 initialization. If this is called after initialization a warning
741 will be issued.
742
744 See the "/Examples/" directory included in this distribution.
745
747 Test2::API::Context - Detailed documentation of the context object.
748 Test2::IPC - The IPC system used for threading/fork support.
750 Test2::Formatter - Formatters such as TAP live here.
752 Test2::Event - Events live in this namespace.
754 Test2::Hub - All events eventually funnel through a hub. Custom hubs
756 are how "intercept()" and "run_subtest()" are implemented.
757
759 This package has an END block. This END block is responsible for
760 setting the exit code based on the test results. This end block also
761 calls the callbacks that can be added to this package.
762
764 The source code repository for Test2 can be found at
765 http://github.com/Test-More/test-more/.
766
768 Chad Granum <exodist@cpan.org>
769
771 Chad Granum <exodist@cpan.org>
772
774 Copyright 2018 Chad Granum <exodist@cpan.org>.
775 This program is free software; you can redistribute it and/or modify it
777 under the same terms as Perl itself.
778 See http://dev.perl.org/licenses/
780perl v5.28.0 2018-08-13 Test2::API(3)