1Test2::API(3)         User Contributed Perl Documentation        Test2::API(3)
2
3
4

NAME

6       Test2::API - Primary interface for writing Test2 based testing tools.
7

***INTERNALS NOTE***

9       The internals of this package are subject to change at any time! The
10       public methods provided will not change in backwards-incompatible ways
11       (once there is a stable release), but the underlying implementation
12       details might.  Do not break encapsulation here!
13
14       Currently the implementation is to create a single instance of the
15       Test2::API::Instance Object. All class methods defer to the single
16       instance. There is no public access to the singleton, and that is
17       intentional.  The class methods provided by this package provide the
18       only functionality publicly exposed.
19
20       This is done primarily to avoid the problems Test::Builder had by
21       exposing its singleton. We do not want anyone to replace this
22       singleton, rebless it, or directly muck with its internals. If you need
23       to do something and cannot because of the restrictions placed here,
24       then please report it as an issue. If possible, we will create a way
25       for you to implement your functionality without exposing things that
26       should not be exposed.
27

DESCRIPTION

29       This package exports all the functions necessary to write and/or verify
30       testing tools. Using these building blocks you can begin writing test
31       tools very quickly. You are also provided with tools that help you to
32       test the tools you write.
33

SYNOPSIS

35   WRITING A TOOL
36       The "context()" method is your primary interface into the Test2
37       framework.
38
39           package My::Ok;
40           use Test2::API qw/context/;
41
42           our @EXPORT = qw/my_ok/;
43           use base 'Exporter';
44
45           # Just like ok() from Test::More
46           sub my_ok($;$) {
47               my ($bool, $name) = @_;
48               my $ctx = context(); # Get a context
49               $ctx->ok($bool, $name);
50               $ctx->release; # Release the context
51               return $bool;
52           }
53
54       See Test2::API::Context for a list of methods available on the context
55       object.
56
57   TESTING YOUR TOOLS
58       The "intercept { ... }" tool lets you temporarily intercept all events
59       generated by the test system:
60
61           use Test2::API qw/intercept/;
62
63           use My::Ok qw/my_ok/;
64
65           my $events = intercept {
66               # These events are not displayed
67               my_ok(1, "pass");
68               my_ok(0, "fail");
69           };
70
71           my_ok(@$events == 2, "got 2 events, the pass and the fail");
72           my_ok($events->[0]->pass, "first event passed");
73           my_ok(!$events->[1]->pass, "second event failed");
74
75       DEEP EVENT INTERCEPTION
76
77       Normally "intercept { ... }" only intercepts events sent to the main
78       hub (as added by intercept itself). Nested hubs, such as those created
79       by subtests, will not be intercepted. This is normally what you will
80       still see the nested events by inspecting the subtest event. However
81       there are times where you want to verify each event as it is sent, in
82       that case use "intercept_deep { ... }".
83
84           my $events = intercept_Deep {
85               buffered_subtest foo => sub {
86                   ok(1, "pass");
87               };
88           };
89
90       $events in this case will contain 3 items:
91
92       The event from "ok(1, "pass")"
93       The plan event for the subtest
94       The subtest event itself, with the first 2 events nested inside it as
95       children.
96
97       This lets you see the order in which the events were sent, unlike
98       "intercept { ... }" which only lets you see events as the main hub sees
99       them.
100
101   OTHER API FUNCTIONS
102           use Test2::API qw{
103               test2_init_done
104               test2_stack
105               test2_set_is_end
106               test2_get_is_end
107               test2_ipc
108               test2_formatter_set
109               test2_formatter
110           };
111
112           my $init  = test2_init_done();
113           my $stack = test2_stack();
114           my $ipc   = test2_ipc();
115
116           test2_formatter_set($FORMATTER)
117           my $formatter = test2_formatter();
118
119           ... And others ...
120

MAIN API EXPORTS

122       All exports are optional. You must specify subs to import.
123
124           use Test2::API qw/context intercept run_subtest/;
125
126       This is the list of exports that are most commonly needed. If you are
127       simply writing a tool, then this is probably all you need. If you need
128       something and you cannot find it here, then you can also look at "OTHER
129       API EXPORTS".
130
131       These exports lack the 'test2_' prefix because of how important/common
132       they are. Exports in the "OTHER API EXPORTS" section have the 'test2_'
133       prefix to ensure they stand out.
134
135   context(...)
136       Usage:
137
138       $ctx = context()
139       $ctx = context(%params)
140
141       The "context()" function will always return the current context. If
142       there is already a context active, it will be returned. If there is not
143       an active context, one will be generated. When a context is generated
144       it will default to using the file and line number where the currently
145       running sub was called from.
146
147       Please see "CRITICAL DETAILS" in Test2::API::Context for important
148       rules about what you can and cannot do with a context once it is
149       obtained.
150
151       Note This function will throw an exception if you ignore the context
152       object it returns.
153
154       Note On perls 5.14+ a depth check is used to insure there are no
155       context leaks. This cannot be safely done on older perls due to
156       <https://rt.perl.org/Public/Bug/Display.html?id=127774> You can
157       forcefully enable it either by setting "$ENV{T2_CHECK_DEPTH} = 1" or
158       "$Test2::API::DO_DEPTH_CHECK = 1" BEFORE loading Test2::API.
159
160       OPTIONAL PARAMETERS
161
162       All parameters to "context" are optional.
163
164       level => $int
165           If you must obtain a context in a sub deeper than your entry point
166           you can use this to tell it how many EXTRA stack frames to look
167           back. If this option is not provided the default of 0 is used.
168
169               sub third_party_tool {
170                   my $sub = shift;
171                   ... # Does not obtain a context
172                   $sub->();
173                   ...
174               }
175
176               third_party_tool(sub {
177                   my $ctx = context(level => 1);
178                   ...
179                   $ctx->release;
180               });
181
182       wrapped => $int
183           Use this if you need to write your own tool that wraps a call to
184           "context()" with the intent that it should return a context object.
185
186               sub my_context {
187                   my %params = ( wrapped => 0, @_ );
188                   $params{wrapped}++;
189                   my $ctx = context(%params);
190                   ...
191                   return $ctx;
192               }
193
194               sub my_tool {
195                   my $ctx = my_context();
196                   ...
197                   $ctx->release;
198               }
199
200           If you do not do this, then tools you call that also check for a
201           context will notice that the context they grabbed was created at
202           the same stack depth, which will trigger protective measures that
203           warn you and destroy the existing context.
204
205       stack => $stack
206           Normally "context()" looks at the global hub stack. If you are
207           maintaining your own Test2::API::Stack instance you may pass it in
208           to be used instead of the global one.
209
210       hub => $hub
211           Use this parameter if you want to obtain the context for a specific
212           hub instead of whatever one happens to be at the top of the stack.
213
214       on_init => sub { ... }
215           This lets you provide a callback sub that will be called ONLY if
216           your call to "context()" generated a new context. The callback WILL
217           NOT be called if "context()" is returning an existing context. The
218           only argument passed into the callback will be the context object
219           itself.
220
221               sub foo {
222                   my $ctx = context(on_init => sub { 'will run' });
223
224                   my $inner = sub {
225                       # This callback is not run since we are getting the existing
226                       # context from our parent sub.
227                       my $ctx = context(on_init => sub { 'will NOT run' });
228                       $ctx->release;
229                   }
230                   $inner->();
231
232                   $ctx->release;
233               }
234
235       on_release => sub { ... }
236           This lets you provide a callback sub that will be called when the
237           context instance is released. This callback will be added to the
238           returned context even if an existing context is returned. If
239           multiple calls to context add callbacks, then all will be called in
240           reverse order when the context is finally released.
241
242               sub foo {
243                   my $ctx = context(on_release => sub { 'will run second' });
244
245                   my $inner = sub {
246                       my $ctx = context(on_release => sub { 'will run first' });
247
248                       # Neither callback runs on this release
249                       $ctx->release;
250                   }
251                   $inner->();
252
253                   # Both callbacks run here.
254                   $ctx->release;
255               }
256
257   release($;$)
258       Usage:
259
260       release $ctx;
261       release $ctx, ...;
262
263       This is intended as a shortcut that lets you release your context and
264       return a value in one statement. This function will get your context,
265       and an optional return value. It will release your context, then return
266       your value. Scalar context is always assumed.
267
268           sub tool {
269               my $ctx = context();
270               ...
271
272               return release $ctx, 1;
273           }
274
275       This tool is most useful when you want to return the value you get from
276       calling a function that needs to see the current context:
277
278           my $ctx = context();
279           my $out = some_tool(...);
280           $ctx->release;
281           return $out;
282
283       We can combine the last 3 lines of the above like so:
284
285           my $ctx = context();
286           release $ctx, some_tool(...);
287
288   context_do(&;@)
289       Usage:
290
291           sub my_tool {
292               context_do {
293                   my $ctx = shift;
294
295                   my (@args) = @_;
296
297                   $ctx->ok(1, "pass");
298
299                   ...
300
301                   # No need to call $ctx->release, done for you on scope exit.
302               } @_;
303           }
304
305       Using this inside your test tool takes care of a lot of boilerplate for
306       you. It will ensure a context is acquired. It will capture and rethrow
307       any exception. It will insure the context is released when you are
308       done. It preserves the subroutine call context (array, scalar, void).
309
310       This is the safest way to write a test tool. The only two downsides to
311       this are a slight performance decrease, and some extra indentation in
312       your source. If the indentation is a problem for you then you can take
313       a peek at the next section.
314
315   no_context(&;$)
316       Usage:
317
318       no_context { ... };
319       no_context { ... } $hid;
320               sub my_tool(&) {
321                   my $code = shift;
322                   my $ctx = context();
323                   ...
324
325                   no_context {
326                       # Things in here will not see our current context, they get a new
327                       # one.
328
329                       $code->();
330                   };
331
332                   ...
333                   $ctx->release;
334               };
335
336       This tool will hide a context for the provided block of code. This
337       means any tools run inside the block will get a completely new context
338       if they acquire one. The new context will be inherited by tools nested
339       below the one that acquired it.
340
341       This will normally hide the current context for the top hub. If you
342       need to hide the context for a different hub you can pass in the
343       optional $hid parameter.
344
345   intercept(&)
346       Usage:
347
348           my $events = intercept {
349               ok(1, "pass");
350               ok(0, "fail");
351               ...
352           };
353
354       This function takes a codeblock as its only argument, and it has a
355       prototype.  It will execute the codeblock, intercepting any generated
356       events in the process. It will return an array reference with all the
357       generated event objects. All events should be subclasses of
358       Test2::Event.
359
360       This is a very low-level subtest tool. This is useful for writing tools
361       which produce subtests. This is not intended for people simply writing
362       tests.
363
364   run_subtest(...)
365       Usage:
366
367           run_subtest($NAME, \&CODE, $BUFFERED, @ARGS)
368
369           # or
370
371           run_subtest($NAME, \&CODE, \%PARAMS, @ARGS)
372
373       This will run the provided codeblock with the args in @args. This
374       codeblock will be run as a subtest. A subtest is an isolated test state
375       that is condensed into a single Test2::Event::Subtest event, which
376       contains all events generated inside the subtest.
377
378       ARGUMENTS:
379
380       $NAME
381           The name of the subtest.
382
383       \&CODE
384           The code to run inside the subtest.
385
386       $BUFFERED or \%PARAMS
387           If this is a simple scalar then it will be treated as a boolean for
388           the 'buffered' setting. If this is a hash reference then it will be
389           used as a parameters hash. The param hash will be used for hub
390           construction (with the specified keys removed).
391
392           Keys that are removed and used by run_subtest:
393
394           'buffered' => $bool
395               Toggle buffered status.
396
397           'inherit_trace' => $bool
398               Normally the subtest hub is pushed and the sub is allowed to
399               generate its own root context for the hub. When this setting is
400               turned on a root context will be created for the hub that
401               shares the same trace as the current context.
402
403               Set this to true if your tool is producing subtests without
404               user-specified subs.
405
406           'no_fork' => $bool
407               Defaults to off. Normally forking inside a subtest will
408               actually fork the subtest, resulting in 2 final subtest events.
409               This parameter will turn off that behavior, only the original
410               process/thread will return a final subtest event.
411
412       @ARGS
413           Any extra arguments you want passed into the subtest code.
414
415       BUFFERED VS UNBUFFERED (OR STREAMED)
416
417       Normally all events inside and outside a subtest are sent to the
418       formatter immediately by the hub. Sometimes it is desirable to hold off
419       sending events within a subtest until the subtest is complete. This
420       usually depends on the formatter being used.
421
422       Things not effected by this flag
423           In both cases events are generated and stored in an array. This
424           array is eventually used to populate the "subevents" attribute on
425           the Test2::Event::Subtest event that is generated at the end of the
426           subtest.  This flag has no effect on this part, it always happens.
427
428           At the end of the subtest, the final Test2::Event::Subtest event is
429           sent to the formatter.
430
431       Things that are effected by this flag
432           The "buffered" attribute of the Test2::Event::Subtest event will be
433           set to the value of this flag. This means any formatter, listener,
434           etc which looks at the event will know if it was buffered.
435
436       Things that are formatter dependant
437           Events within a buffered subtest may or may not be sent to the
438           formatter as they happen. If a formatter fails to specify then the
439           default is to NOT SEND the events as they are generated, instead
440           the formatter can pull them from the "subevents" attribute.
441
442           A formatter can specify by implementing the "hide_buffered()"
443           method. If this method returns true then events generated inside a
444           buffered subtest will not be sent independently of the final
445           subtest event.
446
447       An example of how this is used is the Test2::Formatter::TAP formatter.
448       For unbuffered subtests the events are rendered as they are generated.
449       At the end of the subtest, the final subtest event is rendered, but the
450       "subevents" attribute is ignored. For buffered subtests the opposite
451       occurs, the events are NOT rendered as they are generated, instead the
452       "subevents" attribute is used to render them all at once. This is
453       useful when running subtests tests in parallel, since without it the
454       output from subtests would be interleaved together.
455

OTHER API EXPORTS

457       Exports in this section are not commonly needed. These all have the
458       'test2_' prefix to help ensure they stand out. You should look at the
459       "MAIN API EXPORTS" section before looking here. This section is one
460       where "Great power comes with great responsibility". It is possible to
461       break things badly if you are not careful with these.
462
463       All exports are optional. You need to list which ones you want at
464       import time:
465
466           use Test2::API qw/test2_init_done .../;
467
468   STATUS AND INITIALIZATION STATE
469       These provide access to internal state and object instances.
470
471       $bool = test2_init_done()
472           This will return true if the stack and IPC instances have already
473           been initialized. It will return false if they have not. Init
474           happens as late as possible. It happens as soon as a tool requests
475           the IPC instance, the formatter, or the stack.
476
477       $bool = test2_load_done()
478           This will simply return the boolean value of the loaded flag. If
479           Test2 has finished loading this will be true, otherwise false.
480           Loading is considered complete the first time a tool requests a
481           context.
482
483       test2_set_is_end()
484       test2_set_is_end($bool)
485           This is used to toggle Test2's belief that the END phase has
486           already started.  With no arguments this will set it to true. With
487           arguments it will set it to the first argument's value.
488
489           This is used to prevent the use of "caller()" in END blocks which
490           can cause segfaults. This is only necessary in some persistent
491           environments that may have multiple END phases.
492
493       $bool = test2_get_is_end()
494           Check if Test2 believes it is the END phase.
495
496       $stack = test2_stack()
497           This will return the global Test2::API::Stack instance. If this has
498           not yet been initialized it will be initialized now.
499
500       test2_ipc_disable
501           Disable IPC.
502
503       $bool = test2_ipc_diabled
504           Check if IPC is disabled.
505
506       test2_ipc_wait_enable()
507       test2_ipc_wait_disable()
508       $bool = test2_ipc_wait_enabled()
509           These can be used to turn IPC waiting on and off, or check the
510           current value of the flag.
511
512           Waiting is turned on by default. Waiting will cause the parent
513           process/thread to wait until all child processes and threads are
514           finished before exiting. You will almost never want to turn this
515           off.
516
517       $bool = test2_no_wait()
518       test2_no_wait($bool)
519           DISCOURAGED: This is a confusing interface, it is better to use
520           "test2_ipc_wait_enable()", "test2_ipc_wait_disable()" and
521           "test2_ipc_wait_enabled()".
522
523           This can be used to get/set the no_wait status. Waiting is turned
524           on by default. Waiting will cause the parent process/thread to wait
525           until all child processes and threads are finished before exiting.
526           You will almost never want to turn this off.
527
528       $fh = test2_stdout()
529       $fh = test2_stderr()
530           These functions return the filehandles that test output should be
531           written to.  They are primarily useful when writing a custom
532           formatter and code that turns events into actual output (TAP, etc.)
533           They will return a dupe of the original filehandles that formatted
534           output can be sent to regardless of whatever state the currently
535           running test may have left STDOUT and STDERR in.
536
537       test2_reset_io()
538           Re-dupe the internal filehandles returned by "test2_stdout()" and
539           "test2_stderr()" from the current STDOUT and STDERR.  You shouldn't
540           need to do this except in very peculiar situations (for example,
541           you're testing a new formatter and you need control over where the
542           formatter is sending its output.)
543
544   BEHAVIOR HOOKS
545       These are hooks that allow you to add custom behavior to actions taken
546       by Test2 and tools built on top of it.
547
548       test2_add_callback_exit(sub { ... })
549           This can be used to add a callback that is called after all testing
550           is done. This is too late to add additional results, the main use
551           of this callback is to set the exit code.
552
553               test2_add_callback_exit(
554                   sub {
555                       my ($context, $exit, \$new_exit) = @_;
556                       ...
557                   }
558               );
559
560           The $context passed in will be an instance of Test2::API::Context.
561           The $exit argument will be the original exit code before anything
562           modified it.  $$new_exit is a reference to the new exit code. You
563           may modify this to change the exit code. Please note that
564           $$new_exit may already be different from $exit
565
566       test2_add_callback_post_load(sub { ... })
567           Add a callback that will be called when Test2 is finished loading.
568           This means the callback will be run once, the first time a context
569           is obtained.  If Test2 has already finished loading then the
570           callback will be run immediately.
571
572       test2_add_callback_context_acquire(sub { ... })
573           Add a callback that will be called every time someone tries to
574           acquire a context. This will be called on EVERY call to
575           "context()". It gets a single argument, a reference to the hash of
576           parameters being used the construct the context. This is your
577           chance to change the parameters by directly altering the hash.
578
579               test2_add_callback_context_acquire(sub {
580                   my $params = shift;
581                   $params->{level}++;
582               });
583
584           This is a very scary API function. Please do not use this unless
585           you need to.  This is here for Test::Builder and backwards
586           compatibility. This has you directly manipulate the hash instead of
587           returning a new one for performance reasons.
588
589       test2_add_callback_context_init(sub { ... })
590           Add a callback that will be called every time a new context is
591           created. The callback will receive the newly created context as its
592           only argument.
593
594       test2_add_callback_context_release(sub { ... })
595           Add a callback that will be called every time a context is
596           released. The callback will receive the released context as its
597           only argument.
598
599       test2_add_callback_pre_subtest(sub { ... })
600           Add a callback that will be called every time a subtest is going to
601           be run. The callback will receive the subtest name, coderef, and
602           any arguments.
603
604       @list = test2_list_context_acquire_callbacks()
605           Return all the context acquire callback references.
606
607       @list = test2_list_context_init_callbacks()
608           Returns all the context init callback references.
609
610       @list = test2_list_context_release_callbacks()
611           Returns all the context release callback references.
612
613       @list = test2_list_exit_callbacks()
614           Returns all the exit callback references.
615
616       @list = test2_list_post_load_callbacks()
617           Returns all the post load callback references.
618
619       @list = test2_list_pre_subtest_callbacks()
620           Returns all the pre-subtest callback references.
621
622       test2_add_uuid_via(sub { ... })
623       $sub = test2_add_uuid_via()
624           This allows you to provide a UUID generator. If provided UUIDs will
625           be attached to all events, hubs, and contexts. This is useful for
626           storing, tracking, and linking these objects.
627
628           The sub you provide should always return a unique identifier. Most
629           things will expect a proper UUID string, however nothing in
630           Test2::API enforces this.
631
632           The sub will receive exactly 1 argument, the type of thing being
633           tagged 'context', 'hub', or 'event'. In the future additional
634           things may be tagged, in which case new strings will be passed in.
635           These are purely informative, you can (and usually should) ignore
636           them.
637
638   IPC AND CONCURRENCY
639       These let you access, or specify, the IPC system internals.
640
641       $bool = test2_has_ipc()
642           Check if IPC is enabled.
643
644       $ipc = test2_ipc()
645           This will return the global Test2::IPC::Driver instance. If this
646           has not yet been initialized it will be initialized now.
647
648       test2_ipc_add_driver($DRIVER)
649           Add an IPC driver to the list. This will add the driver to the
650           start of the list.
651
652       @drivers = test2_ipc_drivers()
653           Get the list of IPC drivers.
654
655       $bool = test2_ipc_polling()
656           Check if polling is enabled.
657
658       test2_ipc_enable_polling()
659           Turn on polling. This will cull events from other processes and
660           threads every time a context is created.
661
662       test2_ipc_disable_polling()
663           Turn off IPC polling.
664
665       test2_ipc_enable_shm()
666           Turn on IPC SHM. Only some IPC drivers use this, and most will turn
667           it on themselves.
668
669       test2_ipc_set_pending($uniq_val)
670           Tell other processes and events that an event is pending. $uniq_val
671           should be a unique value no other thread/process will generate.
672
673           Note: After calling this "test2_ipc_get_pending()" will return 1.
674           This is intentional, and not avoidable.
675
676       $pending = test2_ipc_get_pending()
677           This returns -1 if there is no way to check (assume yes)
678
679           This returns 0 if there are (most likely) no pending events.
680
681           This returns 1 if there are (likely) pending events. Upon return it
682           will reset, nothing else will be able to see that there were
683           pending events.
684
685       $timeout = test2_ipc_get_timeout()
686       test2_ipc_set_timeout($timeout)
687           Get/Set the timeout value for the IPC system. This timeout is how
688           long the IPC system will wait for child processes and threads to
689           finish before aborting.
690
691           The default value is 30 seconds.
692
693   MANAGING FORMATTERS
694       These let you access, or specify, the formatters that can/should be
695       used.
696
697       $formatter = test2_formatter
698           This will return the global formatter class. This is not an
699           instance. By default the formatter is set to Test2::Formatter::TAP.
700
701           You can override this default using the "T2_FORMATTER" environment
702           variable.
703
704           Normally 'Test2::Formatter::' is prefixed to the value in the
705           environment variable:
706
707               $ T2_FORMATTER='TAP' perl test.t     # Use the Test2::Formatter::TAP formatter
708               $ T2_FORMATTER='Foo' perl test.t     # Use the Test2::Formatter::Foo formatter
709
710           If you want to specify a full module name you use the '+' prefix:
711
712               $ T2_FORMATTER='+Foo::Bar' perl test.t     # Use the Foo::Bar formatter
713
714       test2_formatter_set($class_or_instance)
715           Set the global formatter class. This can only be set once. Note:
716           This will override anything specified in the 'T2_FORMATTER'
717           environment variable.
718
719       @formatters = test2_formatters()
720           Get a list of all loaded formatters.
721
722       test2_formatter_add($class_or_instance)
723           Add a formatter to the list. Last formatter added is used at
724           initialization. If this is called after initialization a warning
725           will be issued.
726

OTHER EXAMPLES

728       See the "/Examples/" directory included in this distribution.
729

SEE ALSO

731       Test2::API::Context - Detailed documentation of the context object.
732
733       Test2::IPC - The IPC system used for threading/fork support.
734
735       Test2::Formatter - Formatters such as TAP live here.
736
737       Test2::Event - Events live in this namespace.
738
739       Test2::Hub - All events eventually funnel through a hub. Custom hubs
740       are how "intercept()" and "run_subtest()" are implemented.
741

MAGIC

743       This package has an END block. This END block is responsible for
744       setting the exit code based on the test results. This end block also
745       calls the callbacks that can be added to this package.
746

SOURCE

748       The source code repository for Test2 can be found at
749       http://github.com/Test-More/test-more/.
750

MAINTAINERS

752       Chad Granum <exodist@cpan.org>
753

AUTHORS

755       Chad Granum <exodist@cpan.org>
756
758       Copyright 2018 Chad Granum <exodist@cpan.org>.
759
760       This program is free software; you can redistribute it and/or modify it
761       under the same terms as Perl itself.
762
763       See http://dev.perl.org/licenses/
764
765
766
767perl v5.26.3                      2018-03-30                     Test2::API(3)
Impressum