1Test::Inter(3)        User Contributed Perl Documentation       Test::Inter(3)
2
3
4

NAME

6       Test::Inter - framework for more readable interactive test scripts
7

DESCRIPTION

9       This is another framework for writing test scripts. Much of the syntax
10       is loosely inspired by Test::More, and Test::Inter has most of it's
11       functionality, but it is not a drop-in replacement.
12
13       Test::More (and other existing test frameworks) suffer from two
14       weaknesses, both of which have prevented me from ever using them:
15
16          None offer the ability to access specific tests in
17          a reasonably interactive fashion, primarily for
18          debugging purposes
19
20          None offer the ability to write the tests in
21          whatever format would make the tests the most
22          readable
23
24       The way I write and use test scripts, existing Test::* modules are not
25       nearly as useful as they could be.
26
27       Test scripts written using Test::More work fine when running as part of
28       the test suite, but debugging an individual test requires extra steps,
29       and the tests themselves are not as readable as they should be.
30

INTERACTIVE EXECUTION

32       One requirement that I have of a test framework is the ability to
33       interact with it.
34
35       I do most of my debugging using test scripts. When I find a bug, I
36       write a test case for it (typically by adding it to an existing test
37       script) and then debug it using the test script.  Then I leave the test
38       there to ensure that the bug won't come back (hopefully).
39
40       Since I use test scripts in a very interactive way (often in the
41       debugger), I want to be able to do the following trivially:
42
43       Easy access to a specific test or tests
44           I'd like to be able to run only a single test, or a subset of
45           tests.
46
47       Easily set breakpoints in the debugger
48           Setting a breakpoint in the debugger to run up to the start of the
49           Nth test is one of the most common tasks I want to do when I'm
50           debugging a failed test.
51
52       To illustrate the first point, in Test::More, a series of tests might
53       be specified in a test script as shown in the following example (line
54       numbers added for convenience):
55
56          ...
57
58          100:  # test 1
59          101:  $result = func("apples","bushels");
60          102:  is($result, "enough");
61          103:
62          104:  # test 2
63          105:  $result = func("grapefruit","tons");
64          106:  is($result, "enough");
65          107:
66          108:  # test 3
67          109:  $result = func("oranges","boatloads");
68          110:  is($result, "insufficient");
69          111:
70          112:  # tests 4-6
71          113:  foreach my $arg (qw(pears plums pineapple)) {
72          114:    $result = func($arg,"boxes");
73          115:    is($result, "enough");
74          116:  }
75
76          ...
77
78       Say you ran the test suite, and test 3 failed.  To debug it you have to
79       open up the test script, find the 3rd test, and set the appropriate
80       breakpoint.  In this case, you'll want to break at line 109.
81
82       None of these steps are impossible of course, but it will take some
83       time to get it right.  It becomes harder when there are lots of tests
84       (imagine that you want to test the 117th test instead of the 3rd test)
85       or when tests are wrapped up in loops, embedded in subroutines, or
86       other similar situations.
87
88       As an example, what if it's the 5th test that fails in the example
89       above.  Now the break point will be a conditional one, so you have to
90       figure out not only the line, but the condition the appropriate state
91       during that test.  In this case, you need to stop at line 114 when $arg
92       is 'plums'.
93
94       Wouldn't it be far better to set a break point in func when the Nth
95       test is reached?  With Test::Inter, you can.
96
97       So for the above script, the debugger commands that you would use to
98       debug the 3rd test are:
99
100          Test::More :   b 109
101          Test::Inter:   b func ($::TI_NUM==3)
102
103       and the 5th test are:
104
105          Test::More :   b 114  ($arg eq 'plums')
106          Test::Inter:   b func ($::TI_NUM==5)
107
108       It would also be nice to be able to skip the first two tests... perhaps
109       they take a long time to run, and I want to get right to work on test
110       3.  You can do this easily too by setting the $::TI_START variable.
111
112       There are some other variables that can be used to specify which test
113       or tests to run described in the "TEST::INTER VARIABLES" section below.
114
115       The other thing I want to do when I run the test scripts interactively
116       is to see more information which will assist in debugging a failed
117       test.
118
119       This can be controlled with variables such as TI_QUIET, TI_MODE, and
120       TI_WIDTH described below in the "TEST::INTER VARIABLES" section.
121

READABLE TESTS

123       The other feature that I wanted in a test suite is the ability to
124       define the tests in a format that is natural and readable FOR THE
125       TESTS.  In almost every case, it is best to think of a test script as
126       consisting of two separate parts: a script part, and a test part.
127
128       The script part of a test script is the least important part! It's
129       usually fairly trivial, rarely needs to be changed, and is not the
130       focus of the test script.
131
132       The tests part of the script IS the important part, and these should be
133       expressed in a form that is natural to them, easy to maintain, easy to
134       read, and easy to modify, and none of these should involve modifying
135       the script portion of the test script in general. Because the content
136       of the tests is the important part of the script, the emphasis should
137       be in making them more readable, even at the expense of the script
138       portion.  As a general rule, if the script portion of the test script
139       obscures the tests in any way, it's not written correctly!
140
141       The solution to this is well understood, and is common to many other
142       systems where you are mixing two "languages".  The task of correctly
143       specifying both the tests and the test script is virtually identical to
144       the task of creating a PHP script which consists of a mixture of PHP
145       and HTML, or the task of creating a template file using some templating
146       system where the file consists of a mixture of text to be displayed and
147       templating commands.  It is well understood in each of these cases that
148       the more the two "languages" are interwoven, the less readable both
149       are, and the harder it is to maintain.  The more you are able to
150       separate the two, the easier both are to read and maintain.
151
152       As often as possible, I want the tests to be written in some sort of
153       text format which can be easily viewed and modified (usually as a
154       simple table) with no perl commands interspersed. I want to the freedom
155       to define the tests in one section (a long string, the DATA section, or
156       even in a separate file) which is easily readable. This may introduce
157       the necessity of parsing it, but it makes it significantly easier to
158       maintain the tests.
159
160       This flexibility makes it much easier to read the tests (as opposed to
161       the script) which is the fundamental content of a test script.
162
163       Looking again at the example test script, you can see that there is far
164       too much perl interspersed with the tests.
165
166       It's difficult to read the tests individually in this script because
167       there is too much perl code among them, and virtually impossible to
168       look at them as a whole.
169
170       It is true that looking at this particular example, it is very
171       simple... but the script ISN'T the content you're interested in (and
172       bear in mind that many test scripts are nowhere near this simple). The
173       REAL content of this script are the tests, which consist of the
174       function arguments and the expected result. Although it's not
175       impossible to see each of these in the script above, it's not in a
176       format that is conducive to studying the tests, and especially not for
177       examining the list of tests as a whole.
178
179       Now, look at an alternate way of specifying the tests using this
180       module:
181
182          $tests = "
183
184            apples     bushels   => enough
185
186            grapefruit tons      => enough
187
188            oranges    boatloads => insufficient
189
190            pears      boxes     => enough
191
192            plums      boxes     => enough
193
194            pineapple  boxes     => enough
195
196          ";
197
198          $o->tests(tests => $tests,
199                    func  => \&func);
200
201       Here, it's easy to see the list of tests, and adding additional tests
202       is a breeze.
203

CREATING A TEST

205       This module supports a number of methods for defining tests, so you can
206       use whichever one is most convenient (including methods that are
207       identical to Test::More if that really is the best method).
208
209       Every test may have several pieces of information:
210
211       A name
212           Every test is automatically assigned a number, but it may be useful
213           to specify a name of a test (which is actually a short description
214           of the test). Whenever a test result is reported, the name will be
215           given (if one was specified).
216
217           The name may not have a '#' in it.
218
219           The name is completely optional, but makes the results more
220           readable.
221
222       An expected result
223           In order to test something, you need to know what result was
224           expected (or in some cases, what result was NOT expected).
225
226       A function and arguments OR a result
227           You also need to know the results that you're comparing to the
228           expected results.
229
230           This can be obtained by simply working with a set of results, or a
231           function name and a set of arguments to pass to it.
232
233       Conditions
234           It is useful to be able to specify state information at the start
235           of the test suite (for example, to see if certain features are
236           available), and some tests may only run if those conditions are
237           met.
238
239           If no conditions are set for a test, it will always run.
240
241       Todo tests
242           Some tests may be marked as 'todo' tests. These are test which are
243           allowed to fail (meaning that they have been put in place for an
244           as-yet unimplemented feature). Since it is expected that the test
245           will fail, the test suite will still pass, even if these tests
246           fail.
247
248           The tests will still run and if they pass, a message is issued
249           saying that the feature is now implemented, and the tests should be
250           graduated to non-todo state.
251

BASE METHODS

253       new
254              $o = new Test::Inter [$name] [%options];
255
256           This creates a new test framework. There are several options which
257           may be used to specify which tests are run, how they are run, and
258           what output is given.
259
260           The entire test script can be named by passing in $name.
261
262           All options can be set in four different ways.
263
264           First, you can pass in a hash of OPT = VAL> pairs in the new
265           method.  So, to set the start option, the %options) hash would
266           contain:
267
268              start => VALUE
269
270           Second, you can set an environment variable.  This will override
271           any value passed in the first way.  The environment variable is
272           named TI_XXX where XXX is the fully capitalized option.  So:
273
274              $ENV{TI_START} = VALUE
275
276           The third method, which overrides the previous two, is to set a
277           global variable.  It is also named TI_XXX in the main namespace, so
278           to set it this way, set:
279
280              $::TI_START = VALUE
281
282           The final method is to call one of the methods below and these
283           override all other methods.
284
285           Each of the allowed options are described below in the following
286           base methods:
287
288              start
289              end
290              testnum
291              plan
292              abort
293              quiet
294              mode
295              skip_all
296              width
297              use_lib
298
299       version
300              $o->version();
301
302           Returns the version of the module.
303
304       encoding
305              $o->encoding($encoding);
306
307           $encoding is any value that can be passed as an encoding to perl's
308           Encode::decode function.
309
310           Use this if your test strings contain characters in other
311           encodings.
312
313       start
314              $o = new Test::Inter 'start' => $N;
315              $o->start($N)
316
317           To define which test you want to start with, set the start option
318           as described in the new method above.
319
320           When the start test is defined, most tests numbered less than N are
321           completely ignored. If the tests are being run quietly (see the
322           quiet method below), nothing is printed out for these tests.
323           Otherwise, a skip message is printed out.
324
325           One class of tests IS still executed. Tests run using the
326           require_ok or use_ok methods (to test the loading of modules) are
327           still run.
328
329           If no value (or a value of 0) is used, tests run from the first
330           test.
331
332       end
333              $o = new Test::Inter 'end' => $M;
334              $o->end($M);
335
336           To define which test you want to end with, set the end option as
337           described in the new method above.
338
339           When the end test is defined, all tests numbered more than M are
340           completely ignored. If the tests are being run quietly (see the
341           quiet method below), nothing is printed out for these tests.
342           Otherwise, a skip message is printed out.
343
344           If no value is given, it defaults to 0 (which means that all
345           remaining tests are run).
346
347       testnum
348              $o = new Test::Inter 'testnum' => $N;
349              $o->testnum($N);
350
351           To run only a single test, set the testnum option as described in
352           the new method above.
353
354           It is equivalent to setting both the start and end tests to $N.
355
356       plan
357       done_testing
358              $o = new Test::Inter 'plan' => $N;
359              $o->plan($n);
360
361              $o->done_testing();
362              $o->done_testing($n);
363
364           The TAP API (the 'language' used to run a sequence of tests and see
365           which ones failed and which ones passed) requires a statement of
366           the number of tests that are expected to run.
367
368           This statement can appear at the start of the test suite, or at the
369           end.
370
371           If you know in advance how many tests should run in the test
372           script, you can set the plan option as described in the new method
373           above to the number of tests.
374
375           If you know how many tests should run at the end of the test
376           script, you can pass in a non-zero integer to the done_testing
377           method.
378
379           Frequently, you don't really care how many tests are in the script
380           (especially if new tests are added on a regular basis). In this
381           case, you still need to include a statement that says that the
382           number of tests expected is however many were run. To do this, call
383           the done_testing method with no argument.
384
385           NOTE: if the plan method is used, it MUST be used before any tests
386           are run (including those that test the loading of modules). If the
387           done_testing method is used, it MUST be called after all tests are
388           run. You must specify a plan or use a done_testing statement, but
389           you cannot do both.
390
391           It is NOT strictly required to set a plan if the script is only run
392           interactively, so if for some reason this module is used for test
393           scripts which are not part of a standard perl test suite, the plan
394           and done_testing statements are optional. As a matter of fact, the
395           script will run just fine without them... but a perl installer will
396           report a failure in the test suite.
397
398       abort
399              $o = new Test::Inter 'abort' => 0/1/2;
400              $o->abort(0/1/2);
401
402           To define how you want a failure to be treated, set the abort
403           option as described in the new method above.  The abort option can
404           take a value of 0, 1, or 2.
405
406           If this is set to 1, the test script will run unmodified until a
407           test fails. At that point, all remaining tests will be skipped.  If
408           it is set to 2, the test script will run until a test fails at
409           which point it will exit with an error code of 1.  With a value of
410           0, failed tests will be reported, but the script will continue.
411
412           In both cases, todo tests will NOT trigger the abort behavior.
413
414       quiet
415              $o = new Test::Inter 'quiet' => 0/1/2;
416              $o->quiet(0/1/2);
417
418           To define how you want failures to be reported, set the quiet
419           option as described in the new method above.  The quiet option can
420           take a value of 0, 1, or 2.
421
422           If this is set to 0 (the default), all information will be printed
423           out. If it is set to 1, some optional information will not be
424           printed.  If it is set to 2, all optional information will not be
425           printed.
426
427       mode
428              $o = new Test::Inter 'mode' => MODE;
429              $o->mode(MODE);
430
431           Test::Inter scripts can be run in either an interactive mode, or as
432           part of a test suite with different behaviors.  To select the mode,
433           set the mode option as described in the new method above.  The mode
434           option can take a value of 'inter' or 'test'.
435
436           When run in test mode, it prints out the results using the TAP
437           grammar (i.e. 'ok 1', 'not ok 3', etc.).
438
439           When run in interactive mode, it prints out results in a more human
440           readable format.
441
442       width
443              $o = new Test::Inter 'width' => WIDTH;
444              $o->width(WIDTH);
445
446           The width option can be set as described in the new method above.
447
448           WIDTH is the width of the terminal (for printing out failed test
449           information). It defaults to 80, but it can be set to any width
450           (and lines longer then this are truncated). If WIDTH is set to 0,
451           no truncation is done.
452
453       use_lib
454              $o = new Test::Inter 'use_lib' => VALUE;
455              $o->use_lib(VALUE);
456              $o->use_lib();
457
458           By default, the library included in the module distribution will be
459           added to the search path for modules, so a 'use MODULE' line should
460           find the version stored in this module distribution.
461
462           If VALUE is set to 'off', the search path will not be modified
463           automatically.
464
465           You may add the library path at a later time by calling:
466
467              $o->use_lib('on');
468              $o->use_lib();
469
470           Note: both calls must be used. The first sets the option, the
471           second actually modifies the search path.
472
473       skip_all
474              $o = new Test::Inter 'skip_all' => REASON;
475              $o->skip_all(REASON);
476
477           The skip_all option can be set as described in the new method
478           above.
479
480           If this is set, the entire test script will be skipped for the
481           reason given. This must be done before any test is run, and before
482           any plan number is set.
483
484           The skip_all can also be called at any point during the script
485           (i.e.  after tests have been run). In this case, all remaining
486           scripts will be skipped.
487
488              $o->skip_all(REASON,FEATURE,FEATURE,...);
489              $o->skip_all('',FEATURE,FEATURE,...);
490
491           This will skip all tests (or all remaining tests) unless all
492           <FEATURE>s are available.  REASON can be entered as an empty string
493           and the reason the tests are skipped will be a message about the
494           missing feature.
495
496       feature
497              $o->feature($feature,$val);
498
499           This defines a feature. If $val is non-zero, the feature is
500           available.  Otherwise it is not.
501
502       diag
503       note
504              $o->diag($message);
505              $o->note($message);
506
507           Both of these print an optional message. Messages printed with the
508           "note" method are always optional and will be omitted if the quiet
509           option is set to 1 or 2. Messages printed with the "diag" method
510           are optional and will not be printed if the quiet option is set to
511           2, but they will be printed if the quiet method is set to 1.
512
513       testdir
514              $o->testdir();
515              $o->testdir('mod');
516              $o->testdir('lib');
517
518           Occasionally, it may be necessary to know the directory where
519           Test::Inter gets some of it's information.  By default, the
520           directory containing the tests will be returned, but if the
521           optional argument 'mod' is included, it will return the path to the
522           module distribution (which should include both a lib and t
523           subdirerctory).  If the argument 'lib' is included, it will return
524           the directory where the libraries are stored.
525

METHODS FOR LOADING MODULES

527       Test scripts can load other modules (using either the perl "use" or
528       "require" commands).  There are three different modes for doing this
529       which determine how this is done.
530
531       required
532           By default, this is used to test for a module that is required for
533           all tests in the test script.
534
535           Loading the module is treated as an actual test in the test suite.
536           The test is to determine whether the module is available and can be
537           loaded. If it can be loaded, it is, and it is reported as a
538           successful test. If it cannot be loaded, it is reported as a failed
539           test.
540
541           In the result of a failed test, all remaining tests will be skipped
542           automatically (except for other tests which load modules).
543
544       feature
545           In feature mode, loading the module is not treated as a test (i.e.
546           it will not print out an 'ok' or 'not ok' line. Instead, it will
547           set a feature (named the same as the module) which can be used to
548           determine whether other tests should run or not.
549
550       forbid
551           In a few very rare cases, we may want to test for a module but
552           expect that it not be present. This is the exact opposite of the
553           required mode.
554
555           Successfully loading the module is treated as a test failure. In
556           the event of a failure, all remaining tests will be skipped.
557
558       The methods available are:
559
560       require_ok
561              $o->require_ok($module [,$mode]);
562
563           This is used to load a module using the perl "require" function. If
564           $mode is not passed in, the default mode (required) is used to test
565           the existence of the module.
566
567           If $mode is passed in, it must be either the string 'forbid' or
568           'feature'.
569
570           If $mode is 'feature', a feature named $module is set if the module
571           was able to be loaded.
572
573       use_ok
574              $o->use_ok(@args [,$mode]);
575
576           This is used to load a module with "use", or check a perl version.
577
578              BEGIN { $o->use_ok('5.010'); }
579              BEGIN { $o->use_ok('Some::Module'); }
580              BEGIN { $o->use_ok('Some::Module',2.05); }
581              BEGIN { $o->use_ok('Some::Module','foo','bar'); }
582              BEGIN { $o->use_ok('Some::Module',2.05,'foo','bar'); }
583
584           are the same as:
585
586              use 5.010;
587              use Some::Module;
588              use Some::Module 2.05;
589              use Some::Module qw(foo bar);
590              use Some::Module 2.05 qw(foo bar);
591
592           Putting the use_ok call in a BEGIN block allows the functions to be
593           imported at compile-time and prototypes are properly honored.
594           You'll also need to load the Test::Inter module, and create the
595           object in a BEGIN block.
596
597           $mode acts the same as in the require_ok method.
598

METHODS FOR RUNNING TEST

600       There are several methods for running tests. The ok, is, and isnt
601       methods are included for those already comfortable with Test::More and
602       wishing to stick with the same format of test script. The tests method
603       is the suggested method though since it makes use of the full power of
604       this module.
605
606       ok
607              $o->ok(TESTS);
608
609           A test run with ok looks at a result, and if it evaluates to 0 (or
610           false), it fails. If it evaluates to non-zero (or true), it passes.
611
612           These tests do not require you to specify the expected results.  If
613           expected results are given, they will be compared against the
614           result received, and if they differ, a diagnostic message will be
615           printed, but the test will still succeed or fail based only on the
616           actual result produced.
617
618           These tests require a single result and either zero or one expected
619           results.
620
621           To run a single test, use any of the following:
622
623              $o->ok();          # always succeeds
624
625              $o->ok($result);
626              $o->ok($result,$name);
627              $o->ok($result,$expected,$name);
628
629              $o->ok(\&func);
630              $o->ok(\&func,$name);
631              $o->ok(\&func,$expected,$name);
632
633              $o->ok(\&func,\@args);
634              $o->ok(\&func,\@args,$name);
635              $o->ok(\&func,\@args,$expected,$name);
636
637           If $result is a scalar, the test passes if $result is true. If
638           $result is a list reference, the test succeeds if the list contains
639           any defined values. If $result is a hash reference, the test
640           succeeds if the hash contains any key with a value that is not
641           "undef".
642
643           If "\&func" and "\@args" are passed in, then $result is generated
644           by passing @args to &func and behaves identically to the calls
645           where $result is passed in.  If "\&func" is passed in but no
646           arguments, the function takes no arguments, but still produces a
647           result.
648
649           If an expected value is passed in and the result does not match it,
650           a diagnostic warning will be printed, even if the test passes.
651
652       is
653       isnt
654              $o->is(TESTS);
655              $o->isnt(TESTS);
656
657           A test run with is looks at a result and tests to see if it is
658           identical to an expected result. If it is, the test passes.
659           Otherwise it fails. In the case of a failure, a diagnostic message
660           will show what result was actually obtained and what was expected.
661
662           A test run with isnt looks at a result and tests to see if the
663           result obtained is different than an expected result. If it is
664           different, the test passes.  Otherwise it fails.
665
666           The is method can be called in any of the following ways:
667
668              $o->is($result,$expected);
669              $o->is($result,$expected,$name);
670
671              $o->is(\&func,$expected);
672              $o->is(\&func,$expected,$name);
673
674              $o->is(\&func,\@args,$expected);
675              $o->is(\&func,\@args,$expected,$name);
676
677           The isnt method can be called in exactly the same way.
678
679           As with the ok method, the result can be a scalar, hashref, or
680           listref. If it is a hashref or listref, the entire structure must
681           match the expected value.
682
683       tests
684              $o->tests($opt=>$val, $opt=>$val, ...)
685
686           The options available are described in the following section.
687
688       file
689              $o->file($func,$input,$outputdir,$expected,$name [,@args]);
690
691           Sometimes it may be easiest to store the input, output, and
692           expected output from a test in a text file. In this case, each line
693           of output will be treated as a single test, so the output and
694           expected output must match up exactly.
695
696           $func is a reference to a function which will produce a temporary
697           output file.
698
699           If $input is specified, it is the name of the input file.  If it is
700           empty, no input file will be used.  The input file can be fully
701           specified, or it can be relative to the test directory.
702
703           If $outputdir is passed in, it is the directory where the output
704           file will be written.  It can be fully specified, or relative to
705           the test directory.  If $outputdir is left blank, the temporary
706           file will be written to the test directory.
707
708           $expected is the name of a file which contains the expected output.
709           It can be fully specified, or it will be checked for in the test
710           directory.
711
712           $name is the name of this series of tests.
713
714           @args are extra arguments to pass to the test function.
715
716           The function will be called with the arguments:
717
718              &$func( [$input,] $output,@args);
719
720           $input is only passed in if it was passed in to this method.  If no
721           input file is specified, nothing will be passed to the function.
722
723           $output is the name of a temporary file where the output will be
724           written to.
725

USING THE TESTS METHOD

727       It is expected that most tests (except for those that load a module)
728       will be run using the tests method called as:
729
730          $o->tests(%options);
731
732       The following options are available:
733
734       name
735              name => NAME
736
737           This sets the name of this set of tests. All tests will be given
738           the same name.
739
740       tests
741       func
742       expected
743           In order to specify a series of tests, you have to specify either a
744           function and a list of arguments, or a list of results.
745
746           Specifying the function and list of arguments can be done using the
747           pair:
748
749              func  => \&FUNCTION
750              tests => TESTS
751
752           If the func option is not set, tests contains a list of results.
753
754           A list of expected results may also be given. They can be included
755           in the
756
757              tests => TESTS
758
759           option or included separately as:
760
761              expected => RESULTS
762
763           The way to specify these are covered in the next section SPECIFYING
764           THE TESTS.
765
766       feature
767       disable
768              feature => [FEATURE1, FEATURE2, ...]
769
770              disable => [FEATURE1, FEATURE2, ...]
771
772           The default set of tests to run is determined using the start, end,
773           and skip_all methods discussed above. Using those methods, a list
774           of tests is obtained, and it is expected that these will run.
775
776           The feature and disable options modify the list.
777
778           If the feature option is included, the tests given in this call
779           will only run if ALL of the features listed are available.
780
781           If the disable option is included, the tests will be run unless ANY
782           of the features listed are available.
783
784       skip
785              skip => REASON
786
787           Skip these tests for the reason given.
788
789       todo
790              todo => 0/1
791
792           Setting this to 1 says that these tests are allowed to fail. They
793           represent a feature that is not yet implemented.
794
795           If the tests succeed, a message will be printed notifying the
796           developer that the tests are now ready to promote to actual use.
797

SPECIFYING THE TESTS

799       A series of tests can be specified in two different ways. The tests can
800       be written in a very simple string format, or stored as a list.
801
802       Demonstrating how this can be done is best done by example, so let's
803       say that there is a function (func) which takes two arguments, and
804       returns a single value.  Let's say that the expected output (and the
805       actual output) from 3 different sets of arguments is:
806
807          Input   Expected Output  Actual Output
808          -----   ---------------  -------------
809          1,2     a                a
810          3,4     b                x
811          5,6     c                c
812
813       (so in this case, the first and third tests pass, but the 2nd one will
814       fail).
815
816       Specifying these tests as lists could be done as:
817
818          $o->tests(
819             func     => &func,
820             tests    => [ [1,2], [3,4], [5,6] ],
821             expected => [ [a],   [b],   [c] ],
822          );
823
824       Here, the tests are stored as a list, and each element in the list is a
825       listref containing the set of arguments.
826
827       If the func option is not passed in, the tests option is set to a list
828       of results to compare with the expected results, so the following is
829       equivalent to the above:
830
831          $o->tests(
832             tests    => [ [a],   [x],   [c] ],
833             expected => [ [a],   [b],   [c] ],
834          );
835
836       If an argument (or actual result) or an expected result is only a
837       single value, it can be entered as a scalar instead of a list ref, so
838       the following is also equivalent:
839
840          $o->tests(
841             func     => &func,
842             tests    => [ [1,2], [3,4], [5,6] ],
843             expected => [ a,     b,     [c] ],
844          );
845
846       The only exception to this is if the single value is itself a list
847       reference.  In this case it MUST be included as a reference. In other
848       words, if you have a single test, and the expected value for this test
849       is a list reference, it must be passed in as:
850
851          expected => [ [ \@r ] ]
852
853       NOT as:
854
855          expected => [ \@r ]
856
857       Passing in a set of expected results is optional. If none are passed
858       in, the tests are treated as if they had been passed to the ok method
859       (i.e. if they return something true, they pass, otherwise they fail).
860
861       The second way to specify tests is as a string. The string is a multi-
862       line string with each tests being separate from the next test by a
863       blank line.  Comments (lines which begin with '#') are allowed, and are
864       ignored. Whitespace at the start and end of the line is ignored.
865
866       The string may contain the results directly, or results may be passed
867       in separately. For example, the following all give the same sets of
868       tests as the example above:
869
870          $o->tests(
871             func     => &func,
872             tests    => "
873                          # Test 1
874                          1 2 => a
875
876                          # Test 2
877                          3 4 => b
878
879                          5 6 => c
880                         ",
881          );
882
883          $o->tests(
884             func     => &func,
885             tests    => "
886                          1 2
887
888                          3 4
889
890                          5 6
891                         ",
892              expected => [ [a], [b], [c] ]
893          );
894
895          $o->tests(
896             func     => &func,
897             tests    => [ [1,2], [3,4], [5,6] ],
898             expected => "
899                          a
900
901                          b
902
903                          c
904                         ",
905          );
906
907          $o->tests(
908             func     => &func,
909             tests    => "
910                          1 2
911
912                          3 4
913
914                          5 6
915                         ",
916             expected => "
917                          a
918
919                          b
920
921                          c
922                         ",
923          );
924
925       The expected results may also consist of only a single set of results
926       (in this case, it must be passed in as a listref). In this case, all of
927       the tests are expected to have the same results.
928
929       So, the following are equivalent:
930
931          $o->tests(
932             func     => &func,
933             tests    => "
934                          1 2 => a b
935
936                          3 4 => a b
937
938                          5 6 => a b
939                         ",
940          );
941
942          $o->tests(
943             func     => &func,
944             tests    => "
945                          1 2
946
947                          3 4
948
949                          5 6
950                         ",
951             expected  => [ [a, b] ],
952          );
953
954          $o->tests(
955             func     => &func,
956             tests    => "
957                          1 2
958
959                          3 4
960
961                          5 6
962                         ",
963             expected  => "a b",
964          );
965
966       The number of expected values must either be 1 (i.e. all of the tests
967       are expected to produce the same value) or exactly the same number as
968       the number of tests.
969
970       The parser is actually quite powerful, and can handle multi-line tests,
971       quoted strings, and nested data structures.
972
973       The test may be split across any number of lines, provided there is not
974       a completely blank line (which signals the end of the test), so the
975       following are equivalent:
976
977          tests => "a b c",
978          tests => "a b
979                    c",
980
981       Arguments (or expected results) may include data structures. For
982       example, the following are equivalent:
983
984          tests => "[ a b ] { a 1 b 2 }"
985          tests => [ [ [a,b], { a=>1, b=>2 } ] ]
986
987       Whitespace is mostly optional, but there is one exception. An item must
988       end with some kind of delimiter, so the following will fail:
989
990          tests => "[a b][c d]"
991
992       The first element (the list ref [a b]) must be separated from the
993       second element by the delimiter (which is whitespace in this case), so
994       it must be written as:
995
996          tests => "[a b] [c d]"
997
998       As already demonstrated, hashrefs and listrefs may be included and
999       nested. Elements may also be included inside parens, but this is
1000       optional since all arguments and expected results are already treated
1001       as lists, so the following are equivalent:
1002
1003          tests => "a b c"
1004          tests => "(a b) c"
1005
1006       Although parens are optional, they may make things more readable, and
1007       allow you to use something other than whitespace as the delimiter.
1008       Since parens are actually ignored, a string '()' is also ignored, so do
1009       not use empty parentheses.
1010
1011       If the character immediately following the opening paren, brace, or
1012       bracket is a punctuation mark, then it is used as the delimiter instead
1013       of whitespace. For example, the following are all equivalent:
1014
1015          [ a b c ]
1016          [a b c]
1017          [, a,b,c ]
1018          [, a, b, c ]
1019
1020       A delimiter is a single character, and the following may not be used as
1021       a delimiter:
1022
1023          any opening/closing characters () [] {}
1024          single or double quotes
1025          alphanumeric characters
1026          underscore
1027
1028       Whitespace (including newlines) around the delimiter is ignored, so the
1029       following is valid:
1030
1031          [, a,
1032             b,
1033             c ]
1034
1035       Two delimiters next to each other or a trailing delimiter produce an
1036       empty string.
1037
1038          "(,a,b,)" => (a, b, '')
1039          "(,a,,b)" => (a, '', b)
1040
1041       Hashrefs may be specified by braces and the following are equivalent:
1042
1043          { a 1 b 2 }
1044          {, a,1,b,2 }
1045          {, a,1,b,2, }
1046
1047       Note that a trailing delimiter is ignored if there are already an even
1048       number of elements, or an empty string otherwise.
1049
1050       Nested structures are allowed:
1051
1052          "[ [1 2] [3 4] ]"
1053
1054       For example,
1055
1056          $o->tests(
1057             func     => &func,
1058             tests    => "a [ b c ] { d 1 e 2 } => x y"
1059          );
1060
1061       is equivalent to:
1062
1063          $o->tests(
1064             func     => &func,
1065             tests    => [ [a, [b,c], {d=>1,e=>2}] ],
1066             results  => [ [x,y] ],
1067          );
1068
1069       Any single value can be surrounded by single or double quotes in order
1070       to include the delimiter. So:
1071
1072          "(, a,'b,c',e )"
1073
1074       is equivalent to:
1075
1076          "( a b,c e )"
1077
1078       Any single value can be the string '__undef__' which will be turned
1079       into an actual undef. If the value is '__blank__' it is turned into an
1080       empty string (''), though it can also be specified as '' directly. Any
1081       value can have an embedded newline by including a __nl__ in the value,
1082       but the value must be written on a single line.
1083
1084       Expected results are separated from arguments by ' => '.
1085

TEST::INTER VARIABLES

1087       To summarize the information above, the following variables are used by
1088       Test::Inter.  Each variable can be set in two different ways: as an
1089       environment variable and as a perl variable in the main namespace.
1090
1091       For example, the TI_END variable can be set as:
1092
1093          $::TI_END
1094          $ENV{TI_END}
1095
1096       The following variables can be used to define which tests are run:
1097
1098       TI_START
1099           Set this to define the test you want to start with.
1100
1101           Example: If you have a perl test script and you want to start
1102           running it at test 12, run the following shell commands:
1103
1104              TI_START=12
1105              ./my_test_script.t
1106
1107       TI_END
1108           Set this to define the test you want to end with.
1109
1110       TI_TESTNUM
1111           Set this to run only a single test
1112
1113       There is also a variable TI_NUM (available only as $::TI_NUM) which is
1114       set automatically by Test::Inter to be the test currently being run.
1115
1116       The following variables control what is output from the tests, and how
1117       it is formatted:
1118
1119       TI_QUIET
1120           How verbose the test script is.  Values are 0 (most verbose) to 2
1121           (least verbose).
1122
1123       TI_MODE
1124           How the output is formatted.  Values are 'inter' (interactive mode)
1125           or 'test' (test suite mode).  Interactive mode is easier to read.
1126           Test mode is for running as part of a test suite.
1127
1128       TI_WIDTH
1129           The width of the terminal.
1130
1131       The following variables control how some tests are run:
1132
1133       TI_NOCLEAN
1134           When running a file test, the temporary output file will not be
1135           removed if this is set.
1136

HISTORY

1138       The history of this module dates back to 1996 when I needed to write a
1139       test suite for my Date::Manip module. At that time, none of the Test::*
1140       modules currently available in CPAN existed (the earliest ones didn't
1141       come along until 1998), so I was left completely on my own writing my
1142       test scripts.
1143
1144       I wrote a very basic version of my test framework which allowed me to
1145       write all of the tests as a string, it would parse the string, count
1146       the tests, and then run them.
1147
1148       Over the years, the functionality I wanted grew, and periodically, I'd
1149       go back and reexamine other Test frameworks (primarily Test::More) to
1150       see if I could replace my framework with an existing module... and I've
1151       always found them wanting, and chosen to extend my existing framework
1152       instead.
1153
1154       As I've written other modules, I've wanted to use the framework in them
1155       too, so I've always just copied it in, but this is obviously tedious
1156       and error prone. I'm not sure why it took me so long... but in 2010, I
1157       finally decided it was time to rework the framework in a module form.
1158
1159       I loosely based my module on Test::More. I like the functionality of
1160       that module, and wanted most of it (and I plan on adding more in future
1161       versions).  So this module uses some similar syntax to Test::More
1162       (though it allows a great deal more flexibility in how the tests are
1163       specified).
1164
1165       One thing to note is that I may have been able to write this module as
1166       an extension to Test::More, but after looking into that possibility, I
1167       decided that it would be faster to not do that. I did "borrow" a couple
1168       of routines from it (though they've been modified quite heavily) as a
1169       starting point for a few of the functions in this module, and I thank
1170       the authors of Test::More for their work.
1171

KNOWN BUGS AND LIMITATIONS

1173       None known.
1174

SEE ALSO

1176       Test::More - the 'industry standard' of perl test frameworks
1177

BUGS AND QUESTIONS

1179       If you find a bug in Test::Inter, there are three ways to send it to
1180       me.  Any of them are fine, so use the method that is easiest for you.
1181
1182       Direct email
1183           You are welcome to send it directly to me by email.  The email
1184           address to use is:  sbeck@cpan.org.
1185
1186       CPAN Bug Tracking
1187           You can submit it using the CPAN tracking too.  This can be done at
1188           the following URL:
1189
1190           <http://rt.cpan.org/Public/Dist/Display.html?Name=Test-Inter>
1191
1192       GitHub
1193           You can submit it as an issue on GitHub.  This can be done at the
1194           following URL:
1195
1196           <https://github.com/SBECK-github/Test-Inter>
1197
1198       Please do not use other means to report bugs (such as forums for a
1199       specific OS or Linux distribution) as it is impossible for me to keep
1200       up with all of them.
1201
1202       When filing a bug report, please include the following information:
1203
1204       Test::Inter version
1205           Please include the version of Test::Inter you are using.  You can
1206           get this by using the script:
1207
1208              use Test::Inter;
1209              print $Test::Inter::VERSION,"\n";
1210
1211       If you want to report missing or incorrect codes, you must be running
1212       the most recent version of Test::Inter.
1213
1214       If you find any problems with the documentation (errors, typos, or
1215       items that are not clear), please send them to me. I welcome any
1216       suggestions that will allow me to improve the documentation.
1217

LICENSE

1219       This script is free software; you can redistribute it and/or modify it
1220       under the same terms as Perl itself.
1221

AUTHOR

1223       Sullivan Beck (sbeck@cpan.org)
1224
1225
1226
1227perl v5.30.1                      2020-01-30                    Test::Inter(3)
Impressum