1Test::Inter(3) User Contributed Perl Documentation Test::Inter(3)
2
6 Test::Inter - framework for more readable interactive test scripts
7
9 This is another framework for writing test scripts. Much of the syntax
10 is loosely inspired by Test::More, and Test::Inter has most of it's
11 functionality, but it is not a drop-in replacement.
12 Test::More (and other existing test frameworks) suffer from two
14 weaknesses, both of which have prevented me from ever using them:
15 None offer the ability to access specific tests in
17 a reasonably interactive fashion, primarily for
18 debugging purposes
19 None offer the ability to write the tests in
21 whatever format would make the tests the most
22 readable
23 The way I write and use test scripts, existing Test::* modules are not
25 nearly as useful as they could be.
26 Test scripts written using Test::More work fine when running as part of
28 the test suite, but debugging an individual test requires extra steps,
29 and the tests themselves are not as readable as they should be.
30
32 One requirement that I have of a test framework is the ability to
33 interact with it.
34 I do most of my debugging using test scripts. When I find a bug, I
36 write a test case for it (typically by adding it to an existing test
37 script) and then debug it using the test script. Then I leave the test
38 there to ensure that the bug won't come back (hopefully).
39 Since I use test scripts in a very interactive way (often in the
41 debugger), I want to be able to do the following trivially:
42 Easy access to a specific test or tests
44 I'd like to be able to run only a single test, or a subset of
45 tests.
46 Easily set breakpoints in the debugger
48 Setting a breakpoint in the debugger to run up to the start of the
49 Nth test is one of the most common tasks I want to do when I'm
50 debugging a failed test.
51 To illustrate the first point, in Test::More, a series of tests might
53 be specified in a test script as shown in the following example (line
54 numbers added for convenience):
55 ...
57 100: # test 1
59 101: $result = func("apples","bushels");
60 102: is($result, "enough");
61 103:
62 104: # test 2
63 105: $result = func("grapefruit","tons");
64 106: is($result, "enough");
65 107:
66 108: # test 3
67 109: $result = func("oranges","boatloads");
68 110: is($result, "insufficient");
69 111:
70 112: # tests 4-6
71 113: foreach my $arg (qw(pears plums pineapple)) {
72 114: $result = func($arg,"boxes");
73 115: is($result, "enough");
74 116: }
75 ...
77 Say you ran the test suite, and test 3 failed. To debug it you have to
79 open up the test script, find the 3rd test, and set the appropriate
80 breakpoint. In this case, you'll want to break at line 109.
81 None of these steps are impossible of course, but it will take some
83 time to get it right. It becomes harder when there are lots of tests
84 (imagine that you want to test the 117th test instead of the 3rd test)
85 or when tests are wrapped up in loops, embedded in subroutines, or
86 other similar situations.
87 As an example, what if it's the 5th test that fails in the example
89 above. Now the break point will be a conditional one, so you have to
90 figure out not only the line, but the condition the appropriate state
91 during that test. In this case, you need to stop at line 114 when $arg
92 is 'plums'.
93 Wouldn't it be far better to set a break point in func when the Nth
95 test is reached? With Test::Inter, you can.
96 So for the above script, the debugger commands that you would use to
98 debug the 3rd test are:
99 Test::More : b 109
101 Test::Inter: b func ($::TI_NUM==3)
102 and the 5th test are:
104 Test::More : b 114 ($arg eq 'plums')
106 Test::Inter: b func ($::TI_NUM==5)
107 It would also be nice to be able to skip the first two tests... perhaps
109 they take a long time to run, and I want to get right to work on test
110 3. You can do this easily too by setting the $::TI_START variable.
111 There are some other variables that can be used to specify which test
113 or tests to run described in the "TEST::INTER VARIABLES" section below.
114 The other thing I want to do when I run the test scripts interactively
116 is to see more information which will assist in debugging a failed
117 test.
118 This can be controlled with variables such as TI_QUIET, TI_MODE, and
120 TI_WIDTH described below in the "TEST::INTER VARIABLES" section.
121
123 The other feature that I wanted in a test suite is the ability to
124 define the tests in a format that is natural and readable FOR THE
125 TESTS. In almost every case, it is best to think of a test script as
126 consisting of two separate parts: a script part, and a test part.
127 The script part of a test script is the least important part! It's
129 usually fairly trivial, rarely needs to be changed, and is not the
130 focus of the test script.
131 The tests part of the script IS the important part, and these should be
133 expressed in a form that is natural to them, easy to maintain, easy to
134 read, and easy to modify, and none of these should involve modifying
135 the script portion of the test script in general. Because the content
136 of the tests is the important part of the script, the emphasis should
137 be in making them more readable, even at the expense of the script
138 portion. As a general rule, if the script portion of the test script
139 obscures the tests in any way, it's not written correctly!
140 The solution to this is well understood, and is common to many other
142 systems where you are mixing two "languages". The task of correctly
143 specifying both the tests and the test script is virtually identical to
144 the task of creating a PHP script which consists of a mixture of PHP
145 and HTML, or the task of creating a template file using some templating
146 system where the file consists of a mixture of text to be displayed and
147 templating commands. It is well understood in each of these cases that
148 the more the two "languages" are interwoven, the less readable both
149 are, and the harder it is to maintain. The more you are able to
150 separate the two, the easier both are to read and maintain.
151 As often as possible, I want the tests to be written in some sort of
153 text format which can be easily viewed and modified (usually as a
154 simple table) with no perl commands interspersed. I want to the freedom
155 to define the tests in one section (a long string, the DATA section, or
156 even in a separate file) which is easily readable. This may introduce
157 the necessity of parsing it, but it makes it significantly easier to
158 maintain the tests.
159 This flexibility makes it much easier to read the tests (as opposed to
161 the script) which is the fundamental content of a test script.
162 Looking again at the example test script, you can see that there is far
164 too much perl interspersed with the tests.
165 It's difficult to read the tests individually in this script because
167 there is too much perl code among them, and virtually impossible to
168 look at them as a whole.
169 It is true that looking at this particular example, it is very
171 simple... but the script ISN'T the content you're interested in (and
172 bear in mind that many test scripts are nowhere near this simple). The
173 REAL content of this script are the tests, which consist of the
174 function arguments and the expected result. Although it's not
175 impossible to see each of these in the script above, it's not in a
176 format that is conducive to studying the tests, and especially not for
177 examining the list of tests as a whole.
178 Now, look at an alternate way of specifying the tests using this
180 module:
181 $tests = "
183 apples bushels => enough
185 grapefruit tons => enough
187 oranges boatloads => insufficient
189 pears boxes => enough
191 plums boxes => enough
193 pineapple boxes => enough
195 ";
197 $o->tests(tests => $tests,
199 func => \&func);
200 Here, it's easy to see the list of tests, and adding additional tests
202 is a breeze.
203
205 This module supports a number of methods for defining tests, so you can
206 use whichever one is most convenient (including methods that are
207 identical to Test::More if that really is the best method).
208 Every test may have several pieces of information:
210 A name
212 Every test is automatically assigned a number, but it may be useful
213 to specify a name of a test (which is actually a short description
214 of the test). Whenever a test result is reported, the name will be
215 given (if one was specified).
216 The name may not have a '#' in it.
218 The name is completely optional, but makes the results more
220 readable.
221 An expected result
223 In order to test something, you need to know what result was
224 expected (or in some cases, what result was NOT expected).
225 A function and arguments OR a result
227 You also need to know the results that you're comparing to the
228 expected results.
229 This can be obtained by simply working with a set of results, or a
231 function name and a set of arguments to pass to it.
232 Conditions
234 It is useful to be able to specify state information at the start
235 of the test suite (for example, to see if certain features are
236 available), and some tests may only run if those conditions are
237 met.
238 If no conditions are set for a test, it will always run.
240 Todo tests
242 Some tests may be marked as 'todo' tests. These are test which are
243 allowed to fail (meaning that they have been put in place for an
244 as-yet unimplemented feature). Since it is expected that the test
245 will fail, the test suite will still pass, even if these tests
246 fail.
247 The tests will still run and if they pass, a message is issued
249 saying that the feature is now implemented, and the tests should be
250 graduated to non-todo state.
251
253 new
254 $o = new Test::Inter [$name] [%options];
255 This creates a new test framework. There are several options which
257 may be used to specify which tests are run, how they are run, and
258 what output is given.
259 The entire test script can be named by passing in $name.
261 All options can be set in four different ways.
263 First, you can pass in a hash of OPT = VAL> pairs in the new
265 method. So, to set the start option, the %options) hash would
266 contain:
267 start => VALUE
269 Second, you can set an environment variable. This will override
271 any value passed in the first way. The environment variable is
272 named TI_XXX where XXX is the fully capitalized option. So:
273 $ENV{TI_START} = VALUE
275 The third method, which overrides the previous two, is to set a
277 global variable. It is also named TI_XXX in the main namespace, so
278 to set it this way, set:
279 $::TI_START = VALUE
281 The final method is to call one of the methods below and these
283 override all other methods.
284 Each of the allowed options are described below in the following
286 base methods:
287 start
289 end
290 testnum
291 plan
292 abort
293 quiet
294 mode
295 skip_all
296 width
297 use_lib
298 version
300 $o->version();
301 Returns the version of the module.
303 encoding
305 $o->encoding($encoding);
306 $encoding is any value that can be passed as an encoding to perl's
308 Encode::decode function.
309 Use this if your test strings contain characters in other
311 encodings.
312 start
314 $o = new Test::Inter 'start' => $N;
315 $o->start($N)
316 To define which test you want to start with, set the start option
318 as described in the new method above.
319 When the start test is defined, most tests numbered less than N are
321 completely ignored. If the tests are being run quietly (see the
322 quiet method below), nothing is printed out for these tests.
323 Otherwise, a skip message is printed out.
324 One class of tests IS still executed. Tests run using the
326 require_ok or use_ok methods (to test the loading of modules) are
327 still run.
328 If no value (or a value of 0) is used, tests run from the first
330 test.
331 end
333 $o = new Test::Inter 'end' => $M;
334 $o->end($M);
335 To define which test you want to end with, set the end option as
337 described in the new method above.
338 When the end test is defined, all tests numbered more than M are
340 completely ignored. If the tests are being run quietly (see the
341 quiet method below), nothing is printed out for these tests.
342 Otherwise, a skip message is printed out.
343 If no value is given, it defaults to 0 (which means that all
345 remaining tests are run).
346 testnum
348 $o = new Test::Inter 'testnum' => $N;
349 $o->testnum($N);
350 To run only a single test, set the testnum option as described in
352 the new method above.
353 It is equivalent to setting both the start and end tests to $N.
355 plan
357 done_testing
358 $o = new Test::Inter 'plan' => $N;
359 $o->plan($n);
360 $o->done_testing();
362 $o->done_testing($n);
363 The TAP API (the 'language' used to run a sequence of tests and see
365 which ones failed and which ones passed) requires a statement of
366 the number of tests that are expected to run.
367 This statement can appear at the start of the test suite, or at the
369 end.
370 If you know in advance how many tests should run in the test
372 script, you can set the plan option as described in the new method
373 above to the number of tests.
374 If you know how many tests should run at the end of the test
376 script, you can pass in a non-zero integer to the done_testing
377 method.
378 Frequently, you don't really care how many tests are in the script
380 (especially if new tests are added on a regular basis). In this
381 case, you still need to include a statement that says that the
382 number of tests expected is however many were run. To do this, call
383 the done_testing method with no argument.
384 NOTE: if the plan method is used, it MUST be used before any tests
386 are run (including those that test the loading of modules). If the
387 done_testing method is used, it MUST be called after all tests are
388 run. You must specify a plan or use a done_testing statement, but
389 you cannot do both.
390 It is NOT strictly required to set a plan if the script is only run
392 interactively, so if for some reason this module is used for test
393 scripts which are not part of a standard perl test suite, the plan
394 and done_testing statements are optional. As a matter of fact, the
395 script will run just fine without them... but a perl installer will
396 report a failure in the test suite.
397 abort
399 $o = new Test::Inter 'abort' => 0/1/2;
400 $o->abort(0/1/2);
401 To define how you want a failure to be treated, set the abort
403 option as described in the new method above. The abort option can
404 take a value of 0, 1, or 2.
405 If this is set to 1, the test script will run unmodified until a
407 test fails. At that point, all remaining tests will be skipped. If
408 it is set to 2, the test script will run until a test fails at
409 which point it will exit with an error code of 1. With a value of
410 0, failed tests will be reported, but the script will continue.
411 In both cases, todo tests will NOT trigger the abort behavior.
413 quiet
415 $o = new Test::Inter 'quiet' => 0/1/2;
416 $o->quiet(0/1/2);
417 To define how you want failures to be reported, set the quiet
419 option as described in the new method above. The quiet option can
420 take a value of 0, 1, or 2.
421 If this is set to 0 (the default), all information will be printed
423 out. If it is set to 1, some optional information will not be
424 printed. If it is set to 2, all optional information will not be
425 printed.
426 mode
428 $o = new Test::Inter 'mode' => MODE;
429 $o->mode(MODE);
430 Test::Inter scripts can be run in either an interactive mode, or as
432 part of a test suite with different behaviors. To select the mode,
433 set the mode option as described in the new method above. The mode
434 option can take a value of 'inter' or 'test'.
435 When run in test mode, it prints out the results using the TAP
437 grammar (i.e. 'ok 1', 'not ok 3', etc.).
438 When run in interactive mode, it prints out results in a more human
440 readable format.
441 width
443 $o = new Test::Inter 'width' => WIDTH;
444 $o->width(WIDTH);
445 The width option can be set as described in the new method above.
447 WIDTH is the width of the terminal (for printing out failed test
449 information). It defaults to 80, but it can be set to any width
450 (and lines longer then this are truncated). If WIDTH is set to 0,
451 no truncation is done.
452 use_lib
454 $o = new Test::Inter 'use_lib' => VALUE;
455 $o->use_lib(VALUE);
456 $o->use_lib();
457 By default, the library included in the module distribution will be
459 added to the search path for modules, so a 'use MODULE' line should
460 find the version stored in this module distribution.
461 If VALUE is set to 'off', the search path will not be modified
463 automatically.
464 You may add the library path at a later time by calling:
466 $o->use_lib('on');
468 $o->use_lib();
469 Note: both calls must be used. The first sets the option, the
471 second actually modifies the search path.
472 skip_all
474 $o = new Test::Inter 'skip_all' => REASON;
475 $o->skip_all(REASON);
476 The skip_all option can be set as described in the new method
478 above.
479 If this is set, the entire test script will be skipped for the
481 reason given. This must be done before any test is run, and before
482 any plan number is set.
483 The skip_all can also be called at any point during the script
485 (i.e. after tests have been run). In this case, all remaining
486 scripts will be skipped.
487 $o->skip_all(REASON,FEATURE,FEATURE,...);
489 $o->skip_all('',FEATURE,FEATURE,...);
490 This will skip all tests (or all remaining tests) unless all
492 <FEATURE>s are available. REASON can be entered as an empty string
493 and the reason the tests are skipped will be a message about the
494 missing feature.
495 feature
497 $o->feature($feature,$val);
498 This defines a feature. If $val is non-zero, the feature is
500 available. Otherwise it is not.
501 diag
503 note
504 $o->diag($message);
505 $o->note($message);
506 Both of these print an optional message. Messages printed with the
508 "note" method are always optional and will be omitted if the quiet
509 option is set to 1 or 2. Messages printed with the "diag" method
510 are optional and will not be printed if the quiet option is set to
511 2, but they will be printed if the quiet method is set to 1.
512 testdir
514 $o->testdir();
515 $o->testdir('mod');
516 $o->testdir('lib');
517 Occasionally, it may be necessary to know the directory where
519 Test::Inter gets some of it's information. By default, the
520 directory containing the tests will be returned, but if the
521 optional argument 'mod' is included, it will return the path to the
522 module distribution (which should include both a lib and t
523 subdirerctory). If the argument 'lib' is included, it will return
524 the directory where the libraries are stored.
525
527 Test scripts can load other modules (using either the perl "use" or
528 "require" commands). There are three different modes for doing this
529 which determine how this is done.
530 required
532 By default, this is used to test for a module that is required for
533 all tests in the test script.
534 Loading the module is treated as an actual test in the test suite.
536 The test is to determine whether the module is available and can be
537 loaded. If it can be loaded, it is, and it is reported as a
538 successful test. If it cannot be loaded, it is reported as a failed
539 test.
540 In the result of a failed test, all remaining tests will be skipped
542 automatically (except for other tests which load modules).
543 feature
545 In feature mode, loading the module is not treated as a test (i.e.
546 it will not print out an 'ok' or 'not ok' line. Instead, it will
547 set a feature (named the same as the module) which can be used to
548 determine whether other tests should run or not.
549 forbid
551 In a few very rare cases, we may want to test for a module but
552 expect that it not be present. This is the exact opposite of the
553 required mode.
554 Successfully loading the module is treated as a test failure. In
556 the event of a failure, all remaining tests will be skipped.
557 The methods available are:
559 require_ok
561 $o->require_ok($module [,$mode]);
562 This is used to load a module using the perl "require" function. If
564 $mode is not passed in, the default mode (required) is used to test
565 the existence of the module.
566 If $mode is passed in, it must be either the string 'forbid' or
568 'feature'.
569 If $mode is 'feature', a feature named $module is set if the module
571 was able to be loaded.
572 use_ok
574 $o->use_ok(@args [,$mode]);
575 This is used to load a module with "use", or check a perl version.
577 BEGIN { $o->use_ok('5.010'); }
579 BEGIN { $o->use_ok('Some::Module'); }
580 BEGIN { $o->use_ok('Some::Module',2.05); }
581 BEGIN { $o->use_ok('Some::Module','foo','bar'); }
582 BEGIN { $o->use_ok('Some::Module',2.05,'foo','bar'); }
583 are the same as:
585 use 5.010;
587 use Some::Module;
588 use Some::Module 2.05;
589 use Some::Module qw(foo bar);
590 use Some::Module 2.05 qw(foo bar);
591 Putting the use_ok call in a BEGIN block allows the functions to be
593 imported at compile-time and prototypes are properly honored.
594 You'll also need to load the Test::Inter module, and create the
595 object in a BEGIN block.
596 $mode acts the same as in the require_ok method.
598
600 There are several methods for running tests. The ok, is, and isnt
601 methods are included for those already comfortable with Test::More and
602 wishing to stick with the same format of test script. The tests method
603 is the suggested method though since it makes use of the full power of
604 this module.
605 ok
607 $o->ok(TESTS);
608 A test run with ok looks at a result, and if it evaluates to 0 (or
610 false), it fails. If it evaluates to non-zero (or true), it passes.
611 These tests do not require you to specify the expected results. If
613 expected results are given, they will be compared against the
614 result received, and if they differ, a diagnostic message will be
615 printed, but the test will still succeed or fail based only on the
616 actual result produced.
617 These tests require a single result and either zero or one expected
619 results.
620 To run a single test, use any of the following:
622 $o->ok(); # always succeeds
624 $o->ok($result);
626 $o->ok($result,$name);
627 $o->ok($result,$expected,$name);
628 $o->ok(\&func);
630 $o->ok(\&func,$name);
631 $o->ok(\&func,$expected,$name);
632 $o->ok(\&func,\@args);
634 $o->ok(\&func,\@args,$name);
635 $o->ok(\&func,\@args,$expected,$name);
636 If $result is a scalar, the test passes if $result is true. If
638 $result is a list reference, the test succeeds if the list contains
639 any defined values. If $result is a hash reference, the test
640 succeeds if the hash contains any key with a value that is not
641 "undef".
642 If "\&func" and "\@args" are passed in, then $result is generated
644 by passing @args to &func and behaves identically to the calls
645 where $result is passed in. If "\&func" is passed in but no
646 arguments, the function takes no arguments, but still produces a
647 result.
648 If an expected value is passed in and the result does not match it,
650 a diagnostic warning will be printed, even if the test passes.
651 is
653 isnt
654 $o->is(TESTS);
655 $o->isnt(TESTS);
656 A test run with is looks at a result and tests to see if it is
658 identical to an expected result. If it is, the test passes.
659 Otherwise it fails. In the case of a failure, a diagnostic message
660 will show what result was actually obtained and what was expected.
661 A test run with isnt looks at a result and tests to see if the
663 result obtained is different than an expected result. If it is
664 different, the test passes. Otherwise it fails.
665 The is method can be called in any of the following ways:
667 $o->is($result,$expected);
669 $o->is($result,$expected,$name);
670 $o->is(\&func,$expected);
672 $o->is(\&func,$expected,$name);
673 $o->is(\&func,\@args,$expected);
675 $o->is(\&func,\@args,$expected,$name);
676 The isnt method can be called in exactly the same way.
678 As with the ok method, the result can be a scalar, hashref, or
680 listref. If it is a hashref or listref, the entire structure must
681 match the expected value.
682 tests
684 $o->tests($opt=>$val, $opt=>$val, ...)
685 The options available are described in the following section.
687 file
689 $o->file($func,$input,$outputdir,$expected,$name [,@args]);
690 Sometimes it may be easiest to store the input, output, and
692 expected output from a test in a text file. In this case, each line
693 of output will be treated as a single test, so the output and
694 expected output must match up exactly.
695 $func is a reference to a function which will produce a temporary
697 output file.
698 If $input is specified, it is the name of the input file. If it is
700 empty, no input file will be used. The input file can be fully
701 specified, or it can be relative to the test directory.
702 If $outputdir is passed in, it is the directory where the output
704 file will be written. It can be fully specified, or relative to
705 the test directory. If $outputdir is left blank, the temporary
706 file will be written to the test directory.
707 $expected is the name of a file which contains the expected output.
709 It can be fully specified, or it will be checked for in the test
710 directory.
711 $name is the name of this series of tests.
713 @args are extra arguments to pass to the test function.
715 The function will be called with the arguments:
717 &$func( [$input,] $output,@args);
719 $input is only passed in if it was passed in to this method. If no
721 input file is specified, nothing will be passed to the function.
722 $output is the name of a temporary file where the output will be
724 written to.
725
727 It is expected that most tests (except for those that load a module)
728 will be run using the tests method called as:
729 $o->tests(%options);
731 The following options are available:
733 name
735 name => NAME
736 This sets the name of this set of tests. All tests will be given
738 the same name.
739 tests
741 func
742 expected
743 In order to specify a series of tests, you have to specify either a
744 function and a list of arguments, or a list of results.
745 Specifying the function and list of arguments can be done using the
747 pair:
748 func => \&FUNCTION
750 tests => TESTS
751 If the func option is not set, tests contains a list of results.
753 A list of expected results may also be given. They can be included
755 in the
756 tests => TESTS
758 option or included separately as:
760 expected => RESULTS
762 The way to specify these are covered in the next section SPECIFYING
764 THE TESTS.
765 feature
767 disable
768 feature => [FEATURE1, FEATURE2, ...]
769 disable => [FEATURE1, FEATURE2, ...]
771 The default set of tests to run is determined using the start, end,
773 and skip_all methods discussed above. Using those methods, a list
774 of tests is obtained, and it is expected that these will run.
775 The feature and disable options modify the list.
777 If the feature option is included, the tests given in this call
779 will only run if ALL of the features listed are available.
780 If the disable option is included, the tests will be run unless ANY
782 of the features listed are available.
783 skip
785 skip => REASON
786 Skip these tests for the reason given.
788 todo
790 todo => 0/1
791 Setting this to 1 says that these tests are allowed to fail. They
793 represent a feature that is not yet implemented.
794 If the tests succeed, a message will be printed notifying the
796 developer that the tests are now ready to promote to actual use.
797
799 A series of tests can be specified in two different ways. The tests can
800 be written in a very simple string format, or stored as a list.
801 Demonstrating how this can be done is best done by example, so let's
803 say that there is a function (func) which takes two arguments, and
804 returns a single value. Let's say that the expected output (and the
805 actual output) from 3 different sets of arguments is:
806 Input Expected Output Actual Output
808 ----- --------------- -------------
809 1,2 a a
810 3,4 b x
811 5,6 c c
812 (so in this case, the first and third tests pass, but the 2nd one will
814 fail).
815 Specifying these tests as lists could be done as:
817 $o->tests(
819 func => &func,
820 tests => [ [1,2], [3,4], [5,6] ],
821 expected => [ [a], [b], [c] ],
822 );
823 Here, the tests are stored as a list, and each element in the list is a
825 listref containing the set of arguments.
826 If the func option is not passed in, the tests option is set to a list
828 of results to compare with the expected results, so the following is
829 equivalent to the above:
830 $o->tests(
832 tests => [ [a], [x], [c] ],
833 expected => [ [a], [b], [c] ],
834 );
835 If an argument (or actual result) or an expected result is only a
837 single value, it can be entered as a scalar instead of a list ref, so
838 the following is also equivalent:
839 $o->tests(
841 func => &func,
842 tests => [ [1,2], [3,4], [5,6] ],
843 expected => [ a, b, [c] ],
844 );
845 The only exception to this is if the single value is itself a list
847 reference. In this case it MUST be included as a reference. In other
848 words, if you have a single test, and the expected value for this test
849 is a list reference, it must be passed in as:
850 expected => [ [ \@r ] ]
852 NOT as:
854 expected => [ \@r ]
856 Passing in a set of expected results is optional. If none are passed
858 in, the tests are treated as if they had been passed to the ok method
859 (i.e. if they return something true, they pass, otherwise they fail).
860 The second way to specify tests is as a string. The string is a multi-
862 line string with each tests being separate from the next test by a
863 blank line. Comments (lines which begin with '#') are allowed, and are
864 ignored. Whitespace at the start and end of the line is ignored.
865 The string may contain the results directly, or results may be passed
867 in separately. For example, the following all give the same sets of
868 tests as the example above:
869 $o->tests(
871 func => &func,
872 tests => "
873 # Test 1
874 1 2 => a
875 # Test 2
877 3 4 => b
878 5 6 => c
880 ",
881 );
882 $o->tests(
884 func => &func,
885 tests => "
886 1 2
887 3 4
889 5 6
891 ",
892 expected => [ [a], [b], [c] ]
893 );
894 $o->tests(
896 func => &func,
897 tests => [ [1,2], [3,4], [5,6] ],
898 expected => "
899 a
900 b
902 c
904 ",
905 );
906 $o->tests(
908 func => &func,
909 tests => "
910 1 2
911 3 4
913 5 6
915 ",
916 expected => "
917 a
918 b
920 c
922 ",
923 );
924 The expected results may also consist of only a single set of results
926 (in this case, it must be passed in as a listref). In this case, all of
927 the tests are expected to have the same results.
928 So, the following are equivalent:
930 $o->tests(
932 func => &func,
933 tests => "
934 1 2 => a b
935 3 4 => a b
937 5 6 => a b
939 ",
940 );
941 $o->tests(
943 func => &func,
944 tests => "
945 1 2
946 3 4
948 5 6
950 ",
951 expected => [ [a, b] ],
952 );
953 $o->tests(
955 func => &func,
956 tests => "
957 1 2
958 3 4
960 5 6
962 ",
963 expected => "a b",
964 );
965 The number of expected values must either be 1 (i.e. all of the tests
967 are expected to produce the same value) or exactly the same number as
968 the number of tests.
969 The parser is actually quite powerful, and can handle multi-line tests,
971 quoted strings, and nested data structures.
972 The test may be split across any number of lines, provided there is not
974 a completely blank line (which signals the end of the test), so the
975 following are equivalent:
976 tests => "a b c",
978 tests => "a b
979 c",
980 Arguments (or expected results) may include data structures. For
982 example, the following are equivalent:
983 tests => "[ a b ] { a 1 b 2 }"
985 tests => [ [ [a,b], { a=>1, b=>2 } ] ]
986 Whitespace is mostly optional, but there is one exception. An item must
988 end with some kind of delimiter, so the following will fail:
989 tests => "[a b][c d]"
991 The first element (the list ref [a b]) must be separated from the
993 second element by the delimiter (which is whitespace in this case), so
994 it must be written as:
995 tests => "[a b] [c d]"
997 As already demonstrated, hashrefs and listrefs may be included and
999 nested. Elements may also be included inside parens, but this is
1000 optional since all arguments and expected results are already treated
1001 as lists, so the following are equivalent:
1002 tests => "a b c"
1004 tests => "(a b) c"
1005 Although parens are optional, they may make things more readable, and
1007 allow you to use something other than whitespace as the delimiter.
1008 Since parens are actually ignored, a string '()' is also ignored, so do
1009 not use empty parentheses.
1010 If the character immediately following the opening paren, brace, or
1012 bracket is a punctuation mark, then it is used as the delimiter instead
1013 of whitespace. For example, the following are all equivalent:
1014 [ a b c ]
1016 [a b c]
1017 [, a,b,c ]
1018 [, a, b, c ]
1019 A delimiter is a single character, and the following may not be used as
1021 a delimiter:
1022 any opening/closing characters () [] {}
1024 single or double quotes
1025 alphanumeric characters
1026 underscore
1027 Whitespace (including newlines) around the delimiter is ignored, so the
1029 following is valid:
1030 [, a,
1032 b,
1033 c ]
1034 Two delimiters next to each other or a trailing delimiter produce an
1036 empty string.
1037 "(,a,b,)" => (a, b, '')
1039 "(,a,,b)" => (a, '', b)
1040 Hashrefs may be specified by braces and the following are equivalent:
1042 { a 1 b 2 }
1044 {, a,1,b,2 }
1045 {, a,1,b,2, }
1046 Note that a trailing delimiter is ignored if there are already an even
1048 number of elements, or an empty string otherwise.
1049 Nested structures are allowed:
1051 "[ [1 2] [3 4] ]"
1053 For example,
1055 $o->tests(
1057 func => &func,
1058 tests => "a [ b c ] { d 1 e 2 } => x y"
1059 );
1060 is equivalent to:
1062 $o->tests(
1064 func => &func,
1065 tests => [ [a, [b,c], {d=>1,e=>2}] ],
1066 results => [ [x,y] ],
1067 );
1068 Any single value can be surrounded by single or double quotes in order
1070 to include the delimiter. So:
1071 "(, a,'b,c',e )"
1073 is equivalent to:
1075 "( a b,c e )"
1077 Any single value can be the string '__undef__' which will be turned
1079 into an actual undef. If the value is '__blank__' it is turned into an
1080 empty string (''), though it can also be specified as '' directly. Any
1081 value can have an embedded newline by including a __nl__ in the value,
1082 but the value must be written on a single line.
1083 Expected results are separated from arguments by ' => '.
1085
1087 To summarize the information above, the following variables are used by
1088 Test::Inter. Each variable can be set in two different ways: as an
1089 environment variable and as a perl variable in the main namespace.
1090 For example, the TI_END variable can be set as:
1092 $::TI_END
1094 $ENV{TI_END}
1095 The following variables can be used to define which tests are run:
1097 TI_START
1099 Set this to define the test you want to start with.
1100 Example: If you have a perl test script and you want to start
1102 running it at test 12, run the following shell commands:
1103 TI_START=12
1105 ./my_test_script.t
1106 TI_END
1108 Set this to define the test you want to end with.
1109 TI_TESTNUM
1111 Set this to run only a single test
1112 There is also a variable TI_NUM (available only as $::TI_NUM) which is
1114 set automatically by Test::Inter to be the test currently being run.
1115 The following variables control what is output from the tests, and how
1117 it is formatted:
1118 TI_QUIET
1120 How verbose the test script is. Values are 0 (most verbose) to 2
1121 (least verbose).
1122 TI_MODE
1124 How the output is formatted. Values are 'inter' (interactive mode)
1125 or 'test' (test suite mode). Interactive mode is easier to read.
1126 Test mode is for running as part of a test suite.
1127 TI_WIDTH
1129 The width of the terminal.
1130 The following variables control how some tests are run:
1132 TI_NOCLEAN
1134 When running a file test, the temporary output file will not be
1135 removed if this is set.
1136
1138 The history of this module dates back to 1996 when I needed to write a
1139 test suite for my Date::Manip module. At that time, none of the Test::*
1140 modules currently available in CPAN existed (the earliest ones didn't
1141 come along until 1998), so I was left completely on my own writing my
1142 test scripts.
1143 I wrote a very basic version of my test framework which allowed me to
1145 write all of the tests as a string, it would parse the string, count
1146 the tests, and then run them.
1147 Over the years, the functionality I wanted grew, and periodically, I'd
1149 go back and reexamine other Test frameworks (primarily Test::More) to
1150 see if I could replace my framework with an existing module... and I've
1151 always found them wanting, and chosen to extend my existing framework
1152 instead.
1153 As I've written other modules, I've wanted to use the framework in them
1155 too, so I've always just copied it in, but this is obviously tedious
1156 and error prone. I'm not sure why it took me so long... but in 2010, I
1157 finally decided it was time to rework the framework in a module form.
1158 I loosely based my module on Test::More. I like the functionality of
1160 that module, and wanted most of it (and I plan on adding more in future
1161 versions). So this module uses some similar syntax to Test::More
1162 (though it allows a great deal more flexibility in how the tests are
1163 specified).
1164 One thing to note is that I may have been able to write this module as
1166 an extension to Test::More, but after looking into that possibility, I
1167 decided that it would be faster to not do that. I did "borrow" a couple
1168 of routines from it (though they've been modified quite heavily) as a
1169 starting point for a few of the functions in this module, and I thank
1170 the authors of Test::More for their work.
1171
1173 None known.
1174
1176 Test::More - the 'industry standard' of perl test frameworks
1177
1179 If you find a bug in Test::Inter, there are three ways to send it to
1180 me. Any of them are fine, so use the method that is easiest for you.
1181 Direct email
1183 You are welcome to send it directly to me by email. The email
1184 address to use is: sbeck@cpan.org.
1185 CPAN Bug Tracking
1187 You can submit it using the CPAN tracking too. This can be done at
1188 the following URL:
1189 <http://rt.cpan.org/Public/Dist/Display.html?Name=Test-Inter>
1191 GitHub
1193 You can submit it as an issue on GitHub. This can be done at the
1194 following URL:
1195 <https://github.com/SBECK-github/Test-Inter>
1197 Please do not use other means to report bugs (such as forums for a
1199 specific OS or Linux distribution) as it is impossible for me to keep
1200 up with all of them.
1201 When filing a bug report, please include the following information:
1203 Test::Inter version
1205 Please include the version of Test::Inter you are using. You can
1206 get this by using the script:
1207 use Test::Inter;
1209 print $Test::Inter::VERSION,"\n";
1210 If you want to report missing or incorrect codes, you must be running
1212 the most recent version of Test::Inter.
1213 If you find any problems with the documentation (errors, typos, or
1215 items that are not clear), please send them to me. I welcome any
1216 suggestions that will allow me to improve the documentation.
1217
1219 This script is free software; you can redistribute it and/or modify it
1220 under the same terms as Perl itself.
1221
1223 Sullivan Beck (sbeck@cpan.org)
1224perl v5.30.1 2020-01-30 Test::Inter(3)