1Test::Inter(3) User Contributed Perl Documentation Test::Inter(3)
2
6 Test::Inter - framework for more readable interactive test scripts
7
9 This is another framework for writing test scripts. It is loosely
10 inspired by Test::More, and has most of it's functionality, but it is
11 not a drop-in replacement.
12 Test::More (and other existing test frameworks) suffer from two
14 weaknesses, both of which have prevented me from ever using them:
15 None offer the ability to access specific tests in
17 a reasonably interactive fashion
18 None offer the ability to write the tests in
20 whatever format would make the tests the most
21 readable
22 The way I write and use test scripts, existing Test::* modules are not
24 nearly as useful as they could be. Test scripts written using
25 Test::More work fine when running as part of the test suite, but
26 debugging an individual test requires extra steps, and the tests
27 themselves are not as readable as they should be.
28 I do most of my debugging using test scripts. When I find a bug, I
30 write a test case for it, debug it using the test script, and then
31 leave the test there so the bug won't come back (hopefully).
32 Since I use test scripts in two ways (part of a standard test suite and
34 to run the scripts in some interactive way to debug problems), I want
35 to be able to do the follwing trivially:
36 Easy access to a specific test or tests
38 If I'm running the test script interactively (perhaps in the
39 debugger), there are several common functions that I want to have
40 available, including:
41 Run only a single test, or a subset of tests
43 Set a breakpoint in the debugger to run
45 up to the start of the Nth test
46 Better diagnostics
48 When running a test script as part of a test suite, the pass/fail
49 status is really the only thing of interest. You just want to know
50 if the module passes all the tests.
51 When running interactively, additional information may allow me to
53 quickly track down the problem without even resorting to a
54 debugger.
55 If a test fails, I almost always want to see why it failed if I'm
57 running it interactively. If reasonable, I want to see a list of
58 what was input, what was output, and what was expected.
59 The other feature that I wanted in a test suite is the ability to
61 define the tests in a readable format. In almost every case, it is best
62 to think of a test script as consisting of two separate parts: a script
63 part, and a test part, and the more you can keep the two separate, the
64 better.
65 The script part of a test script is the least important part! It's
67 usually fairly trivial, rarely needs to be changed, and is quite simply
68 not the focus of the test script.
69 The tests part of the script IS the important part, and these should be
71 expressed in a form that is easy to maintain, easy to read, and easy to
72 modify, and none of these should involve modifying the script portion
73 of the test script in general. As a general rule, if the script portion
74 of the test script obscures the tests in any way, it's not written
75 correctly!
76 Compare this to any other systems where you are mixing two "languages".
78 For example, a PHP script where you have a mixture of PHP and HTML or a
79 templating system consisting of text and template commands. The more
80 the two languages are interwoven, the less readable both are, and the
81 harder it is to maintain.
82 As often as possible, I want the tests to be written in some sort of
84 text format which can be easily read as a table with no perl commands
85 interspersed. I want to the freedom to define the tests in one big
86 string (perhaps a DATA section, or even in a separate file) which is
87 easily readable. This may introduce the necessity of parsing it, but it
88 makes it significantly easier to maintain the tests.
89 This flexibilty makes it much easier to read the tests (as opposed to
91 the script) which is the fundamental content of a test script.
92 To illustrate some of this, in Test::More, a series of tests might be
94 specified as:
95 # test 1
97 $result = func("apples","bushels");
98 is($result, "enough");
99 # test 2
101 $result = func("grapefruit","tons");
102 is($result, "enough");
103 # test 3
105 $result = func("oranges","boatloads");
106 is($result, "insufficient");
107 Thinking about the features I want that I listed above, there are
109 several difficulties with this.
110 Debugging the script is tedious
112 To debug the 3rd test, you have to open up the file and get the
113 line number of the 3rd test. You set a breakpoint there and then
114 you can run the script.
115 It's typically not very hard to do this, but if you have to check
117 out the line number every time you want to visit a test, it can
118 break your chain of thought.
119 How much better to be able to say:
121 break func ($::TI_NUM==3)
123 It would also be nice to be able to skip the first two tests...
125 perhaps they take a long time to run, and I want to get right to
126 work on test 3. You can do this easily too by setting the
127 $::TI_START variable.
128 Way too much perl interspersed with the tests
130 It's difficult to read the tests individually in this script
131 because there is too much perl code among them, and virtually
132 impossible to look at them as a whole.
133 It is true that looking at this as a perl script, it is very
135 simple... but the script ISN'T the content you're interested in.
136 The REAL content of this script are the tests, which consist of the
137 function arguments and the expected result. Although it's not
138 impossible to see each of these in the script above, it's not in a
139 format that is conducive to studying the tests, and especially not
140 for examing the list of tests as a whole.
141 Now, look at an alternate way of specifying the tests using this
143 module:
144 $tests = "
146 apples bushels => enough
148 grapefruit tons => enough
150 oranges boatloads => insufficient
152 ";
154 $o->tests(tests => $tests,
156 func => \&func);
157 Here, it's easy to see the list of tests, and adding additional tests
159 is a breeze.
160 This module supports a number of methods for defining tests, so you can
162 use whichever one is most convenient (including methods that are
163 identical to Test::More).
164 In addition, the following debugger command works as desired:
166 b func ($::TI_NUM==3)
168 and you're ready to debug.
170
172 Every test may have several pieces of information:
173 A name
175 Every test is automatically assigned a number, but it may be useful
176 to specify a name of a test (which is actually a short description
177 of the test). Whenever a test result is reported, the name will be
178 given (if one was specified).
179 The name may not have a '#' in it.
181 The name is completely optional, but makes the results more
183 readable.
184 An expected result
186 In order to test something, you need to know what result was
187 expected (or in some cases, what result was NOT expected).
188 A function and arguments OR a result
190 You also need to know the results that you're comparing to the
191 expected results.
192 This can be obtained by simply working with a set of results, or a
194 function name and a set of arguments to pass to it.
195 Conditions
197 It is useful to be able to specify state information at the start
198 of the test suite (for example, to see if certain features are
199 available), and some tests may only run if those conditions are
200 met.
201 If no conditions are set for a test, it will always run.
203 Todo tests
205 Some tests may be marked as 'todo' tests. These are test which are
206 allowed to fail (meaning that they have been put in place for an
207 as-yet unimplemented feature). Since it is expected that the test
208 will fail, the test suite will still pass, even if these tests
209 fail.
210 The tests will still run and if they pass, a message is issued
212 saying that the feature is now implemented, and the tests should be
213 graduated to non-todo state.
214
216 new
217 $o = new Test::Inter [$name] [%options];
218 This creates a new test framework. There are several options which
220 may be used to specify which tests are run, how they are run, and
221 what output is given.
222 The entire test script can be named by passing in $name. Options
224 can be passed in as a hash of ($opt,$val) pairs.
225 Options can be set in four different ways. First, you can pass in
227 an ($opt,$val) pair in the new method. Second, you can set an
228 environment variable (which overrides any value passed to the new
229 method). Third, you can set a global variable (which overrides both
230 the environment variable and options passed to the new method).
231 Fouth, you can call the appropriate method to set the option. This
232 overrides all other methods.
233 Each of the allowed options are described below in the following
235 base methods:
236 start
238 end
239 testnum
240 plan
241 abort
242 quiet
243 mode
244 skip_all
245 width
246 version
248 $o->version();
249 Returns the version of the module.
251 start
253 $o = new Test::Inter 'start' => $N;
254 $o->start($N)
255 To define which test you want to start with, pass in an ($opt,$val)
257 pair of ('start',N), set an environment variable TI_START=N, or a
258 global variable $::TI_START=N.
259 When the start test is defined, most tests numbered less than N are
261 completely ignored. If the tests are being run quietly (see the
262 quiet method below), nothing is printed out for these tests.
263 Otherwise, a skip message is printed out.
264 One class of tests IS still executed. Tests run using the
266 require_ok or use_ok methods (to test the loading of modules) are
267 still run.
268 If no value (or a value of 0) is used, it defaults to the first
270 test.
271 end
273 $o = new Test::Inter 'end' => $M;
274 $o->end($M);
275 To define which test you want to end with, pass in an ($opt,$val)
277 pair of ('end',M), set an environment variable TI_END=M, or set a
278 global variable $::TI_END=M.
279 When the end test is defined, all tests numbered more than M are
281 completely ignored. If the tests are being run quietly (see the
282 quiet method below), nothing is printed out for these tests.
283 Otherwise, a skip message is printed out.
284 If no value is given, it defaults to 0 (which means that all
286 reamining tests are run).
287 testnum
289 $o = new Test::Inter 'testnum' => $N;
290 $o->testnum($N);
291 This is used to run only a single test. It is equivalent to setting
293 both the start and end tests to $N.
294 plan
296 done_testing
297 $o = new Test::Inter 'plan' => $N;
298 $o->plan($n);
299 $o->done_testing();
301 $o->done_testing($n);
302 The TAP API (the 'language' used to run a sequence of tests and see
304 which ones failed and which ones passedd) requires a statement of
305 the number of tests that are expected to run.
306 This statement can appear at the start of the test suite, or at the
308 end.
309 If you know in advance how many tests should run in the test
311 script, you can pass in a non-zero integer in a ('plan',N) pair to
312 the new method, or set the TI_PLAN environment variable or the
313 $::TI_PLAN global variable, or call the plan method.
314 If you know how many tests should run at the end of the test
316 script, you can pass in a non-zero integer to the done_testing
317 method.
318 Frequently, you don't really care how many tests are in the script
320 (especially if new tests are added on a regular basis). In this
321 case, you still need to include a statement that says that the
322 number of tests expected is however many were run. To do this, call
323 the done_testing method with no argument.
324 NOTE: if the plan method is used, it MUST be used before any tests
326 are run (including those that test the loading of modules). If the
327 done_testing method is used, it MUST be called after all tests are
328 run. You must specify a plan or use a done_testing statement, but
329 you cannot do both.
330 It is NOT strictly required to set a plan if the script is only run
332 interactively, so if for some reason this module is used for test
333 scritps which are not part of a standard perl test suite, the plan
334 and done_testing statements are optional. As a matter of fact, the
335 script will run just fine without them... but a perl installer will
336 report a failure in the test suite.
337 abort
339 $o = new Test::Inter 'abort' => 0/1/2;
340 $o->abort(0/1/2);
341 The abort option can be set using an ('abort',0/1/2) option pair,
343 or by setting the TI_ABORT environment variable, or the $::TI_ABORT
344 global variable.
345 If this is set to 1, the test script will run unmodified until a
347 test fails. At that point, all remaining tests will be skipped. If
348 it is set to 2, the test script will run until a test fails at
349 which point it will exit with an error code of 1.
350 In both cases, todo tests will NOT trigger the abort behavior.
352 quiet
354 $o = new Test::Inter 'quiet' => 0/1/2;
355 $o->quiet(0/1/2);
356 The quiet option can be set using an ('quiet',0/1/2) option pair,
358 or by setting the TI_QUIET environment variable, or the $::TI_QUIET
359 global variable.
360 If this is set to 0 (the default), all information will be printed
362 out. If it is set to 1, some optional information will not be
363 printed. If it is set to 2, all optional information will not be
364 printed.
365 mode
367 $o = new Test::Inter 'mode' => MODE;
368 $o->mode(MODE);
369 The mode option can be set using a ('mode',MODE) option pair, or by
371 setting the TI_MODE environment variable, or the $::TI_MODE global
372 variable.
373 Currently, MODE can be 'test' or 'inter' meaning that the script is
375 run as part of a test suite, or interactively.
376 When run in test mode, it prints out the results using the TAP
378 grammar (i.e. 'ok 1', 'not ok 3', etc.).
379 When run in interactive mode, it prints out results in a more human
381 readable format.
382 width
384 $o = new Test::Inter 'width' => WIDTH;
385 $o->width(WIDTH);
386 The width option can be set using a ('width',WIDTH) option pair, or
388 by setting the TI_WIDTH environment variable, or the $::TI_WIDTH
389 global variable.
390 WIDTH is the width of the terminal (for printing out failed test
392 information). It defaults to 80, but it can be set to any width
393 (and lines longer then this are truncated). If WIDTH is set to 0,
394 no truncation is done.
395 skip_all
397 $o = new Test::Inter 'skip_all' => REASON;
398 $o->skip_all(REASON);
399 The skip_all option can be set using an ('skip_all',REASON) option
401 pair, or by setting the TI_SKIP_ALL environment variable, or the
402 $::TI_SKIP_ALL global variable.
403 If this is set, the entire test script will be skipped for the
405 reason given. This must be done before any test is run, and before
406 any plan number is set.
407 The skip_all can also be called at any point during the script
409 (i.e. after tests have been run). In this case, all remaining
410 scripts will be skipped.
411 $o->skip_all(REASON,FEATURE,FEATURE,...);
413 $o->skip_all('',FEATURE,FEATURE,...);
414 This will skip all tests (or all remaining tests) unless all
416 features are available. REASON can be entered as an empty string
417 and the reason the tests are skipped will be a message about the
418 missing feature.
419 feature
421 $o->feature($feature,$val);
422 This defines a feature. If $val is non-zero, the feature is
424 available. Otherwise it is not.
425 diag
427 note
428 $o->diag($message);
429 $o->note($message);
430 Both of these print an optional message. Messages printed with the
432 note method are always optional and will be omitted if the quiet
433 option is set to 1 or 2. Messages printed with the diag method are
434 optional and will not be printed if the quiet option is set to 2,
435 but they will be printed if the quiet method is set to 1.
436 testdir
438 Occasionally, it may be necessary to know the directory where the
439 tests live (for example, there may be a config or data file in
440 there). This method will return the directory.
441
443 Test scripts can load other modules (using either the perl 'use' or
444 'require' commands). There are three different modes for doing this
445 which determine how this is done.
446 required mode
448 By default, this is used to test for a module that is required for
449 all tests in the test script.
450 Loading the module is treated as an actual test in the test suite.
452 The test is to determine whether the module is available and can be
453 loaded. If it can be loaded, it is, and it is reported as a
454 successful test. If it cannot be loaded, it is reported as a failed
455 test.
456 In the result of a failed test, all remaining tests will be skipped
458 automatically (except for other tests which load modules).
459 feature mode
461 In feature mode, loading the module is not treated as a test (i.e.
462 it will not print out an 'ok' or 'not ok' line. Instead, it will
463 set a feature (named the same as the module) which can be used to
464 determine whether other tests should run or not.
465 forbid mode
467 In a few very rare cases, we may want to test for a module but
468 expect that it not be present. This is the exact opposite of the
469 'required' mode.
470 Successfully loading the module is treated as a test failure. In
472 the event of a failure, all remaining tests will be skipped.
473 The methods available are:
475 require_ok
477 $o->require_ok($module [,$mode]);
478 This is used to load a module using the perl 'require' function. If
480 $mode is not passed in, the default mode (required) is used to test
481 the existance of the module.
482 If $mode is passed in, it must be either the string 'forbid' or
484 'feature'.
485 If $mode is 'feature', a feature named $module is set if the module
487 was able to be loaded.
488 use_ok
490 $o->use_ok(@args [,$mode]);
491 This is used to load a module with 'use', or check a perl version.
493 BEGIN { $o->use_ok('5.010'); }
495 BEGIN { $o->use_ok('Some::Module'); }
496 BEGIN { $o->use_ok('Some::Module',2.05); }
497 BEGIN { $o->use_ok('Some::Module','foo','bar'); }
498 BEGIN { $o->use_ok('Some::Module',2.05,'foo','bar'); }
499 are the same as:
501 use 5.010;
503 use Some::Module;
504 use Some::Module 2.05;
505 use Some::Module qw(foo bar);
506 use Some::Module 2.05 qw(foo bar);
507 Putting the use_ok call in a BEGIN block allows the functions to be
509 imported at compile-time and prototypes are properly honored.
510 You'll also need to load the Test::Inter module, and create the
511 object in a BEGIN block.
512 $mode acts the same as in the require_ok method.
514
516 There are several methods for running tests. The ok, is, and isnt
517 methods are included for those already comfortable with Test::More and
518 wishing to stick with the same format of test script. The tests method
519 is the suggested method though since it makes use of the full power of
520 this module.
521 ok
523 $o->ok(TESTS);
524 A test run with ok looks at a result, and if it evaluates to 0 (or
526 false), it fails. If it evaluates to non-zero (or true), it passes.
527 These tests do not require you to specify the expected results. If
529 expected results are given, they will be compared against the
530 result received, and if they differ, a diagnostic message will be
531 printed, but the test will still succeed or fail based only on the
532 actual result produced.
533 These tests require a single result and either zero or one expected
535 results.
536 To run a single test, use any of the following:
538 $o->ok(); # always succeeds
540 $o->ok($result);
542 $o->ok($result,$name);
543 $o->ok($result,$expected,$name);
544 $o->ok(\&func);
546 $o->ok(\&func,$name);
547 $o->ok(\&func,$expected,$name);
548 $o->ok(\&func,\@args);
550 $o->ok(\&func,\@args,$name);
551 $o->ok(\&func,\@args,$expected,$name);
552 If $result is a scalar, the test passes if $result is true. If
554 $result is a list reference, and the list is either empty, or the
555 first element is a scalar), the test succeeds if the list contains
556 any values (except for undef). If $result is a hash reference, the
557 test succeeds if the hash contains any key with a value that is not
558 undef.
559 If \&func and \@args are passed in, then $result is generated by
561 passing @args to &func and behaves identically to the calls where
562 $result is passed in. If \&func is passed in but no arguments, the
563 function takes no arguments, but still produces a result.
564 $result may be a scalar, list reference, or hash reference. If it
566 is a list reference, the test passes is the list contains any
567 defined values. If it is a hash reference, the test passes if any
568 of the keys contain defined values.
569 If an expected value is passed in and the result does not match it,
571 a diagnostic warning will be printed, even if the test passes.
572 is
574 isnt
575 $o->is(TESTS);
576 $o->isnt(TESTS);
577 A test run with is looks as a result and tests to see if it is
579 identical to an expected result. If it is, the test passes.
580 Otherwise it fails. In the case of a failure, a diagnostic message
581 will show what result was actually obtained and what was expected.
582 A test run with isnt looks at a result and tests to see if the
584 result obtained is different than an expected result. If it is
585 different, the test passes. Otherwise it fails.
586 The is method can be called in any of the following ways:
588 $o->is($result,$expected);
590 $o->is($result,$expected,$name);
591 $o->is(\&func,$expected);
593 $o->is(\&func,$expected,$name);
594 $o->is(\&func,\@args,$expected);
596 $o->is(\&func,\@args,$expected,$name);
597 The isnt method can be called in exactly the same way.
599 As with the ok method, the result can be a scalar, hashref, or
601 listref. If it is a hashref or listref, the entire structure must
602 match the expected value.
603 tests
605 $o->tests($opt=>$val, $opt=>$val, ...)
606 The options available are described in the following section.
608 file
610 $o->file($func,$input,$outputdir,$expected,$name [,@args]);
611 Sometimes it may be easiest to store the input, output, and
613 expected output from a series of tests in files. In this case, each
614 line of output will be treated as a single test, so the output and
615 expected output must match up exactly.
616 $func is a reference to a function which will produce a temporary
618 output file. If $input is specified, it is the name of the input
619 file, and it will be passed to the function as the first argument.
620 If $input is left blank, no input file will be used. The input file
621 may be specified as a full path, or just the file name (in which
622 case it will be looked for in the test directory and the current
623 directory).
624 $func also takes a arequired argument which is the output file.
626 The tests method will create a tempoary file containing the output.
627 If $outputdir is passed in, it is the directory where the output
628 file will be written. If $outputdir is left blank, the temporary
629 file will be written to the test directory.
630 If @args is passed in, it is a list of additional arguments which
632 will be passed to $func.
633 $expected is the name of a file which contains the expeccted
635 output. It can be fully specified, or it will be checked for in
636 the test directory.
637
639 It is expected that most tests (except for those that load a module)
640 will be run using the tests method called as:
641 $o->tests($opt => $val, $opt => $val, ...);
643 The following options are available:
645 name
647 name => NAME
648 This sets the name of this set of tests. All tests will be given
650 the same name.
651 tests
653 func
654 expected
655 In order to specify a series of tests, you have to specify either a
656 function and a list of arguments, or a list of results.
657 Specifying the function and list of arguments can be done using the
659 pair:
660 func => \&FUNCTION
662 tests => TESTS
663 If the func option is not set, tests contains a list of results.
665 A list of expected results may also be given. They can be included
667 in the
668 tests => TESTS
670 option or included separately as:
672 expected => RESULTS
674 The way to specify these are covered in the next section SPECIFYING
676 THE TESTS.
677 feature
679 disable
680 feature => [FEATURE1, FEATURE2, ...]
681 disable => [FEATURE1, FEATURE2, ...]
683 The default set of tests to run is determined using the start, end,
685 and skip_all methods discussed above. Using those methods, a list
686 of tests is obtained, and it is expected that these will run.
687 The feature and disable options modify the list.
689 If the feature option is included, the tests given in this call
691 will only run if ALL of the features listed are available.
692 If the disable option is included, the tests will be run unless ANY
694 of the features listed are available.
695 skip
697 skip => REASON
698 Skip these tests for the reason given.
700 todo
702 todo => 0/1
703 Setting this to 1 says that these tests are allowed to fail. They
705 represent a feature that is not yet implemented.
706 If the tests succeed, a message will be printed notifying the
708 developer that the tests are now ready to promote to actual use.
709
711 A series of tests can be specified in two different ways. The tests can
712 be written in a very simple string format, or stored as a list.
713 Demonstrating how this can be done is best done by example, so let's
715 say that there is a function (func) which takes two arguments, and
716 returns a single value. Let's say that the expected output (and the
717 actual output) from 3 different sets of arguments is:
718 Input Expected Output Actual Output
720 ----- --------------- -------------
721 1,2 a a
722 3,4 b x
723 5,6 c c
724 (so in this case, the first and third tests pass, but the 2nd one will
726 fail).
727 Specifying these tests as lists could be done as:
729 $o->tests(
731 func => &func,
732 tests => [ [1,2], [3,4], [5,6] ],
733 expected => [ [a], [b], [c] ],
734 );
735 Here, the tests are stored as a list, and each element in the list is a
737 listref containing the set of arguments.
738 If the func option is not passed in, the tests option is set to a list
740 of results to compare with the expected results, so the following is
741 equivalent to the above:
742 $o->tests(
744 tests => [ [a], [x], [c] ],
745 expected => [ [a], [b], [c] ],
746 );
747 If an argument (or actual result) or an expected result is only a
749 single value, it can be entered as a scalar instead of a list ref, so
750 the following is also equivalent:
751 $o->tests(
753 func => &func,
754 tests => [ [1,2], [3,4], [5,6] ],
755 expected => [ a, b, [c] ],
756 );
757 The only exception to this is if the single value is itself a list
759 reference. In this case it MUST be included as a reference. In other
760 words, if you have a single test, and the expected value for this test
761 is a list reference, it must be passed in as:
762 expected => [ [ \@r ] ]
764 NOT as:
766 expected => [ \@r ]
768 Passing in a set of expected results is optional. If none are passed
770 in, the tests are treated as if they had been passed to the 'ok' method
771 (i.e. if they return something true, they pass, otherwise they fail).
772 The second way to specify tests is as a string. The string is a multi-
774 line string with each tests being separate from the next test by a
775 blank line. Comments (lines which begin with '#') are allowed, and are
776 ignored. Whitespace at the start and end of the line is ignored.
777 The string may contain the results directly, or results may be passed
779 in separately. For example, the following all give the same sets of
780 tests as the example above:
781 $o->tests(
783 func => &func,
784 tests => "
785 # Test 1
786 1 2 => a
787 # Test 2
789 3 4 => b
790 5 6 => c
792 ",
793 );
794 $o->tests(
796 func => &func,
797 tests => "
798 1 2
799 3 4
801 5 6
803 ",
804 expected => [ [a], [b], [c] ]
805 );
806 $o->tests(
808 func => &func,
809 tests => [ [1,2], [3,4], [5,6] ],
810 expected => "
811 a
812 b
814 c
816 ",
817 );
818 $o->tests(
820 func => &func,
821 tests => "
822 1 2
823 3 4
825 5 6
827 ",
828 expected => "
829 a
830 b
832 c
834 ",
835 );
836 The expected results may also consist of only a single set of results
838 (in this case, it must be passed in as a listref). In this case, all of
839 the tests are expected to have the same results.
840 So, the following are equivalent:
842 $o->tests(
844 func => &func,
845 tests => "
846 1 2 => a b
847 3 4 => a b
849 5 6 => a b
851 ",
852 );
853 $o->tests(
855 func => &func,
856 tests => "
857 1 2
858 3 4
860 5 6
862 ",
863 expected => [ [a, b] ],
864 );
865 $o->tests(
867 func => &func,
868 tests => "
869 1 2
870 3 4
872 5 6
874 ",
875 expected => "a b",
876 );
877 The number of expected values must either be 1 (i.e. all of the tests
879 are expected to produce the same value) or exactly the same number as
880 the number of tests.
881 The parser is actually quite powerful, and can handle multi-line tests,
883 quoted strings, and nested data structures.
884 The test may be split across any number of lines, provided there is not
886 a completely blank line (which signals the end of the test), so the
887 following are eqivalent:
888 tests => "a b c",
890 tests => "a b
891 c",
892 Arguments (or expected results) may include data structures. For
894 example, the following are equivalent:
895 tests => "[ a b ] { a 1 b 2 }"
897 tests => [ [ [a,b], { a=>1, b=>2 } ] ]
898 Whitespace is mostly optional, but there is one exception. An item must
900 end with some kind of delimiter, so the following will fail:
901 tests => "[a b][c d]"
903 The first element (the list ref [a b]) must be separated from the
905 second element by the delimiter (which is whitespace in this case), so
906 it must be written as:
907 tests => "[a b] [c d]"
909 As already demonstrated, hashrefs and listrefs may be included and
911 nested. Elements may also be included inside parens, but this is
912 optional since all arguments and expected results are already treated
913 as lists, so the following are equivalent:
914 tests => "a b c"
916 tests => "(a b) c"
917 Although parens are optional, they may make things more readable, and
919 allow you to use something other than whitespace as the delimiter.
920 If the character immediately following the opening paren, brace, or
922 bracket is a punctuation mark, then it is used as the delimiter instead
923 of whitespace. For example, the following are all equivalent:
924 [ a b c ]
926 [a b c]
927 [, a,b,c ]
928 [, a, b, c ]
929 A delimiter is a single character, and the following may not be used as
931 a delimiter:
932 any opening/closing characters () [] {}
934 single or double quotes
935 alphanumeric characters
936 underscore
937 Whitespace (including newlines) around the delimiter is ignored, so the
939 following is valid:
940 [, a,
942 b,
943 c ]
944 Two delimiters next to each other or a trailing delimiter produce an
946 empty string.
947 "(,a,b,)" => (a, b, '')
949 "(,a,,b)" => (a, '', b)
950 Hashrefs may be specified by braces and the following are equivalent:
952 { a 1 b 2 }
954 {, a,1,b,2 }
955 {, a,1,b,2, }
956 Note that a trailing delimiter is ignored if there are already an even
958 number of elements, or an empty string otherwise.
959 Nested structures are allowed:
961 "[ [1 2] [3 4] ]"
963 For example,
965 $o->tests(
967 func => &func,
968 tests => "a [ b c ] { d 1 e 2 } => x y"
969 );
970 is equivalent to:
972 $o->tests(
974 func => &func,
975 tests => [ [a, [b,c], {d=>1,e=>2}] ],
976 results => [ [x,y] ],
977 );
978 Any single value can be surrounded by single or double quotes in order
980 to include the delimiter. So:
981 "(, a,'b,c',e )"
983 is equivalent to:
985 "( a b,c e )"
987 Any single value can be the string '__undef__' which will be turned
989 into an actual undef. If the value is '__blank__' it is turned into an
990 empty string (''), though it can also be specified as '' directly. Any
991 value can have an embedded newline by including a __nl__ in the value,
992 but the value must be written on a single line.
993 Expected results are separated from arguments by ' => '.
995
997 To summarize the information above, the following environment variables
998 (and main:: variables) exist. Each can be set in a perl script as a
999 variable in the main namespace:
1000 $::TI_END
1002 or as an environment variable:
1004 $ENV{TI_END}
1006 TI_START
1008 Set this to define the test you want to start with.
1009 Example: If you have a perl test script (my_test_script) and you
1011 want to start running it at test 12, run the following shell
1012 commands:
1013 TI_START=12
1015 ./my_test_script.t
1016 TI_END
1018 Set this to define the test you want to end with.
1019 TI_TESTNUM
1021 Set this to run only a single test
1022 TI_QUIET
1024 How verbose the test script is.
1025 TI_MODE
1027 How the output is formatted.
1028 TI_WIDTH
1030 The width of the terminal.
1031
1033 The history of this module dates back to 1996 when I needed to write a
1034 test suite for my Date::Manip module. At that time, none of the Test::*
1035 modules currently available in CPAN existed (the earliest ones didn't
1036 come along until 1998), so I was left completely on my own writing my
1037 test scripts.
1038 I wrote a very basic version of my test framework which allowed me to
1040 write all of the tests as a string, it would parse the string, count
1041 the tests, and then run them.
1042 Over the years, the functionality I wanted grew, and periodically, I'd
1044 go back and reexamine other Test frameworks (primarily Test::More) to
1045 see if I could replace my framework with an existing module... and I've
1046 always found them wanting, and chosen to extend my existing framework
1047 instead.
1048 As I've written other modules, I've wanted to use the framework in them
1050 too, so I've always just copied it in, but this is obviously tedious
1051 and error prone. I'm not sure why it took me so long... but in 2010, I
1052 finally decided it was time to rework the framework in a module form.
1053 I loosely based my module on Test::More. I like the functionality of
1055 that module, and wanted most of it (and I plan on adding more in future
1056 versions). So this module uses some similar syntax to Test::More
1057 (though it allows a great deal more flexibility in how the tests are
1058 specified).
1059 One thing to note is that I may have been able to write this module as
1061 an extension to Test::More, but after looking into that possibility, I
1062 decided that it would be faster to not do that. I did "borrow" a couple
1063 of routines from it (though they've been modified quite heavily) as a
1064 starting point for a few of the functions in this module, and I thank
1065 the authors of Test::More for their work.
1066
1068 None known.
1069
1071 Test::More - the 'industry standard' of perl test frameworks
1072
1074 This script is free software; you can redistribute it and/or modify it
1075 under the same terms as Perl itself.
1076
1078 Sullivan Beck (sbeck@cpan.org)
1079perl v5.16.3 2014-06-10 Test::Inter(3)