1Config::Model::Tester(3U)ser Contributed Perl DocumentatiCoonnfig::Model::Tester(3)
2
6 Config::Model::Tester - Test framework for Config::Model
7
9 version 3.007
10
12 In your test file (typically "t/model_test.t"):
13 use warnings;
15 use strict;
16 use Config::Model::Tester ;
18 use ExtUtils::testlib;
19 run_tests() ;
21 Run tests with:
23 perl t/model_test.t [ --log ] [--error] [--trace] [ subtest [ test_case ] ]
25
27 This class provides a way to test configuration models with tests
28 files. This class was designed to tests several models and several
29 tests cases per model.
30 A specific layout for test files must be followed.
32 Sub test specification
34 Each subtest is defined in a file like:
35 t/model_tests.d/<app-name>-test-conf.pl
37 This file specifies that "app-name" (which is defined in
39 "lib/Config/Model/*.d" directory) will be used for the test cases
40 defined in the "*-test-conf.pl" file.
41 This file contains a list of test case (explained below) and expects a
43 set of files used as test data. The layout of these test data files is
44 explained in next section.
45 Simple test file layout
47 Each test case is represented by a configuration file (not a directory)
48 in the "*-examples" directory. This configuration file will be used by
49 the model to test and is copied as "$confdir/$conf_file_name" using the
50 global variables explained below.
51 In the example below, we have 1 app model to test: "lcdproc" and 2
53 tests cases. The app name matches the file specified in
54 "lib/Config/Model/*.d" directory. In this case, the app name matches
55 "lib/Config/Model/system.d/lcdproc"
56 t
58 |-- model_test.t
59 \-- model_tests.d # do not change directory name
60 |-- lcdproc-test-conf.pl # subtest specification for lcdproc app
61 \-- lcdproc-examples
62 |-- t0 # test case t0
63 \-- LCDD-0.5.5 # test case for older LCDproc
64 Subtest specification is written in "lcdproc-test-conf.pl" file (i.e.
66 this modules looks for files named like "<app-name>-test-conf.pl>").
67 Subtests data is provided in files in directory "lcdproc-examples" (
69 i.e. this modules looks for test data in directory
70 "<model-name>-examples>". "lcdproc-test-conf.pl" contains instructions
71 so that each file will be used as a "/etc/LCDd.conf" file during each
72 test case.
73 "lcdproc-test-conf.pl" can contain specifications for more test cases.
75 Each test case requires a new file in "lcdproc-examples" directory.
76 See "Examples" for a link to the actual LCDproc model tests
78 Test file layout for multi-file configuration
80 When a configuration is spread over several files, each test case is
81 provided in a sub-directory. This sub-directory is copied in $conf_dir
82 (a global variable as explained below)
83 In the example below, the test specification is written in
85 "dpkg-test-conf.pl". Dpkg layout requires several files per test case.
86 "dpkg-test-conf.pl" will contain instructions so that each directory
87 under "dpkg-examples" will be used.
88 t/model_tests.d
90 \-- dpkg-test-conf.pl # subtest specification
91 \-- dpkg-examples
92 \-- libversion # example subdir, used as test case name
93 \-- debian # directory for used by test case
94 |-- changelog
95 |-- compat
96 |-- control
97 |-- copyright
98 |-- rules
99 |-- source
100 | \-- format
101 \-- watch
102 See "Examples" for a link to the (many) Dpkg model tests
104 More complex file layout
106 Each test case is a sub-directory on the "*-examples" directory and
107 contains several files. The destination of the test files may depend on
108 the system (e.g. the OS). For instance, system wide "ssh_config" is
109 stored in "/etc/ssh" on Linux, and directly in "/etc" on MacOS.
110 These files are copied in a test directory using a "setup" parameter:
112 setup => {
114 test_file_in_example_dir => 'destination'
115 }
116 Let's consider this example of 2 tests cases for ssh:
118 t/model_tests.d/
120 |-- ssh-test-conf.pl
121 |-- ssh-examples
122 \-- basic
123 |-- system_ssh_config
124 \-- user_ssh_config
125 Unfortunately, "user_ssh_config" is a user file, so you specify where
127 the home directory for the tests with another global variable:
128 $home_for_test = '/home/joe' ;
130 For Linux only, the "setup" parameter is:
132 setup => {
134 'system_ssh_config' => '/etc/ssh/ssh_config',
135 'user_ssh_config' => "~/.ssh/config"
136 }
137 On the other hand, system wide config file is different on MacOS and
139 the test file must be copied in the correct location. When the value of
140 the "setup" hash is another hash, the key of this other hash is used as
141 to specify the target location for other OS (as returned by Perl $^O
142 variable:
143 setup => {
145 'system_ssh_config' => {
146 'darwin' => '/etc/ssh_config',
147 'default' => '/etc/ssh/ssh_config',
148 },
149 'user_ssh_config' => "~/.ssh/config"
150 }
151 See the actual Ssh and Sshd model tests
153 <https://github.com/dod38fr/config-model-
154 openssh/tree/master/t/model_tests.d>
155 Basic test specification
157 Each model subtest is specified in "<model>-test-conf.pl". This file
158 contains a set of global variables. (yes, global variables are often
159 bad ideas in programs, but they are handy for tests):
160 # config file name (used to copy test case into test wr_root/model_tests directory)
162 $conf_file_name = "fstab" ;
163 # config dir where to copy the file (optional)
164 #$conf_dir = "etc" ;
165 # home directory for this test
166 $home_for_test = '/home/joe' ;
167 Here, "t0" file will be copied in
169 "wr_root/model_tests/test-t0/etc/fstab".
170 # config model name to test
172 $model_to_test = "Fstab" ;
173 # list of tests. This modules looks for @tests global variable
175 @tests = (
176 {
177 # test name
178 name => 't0',
179 # add optional specification here for t0 test
180 },
181 {
182 name => 't1',
183 # add optional specification here for t1 test
184 },
185 );
186 1; # to keep Perl happy
188 You can suppress warnings by specifying "no_warnings => 1". On the
190 other hand, you may also want to check for warnings specified to your
191 model. In this case, you should avoid specifying "no_warnings" here and
192 specify warning tests or warning filters as mentioned below.
193 See actual fstab test <https://github.com/dod38fr/config-
195 model/blob/master/t/model_tests.d/fstab-test-conf.pl>.
196 Skip a test
198 A test file can be skipped using $skip global variable.
199 In this example, test is skipped when not running on a Debian system:
201 eval { require AptPkg::Config; };
203 $skip = ( $@ or not -r '/etc/debian_version' ) ? 1 : 0;
204 Internal tests or backend tests
206 Some tests will require the creation of a configuration class dedicated
207 for test (typically to test corner cases on a backend).
208 This test class can be created directly in the test specification by
210 calling create_config_class on $model variable. See for instance the
211 layer test <https://github.com/dod38fr/config-
212 model/blob/master/t/model_tests.d/layer-test-conf.pl> or the test for
213 shellvar backend <https://github.com/dod38fr/config-
214 model/blob/master/t/model_tests.d/backend-shellvar-test-conf.pl>.
215 Test specification with arbitrary file names
217 In some models like "Multistrap", the config file is chosen by the
218 user. In this case, the file name must be specified for each tests
219 case:
220 # not needed if test file is named multistrap-test-conf.pl
222 $model_to_test = "Multistrap";
223 @tests = (
225 {
226 name => 'arm',
227 config_file => '/home/foo/my_arm.conf',
228 check => {},
229 },
230 );
231 See the actual multistrap test <https://github.com/dod38fr/config-
233 model/blob/master/t/model_tests.d/multistrap-test-conf.pl>.
234 Backend argument
236 Some application like systemd requires a backend argument specified by
237 User (e.g. a service name for systemd). The parameter "backend_arg" can
238 be specified to emulate this behavior.
239 Re-use test data
241 When the input data for test is quite complex (several files), it may
242 be interested to re-use these data for other tests case. Knowing that
243 test name must must unique, you can re-use test data with "data_from"
244 parameter. For instance:
245 @tests = (
247 {
248 name => 'some-test',
249 # ...
250 },
251 {
252 name => 'some-other-test',
253 data_from => 'some-test', # re-use data from test above
254 # ...
255 },
256 See plainfile backend test <https://github.com/dod38fr/config-
258 model/blob/master/t/model_tests.d/backend-plainfile-test-conf.pl> for a
259 real life example.
260 Test scenario
262 Each subtest follow a sequence explained below. Each step of this
263 sequence may be altered by adding specification in
264 "<model-to-test>-test-conf.pl":
265 · Setup test in "wr_root/model_tests/<subtest name>/". If your
267 configuration file layout depend on the target system, you will
268 have to specify the path using "setup" parameter. See "Test file
269 layout depending on system".
270 · Create configuration instance, load config data and check its
272 validity. Use "load_check => 'no'" if your file is not valid.
273 · Check for config data warnings. You should pass the list of
275 expected warnings that are emitted through Log::Log4Perl. The array
276 ref is passed as is to the "expect" function of "expect" in
277 Test::Log::Lo4Perl. E.g:
278 log4perl_load_warnings => [
280 [ 'Tree.Node', (warn => qr/deprecated/) x 2 ] ,
281 [ 'Tree.Element.Value' , ( warn => qr/skipping/) x 2 ]
282 ]
283 The Log classes are specified in "cme/Logging".
285 Log levels below "warn" are ignored.
287 Config::Model is currently transitioning from traditional "warn" to
289 warn logs. To avoid breaking all tests based on this module, the
290 warnings are emitted through Log::Log4Perl only when
291 c<$::_use_log4perl_to_warn> is set. This hack will be removed once
292 all warnings checks in tests are ported to log4perl checks.
293 · DEPRECATED. Check for config data warning. You should pass the list
295 of expected warnings. E.g.
296 load_warnings => [ qr/Missing/, (qr/deprecated/) x 3 , ],
298 Use an empty array_ref to mask load warnings.
300 · Optionally run update command:
302 update => {
304 [ returns => 'foo' , ]
305 no_warnings => [ 0 | 1 ], # default 0
306 quiet => [ 0 | 1], # default 0, passed to update method
307 loag4perl_update_warnings => [ ... ] # Test::Log::Log4Perl::expect arguments
308 }
309 Where:
311 · "returns" is the expected return value (optional).
313 · "no_warnings" to suppress the warnings coming from
315 Config::Model::Value. Note that "no_warnings => 1" may be
316 useful for less verbose test.
317 · "quiet" to suppress progress messages during update.
319 · "log4perl_update_warnings" is used to check the warnings
321 produced
322 during update. The argument is passed to "expect" function of
323 Test::Log::Log4Perl. See "load_warnings" parameter above for
324 more
325 details.
326 · DEPRECATED. "update_warnings" is an array ref of quoted regexp
328 (See qr operator) to check the warnings produced during update.
329 use "update => []" to check that no warnings are issued during
330 update.
331 All other arguments are passed to "update" method.
333 · Optionally load configuration data. You should design this config
335 data to suppress any error or warning mentioned above. E.g:
336 load => 'binary:seaview Synopsis="multiplatform interface for sequence alignment"',
338 See Config::Model::Loader for the syntax of the string accepted by
340 "load" parameter.
341 · Optionally, run a check before running apply_fix (if any). This
343 step is useful to check warning messages:
344 check_before_fix => {
346 dump_errors => [ ... ] # optional, see below
347 load4perl_dump_warnings => [ ... ] # optional, see below
348 }
349 Use "dump_errors" if you expect issues:
351 check_before_fix => {
353 dump_errors => [
354 # the issues and a way to fix the issue using Config::Model::Node::load
355 qr/mandatory/ => 'Files:"*" Copyright:0="(c) foobar"',
356 qr/mandatory/ => ' License:FOO text="foo bar" ! Files:"*" License short_name="FOO" '
357 ],
358 }
359 Likewise, specify any expected warnings:
361 check_before_fix => {
363 log4perl_dump_warnings => [ ... ],
364 }
365 "log4perl_dump_warnings" passes the array ref content to "expect"
367 function of Test::Log::Log4Perl.
368 Both "log4perl_dump_warnings" and "dump_errors" can be specified in
370 "check_before_fix" hash.
371 · Optionally, call apply_fixes:
373 apply_fix => 1,
375 · Call dump_tree to check the validity of the data after optional
377 "apply_fix". This step is not optional.
378 As with "check_before_fix", both "dump_errors" or "dump_warnings"
380 can be used.
381 · Run specific content check to verify that configuration data was
383 retrieved correctly:
384 check => {
386 'fs:/proc fs_spec' => "proc",
387 'fs:/proc fs_file' => "/proc",
388 'fs:/home fs_file' => "/home",
389 },
390 The keys of the hash points to the value to be checked using the
392 syntax described in "grab" in Config::Model::Role::Grab.
393 Multiple check on the same item can be applied with a array ref:
395 check => [
397 Synopsis => 'fix undefined path_max for st_size zero',
398 Description => [ qr/^The downstream/, qr/yada yada/ ]
399 ]
400 You can run check using different check modes (See "fetch" in
402 Config::Model::Value) by passing a hash ref instead of a scalar :
403 check => {
405 'sections:debian packages:0' => { mode => 'layered', value => 'dpkg-dev' },
406 'sections:base packages:0' => { mode => 'layered', value => "gcc-4.2-base' },
407 },
408 The whole hash content (except "value") is passed to grab and
410 fetch
411 A regexp can also be used to check value:
413 check => {
415 "License text" => qr/gnu/i,
416 }
417 And specification can nest hash or array style:
419 check => {
421 "License:0 text" => qr/gnu/i,
422 "License:1 text" => [ qr/gnu/i, qr/Stallman/ ],
423 "License:2 text" => { mode => 'custom', value => [ qr/gnu/i , qr/Stallman/ ] },
424 "License:3 text" => [ qr/General/], { mode => 'custom', value => [ qr/gnu/i , qr/Stallman/ ] },
425 }
426 · Verify if a hash contains one or more keys (or keys matching a
428 regexp):
429 has_key => [
431 'sections' => 'debian', # sections must point to a hash element
432 'control' => [qw/source binary/],
433 'copyright Files' => qr/.c$/,
434 'copyright Files' => [qr/\.h$/], qr/\.c$/],
435 ],
436 · Verify that a hash has not a key (or a key matching a regexp):
438 has_not_key => [
440 'copyright Files' => qr/.virus$/ # silly, isn't ?
441 ],
442 · Verify annotation extracted from the configuration file comments:
444 verify_annotation => {
446 'source Build-Depends' => "do NOT add libgtk2-perl to build-deps (see bug #554704)",
447 'source Maintainer' => "what a fine\nteam this one is",
448 },
449 · Write back the config data in "wr_root/model_tests/<subtest
451 name>/". Note that write back is forced, so the tested
452 configuration files are written back even if the configuration
453 values were not changed during the test.
454 You can skip warning when writing back with the global :
456 no_warnings => 1,
458 · Check the content of the written files(s) with
460 Test::File::Contents. Tests can be grouped in an array ref:
461 file_contents => {
463 "/home/foo/my_arm.conf" => "really big string" ,
464 "/home/bar/my_arm.conf" => [ "really big string" , "another"], ,
465 }
466 file_contents_like => {
468 "/home/foo/my_arm.conf" => [ qr/should be there/, qr/as well/ ] ,
469 }
470 file_contents_unlike => {
472 "/home/foo/my_arm.conf" => qr/should NOT be there/ ,
473 }
474 · Check the mode of the written files:
476 file_mode => {
478 "~/.ssh/ssh_config" => 0600, # octal mode
479 "debian/stuff.postinst" => 0755,
480 }
481 Only the last four octets of the mode are tested. I.e. the test is
483 done with " $file_mode & 07777 "
484 Note: this test is skipped on Windows
486 · Check added or removed configuration files. If you expect changes,
488 specify a subref to alter the file list:
489 file_check_sub => sub {
491 my $list_ref = shift ;
492 # file added during tests
493 push @$list_ref, "/debian/source/format" ;
494 },
495 Note that actual and expected file lists are sorted before check,
497 adding a file can be done with "push".
498 · Copy all config data from "wr_root/model_tests/<subtest name>/" to
500 "wr_root/model_tests/<subtest name>-w/". This steps is necessary to
501 check that configuration written back has the same content as the
502 original configuration.
503 · Create a second configuration instance to read the conf file that
505 was just copied (configuration data is checked.)
506 · You can skip the load check if the written file still contain
508 errors (e.g. some errors were ignored and cannot be fixed) with
509 "load_check2 => 'no'"
510 · Optionally load configuration data in the second instance. You
512 should design this config data to suppress any error or warning
513 that occur in the step below. E.g:
514 load2 => 'binary:seaview',
516 See Config::Model::Loader for the syntax of the string accepted by
518 "load2" parameter.
519 · Compare data read from original data.
521 · Run specific content check on the written config file to verify
523 that configuration data was written and retrieved correctly:
524 wr_check => {
526 'fs:/proc fs_spec' => "proc" ,
527 'fs:/proc fs_file' => "/proc",
528 'fs:/home fs_file' => "/home",
529 },
530 Like the "check" item explained above, you can run "wr_check" using
532 different check modes.
533 Running the test
535 Run all tests with one of these commands:
536 prove -l t/model_test.t :: [ --trace ] [ --log ] [ --error ] [ <model_name> [ <regexp> ]]
538 perl -Ilib t/model_test.t [ --trace ] [ --log ] [ --error ] [ <model_name> [ <regexp> ]]
539 By default, all tests are run on all models.
541 You can pass arguments to "t/model_test.t":
543 · Optional parameters: "--trace" to get test traces. "--error" to get
545 stack trace in case of errors, "--log" to have logs. E.g.
546 # run with log and error traces
548 prove -lv t/model_test.t :: --error --logl
549 · The model name to tests. E.g.:
551 # run only fstab tests
553 prove -lv t/model_test.t :: fstab
554 · A regexp to filter subtest E.g.:
556 # run only fstab tests foobar subtest
558 prove -lv t/model_test.t :: fstab foobar
559 # run only fstab tests foo subtest
561 prove -lv t/model_test.t :: fstab '^foo$'
562
564 · LCDproc <http://lcdproc.org> has a single configuration file:
565 "/etc/LCDd.conf". Here's LCDproc test layout
566 <https://github.com/dod38fr/config-model-
567 lcdproc/tree/master/t/model_tests.d> and the test specification
568 <https://github.com/dod38fr/config-model-
569 lcdproc/blob/master/t/model_tests.d/lcdd-test-conf.pl>
570 · Dpkg packages are constructed from several files. These files are
572 handled like configuration files by Config::Model::Dpkg. The test
573 layout <http://anonscm.debian.org/gitweb/?p=pkg-
574 perl/packages/libconfig-model-dpkg-
575 perl.git;a=tree;f=t/model_tests.d;hb=HEAD> features test with
576 multiple file in dpkg-examples
577 <http://anonscm.debian.org/gitweb/?p=pkg-perl/packages/libconfig-
578 model-dpkg-perl.git;a=tree;f=t/model_tests.d/dpkg-
579 examples;hb=HEAD>. The test is specified in dpkg-test-conf.pl
580 <http://anonscm.debian.org/gitweb/?p=pkg-perl/packages/libconfig-
581 model-dpkg-perl.git;a=blob_plain;f=t/model_tests.d/dpkg-test-
582 conf.pl;hb=HEAD>
583 · multistrap-test-conf.pl <https://github.com/dod38fr/config-
585 model/blob/master/t/model_tests.d/multistrap-test-conf.pl> and
586 multistrap-examples <https://github.com/dod38fr/config-
587 model/tree/master/t/model_tests.d/multistrap-examples> specify a
588 test where the configuration file name is not imposed by the
589 application. The file name must then be set in the test
590 specification.
591 · backend-shellvar-test-conf.pl <https://github.com/dod38fr/config-
593 model/blob/master/t/model_tests.d/backend-shellvar-test-conf.pl> is
594 a more complex example showing how to test a backend. The test is
595 done creating a dummy model within the test specification.
596
598 · Config::Model
599 · Test::More
601
603 Dominique Dumont
604
606 This software is Copyright (c) 2013-2018 by Dominique Dumont.
607 This is free software, licensed under:
609 The GNU Lesser General Public License, Version 2.1, February 1999
611
613 Websites
614 The following websites have more information about this module, and may
615 be of help to you. As always, in addition to those websites please use
616 your favorite search engine to discover more resources.
617 · Search CPAN
619 The default CPAN search engine, useful to view POD in HTML format.
621 <http://search.cpan.org/dist/Config-Model-Tester>
623 · AnnoCPAN
625 The AnnoCPAN is a website that allows community annotations of Perl
627 module documentation.
628 <http://annocpan.org/dist/Config-Model-Tester>
630 · CPAN Ratings
632 The CPAN Ratings is a website that allows community ratings and
634 reviews of Perl modules.
635 <http://cpanratings.perl.org/d/Config-Model-Tester>
637 · CPANTS
639 The CPANTS is a website that analyzes the Kwalitee ( code metrics )
641 of a distribution.
642 <http://cpants.cpanauthors.org/dist/Config-Model-Tester>
644 · CPAN Testers
646 The CPAN Testers is a network of smoke testers who run automated
648 tests on uploaded CPAN distributions.
649 <http://www.cpantesters.org/distro/C/Config-Model-Tester>
651 · CPAN Testers Matrix
653 The CPAN Testers Matrix is a website that provides a visual
655 overview of the test results for a distribution on various
656 Perls/platforms.
657 <http://matrix.cpantesters.org/?dist=Config-Model-Tester>
659 · CPAN Testers Dependencies
661 The CPAN Testers Dependencies is a website that shows a chart of
663 the test results of all dependencies for a distribution.
664 <http://deps.cpantesters.org/?module=Config::Model::Tester>
666 Bugs / Feature Requests
668 Please report any bugs or feature requests by email to "ddumont at
669 cpan.org", or through the web interface at
670 <https://github.com/dod38fr/config-model-tester/issues>. You will be
671 automatically notified of any progress on the request by the system.
672 Source Code
674 The code is open to the world, and available for you to hack on. Please
675 feel free to browse it and play with it, or whatever. If you want to
676 contribute patches, please send me a diff or prod me to pull from your
677 repository :)
678 <http://github.com/dod38fr/config-model-tester.git>
680 git clone git://github.com/dod38fr/config-model-tester.git
682perl v5.28.1 2019-02-02 Config::Model::Tester(3)