1Config::Model::Tester(3U)ser Contributed Perl DocumentatiCoonnfig::Model::Tester(3)
2
6 Config::Model::Tester - Test framework for Config::Model
7
9 version 3.006
10
12 In your test file (typically "t/model_test.t"):
13 use warnings;
15 use strict;
16 use Config::Model::Tester ;
18 use ExtUtils::testlib;
19 run_tests() ;
21 Run tests with:
23 perl t/model_test.t [ --log ] [--error] [--trace] [ subtest [ test_case ] ]
25
27 This class provides a way to test configuration models with tests
28 files. This class was designed to tests several models and several
29 tests cases per model.
30 A specific layout for test files must be followed.
32 Sub test specification
34 Each subtest is defined in a file like:
35 t/model_tests.d/<app-name>-test-conf.pl
37 This file specifies that "app-name" (which is defined in
39 "lib/Config/Model/*.d" directory) will be used for the test cases
40 defined in the "*-test-conf.pl" file.
41 This file contains a list of test case (explained below) and expects a
43 set of files used as test data. The layout of these test data files is
44 explained in next section.
45 Simple test file layout
47 Each test case is represented by a configuration file (not a directory)
48 in the "*-examples" directory. This configuration file will be used by
49 the model to test and is copied as "$confdir/$conf_file_name" using the
50 global variables explained below.
51 In the example below, we have 1 app model to test: "lcdproc" and 2
53 tests cases. The app name matches the file specified in
54 "lib/Config/Model/*.d" directory. In this case, the app name matches
55 "lib/Config/Model/system.d/lcdproc"
56 t
58 |-- model_test.t
59 \-- model_tests.d # do not change directory name
60 |-- lcdproc-test-conf.pl # subtest specification for lcdproc app
61 \-- lcdproc-examples
62 |-- t0 # test case t0
63 \-- LCDD-0.5.5 # test case for older LCDproc
64 Subtest specification is written in "lcdproc-test-conf.pl" file (i.e.
66 this modules looks for files named like "<app-name>-test-conf.pl>").
67 Subtests data is provided in files in directory "lcdproc-examples" (
69 i.e. this modules looks for test data in directory
70 "<model-name>-examples>". "lcdproc-test-conf.pl" contains instructions
71 so that each file will be used as a "/etc/LCDd.conf" file during each
72 test case.
73 "lcdproc-test-conf.pl" can contain specifications for more test cases.
75 Each test case requires a new file in "lcdproc-examples" directory.
76 See "Examples" for a link to the actual LCDproc model tests
78 Test file layout for multi-file configuration
80 When a configuration is spread over several files, each test case is
81 provided in a sub-directory. This sub-directory is copied in $conf_dir
82 (a global variable as explained below)
83 In the example below, the test specification is written in
85 "dpkg-test-conf.pl". Dpkg layout requires several files per test case.
86 "dpkg-test-conf.pl" will contain instructions so that each directory
87 under "dpkg-examples" will be used.
88 t/model_tests.d
90 \-- dpkg-test-conf.pl # subtest specification
91 \-- dpkg-examples
92 \-- libversion # example subdir, used as test case name
93 \-- debian # directory for used by test case
94 |-- changelog
95 |-- compat
96 |-- control
97 |-- copyright
98 |-- rules
99 |-- source
100 | \-- format
101 \-- watch
102 See "Examples" for a link to the (many) Dpkg model tests
104 More complex file layout
106 Each test case is a sub-directory on the "*-examples" directory and
107 contains several files. The destination of the test files may depend on
108 the system (e.g. the OS). For instance, system wide "ssh_config" is
109 stored in "/etc/ssh" on Linux, and directly in "/etc" on MacOS.
110 These files are copied in a test directory using a "setup" parameter:
112 setup => {
114 test_file_in_example_dir => 'destination'
115 }
116 Let's consider this example of 2 tests cases for ssh:
118 t/model_tests.d/
120 |-- ssh-test-conf.pl
121 |-- ssh-examples
122 \-- basic
123 |-- system_ssh_config
124 \-- user_ssh_config
125 Unfortunately, "user_ssh_config" is a user file, so you specify where
127 the home directory for the tests with another global variable:
128 $home_for_test = '/home/joe' ;
130 For Linux only, the "setup" parameter is:
132 setup => {
134 'system_ssh_config' => '/etc/ssh/ssh_config',
135 'user_ssh_config' => "~/.ssh/config"
136 }
137 On the other hand, system wide config file is different on MacOS and
139 the test file must be copied in the correct location. When the value of
140 the "setup" hash is another hash, the key of this other hash is used as
141 to specify the target location for other OS (as returned by Perl $^O
142 variable:
143 setup => {
145 'system_ssh_config' => {
146 'darwin' => '/etc/ssh_config',
147 'default' => '/etc/ssh/ssh_config',
148 },
149 'user_ssh_config' => "~/.ssh/config"
150 }
151 See the actual Ssh and Sshd model tests
153 <https://github.com/dod38fr/config-model-
154 openssh/tree/master/t/model_tests.d>
155 Basic test specification
157 Each model subtest is specified in "<model>-test-conf.pl". This file
158 contains a set of global variables. (yes, global variables are often
159 bad ideas in programs, but they are handy for tests):
160 # config file name (used to copy test case into test wr_root/model_tests directory)
162 $conf_file_name = "fstab" ;
163 # config dir where to copy the file (optional)
164 #$conf_dir = "etc" ;
165 # home directory for this test
166 $home_for_test = '/home/joe' ;
167 Here, "t0" file will be copied in
169 "wr_root/model_tests/test-t0/etc/fstab".
170 # config model name to test
172 $model_to_test = "Fstab" ;
173 # list of tests. This modules looks for @tests global variable
175 @tests = (
176 {
177 # test name
178 name => 't0',
179 # add optional specification here for t0 test
180 },
181 {
182 name => 't1',
183 # add optional specification here for t1 test
184 },
185 );
186 1; # to keep Perl happy
188 You can suppress warnings by specifying "no_warnings => 1". On the
190 other hand, you may also want to check for warnings specified to your
191 model. In this case, you should avoid specifying "no_warnings" here and
192 specify warning tests or warning filters as mentioned below.
193 See actual fstab test <https://github.com/dod38fr/config-
195 model/blob/master/t/model_tests.d/fstab-test-conf.pl>.
196 Skip a test
198 A test file can be skipped using $skip global variable.
199 In this example, test is skipped when not running on a Debian system:
201 eval { require AptPkg::Config; };
203 $skip = ( $@ or not -r '/etc/debian_version' ) ? 1 : 0;
204 Internal tests or backend tests
206 Some tests will require the creation of a configuration class dedicated
207 for test (typically to test corner cases on a backend).
208 This test class can be created directly in the test specification by
210 calling create_config_class on $model variable. See for instance the
211 layer test <https://github.com/dod38fr/config-
212 model/blob/master/t/model_tests.d/layer-test-conf.pl> or the test for
213 shellvar backend <https://github.com/dod38fr/config-
214 model/blob/master/t/model_tests.d/backend-shellvar-test-conf.pl>.
215 Test specification with arbitrary file names
217 In some models like "Multistrap", the config file is chosen by the
218 user. In this case, the file name must be specified for each tests
219 case:
220 # not needed if test file is named multistrap-test-conf.pl
222 $model_to_test = "Multistrap";
223 @tests = (
225 {
226 name => 'arm',
227 config_file => '/home/foo/my_arm.conf',
228 check => {},
229 },
230 );
231 See the actual multistrap test <https://github.com/dod38fr/config-
233 model/blob/master/t/model_tests.d/multistrap-test-conf.pl>.
234 Backend argument
236 Some application like systemd requires a backend argument specified by
237 User (e.g. a service name for systemd). The parameter "backend_arg" can
238 be specified to emulate this behavior.
239 Re-use test data
241 When the input data for test is quite complex (several files), it may
242 be interested to re-use these data for other tests case. Knowing that
243 test name must must unique, you can re-use test data with "data_from"
244 parameter. For instance:
245 @tests = (
247 {
248 name => 'some-test',
249 # ...
250 },
251 {
252 name => 'some-other-test',
253 data_from => 'some-test', # re-use data from test above
254 # ...
255 },
256 See plainfile backend test <https://github.com/dod38fr/config-
258 model/blob/master/t/model_tests.d/backend-plainfile-test-conf.pl> for a
259 real life example.
260 Test scenario
262 Each subtest follow a sequence explained below. Each step of this
263 sequence may be altered by adding specification in
264 "<model-to-test>-test-conf.pl":
265 · Setup test in "wr_root/model_tests/<subtest name>/". If your
267 configuration file layout depend on the target system, you will
268 have to specify the path using "setup" parameter. See "Test file
269 layout depending on system".
270 · Create configuration instance, load config data and check its
272 validity. Use "load_check => 'no'" if your file is not valid.
273 · Check for config data warnings. You should pass the list of
275 expected warnings that are emitted through Log::Log4Perl. The array
276 ref is passed as is to the "expect" function of "expect" in
277 Test::Log::Lo4Perl. E.g:
278 log4perl_load_warnings => [
280 [ 'Tree.Node', (warn => qr/deprecated/) x 2 ] ,
281 [ 'Tree.Element.Value' , ( warn => qr/skipping/) x 2 ]
282 ]
283 The Log classes are specified in "cme/Logging".
285 Log levels below "warn" are ignored.
287 Config::Model is currently transitioning from traditional "warn" to
289 warn logs. To avoid breaking all tests based on this module, the
290 warnings are emitted through Log::Log4Perl only when
291 c<$::_use_log4perl_to_warn> is set. This hack will be removed once
292 all warnings checks in tests are ported to log4perl checks.
293 · DEPRECATED. Check for config data warning. You should pass the list
295 of expected warnings. E.g.
296 load_warnings => [ qr/Missing/, (qr/deprecated/) x 3 , ],
298 Use an empty array_ref to mask load warnings.
300 · Optionally run update command:
302 update => {
304 [ returns => 'foo' , ]
305 no_warnings => [ 0 | 1 ], # default 0
306 quiet => [ 0 | 1], # default 0, passed to update method
307 loag4perl_update_warnings => [ ... ] # Test::Log::Log4Perl::expect arguments
308 }
309 Where:
311 · "returns" is the expected return value (optional).
313 · "no_warnings" to suppress the warnings coming from
315 Config::Model::Value. Note that "no_warnings => 1" may be
316 useful for less verbose test.
317 · "quiet" to suppress progress messages during update.
319 · "log4perl_update_warnings" is used to check the warnings
321 produced
322 during update. The argument is passed to "expect" function of
323 Test::Log::Log4Perl. See "load_warnings" parameter above for
324 more
325 details.
326 · DEPRECATED. "update_warnings" is an array ref of quoted regexp
328 (See qr operator) to check the warnings produced during update.
329 use "update => []" to check that no warnings are issued during
330 update.
331 All other arguments are passed to "update" method.
333 · Optionally load configuration data. You should design this config
335 data to suppress any error or warning mentioned above. E.g:
336 load => 'binary:seaview Synopsis="multiplatform interface for sequence alignment"',
338 See Config::Model::Loader for the syntax of the string accepted by
340 "load" parameter.
341 · Optionally, run a check before running apply_fix (if any). This
343 step is useful to check warning messages:
344 check_before_fix => {
346 dump_errors => [ ... ] # optional, see below
347 load4perl_dump_warnings => [ ... ] # optional, see below
348 }
349 Use "dump_errors" if you expect issues:
351 check_before_fix => {
353 dump_errors => [
354 # the issues and a way to fix the issue using Config::Model::Node::load
355 qr/mandatory/ => 'Files:"*" Copyright:0="(c) foobar"',
356 qr/mandatory/ => ' License:FOO text="foo bar" ! Files:"*" License short_name="FOO" '
357 ],
358 }
359 Likewise, specify any expected warnings:
361 check_before_fix => {
363 log4perl_dump_warnings => [ ... ],
364 }
365 "log4perl_dump_warnings" passes the array ref content to "expect"
367 function of Test::Log::Log4Perl.
368 Both "log4perl_dump_warnings" and "dump_errors" can be specified in
370 "check_before_fix" hash.
371 · Optionally, call apply_fixes:
373 apply_fix => 1,
375 · Call dump_tree to check the validity of the data after optional
377 "apply_fix". This step is not optional.
378 As with "check_before_fix", both "dump_errors" or "dump_warnings"
380 can be used.
381 · Run specific content check to verify that configuration data was
383 retrieved correctly:
384 check => {
386 'fs:/proc fs_spec', "proc" ,
387 'fs:/proc fs_file', "/proc" ,
388 'fs:/home fs_file', "/home",
389 },
390 The keys of the hash points to the value to be checked using the
392 syntax described in "grab(...)" in Config::Model::AnyThing:.
393 You can run check using different check modes (See "fetch( ... )"
395 in Config::Model::Value) by passing a hash ref instead of a scalar
396 :
397 check => {
399 'sections:debian packages:0' , { mode => 'layered', value => 'dpkg-dev' },
400 'sections:base packages:0', { mode => 'layered', value => "gcc-4.2-base' },
401 },
402 The whole hash content (except "value") is passed to grab and
404 fetch
405 A regexp can also be used to check value:
407 check => {
409 "License text" => qr/gnu/i,
410 "License text" => { mode => 'custom', value => qr/gnu/i },
411 }
412 · Verify if a hash contains one or more keys (or keys matching a
414 regexp):
415 has_key => [
417 'sections' => 'debian', # sections must point to a hash element
418 'control' => [qw/source binary/],
419 'copyright Files' => qr/.c$/,
420 'copyright Files' => [qr/\.h$/], qr/\.c$/],
421 ],
422 · Verify that a hash has not a key (or a key matching a regexp):
424 has_not_key => [
426 'copyright Files' => qr/.virus$/ # silly, isn't ?
427 ],
428 · Verify annotation extracted from the configuration file comments:
430 verify_annotation => {
432 'source Build-Depends' => "do NOT add libgtk2-perl to build-deps (see bug #554704)",
433 'source Maintainer' => "what a fine\nteam this one is",
434 },
435 · Write back the config data in "wr_root/model_tests/<subtest
437 name>/". Note that write back is forced, so the tested
438 configuration files are written back even if the configuration
439 values were not changed during the test.
440 You can skip warning when writing back with the global :
442 no_warnings => 1,
444 · Check the content of the written files(s) with
446 Test::File::Contents. Tests can be grouped in an array ref:
447 file_contents => {
449 "/home/foo/my_arm.conf" => "really big string" ,
450 "/home/bar/my_arm.conf" => [ "really big string" , "another"], ,
451 }
452 file_contents_like => {
454 "/home/foo/my_arm.conf" => [ qr/should be there/, qr/as well/ ] ,
455 }
456 file_contents_unlike => {
458 "/home/foo/my_arm.conf" => qr/should NOT be there/ ,
459 }
460 · Check the mode of the written files:
462 file_mode => {
464 "~/.ssh/ssh_config" => 0600, # octal mode
465 "debian/stuff.postinst" => 0755,
466 }
467 Only the last four octets of the mode are tested. I.e. the test is
469 done with " $file_mode & 07777 "
470 Note: this test is skipped on Windows
472 · Check added or removed configuration files. If you expect changes,
474 specify a subref to alter the file list:
475 file_check_sub => sub {
477 my $list_ref = shift ;
478 # file added during tests
479 push @$list_ref, "/debian/source/format" ;
480 },
481 Note that actual and expected file lists are sorted before check,
483 adding a file can be done with "push".
484 · Copy all config data from "wr_root/model_tests/<subtest name>/" to
486 "wr_root/model_tests/<subtest name>-w/". This steps is necessary to
487 check that configuration written back has the same content as the
488 original configuration.
489 · Create a second configuration instance to read the conf file that
491 was just copied (configuration data is checked.)
492 · You can skip the load check if the written file still contain
494 errors (e.g. some errors were ignored and cannot be fixed) with
495 "load_check2 => 'no'"
496 · Optionally load configuration data in the second instance. You
498 should design this config data to suppress any error or warning
499 that occur in the step below. E.g:
500 load2 => 'binary:seaview',
502 See Config::Model::Loader for the syntax of the string accepted by
504 "load2" parameter.
505 · Compare data read from original data.
507 · Run specific content check on the written config file to verify
509 that configuration data was written and retrieved correctly:
510 wr_check => {
512 'fs:/proc fs_spec' => "proc" ,
513 'fs:/proc fs_file' => "/proc",
514 'fs:/home fs_file' => "/home",
515 },
516 Like the "check" item explained above, you can run "wr_check" using
518 different check modes.
519 Running the test
521 Run all tests with one of these commands:
522 prove -l t/model_test.t :: [ t|l|e [ <model_name> [ <regexp> ]]]
524 perl -Ilib t/model_test.t [ t|l|e [ <model_name> [ <regexp> ]]]
525 By default, all tests are run on all models.
527 You can pass arguments to "t/model_test.t":
529 · a bunch of letters. 't' to get test traces. 'e' to get stack trace
531 in case of errors, 'l' to have logs. All other letters are ignored.
532 E.g.
533 # run with log and error traces
535 prove -lv t/model_test.t :: el
536 · The model name to tests. E.g.:
538 # run only fstab tests
540 prove -lv t/model_test.t :: x fstab
541 · A regexp to filter subtest E.g.:
543 # run only fstab tests foobar subtest
545 prove -lv t/model_test.t :: x fstab foobar
546 # run only fstab tests foo subtest
548 prove -lv t/model_test.t :: x fstab '^foo$'
549
551 · LCDproc <http://lcdproc.org> has a single configuration file:
552 "/etc/LCDd.conf". Here's LCDproc test layout
553 <https://github.com/dod38fr/config-model-
554 lcdproc/tree/master/t/model_tests.d> and the test specification
555 <https://github.com/dod38fr/config-model-
556 lcdproc/blob/master/t/model_tests.d/lcdd-test-conf.pl>
557 · Dpkg packages are constructed from several files. These files are
559 handled like configuration files by Config::Model::Dpkg. The test
560 layout <http://anonscm.debian.org/gitweb/?p=pkg-
561 perl/packages/libconfig-model-dpkg-
562 perl.git;a=tree;f=t/model_tests.d;hb=HEAD> features test with
563 multiple file in dpkg-examples
564 <http://anonscm.debian.org/gitweb/?p=pkg-perl/packages/libconfig-
565 model-dpkg-perl.git;a=tree;f=t/model_tests.d/dpkg-
566 examples;hb=HEAD>. The test is specified in dpkg-test-conf.pl
567 <http://anonscm.debian.org/gitweb/?p=pkg-perl/packages/libconfig-
568 model-dpkg-perl.git;a=blob_plain;f=t/model_tests.d/dpkg-test-
569 conf.pl;hb=HEAD>
570 · multistrap-test-conf.pl <https://github.com/dod38fr/config-
572 model/blob/master/t/model_tests.d/multistrap-test-conf.pl> and
573 multistrap-examples <https://github.com/dod38fr/config-
574 model/tree/master/t/model_tests.d/multistrap-examples> specify a
575 test where the configuration file name is not imposed by the
576 application. The file name must then be set in the test
577 specification.
578 · backend-shellvar-test-conf.pl <https://github.com/dod38fr/config-
580 model/blob/master/t/model_tests.d/backend-shellvar-test-conf.pl> is
581 a more complex example showing how to test a backend. The test is
582 done creating a dummy model within the test specification.
583
585 · Config::Model
586 · Test::More
588
590 Dominique Dumont
591
593 This software is Copyright (c) 2013-2018 by Dominique Dumont.
594 This is free software, licensed under:
596 The GNU Lesser General Public License, Version 2.1, February 1999
598
600 Websites
601 The following websites have more information about this module, and may
602 be of help to you. As always, in addition to those websites please use
603 your favorite search engine to discover more resources.
604 · Search CPAN
606 The default CPAN search engine, useful to view POD in HTML format.
608 <http://search.cpan.org/dist/Config-Model-Tester>
610 · AnnoCPAN
612 The AnnoCPAN is a website that allows community annotations of Perl
614 module documentation.
615 <http://annocpan.org/dist/Config-Model-Tester>
617 · CPAN Ratings
619 The CPAN Ratings is a website that allows community ratings and
621 reviews of Perl modules.
622 <http://cpanratings.perl.org/d/Config-Model-Tester>
624 · CPANTS
626 The CPANTS is a website that analyzes the Kwalitee ( code metrics )
628 of a distribution.
629 <http://cpants.cpanauthors.org/dist/Config-Model-Tester>
631 · CPAN Testers
633 The CPAN Testers is a network of smoke testers who run automated
635 tests on uploaded CPAN distributions.
636 <http://www.cpantesters.org/distro/C/Config-Model-Tester>
638 · CPAN Testers Matrix
640 The CPAN Testers Matrix is a website that provides a visual
642 overview of the test results for a distribution on various
643 Perls/platforms.
644 <http://matrix.cpantesters.org/?dist=Config-Model-Tester>
646 · CPAN Testers Dependencies
648 The CPAN Testers Dependencies is a website that shows a chart of
650 the test results of all dependencies for a distribution.
651 <http://deps.cpantesters.org/?module=Config::Model::Tester>
653 Bugs / Feature Requests
655 Please report any bugs or feature requests by email to "ddumont at
656 cpan.org", or through the web interface at
657 <https://github.com/dod38fr/config-model-tester/issues>. You will be
658 automatically notified of any progress on the request by the system.
659 Source Code
661 The code is open to the world, and available for you to hack on. Please
662 feel free to browse it and play with it, or whatever. If you want to
663 contribute patches, please send me a diff or prod me to pull from your
664 repository :)
665 <http://github.com/dod38fr/config-model-tester.git>
667 git clone git://github.com/dod38fr/config-model-tester.git
669perl v5.28.0 2018-07-14 Config::Model::Tester(3)