1TESTING.SH(1) User Contributed Perl Documentation TESTING.SH(1)
2
6 BeakerLib - testing - asserting functions, watchdog and report
7
9 This file contains functions related directly to testing. These
10 functions are various asserts affecting final result of the phase.
11 Watchdog and the report result function is included as well.
12
14 Manual Asserts
15 rlPass
16 Manual assertion, asserts and logs PASS.
18 rlPass comment
20 comment
22 Short test summary.
23 Returns 0 and asserts PASS.
25 rlFail
27 Manual assertion, asserts and logs FAIL.
29 rlFail comment
31 comment
33 Short test summary.
34 Returns 1 and asserts FAIL.
36 Arithmetic Asserts
38 rlAssert0
39 Assertion checking for the equality of parameter to zero.
41 rlAssert0 comment value
43 comment
45 Short test summary, e.g. "Test if compilation ended successfully".
46 value
48 Integer value (usually return code of a command).
49 Returns 0 and asserts PASS when "value == 0".
51 rlAssertEquals
53 Assertion checking for the equality of two parameters.
55 rlAssertEquals comment value1 value2
57 comment
59 Short test summary, e.g. "Test if all 3 packages have been
60 downloaded".
61 value1
63 First parameter to compare, can be a number or a string
64 value2
66 Second parameter to compare, can be a number or a string
67 Returns 0 and asserts PASS when "value1 == value2".
69 rlAssertNotEquals
71 Assertion checking for the non-equality of two parameters.
73 rlAssertNotEquals comment value1 value2
75 comment
77 Short test summary, e.g. "Test if return code is not 139".
78 value1
80 First parameter to compare, can be a number or a string
81 value2
83 Second parameter to compare, can be a number or a string
84 Returns 0 and asserts PASS when "value1 != value2".
86 rlAssertGreater
88 Assertion checking whether first parameter is greater than the second
90 one.
91 rlAssertGreater comment value1 value2
93 comment
95 Short test summary, e.g. "Test whether there are running more
96 instances of program."
97 value1
99 Integer value.
100 value2
102 Integer value.
103 Returns 0 and asserts PASS when "value1 > value2".
105 rlAssertGreaterOrEqual
107 Assertion checking whether first parameter is greater or equal to the
109 second one.
110 rlAssertGreaterOrEqual comment value1 value2
112 comment
114 Short test summary (e.g. "There should present at least one...")
115 value1
117 Integer value.
118 value2
120 Integer value.
121 Returns 0 and asserts PASS when "value1 X= value2".
123 rlAssertLesser
125 Assertion checking whether first parameter is lesser than the second
127 one.
128 rlAssertLesser comment value1 value2
130 comment
132 Short test summary, e.g. "Test whether there are running more
133 instances of program."
134 value1
136 Integer value.
137 value2
139 Integer value.
140 Returns 0 and asserts PASS when "value1 < value2".
142 rlAssertLesserOrEqual
144 Assertion checking whether first parameter is lesser or equal to the
146 second one.
147 rlAssertLesserOrEqual comment value1 value2
149 comment
151 Short test summary (e.g. "There should present at least one...")
152 value1
154 Integer value.
155 value2
157 Integer value.
158 Returns 0 and asserts PASS when "value1 X= value2".
160 File Asserts
162 rlAssertExists
163 Assertion checking for the existence of a file or a directory.
165 rlAssertExists file|directory
167 file|directory
169 Path to the file or directory.
170 Returns 0 and asserts PASS when "file" exists.
172 rlAssertNotExists
174 Assertion checking for the non-existence of a file or a directory.
176 rlAssertNotExists file|directory
178 file|directory
180 Path to the file or directory.
181 Returns 0 and asserts PASS when "file" does not exist.
183 rlAssertGrep
185 Assertion checking if the file contains a pattern.
187 rlAssertGrep pattern file [options]
189 pattern
191 Regular expression to be searched for.
192 file
194 Path to the file.
195 options
197 Optional parameters to be passed to grep, default is "-q". Can be
198 used to perform case insensitive matches (-i), or using extended
199 (-E) or perl (-P) regular expressions.
200 Returns 0 and asserts PASS when "file" exists and contains given
202 "pattern".
203 rlAssertNotGrep
205 Assertion checking that the file does not contain a pattern.
207 rlAssertNotGrep pattern file [options]
209 pattern
211 Regular expression to be searched for.
212 file
214 Path to the file.
215 options
217 Optional parameters to be passed to grep, default is "-q". Can be
218 used to perform case insensitive matches (-i), or using extended
219 (-E) or perl (-P) regular expressions.
220 Returns 0 and asserts PASS when "file" exists and does not contain
222 given "pattern".
223 rlAssertDiffer
225 Assertion checking that two files differ (are not identical).
227 rlAssertDiffer file1 file2
229 file1
231 Path to first file1
232 file2
234 Path to second file
235 Returns 0 and asserts PASS when "file1" and "file2" differs.
237 rlAssertNotDiffer
239 Assertion checking that two files do not differ (are identical).
241 rlAssertNotDiffer file1 file2
243 file1
245 Path to first file1
246 file2
248 Path to second file
249 Returns 0 and asserts PASS when "file1" and "file2" do not differ.
251 Run, Watch, Report
253 rlRun
254 Run command with optional comment and make sure its exit code matches
256 expectations.
257 rlRun [-t] [-l] [-c] [-s] command [status[,status...] [comment]]
259 -t If specified, stdout and stderr of the command output will be
261 tagged with strings 'STDOUT: ' and 'STDERR: '.
262 -l If specified, output of the command (tagged, if -t was specified)
264 is logged using rlLog function. This is intended for short outputs,
265 and therefore only last 50 lines are logged this way. Longer
266 outputs should be analysed separately, or uploaded via rlFileSubmit
267 or rlBundleLogs.
268 -c Same as "-l", but only log the command output if it failed.
270 -s Store stdout and stderr to a file (mixed together, as the user
272 would see it on a terminal) and set $rlRun_LOG variable to name of
273 the file. $rlRun_LOG is now actually an array where the first index
274 holds the last reference to the file. Thus its behavior is not
275 changed if used without an index. The array is consumed by the
276 rlJournalEnd function to remove the leftover files, so no manual
277 files removal is needed anymore. Note that if you need to use such
278 a file after calling the rlJournalEnd function you need to create
279 your own copy of the respective file. When -t option is used, the
280 content of the file becomes tagged too.
281 If the -s option is not used, $rlRun_LOG is not modified and keeps
283 its content from the last "rlRun -s".
284 command
286 Command to run.
287 status
289 Expected exit code(s). Optional, default 0. If you expect more exit
290 codes, separate them with comma (e.g. "0,1" when both 0 and 1 are
291 OK and expected), or use from-to notation (i.e. "2-5" for
292 "2,3,4,5"), or combine them (e.g. "2-4,26" for "2,3,4,26").
293 comment
295 Short summary describing the action (optional, but recommended -
296 explain what you are doing here).
297 Returns the exit code of the command run. Asserts PASS when command\'s
299 exit status is in the list of expected exit codes.
300 Note:
302 • The output of rlRun is buffered when using "-t", "-l" or "-s"
304 option (they use unix pipes, which are buffered by nature). If you
305 need an unbuffered output just make sure that "expect" package is
306 installed on your system (its "unbuffer" tool will automatically be
307 used to produce unbuffered output).
308 • Be aware that there are some variables which can collide with your
310 code executed within rlRun. You should avoid using
311 __INTERNAL_rlRun_* variables.
312 • When any of "-t" "-l", "-c", or "-s" option is used, special file
314 descriptors 111 and 112 are used to avoid the issue with incomplete
315 log file, bz1361246. As there might be an indefinite loop, there's
316 a timeout of two minutes implemented as a fix for bz1416796. Also
317 an error message is issued to signal the possibility of running
318 subprocess which keeps the file descriptors open.
319 Do not use these options if you expect process forking and
321 continuouse run. Try your own apropriate solution instead.
322 Warning: using "unbuffer" tool is now disabled because of bug 547686.
324 rlWatchdog
326 Run "command". If it does not finish in specified time, then kill it
328 using "signal". Note that the command is executed in a sub-shell so
329 modified environment (e.g. set variables) is not relfected in the test
330 environment.
331 rlWatchdog command timeout [signal] [callback]
333 command
335 Command to run.
336 timeout
338 Timeout to wait, in seconds.
339 signal
341 Signal to use (optional, default KILL).
342 callback
344 Callback function to be called before the signal is send (optional,
345 none by default). The callback function will have one argument
346 available -- PGID of the process group.
347 Returns 0 if the command ends normally, without need to be killed.
349 rlReport
351 Report test result to the test harness. The command to be used for
353 reporting is set by the respective plugin. You can also use the
354 $BEAKERLIB_COMMAND_REPORT_RESULT variable to use your custom command.
355 rlReport name result [score] [log]
357 name
359 Name of the test result.
360 result
362 Result (one of PASS, WARN, FAIL). If called with something else,
363 will use WARN.
364 score
366 Test score (optional).
367 log Optional log file to be submitted instead of default "OUTPUTFILE".
369 rlCmpVersion
371 Compare two given versions composed by numbers and letters divided by
373 dot (.), underscore (_), or dash (-).
374 rlCmpVersion ver1 ver2
376 If ver1 = ver2, sign '=' is printed to stdout and 0 is returned. If
378 ver1 > ver2, sign '>' is printed to stdout and 1 is returned. If ver1
379 < ver2, sign '<' is printed to stdout and 2 is returned.
380 rlTestVersion
382 Test releation between two given versions based on given operator.
384 rlTestVersion ver1 op ver2
386 op Operator defining the logical expression. It can be '=', '==',
388 '!=', '<', '<=', '=<', '>', '>=', or '=>'.
389 Returns 0 if the expresison ver1 op ver2 is true; 1 if the expression
391 is false and 2 if something went wrong.
392 Release Info
394 rlIsRHEL
395 rlIsRHEL [VERSION_SPEC]...
397 VERSION_SPEC
399 Argument specifies version of particular RHEL distribution, eg. 8,
400 8.4 ">8.4".
401 For more details see description of "rlIsOSVersion".
403 Returns 0 when we're running on RHEL. With given version specification
405 in arguments returns 0 if the particular RHEL version is running.
406 Prototype:
408 rlIsRHEL
410 Returns 0 if we are running on RHEL.
412 rlIsRHEL 7 '>=8.5'
414 Returns 0 if we are running any RHEL 7 or RHEL 8.5 and higher.
416 rlIsFedora
418 rlIsFedora [VERSION_SPEC]...
420 VERSION_SPEC
422 Argument specifies version of particular Fedora distribution, eg.
423 30, ">30".
424 For more details see description of "rlIsOSVersion".
426 Returns 0 when we're running on Fedora. With given version
428 specification in arguments returns 0 if the particular Fedora version
429 is running.
430 Prototype:
432 rlIsFedora
434 Returns 0 if we are running on Fedora.
436 rlIsFedora 30 32
438 Returns 0 if we are running Fedora 30 or 32.
440 rlIsCentOS
442 rlIsCentOS [VERSION_SPEC]...
444 VERSION_SPEC
446 Argument specifies version of particular CentOS distribution, eg.
447 7, ">7".
448 For more details see description of "rlIsOSVersion".
450 Returns 0 when we're running on CentOS. With given version
452 specification in arguments returns 0 if the particular CentOS version
453 is running.
454 Prototype:
456 rlIsCentOS
458 Returns 0 if we are running on CentOS.
460 rlIsCentOS 7.1 6
462 Returns 0 if we are running CentOS 7.1 or any CentOS 6.
464 rlIsOS
466 rlIsOS ID
468 ID Argument is based on contents of /etc/os-release. Possible options
470 of ID are e.g. fedora, rhel, centos, ol.
471 Returns 0 when we're running on system with respective ID. Returns 1
473 when parameter does not match with ID in os-release. Returns 2 when
474 there is no ID defined. Returns 3 when no argument is given.
475 Prototype:
477 rlIsOS rhel
479 Returns 0 if we are running on RHEL.
481 rlIsOSLike
483 rlIsOSLike ID_LIKE
485 ID_LIKE
487 Argument is based on contents of /etc/os-release. Possible options
488 of ID_LIKE are e.g. fedora, rhel.
489 Returns 0 when we're running on system with requested ID_LIKE. Returns
491 1 when parameter does not match with ID nor ID_LIKE in os-release.
492 Returns 2 when there is no ID or ID_LIKE defined. Returns 3 when no
493 argument is given.
494 Prototype:
496 rlIsOSLike rhel
498 Returns 0 if we are running on RHEL, CentOS, Rocky Linux, etc..
500 rlIsOSLike fedora
502 Returns 0 if we are running on Fedora, RHEL, CentOS, Oracle Linux, etc..
504 rlIsOSVersion
506 rlIsOsVersion VERSION_SPEC...
508 VERSION_SPEC
510 Parameter is based on VERSION_ID in /etc/os-release. It consists
511 of either "major" or "major"."minor" refering to a particular
512 release.
513 It accepts multiple arguments separated by space (8.1 8.2 8.3 9).
515 To specify a group of distributions optional operator can be passed
517 in argument together with the version number as one string
518 ('>8.5'). Operator can be '<', '<=', '=<', '=', '>', '>=' matching
519 whether the currently installed version is lesser, lesser or equal,
520 equal, equal or greater, greater than the version number supplied
521 as second half of the argument.
522 Note that when using >/< operators you have to either put the
524 argument in quotes or escape the operators to avoid them being
525 interpreted as bash redirection operator.
526 Returns 0 when we're running distribution of the particular version
528 requested by the argument.
529 It usually follows after "rlIsOS" and "rlIsOSLike".
531 Be cautious when using together with "rlIsOSLike" as different
533 distributions may use different versioning schema.
534 Prototype:
536 rlIsOSVersion 6 7 9
538 Returns 0 if we are running distribution with VERSION_ID 6, 7 or 9.
540 rlIsOSVersion 7.8 8
542 Returns 0 if we are running distribution 7.8 or any 8.
544 rlIsOSVersion ">=7.5" or rlIsOSVersion \>=7.5
546 Returns 0 if we are running distribution 7.5 or higher (both minors or majors).
548 Note:
550 rlIsOSVersion '<7.5' || rlIsOSVersion '<8.5'
552 would also cover 7.9 as it is less than 8.5, which is not what you want.
554 So if you want to construct a condition for a distribution <7.5 within the major 7 or
555 a distribution <8.5 within the major 8 you actually need to use following construct:
556 rlIsOSVersion 7 && rlIsOSVersion '<7.5' || rlIsOSVersion 8 && rlIsOSVersion '<8.5'
558 This returns 0 when running distribution less than 7.5 and less then 8.5, but not 7.9 (nor 6.9).
560 rlIsRHELLike
562 rlIsRHELLike [VERSION_SPEC]...
564 VERSION_SPEC
566 Argument specifies version of particular RHEL distribution, eg. 8,
567 8.4 ">8.4".
568 For more details see description of "rlIsOSVersion".
570 Returns 0 when we're running on RHEL-like distribution. These are
572 considered to be RHEL, CentOS, Rocky Linux, etc.. With given version
573 specification in arguments returns 0 if the particular version of RHEL-
574 like distribution is running.
575 Prototype:
577 rlIsRHELLike
579 Returns 0 if we are running on RHEL-like system.
581 rlIsRHELLike ">=6"
583 Returns 0 if we are running on RHEL-like distribution of version 6.0 or higher.
585
587 • Ondrej Hudlicky <ohudlick@redhat.com>
588 • Petr Muller <pmuller@redhat.com>
590 • Jan Hutar <jhutar@redhat.com>
592 • Petr Splichal <psplicha@redhat.com>
594 • Ales Zelinka <azelinka@redhat.com>
596perl v5.34.0 2021-11-09 TESTING.SH(1)