1TESTING.SH(1) User Contributed Perl Documentation TESTING.SH(1)
2
6 BeakerLib - testing - asserting functions, watchdog and report
7
9 This file contains functions related directly to testing. These
10 functions are various asserts affecting final result of the phase.
11 Watchdog and the report result function is included as well.
12
14 Manual Asserts
15 rlPass
16 Manual assertion, asserts and logs PASS.
18 rlPass comment
20 comment
22 Short test summary.
23 Returns 0 and asserts PASS.
25 rlFail
27 Manual assertion, asserts and logs FAIL.
29 rlFail comment
31 comment
33 Short test summary.
34 Returns 1 and asserts FAIL.
36 Arithmetic Asserts
38 rlAssert0
39 Assertion checking for the equality of parameter to zero.
41 rlAssert0 comment value
43 comment
45 Short test summary, e.g. "Test if compilation ended successfully".
46 value
48 Integer value (usually return code of a command).
49 Returns 0 and asserts PASS when "value == 0".
51 rlAssertEquals
53 Assertion checking for the equality of two parameters.
55 rlAssertEquals comment value1 value2
57 comment
59 Short test summary, e.g. "Test if all 3 packages have been
60 downloaded".
61 value1
63 First parameter to compare, can be a number or a string
64 value2
66 Second parameter to compare, can be a number or a string
67 Returns 0 and asserts PASS when "value1 == value2".
69 rlAssertNotEquals
71 Assertion checking for the non-equality of two parameters.
73 rlAssertNotEquals comment value1 value2
75 comment
77 Short test summary, e.g. "Test if return code is not 139".
78 value1
80 First parameter to compare, can be a number or a string
81 value2
83 Second parameter to compare, can be a number or a string
84 Returns 0 and asserts PASS when "value1 != value2".
86 rlAssertGreater
88 Assertion checking whether first parameter is greater than the second
90 one.
91 rlAssertGreater comment value1 value2
93 comment
95 Short test summary, e.g. "Test whether there are running more
96 instances of program."
97 value1
99 Integer value.
100 value2
102 Integer value.
103 Returns 0 and asserts PASS when "value1 > value2".
105 rlAssertGreaterOrEqual
107 Assertion checking whether first parameter is greater or equal to the
109 second one.
110 rlAssertGreaterOrEqual comment value1 value2
112 comment
114 Short test summary (e.g. "There should present at least one...")
115 value1
117 Integer value.
118 value2
120 Integer value.
121 Returns 0 and asserts PASS when "value1 X= value2".
123 rlAssertLesser
125 Assertion checking whether first parameter is lesser than the second
127 one.
128 rlAssertLesser comment value1 value2
130 comment
132 Short test summary, e.g. "Test whether there are running more
133 instances of program."
134 value1
136 Integer value.
137 value2
139 Integer value.
140 Returns 0 and asserts PASS when "value1 < value2".
142 rlAssertLesserOrEqual
144 Assertion checking whether first parameter is lesser or equal to the
146 second one.
147 rlAssertLesserOrEqual comment value1 value2
149 comment
151 Short test summary (e.g. "There should present at least one...")
152 value1
154 Integer value.
155 value2
157 Integer value.
158 Returns 0 and asserts PASS when "value1 X= value2".
160 File Asserts
162 rlAssertExists
163 Assertion checking for the existence of a file or a directory.
165 rlAssertExists file|directory
167 file|directory
169 Path to the file or directory.
170 Returns 0 and asserts PASS when "file" exists.
172 rlAssertNotExists
174 Assertion checking for the non-existence of a file or a directory.
176 rlAssertNotExists file|directory
178 file|directory
180 Path to the file or directory.
181 Returns 0 and asserts PASS when "file" does not exist.
183 rlAssertGrep
185 Assertion checking if the file contains a pattern.
187 rlAssertGrep pattern file [options]
189 pattern
191 Regular expression to be searched for.
192 file
194 Path to the file.
195 options
197 Optional parameters to be passed to grep, default is "-q". Can be
198 used to perform case insensitive matches (-i), or using extended
199 (-E) or perl (-P) regular expressions.
200 Returns 0 and asserts PASS when "file" exists and contains given
202 "pattern".
203 rlAssertNotGrep
205 Assertion checking that the file does not contain a pattern.
207 rlAssertNotGrep pattern file [options]
209 pattern
211 Regular expression to be searched for.
212 file
214 Path to the file.
215 options
217 Optional parameters to be passed to grep, default is "-q". Can be
218 used to perform case insensitive matches (-i), or using extended
219 (-E) or perl (-P) regular expressions.
220 Returns 0 and asserts PASS when "file" exists and does not contain
222 given "pattern".
223 rlAssertDiffer
225 Assertion checking that two files differ (are not identical).
227 rlAssertDiffer file1 file2
229 file1
231 Path to first file1
232 file2
234 Path to second file
235 Returns 0 and asserts PASS when "file1" and "file2" differs.
237 rlAssertNotDiffer
239 Assertion checking that two files do not differ (are identical).
241 rlAssertNotDiffer file1 file2
243 file1
245 Path to first file1
246 file2
248 Path to second file
249 Returns 0 and asserts PASS when "file1" and "file2" do not differ.
251 Run, Watch, Report
253 rlRun
254 Run command with optional comment and make sure its exit code matches
256 expectations.
257 rlRun [-t] [-l] [-c] [-s] command [status[,status...] [comment]]
259 -t If specified, stdout and stderr of the command output will be
261 tagged with strings 'STDOUT: ' and 'STDERR: '.
262 -l If specified, output of the command (tagged, if -t was specified)
264 is logged using rlLog function. This is intended for short outputs,
265 and therefore only last 50 lines are logged this way. Longer
266 outputs should be analysed separately, or uploaded via rlFileSubmit
267 or rlBundleLogs.
268 -c Same as "-l", but only log the command output if it failed.
270 -s Store stdout and stderr to a file (mixed together, as the user
272 would see it on a terminal) and set $rlRun_LOG variable to name of
273 the file. Caller is responsible for removing the file. When -t
274 option is used, the content of the file becomes tagged too.
275 If the -s option is not used, $rlRun_LOG is not modified and keeps
277 its content from the last "rlRun -s".
278 command
280 Command to run.
281 status
283 Expected exit code(s). Optional, default 0. If you expect more exit
284 codes, separate them with comma (e.g. "0,1" when both 0 and 1 are
285 OK and expected), or use from-to notation (i.e. "2-5" for
286 "2,3,4,5"), or combine them (e.g. "2-4,26" for "2,3,4,26").
287 comment
289 Short summary describing the action (optional, but recommended -
290 explain what you are doing here).
291 Returns the exit code of the command run. Asserts PASS when command\'s
293 exit status is in the list of expected exit codes.
294 Note:
296 · The output of rlRun is buffered when using "-t", "-l" or "-s"
298 option (they use unix pipes, which are buffered by nature). If you
299 need an unbuffered output just make sure that "expect" package is
300 installed on your system (its "unbuffer" tool will automatically be
301 used to produce unbuffered output).
302 · Be aware that there are some variables which can collide with your
304 code executed within rlRun. You should avoid using
305 __INTERNAL_rlRun_* variables.
306 · When any of "-t" "-l", "-c", or "-s" option is used, special file
308 descriptors 111 and 112 are used to avoid the issue with incomplete
309 log file, bz1361246. As there might be an indefinite loop, there's
310 a timeout of two minutes implemented as a fix for bz1416796. Also
311 an error message is issued to signal the possibility of running
312 subprocess which keeps the file descriptors open.
313 Do not use these options if you expect process forking and
315 continuouse run. Try your own apropriate solution instead.
316 Warning: using "unbuffer" tool is now disabled because of bug 547686.
318 rlWatchdog
320 Run "command". If it does not finish in specified time, then kill it
322 using "signal". Note that the command is executed in a sub-shell so
323 modified environment (e.g. set variables) is not relfected in the test
324 environment.
325 rlWatchdog command timeout [signal] [callback]
327 command
329 Command to run.
330 timeout
332 Timeout to wait, in seconds.
333 signal
335 Signal to use (optional, default KILL).
336 callback
338 Callback function to be called before the signal is send (optional,
339 none by default). The callback function will have one argument
340 available -- PGID of the process group.
341 Returns 0 if the command ends normally, without need to be killed.
343 rlReport
345 Report test result to the test harness. The command to be used for
347 reporting is set by the respective plugin. You can also use the
348 $BEAKERLIB_COMMAND_REPORT_RESULT variable to use your custom command.
349 rlReport name result [score] [log]
351 name
353 Name of the test result.
354 result
356 Result (one of PASS, WARN, FAIL). If called with something else,
357 will use WARN.
358 score
360 Test score (optional).
361 log Optional log file to be submitted instead of default "OUTPUTFILE".
363 rlCmpVersion
365 Compare two given versions composed by numbers and letters divided by
367 dot (.), underscore (_), or dash (-).
368 rlCmpVersion ver1 ver2
370 If ver1 = ver2, sign '=' is printed to stdout and 0 is returned. If
372 ver1 > ver2, sign '>' is printed to stdout and 1 is returned. If ver1
373 < ver2, sign '<' is printed to stdout and 2 is returned.
374 rlTestVersion
376 Test releation between two given versions based on given operator.
378 rlTestVersion ver1 op ver2
380 op Operator defining the logical expression. It can be '=', '==',
382 '!=', '<', '<=', '=<', '>', '>=', or '=>'.
383 Returns 0 if the expresison ver1 op ver2 is true; 1 if the expression
385 is false and 2 if something went wrong.
386 Release Info
388 rlIsRHEL
389 rlIsRHEL [Num|opNum]
391 Num When used function returns 0 if the particular RHEL version is
393 running. Multiple arguments can be passed separated with space as
394 well as any particular release (5.1 5.2 5.3).
395 opNum
397 Argument consisting of operator and number written together as one
398 string. Operator can be '<', '<=', '=<', '=', '>', '>=' matching
399 whether the currently installed version is lesser, lesser or equal,
400 equal, equal or greater, greater than the version number supplied
401 as second half of the argument. Note that ie. '=5' (unlike just
402 '5') matches exactly 5 (5.0), not 5.N, where N > 0. Also note when
403 that using >/< operators you have to either put the argument in
404 quotes or escape the operators to avoid them being interpreted as
405 bash redirection operator.
406 Returns 0 when we're running on RHEL.
408 Note that
410 rlIsRHEL '<6.9' || rlIsRHEL '<7.5'
412 would also cover 6.10 as it is less than 7.5, which is not what you
414 want. So if you want to construct a condition for rhel<6.9 for rhel6
415 or rhel<7.5 for rhel7 you actually need to use following construct:
416 rlIsRHEL 6 && rlIsRHEL '<6.9' || rlIsRHEL 7 && rlIsRHEL '<7.5'
418 Prototype:
420 rlIsRHEL
422 Returns 0 if we are running on RHEL.
424 rlIsRHEL 4.8 5
426 Returns 0 if we are running RHEL 4.8 or any RHEL 5.
428 rlIsRHEL ">=6" or rlIsRHEL \>=6
430 Returns 0 if we are running RHEL 6 or higher.
432 rlIsFedora
434 rlIsFedora [Num|opNum]
436 Returns 0 when we're running on Fedora. With given number of version
438 as parameter returns 0 if the particular Fedora version is running.
439 Range matching can be used in the form used by rlIsRHEL.
440 rlIsFedora
442 Returns 0 if we are running on Fedora.
444 rlIsFedora 9 10
446 Returns 0 if we are running Fedora 9 or 10.
448 Release Info
450 rlIsCentOS
451 rlIsCentOS [Num|opNum]
453 Returns 0 when we're running on CentOS. With given number of version
455 as parameter returns 0 if the particular CentOS version is running.
456 Range matching can be used in the form used by rlIsRHEL.
457 rlIsCentOS
459 Returns 0 if we are running on CentOS.
461 rlIsCentOS 7.1 6
463 Returns 0 if we are running CentOS 7.1 or any CentOS 6.
465
467 · Ondrej Hudlicky <ohudlick@redhat.com>
468 · Petr Muller <pmuller@redhat.com>
470 · Jan Hutar <jhutar@redhat.com>
472 · Petr Splichal <psplicha@redhat.com>
474 · Ales Zelinka <azelinka@redhat.com>
476perl v5.32.1 2021-03-25 TESTING.SH(1)