1TESTING.SH(1) User Contributed Perl Documentation TESTING.SH(1)
2
6 BeakerLib - testing - asserting functions, watchdog and report
7
9 This file contains functions related directly to testing. These
10 functions are various asserts affecting final result of the phase.
11 Watchdog and the report result function is included as well.
12
14 Manual Asserts
15 rlPass
16 Manual assertion, asserts and logs PASS.
18 rlPass comment
20 comment
22 Short test summary.
23 Returns 0 and asserts PASS.
25 rlFail
27 Manual assertion, asserts and logs FAIL.
29 rlFail comment
31 comment
33 Short test summary.
34 Returns 1 and asserts FAIL.
36 Arithmetic Asserts
38 rlAssert0
39 Assertion checking for the equality of parameter to zero.
41 rlAssert0 comment value
43 comment
45 Short test summary, e.g. "Test if compilation ended successfully".
46 value
48 Integer value (usually return code of a command).
49 Returns 0 and asserts PASS when "value == 0".
51 rlAssertEquals
53 Assertion checking for the equality of two parameters.
55 rlAssertEquals comment value1 value2
57 comment
59 Short test summary, e.g. "Test if all 3 packages have been
60 downloaded".
61 value1
63 First parameter to compare, can be a number or a string
64 value2
66 Second parameter to compare, can be a number or a string
67 Returns 0 and asserts PASS when "value1 == value2".
69 rlAssertNotEquals
71 Assertion checking for the non-equality of two parameters.
73 rlAssertNotEquals comment value1 value2
75 comment
77 Short test summary, e.g. "Test if return code is not 139".
78 value1
80 First parameter to compare, can be a number or a string
81 value2
83 Second parameter to compare, can be a number or a string
84 Returns 0 and asserts PASS when "value1 != value2".
86 rlAssertGreater
88 Assertion checking whether first parameter is greater than the second
90 one.
91 rlAssertGreater comment value1 value2
93 comment
95 Short test summary, e.g. "Test whether there are running more
96 instances of program."
97 value1
99 Integer value.
100 value2
102 Integer value.
103 Returns 0 and asserts PASS when "value1 > value2".
105 rlAssertGreaterOrEqual
107 Assertion checking whether first parameter is greater or equal to the
109 second one.
110 rlAssertGreaterOrEqual comment value1 value2
112 comment
114 Short test summary (e.g. "There should present at least one...")
115 value1
117 Integer value.
118 value2
120 Integer value.
121 Returns 0 and asserts PASS when "value1 X= value2".
123 rlAssertLesser
125 Assertion checking whether first parameter is lesser than the second
127 one.
128 rlAssertLesser comment value1 value2
130 comment
132 Short test summary, e.g. "Test whether there are running more
133 instances of program."
134 value1
136 Integer value.
137 value2
139 Integer value.
140 Returns 0 and asserts PASS when "value1 X value2".
142 rlAssertLesserOrEqual
144 Assertion checking whether first parameter is lesser or equal to the
146 second one.
147 rlAssertLesserOrEqual comment value1 value2
149 comment
151 Short test summary (e.g. "There should present at least one...")
152 value1
154 Integer value.
155 value2
157 Integer value.
158 Returns 0 and asserts PASS when "value1 X= value2".
160 File Asserts
162 rlAssertExists
163 Assertion checking for the existence of a file or a directory.
165 rlAssertExists file|directory
167 file|directory
169 Path to the file or directory.
170 Returns 0 and asserts PASS when "file" exists.
172 rlAssertNotExists
174 Assertion checking for the non-existence of a file or a directory.
176 rlAssertNotExists file|directory
178 file|directory
180 Path to the file or directory.
181 Returns 0 and asserts PASS when "file" does not exist.
183 rlAssertGrep
185 Assertion checking if the file contains a pattern.
187 rlAssertGrep pattern file [options]
189 pattern
191 Regular expression to be searched for.
192 file
194 Path to the file.
195 options
197 Optional parameters to be passed to grep, default is "-q". Can be
198 used to perform case insensitive matches (-i), or using extended
199 (-E) or perl (-P) regular expressions.
200 Returns 0 and asserts PASS when "file" exists and contains given
202 "pattern".
203 rlAssertNotGrep
205 Assertion checking that the file does not contain a pattern.
207 rlAssertNotGrep pattern file [options]
209 pattern
211 Regular expression to be searched for.
212 file
214 Path to the file.
215 options
217 Optional parameters to be passed to grep, default is "-q". Can be
218 used to perform case insensitive matches (-i), or using extended
219 (-E) or perl (-P) regular expressions.
220 Returns 0 and asserts PASS when "file" exists and does not contain
222 given "pattern".
223 rlAssertDiffer
225 Assertion checking that two files differ (are not identical).
227 rlAssertDiffer file1 file2
229 file1
231 Path to first file1
232 file2
234 Path to second file
235 Returns 0 and asserts PASS when "file1" and "file2" differs.
237 rlAssertNotDiffer
239 Assertion checking that two files do not differ (are identical).
241 rlAssertNotDiffer file1 file2
243 file1
245 Path to first file1
246 file2
248 Path to second file
249 Returns 0 and asserts PASS when "file1" and "file2" do not differ.
251 Run, Watch, Report
253 rlRun
254 Run command with optional comment and make sure its exit code matches
256 expectations.
257 rlRun [-t] [-l] [-c] [-s] command [status[,status...] [comment]]
259 -t If specified, stdout and stderr of the command output will be
261 tagged with strigs 'STDOUT: ' and 'STDERR: '.
262 -l If specified, output of the command (tagged, if -t was specified)
264 is logged using rlLog function. This is intended for short outputs,
265 and therefore only last 50 lines are logged this way. Longer
266 outputs should be analysed separately, or uploaded via rlFileSubmit
267 or rlBundleLogs.
268 -c Same as "-l", but only log the commands output if it failed.
270 -s Store stdout and stderr to a file (mixed together, as the user
272 would see it on a terminal) and set $rlRun_LOG variable to name of
273 the file. Caller is responsible for removing the file. When -t
274 option is used, the content of the file becomes tagged too.
275 If the -s option is not used, $rlRun_LOG is not modified and keeps
277 its content from the last "rlRun -s".
278 command
280 Command to run.
281 status
283 Expected exit code(s). Optional, default 0. If you expect more exit
284 codes, separate them with comma (e.g. "0,1" when both 0 and 1 are
285 OK and expected), or use from-to notation (i.e. "2-5" for
286 "2,3,4,5"), or combine them (e.g. "2-4,26" for "2,3,4,26").
287 comment
289 Short summary describing the action (optional, but recommended -
290 explain what are you doing here).
291 Returns the exit code of the command run. Asserts PASS when command\'s
293 exit status is in the list of expected exit codes.
294 Note:
296 · The output of rlRun is buffered when using "-t", "-l" or "-s"
298 option (they use unix pipes, which are buffered by nature). If you
299 need an unbuffered output just make sure that "expect" package is
300 installed on your system (its "unbuffer" tool will automatically be
301 used to produce unbuffered output).
302 · Be aware that there are some variables which can collide with your
304 code executed within rlRun. You should avoid using
305 __INTERNAL_rlRun_* variables.
306 · When any of "-t" "-l", "-c", or "-s" option is used, special file
308 descriptors 111 and 112 are used to avoid the issue with incomplete
309 log file, bz1361246. As there might be an indefinite loop, there's
310 a timeout of two minutes implemented as a fix for bz1416796. Also
311 an error message is issued to signal the possibility of running
312 subprocess which keeps the file descriptors open.
313 Do not use these options if you expect process forking and
315 continuouse run. Try your own apropriate solution instead.
316 Warning: using "unbuffer" tool is now disabled because of bug 547686.
318 rlWatchdog
320 Run "command". If it does not finish in specified time, then kill it
322 using "signal".
323 rlWatchdog command timeout [signal] [callback]
325 command
327 Command to run.
328 timeout
330 Timeout to wait, in seconds.
331 signal
333 Signal to use (optional, default KILL).
334 callback
336 Callback function to be called before the signal is send (optional,
337 none by default). The callback function will have one argument
338 available -- PGID of the process group.
339 Returns 0 if the command ends normally, without need to be killed.
341 rlReport
343 Report test result to the test harness. The command to be used for
345 reporting is set by the respective plugin. You can also use the
346 $BEAKERLIB_COMMAND_REPORT_RESULT variable to use your custom command.
347 rlReport name result [score] [log]
349 name
351 Name of the test result.
352 result
354 Result (one of PASS, WARN, FAIL). If called with something else,
355 will use WARN.
356 score
358 Test score (optional).
359 log Optional log file to be submitted instead of default "OUTPUTFILE".
361 rlCmpVersion
363 Compare two given versions composed by numbers and letters divided by
365 dot (.), underscore (_), or dash (-).
366 rlCmpVersion ver1 ver2
368 If ver1 = ver2, sign '=' is printed to stdout and 0 is returned. If
370 ver1 > ver2, sign '>' is printed to stdout and 1 is returned. If ver1
371 < ver2, sign '<' is printed to stdout and 2 is returned.
372 rlTestVersion
374 Test releation between two given versions based on given operator.
376 rlTestVersion ver1 op ver2
378 op Operator defining the logical expression. It can be '=', '==',
380 '!=', '<', '<=', '=<', '>', '>=', or '=>'.
381 Returns 0 if the expresison ver1 op ver2 is true; 1 if the expression
383 is false and 2 if something went wrong.
384 Release Info
386 rlIsRHEL
387 rlIsRHEL [Num|opNum]
389 Num When used function returns 0 if the particular RHEL version is
391 running. Multiple arguments can be passed separated with space as
392 well as any particular release (5.1 5.2 5.3).
393 opNum
395 Argument consisting of operator and number written together as one
396 string. Operator can be '<', '<=', '=<', '=', '>', '>=' matching
397 whether the currently installed version is lesser, lesser or equal,
398 equal, equal or greater, greater than the version number supplied
399 as second half of the argument. Note that ie. '=5' (unlike just
400 '5') matches exactly 5 (5.0), not 5.N, where N > 0. Also note when
401 that using >/< operators you have to either put the argument in
402 quotes or escape the operators to avoid them being interpreted as
403 bash redirection operator.
404 Returns 0 when we're running on RHEL.
406 Note that
408 rlIsRHEL '<6.9' || rlIsRHEL '<7.5'
410 would also cover 6.10 as it is less than 7.5, which is not what you
412 want. So if you want to construct a condition for rhel<6.9 for rhel6
413 or rhel<7.5 for rhel7 you actually need to use following construct:
414 rlIsRHEL 6 && rlIsRHEL '<6.9' || rlIsRHEL 7 && rlIsRHEL '<7.5'
416 Prototype:
418 rlIsRHEL
420 Returns 0 if we are running on RHEL.
422 rlIsRHEL 4.8 5
424 Returns 0 if we are running RHEL 4.8 or any RHEL 5.
426 rlIsRHEL ">=6" or rlIsRHEL \>=6
428 Returns 0 if we are running RHEL 6 or higher.
430 rlIsFedora
432 rlIsFedora [Num|opNum]
434 Returns 0 when we're running on Fedora. With given number of version
436 as parameter returns 0 if the particular Fedora version is running.
437 Range matching can be used in the form used by rlIsRHEL.
438 rlIsFedora
440 Returns 0 if we are running on Fedora.
442 rlIsFedora 9 10
444 Returns 0 if we are running Fedora 9 or 10.
446 Release Info
448 rlIsCentOS
449 rlIsCentOS [Num|opNum]
451 Returns 0 when we're running on CentOS. With given number of version
453 as parameter returns 0 if the particular CentOS version is running.
454 Range matching can be used in the form used by rlIsRHEL.
455 rlIsCentOS
457 Returns 0 if we are running on CentOS.
459 rlIsCentOS 7.1 6
461 Returns 0 if we are running CentOS 7.1 or any CentOS 6.
463
465 · Ondrej Hudlicky <ohudlick@redhat.com>
466 · Petr Muller <pmuller@redhat.com>
468 · Jan Hutar <jhutar@redhat.com>
470 · Petr Splichal <psplicha@redhat.com>
472 · Ales Zelinka <azelinka@redhat.com>
474perl v5.28.1 2019-04-04 TESTING.SH(1)