1ATF-SH(3) BSD Library Functions Manual ATF-SH(3)
2
4 atf_add_test_case, atf_check, atf_check_equal, atf_config_get,
5 atf_config_has, atf_expect_death, atf_expect_exit, atf_expect_fail,
6 atf_expect_pass, atf_expect_signal, atf_expect_timeout, atf_fail,
7 atf_get, atf_get_srcdir, atf_pass, atf_require_prog, atf_set, atf_skip,
8 atf_test_case — POSIX shell API to write ATF-based test programs
9
11 atf_add_test_case "name"
12 atf_check "command"
13 atf_check_equal "expected_expression" "actual_expression"
14 atf_config_get "var_name"
15 atf_config_has "var_name"
16 atf_expect_death "reason" "..."
17 atf_expect_exit "exitcode" "reason" "..."
18 atf_expect_fail "reason" "..."
19 atf_expect_pass ""
20 atf_expect_signal "signo" "reason" "..."
21 atf_expect_timeout "reason" "..."
22 atf_fail "reason"
23 atf_get "var_name"
24 atf_get_srcdir
25 atf_pass
26 atf_require_prog "prog_name"
27 atf_set "var_name" "value"
28 atf_skip "reason"
29 atf_test_case "name" "cleanup"
30
32 ATF provides a simple but powerful interface to easily write test pro‐
33 grams in the POSIX shell language. These are extremely helpful given
34 that they are trivial to write due to the language simplicity and the
35 great deal of available external tools, so they are often ideal to test
36 other applications at the user level.
37 Test programs written using this library must be run using the atf-sh(1)
39 interpreter by putting the following on their very first line:
40 #! /usr/bin/env atf-sh
42 Shell-based test programs always follow this template:
44 atf_test_case tc1
46 tc1_head() {
47 ... first test case's header ...
48 }
49 tc1_body() {
50 ... first test case's body ...
51 }
52 atf_test_case tc2 cleanup
54 tc2_head() {
55 ... second test case's header ...
56 }
57 tc2_body() {
58 ... second test case's body ...
59 }
60 tc2_cleanup() {
61 ... second test case's cleanup ...
62 }
63 ... additional test cases ...
65 atf_init_test_cases() {
67 atf_add_test_case tc1
68 atf_add_test_case tc2
69 ... add additional test cases ...
70 }
71 Definition of test cases
73 Test cases have an identifier and are composed of three different parts:
74 the header, the body and an optional cleanup routine, all of which are
75 described in atf-test-case(4). To define test cases, one can use the
76 atf_test_case function, which takes a first parameter specifiying the
77 test case's name and instructs the library to set things up to accept it
78 as a valid test case. The second parameter is optional and, if provided,
79 must be ‘cleanup’; providing this parameter allows defining a cleanup
80 routine for the test case. It is important to note that this function
81 does not set the test case up for execution when the program is run. In
82 order to do so, a later registration is needed through the
83 atf_add_test_case function detailed in Program initialization.
84 Later on, one must define the three parts of the body by providing two or
86 three functions (remember that the cleanup routine is optional). These
87 functions are named after the test case's identifier, and are <id>_head,
88 <id>_body and <id>_cleanup. None of these take parameters when executed.
89 Program initialization
91 The test program must define an atf_init_test_cases function, which is in
92 charge of registering the test cases that will be executed at run time by
93 using the atf_add_test_case function, which takes the name of a test case
94 as its single parameter. This main function should not do anything else,
95 except maybe sourcing auxiliary source files that define extra variables
96 and functions.
97 Configuration variables
99 The test case has read-only access to the current configuration variables
100 through the atf_config_has and atf_config_get methods. The former takes
101 a single parameter specifying a variable name and returns a boolean indi‐
102 cating whether the variable is defined or not. The latter can take one
103 or two parameters. If it takes only one, it specifies the variable from
104 which to get the value, and this variable must be defined. If it takes
105 two, the second one specifies a default value to be returned if the vari‐
106 able is not available.
107 Access to the source directory
109 It is possible to get the path to the test case's source directory from
110 anywhere in the test program by using the atf_get_srcdir function. It is
111 interesting to note that this can be used inside atf_init_test_cases to
112 silently include additional helper files from the source directory.
113 Requiring programs
115 Aside from the require.progs meta-data variable available in the header
116 only, one can also check for additional programs in the test case's body
117 by using the atf_require_prog function, which takes the base name or full
118 path of a single binary. Relative paths are forbidden. If it is not
119 found, the test case will be automatically skipped.
120 Test case finalization
122 The test case finalizes either when the body reaches its end, at which
123 point the test is assumed to have passed, or at any explicit call to
124 atf_pass, atf_fail or atf_skip. These three functions terminate the exe‐
125 cution of the test case immediately. The cleanup routine will be pro‐
126 cessed afterwards in a completely automated way, regardless of the test
127 case's termination reason.
128 atf_pass does not take any parameters. atf_fail and atf_skip take a sin‐
130 gle string parameter that describes why the test case failed or was
131 skipped, respectively. It is very important to provide a clear error
132 message in both cases so that the user can quickly know why the test did
133 not pass.
134 Expectations
136 Everything explained in the previous section changes when the test case
137 expectations are redefined by the programmer.
138 Each test case has an internal state called ‘expect’ that describes what
140 the test case expectations are at any point in time. The value of this
141 property can change during execution by any of:
142 atf_expect_death "reason" "..."
144 Expects the test case to exit prematurely regardless of the na‐
145 ture of the exit.
146 atf_expect_exit "exitcode" "reason" "..."
148 Expects the test case to exit cleanly. If exitcode is not ‘-1’,
149 the runtime engine will validate that the exit code of the test
150 case matches the one provided in this call. Otherwise, the exact
151 value will be ignored.
152 atf_expect_fail "reason"
154 Any failure raised in this mode is recorded, but such failures do
155 not report the test case as failed; instead, the test case final‐
156 izes cleanly and is reported as ‘expected failure’; this report
157 includes the provided reason as part of it. If no error is
158 raised while running in this mode, then the test case is reported
159 as ‘failed’.
160 This mode is useful to reproduce actual known bugs in tests.
162 Whenever the developer fixes the bug later on, the test case will
163 start reporting a failure, signaling the developer that the test
164 case must be adjusted to the new conditions. In this situation,
165 it is useful, for example, to set reason as the bug number for
166 tracking purposes.
167 atf_expect_pass
169 This is the normal mode of execution. In this mode, any failure
170 is reported as such to the user and the test case is marked as
171 ‘failed’.
172 atf_expect_signal "signo" "reason" "..."
174 Expects the test case to terminate due to the reception of a sig‐
175 nal. If signo is not ‘-1’, the runtime engine will validate that
176 the signal that terminated the test case matches the one provided
177 in this call. Otherwise, the exact value will be ignored.
178 atf_expect_timeout "reason" "..."
180 Expects the test case to execute for longer than its timeout.
181 Helper functions for common checks
183 atf_check "[options]" "command" "[args]"
184 Executes a command, performs checks on its exit code and its out‐
185 put, and fails the test case if any of the checks is not success‐
186 ful. This function is particularly useful in integration tests
187 that verify the correct functioning of a binary.
188 Internally, this function is just a wrapper over the atf-check(1)
190 tool (whose manual page provides all details on the calling syn‐
191 tax). You should always use the atf_check function instead of
192 the atf-check(1) tool in your scripts; the latter is not even in
193 the path.
194 atf_check_equal "expected_expression" "actual_expression"
196 This function takes two expressions, evaluates them and, if their
197 results differ, aborts the test case with an appropriate failure
198 message. The common style is to put the expected value in the
199 first parameter and the actual value in the second parameter.
200
202 The following shows a complete test program with a single test case that
203 validates the addition operator:
204 atf_test_case addition
206 addition_head() {
207 atf_set "descr" "Sample tests for the addition operator"
208 }
209 addition_body() {
210 atf_check_equal 0 $((0 + 0))
211 atf_check_equal 1 $((0 + 1))
212 atf_check_equal 1 $((1 + 0))
213 atf_check_equal 2 $((1 + 1))
215 atf_check_equal 300 $((100 + 200))
217 }
218 atf_init_test_cases() {
220 atf_add_test_case addition
221 }
222 This other example shows how to include a file with extra helper func‐
224 tions in the test program:
225 ... definition of test cases ...
227 atf_init_test_cases() {
229 . $(atf_get_srcdir)/helper_functions.sh
230 atf_add_test_case foo1
232 atf_add_test_case foo2
233 }
234 This example demonstrates the use of the very useful atf_check function:
236 # Check for silent output
238 atf_check -s exit:0 -o empty -e empty 'true'
239 # Check for silent output and failure
241 atf_check -s exit:1 -o empty -e empty 'false'
242 # Check for known stdout and silent stderr
244 echo foo >expout
245 atf_check -s exit:0 -o file:expout -e empty 'echo foo'
246 # Generate a file for later inspection
248 atf_check -s exit:0 -o save:stdout -e empty 'ls'
249 grep foo ls || atf_fail "foo file not found in listing"
250 # Or just do the match along the way
252 atf_check -s exit:0 -o match:"^foo$" -e empty 'ls'
253
255 atf-check(1), atf-sh(1), atf-test-program(1), atf-test-case(4)
256BSD October 13, 2014 BSD