1KYUA-DEBUG(1) BSD General Commands Manual KYUA-DEBUG(1)
2
4 kyua debug — Executes a single test case with facilities for debugging
5
7 kyua debug [--build-root path] [--kyuafile file] [--stdout path]
8 [--stderr path] test_case
9
11 The kyua debug command provides a mechanism to execute a single test case
12 bypassing some of the Kyua infrastructure and allowing the user to poke
13 into the execution behavior of the test.
14 The test case to run is selected by providing a test filter, described
16 below in Test filters, that matches a single test case. The test case is
17 executed and its result is printed as the last line of the output of the
18 tool.
19 The test executed by kyua debug is run under a controlled environment as
21 described in Test isolation.
22 At the moment, the kyua debug command allows the following aspects of a
24 test case execution to be tweaked:
25 · Redirection of the test case's stdout and stderr to the console (the
27 default) or to arbitraty files. See the --stdout and --stderr
28 options below.
29 The following subcommand options are recognized:
31 --build-root path
33 Specifies the build root in which to find the test programs refer‐
34 enced by the Kyuafile, if different from the Kyuafile's directory.
35 See Build directories below for more information.
36 --kyuafile file, -k file
38 Specifies the Kyuafile to process. Defaults to Kyuafile file in the
39 current directory.
40 --stderr path
42 Specifies the file to which to send the standard error of the test
43 program's body. The default is /dev/stderr, which is a special that
44 redirects the output to the console.
45 --stdout path
47 Specifies the file to which to send the standard output of the test
48 program's body. The default is /dev/stdout, which is a special that
49 redirects the output to the console.
50 For example, consider the following Kyua session:
52 $ kyua test
54 kernel/fs:mkdir -> passed
55 kernel/fs:rmdir -> failed: Invalid argument
56 1/2 passed (1 failed)
58 At this point, we do not have a lot of information regarding the failure
60 of the ‘kernel/fs:rmdir’ test. We can run this test through the kyua
61 debug command to inspect its output a bit closer, hoping that the test
62 case is kind enough to log its progress:
63 $ kyua debug kernel/fs:rmdir
65 Trying rmdir('foo')
66 Trying rmdir(NULL)
67 kernel/fs:rmdir -> failed: Invalid argument
68 Luckily, the offending test case was printing status lines as it pro‐
70 gressed, so we could see the last attempted call and we can know match
71 the failure message to the problem.
72 Build directories
74 Build directories (or object directories, target directories, product
75 directories, etc.) is the concept that allows a developer to keep the
76 source tree clean from build products by asking the build system to place
77 such build products under a separate subtree.
78 Most build systems today support build directories. For example, the GNU
80 Automake/Autoconf build system exposes such concept when invoked as fol‐
81 lows:
82 $ cd my-project-1.0
84 $ mkdir build
85 $ cd build
86 $ ../configure
87 $ make
88 Under such invocation, all the results of the build are left in the
90 my-project-1.0/build/ subdirectory while maintaining the contents of
91 my-project-1.0/ intact.
92 Because build directories are an integral part of most build systems, and
94 because they are a tool that developers use frequently, kyua debug sup‐
95 ports build directories too. This manifests in the form of kyua debug
96 being able to run tests from build directories while reading the (often
97 immutable) test suite definition from the source tree.
98 One important property of build directories is that they follow (or need
100 to follow) the exact same layout as the source tree. For example, con‐
101 sider the following directory listings:
102 src/Kyuafile
104 src/bin/ls/
105 src/bin/ls/Kyuafile
106 src/bin/ls/ls.c
107 src/bin/ls/ls_test.c
108 src/sbin/su/
109 src/sbin/su/Kyuafile
110 src/sbin/su/su.c
111 src/sbin/su/su_test.c
112 obj/bin/ls/
114 obj/bin/ls/ls*
115 obj/bin/ls/ls_test*
116 obj/sbin/su/
117 obj/sbin/su/su*
118 obj/sbin/su/su_test*
119 Note how the directory layout within src/ matches that of obj/. The src/
121 directory contains only source files and the definition of the test suite
122 (the Kyuafiles), while the obj/ directory contains only the binaries gen‐
123 erated during a build.
124 All commands that deal with the workspace support the --build-root path
126 option. When this option is provided, the directory specified by the
127 option is considered to be the root of the build directory. For example,
128 considering our previous fake tree layout, we could invoke kyua debug as
129 any of the following:
130 $ kyua debug --kyuafile=src/Kyuafile --build-root=obj
132 $ cd src && kyua debug --build-root=../obj
133 Test filters
135 A test filter is a string that is used to match test cases or test pro‐
136 grams in a test suite. Filters have the following form:
137 test_program_name[:test_case_name]
139 Where ‘test_program_name’ is the name of a test program or a subdirectory
141 in the test suite, and ‘test_case_name’ is the name of a test case.
142 Test isolation
144 The test programs and test cases run by kyua debug are all executed in a
145 deterministic environment. This known, clean environment serves to make
146 the test execution as reproducible as possible and also to prevent
147 clashes between tests that may, for example, create auxiliary files with
148 overlapping names.
149 For plain test programs and for TAP test programs, the whole test program
151 is run under a single instance of the environment described in this page.
152 For ATF test programs (see atf(7)), each individual test case and test
153 cleanup routine are executed in separate environments.
154 Process space
156 Each test is executed in an independent processes. Corollary: the
157 test can do whatever it wants to the current process (such as modify
158 global variables) without having to undo such changes.
159 Session and process group
161 The test is executed in its own session and its own process group.
162 There is no controlling terminal attached to the session.
163 Should the test spawn any children, the children should maintain the
165 same session and process group. Modifying any of these settings pre‐
166 vents kyua debug from being able to kill any stray subprocess as part
167 of the cleanup phase. If modifying these settings is necessary, or
168 if any subprocess started by the test decides to use a different
169 process group or session, it is the responsibility of the test to
170 ensure those subprocesses are forcibly terminated during cleanup.
171 Work directory
173 The test is executed in a temporary directory automatically created
174 by the runtime engine. Corollary: the test can write to its current
175 directory without needing to clean any files and/or directories it
176 creates. The runtime engine takes care to recursively delete the
177 temporary directories after the execution of a test case. Any file
178 systems mounted within the temporary directory are also unmounted.
179 Home directory
181 The HOME environment variable is set to the absolute path of the work
182 directory.
183 Umask
185 The value of the umask is set to 0022.
186 Environment
188 The LANG, LC_ALL, LC_COLLATE, LC_CTYPE, LC_MESSAGES, LC_MONETARY,
189 LC_NUMERIC and LC_TIME variables are unset.
190 The TZ variable is set to ‘UTC’.
192 The TMPDIR variable is set to the absolute path of the work direc‐
194 tory. This is to prevent the test from mistakenly using a temporary
195 directory outside of the automatically-managed work directory, should
196 the test use the mktemp(3) familiy of functions.
197 Process limits
199 The maximum soft core size limit is raised to its corresponding hard
200 limit. This is a simple, best-effort attempt at allowing tests to
201 dump core for further diagnostic purposes.
202 Configuration varibles
204 The test engine may pass run-time configuration variables to the test
205 program via the environment. The name of the configuration variable
206 is prefixed with ‘TEST_ENV_’ so that a configuration variable of the
207 form ‘foo=bar’ becomes accessible in the environment as
208 ‘TEST_ENV_foo=bar’.
209
211 The kyua debug command returns 0 if the test case passes or 1 if the test
212 case fails.
213 Additional exit codes may be returned as described in kyua(1).
215
217 kyua(1), kyuafile(5)
218BSD October 13, 2014 BSD