1KYUA-DEBUG(1)             BSD General Commands Manual            KYUA-DEBUG(1)
2

NAME

4     kyua debug — Executes a single test case with facilities for debugging
5

SYNOPSIS

7     kyua debug [--build-root path] [--kyuafile file] [--stdout path]
8          [--stderr path] test_case
9

DESCRIPTION

11     The kyua debug command provides a mechanism to execute a single test case
12     bypassing some of the Kyua infrastructure and allowing the user to poke
13     into the execution behavior of the test.
14
15     The test case to run is selected by providing a test filter, described
16     below in Test filters, that matches a single test case.  The test case is
17     executed and its result is printed as the last line of the output of the
18     tool.
19
20     The test executed by kyua debug is run under a controlled environment as
21     described in Test isolation.
22
23     At the moment, the kyua debug command allows the following aspects of a
24     test case execution to be tweaked:
25
26     ·   Redirection of the test case's stdout and stderr to the console (the
27         default) or to arbitraty files.  See the --stdout and --stderr
28         options below.
29
30     The following subcommand options are recognized:
31
32     --build-root path
33         Specifies the build root in which to find the test programs refer‐
34         enced by the Kyuafile, if different from the Kyuafile's directory.
35         See Build directories below for more information.
36
37     --kyuafile file, -k file
38         Specifies the Kyuafile to process.  Defaults to Kyuafile file in the
39         current directory.
40
41     --stderr path
42         Specifies the file to which to send the standard error of the test
43         program's body.  The default is /dev/stderr, which is a special that
44         redirects the output to the console.
45
46     --stdout path
47         Specifies the file to which to send the standard output of the test
48         program's body.  The default is /dev/stdout, which is a special that
49         redirects the output to the console.
50
51     For example, consider the following Kyua session:
52
53           $ kyua test
54           kernel/fs:mkdir  ->  passed
55           kernel/fs:rmdir  ->  failed: Invalid argument
56
57           1/2 passed (1 failed)
58
59     At this point, we do not have a lot of information regarding the failure
60     of the ‘kernel/fs:rmdir’ test.  We can run this test through the kyua
61     debug command to inspect its output a bit closer, hoping that the test
62     case is kind enough to log its progress:
63
64           $ kyua debug kernel/fs:rmdir
65           Trying rmdir('foo')
66           Trying rmdir(NULL)
67           kernel/fs:rmdir  ->  failed: Invalid argument
68
69     Luckily, the offending test case was printing status lines as it pro‐
70     gressed, so we could see the last attempted call and we can know match
71     the failure message to the problem.
72
73   Build directories
74     Build directories (or object directories, target directories, product
75     directories, etc.) is the concept that allows a developer to keep the
76     source tree clean from build products by asking the build system to place
77     such build products under a separate subtree.
78
79     Most build systems today support build directories.  For example, the GNU
80     Automake/Autoconf build system exposes such concept when invoked as fol‐
81     lows:
82
83           $ cd my-project-1.0
84           $ mkdir build
85           $ cd build
86           $ ../configure
87           $ make
88
89     Under such invocation, all the results of the build are left in the
90     my-project-1.0/build/ subdirectory while maintaining the contents of
91     my-project-1.0/ intact.
92
93     Because build directories are an integral part of most build systems, and
94     because they are a tool that developers use frequently, kyua debug sup‐
95     ports build directories too.  This manifests in the form of kyua debug
96     being able to run tests from build directories while reading the (often
97     immutable) test suite definition from the source tree.
98
99     One important property of build directories is that they follow (or need
100     to follow) the exact same layout as the source tree.  For example, con‐
101     sider the following directory listings:
102
103           src/Kyuafile
104           src/bin/ls/
105           src/bin/ls/Kyuafile
106           src/bin/ls/ls.c
107           src/bin/ls/ls_test.c
108           src/sbin/su/
109           src/sbin/su/Kyuafile
110           src/sbin/su/su.c
111           src/sbin/su/su_test.c
112
113           obj/bin/ls/
114           obj/bin/ls/ls*
115           obj/bin/ls/ls_test*
116           obj/sbin/su/
117           obj/sbin/su/su*
118           obj/sbin/su/su_test*
119
120     Note how the directory layout within src/ matches that of obj/.  The src/
121     directory contains only source files and the definition of the test suite
122     (the Kyuafiles), while the obj/ directory contains only the binaries gen‐
123     erated during a build.
124
125     All commands that deal with the workspace support the --build-root path
126     option.  When this option is provided, the directory specified by the
127     option is considered to be the root of the build directory.  For example,
128     considering our previous fake tree layout, we could invoke kyua debug as
129     any of the following:
130
131           $ kyua debug --kyuafile=src/Kyuafile --build-root=obj
132           $ cd src && kyua debug --build-root=../obj
133
134   Test filters
135     A test filter is a string that is used to match test cases or test pro‐
136     grams in a test suite.  Filters have the following form:
137
138           test_program_name[:test_case_name]
139
140     Where ‘test_program_name’ is the name of a test program or a subdirectory
141     in the test suite, and ‘test_case_name’ is the name of a test case.
142
143   Test isolation
144     The test programs and test cases run by kyua debug are all executed in a
145     deterministic environment.  This known, clean environment serves to make
146     the test execution as reproducible as possible and also to prevent
147     clashes between tests that may, for example, create auxiliary files with
148     overlapping names.
149
150     For plain test programs and for TAP test programs, the whole test program
151     is run under a single instance of the environment described in this page.
152     For ATF test programs (see atf(7)), each individual test case and test
153     cleanup routine are executed in separate environments.
154
155     Process space
156         Each test is executed in an independent processes.  Corollary: the
157         test can do whatever it wants to the current process (such as modify
158         global variables) without having to undo such changes.
159
160     Session and process group
161         The test is executed in its own session and its own process group.
162         There is no controlling terminal attached to the session.
163
164         Should the test spawn any children, the children should maintain the
165         same session and process group.  Modifying any of these settings pre‐
166         vents kyua debug from being able to kill any stray subprocess as part
167         of the cleanup phase.  If modifying these settings is necessary, or
168         if any subprocess started by the test decides to use a different
169         process group or session, it is the responsibility of the test to
170         ensure those subprocesses are forcibly terminated during cleanup.
171
172     Work directory
173         The test is executed in a temporary directory automatically created
174         by the runtime engine.  Corollary: the test can write to its current
175         directory without needing to clean any files and/or directories it
176         creates.  The runtime engine takes care to recursively delete the
177         temporary directories after the execution of a test case.  Any file
178         systems mounted within the temporary directory are also unmounted.
179
180     Home directory
181         The HOME environment variable is set to the absolute path of the work
182         directory.
183
184     Umask
185         The value of the umask is set to 0022.
186
187     Environment
188         The LANG, LC_ALL, LC_COLLATE, LC_CTYPE, LC_MESSAGES, LC_MONETARY,
189         LC_NUMERIC and LC_TIME variables are unset.
190
191         The TZ variable is set to ‘UTC’.
192
193         The TMPDIR variable is set to the absolute path of the work direc‐
194         tory.  This is to prevent the test from mistakenly using a temporary
195         directory outside of the automatically-managed work directory, should
196         the test use the mktemp(3) familiy of functions.
197
198     Process limits
199         The maximum soft core size limit is raised to its corresponding hard
200         limit.  This is a simple, best-effort attempt at allowing tests to
201         dump core for further diagnostic purposes.
202
203     Configuration varibles
204         The test engine may pass run-time configuration variables to the test
205         program via the environment.  The name of the configuration variable
206         is prefixed with ‘TEST_ENV_’ so that a configuration variable of the
207         form ‘foo=bar’ becomes accessible in the environment as
208         ‘TEST_ENV_foo=bar’.
209

EXIT STATUS

211     The kyua debug command returns 0 if the test case passes or 1 if the test
212     case fails.
213
214     Additional exit codes may be returned as described in kyua(1).
215

SEE ALSO

217     kyua(1), kyuafile(5)
218
219BSD                            October 13, 2014                            BSD
Impressum