1KYUA(1) BSD General Commands Manual KYUA(1)
2
4 kyua — Testing framework for infrastructure software
5
7 kyua [--config file] [--logfile file] [--loglevel level]
8 [--variable name=value] command [command_options]
9 [command_arguments]
10
12 If you are here looking for details on how to run the test suite in
13 /usr/tests (or /usr/libexec/kyua/tests), please start by reading the
14 tests(7) manual page that should be supplied by your system.
15 Kyua is a testing framework for infrastructure software, originally
17 designed to equip BSD-based operating systems with a test suite. This
18 means that Kyua is lightweight and simple, and that Kyua integrates well
19 with various build systems and continuous integration frameworks.
20 Kyua features an expressive test suite definition language, a safe run‐
22 time engine for test suites and a powerful report generation engine.
23 Kyua is for both developers and users, from the developer applying a sim‐
25 ple fix to a library to the system administrator deploying a new release
26 on a production machine.
27 Kyua is able to execute test programs written with a plethora of testing
29 libraries and languages. The library of choice is ATF, for which Kyua
30 was originally designed, but simple, framework-less test programs and
31 TAP-compliant test programs can also be executed through Kyua.
32 Overview
34 As can be observed in the synopsis, the interface of kyua implements a
35 common subcommand-based interface. The arguments to the tool specify, in
36 this order: a set of common options that all the commands accept, a
37 required command name that specifies what kyua should do, and a set of
38 possibly-optional command_options and command_arguments that are specific
39 to the chosen command.
40 The following options are recognized by all the commands. Keep in mind
42 that these must always be specified before the command name.
43 --config path, -c path
45 Specifies the configuration file to process, which must be in the
46 format described in kyua.conf(5). The special value ‘none’ explic‐
47 itly disables the loading of any configuration file.
48 Defaults to ~/.kyua/kyua.conf if it exists, otherwise to
50 /etc/kyua/kyua.conf if it exists, or else to ‘none’.
51 --logfile path
53 Specifies the location of the file to which kyua will log run time
54 events useful for postmortem debugging.
55 The default depends on different environment variables as described
57 in Logging, but typically the file will be stored within the user's
58 home directory.
59 --loglevel level
61 Specifies the maximum logging level to record in the log file. See
62 Logging for more details.
63 The default is ‘info’.
65 --variable name=value, -v name=value
67 Sets the name configuration variable to value. The values set
68 through this option have preference over the values set in the con‐
69 figuration file.
70 The specified variable can either be a builtin variable or a test-
72 suite specific variable. See kyua.conf(5) for more details.
73 The following commands are generic and do not have any relation to the
75 execution of tests or the inspection of their results:
76 about Shows general program information. See
78 kyua-about(1).
79 config Inspects the values of the configuration variables.
81 See kyua-config(1).
82 db-exec Executes an arbitrary SQL statement on a results
84 file and prints the resulting table. See
85 kyua-db-exec(1).
86 help Shows usage information. See kyua-help(1).
88 The following commands are used to generate reports based on the data
90 previously recorded in a results file:
91 report Generates a plain-text report. Combined with its
93 --verbose flag and the ability to only display spe‐
94 cific test cases, this command can also be used to
95 debug test failures post-facto on the console. See
96 kyua-report(1).
97 report-html Generates an HTML report. See kyua-report-html(1).
99 report-junit Generates a JUnit report. See
101 kyua-report-junit(1).
102 The following commands are used to interact with a test suite:
104 debug Executes a single test case in a controlled envi‐
106 ronment for debugging purposes. See kyua-debug(1).
107 list Lists test cases defined in a test suite by a
109 kyuafile(5) and, optionally, displays their meta‐
110 data. See kyua-list(1).
111 test Runs tests defined in a test suite by a
113 kyuafile(5). See kyua-test(1).
114 Logging
116 kyua has a logging facility that collects all kinds of events at run
117 time. These events are always logged to a file so that the log is avail‐
118 able when it is most needed: right after a non-reproducible problem hap‐
119 pens. The only way to disable logging is by sending the log to
120 /dev/null.
121 The location of the log file can be manually specified with the --logfile
123 option, which applies to all commands. If no file is explicitly speci‐
124 fied, the location of the log files is chosen in this order:
125 1. ${HOME}/.kyua/logs/ if HOME is defined.
127 2. ${TMPDIR}/ if TMPDIR is defined.
129 3. /tmp/.
131 And the default naming scheme of the log files is:
133 ‘<progname>.<timestamp>.log’.
134 The messages stored in the log file have a level (or severity) attached
136 to them. These are:
137 error Fatal error messages. The program generally terminates
139 after these, either in a clean manner or by crashing.
140 warning Non-fatal error messages. These generally report a con‐
142 dition that must be addressed but the application can
143 continue to run.
144 info Informational messages. These tell the user what the
146 program was doing at a general level of operation.
147 debug Detailed informational messages. These are often useful
149 when debugging problems in the application, as they con‐
150 tain lots of internal details.
151 The default log level is ‘info’ unless expicitly overriden with
153 --loglevel.
154 The log file is a plain text file containing one line per log record.
156 The format of each line is as follows:
157 timestamp entry_type pid file:line: message
159 entry_type can be one of: ‘E’ for an error, ‘W’ for a warning, ‘I’ for an
161 informational message and ‘D’ for a debug message.
162 Bug reporting
164 If you think you have encountered a bug in kyua, please take the time to
165 let the developers know about it. This will ensure that the bug is
166 addressed and potentially fixed in the next Kyua release.
167 The first step in reporting a bug is to check if there already is a simi‐
169 lar bug in the database. You can check what issues are currently in the
170 database by going to:
171 https://github.com/jmmv/kyua/issues/
173 If there is no existing issue that describes an issue similar to the one
175 you are experiencing, you can open a new one by visiting:
176 https://github.com/jmmv/kyua/issues/new/
178 When doing so, please include as much detail as possible. Among other
180 things, explain what operating system and platform you are running kyua
181 on, what were you trying to do, what exact messages you saw on the
182 screen, how did you expect the program to behave, and any other details
183 that you may find relevant.
184 Also, please include a copy of the log file corresponding to the problem
186 you are experiencing. Unless you have changed the location of the log
187 files, you can most likely find them in ~/.kyua/logs/. If the problem is
188 reproducible, it is good idea to regenerate the log file with an
189 increased log level so as to provide more information. For example:
190 $ kyua --logfile=problem.log --loglevel=debug \
192 [rest of the command line]
193
195 The following variables are recognized and can be freely tuned by the end
196 user:
197 COLUMNS The width of the screen, in number of characters. kyua uses
199 this to wrap long lines. If not present, the width of the
200 screen is determined from the terminal stdout is connected to,
201 and, if the guessing fails, this defaults to infinity.
202 HOME Path to the user's home directory. kyua uses this location to
204 determine paths to configuration files and default log files.
205 TMPDIR Path to the system-wide temporary directory. kyua uses this
207 location to place the work directory of test cases, among
208 other things.
209 The default value of this variable depends on the operating
211 system. In general, it is /tmp.
212 The following variables are also recognized, but you should not need to
214 set them during normal operation. They are only provided to override the
215 value of built-in values, which is useful when testing kyua itself:
216 KYUA_CONFDIR Path to the system-wide configuration files for kyua.
218 Defaults to /etc/kyua.
220 KYUA_DOCDIR Path to the location of installed documentation.
222 Defaults to /usr/share/doc/kyua.
224 KYUA_MISCDIR Path to the location of the installed miscellaneous
226 scripts and data files provided by kyua.
227 Defaults to /usr/share/kyua/misc.
229 KYUA_STOREDIR Path to the location of the installed store support
231 files; e.g. the directory containing the SQL database
232 schema.
233 Defaults to /usr/share/kyua/store.
235
237 ~/.kyua/store/
238 Default location for the results files.
239 ~/.kyua/kyua.conf
241 User-specific configuration file.
242 ~/.kyua/logs/
244 Default location for the collected log files.
245 /etc/kyua/kyua.conf
247 System-wide configuration file.
248
250 kyua returns 0 on success, 1 on a controlled error condition in the given
251 subcommand, 2 on a general unexpected error and 3 on a usage error.
252 The documentation of the subcommands in the corresponding manual pages
254 only details the difference between a successful exit (0) and the detec‐
255 tion of a controlled error (1). Even though when those manual pages do
256 not describe any other exit statuses, codes above 1 can be returned.
257
259 kyua.conf(5), kyuafile(5), atf(7), tests(7)
260
262 For more details on the people that made kyua possible and the license
263 terms, run:
264 $ kyua about
266BSD May 12, 2015 BSD