1KYUA-REPORT(1) BSD General Commands Manual KYUA-REPORT(1)
2
4 kyua report — Generates reports with the results of a test suite run
5
7 kyua report [--output path] [--results-file file]
8 [--results-filter types] [--verbose] [test_filter1 .. test_filterN]
9
11 The kyua report command parses a results file and generates a user-
12 friendly, plain-text report for user consumption on the terminal. By
13 default, these reports only display a summary of the execution of the
14 full test suite to highlight where problems may lie.
15 The output of kyua report can be customized to display full details on
17 all executed test cases. Additionally, the optional arguments to kyua
18 report are used to select which test programs or test cases to display.
19 These are filters and are described below in Test filters.
20 Reports generated by kyua report are not intended to be
22 machine-parseable.
23 The following subcommand options are recognized:
25 --output path
27 Specifies the path to which the report should be written to. The
28 special values /dev/stdout and /dev/stderr can be used to specify the
29 standard output and the standard error respectively.
30 --results-file path, -s path
32 Specifies the results file to operate on. Defaults to ‘LATEST’,
33 which causes kyua report to automatically load the latest results
34 file from the current test suite.
35 The following values are accepted:
37 ‘LATEST’
39 Requests the load of the latest results file available for the
40 test suite rooted at the current directory.
41 Directory
43 Requests the load of the latest results file available for the
44 test suite rooted at the given directory.
45 Test suite name
47 Requests the load of the latest results file available for the
48 given test suite.
49 Results identifier
51 Requests the load of a specific results file.
52 Explicit file name (aka everything else)
54 Load the specified results file.
55 See Results files for more details.
57 --results-filter types
59 Comma-separated list of the test result types to include in the
60 report. The ordering of the values is respected so that you can
61 determine how you want the list of tests to be shown.
62 The valid values are: ‘broken’, ‘failed’, ‘passed’, ‘skipped’ and
64 ‘xfail’. If the parameter supplied to the option is empty, filtering
65 is suppressed and all result types are shown in the report.
66 The default value for this flag includes all the test results except
68 the passed tests. Showing the passed tests by default clutters the
69 report with too much information, so only abnormal conditions are
70 included.
71 --verbose
73 Prints a detailed report of the execution. In addition to all the
74 information printed by default, verbose reports include the runtime
75 context of the test suite run, the metadata of each test case, and
76 the verbatim output of the test cases.
77 Results files
79 Results files contain, as their name implies, the results of the execu‐
80 tion of a test suite. Each test suite executed by kyua-test(1) generates
81 a new results file, and such results files can be loaded later on by
82 inspection commands such as kyua-report(1) to analyze their contents.
83 Results files support identifier-based lookups and also path name
85 lookups. The differences between the two are described below.
86 The default naming scheme for the results files provides simple support
88 for identifier-based lookups and historical recording of test suite runs.
89 Each results file is given an identifier derived from the test suite that
90 generated it and the time the test suite was run. Kyua can later look up
91 results files by these fileds.
92 The identifier follows this pattern:
94 <test_suite>.<YYYYMMDD>-<HHMMSS>-<uuuuuu>
96 where ‘test_suite’ is the path to the root of the test suite that was run
98 with all slashes replaced by underscores and ‘YYYYMMDD-HHMMSS-uuuuuu’ is
99 a timestamp with microsecond resolution.
100 When using the default naming scheme, results files are stored in the
102 ~/.kyua/store/ subdirectory and each file holds a name of the form:
103 ~/.kyua/store/results.<identifier>.db
105 Results files are simple SQLite databases with the schema described in
107 the /usr/share/kyua/store/schema_v?.sql files. For details on the
108 schema, please refer to the heavily commented SQL file.
109 Test filters
111 A test filter is a string that is used to match test cases or test pro‐
112 grams in a test suite. Filters have the following form:
113 test_program_name[:test_case_name]
115 Where ‘test_program_name’ is the name of a test program or a subdirectory
117 in the test suite, and ‘test_case_name’ is the name of a test case.
118
120 The kyua report command returns 0 if no filters were specified or if all
121 filters match one or more test cases. If any filter fails to match any
122 test case, the command returns 1.
123 Additional exit codes may be returned as described in kyua(1).
125
127 Workflow with results files
128 Let's say you run the following command twice in a row:
129 kyua test -k /usr/tests/Kyuafile
131 The two executions will generate two files with names like these:
133 ~/.kyua/store/results.usr_tests.20140731-150500-196784.db
135 ~/.kyua/store/results.usr_tests.20140731-151730-997451.db
136 Taking advantage of the default naming scheme, the following commands
138 would all generate a report for the results of the latest execution of
139 the test suite:
140 cd /usr/tests && kyua report
142 cd /usr/tests && kyua report --results-file=LATEST
143 kyua report --results-file=/usr/tests
144 kyua report --results-file=usr_tests
145 kyua report --results-file=usr_tests.20140731-151730-997451
146 But it is also possible to explicitly load data for older runs or from
148 explicitly-named files:
149 kyua report \
151 --results-file=usr_tests.20140731-150500-196784
152 kyua report \
153 --results-file=~/.kyua/store/results.usr_tests.20140731-150500-196784.db
154
156 kyua(1), kyua-report-html(1), kyua-report-junit(1)
157BSD October 13, 2014 BSD