1TMT(1) User Commands TMT(1)
2
6 tmt - Test Management Tool
7
9 The tmt tool provides a user-friendly way to work with tests. You can
10 comfortably create new tests, safely and easily run tests across dif‐
11 ferent environments, review test results, debug test code and enable
12 tests in the CI using a consistent and concise config.
13 The python module and command-line tool implement the Metadata Specifi‐
15 cation which allows storing all needed test execution data directly
16 within a git repository. Together with possibility to reference remote
17 repositories it makes it easy to share test coverage across projects
18 and distros.
19 The Flexible Metadata Format fmf is used to store data in both human
21 and machine readable way close to the source code. Thanks to inheri‐
22 tance and elasticity metadata are organized in the structure effi‐
23 ciently, preventing unnecessary duplication.
24
26 There are several metadata levels defined by the specification:
27 Core attributes such as summary or description which are common across
29 all levels are defined by the special L0 metadata.
30 Tests, or L1 metadata, define attributes which are closely related to
32 individual test cases such as test script, framework, directory path
33 where the test should be executed, maximum test duration or packages
34 required to run the test.
35 Plans, also called L2 metadata, are used to group relevant tests and
37 enable them in the CI. They describe how to discover tests for execu‐
38 tion, how to provision the environment, how to prepare it for testing,
39 how to execute tests and report test results.
40 Stories, which implement the L3 metadata, can be used to track imple‐
42 mentation, test and documentation coverage for individual features or
43 requirements. Thanks to this you can track everything in one place, in‐
44 cluding the project implementation progress.
45
47 Command line usage is straightforward:
48 tmt command [options]
50
52 Let's see which tests, plans and stories are available:
53 tmt
55 Initialize the metadata tree in the current directory, optionally with
57 example content based on templates:
58 tmt init
60 tmt init --template base
61 Run all or selected steps for each plan:
63 tmt run
65 tmt run discover
66 tmt run prepare execute
67 List tests, show details, check against the specification:
69 tmt test ls
71 tmt test show
72 tmt test lint
73 Create a new test, import test metadata from other formats:
75 tmt test create
77 tmt test import
78 List plans, show details, check against the specification:
80 tmt plan ls
82 tmt plan show
83 tmt plan lint
84 List stories, check details, show coverage status:
86 tmt story ls
88 tmt story show
89 tmt story coverage
90 Many commands support regular expression filtering and other specific
92 options:
93 tmt story ls cli
95 tmt story show create
96 tmt story coverage --implemented
97 Check help message of individual commands for the full list of avail‐
99 able options.
100
102 Here is the list of the most frequently used commands and options.
103 Run
105 The run command is used to execute test steps. By default all test
106 steps are run. See the L2 Metadata specification for detailed descrip‐
107 tion of individual steps. Here is a brief overview:
108 discover
110 Gather information about test cases to be executed.
111 provision
113 Provision an environment for testing or use localhost.
114 prepare
116 Prepare the environment for testing.
117 execute
119 Run tests using the specified executor.
120 report Provide test results overview and send reports.
122 finish Perform the finishing tasks and clean up provisioned guests.
124 Test
126 Manage tests (L1 metadata). Check available tests, inspect their meta‐
127 data, gather old metadata from various sources and stored them in the
128 new fmf format.
129 ls List available tests.
131 show Show test details.
133 lint Check tests against the L1 metadata specification.
135 create Create a new test based on given template.
137 import Convert old test metadata into the new fmf format.
139 Plan
141 Manage test plans (L2 metadata). Search for available plans. Explore
142 detailed test step configuration.
143 ls List available plans.
145 show Show plan details.
147 lint Check plans against the L2 metadata specification.
149 Story
151 Manage user stories. Check available user stories. Explore coverage
152 (test, implementation, documentation).
153 ls List available stories.
155 show Show story details.
157 coverage
159 Show code, test and docs coverage for given stories.
160 export Export selected stories into desired format.
162 Utils
164 Various utility options.
165 --root PATH
167 Path to the metadata tree, current directory used by default.
168 --verbose
170 Print additional information.
171 --debug
173 Turn on debugging output.
174 Check help message of individual commands for the full list of avail‐
176 able options.
177
179 The main tmt package provides the core features with a minimal set of
180 dependencies:
181 sudo dnf install tmt
183 In order to enable additional functionality, such as particular provi‐
185 sion or report plugins, install the respective subpackage:
186 sudo dnf install tmt-test-convert
188 sudo dnf install tmt-report-html
189 sudo dnf install tmt-provision-container
190 sudo dnf install tmt-provision-virtual
191 If you don't care about disk space and want to have all available fea‐
193 tures right at hand install everything:
194 sudo dnf install tmt-all
196 For RHEL 8 and CentOS 8, first make sure that you have available the
198 EPEL repository. You might also have to enable additional repositories:
199 sudo dnf install https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm
201 sudo dnf config-manager --enable powertools # CentOS 8
202 sudo dnf config-manager --enable rhel-CRB # RHEL 8
203 sudo dnf install tmt
204 For plugins which cannot work outside of VPN and so live within its
206 walls you need to enable the internal copr repository first. Then you
207 can install either everything or only those you need:
208 sudo dnf install tmt-redhat-all
210 sudo dnf install tmt-redhat-*
211 Impatient to try the fresh features as soon as possible? Install the
213 latest greatest version from the copr repository:
214 sudo dnf copr enable psss/tmt
216 sudo dnf install tmt
217 Not sure, just want to try out how it works? Experiment safely and eas‐
219 ily inside a container:
220 podman run -it --rm quay.io/testing-farm/tmt bash
222 podman run -it --rm quay.io/testing-farm/tmt-all bash
223 When installing using pip you might need to install additional packages
225 on your system:
226 sudo dnf install gcc {python3,libvirt,krb5,libpq}-devel
228 pip install --user tmt
229 Note: You can omit the --user flag if in a virtual environment.
231
233 The rpm package includes a system wide script which enables the command
234 line completion for bash so no additional config should be needed. If
235 you use a different installation method or prefer another shell, see
236 the instructions below.
237 For Bash, add this to ~/.bashrc:
239 eval "$(_TMT_COMPLETE=source_bash tmt)"
241 For Zsh, add this to ~/.zshrc:
243 eval "$(_TMT_COMPLETE=source_zsh tmt)"
245 For Fish, add this to ~/.config/fish/completions/tmt.fish:
247 eval (env _TMT_COMPLETE=source_fish tmt)
249 Open a new shell to enable completion. Or run the eval command directly
251 in your current shell to enable it temporarily.
252
254 The following exit codes are returned from tmt run. Note that you can
255 use the --quiet option to completely disable output and only check for
256 the exit code.
257 0 At least one test passed, there was no fail, warn or error.
259 1 There was a fail or warn identified, but no error.
261 2 Errors occured during test execution.
263 3 No test results found.
265
267 The following environment variables can be used to modify behaviour of
268 the tmt command.
269 TMT_DEBUG
271 Enable the desired debug level. Most of the commands support
272 levels from 1 to 3. However, some of the plugins go even deeper
273 when needed.
274 NO_COLOR
276 Disable colors in the terminal output. Output only plain,
277 non-colored text. See https://no-color.org/ for more informa‐
278 tion.
279
281 Git: https://github.com/psss/tmt
282 Docs: http://tmt.readthedocs.io/
284 Stories: https://tmt.readthedocs.io/en/latest/stories.html
286 Issues: https://github.com/psss/tmt/issues
288 Releases: https://github.com/psss/tmt/releases
290 Copr: http://copr.fedoraproject.org/coprs/psss/tmt
292 PIP: https://pypi.org/project/tmt/
294 Travis: https://travis-ci.org/psss/tmt
296 Coveralls: https://coveralls.io/github/psss/tmt
298 Metadata Specification: https://tmt.readthedocs.io/en/latest/spec.html
300 Flexible Metadata Format: http://fmf.readthedocs.io/
302 Packit & Testing Farm: https://packit.dev/testing-farm/
304
306 Petr Šplíchal, Miro Hrončok, Alexander Sosedkin, Lukáš Zachar, Petr
307 Menšík, Leoš Pol, Miroslav Vadkerti, Pavel Valena, Jakub Heger, Honza
308 Horák, Rachel Sibley, František Nečas, Michal Ruprich, Martin Kyral,
309 Miloš Prchlík, Tomáš Navrátil, František Lachman, Patrik Kis, Ondrej
310 Mosnáček, Andrea Ficková, Denis Karpelevich, Michal Srb, Jan Ščotka,
311 Artem Zhukov, Vinzenz Feenstra, Inessa Vasilevskaya, Štěpán Němec,
312 Robin Hack and Yulia Kopkova.
313
315 Copyright (c) 2019 Red Hat, Inc.
316 This program is free software; you can redistribute it and/or modify it
318 under the terms of the MIT License.
319 September 2019 TMT(1)