1TMT(1) User Commands TMT(1)
2
6 tmt - Test Management Tool
7
9 The tmt python module and command-line tool implement the Metadata
10 Specification which allows storing all needed test execution data
11 directly within a git repository. In this way, it makes testing inde‐
12 pendent on any external test management system.
13 The Flexible Metadata Format fmf is used to store data in both human
15 and machine readable way close to the source code. Thanks to inheri‐
16 tance and elasticity metadata are organized in the structure effi‐
17 ciently, preventing unnecessary duplication.
18 The tool provides a user-friendly way to create, debug and easily run
20 tests from your laptop across different environments. It also allows to
21 easily convert old metadata, list and filter available tests and verify
22 them against the L1 specification.
23 Plans are used to group tests and configure individual test steps
25 defined by the L2 specification. They describe how to select tests for
26 execution, how to provision the environment, how to prepare it for
27 testing or how the test results should be reported.
28 Stories, defined by the L3 specification, can be used to track imple‐
30 mentation, test and documentation coverage for individual features or
31 requirements. Thanks to this you can track everything in one place,
32 including the project implementation progress.
33
35 Command line usage is straightforward:
36 tmt command [options]
38
40 Let's see which tests, plans and stories are available:
41 tmt
43 Initialize the metadata tree in the current directory, optionally with
45 example content based on templates:
46 tmt init
48 tmt init --template base
49 Run all or selected steps for each plan:
51 tmt run
53 tmt run discover
54 tmt run prepare execute
55 List tests, show details, check against the specification:
57 tmt test ls
59 tmt test show
60 tmt test lint
61 Create a new test, import test metadata from other formats:
63 tmt test create
65 tmt test import
66 List plans, show details, check against the specification:
68 tmt plan ls
70 tmt plan show
71 tmt plan lint
72 List stories, check details, show coverage status:
74 tmt story ls
76 tmt story show
77 tmt story coverage
78 Many commands support regular expression filtering and other specific
80 options:
81 tmt story ls cli
83 tmt story show create
84 tmt story coverage --implemented
85 Check help message of individual commands for the full list of avail‐
87 able options.
88
90 Here is the list of the most frequently used commands and options.
91 Run
93 The run command is used to execute test steps. By default all test
94 steps are run. See the L2 Metadata specification for detailed descrip‐
95 tion of individual steps. Here is a brief overview:
96 discover
98 Gather information about test cases to be executed.
99 provision
101 Provision an environment for testing or use localhost.
102 prepare
104 Prepare the environment for testing.
105 execute
107 Run tests using the specified executor.
108 report Provide test results overview and send reports.
110 finish Perform the finishing tasks and clean up provisioned guests.
112 Test
114 Manage tests (L1 metadata). Check available tests, inspect their meta‐
115 data, gather old metadata from various sources and stored them in the
116 new fmf format.
117 ls List available tests.
119 show Show test details.
121 lint Check tests against the L1 metadata specification.
123 create Create a new test based on given template.
125 import Convert old test metadata into the new fmf format.
127 Plan
129 Manage test plans (L2 metadata). Search for available plans. Explore
130 detailed test step configuration.
131 ls List available plans.
133 show Show plan details.
135 lint Check plans against the L2 metadata specification.
137 Story
139 Manage user stories. Check available user stories. Explore coverage
140 (test, implementation, documentation).
141 ls List available stories.
143 show Show story details.
145 coverage
147 Show code, test and docs coverage for given stories.
148 export Export selected stories into desired format.
150 Utils
152 Various utility options.
153 --root PATH
155 Path to the metadata tree, current directory used by default.
156 --verbose
158 Print additional information.
159 --debug
161 Turn on debugging output.
162 Check help message of individual commands for the full list of avail‐
164 able options.
165
167 The main tmt package provides the core features with a minimal set of
168 dependencies:
169 sudo dnf install tmt
171 In order to enable additional functionality, such as particular provi‐
173 sion or report plugins, install the respective subpackage:
174 sudo dnf install tmt-test-convert
176 sudo dnf install tmt-report-html
177 sudo dnf install tmt-provision-container
178 sudo dnf install tmt-provision-virtual
179 If you don't care about disk space and want to have all available fea‐
181 tures right at hand install everything:
182 sudo dnf install tmt-all
184 For RHEL 8 and CentOS 8, first make sure that you have enabled the EPEL
186 repository:
187 sudo dnf install epel-release
189 sudo dnf install tmt
190 Impatient to try the fresh features as soon as possible? Install the
192 latest greatest version from the copr repository:
193 sudo dnf copr enable psss/tmt
195 sudo dnf install tmt
196 Not sure, just want to try out how it works? Experiment safely and eas‐
198 ily inside a container:
199 podman run -it --rm quay.io/testing-farm/tmt bash
201 podman run -it --rm quay.io/testing-farm/tmt-all bash
202 When installing using pip you might need to install additional packages
204 on your system:
205 sudo dnf install gcc {python3,libvirt,krb5,libpq}-devel
207 pip install --user tmt
208 Note: You can omit the --user flag if in a virtual environment.
210
212 In order to experiment, play with the latest bits and develop improve‐
213 ments it is best to use a virtual environment:
214 mkvirtualenv tmt
216 git clone https://github.com/psss/tmt
217 cd tmt
218 pip install -e .
219 Install python3-virtualenvwrapper to easily create and enable virtual
221 environments using mkvirtualenv and workon. Note that if you have
222 freshly installed the package you need to open a new shell session to
223 enable the wrapper functions.
224 The main tmt package contains only the core dependencies. For building
226 documentation, testing changes, importing/exporting test cases or
227 advanced provisioning options install the extra deps:
228 pip install '.[docs]'
230 pip install '.[tests]'
231 pip install '.[convert]'
232 pip install '.[provision]'
233 Or simply install all extra dependencies to make sure you have every‐
235 thing needed for the tmt development ready on your system:
236 pip install '.[all]'
238
240 The following exit codes are returned from tmt run. Note that you can
241 use the --quiet option to completely disable output and only check for
242 the exit code.
243 0 At least one test passed, there was no fail, warn or error.
245 1 There was a fail or warn identified, but no error.
247 2 Errors occured during test execution.
249 3 No test results found.
251
253 The following environment variables can be used to modify behaviour of
254 the tmt command.
255 TMT_DEBUG
257 Enable the desired debug level. Most of the commands support
258 levels from 1 to 3. However, some of the plugins go even deeper
259 when needed.
260
262 Git: https://github.com/psss/tmt
263 Docs: http://tmt.readthedocs.io/
265 Stories: https://tmt.readthedocs.io/en/latest/stories.html
267 Issues: https://github.com/psss/tmt/issues
269 Releases: https://github.com/psss/tmt/releases
271 Copr: http://copr.fedoraproject.org/coprs/psss/tmt
273 PIP: https://pypi.org/project/tmt/
275 Travis: https://travis-ci.org/psss/tmt
277 Coveralls: https://coveralls.io/github/psss/tmt
279 Metadata Specification: https://tmt.readthedocs.io/en/latest/spec.html
281 Flexible Metadata Format: http://fmf.readthedocs.io/
283 Packit & Testing Farm: https://packit.dev/testing-farm/
285
287 Petr Šplíchal, Miro Hrončok, Alexander Sosedkin, Lukáš Zachar, Petr
288 Menšík, Leoš Pol, Miroslav Vadkerti, Pavel Valena, Jakub Heger, Honza
289 Horák, Rachel Sibley, František Nečas, Michal Ruprich, Martin Kyral,
290 Miloš Prchlík, Tomáš Navrátil, František Lachman, Patrik Kis, Ondrej
291 Mosnáček, Andrea Ficková, Denis Karpelevich and Michal Srb.
292
294 Copyright (c) 2019 Red Hat, Inc.
295 This program is free software; you can redistribute it and/or modify it
297 under the terms of the MIT License.
298 September 2019 TMT(1)