1TRIAL(1) TRIAL(1)
2
6 trial - run unit tests
7
9 trial [ options ] [ file | package | module | TestCase | testmethod ]
10 ...
11 trial --help | -h
13
15 trial loads and executes a suite of unit tests, obtained from modules,
16 packages and files listed on the command line.
17 trial will take either filenames or fully qualified Python names as
19 arguments. Thus `trial myproject/foo.py', `trial myproject.foo' and
20 `trial myproject.foo.SomeTestCase.test_method' are all valid ways to
21 invoke trial. Multiple such arguments are also accepted, and their
22 order will determine the order in which the corresponding tests are
23 run.
24 After running the given test suite, the default test reporter prints a
26 summary of the test run. This consists of the word "PASSED" (if all
27 tests ran as expected) or "FAILED" (if any test behaved unexpectedly)
28 followed by a count of the different kinds of test results encountered.
29 The possible kinds of test results includes:
30 successes
32 Tests that passed all their assertions and completed without
33 error. These are marked "PASSED" in the normal test output.
34 failures
36 Tests that failed an assertion, called self.fail() or explicitly
37 raised self.failureException for some reason. These are marked
38 "FAILED" in the normal test output.
39 errors Tests that raised an unexpected exception (including Assertion‐
41 Error), tests that caused the tearDown() method to raise an
42 exception, tests that run for longer than the timeout interval,
43 tests that caused something to call twisted.python.log.err()
44 without subsequently calling self.flushLoggedErrors(), tests
45 that leave the reactor in an unclean state, etc. These are
46 marked "ERROR" in the normal test output.
47 Note that because errors can be caused after the actual test
49 method returns, it is possible for a single test to be reported
50 as both an error and a failure, and hence the total number of
51 test results can be greater than the total number of tests exe‐
52 cuted.
53 skips Tests that were skipped, usually because of missing dependen‐
55 cies. These are marked "SKIPPED" in the normal test output.
56 expectedFailures
58 Tests that failed, but were expected to fail, usually because
59 the test is for a feature that hasn't been implemented yet.
60 These are marked "TODO" in the normal test output.
61 unexpectedSuccesses
63 Tests that should have been listed under expectedFailures,
64 except that for some reason the test succeeded. These are marked
65 "SUCCESS!?!" in the normal test output.
66
68 -j, --jobs number
69 Set the number of process workers to run. It conflicts with the
70 debug, exitfirst and profile options.
71 -b, --debug
73 Run the tests in a debugger. If that debugger is 'pdb' (which is
74 the default if unspecified), a `.pdbrc' will be loaded from the
75 current directory if it exists. Also does post-mortem debugging
76 on exceptions.
77 -B, --debug-stacktraces
79 Report Deferred creation and callback stack traces.
80 --coverage
82 Generate coverage information in the `coverage' subdirectory of
83 the trial temp directory (`_trial_temp' by default). For each
84 Python module touched by the execution of the given tests, a
85 file will be created in the coverage directory named for the
86 module's fully-qualified name with the suffix `.cover'. For
87 example, because the trial test runner is written in Python, the
88 coverage directory will almost always contain a file named
89 `twisted.trial.runner.cover'.
90 Each `.cover' file contains a copy of the Python source of the
92 module in question, with a prefix at the beginning of each line
93 containing coverage information. For lines that are not exe‐
94 cutable (blank lines, comments, etc.) the prefix is blank. For
95 executable lines that were run in the course of the test suite,
96 the prefix is a number indicating the number of times that line
97 was executed. The string `>>>>>>' prefixes executable lines
98 that were not executed in the course of the test suite.
99 Note that this functionality uses Python's sys.settrace() func‐
101 tion, so tests that call sys.settrace() themselves are likely to
102 break trial's coverage functionality.
103 --debugger
105 Specifies the debugger to use when the --debug option is passed.
106 The argument should be the fully qualified name of an object
107 that implements the same interface as the standard library's
108 `pdb'.
109 --disablegc
111 Disable the garbage collector for the duration of the test run.
112 As each test is run, trial saves the TestResult objects, which
113 means that Python's garbage collector has more non-garbage
114 objects to wade through, making each garbage-collection run
115 slightly slower. Disabling garbage collection entirely will make
116 some test suites complete faster (contrast --force-gc, below),
117 at the cost of increasing (possibly greatly) memory consumption.
118 This option also makes tests slightly more deterministic, which
119 might help debugging in extreme circumstances.
120 -e, --rterrors
122 Print tracebacks to standard output as soon as they occur.
123 --force-gc
125 Run gc.collect() before and after each test case. This can be
126 used to isolate errors that occur when objects get collected.
127 This option would be the default, except it makes tests run
128 about ten times slower.
129 -h, --help
131 Print a usage message to standard output, then exit.
132 --help-order
134 Print a list of possible orders that TestCase test methods can
135 be run in, then exit. The orders can be used with the --order
136 option described below.
137 --help-reporters
139 Print a list of valid reporters to standard output, then exit.
140 Reporters can be selected with the --reporter option described
141 below.
142 --help-reactors
144 Print a list of possible reactors to standard output, then exit.
145 Not all listed reactors are available on every platform. Reac‐
146 tors can be selected with the --reactor option described below.
147 -l, --logfile logfile
149 Direct the log to a different file. The default file is
150 `test.log'. logfile is relative to _trial_temp.
151 -n, --dry-run
153 Go through all the tests and make them pass without running.
154 -N, --no-recurse
156 By default, trial recurses through packages to find every module
157 inside every subpackage. Unless, that is, you specify this
158 option.
159 --nopm Don't automatically jump into debugger for post-mortem analysis
161 of exceptions. Only usable in conjunction with --debug.
162 --order order
164 Specify what order to run the individual test methods within the
165 given TestCases. By default, they are run alphabetically. See
166 --help-order for a list of other valid values.
167 --profile
169 Run tests under the Python profiler.
170 -r, --reactor reactor
172 Choose which reactor to use. See --help-reactors for a list.
173 --recursionlimit limit
175 Set Python's recursion limit. See sys.setrecursionlimit().
176 --reporter reporter
178 Select the reporter to use for trial's output. Use the --help-
179 reporters option to see a list of valid reporters.
180 -x, --exitfirst
182 Stop the test run after the first test which does not succeed.
183 This includes failures, errors, or unexpected successes. Won't
184 work with the --jobs option currently.
185 --spew Print an insanely verbose log of everything that happens. Useful
187 when debugging freezes or locks in complex code.
188 --tbformat format
190 Format to display tracebacks with. Acceptable values are
191 `default', `brief' and `verbose'. `brief' produces tracebacks
192 that play nicely with Emacs' GUD.
193 --temp-directory directory
195 WARNING: Do not use this option unless you know what you are
196 doing. By default, trial creates a directory called _trial_temp
197 under the current working directory. When trial runs, it first
198 deletes this directory, then creates it, then changes into the
199 directory to run the tests. The log file and any coverage files
200 are stored here. Use this option if you wish to have trial run
201 in a directory other than _trial_temp. Be warned, trial will
202 delete the directory before re-creating it.
203 --testmodule filename
205 Ask trial to look into filename and run any tests specified
206 using the Emacs-style buffer variable `test-case-name'.
207 --unclean-warnings
209 As of Twisted 8.0, trial will report an error if the reactor is
210 left unclean at the end of the test. This option is provided to
211 assist in migrating from Twisted 2.5 to Twisted 8.0 and later.
212 Enabling this option will turn the errors into warnings.
213 -u, --until-failure
215 Keep looping the tests until one of them raises an error or a
216 failure. This is particularly useful for reproducing intermit‐
217 tent failures.
218 --version
220 Prints the Twisted version number and exit.
221 --without-module modulenames
223 Simulate the lack of the specified comma-separated list of mod‐
224 ules. This makes it look like the modules are not present in the
225 system, causing tests to check the behavior for that configura‐
226 tion.
227 -z, --random seed
229 Run the tests in random order using the specified seed. Don't
230 pass this option if you also are passing --order.
231
233 The latest version of the trial documentation can be found at
234 http://twistedmatrix.com/documents/current/core/howto/testing.html
235
237 Written by Jonathan M. Lange
238
240 To report a bug, visit http://twistedmatrix.com/trac/newticket
241
243 Copyright © 2003-2013 Twisted Matrix Laboratories
244 This is free software; see the source for copying conditions. There is
245 NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
246 PURPOSE.
247 Jun 2013 TRIAL(1)