1TRIAL(1) TRIAL(1)
2
6 trial - run unit tests
7
9 trial [ options ] [ file | package | module | TestCase | testmethod ]
10 ...
11 trial --help | -h
13
15 trial loads and executes a suite of unit tests, obtained from modules,
16 packages and files listed on the command line.
17 trial will take either filenames or fully qualified Python names as
19 arguments. Thus `trial myproject/foo.py', `trial myproject.foo' and
20 `trial myproject.foo.SomeTestCase.test_method' are all valid ways to
21 invoke trial.
22 After running the given test suite, the default test reporter prints a
24 summary of the test run. This consists of the word "PASSED" (if all
25 tests ran as expected) or "FAILED" (if any test behaved unexpectedly)
26 followed by a count of the different kinds of test results encountered.
27 The possible kinds of test results includes:
28 successes
30 Tests that passed all their assertions and completed without
31 error. These are marked "PASSED" in the normal test output.
32 failures
34 Tests that failed an assertion, called self.fail() or explicitly
35 raised self.failureException for some reason. These are marked
36 "FAILED" in the normal test output.
37 errors Tests that raised an unexpected exception (including Assertion‐
39 Error), tests that caused the tearDown() method to raise an
40 exception, tests that run for longer than the timeout interval,
41 tests that caused something to call twisted.python.log.err()
42 without subsequently calling self.flushLoggedErrors(), tests
43 that leave the reactor in an unclean state, etc. These are
44 marked "ERROR" in the normal test output.
45 Note that because errors can be caused after the actual test
47 method returns, it is possible for a single test to be reported
48 as both an error and a failure, and hence the total number of
49 test results can be greater than the total number of tests exe‐
50 cuted.
51 skips Tests that were skipped, usually because of missing dependen‐
53 cies. These are marked "SKIPPED" in the normal test output.
54 expectedFailures
56 Tests that failed, but were expected to fail, usually because
57 the test is for a feature that hasn't been implemented yet.
58 These are marked "TODO" in the normal test output.
59 unexpectedSuccesses
61 Tests that should have been listed under expectedFailures,
62 except that for some reason the test succeeded. These are marked
63 "SUCCESS!?!" in the normal test output.
64
66 -b, --debug
67 Run the tests in the Python debugger. Also does post-mortem
68 debugging on exceptions. Will load `.pdbrc' from current direc‐
69 tory if it exists.
70 -B, --debug-stacktraces
72 Report Deferred creation and callback stack traces
73 --coverage
75 Generate coverage information in the `coverage' subdirectory of
76 the trial temp directory (`_trial_temp' by default). For each
77 Python module touched by the execution of the given tests, a
78 file will be created in the coverage directory named for the
79 module's fully-qualified name with the suffix `.cover'. For
80 example, because the trial test runner is written in Python, the
81 coverage directory will almost always contain a file named
82 `twisted.trial.runner.cover'.
83 Each `.cover' file contains a copy of the Python source of the
85 module in question, with a prefix at the beginning of each line
86 containing coverage information. For lines that are not exe‐
87 cutable (blank lines, comments, etc.) the prefix is blank. For
88 executable lines that were run in the course of the test suite,
89 the prefix is a number indicating the number of times that line
90 was executed. The string `>>>>>>' prefixes executable lines
91 that were not executed in the course of the test suite.
92 Note that this functionality uses Python's sys.settrace() func‐
94 tion, so tests that call sys.settrace() themselves are likely to
95 break trial's coverage functionality.
96 --disablegc
98 Disable the garbage collector for the duration of the test run.
99 As each test is run, trial saves the TestResult objects, which
100 means that Python's garbage collector has more non-garbage
101 objects to wade through, making each garbage-collection run
102 slightly slower. Disabling garbage collection entirely will make
103 some test suites complete faster (contrast --force-gc, below),
104 at the cost of increasing (possibly greatly) memory consumption.
105 This option also makes tests slightly more deterministic, which
106 might help debugging in extreme circumstances.
107 -e, --rterrors
109 Print tracebacks to standard output as soon as they occur
110 --force-gc
112 Run gc.collect() before and after each test case. This can be
113 used to isolate errors that occur when objects get collected.
114 This option would be the default, except it makes tests run
115 about ten times slower.
116 -h, --help
118 Print a usage message to standard output, then exit.
119 --help-reporters
121 Print a list of valid reporters to standard output, then exit.
122 Reporters can be selected with the --reporter option described
123 below.
124 --help-reactors
126 Print a list of possible reactors to standard output, then exit.
127 Not all listed reactors are available on every platform. Reac‐
128 tors can be selected with the --reactor option described below.
129 -l, --logfile logfile
131 Direct the log to a different file. The default file is
132 `test.log'. logfile is relative to _trial_temp.
133 -n, --dry-run
135 Go through all the tests and make them pass without running.
136 -N, --no-recurse
138 By default, trial recurses through packages to find every module
139 inside every subpackage. Unless, that is, you specify this
140 option.
141 --nopm Don't automatically jump into debugger for post-mortem analysis
143 of exceptions. Only usable in conjunction with --debug.
144 --profile
146 Run tests under the Python profiler.
147 -r, --reactor reactor
149 Choose which reactor to use. See --help-reactors for a list.
150 --recursionlimit
152 Set Python's recursion limit. See sys.setrecursionlimit()
153 --reporter
155 Select the reporter to use for trial's output. Use the --help-
156 reporters option to see a list of valid reporters.
157 --spew Print an insanely verbose log of everything that happens. Useful
159 when debugging freezes or locks in complex code.
160 --tbformat format
162 Format to display tracebacks with. Acceptable values are
163 `default', `brief' and `verbose'. `brief' produces tracebacks
164 that play nicely with Emacs' GUD.
165 --temp-directory directory
167 WARNING: Do not use this options unless you know what you are
168 doing. By default, trial creates a directory called _trial_temp
169 under the current working directory. When trial runs, it first
170 deletes this directory, then creates it, then changes into the
171 directory to run the tests. The log file and any coverage files
172 are stored here. Use this option if you wish to have trial run
173 in a directory other than _trial_temp. Be warned, trial will
174 delete the directory before re-creating it.
175 --testmodule filename
177 Ask trial to look into filename and run any tests specified
178 using the Emacs-style buffer variable `test-case-name'.
179 --unclean-warnings
181 As of Twisted 8.0, trial will report an error if the reactor is
182 left unclean at the end of the test. This option is provided to
183 assist in migrating from Twisted 2.5 to Twisted 8.0 and later.
184 Enabling this option will turn the errors into warnings.
185 -u, --until-failure
187 Keep looping the tests until one of them raises an error or a
188 failure. This is particularly useful for reproducing intermit‐
189 tent failures.
190 --version
192 Prints the Twisted version number and exit.
193 --without-module modulenames
195 Simulate the lack of the specified comma-separated list of mod‐
196 ules. This makes it look like the modules are not present in the
197 system, causing tests to check the behavior for that configura‐
198 tion.
199 -z, --random [seed]
201 Run the tests in random order using the specified seed.
202
204 The latest version of the trial documentation can be found at
205 http://twistedmatrix.com/documents/current/core/howto/testing.html
206
208 Written by Jonathan M. Lange
209
211 To report a bug, visit http://twistedmatrix.com/trac/newticket
212
214 Copyright © 2003-2011 Twisted Matrix Laboratories
215 This is free software; see the source for copying conditions. There is
216 NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
217 PURPOSE.
218 Oct 2007 TRIAL(1)