1ZOPE.TESTRUNNER(1)               User Commands              ZOPE.TESTRUNNER(1)
2
3
4

NAME

6       zope.testrunner - Zope testrunner script
7

DESCRIPTION

9       usage: zope-testrunner-3.7 [-h] [--package PACKAGE] [--module MODULE]
10
11       [--test TEST] [--unit] [--non-unit] [--layer LAYER]
12              [-a  AT_LEVEL] [--all] [--list-tests] [--require-unique] [--ver‐
13              bose] [--quiet] [--progress]  [--no-progress]  [--auto-progress]
14              [--color] [--no-color] [--auto-color] [--subunit] [--subunit-v2]
15              [--slow-test  N]  [-1]   [--show-secondary-failures]   [--ndiff]
16              [--udiff]       [--cdiff]      [--ignore-new-thread      REGEXP]
17              [--stop-on-error]   [--post-mortem]   [--gc   GC]   [--gc-option
18              {DEBUG_INSTANCES,DEBUG_LEAK,DEBUG_UNCOL‐
19              LECTABLE,DEBUG_STATS,DEBUG_SAVEALL,DEBUG_COL‐
20              LECTABLE,DEBUG_OBJECTS}]  [--repeat REPEAT] [--report-refcounts]
21              [--coverage COVERAGE] [--profile  {cProfile}]  [--profile-direc‐
22              tory  PROF_DIR]  [--path  PATH] [--test-path TEST_PATH] [--pack‐
23              age-path PACKAGE_PATH PACKAGE_PATH] [--tests-pattern  TESTS_PAT‐
24              TERN]     [--suite-name     SUITE_NAME]     [--test-file-pattern
25              TEST_FILE_PATTERN]   [--ignore_dir    IGNORE_DIR]    [--shuffle]
26              [--shuffle-seed   SHUFFLE_SEED]   [--version]   [-j   PROCESSES]
27              [--keepbytecode]      [--usecompiled]       [--exit-with-status]
28              [legacy_module_filter] [legacy_test_filter]
29
30       Discover and run unittest tests
31
32   positional arguments:
33       legacy_module_filter
34              DEPRECATED: Prefer to use --module.
35
36       legacy_test_filter
37              DEPRECATED: Prefer to use --test.
38
39   optional arguments:
40       -h, --help
41              show this help message and exit
42
43   Searching and filtering:
44              Options in this group are used to define which tests to run.
45
46       --package PACKAGE, --dir PACKAGE, -s PACKAGE
47              Search  the  given  package's directories for tests. This can be
48              specified more than once to run tests in multiple parts  of  the
49              source  tree.  For example, if refactoring interfaces, you don't
50              want to see the way you have broken setups for  tests  in  other
51              packages.   You *just* want to run the interface tests. Packages
52              are supplied as dotted names. For  compatibility  with  the  old
53              test  runner,  forward and backward slashed in package names are
54              converted to dots. (In the special case of packages spread  over
55              multiple  directories,  only  directories within the test search
56              path are searched. See the --path option.)
57
58       --module MODULE, -m MODULE
59              Specify a test-module filter as a regular expression.  This is a
60              case-sensitive  regular  expression,  used in search (not match)
61              mode, to limit which test modules are searched  for  tests.  The
62              regular  expressions are checked against dotted module names. In
63              an extension  of  Python  regexp  notation,  a  leading  "!"  is
64              stripped  and  causes  the  sense  of the remaining regexp to be
65              negated (so "!bc" matches any string that does not  match  "bc",
66              and  vice versa). The option can be specified multiple test-mod‐
67              ule filters. Test modules matching any of the test  filters  are
68              searched.  If  no test-module filter is specified, then all test
69              modules are used.
70
71       --test TEST, -t TEST
72              Specify a test  filter  as  a  regular  expression.  This  is  a
73              case-sensitive  regular  expression,  used in search (not match)
74              mode, to limit which tests are run. In an  extension  of  Python
75              regexp  notation, a leading "!" is stripped and causes the sense
76              of the remaining regexp to be  negated  (so  "!bc"  matches  any
77              string that does not match "bc", and vice versa). The option can
78              be specified multiple test filters. Tests matching  any  of  the
79              test  filters are included. If no test filter is specified, then
80              all tests are run.
81
82       --unit, -u
83              Run only unit tests, ignoring any layer options.
84
85       --non-unit, -f
86              Run tests other than unit tests.
87
88       --layer LAYER
89              Specify a test layer to run. The option can  be  given  multiple
90              times to specify more than one layer. If not specified, all lay‐
91              ers are run. It is common for  the  running  script  to  provide
92              default  values  for  this  option. Layers are specified regular
93              expressions, used in search mode, for dotted  names  of  objects
94              that  define a layer. In an extension of Python regexp notation,
95              a leading "!" is stripped and causes the sense of the  remaining
96              regexp  to be negated (so "!bc" matches any string that does not
97              match "bc", and vice  versa).  The  layer  named  'zope.testrun‐
98              ner.layer.UnitTests'  is  reserved for unit tests, however, take
99              note of the --unit and non-unit options.
100
101       -a AT_LEVEL, --at-level AT_LEVEL
102              Run the tests at the given level. Any test  at  a  level  at  or
103              below  this  is  run, any test at a level above this is not run.
104              Level 0 runs all tests.
105
106       --all  Run tests at all levels.
107
108       --list-tests
109              List all tests that matched your filters. Do not run any tests.
110
111       --require-unique
112              Require that all test IDs be unique and raise an error if dupli‐
113              cates are encountered.
114
115   Reporting:
116              Reporting options control basic aspects of test-runner output
117
118       --verbose, -v
119              Make output more verbose. Increment the verbosity level.
120
121       --quiet, -q
122              Make the output minimal, overriding any verbosity options.
123
124       --progress, -p
125              Output progress status
126
127       --no-progress
128              Do  not  output progress status. This is the default, but can be
129              used to counter a previous use of --progress or -p.
130
131       --auto-progress
132              Output progress status, but only when stdout is a terminal.
133
134       --color, -c
135              Colorize the output.
136
137       --no-color, -C
138              Do not colorize the output. This is the default, but can be used
139              to counter a previous use of --color or -c.
140
141       --auto-color
142              Colorize the output, but only when stdout is a terminal.
143
144       --subunit
145              Use subunit v1 output. Will not be colorized.
146
147       --subunit-v2
148              Use subunit v2 output. Will not be colorized.
149
150       --slow-test N
151              With  -c  and -vvv, highlight tests that take longer than N sec‐
152              onds (default: 10).
153
154       -1, --hide-secondary-failures
155              Report only the first failure in a doctest. (Examples after  the
156              failure are still executed, in case they do any cleanup.)
157
158       --show-secondary-failures
159              Report  all  failures in a doctest. This is the default, but can
160              be used to counter a default use of -1 or --hide-secondary-fail‐
161              ures.
162
163       --ndiff
164              When  there  is  a  doctest failure, show it as a diff using the
165              ndiff.py utility.
166
167       --udiff
168              When there is a doctest failure, show it as a unified diff.
169
170       --cdiff
171              When there is a doctest failure, show it as a context diff.
172
173       --ignore-new-thread REGEXP
174              If a thread with this name is left behind, don't report this  at
175              the  end.  This  is a case-sensitive regular expression, used in
176              match mode. This option can be used multiple times. If a  thread
177              name matches any of them, it will be ignored.
178
179   Analysis:
180              Analysis options provide tools for analysing test output.
181
182       --stop-on-error, --stop, -x
183              Stop running tests after first test failure or error.
184
185       --post-mortem, --pdb, -D
186              Enable post-mortem debugging of test failures
187
188       --gc GC, -g GC
189              Set the garbage collector generation threshold. This can be used
190              to stress memory and  gc  correctness.  Some  crashes  are  only
191              reproducible  when the threshold is set to 1 (aggressive garbage
192              collection). Do "--gc 0" to  disable  garbage  collection  alto‐
193              gether.  The --gc option can be used up to 3 times to specify up
194              to 3 of the 3 Python gc_threshold settings.
195
196       --gc-option                    {DEBUG_INSTANCES,DEBUG_LEAK,DEBUG_UNCOL‐
197       LECTABLE,DEBUG_STATS,DEBUG_SAVEALL,DEBUG_COLLECTABLE,DEBUG_OBJECTS}, -G
198       {DEBUG_INSTANCES,DEBUG_LEAK,DEBUG_UNCOL‐
199       LECTABLE,DEBUG_STATS,DEBUG_SAVEALL,DEBUG_COLLECTABLE,DEBUG_OBJECTS}
200              Set  a Python gc-module debug flag. This option can be used more
201              than once to set multiple flags.
202
203       --repeat REPEAT, -N REPEAT
204              Repeat the tests the given number of times. This option is  used
205              to  make  sure  that  tests leave their environment in the state
206              they found it and, with the --report-refcounts  option  to  look
207              for memory leaks.
208
209       --report-refcounts, -r
210              After each run of the tests, output a report summarizing changes
211              in refcounts by object type.  This  option  that  requires  that
212              Python was built with the --with-pydebug option to configure.
213
214       --coverage COVERAGE
215              Perform  code-coverage analysis, saving trace data to the direc‐
216              tory with the given name. A code coverage summary is printed  to
217              standard out.
218
219       --profile {cProfile}
220              Run  the  tests  under  cProfiler  and display the top 50 stats,
221              sorted by cumulative time and number of calls.
222
223       --profile-directory PROF_DIR
224              Directory  for  temporary  profiler  files.  All   files   named
225              tests_profile.*.prof  in  this directory will be removed. If you
226              intend to run multiple instances of the test runner in parallel,
227              be sure to tell them to use different directories, so they won't
228              step on each other's toes.
229
230   Setup:
231              Setup options are normally supplied by  the  testrunner  script,
232              although they can be overridden by users.
233
234       --path PATH
235              Specify a path to be added to Python's search path.  This option
236              can be used multiple times to specify multiple search paths. The
237              path  is  usually  specified  by  the test-runner script itself,
238              rather than by users of the script, although it can be  overrid‐
239              den  by  users.   Only tests found in the path will be run. This
240              option also specifies directories to be searched for tests.  See
241              the search_directory.
242
243       --test-path TEST_PATH
244              Specify  a  path  to be searched for tests, but not added to the
245              Python search path. This option can be used  multiple  times  to
246              specify  multiple search paths. The path is usually specified by
247              the test-runner script itself,  rather  than  by  users  of  the
248              script, although it can be overridden by users. Only tests found
249              in the path will be run.
250
251       --package-path PACKAGE_PATH PACKAGE_PATH
252              Specify a path to be searched for tests, but not  added  to  the
253              Python  search  path.  Also specify a package for files found in
254              this path. This is  used  to  deal  with  directories  that  are
255              stitched  into  packages  that  are  not  otherwise searched for
256              tests. This option takes 2 arguments. The first is a path  name.
257              The second is the package name. This option can be used multiple
258              times to specify multiple search  paths.  The  path  is  usually
259              specified by the test-runner script itself, rather than by users
260              of the script, although it can  be  overridden  by  users.  Only
261              tests found in the path will be run.
262
263       --tests-pattern TESTS_PATTERN
264              The test runner looks for modules containing tests. It uses this
265              pattern to identify these modules. The  modules  may  be  either
266              packages or python files. If a test module is a package, it uses
267              the value given by  the  test-file-pattern  to  identify  python
268              files within the package containing tests.
269
270       --suite-name SUITE_NAME
271              Specify the name of the object in each test_module that contains
272              the module's test suite.
273
274       --test-file-pattern TEST_FILE_PATTERN
275              Specify a pattern for identifying python files  within  a  tests
276              package. See the documentation for the --tests-pattern option.
277
278       --ignore_dir IGNORE_DIR
279              Specifies  the  name  of  a directory to ignore when looking for
280              tests.
281
282       --shuffle
283              Shuffles the order in which tests are ran.
284
285       --shuffle-seed SHUFFLE_SEED
286              Value used to initialize the tests shuffler. Specify a value  to
287              create repeatable random ordered tests.
288
289   Other:
290              Other options
291
292       --version
293              Print the version of the testrunner, and exit.
294
295       -j PROCESSES
296              Use  up  to given number of parallel processes to execute tests.
297              May decrease test run time substantially. Defaults to 1.
298
299       --keepbytecode, -k
300              Normally, the test runner scans the  test  paths  and  the  test
301              directories  looking  for  and deleting pyc or pyo files without
302              corresponding py files. This is to prevent spurious  test  fail‐
303              ures  due  to finding compiled modules where source modules have
304              been deleted. This scan can be time consuming. Using this option
305              disables  this scan. If you know you haven't removed any modules
306              since last running the tests, can make  the  test  run  go  much
307              faster.
308
309       --usecompiled
310              Normally,  a  package must contain an __init__.py file, and only
311              .py files can contain test code. When this option is  specified,
312              compiled  Python  files  (.pyc  and .pyo) can be used instead: a
313              directory containing __init__.pyc or __init__.pyo is  also  con‐
314              sidered  to  be a package, and if file XYZ.py contains tests but
315              is absent while XYZ.pyc or  XYZ.pyo  exists  then  the  compiled
316              files will be used. This is necessary when running tests against
317              a tree where the .py files have been removed  after  compilation
318              to .pyc/.pyo. Use of this option implies --keepbytecode.
319
320       --exit-with-status
321              DEPRECATED: The test runner will always exit with a status.
322
323
324
325zope.testrunner version 5.0       March 2019                ZOPE.TESTRUNNER(1)
Impressum