dotnet-test(1)

1dotnet-test(1)                .NET Documentation                dotnet-test(1)
2
3
4

dotnet test

6       This article applies to: ✔️ .NET Core 3.1 SDK and later versions
7

NAME

9       dotnet-test - .NET test driver used to execute unit tests.
10

SYNOPSIS

12              dotnet test [<PROJECT> | <SOLUTION> | <DIRECTORY> | <DLL> | <EXE>]
13                  [--test-adapter-path <ADAPTER_PATH>]
14                  [-a|--arch <ARCHITECTURE>]
15                  [--blame]
16                  [--blame-crash]
17                  [--blame-crash-dump-type <DUMP_TYPE>]
18                  [--blame-crash-collect-always]
19                  [--blame-hang]
20                  [--blame-hang-dump-type <DUMP_TYPE>]
21                  [--blame-hang-timeout <TIMESPAN>]
22                  [-c|--configuration <CONFIGURATION>]
23                  [--collect <DATA_COLLECTOR_NAME>]
24                  [-d|--diag <LOG_FILE>]
25                  [-f|--framework <FRAMEWORK>]
26                  [-e|--environment <NAME="VALUE">]
27                  [--filter <EXPRESSION>]
28                  [--interactive]
29                  [-l|--logger <LOGGER>]
30                  [--no-build]
31                  [--nologo]
32                  [--no-restore]
33                  [-o|--output <OUTPUT_DIRECTORY>]
34                  [--os <OS>]
35                  [--results-directory <RESULTS_DIR>]
36                  [-r|--runtime <RUNTIME_IDENTIFIER>]
37                  [-s|--settings <SETTINGS_FILE>]
38                  [-t|--list-tests]
39                  [-v|--verbosity <LEVEL>]
40                  [<args>...]
41                  [[--] <RunSettings arguments>]
42
43              dotnet test -h|--help
44

DESCRIPTION

46       The  dotnet test command is used to execute unit tests in a given solu‐
47       tion.  The dotnet test command builds the solution and runs a test host
48       application  for each test project in the solution.  The test host exe‐
49       cutes tests in the given project using a test framework,  for  example:
50       MSTest,  NUnit,  or  xUnit,  and reports the success or failure of each
51       test.  If all tests are successful, the test runner returns 0 as an ex‐
52       it code; otherwise if any test fails, it returns 1.
53
54       For multi-targeted projects, tests are run for each targeted framework.
55       The test host and the unit test framework are packaged as  NuGet  pack‐
56       ages and are restored as ordinary dependencies for the project.
57
58       Test  projects specify the test runner using an ordinary <PackageRefer‐
59       ence> element, as seen in the following sample project file:
60
61       [!code-xmlXUnit Basic Template]
62
63       Where Microsoft.NET.Test.Sdk is the test host, xunit is the test frame‐
64       work.   And  xunit.runner.visualstudio  is a test adapter, which allows
65       the xUnit framework to work with the test host.
66
67   Implicit restore
68       You don’t have to run dotnet restore because it’s run implicitly by all
69       commands  that  require  a restore to occur, such as dotnet new, dotnet
70       build, dotnet run, dotnet test, dotnet publish, and  dotnet  pack.   To
71       disable implicit restore, use the --no-restore option.
72
73       The  dotnet  restore command is still useful in certain scenarios where
74       explicitly restoring makes sense, such as continuous integration builds
75       in  Azure  DevOps  Services or in build systems that need to explicitly
76       control when the restore occurs.
77
78       For information about how to manage NuGet feeds, see the dotnet restore
79       documentation.
80
81   Workload manifest downloads
82       When  you  run  this  command,  it initiates an asynchronous background
83       download of advertising manifests for workloads.  If  the  download  is
84       still running when this command finishes, the download is stopped.  For
85       more information, see Advertising manifests.
86

ARGUMENTS

88       • PROJECT | SOLUTION | DIRECTORY | DLL | EXE
89
90         • Path to the test project.
91
92         • Path to the solution.
93
94         • Path to a directory that contains a project or a solution.
95
96         • Path to a test project .dll file.
97
98         • Path to a test project .exe file.
99
100         If not specified, the effect is the same as using the DIRECTORY argu‐
101         ment to specify the current directory.
102

OPTIONS

104              [!WARNING] Breaking changes in options:
105
106              • Starting  in  .NET  7:  switch  -a  to alias --arch instead of
107                --test-adapter-path
108
109              • Starting in .NET 7: switch -r to alias  --runtime  instead  of
110                --results-dir
111
112       • --test-adapter-path <ADAPTER_PATH>
113
114         Path to a directory to be searched for additional test adapters.  On‐
115         ly .dll files with suffix .TestAdapter.dll  are  inspected.   If  not
116         specified, the directory of the test .dll is searched.
117
118         Short form -a available in .NET SDK versions earlier than 7.
119
120       • --arch <ARCHITECTURE>
121
122         Specifies  the  target  architecture.  This is a shorthand syntax for
123         setting the Runtime Identifier (RID), where  the  provided  value  is
124         combined  with  the  default RID.  For example, on a win-x64 machine,
125         specifying --arch x86 sets the RID to win-x86.  If you use  this  op‐
126         tion, don’t use the -r|--runtime option.  Available since .NET 6 Pre‐
127         view 7.
128
129       • --blame
130
131         Runs the tests in blame mode.  This option is  helpful  in  isolating
132         problematic tests that cause the test host to crash.  When a crash is
133         detected, it creates a sequence file in TestResults/<Guid>/<Guid>_Se‐
134         quence.xml  that captures the order of tests that were run before the
135         crash.
136
137         This option does not create a memory dump and is not helpful when the
138         test is hanging.
139
140       • --blame-crash (Available since .NET 5.0 SDK)
141
142         Runs  the tests in blame mode and collects a crash dump when the test
143         host exits unexpectedly.  This option depends on the version of  .NET
144         used, the type of error, and the operating system.
145
146         For exceptions in managed code, a dump will be automatically collect‐
147         ed on .NET 5.0 and later versions.   It  will  generate  a  dump  for
148         testhost  or any child process that also ran on .NET 5.0 and crashed.
149         Crashes in native code will not generate a dump.  This  option  works
150         on Windows, macOS, and Linux.
151
152         Crash  dumps  in  native code, or when using .NET Core 3.1 or earlier
153         versions, can only be collected on Windows, by using Procdump.  A di‐
154         rectory  that contains procdump.exe and procdump64.exe must be in the
155         PATH or PROCDUMP_PATH environment variable.  Download the tools.  Im‐
156         plies --blame.
157
158         To collect a crash dump from a native application running on .NET 5.0
159         or later, the  usage  of  Procdump  can  be  forced  by  setting  the
160         VSTEST_DUMP_FORCEPROCDUMP environment variable to 1.
161
162       • --blame-crash-dump-type <DUMP_TYPE> (Available since .NET 5.0 SDK)
163
164         The  type  of  crash  dump to be collected.  Supported dump types are
165         full (default), and mini.  Implies --blame-crash.
166
167       • --blame-crash-collect-always (Available since .NET 5.0 SDK)
168
169         Collects a crash dump on expected as well as unexpected test host ex‐
170         it.
171
172       • --blame-hang (Available since .NET 5.0 SDK)
173
174         Run  the tests in blame mode and collects a hang dump when a test ex‐
175         ceeds the given timeout.
176
177       • --blame-hang-dump-type <DUMP_TYPE> (Available since .NET 5.0 SDK)
178
179         The type of crash dump to be collected.  It should be full, mini,  or
180         none.   When  none  is specified, test host is terminated on timeout,
181         but no dump is collected.  Implies --blame-hang.
182
183       • --blame-hang-timeout <TIMESPAN> (Available since .NET 5.0 SDK)
184
185         Per-test timeout, after which a hang dump is triggered and  the  test
186         host  process and all of its child processes are dumped and terminat‐
187         ed.  The timeout value is specified in one of the following formats:
188
189         • 1.5h, 1.5hour, 1.5hours
190
191         • 90m, 90min, 90minute, 90minutes
192
193         • 5400s, 5400sec, 5400second, 5400seconds
194
195         • 5400000ms, 5400000mil, 5400000millisecond, 5400000milliseconds
196
197         When no unit is used (for example, 5400000), the value is assumed  to
198         be  in  milliseconds.  When used together with data driven tests, the
199         timeout behavior depends on the test adapter used.  For xUnit, NUnit.
200         and MSTest 2.2.4+, the timeout is renewed after every test case.  For
201         MSTest before version 2.2.4, the timeout is used for all test  cases.
202         This  option is supported on Windows with netcoreapp2.1 and later, on
203         Linux with netcoreapp3.1 and later, and on macOS with net5.0 or  lat‐
204         er.  Implies --blame and --blame-hang.
205
206       • -c|--configuration <CONFIGURATION>
207
208         Defines  the  build  configuration.  The default for most projects is
209         Debug, but you can override the build configuration settings in  your
210         project.
211
212       • --collect <DATA_COLLECTOR_NAME>
213
214         Enables  data  collector for the test run.  For more information, see
215         Monitor and analyze test run (https://aka.ms/vstest-collect).
216
217         On Windows (x86, x64 and Arm64), Linux (x64) and macOS (x64), you can
218         collect  code coverage by using the --collect "Code Coverage" option.
219         For more information, see Use code coverage and Customize code cover‐
220         age analysis.
221
222         To  collect  code  coverage on any platform that is supported by .NET
223         Core, install  Coverlet  (https://github.com/coverlet-coverage/cover‐
224         let/blob/master/README.md)  and  use the --collect "XPlat Code Cover‐
225         age" option.
226
227       • -d|--diag <LOG_FILE>
228
229         Enables diagnostic mode for the test platform and  writes  diagnostic
230         messages  to the specified file and to files next to it.  The process
231         that is logging the messages determines which files are created, such
232         as   *.host_<date>.txt   for   test   host   log,  and  *.datacollec‐
233         tor_<date>.txt for data collector log.
234
235       • -e|--environment <NAME="VALUE">
236
237         Sets the value of an environment variable.  Creates the  variable  if
238         it  does  not  exist, overrides if it does exist.  Use of this option
239         will force the tests to be run in an isolated  process.   The  option
240         can be specified multiple times to provide multiple variables.
241
242       • -f|--framework <FRAMEWORK>
243
244         The  target  framework  moniker  (TFM) of the target framework to run
245         tests for.  The target  framework  must  also  be  specified  in  the
246         project file.
247
248       • --filter <EXPRESSION>
249
250         Filters  out tests in the current project using the given expression.
251         For more information, see the Filter  option  details  section.   For
252         more  information and examples on how to use selective unit test fil‐
253         tering, see Running selective unit tests.
254
255       • -?|-h|--help
256
257         Prints out a description of how to use the command.
258
259       • --interactive
260
261         Allows the command to stop and wait for user input  or  action.   For
262         example,  to  complete authentication.  Available since .NET Core 3.0
263         SDK.
264
265       • -l|--logger <LOGGER>
266
267         Specifies a logger for test results.   Unlike  MSBuild,  dotnet  test
268         doesn’t  accept  abbreviations:  instead  of  -l "console;v=d" use -l
269         "console;verbosity=detailed".  Specify the parameter  multiple  times
270         to enable multiple loggers.  For more information, see Reporting test
271         results  (https://github.com/Microsoft/vstest-docs/blob/main/docs/re‐
272         port.md),  Switches for loggers, and the examples later in this arti‐
273         cle.
274
275       • --no-build
276
277         Doesn’t build the test project before running it.  It also implicitly
278         sets the --no-restore flag.
279
280       • --nologo
281
282         Run  tests  without  displaying  the  Microsoft  TestPlatform banner.
283         Available since .NET Core 3.0 SDK.
284
285       • --no-restore
286
287         Doesn’t execute an implicit restore when running the command.
288
289       • -o|--output <OUTPUT_DIRECTORY>
290
291         Directory in which to find the binaries to run.   If  not  specified,
292         the default path is ./bin/<configuration>/<framework>/.  For projects
293         with multiple target frameworks (via the TargetFrameworks  property),
294         you  also  need  to  define --framework when you specify this option.
295         dotnet test always runs tests from the output directory.  You can use
296         <xref:System.AppDomain.BaseDirectory%2A?displayProperty=nameWithType>
297         to consume test assets in the output directory.
298
299       • --os <OS>
300
301         Specifies the target operating system (OS).  This is a shorthand syn‐
302         tax for setting the Runtime Identifier (RID), where the provided val‐
303         ue is combined with the default RID.  For example, on a  win-x64  ma‐
304         chine,  specifying  --os linux sets the RID to linux-x64.  If you use
305         this option, don’t use the -r|--runtime option.  Available since .NET
306         6.
307
308       • --results-directory <RESULTS_DIR>
309
310         The  directory where the test results are going to be placed.  If the
311         specified directory doesn’t exist,  it’s  created.   The  default  is
312         TestResults in the directory that contains the project file.
313
314         Short form -r available in .NET SDK versions earlier than 7.
315
316       • -r|--runtime <RUNTIME_IDENTIFIER>
317
318         The target runtime to test for.
319
320         Short form -r available starting in .NET SDK 7.
321
322       • -s|--settings <SETTINGS_FILE>
323
324         The  .runsettings file to use for running the tests.  The TargetPlat‐
325         form element (x86|x64) has no effect for dotnet test.  To  run  tests
326         that  target  x86, install the x86 version of .NET Core.  The bitness
327         of the dotnet.exe that is on the path is what will be used  for  run‐
328         ning tests.  For more information, see the following resources:
329
330         • Configure unit tests by using a .runsettings file.
331
332         • Configure    a   test   run   (https://github.com/Microsoft/vstest-
333           docs/blob/main/docs/configure.md)
334
335       • -t|--list-tests
336
337         List the discovered tests instead of running the tests.
338
339       • -v|--verbosity <LEVEL>
340
341         Sets the verbosity level of the command.  Allowed values are q[uiet],
342         m[inimal],  n[ormal],  d[etailed],  and diag[nostic].  The default is
343         minimal.   For  more  information,  see  <xref:Microsoft.Build.Frame‐
344         work.LoggerVerbosity>.
345
346       • args
347
348         Specifies  extra  arguments  to  pass to the adapter.  Use a space to
349         separate multiple arguments.
350
351         The list of possible arguments depends upon the specified behavior:
352
353         • When you specify a project, solution, or a  directory,  or  if  you
354           omit  this  argument,  the  call  is forwarded to msbuild.  In that
355           case, the available arguments can be found in  the  dotnet  msbuild
356           documentation.
357
358         • When  you  specify  a  .dll  or  an  .exe, the call is forwarded to
359           vstest.  In that case, the available arguments can be found in  the
360           dotnet vstest documentation.
361
362       • RunSettings arguments
363
364       Inline RunSettings are passed as the last arguments on the command line
365       after “–” (note the space after –).  Inline RunSettings  are  specified
366       as  [name]=[value]  pairs.   A  space  is  used  to  separate  multiple
367       [name]=[value] pairs.
368
369       Example: dotnet test -- MSTest.DeploymentEnabled=false MSTest.MapIncon‐
370       clusiveToFailed=True
371
372       For more information, see Passing RunSettings arguments through command
373       line   (https://github.com/Microsoft/vstest-docs/blob/main/docs/RunSet‐
374       tingsArguments.md).
375

EXAMPLES

377       • Run the tests in the project in the current directory:
378
379                dotnet test
380
381       • Run the tests in the test1 project:
382
383                dotnet test ~/projects/test1/test1.csproj
384
385       • Run the tests using test1.dll assembly:
386
387                dotnet test ~/projects/test1/bin/debug/test1.dll
388
389       • Run the tests in the project in the current directory, and generate a
390         test results file in the trx format:
391
392                dotnet test --logger trx
393
394       • Run the tests in the project in the current directory, and generate a
395         code      coverage      file      (after      installing     Coverlet
396         (https://github.com/coverlet-coverage/coverlet/blob/master/Documenta‐
397         tion/VSTestIntegration.md) collectors integration):
398
399                dotnet test --collect:"XPlat Code Coverage"
400
401       • Run the tests in the project in the current directory, and generate a
402         code coverage file (Windows only):
403
404                dotnet test --collect "Code Coverage"
405
406       • Run the tests in the project in the current directory, and  log  with
407         detailed verbosity to the console:
408
409                dotnet test --logger "console;verbosity=detailed"
410
411       • Run  the  tests in the project in the current directory, and log with
412         the trx logger to testResults.trx in the TestResults folder:
413
414                dotnet test --logger "trx;logfilename=testResults.trx"
415
416         Since the log file name is specified, the same name is used for  each
417         target framework in the case of a multi-targeted project.  The output
418         for each target framework overwrites the output for preceding  target
419         frameworks.   The  file  is  created in the TestResults folder in the
420         test project folder, because relative  paths  are  relative  to  that
421         folder.   The  following example shows how to produce a separate file
422         for each target framework.
423
424       • Run the tests in the project in the current directory, and  log  with
425         the  trx  logger  to files in the TestResults folder, with file names
426         that are unique for each target framework:
427
428                dotnet test --logger:"trx;LogFilePrefix=testResults"
429
430       • Run the tests in the project in the current directory, and  log  with
431         the html logger to testResults.html in the TestResults folder:
432
433                dotnet test --logger "html;logfilename=testResults.html"
434
435       • Run  the  tests  in  the project in the current directory, and report
436         tests that were in progress when the test host crashed:
437
438                dotnet test --blame
439
440       • Run the tests in the test1 project, providing the  -bl  (binary  log)
441         argument to msbuild:
442
443                dotnet test ~/projects/test1/test1.csproj -bl
444
445       • Run  the  tests  in the test1 project, setting the MSBuild DefineCon‐
446         stants property to DEV:
447
448                dotnet test ~/projects/test1/test1.csproj -p:DefineConstants="DEV"
449
450   Filter option details
451       --filter <EXPRESSION>
452
453       <Expression>  has  the  format   <property><operator><value>[|&<Expres‐
454       sion>].
455
456       <property>  is  an  attribute  of the Test Case.  The following are the
457       properties supported by popular unit test frameworks:
458
459       Test Framework   Supported properties
460       ──────────────────────────────────────
461       MSTest
462       xUnit
463       NUnit
464
465       The <operator> describes the relationship between the property and  the
466       value:
467
468       Operator   Function
469       ───────────────────────────
470          =       Exact match
471          !=      Not exact match
472          ~       Contains
473          !~      Not contains
474
475       <value> is a string.  All the lookups are case insensitive.
476
477       An  expression  without  an <operator> is automatically considered as a
478       contains on  FullyQualifiedName  property  (for  example,  dotnet  test
479       --filter xyz is same as dotnet test --filter FullyQualifiedName~xyz).
480
481       Expressions can be joined with conditional operators:
482
483       Operator   Function
484       ────────────────────
485       |          OR
486       &          AND
487
488       You can enclose expressions in parenthesis when using conditional oper‐
489       ators (for example, (Name~TestMethod1) | (Name~TestMethod2)).
490
491       For more information and examples on how to  use  selective  unit  test
492       filtering, see Running selective unit tests.
493