1FILECHECK(1) LLVM FILECHECK(1)
2
3
4
6 FileCheck - Flexible pattern matching file verifier
7
9 FileCheck match-filename [--check-prefix=XXX] [--strict-whitespace]
10
12 FileCheck reads two files (one from standard input, and one specified
13 on the command line) and uses one to verify the other. This behavior
14 is particularly useful for the testsuite, which wants to verify that
15 the output of some tool (e.g. llc) contains the expected information
16 (for example, a movsd from esp or whatever is interesting). This is
17 similar to using grep, but it is optimized for matching multiple dif‐
18 ferent inputs in one file in a specific order.
19
20 The match-filename file specifies the file that contains the patterns
21 to match. The file to verify is read from standard input unless the
22 --input-file option is used.
23
25 Options are parsed from the environment variable FILECHECK_OPTS and
26 from the command line.
27
28 -help Print a summary of command line options.
29
30 --check-prefix prefix
31 FileCheck searches the contents of match-filename for patterns
32 to match. By default, these patterns are prefixed with
33 "CHECK:". If you'd like to use a different prefix (e.g. because
34 the same input file is checking multiple different tool or op‐
35 tions), the --check-prefix argument allows you to specify one or
36 more prefixes to match. Multiple prefixes are useful for tests
37 which might change for different run options, but most lines re‐
38 main the same.
39
40 --check-prefixes prefix1,prefix2,...
41 An alias of --check-prefix that allows multiple prefixes to be
42 specified as a comma separated list.
43
44 --input-file filename
45 File to check (defaults to stdin).
46
47 --match-full-lines
48 By default, FileCheck allows matches of anywhere on a line. This
49 option will require all positive matches to cover an entire
50 line. Leading and trailing whitespace is ignored, unless
51 --strict-whitespace is also specified. (Note: negative matches
52 from CHECK-NOT are not affected by this option!)
53
54 Passing this option is equivalent to inserting {{^ *}} or {{^}}
55 before, and {{ *$}} or {{$}} after every positive check pattern.
56
57 --strict-whitespace
58 By default, FileCheck canonicalizes input horizontal whitespace
59 (spaces and tabs) which causes it to ignore these differences (a
60 space will match a tab). The --strict-whitespace argument dis‐
61 ables this behavior. End-of-line sequences are canonicalized to
62 UNIX-style \n in all modes.
63
64 --implicit-check-not check-pattern
65 Adds implicit negative checks for the specified patterns between
66 positive checks. The option allows writing stricter tests with‐
67 out stuffing them with CHECK-NOTs.
68
69 For example, "--implicit-check-not warning:" can be useful when
70 testing diagnostic messages from tools that don't have an option
71 similar to clang -verify. With this option FileCheck will verify
72 that input does not contain warnings not covered by any CHECK:
73 patterns.
74
75 --dump-input <mode>
76 Dump input to stderr, adding annotations representing currently
77 enabled diagnostics. Do this either 'always', on 'fail', or
78 'never'. Specify 'help' to explain the dump format and quit.
79
80 --dump-input-on-failure
81 When the check fails, dump all of the original input. This op‐
82 tion is deprecated in favor of --dump-input=fail.
83
84 --enable-var-scope
85 Enables scope for regex variables.
86
87 Variables with names that start with $ are considered global and
88 remain set throughout the file.
89
90 All other variables get undefined after each encountered
91 CHECK-LABEL.
92
93 -D<VAR=VALUE>
94 Sets a filecheck pattern variable VAR with value VALUE that can
95 be used in CHECK: lines.
96
97 -D#<NUMVAR>=<VALUE EXPRESSION>
98 Sets a filecheck numeric variable NUMVAR to the result of evalu‐
99 ating <VALUE EXPRESSION> that can be used in CHECK: lines. See
100 section FileCheck Numeric Variables and Expressions for details
101 on the format and meaning of <VALUE EXPRESSION>.
102
103 -version
104 Show the version number of this program.
105
106 -v Print good directive pattern matches. However, if -in‐
107 put-dump=fail or -input-dump=always, add those matches as input
108 annotations instead.
109
110 -vv Print information helpful in diagnosing internal FileCheck is‐
111 sues, such as discarded overlapping CHECK-DAG: matches, implicit
112 EOF pattern matches, and CHECK-NOT: patterns that do not have
113 matches. Implies -v. However, if -input-dump=fail or -in‐
114 put-dump=always, just add that information as input annotations
115 instead.
116
117 --allow-deprecated-dag-overlap
118 Enable overlapping among matches in a group of consecutive
119 CHECK-DAG: directives. This option is deprecated and is only
120 provided for convenience as old tests are migrated to the new
121 non-overlapping CHECK-DAG: implementation.
122
123 --color
124 Use colors in output (autodetected by default).
125
127 If FileCheck verifies that the file matches the expected contents, it
128 exits with 0. Otherwise, if not, or if an error occurs, it will exit
129 with a non-zero value.
130
132 FileCheck is typically used from LLVM regression tests, being invoked
133 on the RUN line of the test. A simple example of using FileCheck from
134 a RUN line looks like this:
135
136 ; RUN: llvm-as < %s | llc -march=x86-64 | FileCheck %s
137
138 This syntax says to pipe the current file ("%s") into llvm-as, pipe
139 that into llc, then pipe the output of llc into FileCheck. This means
140 that FileCheck will be verifying its standard input (the llc output)
141 against the filename argument specified (the original .ll file speci‐
142 fied by "%s"). To see how this works, let's look at the rest of the
143 .ll file (after the RUN line):
144
145 define void @sub1(i32* %p, i32 %v) {
146 entry:
147 ; CHECK: sub1:
148 ; CHECK: subl
149 %0 = tail call i32 @llvm.atomic.load.sub.i32.p0i32(i32* %p, i32 %v)
150 ret void
151 }
152
153 define void @inc4(i64* %p) {
154 entry:
155 ; CHECK: inc4:
156 ; CHECK: incq
157 %0 = tail call i64 @llvm.atomic.load.add.i64.p0i64(i64* %p, i64 1)
158 ret void
159 }
160
161 Here you can see some "CHECK:" lines specified in comments. Now you
162 can see how the file is piped into llvm-as, then llc, and the machine
163 code output is what we are verifying. FileCheck checks the machine
164 code output to verify that it matches what the "CHECK:" lines specify.
165
166 The syntax of the "CHECK:" lines is very simple: they are fixed strings
167 that must occur in order. FileCheck defaults to ignoring horizontal
168 whitespace differences (e.g. a space is allowed to match a tab) but
169 otherwise, the contents of the "CHECK:" line is required to match some
170 thing in the test file exactly.
171
172 One nice thing about FileCheck (compared to grep) is that it allows
173 merging test cases together into logical groups. For example, because
174 the test above is checking for the "sub1:" and "inc4:" labels, it will
175 not match unless there is a "subl" in between those labels. If it ex‐
176 isted somewhere else in the file, that would not count: "grep subl"
177 matches if "subl" exists anywhere in the file.
178
179 The FileCheck -check-prefix option
180 The FileCheck -check-prefix option allows multiple test configurations
181 to be driven from one .ll file. This is useful in many circumstances,
182 for example, testing different architectural variants with llc. Here's
183 a simple example:
184
185 ; RUN: llvm-as < %s | llc -mtriple=i686-apple-darwin9 -mattr=sse41 \
186 ; RUN: | FileCheck %s -check-prefix=X32
187 ; RUN: llvm-as < %s | llc -mtriple=x86_64-apple-darwin9 -mattr=sse41 \
188 ; RUN: | FileCheck %s -check-prefix=X64
189
190 define <4 x i32> @pinsrd_1(i32 %s, <4 x i32> %tmp) nounwind {
191 %tmp1 = insertelement <4 x i32>; %tmp, i32 %s, i32 1
192 ret <4 x i32> %tmp1
193 ; X32: pinsrd_1:
194 ; X32: pinsrd $1, 4(%esp), %xmm0
195
196 ; X64: pinsrd_1:
197 ; X64: pinsrd $1, %edi, %xmm0
198 }
199
200 In this case, we're testing that we get the expected code generation
201 with both 32-bit and 64-bit code generation.
202
203 The CHECK-NEXT: directive
204 Sometimes you want to match lines and would like to verify that matches
205 happen on exactly consecutive lines with no other lines in between
206 them. In this case, you can use "CHECK:" and "CHECK-NEXT:" directives
207 to specify this. If you specified a custom check prefix, just use
208 "<PREFIX>-NEXT:". For example, something like this works as you'd ex‐
209 pect:
210
211 define void @t2(<2 x double>* %r, <2 x double>* %A, double %B) {
212 %tmp3 = load <2 x double>* %A, align 16
213 %tmp7 = insertelement <2 x double> undef, double %B, i32 0
214 %tmp9 = shufflevector <2 x double> %tmp3,
215 <2 x double> %tmp7,
216 <2 x i32> < i32 0, i32 2 >
217 store <2 x double> %tmp9, <2 x double>* %r, align 16
218 ret void
219
220 ; CHECK: t2:
221 ; CHECK: movl 8(%esp), %eax
222 ; CHECK-NEXT: movapd (%eax), %xmm0
223 ; CHECK-NEXT: movhpd 12(%esp), %xmm0
224 ; CHECK-NEXT: movl 4(%esp), %eax
225 ; CHECK-NEXT: movapd %xmm0, (%eax)
226 ; CHECK-NEXT: ret
227 }
228
229 "CHECK-NEXT:" directives reject the input unless there is exactly one
230 newline between it and the previous directive. A "CHECK-NEXT:" cannot
231 be the first directive in a file.
232
233 The CHECK-SAME: directive
234 Sometimes you want to match lines and would like to verify that matches
235 happen on the same line as the previous match. In this case, you can
236 use "CHECK:" and "CHECK-SAME:" directives to specify this. If you
237 specified a custom check prefix, just use "<PREFIX>-SAME:".
238
239 "CHECK-SAME:" is particularly powerful in conjunction with "CHECK-NOT:"
240 (described below).
241
242 For example, the following works like you'd expect:
243
244 !0 = !DILocation(line: 5, scope: !1, inlinedAt: !2)
245
246 ; CHECK: !DILocation(line: 5,
247 ; CHECK-NOT: column:
248 ; CHECK-SAME: scope: ![[SCOPE:[0-9]+]]
249
250 "CHECK-SAME:" directives reject the input if there are any newlines be‐
251 tween it and the previous directive. A "CHECK-SAME:" cannot be the
252 first directive in a file.
253
254 The CHECK-EMPTY: directive
255 If you need to check that the next line has nothing on it, not even
256 whitespace, you can use the "CHECK-EMPTY:" directive.
257
258 declare void @foo()
259
260 declare void @bar()
261 ; CHECK: foo
262 ; CHECK-EMPTY:
263 ; CHECK-NEXT: bar
264
265 Just like "CHECK-NEXT:" the directive will fail if there is more than
266 one newline before it finds the next blank line, and it cannot be the
267 first directive in a file.
268
269 The CHECK-NOT: directive
270 The "CHECK-NOT:" directive is used to verify that a string doesn't oc‐
271 cur between two matches (or before the first match, or after the last
272 match). For example, to verify that a load is removed by a transforma‐
273 tion, a test like this can be used:
274
275 define i8 @coerce_offset0(i32 %V, i32* %P) {
276 store i32 %V, i32* %P
277
278 %P2 = bitcast i32* %P to i8*
279 %P3 = getelementptr i8* %P2, i32 2
280
281 %A = load i8* %P3
282 ret i8 %A
283 ; CHECK: @coerce_offset0
284 ; CHECK-NOT: load
285 ; CHECK: ret i8
286 }
287
288 The CHECK-COUNT: directive
289 If you need to match multiple lines with the same pattern over and over
290 again you can repeat a plain CHECK: as many times as needed. If that
291 looks too boring you can instead use a counted check
292 "CHECK-COUNT-<num>:", where <num> is a positive decimal number. It will
293 match the pattern exactly <num> times, no more and no less. If you
294 specified a custom check prefix, just use "<PREFIX>-COUNT-<num>:" for
295 the same effect. Here is a simple example:
296
297 Loop at depth 1
298 Loop at depth 1
299 Loop at depth 1
300 Loop at depth 1
301 Loop at depth 2
302 Loop at depth 3
303
304 ; CHECK-COUNT-6: Loop at depth {{[0-9]+}}
305 ; CHECK-NOT: Loop at depth {{[0-9]+}}
306
307 The CHECK-DAG: directive
308 If it's necessary to match strings that don't occur in a strictly se‐
309 quential order, "CHECK-DAG:" could be used to verify them between two
310 matches (or before the first match, or after the last match). For exam‐
311 ple, clang emits vtable globals in reverse order. Using CHECK-DAG:, we
312 can keep the checks in the natural order:
313
314 // RUN: %clang_cc1 %s -emit-llvm -o - | FileCheck %s
315
316 struct Foo { virtual void method(); };
317 Foo f; // emit vtable
318 // CHECK-DAG: @_ZTV3Foo =
319
320 struct Bar { virtual void method(); };
321 Bar b;
322 // CHECK-DAG: @_ZTV3Bar =
323
324 CHECK-NOT: directives could be mixed with CHECK-DAG: directives to ex‐
325 clude strings between the surrounding CHECK-DAG: directives. As a re‐
326 sult, the surrounding CHECK-DAG: directives cannot be reordered, i.e.
327 all occurrences matching CHECK-DAG: before CHECK-NOT: must not fall be‐
328 hind occurrences matching CHECK-DAG: after CHECK-NOT:. For example,
329
330 ; CHECK-DAG: BEFORE
331 ; CHECK-NOT: NOT
332 ; CHECK-DAG: AFTER
333
334 This case will reject input strings where BEFORE occurs after AFTER.
335
336 With captured variables, CHECK-DAG: is able to match valid topological
337 orderings of a DAG with edges from the definition of a variable to its
338 use. It's useful, e.g., when your test cases need to match different
339 output sequences from the instruction scheduler. For example,
340
341 ; CHECK-DAG: add [[REG1:r[0-9]+]], r1, r2
342 ; CHECK-DAG: add [[REG2:r[0-9]+]], r3, r4
343 ; CHECK: mul r5, [[REG1]], [[REG2]]
344
345 In this case, any order of that two add instructions will be allowed.
346
347 If you are defining and using variables in the same CHECK-DAG: block,
348 be aware that the definition rule can match after its use.
349
350 So, for instance, the code below will pass:
351
352 ; CHECK-DAG: vmov.32 [[REG2:d[0-9]+]][0]
353 ; CHECK-DAG: vmov.32 [[REG2]][1]
354 vmov.32 d0[1]
355 vmov.32 d0[0]
356
357 While this other code, will not:
358
359 ; CHECK-DAG: vmov.32 [[REG2:d[0-9]+]][0]
360 ; CHECK-DAG: vmov.32 [[REG2]][1]
361 vmov.32 d1[1]
362 vmov.32 d0[0]
363
364 While this can be very useful, it's also dangerous, because in the case
365 of register sequence, you must have a strong order (read before write,
366 copy before use, etc). If the definition your test is looking for
367 doesn't match (because of a bug in the compiler), it may match further
368 away from the use, and mask real bugs away.
369
370 In those cases, to enforce the order, use a non-DAG directive between
371 DAG-blocks.
372
373 A CHECK-DAG: directive skips matches that overlap the matches of any
374 preceding CHECK-DAG: directives in the same CHECK-DAG: block. Not only
375 is this non-overlapping behavior consistent with other directives, but
376 it's also necessary to handle sets of non-unique strings or patterns.
377 For example, the following directives look for unordered log entries
378 for two tasks in a parallel program, such as the OpenMP runtime:
379
380 // CHECK-DAG: [[THREAD_ID:[0-9]+]]: task_begin
381 // CHECK-DAG: [[THREAD_ID]]: task_end
382 //
383 // CHECK-DAG: [[THREAD_ID:[0-9]+]]: task_begin
384 // CHECK-DAG: [[THREAD_ID]]: task_end
385
386 The second pair of directives is guaranteed not to match the same log
387 entries as the first pair even though the patterns are identical and
388 even if the text of the log entries is identical because the thread ID
389 manages to be reused.
390
391 The CHECK-LABEL: directive
392 Sometimes in a file containing multiple tests divided into logical
393 blocks, one or more CHECK: directives may inadvertently succeed by
394 matching lines in a later block. While an error will usually eventually
395 be generated, the check flagged as causing the error may not actually
396 bear any relationship to the actual source of the problem.
397
398 In order to produce better error messages in these cases, the
399 "CHECK-LABEL:" directive can be used. It is treated identically to a
400 normal CHECK directive except that FileCheck makes an additional as‐
401 sumption that a line matched by the directive cannot also be matched by
402 any other check present in match-filename; this is intended to be used
403 for lines containing labels or other unique identifiers. Conceptually,
404 the presence of CHECK-LABEL divides the input stream into separate
405 blocks, each of which is processed independently, preventing a CHECK:
406 directive in one block matching a line in another block. If --en‐
407 able-var-scope is in effect, all local variables are cleared at the be‐
408 ginning of the block.
409
410 For example,
411
412 define %struct.C* @C_ctor_base(%struct.C* %this, i32 %x) {
413 entry:
414 ; CHECK-LABEL: C_ctor_base:
415 ; CHECK: mov [[SAVETHIS:r[0-9]+]], r0
416 ; CHECK: bl A_ctor_base
417 ; CHECK: mov r0, [[SAVETHIS]]
418 %0 = bitcast %struct.C* %this to %struct.A*
419 %call = tail call %struct.A* @A_ctor_base(%struct.A* %0)
420 %1 = bitcast %struct.C* %this to %struct.B*
421 %call2 = tail call %struct.B* @B_ctor_base(%struct.B* %1, i32 %x)
422 ret %struct.C* %this
423 }
424
425 define %struct.D* @D_ctor_base(%struct.D* %this, i32 %x) {
426 entry:
427 ; CHECK-LABEL: D_ctor_base:
428
429 The use of CHECK-LABEL: directives in this case ensures that the three
430 CHECK: directives only accept lines corresponding to the body of the
431 @C_ctor_base function, even if the patterns match lines found later in
432 the file. Furthermore, if one of these three CHECK: directives fail,
433 FileCheck will recover by continuing to the next block, allowing multi‐
434 ple test failures to be detected in a single invocation.
435
436 There is no requirement that CHECK-LABEL: directives contain strings
437 that correspond to actual syntactic labels in a source or output lan‐
438 guage: they must simply uniquely match a single line in the file being
439 verified.
440
441 CHECK-LABEL: directives cannot contain variable definitions or uses.
442
443 FileCheck Regex Matching Syntax
444 All FileCheck directives take a pattern to match. For most uses of
445 FileCheck, fixed string matching is perfectly sufficient. For some
446 things, a more flexible form of matching is desired. To support this,
447 FileCheck allows you to specify regular expressions in matching
448 strings, surrounded by double braces: {{yourregex}}. FileCheck imple‐
449 ments a POSIX regular expression matcher; it supports Extended POSIX
450 regular expressions (ERE). Because we want to use fixed string matching
451 for a majority of what we do, FileCheck has been designed to support
452 mixing and matching fixed string matching with regular expressions.
453 This allows you to write things like this:
454
455 ; CHECK: movhpd {{[0-9]+}}(%esp), {{%xmm[0-7]}}
456
457 In this case, any offset from the ESP register will be allowed, and any
458 xmm register will be allowed.
459
460 Because regular expressions are enclosed with double braces, they are
461 visually distinct, and you don't need to use escape characters within
462 the double braces like you would in C. In the rare case that you want
463 to match double braces explicitly from the input, you can use something
464 ugly like {{[{][{]}} as your pattern.
465
466 FileCheck String Substitution Blocks
467 It is often useful to match a pattern and then verify that it occurs
468 again later in the file. For codegen tests, this can be useful to al‐
469 low any register, but verify that that register is used consistently
470 later. To do this, FileCheck supports string substitution blocks that
471 allow string variables to be defined and substituted into patterns.
472 Here is a simple example:
473
474 ; CHECK: test5:
475 ; CHECK: notw [[REGISTER:%[a-z]+]]
476 ; CHECK: andw {{.*}}[[REGISTER]]
477
478 The first check line matches a regex %[a-z]+ and captures it into the
479 string variable REGISTER. The second line verifies that whatever is in
480 REGISTER occurs later in the file after an "andw". FileCheck string
481 substitution blocks are always contained in [[ ]] pairs, and string
482 variable names can be formed with the regex [a-zA-Z_][a-zA-Z0-9_]*. If
483 a colon follows the name, then it is a definition of the variable; oth‐
484 erwise, it is a substitution.
485
486 FileCheck variables can be defined multiple times, and substitutions
487 always get the latest value. Variables can also be substituted later
488 on the same line they were defined on. For example:
489
490 ; CHECK: op [[REG:r[0-9]+]], [[REG]]
491
492 Can be useful if you want the operands of op to be the same register,
493 and don't care exactly which register it is.
494
495 If --enable-var-scope is in effect, variables with names that start
496 with $ are considered to be global. All others variables are local.
497 All local variables get undefined at the beginning of each CHECK-LABEL
498 block. Global variables are not affected by CHECK-LABEL. This makes it
499 easier to ensure that individual tests are not affected by variables
500 set in preceding tests.
501
502 FileCheck Numeric Substitution Blocks
503 FileCheck also supports numeric substitution blocks that allow defining
504 numeric variables and checking for numeric values that satisfy a nu‐
505 meric expression constraint based on those variables via a numeric sub‐
506 stitution. This allows CHECK: directives to verify a numeric relation
507 between two numbers, such as the need for consecutive registers to be
508 used.
509
510 The syntax to define a numeric variable is [[#<NUMVAR>:]] where <NUM‐
511 VAR> is the name of the numeric variable to define to the matching
512 value.
513
514 For example:
515
516 ; CHECK: mov r[[#REG:]], 42
517
518 would match mov r5, 42 and set REG to the value 5.
519
520 The syntax of a numeric substitution is [[#<expr>]] where <expr> is an
521 expression. An expression is recursively defined as:
522
523 • a numeric operand, or
524
525 • an expression followed by an operator and a numeric operand.
526
527 A numeric operand is a previously defined numeric variable, or an inte‐
528 ger literal. The supported operators are + and -. Spaces are accepted
529 before, after and between any of these elements.
530
531 For example:
532
533 ; CHECK: load r[[#REG:]], [r0]
534 ; CHECK: load r[[#REG+1]], [r1]
535
536 The above example would match the text:
537
538 load r5, [r0]
539 load r6, [r1]
540
541 but would not match the text:
542
543 load r5, [r0]
544 load r7, [r1]
545
546 due to 7 being unequal to 5 + 1.
547
548 The --enable-var-scope option has the same effect on numeric variables
549 as on string variables.
550
551 Important note: In its current implementation, an expression cannot use
552 a numeric variable defined on the same line.
553
554 FileCheck Pseudo Numeric Variables
555 Sometimes there's a need to verify output that contains line numbers of
556 the match file, e.g. when testing compiler diagnostics. This intro‐
557 duces a certain fragility of the match file structure, as "CHECK:"
558 lines contain absolute line numbers in the same file, which have to be
559 updated whenever line numbers change due to text addition or deletion.
560
561 To support this case, FileCheck expressions understand the @LINE pseudo
562 numeric variable which evaluates to the line number of the CHECK pat‐
563 tern where it is found.
564
565 This way match patterns can be put near the relevant test lines and in‐
566 clude relative line number references, for example:
567
568 // CHECK: test.cpp:[[# @LINE + 4]]:6: error: expected ';' after top level declarator
569 // CHECK-NEXT: {{^int a}}
570 // CHECK-NEXT: {{^ \^}}
571 // CHECK-NEXT: {{^ ;}}
572 int a
573
574 To support legacy uses of @LINE as a special string variable, FileCheck
575 also accepts the following uses of @LINE with string substitution block
576 syntax: [[@LINE]], [[@LINE+<offset>]] and [[@LINE-<offset>]] without
577 any spaces inside the brackets and where offset is an integer.
578
579 Matching Newline Characters
580 To match newline characters in regular expressions the character class
581 [[:space:]] can be used. For example, the following pattern:
582
583 // CHECK: DW_AT_location [DW_FORM_sec_offset] ([[DLOC:0x[0-9a-f]+]]){{[[:space:]].*}}"intd"
584
585 matches output of the form (from llvm-dwarfdump):
586
587 DW_AT_location [DW_FORM_sec_offset] (0x00000233)
588 DW_AT_name [DW_FORM_strp] ( .debug_str[0x000000c9] = "intd")
589
590 letting us set the FileCheck variable DLOC to the desired value
591 0x00000233, extracted from the line immediately preceding "intd".
592
594 Maintained by the LLVM Team (https://llvm.org/).
595
597 2003-2021, LLVM Project
598
599
600
601
6029 2021-05-31 FILECHECK(1)