1FILECHECK(1) LLVM FILECHECK(1)
2
6 FileCheck - Flexible pattern matching file verifier
7
9 FileCheck match-filename [--check-prefix=XXX] [--strict-whitespace]
10
12 FileCheck reads two files (one from standard input, and one specified
13 on the command line) and uses one to verify the other. This behavior
14 is particularly useful for the testsuite, which wants to verify that
15 the output of some tool (e.g. llc) contains the expected information
16 (for example, a movsd from esp or whatever is interesting). This is
17 similar to using grep, but it is optimized for matching multiple dif‐
18 ferent inputs in one file in a specific order.
19 The match-filename file specifies the file that contains the patterns
21 to match. The file to verify is read from standard input unless the
22 --input-file option is used.
23
25 -help Print a summary of command line options.
26 --check-prefix prefix
28 FileCheck searches the contents of match-filename for patterns
29 to match. By default, these patterns are prefixed with
30 "CHECK:". If you'd like to use a different prefix (e.g. because
31 the same input file is checking multiple different tool or op‐
32 tions), the --check-prefix argument allows you to specify one or
33 more prefixes to match. Multiple prefixes are useful for tests
34 which might change for different run options, but most lines re‐
35 main the same.
36 --check-prefixes prefix1,prefix2,...
38 An alias of --check-prefix that allows multiple prefixes to be
39 specified as a comma separated list.
40 --input-file filename
42 File to check (defaults to stdin).
43 --match-full-lines
45 By default, FileCheck allows matches of anywhere on a line. This
46 option will require all positive matches to cover an entire
47 line. Leading and trailing whitespace is ignored, unless
48 --strict-whitespace is also specified. (Note: negative matches
49 from CHECK-NOT are not affected by this option!)
50 Passing this option is equivalent to inserting {{^ *}} or {{^}}
52 before, and {{ *$}} or {{$}} after every positive check pattern.
53 --strict-whitespace
55 By default, FileCheck canonicalizes input horizontal whitespace
56 (spaces and tabs) which causes it to ignore these differences (a
57 space will match a tab). The --strict-whitespace argument dis‐
58 ables this behavior. End-of-line sequences are canonicalized to
59 UNIX-style \n in all modes.
60 --implicit-check-not check-pattern
62 Adds implicit negative checks for the specified patterns between
63 positive checks. The option allows writing stricter tests with‐
64 out stuffing them with CHECK-NOTs.
65 For example, "--implicit-check-not warning:" can be useful when
67 testing diagnostic messages from tools that don't have an option
68 similar to clang -verify. With this option FileCheck will verify
69 that input does not contain warnings not covered by any CHECK:
70 patterns.
71 --dump-input-on-failure
73 When the check fails, dump all of the original input.
74 --enable-var-scope
76 Enables scope for regex variables.
77 Variables with names that start with $ are considered global and
79 remain set throughout the file.
80 All other variables get undefined after each encountered
82 CHECK-LABEL.
83 -D<VAR=VALUE>
85 Sets a filecheck variable VAR with value VALUE that can be used
86 in CHECK: lines.
87 -version
89 Show the version number of this program.
90 -v Print directive pattern matches.
92 -vv Print information helpful in diagnosing internal FileCheck is‐
94 sues, such as discarded overlapping CHECK-DAG: matches, implicit
95 EOF pattern matches, and CHECK-NOT: patterns that do not have
96 matches. Implies -v.
97 --allow-deprecated-dag-overlap
99 Enable overlapping among matches in a group of consecutive
100 CHECK-DAG: directives. This option is deprecated and is only
101 provided for convenience as old tests are migrated to the new
102 non-overlapping CHECK-DAG: implementation.
103
105 If FileCheck verifies that the file matches the expected contents, it
106 exits with 0. Otherwise, if not, or if an error occurs, it will exit
107 with a non-zero value.
108
110 FileCheck is typically used from LLVM regression tests, being invoked
111 on the RUN line of the test. A simple example of using FileCheck from
112 a RUN line looks like this:
113 ; RUN: llvm-as < %s | llc -march=x86-64 | FileCheck %s
115 This syntax says to pipe the current file ("%s") into llvm-as, pipe
117 that into llc, then pipe the output of llc into FileCheck. This means
118 that FileCheck will be verifying its standard input (the llc output)
119 against the filename argument specified (the original .ll file speci‐
120 fied by "%s"). To see how this works, let's look at the rest of the
121 .ll file (after the RUN line):
122 define void @sub1(i32* %p, i32 %v) {
124 entry:
125 ; CHECK: sub1:
126 ; CHECK: subl
127 %0 = tail call i32 @llvm.atomic.load.sub.i32.p0i32(i32* %p, i32 %v)
128 ret void
129 }
130 define void @inc4(i64* %p) {
132 entry:
133 ; CHECK: inc4:
134 ; CHECK: incq
135 %0 = tail call i64 @llvm.atomic.load.add.i64.p0i64(i64* %p, i64 1)
136 ret void
137 }
138 Here you can see some "CHECK:" lines specified in comments. Now you
140 can see how the file is piped into llvm-as, then llc, and the machine
141 code output is what we are verifying. FileCheck checks the machine
142 code output to verify that it matches what the "CHECK:" lines specify.
143 The syntax of the "CHECK:" lines is very simple: they are fixed strings
145 that must occur in order. FileCheck defaults to ignoring horizontal
146 whitespace differences (e.g. a space is allowed to match a tab) but
147 otherwise, the contents of the "CHECK:" line is required to match some
148 thing in the test file exactly.
149 One nice thing about FileCheck (compared to grep) is that it allows
151 merging test cases together into logical groups. For example, because
152 the test above is checking for the "sub1:" and "inc4:" labels, it will
153 not match unless there is a "subl" in between those labels. If it ex‐
154 isted somewhere else in the file, that would not count: "grep subl"
155 matches if "subl" exists anywhere in the file.
156 The FileCheck -check-prefix option
158 The FileCheck -check-prefix option allows multiple test configurations
159 to be driven from one .ll file. This is useful in many circumstances,
160 for example, testing different architectural variants with llc. Here's
161 a simple example:
162 ; RUN: llvm-as < %s | llc -mtriple=i686-apple-darwin9 -mattr=sse41 \
164 ; RUN: | FileCheck %s -check-prefix=X32
165 ; RUN: llvm-as < %s | llc -mtriple=x86_64-apple-darwin9 -mattr=sse41 \
166 ; RUN: | FileCheck %s -check-prefix=X64
167 define <4 x i32> @pinsrd_1(i32 %s, <4 x i32> %tmp) nounwind {
169 %tmp1 = insertelement <4 x i32>; %tmp, i32 %s, i32 1
170 ret <4 x i32> %tmp1
171 ; X32: pinsrd_1:
172 ; X32: pinsrd $1, 4(%esp), %xmm0
173 ; X64: pinsrd_1:
175 ; X64: pinsrd $1, %edi, %xmm0
176 }
177 In this case, we're testing that we get the expected code generation
179 with both 32-bit and 64-bit code generation.
180 The "CHECK-NEXT:" directive
182 Sometimes you want to match lines and would like to verify that matches
183 happen on exactly consecutive lines with no other lines in between
184 them. In this case, you can use "CHECK:" and "CHECK-NEXT:" directives
185 to specify this. If you specified a custom check prefix, just use
186 "<PREFIX>-NEXT:". For example, something like this works as you'd ex‐
187 pect:
188 define void @t2(<2 x double>* %r, <2 x double>* %A, double %B) {
190 %tmp3 = load <2 x double>* %A, align 16
191 %tmp7 = insertelement <2 x double> undef, double %B, i32 0
192 %tmp9 = shufflevector <2 x double> %tmp3,
193 <2 x double> %tmp7,
194 <2 x i32> < i32 0, i32 2 >
195 store <2 x double> %tmp9, <2 x double>* %r, align 16
196 ret void
197 ; CHECK: t2:
199 ; CHECK: movl 8(%esp), %eax
200 ; CHECK-NEXT: movapd (%eax), %xmm0
201 ; CHECK-NEXT: movhpd 12(%esp), %xmm0
202 ; CHECK-NEXT: movl 4(%esp), %eax
203 ; CHECK-NEXT: movapd %xmm0, (%eax)
204 ; CHECK-NEXT: ret
205 }
206 "CHECK-NEXT:" directives reject the input unless there is exactly one
208 newline between it and the previous directive. A "CHECK-NEXT:" cannot
209 be the first directive in a file.
210 The "CHECK-SAME:" directive
212 Sometimes you want to match lines and would like to verify that matches
213 happen on the same line as the previous match. In this case, you can
214 use "CHECK:" and "CHECK-SAME:" directives to specify this. If you
215 specified a custom check prefix, just use "<PREFIX>-SAME:".
216 "CHECK-SAME:" is particularly powerful in conjunction with "CHECK-NOT:"
218 (described below).
219 For example, the following works like you'd expect:
221 !0 = !DILocation(line: 5, scope: !1, inlinedAt: !2)
223 ; CHECK: !DILocation(line: 5,
225 ; CHECK-NOT: column:
226 ; CHECK-SAME: scope: ![[SCOPE:[0-9]+]]
227 "CHECK-SAME:" directives reject the input if there are any newlines be‐
229 tween it and the previous directive. A "CHECK-SAME:" cannot be the
230 first directive in a file.
231 The "CHECK-EMPTY:" directive
233 If you need to check that the next line has nothing on it, not even
234 whitespace, you can use the "CHECK-EMPTY:" directive.
235 foo
237 bar
239 ; CHECK: foo
240 ; CHECK-EMPTY:
241 ; CHECK-NEXT: bar
242 Just like "CHECK-NEXT:" the directive will fail if there is more than
244 one newline before it finds the next blank line, and it cannot be the
245 first directive in a file.
246 The "CHECK-NOT:" directive
248 The "CHECK-NOT:" directive is used to verify that a string doesn't oc‐
249 cur between two matches (or before the first match, or after the last
250 match). For example, to verify that a load is removed by a transforma‐
251 tion, a test like this can be used:
252 define i8 @coerce_offset0(i32 %V, i32* %P) {
254 store i32 %V, i32* %P
255 %P2 = bitcast i32* %P to i8*
257 %P3 = getelementptr i8* %P2, i32 2
258 %A = load i8* %P3
260 ret i8 %A
261 ; CHECK: @coerce_offset0
262 ; CHECK-NOT: load
263 ; CHECK: ret i8
264 }
265 The "CHECK-DAG:" directive
267 If it's necessary to match strings that don't occur in a strictly se‐
268 quential order, "CHECK-DAG:" could be used to verify them between two
269 matches (or before the first match, or after the last match). For exam‐
270 ple, clang emits vtable globals in reverse order. Using CHECK-DAG:, we
271 can keep the checks in the natural order:
272 // RUN: %clang_cc1 %s -emit-llvm -o - | FileCheck %s
274 struct Foo { virtual void method(); };
276 Foo f; // emit vtable
277 // CHECK-DAG: @_ZTV3Foo =
278 struct Bar { virtual void method(); };
280 Bar b;
281 // CHECK-DAG: @_ZTV3Bar =
282 CHECK-NOT: directives could be mixed with CHECK-DAG: directives to ex‐
284 clude strings between the surrounding CHECK-DAG: directives. As a re‐
285 sult, the surrounding CHECK-DAG: directives cannot be reordered, i.e.
286 all occurrences matching CHECK-DAG: before CHECK-NOT: must not fall be‐
287 hind occurrences matching CHECK-DAG: after CHECK-NOT:. For example,
288 ; CHECK-DAG: BEFORE
290 ; CHECK-NOT: NOT
291 ; CHECK-DAG: AFTER
292 This case will reject input strings where BEFORE occurs after AFTER.
294 With captured variables, CHECK-DAG: is able to match valid topological
296 orderings of a DAG with edges from the definition of a variable to its
297 use. It's useful, e.g., when your test cases need to match different
298 output sequences from the instruction scheduler. For example,
299 ; CHECK-DAG: add [[REG1:r[0-9]+]], r1, r2
301 ; CHECK-DAG: add [[REG2:r[0-9]+]], r3, r4
302 ; CHECK: mul r5, [[REG1]], [[REG2]]
303 In this case, any order of that two add instructions will be allowed.
305 If you are defining and using variables in the same CHECK-DAG: block,
307 be aware that the definition rule can match after its use.
308 So, for instance, the code below will pass:
310 ; CHECK-DAG: vmov.32 [[REG2:d[0-9]+]][0]
312 ; CHECK-DAG: vmov.32 [[REG2]][1]
313 vmov.32 d0[1]
314 vmov.32 d0[0]
315 While this other code, will not:
317 ; CHECK-DAG: vmov.32 [[REG2:d[0-9]+]][0]
319 ; CHECK-DAG: vmov.32 [[REG2]][1]
320 vmov.32 d1[1]
321 vmov.32 d0[0]
322 While this can be very useful, it's also dangerous, because in the case
324 of register sequence, you must have a strong order (read before write,
325 copy before use, etc). If the definition your test is looking for
326 doesn't match (because of a bug in the compiler), it may match further
327 away from the use, and mask real bugs away.
328 In those cases, to enforce the order, use a non-DAG directive between
330 DAG-blocks.
331 A CHECK-DAG: directive skips matches that overlap the matches of any
333 preceding CHECK-DAG: directives in the same CHECK-DAG: block. Not only
334 is this non-overlapping behavior consistent with other directives, but
335 it's also necessary to handle sets of non-unique strings or patterns.
336 For example, the following directives look for unordered log entries
337 for two tasks in a parallel program, such as the OpenMP runtime:
338 // CHECK-DAG: [[THREAD_ID:[0-9]+]]: task_begin
340 // CHECK-DAG: [[THREAD_ID]]: task_end
341 //
342 // CHECK-DAG: [[THREAD_ID:[0-9]+]]: task_begin
343 // CHECK-DAG: [[THREAD_ID]]: task_end
344 The second pair of directives is guaranteed not to match the same log
346 entries as the first pair even though the patterns are identical and
347 even if the text of the log entries is identical because the thread ID
348 manages to be reused.
349 The "CHECK-LABEL:" directive
351 Sometimes in a file containing multiple tests divided into logical
352 blocks, one or more CHECK: directives may inadvertently succeed by
353 matching lines in a later block. While an error will usually eventually
354 be generated, the check flagged as causing the error may not actually
355 bear any relationship to the actual source of the problem.
356 In order to produce better error messages in these cases, the
358 "CHECK-LABEL:" directive can be used. It is treated identically to a
359 normal CHECK directive except that FileCheck makes an additional as‐
360 sumption that a line matched by the directive cannot also be matched by
361 any other check present in match-filename; this is intended to be used
362 for lines containing labels or other unique identifiers. Conceptually,
363 the presence of CHECK-LABEL divides the input stream into separate
364 blocks, each of which is processed independently, preventing a CHECK:
365 directive in one block matching a line in another block. If --en‐
366 able-var-scope is in effect, all local variables are cleared at the be‐
367 ginning of the block.
368 For example,
370 define %struct.C* @C_ctor_base(%struct.C* %this, i32 %x) {
372 entry:
373 ; CHECK-LABEL: C_ctor_base:
374 ; CHECK: mov [[SAVETHIS:r[0-9]+]], r0
375 ; CHECK: bl A_ctor_base
376 ; CHECK: mov r0, [[SAVETHIS]]
377 %0 = bitcast %struct.C* %this to %struct.A*
378 %call = tail call %struct.A* @A_ctor_base(%struct.A* %0)
379 %1 = bitcast %struct.C* %this to %struct.B*
380 %call2 = tail call %struct.B* @B_ctor_base(%struct.B* %1, i32 %x)
381 ret %struct.C* %this
382 }
383 define %struct.D* @D_ctor_base(%struct.D* %this, i32 %x) {
385 entry:
386 ; CHECK-LABEL: D_ctor_base:
387 The use of CHECK-LABEL: directives in this case ensures that the three
389 CHECK: directives only accept lines corresponding to the body of the
390 @C_ctor_base function, even if the patterns match lines found later in
391 the file. Furthermore, if one of these three CHECK: directives fail,
392 FileCheck will recover by continuing to the next block, allowing multi‐
393 ple test failures to be detected in a single invocation.
394 There is no requirement that CHECK-LABEL: directives contain strings
396 that correspond to actual syntactic labels in a source or output lan‐
397 guage: they must simply uniquely match a single line in the file being
398 verified.
399 CHECK-LABEL: directives cannot contain variable definitions or uses.
401 FileCheck Pattern Matching Syntax
403 All FileCheck directives take a pattern to match. For most uses of
404 FileCheck, fixed string matching is perfectly sufficient. For some
405 things, a more flexible form of matching is desired. To support this,
406 FileCheck allows you to specify regular expressions in matching
407 strings, surrounded by double braces: {{yourregex}}. FileCheck imple‐
408 ments a POSIX regular expression matcher; it supports Extended POSIX
409 regular expressions (ERE). Because we want to use fixed string matching
410 for a majority of what we do, FileCheck has been designed to support
411 mixing and matching fixed string matching with regular expressions.
412 This allows you to write things like this:
413 ; CHECK: movhpd {{[0-9]+}}(%esp), {{%xmm[0-7]}}
415 In this case, any offset from the ESP register will be allowed, and any
417 xmm register will be allowed.
418 Because regular expressions are enclosed with double braces, they are
420 visually distinct, and you don't need to use escape characters within
421 the double braces like you would in C. In the rare case that you want
422 to match double braces explicitly from the input, you can use something
423 ugly like {{[{][{]}} as your pattern.
424 FileCheck Variables
426 It is often useful to match a pattern and then verify that it occurs
427 again later in the file. For codegen tests, this can be useful to al‐
428 low any register, but verify that that register is used consistently
429 later. To do this, FileCheck allows named variables to be defined and
430 substituted into patterns. Here is a simple example:
431 ; CHECK: test5:
433 ; CHECK: notw [[REGISTER:%[a-z]+]]
434 ; CHECK: andw {{.*}}[[REGISTER]]
435 The first check line matches a regex %[a-z]+ and captures it into the
437 variable REGISTER. The second line verifies that whatever is in REGIS‐
438 TER occurs later in the file after an "andw". FileCheck variable ref‐
439 erences are always contained in [[ ]] pairs, and their names can be
440 formed with the regex [a-zA-Z_][a-zA-Z0-9_]*. If a colon follows the
441 name, then it is a definition of the variable; otherwise, it is a use.
442 FileCheck variables can be defined multiple times, and uses always get
444 the latest value. Variables can also be used later on the same line
445 they were defined on. For example:
446 ; CHECK: op [[REG:r[0-9]+]], [[REG]]
448 Can be useful if you want the operands of op to be the same register,
450 and don't care exactly which register it is.
451 If --enable-var-scope is in effect, variables with names that start
453 with $ are considered to be global. All others variables are local.
454 All local variables get undefined at the beginning of each CHECK-LABEL
455 block. Global variables are not affected by CHECK-LABEL. This makes it
456 easier to ensure that individual tests are not affected by variables
457 set in preceding tests.
458 FileCheck Expressions
460 Sometimes there's a need to verify output which refers line numbers of
461 the match file, e.g. when testing compiler diagnostics. This intro‐
462 duces a certain fragility of the match file structure, as "CHECK:"
463 lines contain absolute line numbers in the same file, which have to be
464 updated whenever line numbers change due to text addition or deletion.
465 To support this case, FileCheck allows using [[@LINE]], [[@LINE+<off‐
467 set>]], [[@LINE-<offset>]] expressions in patterns. These expressions
468 expand to a number of the line where a pattern is located (with an op‐
469 tional integer offset).
470 This way match patterns can be put near the relevant test lines and in‐
472 clude relative line number references, for example:
473 // CHECK: test.cpp:[[@LINE+4]]:6: error: expected ';' after top level declarator
475 // CHECK-NEXT: {{^int a}}
476 // CHECK-NEXT: {{^ \^}}
477 // CHECK-NEXT: {{^ ;}}
478 int a
479 Matching Newline Characters
481 To match newline characters in regular expressions the character class
482 [[:space:]] can be used. For example, the following pattern:
483 // CHECK: DW_AT_location [DW_FORM_sec_offset] ([[DLOC:0x[0-9a-f]+]]){{[[:space:]].*}}"intd"
485 matches output of the form (from llvm-dwarfdump):
487 DW_AT_location [DW_FORM_sec_offset] (0x00000233)
489 DW_AT_name [DW_FORM_strp] ( .debug_str[0x000000c9] = "intd")
490 letting us set the FileCheck variable DLOC to the desired value
492 0x00000233, extracted from the line immediately preceding "intd".
493
495 Maintained by The LLVM Team (http://llvm.org/).
496
498 2003-2023, LLVM Project
4997 2023-07-20 FILECHECK(1)