[FileCheck] Support comment directives

[jdenny]

Summary

Sometimes you want to disable a FileCheck directive without removing
it entirely, or you want to write comments that mention a directive by
name. The COM: directive makes it easy to do this. For example,
you might have:

; X32: pinsrd_1:
; X32:    pinsrd $1, 4(%esp), %xmm0

; COM: FIXME: X64 isn't working correctly yet for this part of codegen, but 
; COM: X64 will have something similar to X32:
; COM:
; COM:   X64: pinsrd_1:
; COM:   X64:    pinsrd $1, %edi, %xmm0

Without this patch, you need to use some combination of rewording and
directive syntax mangling to prevent FileCheck from recognizing the
commented occurrences of X32: and X64: above as directives.
Moreover, FileCheck diagnostics have been proposed that might complain
about the occurrences of X64 that don't have the trailing :
because they look like directive typos:

<http://lists.llvm.org/pipermail/llvm-dev/2020-April/140610.html>

I think dodging all these problems can prove tedious for test authors,
and directive syntax mangling already makes the purpose of existing
test code unclear. COM: can avoid all these problems.

This patch also updates the small set of existing tests that define
COM as a check prefix:

• clang/test/CodeGen/default-address-space.c
• clang/test/CodeGenOpenCL/addr-space-struct-arg.cl
• clang/test/Driver/hip-device-libs.hip
• llvm/test/Assembler/drop-debug-info-nonzero-alloca.ll

I think lit should support COM: as well. Perhaps clang -verify
should too.

[jhenderson]

I hesitate to suggest this, but is it worth using REM as a comment prefix? It's the comment marker for Windows batch files, so has some precedence.

llvm/docs/CommandGuide/FileCheck.rst

57

Nit: the rest of this document uses single spaces after full stops. Please do that in the new text too.

llvm/lib/Support/FileCheck.cpp

1930

It looks to me like the changes related to this function could be done separately (perhaps first). Is that the case, and if so, could you do so? It's a little hard to follow what's just refactoring of the existing stuff and what is new, with the extra comment stuff thrown in.

llvm/test/FileCheck/comment.txt

1 ↗

(On Diff #261575)

You don't need to prefix the lines with '//' or anything else for that matter, since this isn't being used as an assembly/yaml/C/etc input.

If you do want to have the characters (I don't care either way), I'd prefer '#' for RUN/CHECK directive lines and '##' for comments (the latter so they stand out better from directive lines). It's fewer characters, whilst still as explicit ('#' is used widely in some parts of the testsuite at least).

2 ↗

(On Diff #261575)

Don't prefix empty lines.

4 ↗

(On Diff #261575)

begin -> beginning

9 ↗

(On Diff #261575)

Is the lack of space after 'RUN:' deliberate?

10–11 ↗

(On Diff #261575)

I have a personal preference for multi-line commands like this to be formatted like this:

// RUN: <command> <args> ... | \
// RUN:   <next command> <args> ...

with the | and \ on the same line, to show that the next line is the start of a new command (as opposed to just being more arguments for that line), and the extra two space indentation showing that the line is a continuation of the previous.

15 ↗

(On Diff #261575)

Not a big deal, but it might be easier for debugging the test in the future if the %t.in (and %t.chk) of each case is named differently, e.g. %t2.in, %t2.chk etc. That way, the later ones won't overwrite the earlier ones.

38–43 ↗

(On Diff #261575)

I'm struggling a little with this case. Firstly, why the '8' in the column count in the remark? Secondly, if COM: was being treated as a genuine comment here, then the check directive would become CHECK: foo which would still be a match of the input, if I'm not mistaken? I guess the two might somehow be related, but I don't know how if so.

127 ↗

(On Diff #261575)

Maybe worth cases like "-comment-prefixes=,FOO", "-comment-prefixes=FOO," and "-comment-prefixes=FOO,,BAR".

133 ↗

(On Diff #261575)

Maybe worth an additional case where the invalid character is not the first character in the string.

143–147 ↗

(On Diff #261575)

I would naively expect --check-prefixes=FOO,COM to work, and disable the comment prefix. Is that complicated to implement?

llvm/utils/FileCheck/FileCheck.cpp

50

Similar comment to earlier. Every multi-sentence diagnostic I've seen in LLVM uses single spaces, so this probably should too.

[jdenny]

Thanks for the review. I've replied to a few comments, and I'll address the rest later.

In D79276#2017101, @jhenderson wrote:

I hesitate to suggest this, but is it worth using REM as a comment prefix? It's the comment marker for Windows batch files, so has some precedence.

I don't have a strong preference. Does anyone else following this have a preference?

Some data about existing usage:

$ cd llvm.git
$ git grep '\<REM\(-[a-zA-Z0-9_-]*\)\?:' | wc -l
115

In 21 matches (1 test file), REM is a FileCheck prefix.

In 4 matches (2 test files), // REM: is actually used to prefix a comment. I'm not sure why.

The remaining 90 matches (29 test files) use REM: to define a FileCheck variable and shouldn't actually break behavior if REM is also a FileCheck comment prefix. However, to make test code less confusing, people should probably avoid using such a FileCheck comment prefix for other purposes even if it works fine.

$ git grep '\<COM\(-[a-zA-Z0-9_-]*\)\?:' | wc -l
12

In 11 matches (4 test files), COM is a FileCheck prefix. I fixed those as part of this patch.

In 1 match (1 test file), CHECK-COM is a FileCheck prefix. No conflict there.

llvm/test/FileCheck/comment.txt

1 ↗

(On Diff #261575)

I don't much care either. The FileCheck test suite is not consistent in this regard.

What about the following style, making use of COM: (or REM: or whatever we choose) for comments as that's it's purpose:

COM: -------------------------------------------------------------------------
COM: Comment directives successfully comment out check directives.
COM: -------------------------------------------------------------------------

COM: Check all default comment prefixes.
COM: Check that a comment directive at the begin/end of the file is handled.
COM: Check that the preceding/following line's check directive is not affected.
RUN: echo 'foo'                   >  %t.in
RUN: echo 'COM: CHECK: bar'       >  %t.chk
RUN: echo 'CHECK: foo'            >> %t.chk
RUN: echo 'RUN:echo "CHECK: baz"' >> %t.chk
RUN: %ProtectFileCheckOutput FileCheck -vv %t.chk < %t.in 2>&1 \
RUN: | FileCheck -check-prefix=SUPPRESSES-CHECKS -DCHECK_LINE=2 %s

COM: Check the case of one user-specified comment prefix.
COM: Check that a comment directive not at the beginning of a line is handled.
RUN: echo 'foo'                                      >  %t.in
RUN: echo 'CHECK: foo'                               >  %t.chk
RUN: echo 'letters then space MY-PREFIX: CHECK: bar' >> %t.chk
RUN: %ProtectFileCheckOutput \
RUN: FileCheck -vv %t.chk -comment-prefixes=MY-PREFIX < %t.in 2>&1 \
RUN: | FileCheck -check-prefix=SUPPRESSES-CHECKS -DCHECK_LINE=1 %s

COM: Check the case of multiple user-specified comment prefixes.
RUN: echo 'foo'               >  %t.in
RUN: echo 'Bar_2: CHECK: Bar' >  %t.chk
RUN: echo 'CHECK: foo'        >> %t.chk
RUN: echo 'Foo_1: CHECK: Foo' >> %t.chk
RUN: echo 'Baz_3: CHECK: Baz' >> %t.chk
RUN: %ProtectFileCheckOutput \
RUN: FileCheck -vv %t.chk -comment-prefixes=Foo_1,Bar_2 \
RUN:           -comment-prefixes=Baz_3 < %t.in 2>&1 \
RUN: | FileCheck -check-prefix=SUPPRESSES-CHECKS -DCHECK_LINE=2 %s

SUPPRESSES-CHECKS: .chk:[[CHECK_LINE]]:8: remark: CHECK: expected string found in input

2 ↗

(On Diff #261575)

I was doing that to keep related lines together in a block, but I can use "-----" lines to separate blocks instead, as shown in my reply above.

10–11 ↗

(On Diff #261575)

Has the community ever discussed a preferred style for the LLVM coding standards? I don't have strong feelings about what we choose, but settling on one standard would be nice.

143–147 ↗

(On Diff #261575)

It's probably not complicated to implement.

However, there can already be much confusion over check prefixes, and people keep proposing diagnostics to catch mistakes: which check prefixes are enabled, what text is intended to be directives vs. comments, undefined prefixes, unused prefixes, what prefixes are mistyped. The motivation for a comment directive is to address part of this confusion. But I don't want to extend the confusion along a new dimension by forcing people to have to also figure out what comment prefixes are in effect. Instead, I think it would be good for LLVM test suites to settle on a single comment style and stick with it as much as possible. The documentation I wrote specifically discourages using -comment-prefixes unless you're working in some test environment where the default comment prefixes just aren't appropriate.

Following this reasoning, I don't think it should be possible to quietly and perhaps accidentally override the default comment prefixes via -check-prefixes. You should have to do so explicitly with -comment-prefixes.

What do you think?

[jdenny]

In D79276#2017742, @jdenny wrote:

Thanks for the review. I've replied to a few comments, and I'll address the rest later.

In D79276#2017101, @jhenderson wrote:

I hesitate to suggest this, but is it worth using REM as a comment prefix? It's the comment marker for Windows batch files, so has some precedence.

I don't have a strong preference. Does anyone else following this have a preference?

One reason not to choose REM: is that it looks much more like RUN: than COM:. For example, imagine the long code sample I wrote in my last comment, but with COM: replaced by REM:.

[jdenny]

Addressed most reviewer comments.

[jdenny]

llvm/docs/CommandGuide/FileCheck.rst

57

It doesn't matter to me which we use, especially given that it doesn't seem to affect how the HTML renders. However, the majority of periods (ignoring .. marking blocks) in this document that are followed by at least one space are followed by two spaces. Does that change your suggestion?

llvm/lib/Support/FileCheck.cpp

1930

Good idea: D79375

llvm/test/FileCheck/comment.txt

38–43 ↗

(On Diff #261575)

Firstly, why the '8' in the column count in the remark?

Are you asking why it's not 1? Here, FileCheck gives the location of the first character of the pattern from the check file.

Secondly, if COM: was being treated as a genuine comment here, then the check directive would become CHECK: foo which would still be a match of the input, if I'm not mistaken?

Ah, thanks. Fixed.

[jdenny]

Fixed a missed rename in the test code in the last update.

[jhenderson]

llvm/docs/CommandGuide/FileCheck.rst

57

Ah, it looks like it depends on who wrote the respective section. I know @probinson prefers double-spaces, and he wrote a lot of this stuff originally. I personally prefer single spaces (why waste extra characters etc etc). As the document is already inconsistent, I don't mind. When I made my original comment, I obviously didn't look at the right bits of the docs.

llvm/lib/Support/FileCheck.cpp

1917

This can probably be "check and comment prefixes"

1924

Similar to what I mentioned in D79375, perhaps it would make more sense for COM and RUN to be defined by the tool itself rather than the library to allow users to customise their respective front-ends as they wish.

1970

This is incredibly nit-picky, but perhaps this should have a check for PrefixRegexStr.empty(). It's such an edge case that it probably doesn't matter, but assuming you adopt my comment about the default prefixes being defined by the tool, you could imagine a situation where a different front-end defined no check prefixes, and the user then didn't specify any either.

llvm/test/FileCheck/comment.txt

1 ↗

(On Diff #261575)

Honestly, I don't find COM: stands out enough from other non-comment lines, so I generally wouldn't use it unless I needed to workaround an issue, although I'm not opposed to it in principle.

In this particular test, which is all about FileCheck's testing of the comment directive, I'd suggest it might be best to avoid using it for anything other than testing purposes to avoid any risk of confusion.

2 ↗

(On Diff #261575)

Ah, I see what you are doing. Maybe it would just be better to split the related groups into separate test files. That'll make them run faster when run in isolation, and allow for easier debugging too, whilst also keeping relateed stuff clearly together.

10–11 ↗

(On Diff #261575)

I don't think there's ever been a discussion. Feel free to start one, and I'll chime in.

38–43 ↗

(On Diff #261575)

Are you asking why it's not 1? Here, FileCheck gives the location of the first character of the pattern from the check file.

Apparently I can't count. Thanks.

143–147 ↗

(On Diff #261575)

Okay, that sounds reasonable to me. I guess it should be fairly obvious what is going on.

[jdenny]

Applied most reviewer comments.

[jdenny]

llvm/lib/Support/FileCheck.cpp

1924

I suggested there that we handle the check prefix side of that change in a separate patch. Perhaps the same patch could handle the comment prefix side of it.

Do you see a reason to handle this before committing the current patches?

1970

Agreed. As you say, it won't matter until tools can define default prefixes, so it can be addressed wherever that's addressed.

llvm/test/FileCheck/comment.txt

1 ↗

(On Diff #261575)

In this particular test, which is all about FileCheck's testing of the comment directive, I'd suggest it might be best to avoid using it for anything other than testing purposes to avoid any risk of confusion.

Fair enough. We can do something different for this test. But I am interested in pursuing more discussion on the general appeal of COM:....

Honestly, I don't find COM: stands out enough from other non-comment lines, so I generally wouldn't use it unless I needed to workaround an issue, although I'm not opposed to it in principle.

By that logic, the example from this patch's summary might end up like this:

; X32: pinsrd_1:
; X32:    pinsrd $1, 4(%esp), %xmm0

; FIXME: X64 isn't working correctly yet for this part of codegen, but 
; COM: X64 will have something similar to X32:
; 
; COM:   X64: pinsrd_1:
; COM:   X64:    pinsrd $1, %edi, %xmm0

After more FileCheck diagnostics land, the FIXME line might break, and a new COM: would have to be added.

Instead, my hope was that people would start to see COM: (usually with a leading ;, //, etc.) as a general way to style entire comment blocks in test files. Once that habit is in place and eyes are trained, true comments would become very obvious, and test authors would never have to think about whether FileCheck would trip on a comment that looks like a directive.

Of course, I'm not suggesting an overhaul of all test suites once this lands. I'm just suggesting people would use it more and more to improve readability of tests, both for humans and for tools like FileCheck.

Is there anything we can do to improve the appeal of COM:? Or is it just a matter of getting used to it, as with any comment style?

10–11 ↗

(On Diff #261575)

For this patch, I've switched to the style you suggested.

I'm afraid that leading a general discussion on that issue is low on my list of priorities. Maybe some day.

[jdenny]

My last update accidentally included D79375. This strips it back out. Sorry about that.

[jhenderson]

llvm/lib/Support/FileCheck.cpp

1924

I don't, as there's no current use-case for it. Let's leave it as-is for now.

llvm/test/FileCheck/comment/bad-comment-prefix.txt

37

Perhaps worth checking against RUN as well as COM?

llvm/test/FileCheck/comment/blank-comments.txt

10

I'm assuming that FileCheck treats the entire line after the first CHECK: here as part of the CHECK, and doesn't try to start a second CHECK directive?

llvm/test/FileCheck/comment/suffixes.txt

2–3

I don't think you should change anything here, but if I'm following this right, this leads to the amusing limitation that you can "comment out" (via --comment-prefixes) any CHECK lines that use a suffix (CHECK-NEXT, CHECK-NOT etc) but not those that don't without changing --check-prefixes value!

By the way, do we need to have a test-case for that? I.e. that --comment-prefixes=CHECK-NEXT disables the CHECK-NEXT lines (assuming it does of course)?

llvm/test/FileCheck/comment/suppresses-checks.txt

2

Missing '#'

[jdenny]

Addressed remaining reviewer comments except one, which is still under discussion.

[jdenny]

llvm/test/FileCheck/comment/blank-comments.txt

10

That's right.

llvm/test/FileCheck/comment/suffixes.txt

2–3

I don't think you should change anything here, but if I'm following this right, this leads to the amusing limitation that you can "comment out" (via --comment-prefixes) any CHECK lines that use a suffix (CHECK-NEXT, CHECK-NOT etc) but not those that don't without changing --check-prefixes value!

That's right, but check prefixes have this problem too. That is, you can do things like -check-prefixes=FOO,FOO-NOT so that FOO-NOT is not negative.

ValidatePrefixes should be extended to catch such cases, I think. But in a separate patch.

By the way, do we need to have a test-case for that? I.e. that --comment-prefixes=CHECK-NEXT disables the CHECK-NEXT lines (assuming it does of course)?

Hmm. I think it's behavior we don't want to support. Maybe the test case should be added when extending ValidatePrefixes as I described above?

[jhenderson]

LGTM, but might not hurt to have someone else looking at it.

llvm/test/FileCheck/comment/bad-comment-prefix.txt

41

I'm guessing the underscore is to circumvent lit trying to run things? If so, I think we need to fix lit to only run lines where the RUN: is at the start of the line (ignoring whitespace and non alpha-numerics). I don't think anything else makes sense.

Not related to this patch of course, so nothing to do here, unless my guess is incorrect.

llvm/test/FileCheck/comment/suffixes.txt

2–3

I agree it's separate work. FWIW, I just came up with a genuinely useful use-case for it with CHECK directives, but it might just be unnecessary. Imagine the case where you want a test where some specific output is printed under one condition and not another condition. You'd want something like:

# RUN: ... | FileCheck %s --check-prefix=WITH
# RUN: ... | FileCheck %s --check-prefix=WITHOUT

# WITH: some text that should be matched
# WITHOUT-NOT: some text that should be matched

A careleses developer could change the text of "WITH" to match some new behaviour without changing "WITHOUT-NOT", thus breaking the second case. You could instead do:

# RUN: ... | FileCheck %s --check-prefix=CHECK-NOT
# RUN: ... | FileCheck %s

# CHECK-NOT: some text that should be matched

Then, if the output changed, you'd update both the regular and NOT match. I might have used this pattern in a few tests in the past had it occurred to me.

Anyway, I think there are other ways of solving that problem, although not without work on FileCheck (I've been prototyping a method with only limited success so far), and I agree it's otherwise mostly horrible, so I'm not seriously opposing the suggestion.

[jdenny]

In D79276#2024411, @jhenderson wrote:

LGTM, but might not hurt to have someone else looking at it.

I'll probably wait until Monday to give others more time to participate.

Thanks for the review!

llvm/test/FileCheck/comment/bad-comment-prefix.txt

41

I'm guessing the underscore is to circumvent lit trying to run things?

Yep.

If so, I think we need to fix lit to only run lines where the RUN: is at the start of the line (ignoring whitespace and non alpha-numerics). I don't think anything else makes sense.

I agree lit should change. That might be the right condition. People do things like the following, but maybe they shouldn't:

clang/test/Index/complete-access-checks-crash.cpp:    Derived(). // RUN: c-index-test -code-completion-at=%s:10:15 %s | FileCheck %s

At the very least, lit shouldn't recognize RUN: at the ends of other words (where - is considered a word character). Lit even recognizes XRUN:.

llvm/test/FileCheck/comment/suffixes.txt

2–3

I agree the use case is important, and I also agree there must be a better solution.

The underlying issue is that we want to reuse a pattern. Perhaps there should be some way to define a FileCheck variable once and reuse it among multiple FileCheck commands. For example:

# RUN: ... | FileCheck %s --check-prefix=WITH,ALL
# RUN: ... | FileCheck %s --check-prefix=WITHOUT,ALL

# ALL-DEF: [[VAR:some text that should be matched]]
# WITH: [[VAR]]
# WITHOUT-NOT: [[VAR]]

It should probably be possible to use regexes in such a variable. I'm not sure if that's possible now. It might require a special variable type. We currently have # to indicate numeric variables. Perhaps ~ would indicate pattern variables.

[thopre]

llvm/test/FileCheck/comment/after-words.txt

2

How about characters that cannot be part of an identifier? E.g. would "@COM:" be interpreted as a comment? Should we only accept comment prefix if it's a space or beginning of line before?

[jdenny]

llvm/test/FileCheck/comment/after-words.txt

2

Yes, this patch interprets @COM: as a comment directive.

My justification is that, to keep things simple for the FileCheck user and implementation, I tried to keep parsing of comment directives and check directives as similar as possible. One intentional difference is that comment directives are ignored if they have check suffixes, such as -NOT.

[jdenny]

llvm/test/FileCheck/comment/after-words.txt

2

git grep '@RUN' shows that @ without a following space might be a comment style we should support. Here's an example match:

llvm/test/MC/ARM/ldr-pseudo-cond-darwin.s:@RUN: llvm-mc -triple armv7-base-apple-darwin %s | FileCheck --check-prefix=CHECK-ARM --check-prefix=CHECK %s

[thopre]

LGTM. Sorry for the late review.

llvm/test/FileCheck/comment/after-words.txt

2

You're right, I forgot CHECK directives have the same problem. This should be fixed by the patch that was proposed to force a comment mark or beginning of line for a prefix to be recognized as a directive. Patch LGTM then.

[jdenny]

Thanks!

[jhenderson]

llvm/test/FileCheck/comment/suffixes.txt

2–3

That's almost exactly what I've been prototyping on-and-off over the past few months, but I've been running into various ordering issues, which I haven't yet solved to my satisfaction. My original thread that inspired the idea is http://lists.llvm.org/pipermail/llvm-dev/2020-January/138822.html.

[jdenny]

llvm/test/FileCheck/comment/suffixes.txt

2–3

Cool. I think I could benefit from this feature.

clang/test/CodeGen/default-address-space.c

-// RUN: %clang_cc1 -triple amdgcn---amdgiz -emit-llvm < %s | FileCheck -check-prefixes=CHECK,COM %s
+// RUN: %clang_cc1 -triple amdgcn---amdgiz -emit-llvm < %s | FileCheck -check-prefixes=CHECK %s
 
 // CHECK-DAG: @foo = addrspace(1) global i32 0
 int foo;
 
 // CHECK-DAG: @ban = addrspace(1) global [10 x i32] zeroinitializer
 int ban[10];
 
 // CHECK-DAG: @A = addrspace(1) global i32* null
 // CHECK-DAG: @B = addrspace(1) global i32* null
 int *A;
 int *B;
 
-// COM-LABEL: define i32 @test1()
+// CHECK-LABEL: define i32 @test1()
 // CHECK: load i32, i32* addrspacecast{{[^@]+}} @foo
 int test1() { return foo; }
 
-// COM-LABEL: define i32 @test2(i32 %i)
-// COM: %[[addr:.*]] = getelementptr
+// CHECK-LABEL: define i32 @test2(i32 %i)
+// CHECK: %[[addr:.*]] = getelementptr
 // CHECK: load i32, i32* %[[addr]]
 // CHECK-NEXT: ret i32
 int test2(int i) { return ban[i]; }
 
-// COM-LABEL: define void @test3()
+// CHECK-LABEL: define void @test3()
 // CHECK: load i32*, i32** addrspacecast{{.*}} @B
 // CHECK: load i32, i32*
 // CHECK: load i32*, i32** addrspacecast{{.*}} @A
 // CHECK: store i32 {{.*}}, i32*
 void test3() {
   *A = *B;
 }
 

clang/test/CodeGenOpenCL/addr-space-struct-arg.cl

-// RUN: %clang_cc1 %s -emit-llvm -o - -O0 -ffake-address-space-map -triple i686-pc-darwin | FileCheck -enable-var-scope -check-prefixes=COM,X86 %s
-// RUN: %clang_cc1 %s -emit-llvm -o - -O0 -triple amdgcn | FileCheck -enable-var-scope -check-prefixes=COM,AMDGCN %s
-// RUN: %clang_cc1 %s -emit-llvm -o - -cl-std=CL2.0 -O0 -triple amdgcn | FileCheck -enable-var-scope -check-prefixes=COM,AMDGCN,AMDGCN20 %s
+// RUN: %clang_cc1 %s -emit-llvm -o - -O0 -ffake-address-space-map -triple i686-pc-darwin | FileCheck -enable-var-scope -check-prefixes=ALL,X86 %s
+// RUN: %clang_cc1 %s -emit-llvm -o - -O0 -triple amdgcn | FileCheck -enable-var-scope -check-prefixes=ALL,AMDGCN %s
+// RUN: %clang_cc1 %s -emit-llvm -o - -cl-std=CL2.0 -O0 -triple amdgcn | FileCheck -enable-var-scope -check-prefixes=ALL,AMDGCN,AMDGCN20 %s
 // RUN: %clang_cc1 %s -emit-llvm -o - -cl-std=CL1.2 -O0 -triple spir-unknown-unknown-unknown | FileCheck -enable-var-scope -check-prefixes=SPIR %s
 
 typedef int int2 __attribute__((ext_vector_type(2)));
 
 typedef struct {
   int cells[9];
 } Mat3X3;
 

 
 // X86-LABEL: define void @foo(%struct.Mat4X4* noalias sret align 4 %agg.result, %struct.Mat3X3* byval(%struct.Mat3X3) align 4 %in)
 // AMDGCN-LABEL: define %struct.Mat4X4 @foo([9 x i32] %in.coerce)
 Mat4X4 __attribute__((noinline)) foo(Mat3X3 in) {
   Mat4X4 out;
   return out;
 }
 
-// COM-LABEL: define {{.*}} void @ker
+// ALL-LABEL: define {{.*}} void @ker
 // Expect two mem copies: one for the argument "in", and one for
 // the return value.
 // X86: call void @llvm.memcpy.p0i8.p1i8.i32(i8*
 // X86: call void @llvm.memcpy.p1i8.p0i8.i32(i8 addrspace(1)*
 
 // AMDGCN: load [9 x i32], [9 x i32] addrspace(1)*
 // AMDGCN: call %struct.Mat4X4 @foo([9 x i32]
 // AMDGCN: call void @llvm.memcpy.p1i8.p5i8.i64(i8 addrspace(1)*
 kernel void ker(global Mat3X3 *in, global Mat4X4 *out) {
   out[0] = foo(in[1]);
 }
 
 // X86-LABEL: define void @foo_large(%struct.Mat64X64* noalias sret align 4 %agg.result, %struct.Mat32X32* byval(%struct.Mat32X32) align 4 %in)
 // AMDGCN-LABEL: define void @foo_large(%struct.Mat64X64 addrspace(5)* noalias sret align 4 %agg.result, %struct.Mat32X32 addrspace(5)* byval(%struct.Mat32X32) align 4 %in)
 Mat64X64 __attribute__((noinline)) foo_large(Mat32X32 in) {
   Mat64X64 out;
   return out;
 }
 
-// COM-LABEL: define {{.*}} void @ker_large
+// ALL-LABEL: define {{.*}} void @ker_large
 // Expect two mem copies: one for the argument "in", and one for
 // the return value.
 // X86: call void @llvm.memcpy.p0i8.p1i8.i32(i8*
 // X86: call void @llvm.memcpy.p1i8.p0i8.i32(i8 addrspace(1)*
 // AMDGCN: call void @llvm.memcpy.p5i8.p1i8.i64(i8 addrspace(5)*
 // AMDGCN: call void @llvm.memcpy.p1i8.p5i8.i64(i8 addrspace(1)*
 kernel void ker_large(global Mat32X32 *in, global Mat64X64 *out) {
   out[0] = foo_large(in[1]);

clang/test/Driver/hip-device-libs.hip

 // REQUIRES: clang-driver
 // REQUIRES: x86-registered-target
 // REQUIRES: amdgpu-registered-target
 
 // Test if oclc_daz_opt_on or if oclc_daz_opt_off is linked depending on
 // expected denormal mode.
 
 // Test subtarget with flushing on by default.
 // RUN: %clang -### -target x86_64-linux-gnu \
 // RUN:   --cuda-gpu-arch=gfx803 \
 // RUN:   --hip-device-lib-path=%S/Inputs/hip_dev_lib   \
 // RUN:   %S/Inputs/hip_multiple_inputs/b.hip \
-// RUN: 2>&1 | FileCheck %s --check-prefixes=COM,FLUSHD
+// RUN: 2>&1 | FileCheck %s --check-prefixes=ALL,FLUSHD
 
 
 // Test subtarget with flushing off by ddefault.
 // RUN: %clang -### -target x86_64-linux-gnu \
 // RUN:   --cuda-gpu-arch=gfx900 \
 // RUN:   --hip-device-lib-path=%S/Inputs/hip_dev_lib \
 // RUN:   %S/Inputs/hip_multiple_inputs/b.hip \
-// RUN: 2>&1 | FileCheck %s --check-prefixes=COM,NOFLUSHD
+// RUN: 2>&1 | FileCheck %s --check-prefixes=ALL,NOFLUSHD
 
 
 // Test explicit flag, opposite of target default.
 // RUN: %clang -### -target x86_64-linux-gnu \
 // RUN:   --cuda-gpu-arch=gfx900 \
 // RUN:   -fcuda-flush-denormals-to-zero \
 // RUN:   --hip-device-lib-path=%S/Inputs/hip_dev_lib \
 // RUN:   %S/Inputs/hip_multiple_inputs/b.hip \
-// RUN: 2>&1 | FileCheck %s --check-prefixes=COM,FLUSHD
+// RUN: 2>&1 | FileCheck %s --check-prefixes=ALL,FLUSHD
 
 
 // Test explicit flag, opposite of target default.
 // RUN: %clang -### -target x86_64-linux-gnu \
 // RUN:   --cuda-gpu-arch=gfx803 \
 // RUN:   -fno-cuda-flush-denormals-to-zero \
 // RUN:   --hip-device-lib-path=%S/Inputs/hip_dev_lib \
 // RUN:   %S/Inputs/hip_multiple_inputs/b.hip \
-// RUN: 2>&1 | FileCheck %s --check-prefixes=COM,NOFLUSHD
+// RUN: 2>&1 | FileCheck %s --check-prefixes=ALL,NOFLUSHD
 
 
 // Test explicit flag, same as target default.
 // RUN: %clang -### -target x86_64-linux-gnu \
 // RUN:   --cuda-gpu-arch=gfx900 \
 // RUN:   -fno-cuda-flush-denormals-to-zero \
 // RUN:   --hip-device-lib-path=%S/Inputs/hip_dev_lib \
 // RUN:   %S/Inputs/hip_multiple_inputs/b.hip \
-// RUN: 2>&1 | FileCheck %s --check-prefixes=COM,NOFLUSHD
+// RUN: 2>&1 | FileCheck %s --check-prefixes=ALL,NOFLUSHD
 
 
 // Test explicit flag, same as target default.
 // RUN: %clang -### -target x86_64-linux-gnu \
 // RUN:   --cuda-gpu-arch=gfx803 \
 // RUN:   -fcuda-flush-denormals-to-zero \
 // RUN:   --hip-device-lib-path=%S/Inputs/hip_dev_lib \
 // RUN:   %S/Inputs/hip_multiple_inputs/b.hip \
-// RUN: 2>&1 | FileCheck %s --check-prefixes=COM,FLUSHD
+// RUN: 2>&1 | FileCheck %s --check-prefixes=ALL,FLUSHD
 
 
 // Test last flag wins, not flushing
 // RUN: %clang -### -target x86_64-linux-gnu \
 // RUN:   --cuda-gpu-arch=gfx803 \
 // RUN:   -fcuda-flush-denormals-to-zero -fno-cuda-flush-denormals-to-zero \
 // RUN:   --hip-device-lib-path=%S/Inputs/hip_dev_lib \
 // RUN:   %S/Inputs/hip_multiple_inputs/b.hip \
-// RUN: 2>&1 | FileCheck %s --check-prefixes=COM,NOFLUSHD
+// RUN: 2>&1 | FileCheck %s --check-prefixes=ALL,NOFLUSHD
 
 
 // RUN: %clang -### -target x86_64-linux-gnu \
 // RUN:   --cuda-gpu-arch=gfx900 \
 // RUN:   -fcuda-flush-denormals-to-zero -fno-cuda-flush-denormals-to-zero \
 // RUN:   --hip-device-lib-path=%S/Inputs/hip_dev_lib   \
 // RUN:   %S/Inputs/hip_multiple_inputs/b.hip \
-// RUN: 2>&1 | FileCheck %s --check-prefixes=COM,NOFLUSHD
+// RUN: 2>&1 | FileCheck %s --check-prefixes=ALL,NOFLUSHD
 
 
 // RUN: %clang -### -target x86_64-linux-gnu \
 // RUN:   --cuda-gpu-arch=gfx900 \
 // RUN:   -fno-cuda-flush-denormals-to-zero -fcuda-flush-denormals-to-zero \
 // RUN:   --hip-device-lib-path=%S/Inputs/hip_dev_lib   \
 // RUN:   %S/Inputs/hip_multiple_inputs/b.hip \
-// RUN: 2>&1 | FileCheck %s --check-prefixes=COM,FLUSHD
+// RUN: 2>&1 | FileCheck %s --check-prefixes=ALL,FLUSHD
 
 
 // RUN: %clang -### -target x86_64-linux-gnu \
 // RUN:   --cuda-gpu-arch=gfx803 \
 // RUN:   -fno-cuda-flush-denormals-to-zero -fcuda-flush-denormals-to-zero \
 // RUN:   --hip-device-lib-path=%S/Inputs/hip_dev_lib \
 // RUN:   %S/Inputs/hip_multiple_inputs/b.hip \
-// RUN: 2>&1 | FileCheck %s --check-prefixes=COM,FLUSHD
+// RUN: 2>&1 | FileCheck %s --check-prefixes=ALL,FLUSHD
 
 
 // Test environment variable HIP_DEVICE_LIB_PATH
 
 // RUN: env HIP_DEVICE_LIB_PATH=%S/Inputs/hip_dev_lib \
 // RUN:   %clang -### -target x86_64-linux-gnu \
 // RUN:   --cuda-gpu-arch=gfx900 \
 // RUN:   %S/Inputs/hip_multiple_inputs/b.hip \
-// RUN: 2>&1 | FileCheck %s --check-prefixes=COM
+// RUN: 2>&1 | FileCheck %s --check-prefixes=ALL
 
-// COM: {{"[^"]*clang[^"]*"}}
-// COM-SAME: "-mlink-builtin-bitcode" "{{.*}}hip.amdgcn.bc"
-// COM-SAME: "-mlink-builtin-bitcode" "{{.*}}ocml.amdgcn.bc"
-// COM-SAME: "-mlink-builtin-bitcode" "{{.*}}ockl.amdgcn.bc"
+// ALL: {{"[^"]*clang[^"]*"}}
+// ALL-SAME: "-mlink-builtin-bitcode" "{{.*}}hip.amdgcn.bc"
+// ALL-SAME: "-mlink-builtin-bitcode" "{{.*}}ocml.amdgcn.bc"
+// ALL-SAME: "-mlink-builtin-bitcode" "{{.*}}ockl.amdgcn.bc"
 // FLUSHD-SAME: "-mlink-builtin-bitcode" "{{.*}}oclc_daz_opt_on.amdgcn.bc"
 // NOFLUSHD-SAME: "-mlink-builtin-bitcode" "{{.*}}oclc_daz_opt_off.amdgcn.bc"

llvm/docs/CommandGuide/FileCheck.rst

  Print a summary of command line options.
 
 .. option:: --check-prefix prefix
 
  FileCheck searches the contents of ``match-filename`` for patterns to
  match.  By default, these patterns are prefixed with "``CHECK:``".
  If you'd like to use a different prefix (e.g. because the same input
  file is checking multiple different tool or options), the
- :option:`--check-prefix` argument allows you to specify one or more
- prefixes to match. Multiple prefixes are useful for tests which might
- change for different run options, but most lines remain the same.
+ :option:`--check-prefix` argument allows you to specify (without the trailing
+ "``:``") one or more prefixes to match. Multiple prefixes are useful for tests
+ which might change for different run options, but most lines remain the same.
+
+ FileCheck does not permit duplicate prefixes, even if one is a check prefix
+ and one is a comment prefix (see :option:`--comment-prefixes` below).
 
 .. option:: --check-prefixes prefix1,prefix2,...
 
  An alias of :option:`--check-prefix` that allows multiple prefixes to be
  specified as a comma separated list.
 
+.. option:: --comment-prefixes prefix1,prefix2,...
+
+ By default, FileCheck ignores any occurrence in ``match-filename`` of any check
+ prefix if it is preceded on the same line by "``COM:``" or "``RUN:``". See the
+ section `The "COM:" directive`_ for usage details.
+
+ These default comment prefixes can be overridden by
+ :option:`--comment-prefixes` if they are not appropriate for your testing
+ environment. However, doing so is not recommended in LLVM's LIT-based test
+ suites, which should be easier to maintain if they all follow a consistent
+ comment style. In that case, consider proposing a change to the default
+ comment prefixes instead.
+
 .. option:: --input-file filename
 
   File to check (defaults to stdin).
 
 .. option:: --match-full-lines
 
  By default, FileCheck allows matches of anywhere on a line. This
  option will require all positive matches to cover an entire
.. code-block:: llvm
 
    ; X64: pinsrd_1:
    ; X64:    pinsrd $1, %edi, %xmm0
    }
 
 In this case, we're testing that we get the expected code generation with
 both 32-bit and 64-bit code generation.
 
+The "COM:" directive
+~~~~~~~~~~~~~~~~~~~~
+
+Sometimes you want to disable a FileCheck directive without removing it
+entirely, or you want to write comments that mention a directive by name. The
+"``COM:``" directive makes it easy to do this. For example, you might have:
+
+.. code-block:: llvm
+
+   ; X32: pinsrd_1:
+   ; X32:    pinsrd $1, 4(%esp), %xmm0
+
+   ; COM: FIXME: X64 isn't working correctly yet for this part of codegen, but
+   ; COM: X64 will have something similar to X32:
+   ; COM:
+   ; COM:   X64: pinsrd_1:
+   ; COM:   X64:    pinsrd $1, %edi, %xmm0
+
+Without "``COM:``", you would need to use some combination of rewording and
+directive syntax mangling to prevent FileCheck from recognizing the commented
+occurrences of "``X32:``" and "``X64:``" above as directives. Moreover,
+FileCheck diagnostics have been proposed that might complain about the above
+occurrences of "``X64``" that don't have the trailing "``:``" because they look
+like directive typos. Dodging all these problems can be tedious for a test
+author, and directive syntax mangling can make the purpose of test code unclear.
+"``COM:``" avoids all these problems.
+
+A few important usage notes:
+
+* "``COM:``" within another directive's pattern does *not* comment out the
+  remainder of the pattern. For example:
+
+  .. code-block:: llvm
+
+     ; X32: pinsrd $1, 4(%esp), %xmm0 COM: This is part of the X32 pattern!
+
+  If you need to temporarily comment out part of a directive's pattern, move it
+  to another line. The reason is that FileCheck parses "``COM:``" in the same
+  manner as any other directive: only the first directive on the line is
+  recognized as a directive.
+
+* For the sake of LIT, FileCheck treats "``RUN:``" just like "``COM:``". If this
+  is not suitable for your test environment, see :option:`--comment-prefixes`.
+
+* FileCheck does not recognize "``COM``", "``RUN``", or any user-defined comment
+  prefix as a comment directive if it's combined with one of the usual check
+  directive suffixes, such as "``-NEXT:``" or "``-NOT:``", discussed below.
+  FileCheck treats such a combination as plain text instead. If it needs to act
+  as a comment directive for your test environment, define it as such with
+  :option:`--comment-prefixes`.
+
 The "CHECK-NEXT:" directive
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 Sometimes you want to match lines and would like to verify that matches
 happen on exactly consecutive lines with no other lines in between them.  In
 this case, you can use "``CHECK:``" and "``CHECK-NEXT:``" directives to specify
 this.  If you specified a custom check prefix, just use "``<PREFIX>-NEXT:``".
 For example, something like this works as you'd expect:

llvm/include/llvm/Support/FileCheck.h

 #include <string>
 #include <vector>
 
 namespace llvm {
 
 /// Contains info about various FileCheck options.
 struct FileCheckRequest {
   std::vector<StringRef> CheckPrefixes;
+  std::vector<StringRef> CommentPrefixes;
   bool NoCanonicalizeWhiteSpace = false;
   std::vector<StringRef> ImplicitCheckNot;
   std::vector<StringRef> GlobalDefines;
   bool AllowEmptyInput = false;
   bool MatchFullLines = false;
   bool IgnoreCase = false;
   bool IsDefaultCheckPrefix = false;
   bool EnableVarScope = false;
enum FileCheckKind {
   CheckNone = 0,
   CheckPlain,
   CheckNext,
   CheckSame,
   CheckNot,
   CheckDAG,
   CheckLabel,
   CheckEmpty,
+  CheckComment,
 
   /// Indicates the pattern only matches the end of file. This is used for
   /// trailing CHECK-NOTs.
   CheckEOF,
 
   /// Marks when parsing found a -NOT check combined with another CHECK suffix.
   CheckBadNot,
 

llvm/lib/Support/FileCheck.cpp

std::string Check::FileCheckType::getDescription(StringRef Prefix) const {
   case Check::CheckNot:
     return Prefix.str() + "-NOT";
   case Check::CheckDAG:
     return Prefix.str() + "-DAG";
   case Check::CheckLabel:
     return Prefix.str() + "-LABEL";
   case Check::CheckEmpty:
     return Prefix.str() + "-EMPTY";
+  case Check::CheckComment:
+    return std::string(Prefix);
   case Check::CheckEOF:
     return "implicit EOF";
   case Check::CheckBadNot:
     return "bad NOT";
   case Check::CheckBadCount:
     return "bad COUNT";
   }
   llvm_unreachable("unknown FileCheckType");
 }
 
 static std::pair<Check::FileCheckType, StringRef>
-FindCheckType(StringRef Buffer, StringRef Prefix) {
+FindCheckType(const FileCheckRequest &Req, StringRef Buffer, StringRef Prefix) {
   if (Buffer.size() <= Prefix.size())
     return {Check::CheckNone, StringRef()};
 
   char NextChar = Buffer[Prefix.size()];
 
   StringRef Rest = Buffer.drop_front(Prefix.size() + 1);
+
+  // Check for comment.
+  if (Req.CommentPrefixes.end() != std::find(Req.CommentPrefixes.begin(),
+                                             Req.CommentPrefixes.end(),
+                                             Prefix)) {
+    if (NextChar == ':')
+      return {Check::CheckComment, Rest};
+    // Ignore a comment prefix if it has a suffix like "-NOT".
+    return {Check::CheckNone, StringRef()};
+  }
+
   // Verify that the : is present after the prefix.
   if (NextChar == ':')
     return {Check::CheckPlain, Rest};
 
   if (NextChar != '-')
     return {Check::CheckNone, StringRef()};
 
   if (Rest.consume_front("COUNT-")) {

 /// If this routine returns a valid prefix, it will also shrink \p Buffer to
 /// start at the beginning of the returned prefix, increment \p LineNumber for
 /// each new line consumed from \p Buffer, and set \p CheckTy to the type of
 /// check found by examining the suffix.
 ///
 /// If no valid prefix is found, the state of Buffer, LineNumber, and CheckTy
 /// is unspecified.
 static std::pair<StringRef, StringRef>
-FindFirstMatchingPrefix(Regex &PrefixRE, StringRef &Buffer,
-                        unsigned &LineNumber, Check::FileCheckType &CheckTy) {
+FindFirstMatchingPrefix(const FileCheckRequest &Req, Regex &PrefixRE,
+                        StringRef &Buffer, unsigned &LineNumber,
+                        Check::FileCheckType &CheckTy) {
   SmallVector<StringRef, 2> Matches;
 
   while (!Buffer.empty()) {
     // Find the first (longest) match using the RE.
     if (!PrefixRE.match(Buffer, &Matches))
       // No match at all, bail.
       return {StringRef(), StringRef()};
 
while (!Buffer.empty()) {
     // Check that the matched prefix isn't a suffix of some other check-like
     // word.
     // FIXME: This is a very ad-hoc check. it would be better handled in some
     // other way. Among other things it seems hard to distinguish between
     // intentional and unintentional uses of this feature.
     if (Skipped.empty() || !IsPartOfWord(Skipped.back())) {
       // Now extract the type.
       StringRef AfterSuffix;
-      std::tie(CheckTy, AfterSuffix) = FindCheckType(Buffer, Prefix);
+      std::tie(CheckTy, AfterSuffix) = FindCheckType(Req, Buffer, Prefix);
 
       // If we've found a valid check type for this prefix, we're done.
       if (CheckTy != Check::CheckNone)
         return {Prefix, AfterSuffix};
     }
 
     // If we didn't successfully find a prefix, we need to skip this invalid
     // prefix and continue scanning. We directly skip the prefix that was
bool FileCheck::readCheckFile(
   }
 
   std::vector<Pattern> DagNotMatches = ImplicitNegativeChecks;
 
   // LineNumber keeps track of the line on which CheckPrefix instances are
   // found.
   unsigned LineNumber = 1;
 
-  bool FoundUsedPrefix = false;
+  bool FoundUsedCheckPrefix = false;
   while (1) {
     Check::FileCheckType CheckTy;
 
     // See if a prefix occurs in the memory buffer.
     StringRef UsedPrefix;
     StringRef AfterSuffix;
     std::tie(UsedPrefix, AfterSuffix) =
-        FindFirstMatchingPrefix(PrefixRE, Buffer, LineNumber, CheckTy);
+        FindFirstMatchingPrefix(Req, PrefixRE, Buffer, LineNumber, CheckTy);
     if (UsedPrefix.empty())
       break;
-    FoundUsedPrefix = true;
+    if (CheckTy != Check::CheckComment)
+      FoundUsedCheckPrefix = true;
 
     assert(UsedPrefix.data() == Buffer.data() &&
            "Failed to move Buffer's start forward, or pointed prefix outside "
            "of the buffer!");
     assert(AfterSuffix.data() >= Buffer.data() &&
            AfterSuffix.data() < Buffer.data() + Buffer.size() &&
            "Parsing after suffix doesn't start inside of buffer!");
 
if (!(Req.NoCanonicalizeWhiteSpace && Req.MatchFullLines))
       Buffer = Buffer.substr(Buffer.find_first_not_of(" \t"));
 
     // Scan ahead to the end of line.
     size_t EOL = Buffer.find_first_of("\n\r");
 
     // Remember the location of the start of the pattern, for diagnostics.
     SMLoc PatternLoc = SMLoc::getFromPointer(Buffer.data());
 
+    // Extract the pattern from the buffer.
+    StringRef PatternBuffer = Buffer.substr(0, EOL);
+    Buffer = Buffer.substr(EOL);
+
+    // If this is a comment, we're done.
+    if (CheckTy == Check::CheckComment)
+      continue;
+
     // Parse the pattern.
     Pattern P(CheckTy, PatternContext.get(), LineNumber);
-    if (P.parsePattern(Buffer.substr(0, EOL), UsedPrefix, SM, Req))
+    if (P.parsePattern(PatternBuffer, UsedPrefix, SM, Req))
       return true;
 
     // Verify that CHECK-LABEL lines do not define or use variables
     if ((CheckTy == Check::CheckLabel) && P.hasVariable()) {
       SM.PrintMessage(
           SMLoc::getFromPointer(UsedPrefixStart), SourceMgr::DK_Error,
           "found '" + UsedPrefix + "-LABEL:'"
                                    " with variable definition or use");
       return true;
     }
 
-    Buffer = Buffer.substr(EOL);
-
     // Verify that CHECK-NEXT/SAME/EMPTY lines have at least one CHECK line before them.
     if ((CheckTy == Check::CheckNext || CheckTy == Check::CheckSame ||
          CheckTy == Check::CheckEmpty) &&
         CheckStrings->empty()) {
       StringRef Type = CheckTy == Check::CheckNext
                            ? "NEXT"
                            : CheckTy == Check::CheckEmpty ? "EMPTY" : "SAME";
       SM.PrintMessage(SMLoc::getFromPointer(UsedPrefixStart),
while (1) {
     // Okay, add the string we captured to the output vector and move on.
     CheckStrings->emplace_back(P, UsedPrefix, PatternLoc);
     std::swap(DagNotMatches, CheckStrings->back().DagNotStrings);
     DagNotMatches = ImplicitNegativeChecks;
   }
 
   // When there are no used prefixes we report an error except in the case that
   // no prefix is specified explicitly but -implicit-check-not is specified.
-  if (!FoundUsedPrefix &&
+  if (!FoundUsedCheckPrefix &&
       (ImplicitNegativeChecks.empty() || !Req.IsDefaultCheckPrefix)) {
     errs() << "error: no check strings found with prefix"
            << (Req.CheckPrefixes.size() > 1 ? "es " : " ");
     for (size_t I = 0, E = Req.CheckPrefixes.size(); I != E; ++I) {
       if (I != 0)
         errs() << ", ";
       errs() << "\'" << Req.CheckPrefixes[I] << ":'";
     }

 }
 
 // A check prefix must contain only alphanumeric, hyphens and underscores.
 static bool ValidateCheckPrefix(StringRef CheckPrefix) {
   static const Regex Validator("^[a-zA-Z0-9_-]*$");
   return Validator.match(CheckPrefix);
 }
 
-bool FileCheck::ValidateCheckPrefixes() {
-  StringSet<> PrefixSet;
-
-  for (StringRef Prefix : Req.CheckPrefixes) {
-    // Reject empty prefixes.
-    if (Prefix.empty())
+static bool ValidatePrefixes(StringRef Kind, StringSet<> &UniquePrefixes,
+                             const ArrayRef<StringRef> &SuppliedPrefixes) {
+  for (StringRef Prefix : SuppliedPrefixes) {
+    if (Prefix.empty()) {
+      errs() << "error: supplied " << Kind << " prefix must not be the empty "
+             << "string\n";
       return false;
-
-    if (!PrefixSet.insert(Prefix).second)
+    }
+    if (!ValidateCheckPrefix(Prefix)) {
+      errs() << "error: supplied " << Kind << " prefix must start with a "
+             << "letter and contain only alphanumeric characters, hyphens, and "
+             << "underscores: '" << Prefix << "'\n";
       return false;
-
-    if (!ValidateCheckPrefix(Prefix))
+    }
+    if (!UniquePrefixes.insert(Prefix).second) {
+      errs() << "error: supplied " << Kind << " prefix must be unique among "
+             << "check and comment prefixes: '" << Prefix << "'\n";
       return false;
     }
+  }
+  return true;
+}
 
+static const char *DefaultCheckPrefixes[] = {"CHECK"};
+static const char *DefaultCommentPrefixes[] = {"COM", "RUN"};
+
+bool FileCheck::ValidateCheckPrefixes() {
+  StringSet<> UniquePrefixes;
+  // Add default prefixes to catch user-supplied duplicates of them below.
+  if (Req.CheckPrefixes.empty()) {
+    for (const char *Prefix : DefaultCheckPrefixes)
+      UniquePrefixes.insert(Prefix);
+  }
+  if (Req.CommentPrefixes.empty()) {
+    for (const char *Prefix : DefaultCommentPrefixes)
+      UniquePrefixes.insert(Prefix);
+  }
+  // Do not validate the default prefixes, or diagnostics about duplicates might
+  // incorrectly indicate that they were supplied by the user.
+  if (!ValidatePrefixes("check", UniquePrefixes, Req.CheckPrefixes))
+    return false;
+  if (!ValidatePrefixes("comment", UniquePrefixes, Req.CommentPrefixes))
+    return false;
   return true;
 }
 
 Regex FileCheck::buildCheckPrefixRegex() {
-  // I don't think there's a way to specify an initial value for cl::list,
-  // so if nothing was specified, add the default
   if (Req.CheckPrefixes.empty()) {
-    Req.CheckPrefixes.push_back("CHECK");
+    for (const char *Prefix : DefaultCheckPrefixes)
+      Req.CheckPrefixes.push_back(Prefix);
     Req.IsDefaultCheckPrefix = true;
   }
+  if (Req.CommentPrefixes.empty()) {
+    for (const char *Prefix : DefaultCommentPrefixes)
+      Req.CommentPrefixes.push_back(Prefix);
+  }
 
-  // We already validated the contents of CheckPrefixes so just concatenate
-  // them as alternatives.
+  // We already validated the contents of CheckPrefixes and CommentPrefixes so
+  // just concatenate them as alternatives.
   SmallString<32> PrefixRegexStr;
   for (size_t I = 0, E = Req.CheckPrefixes.size(); I != E; ++I) {
     if (I != 0)
       PrefixRegexStr.push_back('|');
     PrefixRegexStr.append(Req.CheckPrefixes[I]);
   }
+  for (StringRef Prefix : Req.CommentPrefixes) {
+    PrefixRegexStr.push_back('|');
+    PrefixRegexStr.append(Prefix);
+  }
 
   return Regex(PrefixRegexStr);
 }
 
 Error FileCheckPatternContext::defineCmdlineVariables(
     ArrayRef<StringRef> CmdlineDefines, SourceMgr &SM) {
   assert(GlobalVariableTable.empty() && GlobalNumericVariableTable.empty() &&
          "Overriding defined variable with command-line variable definitions");

llvm/test/Assembler/drop-debug-info-nonzero-alloca.ll

-; RUN: llvm-as < %s -o %t.bc -data-layout=A5 2>&1 | FileCheck -check-prefixes=COM,AS %s
-; RUN: llvm-dis < %t.bc | FileCheck -check-prefixes=COM,DIS %s
-; RUN: opt < %s -S -data-layout=A5 2>&1 | FileCheck -check-prefixes=COM,AS %s
-; RUN: opt < %t.bc -S | FileCheck -check-prefixes=COM,DIS %s
+; RUN: llvm-as < %s -o %t.bc -data-layout=A5 2>&1 | FileCheck -check-prefixes=ALL,AS %s
+; RUN: llvm-dis < %t.bc | FileCheck -check-prefixes=ALL,DIS %s
+; RUN: opt < %s -S -data-layout=A5 2>&1 | FileCheck -check-prefixes=ALL,AS %s
+; RUN: opt < %t.bc -S | FileCheck -check-prefixes=ALL,DIS %s
 
 define void @foo() {
 entry:
 ; DIS: target datalayout = "A5"
 ; DIS: %tmp = alloca i32, addrspace(5)
   %tmp = alloca i32, addrspace(5)
   call void @llvm.dbg.value(
       metadata i8* undef,
       metadata !DILocalVariable(scope: !1),
       metadata !DIExpression())
-; COM-NOT: Allocation instruction pointer not in the stack address space!
+; ALL-NOT: Allocation instruction pointer not in the stack address space!
 ; AS: llvm.dbg.value intrinsic requires a !dbg attachment
 ; AS: warning: ignoring invalid debug info in <stdin>
 ret void
 }
 
 declare void @llvm.dbg.value(metadata, metadata, metadata)
 
 !llvm.module.flags = !{!0}
 !0 = !{i32 2, !"Debug Info Version", i32 3}
 !1 = distinct !DISubprogram(name: "foo")

llvm/test/FileCheck/comment/after-words.txt

+# Comment prefixes are not recognized at ends of words.
+
+RUN: echo 'foo'                 >  %t.in
+RUN: echo 'bar'                 >> %t.in
+RUN: echo 'foo'                 >> %t.in
+RUN: echo 'bar'                 >> %t.in
+RUN: echo 'FOO-COM: CHECK: foo' >  %t.chk
+RUN: echo 'RUN_COM: CHECK: bar' >> %t.chk
+RUN: echo 'RUN3COM: CHECK: foo' >> %t.chk
+RUN: echo ' COMRUN: CHECK: bar' >> %t.chk
+RUN: %ProtectFileCheckOutput FileCheck -vv %t.chk < %t.in 2>&1 | FileCheck %s
+
+CHECK: .chk:1:17: remark: CHECK: expected string found in input
+CHECK: .chk:2:17: remark: CHECK: expected string found in input
+CHECK: .chk:3:17: remark: CHECK: expected string found in input
+CHECK: .chk:4:17: remark: CHECK: expected string found in input

llvm/test/FileCheck/comment/bad-comment-prefix.txt

+# Bad comment prefixes are diagnosed.
+
+# Check empty comment prefix.
+RUN: %ProtectFileCheckOutput not FileCheck /dev/null < /dev/null 2>&1 \
+RUN:                                      -comment-prefixes= | \
+RUN:   FileCheck -check-prefix=PREFIX-EMPTY %s
+RUN: %ProtectFileCheckOutput not FileCheck /dev/null < /dev/null 2>&1 \
+RUN:                                      -comment-prefixes=,FOO | \
+RUN:   FileCheck -check-prefix=PREFIX-EMPTY %s
+RUN: %ProtectFileCheckOutput not FileCheck /dev/null < /dev/null 2>&1 \
+RUN:                                      -comment-prefixes=FOO, | \
+RUN:   FileCheck -check-prefix=PREFIX-EMPTY %s
+RUN: %ProtectFileCheckOutput not FileCheck /dev/null < /dev/null 2>&1 \
+RUN:                                      -comment-prefixes=FOO,,BAR | \
+RUN:   FileCheck -check-prefix=PREFIX-EMPTY %s
+PREFIX-EMPTY: error: supplied comment prefix must not be the empty string
+
+# Check invalid characters in comment prefix.
+RUN: %ProtectFileCheckOutput not FileCheck /dev/null < /dev/null 2>&1 \
+RUN:                                       -comment-prefixes=. | \
+RUN:   FileCheck -check-prefix=PREFIX-BAD-CHAR1 %s
+PREFIX-BAD-CHAR1: error: supplied comment prefix must start with a letter and contain only alphanumeric characters, hyphens, and underscores: '.'
+RUN: %ProtectFileCheckOutput not FileCheck /dev/null < /dev/null 2>&1 \
+RUN:                                       -comment-prefixes='foo ' | \
+RUN:   FileCheck -check-prefix=PREFIX-BAD-CHAR2 %s
+PREFIX-BAD-CHAR2: error: supplied comment prefix must start with a letter and contain only alphanumeric characters, hyphens, and underscores: 'foo '
+
+# Check duplicate comment prefixes.
+RUN: %ProtectFileCheckOutput not FileCheck /dev/null < /dev/null 2>&1 \
+RUN:                                      -comment-prefixes=FOO,BAR,BAZ,BAR | \
+RUN:   FileCheck -check-prefix=COMMENT-PREFIX-DUP %s
+COMMENT-PREFIX-DUP: error: supplied comment prefix must be unique among check and comment prefixes: 'BAR'
+
+# Check user-supplied check prefix that duplicates a default comment prefix.
+RUN: %ProtectFileCheckOutput not FileCheck /dev/null < /dev/null 2>&1 \
+RUN:                                      -check-prefixes=FOO,COM | \
+RUN:   FileCheck -check-prefix=CHECK-PREFIX-DUP-COMMENT %s
+CHECK-PREFIX-DUP-COMMENT: error: supplied check prefix must be unique among check and comment prefixes: 'COM'
+
+# Check user-supplied comment prefix that duplicates default check prefixes.
+RUN: %ProtectFileCheckOutput not FileCheck /dev/null < /dev/null 2>&1 \
+RUN:                                      -comment-prefixes=CHECK | \
+RUN:   FileCheck -check-prefix=COMMENT-PREFIX-DUP-CHECK %s
+COMMENT-PREFIX-DUP-CHECK: error: supplied comment prefix must be unique among check and comment prefixes: 'CHECK'

llvm/test/FileCheck/comment/blank-comments.txt

+# Comment directives with blank contents are fine.
+
+RUN: echo 'foo'        >  %t.in
+RUN: echo 'COM:'       >  %t.chk
+RUN: echo 'CHECK: foo' >> %t.chk
+RUN: echo ' COM: '     >> %t.chk
+RUN: %ProtectFileCheckOutput FileCheck -vv %t.chk < %t.in 2>&1 | FileCheck %s
+
+CHECK: .chk:2:8: remark: CHECK: expected string found in input

llvm/test/FileCheck/comment/suffixes.txt

+# Comment prefixes plus check directive suffixes are not comment directives
+# and are treated as plain text.
+
+RUN: echo foo                    >  %t.in
+RUN: echo bar                    >> %t.in
+RUN: echo 'COM-NEXT: CHECK: foo' >  %t.chk
+RUN: echo 'RUN-NOT: CHECK: bar'  >> %t.chk
+
+RUN: %ProtectFileCheckOutput FileCheck -vv %t.chk < %t.in 2>&1 | \
+RUN:   FileCheck -check-prefix=CHECK1 %s
+
+CHECK1: .chk:1:18: remark: CHECK: expected string found in input
+CHECK1: .chk:2:17: remark: CHECK: expected string found in input
+
+# But we can define them as comment prefixes.
+
+RUN: %ProtectFileCheckOutput \
+RUN: FileCheck -vv -comment-prefixes=COM,RUN,RUN-NOT %t.chk < %t.in 2>&1 | \
+RUN:   FileCheck -check-prefix=CHECK2 %s
+
+CHECK2: .chk:1:18: remark: CHECK: expected string found in input
+CHECK2-NOT: .chk:2

llvm/test/FileCheck/comment/suppresses-checks.txt

+Comment directives successfully comment out check directives.
+
+# Check all default comment prefixes.
+# Check that a comment directive at the beginning/end of the file is handled.
+# Check that the preceding/following line's check directive is not affected.
+RUN: echo 'foo'                    >  %t-1.in
+RUN: echo 'COM: CHECK: bar'        >  %t-1.chk
+RUN: echo 'CHECK: foo'             >> %t-1.chk
+RUN: echo 'RUN: echo "CHECK: baz"' >> %t-1.chk
+RUN: %ProtectFileCheckOutput FileCheck -vv %t-1.chk < %t-1.in 2>&1 | \
+RUN:   FileCheck -DCHECK_LINE=2 %s
+
+# Check the case of one user-specified comment prefix.
+# Check that a comment directive not at the beginning of a line is handled.
+RUN: echo 'foo'                                      >  %t-2.in
+RUN: echo 'CHECK: foo'                               >  %t-2.chk
+RUN: echo 'letters then space MY-PREFIX: CHECK: bar' >> %t-2.chk
+RUN: %ProtectFileCheckOutput \
+RUN: FileCheck -vv %t-2.chk -comment-prefixes=MY-PREFIX < %t-2.in 2>&1 | \
+RUN:   FileCheck -DCHECK_LINE=1 %s
+
+# Check the case of multiple user-specified comment prefixes.
+RUN: echo 'foo'               >  %t-3.in
+RUN: echo 'Bar_2: CHECK: Bar' >  %t-3.chk
+RUN: echo 'CHECK: foo'        >> %t-3.chk
+RUN: echo 'Foo_1: CHECK: Foo' >> %t-3.chk
+RUN: echo 'Baz_3: CHECK: Baz' >> %t-3.chk
+RUN: %ProtectFileCheckOutput \
+RUN: FileCheck -vv %t-3.chk -comment-prefixes=Foo_1,Bar_2 \
+RUN:           -comment-prefixes=Baz_3 < %t-3.in 2>&1 | \
+RUN:   FileCheck -DCHECK_LINE=2 %s
+
+CHECK: .chk:[[CHECK_LINE]]:8: remark: CHECK: expected string found in input

llvm/test/FileCheck/comment/unused-check-prefixes.txt

+# Comment directives do not suppress diagnostics for unused check prefixes.
+
+RUN: echo 'foo'       >  %t.in
+RUN: echo 'COM: foo'  >  %t.chk
+RUN: echo 'RUN: echo' >> %t.chk
+RUN: %ProtectFileCheckOutput not FileCheck %t.chk < %t.in 2>&1 | FileCheck %s
+
+CHECK: error: no check strings found with prefix 'CHECK:'

llvm/test/FileCheck/comment/unused-comment-prefixes.txt

+# Not using comment directives is always fine.
+
+RUN: echo 'foo'        > %t.in
+RUN: echo 'CHECK: foo' > %t.chk
+
+# Check the case of default comment prefixes.
+RUN: %ProtectFileCheckOutput \
+RUN: FileCheck -vv %t.chk < %t.in 2>&1 | FileCheck %s
+
+# Specifying non-default comment prefixes doesn't mean you have to use them.
+# For example, they might be applied to an entire test suite via
+# FILECHECK_OPTS or via a wrapper command or substitution.
+RUN: %ProtectFileCheckOutput \
+RUN: FileCheck -vv -comment-prefixes=FOO %t.chk < %t.in 2>&1 | FileCheck %s
+
+CHECK: .chk:1:8: remark: CHECK: expected string found in input

llvm/test/FileCheck/comment/within-checks.txt

+# A comment directive is not recognized within a check directive's pattern and
+# thus does not comment out the remainder of the pattern.
+
+RUN: echo 'foo'                 > %t.in
+RUN: echo 'CHECK: foo COM: bar' > %t.chk
+RUN: %ProtectFileCheckOutput not FileCheck %t.chk < %t.in 2>&1 | FileCheck %s
+
+CHECK: .chk:1:8: error: CHECK: expected string not found in input

llvm/test/FileCheck/first-character-match.txt

-RUN: FileCheck -check-prefix=RUN -input-file %s %s
+RUN: FileCheck -check-prefix=RUN --comment-prefixes=COM -input-file %s %s
 // Prefix is at the first character in the file. The run line then matches itself.

llvm/test/FileCheck/validate-check-prefix.txt

 // RUN: %ProtectFileCheckOutput not FileCheck -check-prefix=A! -input-file %s %s 2>&1 | FileCheck -check-prefix=BAD_PREFIX %s
 // RUN: FileCheck -check-prefix=A1a-B_c -input-file %s %s
-// RUN: %ProtectFileCheckOutput not FileCheck -check-prefix=REPEAT -check-prefix=REPEAT -input-file %s %s 2>&1 | FileCheck -check-prefix=BAD_PREFIX %s
+// RUN: %ProtectFileCheckOutput not FileCheck -check-prefix=REPEAT -check-prefix=REPEAT -input-file %s %s 2>&1 | FileCheck -check-prefix=DUPLICATE_PREFIX %s
 // RUN: %ProtectFileCheckOutput not FileCheck -check-prefix=VALID -check-prefix=A! -input-file %s %s 2>&1 | FileCheck -check-prefix=BAD_PREFIX %s
-// RUN: %ProtectFileCheckOutput not FileCheck -check-prefix= -input-file %s %s 2>&1 | FileCheck -check-prefix=BAD_PREFIX %s
+// RUN: %ProtectFileCheckOutput not FileCheck -check-prefix= -input-file %s %s 2>&1 | FileCheck -check-prefix=EMPTY_PREFIX %s
 foobar
 ; A1a-B_c: foobar
 
-; BAD_PREFIX: Supplied check-prefix is invalid! Prefixes must be
-  unique and start with a letter and contain only alphanumeric characters, hyphens and underscores
+; BAD_PREFIX: supplied check prefix must start with a letter and contain only alphanumeric characters, hyphens, and underscores: 'A!'
+
+; DUPLICATE_PREFIX: error: supplied check prefix must be unique among check and comment prefixes: 'REPEAT'
+
+; EMPTY_PREFIX: error: supplied check prefix must not be the empty string

llvm/utils/FileCheck/FileCheck.cpp

static cl::list<std::string> CheckPrefixes(
     "check-prefix",
     cl::desc("Prefix to use from check file (defaults to 'CHECK')"));
 static cl::alias CheckPrefixesAlias(
     "check-prefixes", cl::aliasopt(CheckPrefixes), cl::CommaSeparated,
     cl::NotHidden,
     cl::desc(
         "Alias for -check-prefix permitting multiple comma separated values"));
 
+static cl::list<std::string> CommentPrefixes(
+    "comment-prefixes", cl::CommaSeparated, cl::Hidden,
+    cl::desc("Comma-separated list of comment prefixes to use from check file\n"
+             "(defaults to 'COM,RUN'). Please avoid using this feature in\n"
+             "LLVM's LIT-based test suites, which should be easier to\n"
+             "maintain if they all follow a consistent comment style. This\n"
+             "feature is meant for non-LIT test suites using FileCheck."));
+
 static cl::opt<bool> NoCanonicalizeWhiteSpace(
     "strict-whitespace",
     cl::desc("Do not treat all horizontal whitespace as equivalent"));
 
 static cl::opt<bool> IgnoreCase(
     "ignore-case",
     cl::desc("Use case-insensitive matching"));
 
std::string GetCheckTypeAbbreviation(Check::FileCheckType Ty) {
   case Check::CheckNot:
     return "not";
   case Check::CheckDAG:
     return "dag";
   case Check::CheckLabel:
     return "label";
   case Check::CheckEmpty:
     return "empty";
+  case Check::CheckComment:
+    return "com";
   case Check::CheckEOF:
     return "eof";
   case Check::CheckBadNot:
     return "bad-not";
   case Check::CheckBadCount:
     return "bad-count";
   case Check::CheckNone:
     llvm_unreachable("invalid FileCheckType");
if (CheckFilename.empty()) {
     errs() << "<check-file> not specified\n";
     return 2;
   }
 
   FileCheckRequest Req;
   for (StringRef Prefix : CheckPrefixes)
     Req.CheckPrefixes.push_back(Prefix);
 
+  for (StringRef Prefix : CommentPrefixes)
+    Req.CommentPrefixes.push_back(Prefix);
+
   for (StringRef CheckNot : ImplicitCheckNot)
     Req.ImplicitCheckNot.push_back(CheckNot);
 
   bool GlobalDefineError = false;
   for (StringRef G : GlobalDefines) {
     size_t EqIdx = G.find('=');
     if (EqIdx == std::string::npos) {
       errs() << "Missing equal sign in command-line definition '-D" << G
int main(int argc, char **argv) {
   Req.NoCanonicalizeWhiteSpace = NoCanonicalizeWhiteSpace;
   Req.MatchFullLines = MatchFullLines;
   Req.IgnoreCase = IgnoreCase;
 
   if (VerboseVerbose)
     Req.Verbose = true;
 
   FileCheck FC(Req);
-  if (!FC.ValidateCheckPrefixes()) {
-    errs() << "Supplied check-prefix is invalid! Prefixes must be unique and "
-              "start with a letter and contain only alphanumeric characters, "
-              "hyphens and underscores\n";
+  if (!FC.ValidateCheckPrefixes())
     return 2;
-  }
 
   Regex PrefixRE = FC.buildCheckPrefixRegex();
   std::string REError;
   if (!PrefixRE.isValid(REError)) {
     errs() << "Unable to combine check-prefix strings into a prefix regular "
               "expression! This is likely a bug in FileCheck's verification of "
               "the check-prefix strings. Regular expression parsing failed "
               "with the following error: "