[FileCheck] Suppress old -v/-vv diags if dumping input

[jdenny]

Summary

The old diagnostic form of the trace produced by -v and -vv looks
like:

check1:1:8: remark: CHECK: expected string found in input
CHECK: abc 
       ^
<stdin>:1:3: note: found here
; abc def 
  ^~~

When dumping annotated input is requested (via -dump-input), I find
that this old trace is not useful and is sometimes harmful:

1. The old trace is mostly redundant because the same basic information also appears in the input dump's annotations.

2. The old trace buries any error diagnostic between it and the input dump, but I find it useful to see any error diagnostic up front.

3. FILECHECK_OPTS=-dump-input=fail requests annotated input dumps only for failed FileCheck calls. However, I have to also add -v or -vv to get a full set of annotations, and that can produce massive output from all FileCheck calls in all tests. That's a real problem when I run this in the IDE I use, which grinds to a halt as it tries to capture all that output.

When -dump-input=fail|always, this patch suppresses the old trace from
-v or -vv. Error diagnostics still print as usual. If you want the
old trace, perhaps to see variable expansions, you can set
-dump-input=none (the default).

[jdenny]

Ping.

[probinson]

Thanks for the ping. The code change looks straightforward, but a handful of questions on the tests.

llvm/test/FileCheck/dump-input-annotations.txt

29

This kind of sequence always makes me wonder whether the NOT text might appear after one of the other lines gets a match. If 'remark' shouldn't occur at all, then maybe use -implicit-check-not on the run line?

69

implicit-check-not?

105

implict-check-not?

142

implicit-check-not?

195

implicit-check-not?

214

implicit-check-not?

258

implicit-check-not?

292

implicit-check-not?

327

implicit-check-not?

357

implicit-check-not?

404

implicit-check-not?

452

implicit-check-not?

llvm/test/FileCheck/dump-input-enable.txt

14

So, this first set of changes goes from matching (or not) on the first line, to always matching the first line, and matching (or not) on the second line. Does that make a material difference in the test?

56

Offhand it looks like every use of -dump-input in this file now also has -v which seems excessive. Also like you're losing coverage of that combination. Think about using -v more judiciously here.

[jdenny]

Thanks for reviewing!

llvm/test/FileCheck/dump-input-annotations.txt

29

Currently, the remarks forming the trace all come before the error because FileCheck stops at the error, but that could change in the future, intentionally or not. Good idea. I'll adjust.

llvm/test/FileCheck/dump-input-enable.txt

14

With that change, -v (when -dump-input=none) always produces a remark for the trace so we can always test whether the trace is suppressed.

Without that change, there's only a remark for the trace if the first line matches.

56

Offhand it looks like every use of -dump-input in this file now also has -v which seems excessive.

The purpose of this test file is to check the effects of various ways of disabling/enabling input dumps. Checking for a trace seems important because disabling/enabling input dumps in any manner is now intended to permit/suppress the trace. I don't see a reason to skip that check for just some ways of disabling/enabling input dumps.

Also like you're losing coverage of that combination.

Combinations without -v? I feel like related dimensions of functionality are tested well elsewhere, such as verbose.txt and dump-input-annotations.txt.

Think about using -v more judiciously here.

If the above explanation of my testing strategy isn't satisfying, please explain your objection a little more. Maybe I'm misunderstanding.

[probinson]

llvm/test/FileCheck/dump-input-enable.txt

56

verbose.txt looks at -v/-vv without -dump-input. dump-input-annotations.txt uses only -dump-input=always and not the other flavors. The behavior of -dump-input=never/fail without -v is not verified anywhere that I can find.

[jdenny]

llvm/test/FileCheck/dump-input-enable.txt

56

Ah, I see. I don't want to lose coverage of trace suppression by -dump-input, so I'll add a few new checks here for the cases you've mentioned. Thanks.

[jdenny]

• In dump-input-annotations.txt, use -implicit-check-not to thoroughly check for trace suppression.
• In dump-input-enable.txt, improve comments, and cover cases of -dump-input=never|fail without -v.

[probinson]

LGTM although I do wonder at the usefulness of the wildcards in the implicit-not checks.

llvm/test/FileCheck/dump-input-annotations.txt

20

Is this different from -implicit-check-not='remark:' ? I wouldn't have thought so. I mean, in a positive CHECK, you use the wildcards to hoover up the rest of the line, but in a NOT it doesn't seem useful.

[jdenny]

llvm/test/FileCheck/dump-input-annotations.txt

20

Ah, I hadn't noticed that -match-full-lines only affects positive matches. I'll adjust all those before committing.

Thanks for the review!

llvm/docs/CommandGuide/FileCheck.rst

.. option:: -D<VAR=VALUE>
   ``CHECK:`` lines.
 
 .. option:: -version
 
  Show the version number of this program.
 
 .. option:: -v
 
-  Print directive pattern matches.
+  Print good directive pattern matches.  However, if ``-input-dump=fail`` or
+  ``-input-dump=always``, add those matches as input annotations instead.
 
 .. option:: -vv
 
   Print information helpful in diagnosing internal FileCheck issues, such as
   discarded overlapping ``CHECK-DAG:`` matches, implicit EOF pattern matches,
   and ``CHECK-NOT:`` patterns that do not have matches.  Implies ``-v``.
+  However, if ``-input-dump=fail`` or ``-input-dump=always``, just add that
+  information as input annotations instead.
 
 .. option:: --allow-deprecated-dag-overlap
 
   Enable overlapping among matches in a group of consecutive ``CHECK-DAG:``
   directives.  This option is deprecated and is only provided for convenience
   as old tests are migrated to the new non-overlapping ``CHECK-DAG:``
   implementation.
 

llvm/lib/Support/FileCheck.cpp

 }
 
 static void PrintMatch(bool ExpectedMatch, const SourceMgr &SM,
                        StringRef Prefix, SMLoc Loc, const FileCheckPattern &Pat,
                        int MatchedCount, StringRef Buffer,
                        StringMap<StringRef> &VariableTable, size_t MatchPos,
                        size_t MatchLen, const FileCheckRequest &Req,
                        std::vector<FileCheckDiag> *Diags) {
+  bool PrintDiag = true;
   if (ExpectedMatch) {
     if (!Req.Verbose)
       return;
     if (!Req.VerboseVerbose && Pat.getCheckTy() == Check::CheckEOF)
       return;
+    // Due to their verbosity, we don't print verbose diagnostics here if we're
+    // gathering them for a different rendering, but we always print other
+    // diagnostics.
+    PrintDiag = !Diags;
   }
   SMRange MatchRange = ProcessMatchResult(
       ExpectedMatch ? FileCheckDiag::MatchFoundAndExpected
                     : FileCheckDiag::MatchFoundButExcluded,
       SM, Loc, Pat.getCheckTy(), Buffer, MatchPos, MatchLen, Diags);
+  if (!PrintDiag)
+    return;
+
   std::string Message = formatv("{0}: {1} string found in input",
                                 Pat.getCheckTy().getDescription(Prefix),
                                 (ExpectedMatch ? "expected" : "excluded"))
                             .str();
   if (Pat.getCount() > 1)
     Message += formatv(" ({0} out of {1})", MatchedCount, Pat.getCount()).str();
 
   SM.PrintMessage(

 }
 
 static void PrintNoMatch(bool ExpectedMatch, const SourceMgr &SM,
                          StringRef Prefix, SMLoc Loc,
                          const FileCheckPattern &Pat, int MatchedCount,
                          StringRef Buffer, StringMap<StringRef> &VariableTable,
                          bool VerboseVerbose,
                          std::vector<FileCheckDiag> *Diags) {
-  if (!ExpectedMatch && !VerboseVerbose)
+  bool PrintDiag = true;
+  if (!ExpectedMatch) {
+    if (!VerboseVerbose)
+      return;
+    // Due to their verbosity, we don't print verbose diagnostics here if we're
+    // gathering them for a different rendering, but we always print other
+    // diagnostics.
+    PrintDiag = !Diags;
+  }
+
+  // If the current position is at the end of a line, advance to the start of
+  // the next line.
+  Buffer = Buffer.substr(Buffer.find_first_not_of(" \t\n\r"));
+  SMRange SearchRange = ProcessMatchResult(
+      ExpectedMatch ? FileCheckDiag::MatchNoneButExpected
+                    : FileCheckDiag::MatchNoneAndExcluded,
+      SM, Loc, Pat.getCheckTy(), Buffer, 0, Buffer.size(), Diags);
+  if (!PrintDiag)
     return;
 
-  // Otherwise, we have an error, emit an error message.
+  // Print "not found" diagnostic.
   std::string Message = formatv("{0}: {1} string not found in input",
                                 Pat.getCheckTy().getDescription(Prefix),
                                 (ExpectedMatch ? "expected" : "excluded"))
                             .str();
   if (Pat.getCount() > 1)
     Message += formatv(" ({0} out of {1})", MatchedCount, Pat.getCount()).str();
-
   SM.PrintMessage(
       Loc, ExpectedMatch ? SourceMgr::DK_Error : SourceMgr::DK_Remark, Message);
 
-  // Print the "scanning from here" line.  If the current position is at the
-  // end of a line, advance to the start of the next line.
-  Buffer = Buffer.substr(Buffer.find_first_not_of(" \t\n\r"));
-  SMRange SearchRange = ProcessMatchResult(
-      ExpectedMatch ? FileCheckDiag::MatchNoneButExpected
-                    : FileCheckDiag::MatchNoneAndExcluded,
-      SM, Loc, Pat.getCheckTy(), Buffer, 0, Buffer.size(), Diags);
+  // Print the "scanning from here" line.
   SM.PrintMessage(SearchRange.Start, SourceMgr::DK_Note, "scanning from here");
 
   // Allow the pattern to print additional information if desired.
   Pat.PrintVariableUses(SM, Buffer, VariableTable);
 
   if (ExpectedMatch)
     Pat.PrintFuzzyMatch(SM, Buffer, VariableTable, Diags);
 }
for (auto MI = MatchRanges.begin(), ME = MatchRanges.end(); true; ++MI) {
         }
       }
       if (!Overlap) {
         // Insert non-overlapping match into list.
         MatchRanges.insert(MI, M);
         break;
       }
       if (Req.VerboseVerbose) {
+        // Due to their verbosity, we don't print verbose diagnostics here if
+        // we're gathering them for a different rendering, but we always print
+        // other diagnostics.
+        if (!Diags) {
           SMLoc OldStart = SMLoc::getFromPointer(Buffer.data() + MI->Pos);
           SMLoc OldEnd = SMLoc::getFromPointer(Buffer.data() + MI->End);
           SMRange OldRange(OldStart, OldEnd);
           SM.PrintMessage(OldStart, SourceMgr::DK_Note,
                           "match discarded, overlaps earlier DAG match here",
                           {OldRange});
-        if (Diags)
+        } else
           Diags->rbegin()->MatchTy = FileCheckDiag::MatchFoundButDiscarded;
       }
       MatchPos = MI->End;
     }
     if (!Req.VerboseVerbose)
       PrintMatch(true, SM, Prefix, Pat.getLoc(), Pat, 1, Buffer, VariableTable,
                  MatchPos, MatchLen, Req, Diags);
 

llvm/test/FileCheck/dump-input-annotations.txt

 ; RUN: echo 'goodbye' >> %t.in
 ; RUN: echo 'world' >> %t.in
 ; RUN: echo 'unicorn' >> %t.in
 
 ; RUN: echo 'CHECK: hello' > %t.chk
 ; RUN: echo 'CHECK: universe' >> %t.chk
 
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -v 2>&1 \
-; RUN: | FileCheck -strict-whitespace -match-full-lines -check-prefix=ALIGN %s
+; RUN: | FileCheck -strict-whitespace -match-full-lines -check-prefix=ALIGN \
+; RUN:             -implicit-check-not='{{.*}}remark:{{.*}}' %s
+
+; Verbose diagnostics are suppressed but not errors.
+; ALIGN:{{.*}}error:{{.*}}
+; ALIGN:{{.*}}possible intended match here{{.*}}
 
 ; ALIGN:Full input was:
 ; ALIGN-NEXT:<<<<<<
 ; ALIGN-NEXT:           1: hello world
 ; ALIGN-NEXT:check:1       ^~~~~
 ; ALIGN-NEXT:check:2'0           X~~~~
 ; ALIGN-NEXT:           2: goodbye
 ; ALIGN-NEXT:check:2'0     ~~~~~~~
 ; ALIGN-NEXT:           3: world
 ; ALIGN-NEXT:check:2'0     ~~~~~
 ; ALIGN-NEXT:           4: unicorn
 ; ALIGN-NEXT:check:2'0     ~~~~~~~ error: no match found
 ; ALIGN-NEXT:check:2'1     ?       possible intended match

 ; RUN: echo 'hello'   > %t.in
 ; RUN: echo 'again'   >> %t.in
 ; RUN: echo 'whirled' >> %t.in
 
 ; RUN: echo 'CHECK: hello' > %t.chk
 ; RUN: echo 'CHECK: world' >> %t.chk
 
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk 2>&1 \
-; RUN: | FileCheck -match-full-lines %s -check-prefix=CHK
+; RUN: | FileCheck -match-full-lines %s -check-prefix=CHK \
+; RUN:             -implicit-check-not='{{.*}}remark:{{.*}}'
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -v 2>&1 \
-; RUN: | FileCheck -match-full-lines %s -check-prefixes=CHK,CHK-V
+; RUN: | FileCheck -match-full-lines %s -check-prefixes=CHK,CHK-V \
+; RUN:             -implicit-check-not='{{.*}}remark:{{.*}}'
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -vv 2>&1 \
-; RUN: | FileCheck -match-full-lines %s -check-prefixes=CHK,CHK-V
+; RUN: | FileCheck -match-full-lines %s -check-prefixes=CHK,CHK-V \
+; RUN:             -implicit-check-not='{{.*}}remark:{{.*}}'
+
+; Verbose diagnostics are suppressed but not errors.
+; CHK: {{.*}}error:{{.*}}
+; CHK: {{.*}}possible intended match here{{.*}}
 
 ; CHK:        <<<<<<
 ; CHK-NEXT:              1: hello
 ; CHK-V-NEXT: check:1       ^~~~~
 ; CHK-NEXT:              2: again
 ; CHK-NEXT:   check:2'0     X~~~~
 ; CHK-NEXT:              3: whirled
 ; CHK-NEXT:   check:2'0     ~~~~~~~ error: no match found
 ; CHK-NEXT:   check:2'1     ?       possible intended match
 ; CHK-NEXT:   >>>>>>
 ; CHK-NOT:    {{.}}
 
 ;--------------------------------------------------
 ; CHECK-COUNT-<num>
 ;--------------------------------------------------
 
 ; Good match and no match.
 
 ; RUN: echo 'pete'   > %t.in
 ; RUN: echo 'repete' >> %t.in
 ; RUN: echo 'repeat' >> %t.in
 
 ; RUN: echo 'CHECK-COUNT-3: pete' > %t.chk
 
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk 2>&1 \
-; RUN: | FileCheck -match-full-lines %s -check-prefixes=CNT,CNT-Q
+; RUN: | FileCheck -match-full-lines %s -check-prefixes=CNT,CNT-Q \
+; RUN:             -implicit-check-not='{{.*}}remark:{{.*}}'
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -v 2>&1 \
-; RUN: | FileCheck -match-full-lines %s -check-prefixes=CNT,CNT-V
+; RUN: | FileCheck -match-full-lines %s -check-prefixes=CNT,CNT-V \
+; RUN:             -implicit-check-not='{{.*}}remark:{{.*}}'
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -vv 2>&1 \
-; RUN: | FileCheck -match-full-lines %s -check-prefixes=CNT,CNT-V
+; RUN: | FileCheck -match-full-lines %s -check-prefixes=CNT,CNT-V \
+; RUN:             -implicit-check-not='{{.*}}remark:{{.*}}'
+
+; Verbose diagnostics are suppressed but not errors.
+; CNT: {{.*}}error:{{.*}}
 
 ; CNT:          <<<<<<
 ; CNT-NEXT:                1: pete
 ; CNT-V-NEXT:   count:1'0     ^~~~
 ; CNT-NEXT:                2: repete
 ; CNT-V-NEXT:   count:1'1       ^~~~
 ; CNT-NEXT:                3: repeat
 ; CNT-Q-NEXT:   count:1       X~~~~~ error: no match found
 ; CNT-V-NEXT:   count:1'2     X~~~~~ error: no match found
 ; CNT-NEXT:     >>>>>>
 ; CNT-NOT:      {{.}}
 
 ;--------------------------------------------------
 ; CHECK-NEXT (also: EOF search-range, wrong-line match)
 ;--------------------------------------------------
 
 ; Good match and no match.
 
 ; RUN: echo 'hello' > %t.in
 ; RUN: echo 'again' >> %t.in
 
 ; RUN: echo 'CHECK: hello' > %t.chk
 ; RUN: echo 'CHECK-NEXT: again' >> %t.chk
 ; RUN: echo 'CHECK-NEXT: world' >> %t.chk
 
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk 2>&1 \
-; RUN: | FileCheck -match-full-lines %s -check-prefix=NXT
+; RUN: | FileCheck -match-full-lines %s -check-prefix=NXT \
+; RUN:             -implicit-check-not='{{.*}}remark:{{.*}}'
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -v 2>&1 \
-; RUN: | FileCheck -match-full-lines %s -check-prefixes=NXT,NXT-V
+; RUN: | FileCheck -match-full-lines %s -check-prefixes=NXT,NXT-V \
+; RUN:             -implicit-check-not='{{.*}}remark:{{.*}}'
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -vv 2>&1 \
-; RUN: | FileCheck -match-full-lines %s -check-prefixes=NXT,NXT-V,NXT-VV
+; RUN: | FileCheck -match-full-lines %s -check-prefixes=NXT,NXT-V,NXT-VV \
+; RUN:             -implicit-check-not='{{.*}}remark:{{.*}}'
+
+; Verbose diagnostics are suppressed but not errors.
+; NXT: {{.*}}error:{{.*}}
 
 ; NXT:        <<<<<<
 ; NXT-NEXT:            1: hello
 ; NXT-V-NEXT: check:1     ^~~~~
 ; NXT-NEXT:            2: again
 ; NXT-V-NEXT: next:2      ^~~~~
 ; NXT-NEXT:            3:
 ; NXT-NEXT:   next:3      X error: no match found
 ; NXT-NEXT:   >>>>>>
 ; NXT-NOT:    {{.}}
 

 
 ; RUN: echo 'hello world!' > %t.in
 
 ; RUN: echo 'CHECK: hello' > %t.chk
 ; RUN: echo 'CHECK-SAME: world' >> %t.chk
 ; RUN: echo 'CHECK-SAME: again' >> %t.chk
 
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk 2>&1 \
-; RUN: | FileCheck -match-full-lines %s -check-prefix=SAM
+; RUN: | FileCheck -match-full-lines %s -check-prefix=SAM \
+; RUN:             -implicit-check-not='{{.*}}remark:{{.*}}'
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -v 2>&1 \
-; RUN: | FileCheck -match-full-lines %s -check-prefixes=SAM,SAM-V
+; RUN: | FileCheck -match-full-lines %s -check-prefixes=SAM,SAM-V \
+; RUN:             -implicit-check-not='{{.*}}remark:{{.*}}'
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -vv 2>&1 \
-; RUN: | FileCheck -match-full-lines %s -check-prefixes=SAM,SAM-V,SAM-VV
+; RUN: | FileCheck -match-full-lines %s -check-prefixes=SAM,SAM-V,SAM-VV \
+; RUN:             -implicit-check-not='{{.*}}remark:{{.*}}'
+
+; Verbose diagnostics are suppressed but not errors.
+; SAM: {{.*}}error:{{.*}}
 
 ; SAM:        <<<<<<
 ; SAM-NEXT:            1: hello world!
 ; SAM-V-NEXT: check:1     ^~~~~
 ; SAM-V-NEXT: same:2            ^~~~~
 ; SAM-NEXT:   same:3                 X error: no match found
 ; SAM-NEXT:   >>>>>>
 ; SAM-NOT:    {{.}}
 
 ; Wrong-line match.
 
 ; RUN: echo 'again' >> %t.in
 
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -v 2>&1 \
-; RUN: | FileCheck -match-full-lines %s -check-prefixes=SAM2
+; RUN: | FileCheck -match-full-lines %s -check-prefixes=SAM2 \
+; RUN:             -implicit-check-not='{{.*}}remark:{{.*}}'
+
+; Verbose diagnostics are suppressed but not errors.
+; SAM2: {{.*}}error:{{.*}}
 
 ; SAM2:      <<<<<<
 ; SAM2-NEXT:          1: hello world!
 ; SAM2-NEXT: check:1     ^~~~~
 ; SAM2-NEXT: same:2            ^~~~~
 ; SAM2-NEXT:          2: again
 ; SAM2-NEXT: same:3      !~~~~ error: match on wrong line
 ; SAM2-NEXT: >>>>>>
 ; SAM2-NOT:  {{.}}
 
 ;--------------------------------------------------

 ; RUN: echo 'label' >> %t.in
 
 ; RUN: echo 'CHECK: hello' > %t.chk
 ; RUN: echo 'CHECK-EMPTY:' >> %t.chk
 ; RUN: echo 'CHECK-EMPTY:' >> %t.chk
 ; RUN: echo 'CHECK-LABEL: label' >> %t.chk
 
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk 2>&1 \
-; RUN: | FileCheck -match-full-lines %s -check-prefix=EMP
+; RUN: | FileCheck -match-full-lines %s -check-prefix=EMP \
+; RUN:             -implicit-check-not='{{.*}}remark:{{.*}}'
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -v 2>&1 \
-; RUN: | FileCheck -match-full-lines %s -check-prefixes=EMP,EMP-V
+; RUN: | FileCheck -match-full-lines %s -check-prefixes=EMP,EMP-V \
+; RUN:             -implicit-check-not='{{.*}}remark:{{.*}}'
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -vv 2>&1 \
-; RUN: | FileCheck -match-full-lines %s -check-prefixes=EMP,EMP-V,EMP-VV
+; RUN: | FileCheck -match-full-lines %s -check-prefixes=EMP,EMP-V,EMP-VV \
+; RUN:             -implicit-check-not='{{.*}}remark:{{.*}}'
+
+; Verbose diagnostics are suppressed but not errors.
+; EMP: {{.*}}error:{{.*}}
 
 ; EMP:        <<<<<<
 ; EMP-NEXT:            1: hello
 ; EMP-V-NEXT: check:1     ^~~~~
 ; EMP-NEXT:            2:
 ; EMP-V-NEXT: empty:2     ^
 ; EMP-NEXT:            3: world
 ; EMP-NEXT:   empty:3     X~~~~
 ; EMP-NEXT:            4: label
 ; EMP-NEXT:   empty:3     ~~~~~ error: no match found
 ; EMP-V-NEXT: label:4     ^~~~~
 ; EMP-NEXT:   >>>>>>
 ; EMP-NOT:    {{.}}
 
 ; Wrong-line match.
 
 ; RUN: echo 'hello' > %t.in
 ; RUN: echo 'world' >> %t.in
 
 ; RUN: echo 'CHECK: hello' > %t.chk
 ; RUN: echo 'CHECK-EMPTY:' >> %t.chk
 
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk 2>&1 \
-; RUN: | FileCheck -match-full-lines %s -check-prefix=EMP2
+; RUN: | FileCheck -match-full-lines %s -check-prefix=EMP2 \
+; RUN:             -implicit-check-not='{{.*}}remark:{{.*}}'
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -v 2>&1 \
-; RUN: | FileCheck -match-full-lines %s -check-prefixes=EMP2,EMP2-V
+; RUN: | FileCheck -match-full-lines %s -check-prefixes=EMP2,EMP2-V \
+; RUN:             -implicit-check-not='{{.*}}remark:{{.*}}'
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -vv 2>&1 \
-; RUN: | FileCheck -match-full-lines %s -check-prefixes=EMP2,EMP2-V,EMP2-VV
+; RUN: | FileCheck -match-full-lines %s -check-prefixes=EMP2,EMP2-V,EMP2-VV \
+; RUN:             -implicit-check-not='{{.*}}remark:{{.*}}'
+
+; Verbose diagnostics are suppressed but not errors.
+; EMP2: {{.*}}error:{{.*}}
 
 ; EMP2:        <<<<<<
 ; EMP2-NEXT:            1: hello
 ; EMP2-V-NEXT: check:1     ^~~~~
 ; EMP2-NEXT:            2: world
 ; EMP2-NEXT:            3:
 ; EMP2-NEXT:   empty:2     !     error: match on wrong line
 ; EMP2-NEXT:   >>>>>>
 ; EMP2-NOT:    {{.}}
 
 ;--------------------------------------------------
 ; CHECK-NOT (also: EOF pattern, and multiline range that ends before EOL)
 ;--------------------------------------------------
 
 ; No match (success) and unexpected match (error).
 
 ; RUN: echo 'hello' > %t.in
 ; RUN: echo 'world' >> %t.in
 ; RUN: echo 'again' >> %t.in
 
 ; RUN: echo 'CHECK-NOT: goodbye' > %t.chk
 ; RUN: echo 'CHECK-NOT: world' >> %t.chk
 
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk 2>&1 \
-; RUN: | FileCheck -match-full-lines %s -check-prefix=NOT
+; RUN: | FileCheck -match-full-lines %s -check-prefix=NOT \
+; RUN:             -implicit-check-not='{{.*}}remark:{{.*}}'
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -v 2>&1 \
-; RUN: | FileCheck -match-full-lines %s -check-prefixes=NOT,NOT-V
+; RUN: | FileCheck -match-full-lines %s -check-prefixes=NOT,NOT-V \
+; RUN:             -implicit-check-not='{{.*}}remark:{{.*}}'
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -vv 2>&1 \
-; RUN: | FileCheck -match-full-lines %s -check-prefixes=NOT,NOT-V,NOT-VV
+; RUN: | FileCheck -match-full-lines %s -check-prefixes=NOT,NOT-V,NOT-VV \
+; RUN:             -implicit-check-not='{{.*}}remark:{{.*}}'
+
+; Verbose diagnostics are suppressed but not errors.
+; NOT: {{.*}}error:{{.*}}
 
 ; NOT:         <<<<<<
 ; NOT-NEXT:           1: hello
 ; NOT-VV-NEXT: not:1     X~~~~
 ; NOT-NEXT:           2: world
 ; NOT-VV-NEXT: not:1     ~~~~~
 ; NOT-NEXT:    not:2     !~~~~ error: no match expected
 ; NOT-NEXT:           3: again
 ; NOT-VV-NEXT: not:1     ~~~~~
 ; NOT-VV-NEXT:        4:
 ; NOT-VV-NEXT: eof:2     ^
 ; NOT-NEXT:    >>>>>>
 ; NOT-NOT:     {{.}}
 
 ; Again, but with a CHECK instead of EOF as search range end.
 
 ; RUN: echo 'CHECK: ain' >> %t.chk
 
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk 2>&1 \
-; RUN: | FileCheck -match-full-lines %s -check-prefix=NOT2
+; RUN: | FileCheck -match-full-lines %s -check-prefix=NOT2 \
+; RUN:             -implicit-check-not='{{.*}}remark:{{.*}}'
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -v 2>&1 \
-; RUN: | FileCheck -match-full-lines %s -check-prefixes=NOT2,NOT2-V
+; RUN: | FileCheck -match-full-lines %s -check-prefixes=NOT2,NOT2-V \
+; RUN:             -implicit-check-not='{{.*}}remark:{{.*}}'
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -vv 2>&1 \
-; RUN: | FileCheck -match-full-lines %s -check-prefixes=NOT2,NOT2-V,NOT2-VV
+; RUN: | FileCheck -match-full-lines %s -check-prefixes=NOT2,NOT2-V,NOT2-VV \
+; RUN:             -implicit-check-not='{{.*}}remark:{{.*}}'
+
+; Verbose diagnostics are suppressed but not errors.
+; NOT2: {{.*}}error:{{.*}}
 
 ; NOT2:         <<<<<<
 ; NOT2-NEXT:             1: hello
 ; NOT2-VV-NEXT: not:1       X~~~~
 ; NOT2-NEXT:             2: world
 ; NOT2-VV-NEXT: not:1       ~~~~~
 ; NOT2-NEXT:    not:2       !~~~~ error: no match expected
 ; NOT2-NEXT:             3: again
 ; NOT2-VV-NEXT: not:1       ~~
 ; NOT2-V-NEXT:  check:3       ^~~
 ; NOT2-NEXT:    >>>>>>

 ; Prefixes used here:
 ; DAG    = quiet, -v, or -vv
 ; DAG-Q  = quiet
 ; DAG-V  = -v or -vv (-vv implies -v)
 ; DAG-VQ = -v and not -vv
 ; DAG-VV = -vv
 
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk 2>&1 \
-; RUN: | FileCheck -match-full-lines %s -check-prefixes=DAG,DAG-Q
+; RUN: | FileCheck -match-full-lines %s -check-prefixes=DAG,DAG-Q \
+; RUN:             -implicit-check-not='{{.*}}remark:{{.*}}'
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -v 2>&1 \
-; RUN: | FileCheck -match-full-lines %s -check-prefixes=DAG,DAG-V,DAG-VQ
+; RUN: | FileCheck -match-full-lines %s -check-prefixes=DAG,DAG-V,DAG-VQ \
+; RUN:             -implicit-check-not='{{.*}}remark:{{.*}}'
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -vv 2>&1 \
-; RUN: | FileCheck -match-full-lines %s -check-prefixes=DAG,DAG-V,DAG-VV
+; RUN: | FileCheck -match-full-lines %s -check-prefixes=DAG,DAG-V,DAG-VV \
+; RUN:             -implicit-check-not='{{.*}}remark:{{.*}}'
+
+; Verbose diagnostics are suppressed but not errors.
+; DAG: {{.*}}error:{{.*}}
 
 ; DAG:         <<<<<<
 ; DAG-NEXT:             1: abc
 ; DAG-V-NEXT:  dag:2       ^~~
 ; DAG-VV-NEXT: dag:3'0     !~~ discard: overlaps earlier match
 ; DAG-NEXT:             2: def
 ; DAG-V-NEXT:  dag:1       ^~~
 ; DAG-VV-NEXT: dag:4'0     !~~ discard: overlaps earlier match
 ; DAG-NEXT:             3: abc
 ; DAG-VQ-NEXT: dag:3       ^~~
 ; DAG-VV-NEXT: dag:3'1     ^~~

 ; RUN: echo 'lab1' >> %t.in
 ; RUN: echo 'bar' >> %t.in
 
 ; RUN: echo 'CHECK-LABEL: lab0' > %t.chk
 ; RUN: echo 'CHECK: foo' >> %t.chk
 ; RUN: echo 'CHECK-LABEL: lab2' >> %t.chk
 
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk 2>&1 \
-; RUN: | FileCheck -match-full-lines %s -check-prefixes=LAB
+; RUN: | FileCheck -match-full-lines %s -check-prefixes=LAB \
+; RUN:             -implicit-check-not='{{.*}}remark:{{.*}}'
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -v 2>&1 \
-; RUN: | FileCheck -match-full-lines %s -check-prefixes=LAB,LAB-V
+; RUN: | FileCheck -match-full-lines %s -check-prefixes=LAB,LAB-V \
+; RUN:             -implicit-check-not='{{.*}}remark:{{.*}}'
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -vv 2>&1 \
-; RUN: | FileCheck -match-full-lines %s -check-prefixes=LAB,LAB-V,LAB-VV
+; RUN: | FileCheck -match-full-lines %s -check-prefixes=LAB,LAB-V,LAB-VV \
+; RUN:             -implicit-check-not='{{.*}}remark:{{.*}}'
+
+; Verbose diagnostics are suppressed but not errors.
+; LAB: {{.*}}error:{{.*}}
+; LAB: {{.*}}possible intended match{{.*}}
 
 ; LAB:         <<<<<<
 ; LAB-NEXT:               1: lab0
 ; LAB-V-NEXT:  label:1'0     ^~~~
 ; LAB-V-NEXT:  label:1'1     ^~~~
 ; LAB-NEXT:               2: foo
 ; LAB-NEXT:    label:3'0     X~~
 ; LAB-NEXT:               3: lab1
 ; LAB-NEXT:    label:3'0     ~~~~
 ; LAB-NEXT:    label:3'1     ?    possible intended match
 ; LAB-NEXT:               4: bar
 ; LAB-NEXT:    label:3'0     ~~~ error: no match found
 ; LAB-NEXT:    >>>>>>
 ; LAB-NOT:     {{.}}
-
-

llvm/test/FileCheck/dump-input-enable.txt

-; RUN: echo ciao > %t.good
+;--------------------------------------------------
+; Create the check file, good input, and bad input.
+;
+; For both good and bad input, make sure the -v trace has at least one remark
+; so we can check how trace suppression is affected by -dump-input.
+;--------------------------------------------------
+
+; RUN: echo hello > %t.good
 ; RUN: echo world >> %t.good
 
 ; RUN: echo hello > %t.err
-; RUN: echo world >> %t.err
+; RUN: echo whirled >> %t.err
 
-; RUN: echo 'CHECK: ciao' > %t.check
+; RUN: echo 'CHECK: hello' > %t.check
 ; RUN: echo 'CHECK-NEXT: world' >> %t.check
 
 ;--------------------------------------------------
-; unknown value
+; Check -dump-input=<bad value>.
 ;--------------------------------------------------
 
 ; RUN: not FileCheck -input-file %t.good %t.check -check-prefix=CHECK \
 ; RUN:               -match-full-lines -dump-input=foobar 2>&1 \
 ; RUN: | FileCheck %s -match-full-lines -check-prefix=BADVAL
 
 ; No positional arg.
 ; RUN: not FileCheck -dump-input=foobar 2>&1 \
 ; RUN: | FileCheck %s -match-full-lines -check-prefix=BADVAL
 
 BADVAL: FileCheck{{.*}}: for the -dump-input option: Cannot find option named 'foobar'!
 
 ;--------------------------------------------------
-; help
+; Check -dump-input=help.
 ;--------------------------------------------------
 
 ; Appended to normal command line.
 ; RUN: FileCheck -input-file %t.err -color %t.check -dump-input=help \
 ; RUN: | FileCheck %s -check-prefix=HELP
 
 ; No positional arg.
 ; RUN: FileCheck -dump-input=help | FileCheck %s -check-prefix=HELP
 
 HELP-NOT: {{.}}
 HELP: The following description was requested by -dump-input=help
 HELP: try{{.*}}-color
 HELP-NOT: {{.}}
 
 ;--------------------------------------------------
-; never
+; Check -dump-input=never.
+;
+; Include the case without -v, which isn't covered elsewhere.
 ;--------------------------------------------------
 
 ; RUN: FileCheck -input-file %t.good %t.check -check-prefix=CHECK \
 ; RUN:           -match-full-lines -dump-input=never 2>&1 \
-; RUN: | FileCheck %s -match-full-lines -check-prefix=CHECK-NODUMP -allow-empty
+; RUN: | FileCheck %s -match-full-lines -allow-empty \
+; RUN:             -check-prefixes=NOTRACE,NODUMP
 
 ; RUN: not FileCheck -input-file %t.err %t.check -check-prefix=CHECK \
 ; RUN:               -match-full-lines -dump-input=never 2>&1 \
-; RUN: | FileCheck %s -match-full-lines -check-prefix=CHECK-NODUMP
+; RUN: | FileCheck %s -match-full-lines -check-prefixes=NOTRACE,ERR,NODUMP
+
+; RUN: FileCheck -input-file %t.good %t.check -check-prefix=CHECK \
+; RUN:           -match-full-lines -dump-input=never -v 2>&1 \
+; RUN: | FileCheck %s -match-full-lines -check-prefixes=TRACE,NODUMP
+
+; RUN: not FileCheck -input-file %t.err %t.check -check-prefix=CHECK \
+; RUN:               -match-full-lines -dump-input=never -v 2>&1 \
+; RUN: | FileCheck %s -match-full-lines -check-prefixes=TRACE,ERR,NODUMP
 
 ;--------------------------------------------------
-; default: never
+; Check no -dump-input, which defaults to never.
 ;--------------------------------------------------
 
 ; RUN: FileCheck -input-file %t.good %t.check -check-prefix=CHECK \
-; RUN:           -match-full-lines 2>&1 \
-; RUN: | FileCheck %s -match-full-lines -check-prefix=CHECK-NODUMP -allow-empty
+; RUN:           -match-full-lines -v 2>&1 \
+; RUN: | FileCheck %s -match-full-lines -check-prefixes=TRACE,NODUMP
 
 ; RUN: not FileCheck -input-file %t.err %t.check -check-prefix=CHECK \
-; RUN:               -match-full-lines 2>&1 \
-; RUN: | FileCheck %s -match-full-lines -check-prefix=CHECK-NODUMP
+; RUN:               -match-full-lines -v 2>&1 \
+; RUN: | FileCheck %s -match-full-lines -check-prefixes=TRACE,ERR,NODUMP
 
 ;--------------------------------------------------
-; fail
+; Check -dump-input=fail.
+;
+; Include the case without -v, which isn't covered elsewhere.
 ;--------------------------------------------------
 
 ; RUN: FileCheck -input-file %t.good %t.check -check-prefix=CHECK \
 ; RUN:           -match-full-lines -dump-input=fail 2>&1 \
-; RUN: | FileCheck %s -match-full-lines -check-prefix=CHECK-NODUMP -allow-empty
+; RUN: | FileCheck %s -match-full-lines -allow-empty \
+; RUN:             -check-prefixes=NOTRACE,NODUMP
 
 ; RUN: not FileCheck -input-file %t.err %t.check -check-prefix=CHECK \
 ; RUN:               -match-full-lines -dump-input=fail 2>&1 \
-; RUN: | FileCheck %s -match-full-lines -check-prefix=CHECK-ERR
+; RUN: | FileCheck %s -match-full-lines -check-prefixes=NOTRACE,ERR,DUMP-ERR
+
+; RUN: FileCheck -input-file %t.good %t.check -check-prefix=CHECK \
+; RUN:           -match-full-lines -dump-input=fail -v 2>&1 \
+; RUN: | FileCheck %s -match-full-lines -allow-empty \
+; RUN:             -check-prefixes=NOTRACE,NODUMP
+
+; RUN: not FileCheck -input-file %t.err %t.check -check-prefix=CHECK \
+; RUN:               -match-full-lines -dump-input=fail -v 2>&1 \
+; RUN: | FileCheck %s -match-full-lines \
+; RUN:                -check-prefixes=NOTRACE,ERR,DUMP-ERR,DUMP-ERR-V
 
 ;--------------------------------------------------
-; -dump-input-on-failure
+; Check -dump-input-on-failure.
 ;--------------------------------------------------
 
 ; RUN: FileCheck -input-file %t.good %t.check -check-prefix=CHECK \
-; RUN:           -match-full-lines -dump-input-on-failure 2>&1 \
-; RUN: | FileCheck %s -match-full-lines -check-prefix=CHECK-NODUMP -allow-empty
+; RUN:           -match-full-lines -dump-input-on-failure -v 2>&1 \
+; RUN: | FileCheck %s -match-full-lines -allow-empty \
+; RUN:             -check-prefixes=NOTRACE,NODUMP
 
 ; RUN: not FileCheck -input-file %t.err %t.check -check-prefix=CHECK \
-; RUN:               -match-full-lines -dump-input-on-failure 2>&1 \
-; RUN: | FileCheck %s -match-full-lines -check-prefix=CHECK-ERR
+; RUN:               -match-full-lines -dump-input-on-failure -v 2>&1 \
+; RUN: | FileCheck %s -match-full-lines \
+; RUN:                -check-prefixes=NOTRACE,ERR,DUMP-ERR,DUMP-ERR-V
 
 ; RUN: env FILECHECK_DUMP_INPUT_ON_FAILURE=1 \
 ; RUN: FileCheck -input-file %t.good %t.check -check-prefix=CHECK \
-; RUN:           -match-full-lines 2>&1 \
-; RUN: | FileCheck %s -match-full-lines -check-prefix=CHECK-NODUMP -allow-empty
+; RUN:           -match-full-lines -v 2>&1 \
+; RUN: | FileCheck %s -match-full-lines -allow-empty \
+; RUN:             -check-prefixes=NOTRACE,NODUMP
 
 ; RUN: env FILECHECK_DUMP_INPUT_ON_FAILURE=1 \
 ; RUN: not FileCheck -input-file %t.err %t.check -check-prefix=CHECK \
-; RUN:               -match-full-lines 2>&1 \
-; RUN: | FileCheck %s -match-full-lines -check-prefix=CHECK-ERR
+; RUN:               -match-full-lines -v 2>&1 \
+; RUN: | FileCheck %s -match-full-lines \
+; RUN:                -check-prefixes=NOTRACE,ERR,DUMP-ERR,DUMP-ERR-V
 
 ;--------------------------------------------------
-; always
+; Check -dump-input=always.
 ;--------------------------------------------------
 
 ; RUN: FileCheck -input-file %t.good %t.check -check-prefix=CHECK \
 ; RUN:           -match-full-lines -dump-input=always -v 2>&1 \
-; RUN: | FileCheck %s -match-full-lines -check-prefix=CHECK-GOOD
+; RUN: | FileCheck %s -match-full-lines -check-prefixes=NOTRACE,DUMP-OK
 
 ; RUN: not FileCheck -input-file %t.err %t.check -check-prefix=CHECK \
-; RUN:               -match-full-lines -dump-input=always 2>&1 \
-; RUN: | FileCheck %s -match-full-lines -check-prefix=CHECK-ERR
+; RUN:               -match-full-lines -dump-input=always -v 2>&1 \
+; RUN: | FileCheck %s -match-full-lines \
+; RUN:                -check-prefixes=NOTRACE,ERR,DUMP-ERR,DUMP-ERR-V
 
 ; END.
 
-; CHECK-GOOD: Full input was:
-; CHECK-GOOD-NEXT: <<<<<<
-; CHECK-GOOD-NEXT:          1: ciao
-; CHECK-GOOD-NEXT: check:1     ^~~~
-; CHECK-GOOD-NEXT:          2: world
-; CHECK-GOOD-NEXT: next:2      ^~~~~
-; CHECK-GOOD-NEXT: >>>>>>
-
-; CHECK-ERR: Full input was:
-; CHECK-ERR-NEXT: <<<<<<
-; CHECK-ERR-NEXT:          1: hello
-; CHECK-ERR-NEXT: check:1     X~~~~
-; CHECK-ERR-NEXT:          2: world
-; CHECK-ERR-NEXT: check:1     ~~~~~ error: no match found
-; CHECK-ERR-NEXT: >>>>>>
+;--------------------------------------------------
+; Check the output for all cases that actually process directives.
+;--------------------------------------------------
 
-; CHECK-NODUMP-NOT: <<<<<<
+; Trace is sometimes suppressed.
+; TRACE:       {{.*}}remark:{{.*}}
+; NOTRACE-NOT: {{.*}}remark:{{.*}}
+
+; Error diagnostics are never suppressed.
+; ERR: {{.*}}error:{{.*}}
+
+; NODUMP-NOT: <<<<<<
+
+; DUMP-OK:      Full input was:
+; DUMP-OK-NEXT: <<<<<<
+; DUMP-OK-NEXT:          1: hello
+; DUMP-OK-NEXT: check:1     ^~~~~
+; DUMP-OK-NEXT:          2: world
+; DUMP-OK-NEXT: next:2      ^~~~~
+; DUMP-OK-NEXT: >>>>>>
+
+; DUMP-ERR:        Full input was:
+; DUMP-ERR-NEXT:   <<<<<<
+; DUMP-ERR-NEXT:            1: hello
+; DUMP-ERR-V-NEXT: check:1     ^~~~~
+; DUMP-ERR-NEXT:            2: whirled
+; DUMP-ERR-NEXT:   next:2      X~~~~~~ error: no match found
+; DUMP-ERR-NEXT:   >>>>>>

llvm/utils/FileCheck/FileCheck.cpp

 
 static cl::opt<bool> AllowDeprecatedDagOverlap(
     "allow-deprecated-dag-overlap", cl::init(false),
     cl::desc("Enable overlapping among matches in a group of consecutive\n"
              "CHECK-DAG directives.  This option is deprecated and is only\n"
              "provided for convenience as old tests are migrated to the new\n"
              "non-overlapping CHECK-DAG implementation.\n"));
 
-static cl::opt<bool> Verbose("v", cl::init(false),
-                             cl::desc("Print directive pattern matches.\n"));
+static cl::opt<bool> Verbose(
+    "v", cl::init(false),
+    cl::desc("Print directive pattern matches, or add them to the input dump\n"
+             "if enabled.\n"));
 
 static cl::opt<bool> VerboseVerbose(
     "vv", cl::init(false),
     cl::desc("Print information helpful in diagnosing internal FileCheck\n"
-             "issues.  Implies -v.\n"));
+             "issues, or add it to the input dump if enabled.  Implies\n"
+             "-v.\n"));
 static const char * DumpInputEnv = "FILECHECK_DUMP_INPUT_ON_FAILURE";
 
 static cl::opt<bool> DumpInputOnFailure(
     "dump-input-on-failure", cl::init(std::getenv(DumpInputEnv)),
     cl::desc("Dump original input to stderr before failing.\n"
              "The value can be also controlled using\n"
              "FILECHECK_DUMP_INPUT_ON_FAILURE environment variable.\n"
              "This option is deprecated in favor of -dump-input=fail.\n"));