[FileCheck] Annotate input dump (5/7)

[jdenny]

Summary

This patch implements input annotations for diagnostics enabled by -v,
which report good matches for directives. These annotations mark
match ranges using ^~~.

For example:

$ FileCheck -dump-input=help
The following description was requested by -dump-input=help to
explain the input annotations printed by -dump-input=always and
-dump-input=fail:

  - L:     labels line number L of the input file
  - T:L    labels the only match result for a pattern of type T from line L of
           the check file
  - T:L'N  labels the Nth match result for a pattern of type T from line L of
           the check file
  - ^~~    marks good match (reported if -v)
  - !~~    marks bad match, such as:
           - CHECK-NEXT on same line as previous match (error)
           - CHECK-NOT found (error)
  - X~~    marks search range when no match is found, such as:
           - CHECK-NEXT not found (error)
  - ?      marks fuzzy match when no match is found
  - colors success, error, fuzzy match, unmatched input

If you are not seeing color above or in input dumps, try: -color

$ FileCheck -v -dump-input=always check3 < input3 |& sed -n '/^<<<</,$p'
<<<<<<
         1: abc foobar def
check:1     ^~~
not:2           !~~~~~     error: no match expected
check:3                ^~~
>>>>>>

$ cat check3
CHECK:     abc
CHECK-NOT: foobar
CHECK:     def

$ cat input3
abc foobar def

-vv enables these annotations for FileCheck's implicit EOF patterns as
well. For an example where EOF patterns become relevant, see patch 7
in this series.

If colors are enabled, ^~~ is green to suggest success.

-v plus color enables highlighting of input text that has no final
match for any expected pattern. The highlight uses a cyan background
to suggest a cold section. This highlighting can make it easier to
spot text that was intended to be matched but that failed to be
matched in a long series of good matches.

CHECK-COUNT-<num> good matches are another case where there can be
multiple match results for the same directive.

[jdenny]

Rebased onto updated patches earlier in series.

[jdenny]

Update for changes earlier in patch series.

[jdenny]

Propagate changes started earlier in patch series.

[jdenny]

Fix bug in logic for highlighting unmatched text: it was broken for -allow-deprecated-dag-overlap.

[jdenny]

Continue changes started earlier in patch series.

[jdenny]

llvm/utils/FileCheck/FileCheck.cpp

494

A break statement could be here.

[jdenny]

• Add a break statement to skip useless loop iterations.
• Convert a list to a vector.

[jdenny]

Adjust comment, as suggested by probinson.

[probinson]

The terminology around "possible" "expected" and "final" matches probably warrants a few lines of commentary somewhere, unless I missed it in an earlier patch. There are getting to be enough cases that the casual reader (ahem) can't just Zen their way through them all.
Can be done as a follow-up if you prefer.
One comment nit and LGTM.

llvm/utils/FileCheck/FileCheck.cpp

469–482

For the case...

[jdenny]

In D53897#1331556, @probinson wrote:

The terminology around "possible" "expected" and "final" matches probably warrants a few lines of commentary somewhere, unless I missed it in an earlier patch. There are getting to be enough cases that the casual reader (ahem) can't just Zen their way through them all.

Sure, I'll add some additional comments here.

Can be done as a follow-up if you prefer.

Yes, a follow-up would be easier. I'll just commit something. If you prefer a phab review first, that's absolutely fine for me, but let me know.

[probinson]

In D53897#1331582, @jdenny wrote:

Yes, a follow-up would be easier. I'll just commit something. If you prefer a phab review first, that's absolutely fine for me, but let me know.

No, just do it.

[jdenny]

Remaining suggestions done in D55738.

llvm/include/llvm/Support/FileCheck.h

struct FileCheckDiag {
   /// example, there might be a fuzzy match after a fail.
   ///
   /// We iterate these types, so they must have contiguous values in
   /// [0, MatchTypeCount).  Moreover, keep match types together if they use the
   /// same mark in annotated input dumps or else printing of the annotation key
   /// will malfunction.
   enum MatchType {
     // TODO: More members will appear with later patches in this series.
+    MatchFinalAndExpected, //< the final match for an expected pattern
+    MatchTypeFirst = MatchFinalAndExpected,
     MatchFinalButExcluded, //< the final match for an excluded pattern
-    MatchTypeFirst = MatchFinalButExcluded,
     MatchFinalButIllegal,  //< the final but illegal match for an expected pattern
     MatchNoneButExpected,  //< no match for an expected pattern
     MatchFuzzy,            //< a fuzzy match (because no perfect match)
     MatchTypeCount,
   } MatchTy;
   /// The match range if MatchTy is not MatchNoneButExpected, or the search
   /// range otherwise.
   unsigned InputStartLine, InputStartCol, InputEndLine, InputEndCol;

llvm/lib/Support/FileCheck.cpp

if (!VariableUses.empty()) {
     }
   }
 }
 
 static SMRange ProcessMatchResult(FileCheckDiag::MatchType MatchTy,
                                   const SourceMgr &SM, SMLoc Loc,
                                   Check::FileCheckType CheckTy,
                                   StringRef Buffer, size_t Pos, size_t Len,
-                                  std::list<FileCheckDiag> *Diags) {
+                                  std::list<FileCheckDiag> *Diags,
+                                  bool AdjustPrevDiag = false) {
   SMLoc Start = SMLoc::getFromPointer(Buffer.data() + Pos);
   SMLoc End = SMLoc::getFromPointer(Buffer.data() + Pos + Len);
   SMRange Range(Start, End);
   // TODO: The second condition will disappear when we extend this to handle
   // more match types.
-  if (Diags && MatchTy != FileCheckDiag::MatchTypeCount)
+  if (Diags && MatchTy != FileCheckDiag::MatchTypeCount) {
+    if (AdjustPrevDiag)
+      Diags->rbegin()->MatchTy = MatchTy;
+    else
       Diags->emplace_back(SM, CheckTy, Loc, MatchTy, Range);
+  }
   return Range;
 }
 
 void FileCheckPattern::PrintFuzzyMatch(
     const SourceMgr &SM, StringRef Buffer,
     const StringMap<StringRef> &VariableTable,
     std::list<FileCheckDiag> *Diags) const {
   // Attempt to find the closest/best fuzzy match.  Usually an error happens
static void PrintMatch(bool ExpectedMatch, const SourceMgr &SM,
                        std::list<FileCheckDiag> *Diags) {
   if (ExpectedMatch) {
     if (!Req.Verbose)
       return;
     if (!Req.VerboseVerbose && Pat.getCheckTy() == Check::CheckEOF)
       return;
   }
   SMRange MatchRange = ProcessMatchResult(
-      ExpectedMatch ? FileCheckDiag::MatchTypeCount
+      ExpectedMatch ? FileCheckDiag::MatchFinalAndExpected
                     : FileCheckDiag::MatchFinalButExcluded,
       SM, Loc, Pat.getCheckTy(), Buffer, MatchPos, MatchLen, Diags);
   SM.PrintMessage(
       Loc, ExpectedMatch ? SourceMgr::DK_Remark : SourceMgr::DK_Error,
       CheckTypeName(Prefix, Pat.getCheckTy()) + ": " +
           (ExpectedMatch ? "expected" : "excluded") +
           " string found in input");
   SM.PrintMessage(MatchRange.Start, SourceMgr::DK_Note, "found here",
size_t FileCheckString::Check(const SourceMgr &SM, StringRef Buffer,
   if (!IsLabelScanMode) {
     StringRef SkippedRegion = Buffer.substr(LastPos, MatchPos);
 
     // If this check is a "CHECK-NEXT", verify that the previous match was on
     // the previous line (i.e. that there is one newline between them).
     if (CheckNext(SM, SkippedRegion)) {
       ProcessMatchResult(FileCheckDiag::MatchFinalButIllegal, SM, Loc,
                          Pat.getCheckTy(), MatchBuffer, MatchPos, MatchLen,
-                         Diags);
+                         Diags, Req.Verbose);
       return StringRef::npos;
     }
 
     // If this check is a "CHECK-SAME", verify that the previous match was on
     // the same line (i.e. that there is no newline between them).
     if (CheckSame(SM, SkippedRegion)) {
       ProcessMatchResult(FileCheckDiag::MatchFinalButIllegal, SM, Loc,
                          Pat.getCheckTy(), MatchBuffer, MatchPos, MatchLen,
-                         Diags);
+                         Diags, Req.Verbose);
       return StringRef::npos;
     }
 
     // If this match had "not strings", verify that they don't exist in the
     // skipped region.
     if (CheckNot(SM, SkippedRegion, NotStrings, VariableTable, Req, Diags))
       return StringRef::npos;
   }
for (auto MI = MatchRanges.begin(), ME = MatchRanges.end(); true; ++MI) {
       }
       if (Req.VerboseVerbose) {
         SMLoc OldStart = SMLoc::getFromPointer(Buffer.data() + MI->Pos);
         SMLoc OldEnd = SMLoc::getFromPointer(Buffer.data() + MI->End);
         SMRange OldRange(OldStart, OldEnd);
         SM.PrintMessage(OldStart, SourceMgr::DK_Note,
                         "match discarded, overlaps earlier DAG match here",
                         {OldRange});
+        if (Diags)
+          Diags->pop_back();
       }
       MatchPos = MI->End;
     }
     if (!Req.VerboseVerbose)
       PrintMatch(true, SM, Prefix, Pat.getLoc(), Pat, Buffer, VariableTable,
                  MatchPos, MatchLen, Req, Diags);
 
     // Handle the end of a CHECK-DAG group.

llvm/test/FileCheck/dump-input-annotations.txt

 ; RUN: echo 'CHECK: universe' >> %t.chk
 
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -v 2>&1 \
 ; RUN: | FileCheck -strict-whitespace -match-full-lines -check-prefix=ALIGN %s
 
 ; ALIGN:Full input was:
 ; ALIGN-NEXT:<<<<<<
 ; ALIGN-NEXT:         1: hello world
+; ALIGN-NEXT:check:1     ^~~~~
 ; ALIGN-NEXT:check:2           X~~~~
 ; ALIGN-NEXT:         2: goodbye
 ; ALIGN-NEXT:check:2     ~~~~~~~
 ; ALIGN-NEXT:         3: world
 ; ALIGN-NEXT:check:2     ~~~~~
 ; ALIGN-NEXT:>>>>>>
 ; ALIGN-NOT:{{.}}
 

 ; RUN: | FileCheck -match-full-lines %s -check-prefix=CHK
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -v 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=CHK,CHK-V
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -vv 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=CHK,CHK-V
 
 ; CHK:        <<<<<<
 ; CHK-NEXT:              1: hello
+; CHK-V-NEXT: check:1       ^~~~~
 ; CHK-NEXT:              2: again
 ; CHK-NEXT:   check:2'0     X~~~~
 ; CHK-NEXT:              3: whirled
 ; CHK-NEXT:   check:2'0     ~~~~~~~
 ; CHK-NEXT:   check:2'1     ?
 ; CHK-NEXT:   >>>>>>
 ; CHK-NOT:    {{.}}
 

 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefix=NXT
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -v 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=NXT,NXT-V
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -vv 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=NXT,NXT-V,NXT-VV
 
 ; NXT:        <<<<<<
-; NXT-NEXT:           1: hello
-; NXT-NEXT:           2: again
-; NXT-NEXT:           3:
-; NXT-NEXT:   next:3     X
+; NXT-NEXT:            1: hello
+; NXT-V-NEXT: check:1     ^~~~~
+; NXT-NEXT:            2: again
+; NXT-V-NEXT: next:2      ^~~~~
+; NXT-NEXT:            3:
+; NXT-NEXT:   next:3      X
 ; NXT-NEXT:   >>>>>>
 ; NXT-NOT:    {{.}}
 
 ; Illegal match.
 
 ; RUN: echo 'yonder' >> %t.in
 ; RUN: echo 'world' >> %t.in
 
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefix=NXT2
 
 ; NXT2:      <<<<<<
 ; NXT2-NEXT:         1: hello
 ; NXT2-NEXT:         2: again
 ; NXT2-NEXT:         3: yonder
 ; NXT2-NEXT:         4: world
 ; NXT2-NEXT: next:3     !~~~~
 ; NXT2-NEXT: >>>>>>
 ; NXT2-NOT:  {{.}}
 
 ;--------------------------------------------------
-; CHECK-SAME (also: single-char search range, illegal match)
+; CHECK-SAME (also: multiple annotations per line, single-char search range,
+; illegal match)
 ;--------------------------------------------------
 
 ; Good match and no match.
 
 ; RUN: echo 'hello world!' > %t.in
 
 ; RUN: echo 'CHECK: hello' > %t.chk
 ; RUN: echo 'CHECK-SAME: world' >> %t.chk
 ; RUN: echo 'CHECK-SAME: again' >> %t.chk
 
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefix=SAM
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -v 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=SAM,SAM-V
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -vv 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=SAM,SAM-V,SAM-VV
 
 ; SAM:        <<<<<<
-; SAM-NEXT:           1: hello world!
-; SAM-NEXT:   same:3                X
+; SAM-NEXT:            1: hello world!
+; SAM-V-NEXT: check:1     ^~~~~
+; SAM-V-NEXT: same:2            ^~~~~
+; SAM-NEXT:   same:3                 X
 ; SAM-NEXT:   >>>>>>
 ; SAM-NOT:    {{.}}
 
 ; Illegal match.
 
 ; RUN: echo 'again' >> %t.in
 
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -v 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=SAM2
 
 ; SAM2:      <<<<<<
 ; SAM2-NEXT:          1: hello world!
+; SAM2-NEXT: check:1     ^~~~~
+; SAM2-NEXT: same:2            ^~~~~
 ; SAM2-NEXT:          2: again
 ; SAM2-NEXT: same:3      !~~~~
 ; SAM2-NEXT: >>>>>>
 ; SAM2-NOT:  {{.}}
 
 ;--------------------------------------------------
-; CHECK-EMPTY (also: search range ends at label, illegal match)
+; CHECK-EMPTY (also: search range ends at label, single-char match, illegal
+; match)
 ;--------------------------------------------------
 
 ; Good match and no match.
 ;
 ; CHECK-EMPTY always seems to match an empty line at EOF (illegally when it's
 ; not the next line) unless either (1) the last line is non-empty and has no
 ; newline or (2) there's a CHECK-LABEL to end the search range before EOF.  We
 ; choose scenario 2 to check the case of no match.

 ; RUN: | FileCheck -match-full-lines %s -check-prefix=EMP
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -v 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=EMP,EMP-V
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -vv 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=EMP,EMP-V,EMP-VV
 
 ; EMP:        <<<<<<
 ; EMP-NEXT:            1: hello
+; EMP-V-NEXT: check:1     ^~~~~
 ; EMP-NEXT:            2:
+; EMP-V-NEXT: empty:2     ^
 ; EMP-NEXT:            3: world
 ; EMP-NEXT:   empty:3     X~~~~
 ; EMP-NEXT:            4: label
 ; EMP-NEXT:   empty:3     ~~~~~
+; EMP-V-NEXT: label:4     ^~~~~
 ; EMP-NEXT:   >>>>>>
 ; EMP-NOT:    {{.}}
 
 ; Illegal match.
 
 ; RUN: echo 'hello' > %t.in
 ; RUN: echo 'world' >> %t.in
 
 ; RUN: echo 'CHECK: hello' > %t.chk
 ; RUN: echo 'CHECK-EMPTY:' >> %t.chk
 
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefix=EMP2
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -v 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=EMP2,EMP2-V
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -vv 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=EMP2,EMP2-V,EMP2-VV
 
 ; EMP2:        <<<<<<
 ; EMP2-NEXT:            1: hello
+; EMP2-V-NEXT: check:1     ^~~~~
 ; EMP2-NEXT:            2: world
 ; EMP2-NEXT:            3:
 ; EMP2-NEXT:   empty:2     !
 ; EMP2-NEXT:   >>>>>>
 ; EMP2-NOT:    {{.}}
 
 ;--------------------------------------------------
-; CHECK-NOT
+; CHECK-NOT (also: EOF pattern)
 ;--------------------------------------------------
 
 ; No match (success) and illegal match.
 
 ; RUN: echo 'hello' > %t.in
 ; RUN: echo 'world' >> %t.in
 ; RUN: echo 'again' >> %t.in
 
 ; RUN: echo 'CHECK-NOT: goodbye' > %t.chk
 ; RUN: echo 'CHECK-NOT: world' >> %t.chk
 
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefix=NOT
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -v 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=NOT,NOT-V
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -vv 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=NOT,NOT-V,NOT-VV
 
 ; NOT:         <<<<<<
 ; NOT-NEXT:           1: hello
 ; NOT-NEXT:           2: world
 ; NOT-NEXT:    not:2     !~~~~
 ; NOT-NEXT:           3: again
+; NOT-VV-NEXT:        4:
+; NOT-VV-NEXT: eof:2     ^
 ; NOT-NEXT:    >>>>>>
 ; NOT-NOT:     {{.}}
 
 ; Again, but with a CHECK instead of EOF as search range end.
 
 ; RUN: echo 'CHECK: ain' >> %t.chk
 
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefix=NOT2
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -v 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=NOT2,NOT2-V
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -vv 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=NOT2,NOT2-V,NOT2-VV
 
 ; NOT2:         <<<<<<
 ; NOT2-NEXT:             1: hello
 ; NOT2-NEXT:             2: world
 ; NOT2-NEXT:    not:2       !~~~~
 ; NOT2-NEXT:             3: again
+; NOT2-V-NEXT:  check:3       ^~~
 ; NOT2-NEXT:    >>>>>>
 ; NOT2-NOT:     {{.}}
 
 ;--------------------------------------------------
-; CHECK-DAG
+; CHECK-DAG (also: matches in different order than directives)
 ;--------------------------------------------------
 
 ; Good match, discarded match plus good match, and no match.
 
 ; RUN: echo 'abc' > %t.in
 ; RUN: echo 'def' >> %t.in
 ; RUN: echo 'abc' >> %t.in
 
 ; RUN: echo 'CHECK-DAG: def' > %t.chk
 ; RUN: echo 'CHECK-DAG: abc' >> %t.chk
 ; RUN: echo 'CHECK-DAG: abc' >> %t.chk
 ; RUN: echo 'CHECK-DAG: def' >> %t.chk
 
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk 2>&1 \
-; RUN: | FileCheck -match-full-lines %s -check-prefixes=DAG
+; RUN: | FileCheck -match-full-lines %s -check-prefixes=DAG,DAG-Q
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -v 2>&1 \
-; RUN: | FileCheck -match-full-lines %s -check-prefixes=DAG
+; RUN: | FileCheck -match-full-lines %s -check-prefixes=DAG,DAG-V
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -vv 2>&1 \
-; RUN: | FileCheck -match-full-lines %s -check-prefixes=DAG
+; RUN: | FileCheck -match-full-lines %s -check-prefixes=DAG,DAG-V,DAG-VV
 
 ; DAG:         <<<<<<
-; DAG-NEXT:           1: abc
-; DAG-NEXT:           2: def
-; DAG-NEXT:           3: abc
-; DAG-NEXT:    dag:4     X~~
+; DAG-NEXT:             1: abc
+; DAG-V-NEXT:  dag:2       ^~~
+; DAG-NEXT:             2: def
+; DAG-V-NEXT:  dag:1       ^~~
+; DAG-NEXT:             3: abc
+; DAG-V-NEXT:  dag:3       ^~~
+; DAG-NEXT:    dag:4       X~~
 ; DAG-NEXT:    >>>>>>
 ; DAG-NOT:     {{.}}
 
 ;--------------------------------------------------
 ; CHECK-LABEL
+;
+; FIXME: Labels sometimes produce redundant diagnostics for good matches.
+; That bug is independent of but affects -dump-input.
 ;--------------------------------------------------
 
 ; Good match and no match.
 
 ; RUN: echo 'lab0' > %t.in
 ; RUN: echo 'foo' >> %t.in
 ; RUN: echo 'lab1' >> %t.in
 ; RUN: echo 'bar' >> %t.in
 
 ; RUN: echo 'CHECK-LABEL: lab0' > %t.chk
 ; RUN: echo 'CHECK: foo' >> %t.chk
 ; RUN: echo 'CHECK-LABEL: lab2' >> %t.chk
 
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=LAB
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -v 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=LAB,LAB-V
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -vv 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=LAB,LAB-V,LAB-VV
 
 ; LAB:         <<<<<<
 ; LAB-NEXT:               1: lab0
+; LAB-V-NEXT:  label:1'0     ^~~~
+; LAB-V-NEXT:  label:1'1     ^~~~
 ; LAB-NEXT:               2: foo
 ; LAB-NEXT:    label:3'0     X~~
 ; LAB-NEXT:               3: lab1
 ; LAB-NEXT:    label:3'0     ~~~~
 ; LAB-NEXT:    label:3'1     ?
 ; LAB-NEXT:               4: bar
 ; LAB-NEXT:    label:3'0     ~~~
 ; LAB-NEXT:    >>>>>>
 ; LAB-NOT:     {{.}}
 
 

llvm/test/FileCheck/dump-input-enable.txt

 ; RUN:               -match-full-lines -dump-input=always 2>&1 \
 ; RUN: | FileCheck %s -match-full-lines -check-prefix=CHECK-ERR
 
 ; END.
 
 ; CHECK-GOOD: Full input was:
 ; CHECK-GOOD-NEXT: <<<<<<
 ; CHECK-GOOD-NEXT:          1: ciao
+; CHECK-GOOD-NEXT: check:1     ^~~~
 ; CHECK-GOOD-NEXT:          2: world
+; CHECK-GOOD-NEXT: next:2      ^~~~~
 ; CHECK-GOOD-NEXT: >>>>>>
 
 ; CHECK-ERR: Full input was:
 ; CHECK-ERR-NEXT: <<<<<<
 ; CHECK-ERR-NEXT:          1: hello
 ; CHECK-ERR-NEXT: check:1     X~~~~
 ; CHECK-ERR-NEXT:          2: world
 ; CHECK-ERR-NEXT: check:1     ~~~~~
 ; CHECK-ERR-NEXT: >>>>>>
 
 ; CHECK-NODUMP-NOT: <<<<<<

llvm/utils/FileCheck/FileCheck.cpp

for (int I = 0; I < argc; I++)
     errs() << " " << argv[I];
   errs() << "\n";
 }
 
 struct MatchTypeStyle {
   char Mark;
   bool HasTildes;
   raw_ostream::Colors Color;
+  enum Verbosity { Quiet, Verbose, VerboseVerbose } RequiredVerbosity;
   const char *What;
   MatchTypeStyle(char Mark, bool HasTildes, raw_ostream::Colors Color,
-                 const char *What)
-      : Mark(Mark), HasTildes(HasTildes), Color(Color), What(What) {}
+                 Verbosity RequiredVerbosity, const char *What)
+      : Mark(Mark), HasTildes(HasTildes), Color(Color),
+        RequiredVerbosity(RequiredVerbosity), What(What) {}
 };
 
 static MatchTypeStyle GetMatchTypeStyle(unsigned MatchTy) {
   switch (MatchTy) {
+  case FileCheckDiag::MatchFinalAndExpected:
+    return MatchTypeStyle('^', true, raw_ostream::GREEN,
+                          MatchTypeStyle::Verbose,
+                          "the final match for an expected pattern (e.g., "
+                          "CHECK)");
   case FileCheckDiag::MatchFinalButExcluded:
-    return MatchTypeStyle('!', true, raw_ostream::RED,
+    return MatchTypeStyle('!', true, raw_ostream::RED, MatchTypeStyle::Quiet,
                           "the final match for an excluded pattern (e.g., "
                           "CHECK-NOT)");
   case FileCheckDiag::MatchFinalButIllegal:
-    return MatchTypeStyle('!', true, raw_ostream::RED,
+    return MatchTypeStyle('!', true, raw_ostream::RED, MatchTypeStyle::Quiet,
                           "the final but illegal match for an expected "
                           "pattern (e.g., CHECK-NEXT)");
   case FileCheckDiag::MatchNoneButExpected:
-    return MatchTypeStyle('X', true, raw_ostream::RED,
+    return MatchTypeStyle('X', true, raw_ostream::RED, MatchTypeStyle::Quiet,
                           "the search range for an unmatched expected "
                           "pattern (e.g., CHECK)");
   case FileCheckDiag::MatchFuzzy:
     return MatchTypeStyle('?', false, raw_ostream::MAGENTA,
+                          MatchTypeStyle::Quiet,
                           "a fuzzy match start for an otherwise unmatched "
                           "pattern");
   case FileCheckDiag::MatchTypeCount:
     llvm_unreachable_internal("unexpected match type");
   }
   llvm_unreachable_internal("unexpected match type");
 }
 
static void DumpInputAnnotationExplanation(raw_ostream &OS,
   OS << "  - ";
   WithColor(OS, raw_ostream::SAVEDCOLOR, true) << "T:L'N";
   OS << "  labels the Nth match result for a pattern of type T from line "
      << "L of\n"
      << "           the check file\n";
 
   // Markers on annotation lines.
   OS << "  - ";
+  WithColor(OS, raw_ostream::SAVEDCOLOR, true) << "^~~";
+  OS << "    marks good match (requires -v)\n"
+     << "  - ";
   WithColor(OS, raw_ostream::SAVEDCOLOR, true) << "!~~";
   OS << "    marks bad match\n"
      << "  - ";
   WithColor(OS, raw_ostream::SAVEDCOLOR, true) << "X~~";
   OS << "    marks search range when no match is found\n"
      << "  - ";
   WithColor(OS, raw_ostream::SAVEDCOLOR, true) << "?";
   OS << "      marks fuzzy match when no match is found\n";
 
   // Colors.
   if (WithColor(OS).colorsEnabled()) {
     OS << "  - color  ";
+    if (Req.Verbose) {
+      WithColor(OS, raw_ostream::GREEN, true) << "success";
+      OS << ", ";
+    }
     WithColor(OS, raw_ostream::RED, true) << "error";
+    if (Req.Verbose) {
+      OS << ", ";
+      WithColor(OS, raw_ostream::CYAN, true, true) << "unmatched";
+    }
     OS << ", ";
     WithColor(OS, raw_ostream::MAGENTA, true) << "fuzzy";
     OS << '\n';
   }
 
   OS << "\nDetailed description of currently enabled markers:\n\n";
 
+  MatchTypeStyle::Verbosity Verbosity =
+      Req.VerboseVerbose
+          ? MatchTypeStyle::VerboseVerbose
+          : Req.Verbose ? MatchTypeStyle::Verbose : MatchTypeStyle::Quiet;
   for (unsigned StyleIdx = FileCheckDiag::MatchTypeFirst;
        StyleIdx < FileCheckDiag::MatchTypeCount; ++StyleIdx) {
     MatchTypeStyle Style = GetMatchTypeStyle(StyleIdx);
+    if (Verbosity < Style.RequiredVerbosity)
+      continue;
     if (StyleIdx == FileCheckDiag::MatchTypeFirst ||
+        Verbosity < GetMatchTypeStyle(StyleIdx - 1).RequiredVerbosity ||
         Style.Mark != GetMatchTypeStyle(StyleIdx - 1).Mark ||
         Style.HasTildes != GetMatchTypeStyle(StyleIdx - 1).HasTildes ||
         (WithColor(OS).colorsEnabled() &&
          Style.Color != GetMatchTypeStyle(StyleIdx - 1).Color)) {
       OS << "  - ";
       WithColor(OS, Style.Color, true)
           << Style.Mark << (Style.HasTildes ? "~~" : "  ");
       OS << "    marks ";
       if (StyleIdx + 1 != FileCheckDiag::MatchTypeCount &&
+          Verbosity >= GetMatchTypeStyle(StyleIdx + 1).RequiredVerbosity &&
           Style.Mark == GetMatchTypeStyle(StyleIdx + 1).Mark &&
           Style.HasTildes == GetMatchTypeStyle(StyleIdx + 1).HasTildes &&
           (!WithColor(OS).colorsEnabled() ||
            Style.Color == GetMatchTypeStyle(StyleIdx + 1).Color))
         OS << "either:\n"
            << "           - ";
     } else
       OS << "           - ";
     OS << Style.What << "\n";
   }
 
+  if (WithColor(OS).colorsEnabled() && Req.Verbose) {
+    OS << "  - ";
+    WithColor(OS, raw_ostream::CYAN, true, true) << "input";
+    OS << "  is style of input text with no final match for any expected "
+       << "pattern\n";
+  }
+
   // Files.
   OS << "\nInput file: ";
   if (InputFilename == "-")
     OS << "<stdin>";
   else
     OS << InputFilename;
   OS << "\nCheck file: " << CheckFilename << "\n";
 }
struct InputAnnotation {
   /// multiple lines.
   unsigned InputLine;
   /// The column range (one-origin indexing, open end) in which to to mark the
   /// input line.  If InputEndCol is UINT_MAX, treat it as the last column
   /// before the newline.
   unsigned InputStartCol, InputEndCol;
   /// The starting char (before tildes) for marking the line.
   char Mark;
+  /// Whether this annotation represents a final match for an expected pattern.
+  bool FinalAndExpectedMatch;
   /// What color to use for this annotation.
   raw_ostream::Colors Color;
 };
 
 /// Get an abbreviation for the check type.
 std::string GetCheckTypeAbbreviation(Check::FileCheckType Ty) {
   switch (Ty) {
   case Check::CheckPlain:
for (auto DiagItr = DiagList.begin(), DiagEnd = DiagList.end();
     else
       A.CheckDiagIndex = 0;
     Label.flush();
     LabelWidth = std::max((std::string::size_type)LabelWidth, A.Label.size());
 
     MatchTypeStyle MatchTyStyle = GetMatchTypeStyle(DiagItr->MatchTy);
     A.Mark = MatchTyStyle.Mark;
     A.Color = MatchTyStyle.Color;
+    A.FinalAndExpectedMatch =
+        DiagItr->MatchTy == FileCheckDiag::MatchFinalAndExpected;
 
     // Compute the mark location, and break annotation into multiple
     // annotations if it spans multiple lines.
     A.InputLine = DiagItr->InputStartLine;
     A.InputStartCol = DiagItr->InputStartCol;
     if (DiagItr->InputStartLine == DiagItr->InputEndLine) {
       // Sometimes ranges are empty in order to indicate a specific point, but
       // that would mean nothing would be marked, so adjust the range to
if (DiagItr->InputStartLine == DiagItr->InputEndLine) {
         B.Label = A.Label;
         B.InputLine = L;
         B.Mark = '~';
         B.InputStartCol = 1;
         if (L != E)
           B.InputEndCol = UINT_MAX;
         else
           B.InputEndCol = DiagItr->InputEndCol;
+        B.FinalAndExpectedMatch = A.FinalAndExpectedMatch;
         B.Color = A.Color;
       }
     }
   }
 }
 
 static void DumpAnnotatedInput(
-    raw_ostream &OS, StringRef InputFileText,
+    raw_ostream &OS, const FileCheckRequest &Req, StringRef InputFileText,
     const std::list<InputAnnotation> &AnnotationList, unsigned LabelWidth) {
   OS << "Full input was:\n<<<<<<\n";
 
   // Sort annotations.
   //
   // First, sort in the order of input lines to make it easier to find relevant
   // annotations while iterating input lines in the implementation below.
   // FileCheck diagnostics are not always reported and recorded in the order of
static void DumpAnnotatedInput(
   std::vector<InputAnnotation> Annotations(AnnotationList.begin(),
                                            AnnotationList.end());
   std::sort(Annotations.begin(), Annotations.end(),
             [](const InputAnnotation &A, const InputAnnotation &B) {
               if (A.InputLine != B.InputLine)
                 return A.InputLine < B.InputLine;
               if (A.CheckLine != B.CheckLine)
                 return A.CheckLine < B.CheckLine;
-              assert(A.CheckDiagIndex != B.CheckDiagIndex &&
-                     "expected diagnostic indices to be unique within a "
-                     " check line");
+              // FIXME: Sometimes CHECK-LABEL reports its match twice with
+              // other diagnostics in between, and then diag index incrementing
+              // fails to work properly, and then this assert fails.  We should
+              // suppress one of those diagnostics or do a better job of
+              // computing this index.  For now, we just produce a redundant
+              // CHECK-LABEL annotation.
+              // assert(A.CheckDiagIndex != B.CheckDiagIndex &&
+              //        "expected diagnostic indices to be unique within a "
+              //        " check line");
               return A.CheckDiagIndex < B.CheckDiagIndex;
             });
 
   // Compute the width of the label column.
   const unsigned char *InputFilePtr = InputFileText.bytes_begin(),
                       *InputFileEnd = InputFileText.bytes_end();
   unsigned LineCount = InputFileText.count('\n');
   if (InputFileEnd[-1] != '\n')
for (unsigned Line = 1;
        InputFilePtr != InputFileEnd || AnnotationItr != AnnotationEnd;
        ++Line) {
     const unsigned char *InputFileLine = InputFilePtr;
 
     // Print right-aligned line number.
     WithColor(OS, raw_ostream::BLACK, true)
         << format_decimal(Line, LabelWidth) << ": ";
 
-    // Print numbered line.
+    // For case where -v and colors are enabled, find the annotations for final
+    // matches for expected patterns in order to highlight everything else in
+    // the line.  There are no such annotations if -v is disabled.
+    std::list<InputAnnotation> FinalAndExpectedMatches;
+    if (Req.Verbose && WithColor(OS).colorsEnabled()) {
+      for (auto I = AnnotationItr; I != AnnotationEnd && I->InputLine == Line;
+           ++I) {
+        if (I->FinalAndExpectedMatch)
+          FinalAndExpectedMatches.push_back(*I);
+      }
+    }
+
+    // Print numbered line with highlighting where there are no matches for
+    // expected patterns.
     bool Newline = false;
-    while (InputFilePtr != InputFileEnd && !Newline) {
+    {
+      WithColor COS(OS);
+      if (Req.Verbose)
+        COS.changeColor(raw_ostream::CYAN, true, true);
+      for (unsigned Col = 1; InputFilePtr != InputFileEnd && !Newline; ++Col) {
+        bool StartsMatch = false;
+        bool EndsMatch = false;
+        for (auto M : FinalAndExpectedMatches) {
+          if (Col == M.InputEndCol)
+            EndsMatch = true;
+          else if (Col == M.InputStartCol)
+            StartsMatch = true;
+        }
+        if (StartsMatch)
+          COS.resetColor();
+        else if (EndsMatch)
+          COS.changeColor(raw_ostream::CYAN, true, true);
         if (*InputFilePtr == '\n')
           Newline = true;
         else
-        OS << *InputFilePtr;
+          COS << *InputFilePtr;
         ++InputFilePtr;
       }
+    }
     OS << '\n';
     unsigned InputLineWidth = InputFilePtr - InputFileLine - Newline;
 
     // Print any annotations.
     while (AnnotationItr != AnnotationEnd &&
            AnnotationItr->InputLine == Line) {
       WithColor COS(OS, AnnotationItr->Color, true);
       // The two spaces below are where the ": " appears on input lines.
if (ExitCode == 1 && DumpInputOnFailure)
     errs() << "Full input was:\n<<<<<<\n" << InputFileText << "\n>>>>>>\n";
   if (DumpInput == "always" || (ExitCode == 1 && DumpInput == "fail")) {
     errs() << '\n';
     DumpInputAnnotationExplanation(errs(), Req);
     std::list<InputAnnotation> AnnotationList;
     unsigned LabelWidth;
     BuildInputAnnotations(DiagList, AnnotationList, LabelWidth);
     errs() << '\n';
-    DumpAnnotatedInput(errs(), InputFileText, AnnotationList, LabelWidth);
+    DumpAnnotatedInput(errs(), Req, InputFileText, AnnotationList, LabelWidth);
   }
 
   return ExitCode;
 }