[FileCheck] Annotate input dump (5/7)

[jdenny]

Summary

This patch implements input annotations for diagnostics enabled by -v,
which report good matches for directives. These annotations mark
match ranges using ^~~.

For example:

$ FileCheck -dump-input=help
The following description was requested by -dump-input=help to
explain the input annotations printed by -dump-input=always and
-dump-input=fail:

  - L:     labels line number L of the input file
  - T:L    labels the only match result for a pattern of type T from line L of
           the check file
  - T:L'N  labels the Nth match result for a pattern of type T from line L of
           the check file
  - ^~~    marks good match (reported if -v)
  - !~~    marks bad match, such as:
           - CHECK-NEXT on same line as previous match (error)
           - CHECK-NOT found (error)
  - X~~    marks search range when no match is found, such as:
           - CHECK-NEXT not found (error)
  - ?      marks fuzzy match when no match is found
  - colors success, error, fuzzy match, unmatched input

If you are not seeing color above or in input dumps, try: -color

$ FileCheck -v -dump-input=always check3 < input3 |& sed -n '/^<<<</,$p'
<<<<<<
         1: abc foobar def
check:1     ^~~
not:2           !~~~~~     error: no match expected
check:3                ^~~
>>>>>>

$ cat check3
CHECK:     abc
CHECK-NOT: foobar
CHECK:     def

$ cat input3
abc foobar def

-vv enables these annotations for FileCheck's implicit EOF patterns as
well. For an example where EOF patterns become relevant, see patch 7
in this series.

If colors are enabled, ^~~ is green to suggest success.

-v plus color enables highlighting of input text that has no final
match for any expected pattern. The highlight uses a cyan background
to suggest a cold section. This highlighting can make it easier to
spot text that was intended to be matched but that failed to be
matched in a long series of good matches.

CHECK-COUNT-<num> good matches are another case where there can be
multiple match results for the same directive.

[jdenny]

Rebased onto updated patches earlier in series.

[jdenny]

Update for changes earlier in patch series.

[jdenny]

Propagate changes started earlier in patch series.

[jdenny]

Fix bug in logic for highlighting unmatched text: it was broken for -allow-deprecated-dag-overlap.

[jdenny]

Continue changes started earlier in patch series.

[jdenny]

llvm/utils/FileCheck/FileCheck.cpp

441 ↗

(On Diff #177385)

A break statement could be here.

[jdenny]

• Add a break statement to skip useless loop iterations.
• Convert a list to a vector.

[jdenny]

Adjust comment, as suggested by probinson.

[probinson]

The terminology around "possible" "expected" and "final" matches probably warrants a few lines of commentary somewhere, unless I missed it in an earlier patch. There are getting to be enough cases that the casual reader (ahem) can't just Zen their way through them all.
Can be done as a follow-up if you prefer.
One comment nit and LGTM.

llvm/utils/FileCheck/FileCheck.cpp

418 ↗

(On Diff #177942)

For the case...

[jdenny]

In D53897#1331556, @probinson wrote:

The terminology around "possible" "expected" and "final" matches probably warrants a few lines of commentary somewhere, unless I missed it in an earlier patch. There are getting to be enough cases that the casual reader (ahem) can't just Zen their way through them all.

Sure, I'll add some additional comments here.

Can be done as a follow-up if you prefer.

Yes, a follow-up would be easier. I'll just commit something. If you prefer a phab review first, that's absolutely fine for me, but let me know.

[probinson]

In D53897#1331582, @jdenny wrote:

Yes, a follow-up would be easier. I'll just commit something. If you prefer a phab review first, that's absolutely fine for me, but let me know.

No, just do it.

[jdenny]

Remaining suggestions done in D55738.

llvm/trunk/include/llvm/Support/FileCheck.h

struct FileCheckDiag {
   /// Where is the FileCheck directive for this diagnostic?
   unsigned CheckLine, CheckCol;
   /// What kind of match result does this diagnostic describe?
   ///
   /// There might be more than one of these for the same directive.  For
   /// example, there might be a fuzzy match after a fail.
   enum MatchType {
     // TODO: More members will appear with later patches in this series.
+    /// Indicates the final match for an expected pattern.
+    MatchFinalAndExpected,
     /// Indicates the final match for an excluded pattern.
     MatchFinalButExcluded,
     /// Indicates the final match for an expected pattern, but the match is on
     /// the wrong line.
     MatchFinalButWrongLine,
     /// Indicates no match for an expected pattern.
     MatchNoneButExpected,
     /// Indicates a possible intended match because there's no perfect match.

llvm/trunk/lib/Support/FileCheck.cpp

if (!VariableUses.empty()) {
     }
   }
 }
 
 static SMRange ProcessMatchResult(FileCheckDiag::MatchType MatchTy,
                                   const SourceMgr &SM, SMLoc Loc,
                                   Check::FileCheckType CheckTy,
                                   StringRef Buffer, size_t Pos, size_t Len,
-                                  std::vector<FileCheckDiag> *Diags) {
+                                  std::vector<FileCheckDiag> *Diags,
+                                  bool AdjustPrevDiag = false) {
   SMLoc Start = SMLoc::getFromPointer(Buffer.data() + Pos);
   SMLoc End = SMLoc::getFromPointer(Buffer.data() + Pos + Len);
   SMRange Range(Start, End);
   // TODO: The second condition will disappear when we extend this to handle
   // more match types.
-  if (Diags && MatchTy != FileCheckDiag::MatchTypeCount)
+  if (Diags && MatchTy != FileCheckDiag::MatchTypeCount) {
+    if (AdjustPrevDiag)
+      Diags->rbegin()->MatchTy = MatchTy;
+    else
       Diags->emplace_back(SM, CheckTy, Loc, MatchTy, Range);
+  }
   return Range;
 }
 
 void FileCheckPattern::PrintFuzzyMatch(
     const SourceMgr &SM, StringRef Buffer,
     const StringMap<StringRef> &VariableTable,
     std::vector<FileCheckDiag> *Diags) const {
   // Attempt to find the closest/best fuzzy match.  Usually an error happens
static void PrintMatch(bool ExpectedMatch, const SourceMgr &SM,
                        std::vector<FileCheckDiag> *Diags) {
   if (ExpectedMatch) {
     if (!Req.Verbose)
       return;
     if (!Req.VerboseVerbose && Pat.getCheckTy() == Check::CheckEOF)
       return;
   }
   SMRange MatchRange = ProcessMatchResult(
-      ExpectedMatch ? FileCheckDiag::MatchTypeCount
+      ExpectedMatch ? FileCheckDiag::MatchFinalAndExpected
                     : FileCheckDiag::MatchFinalButExcluded,
       SM, Loc, Pat.getCheckTy(), Buffer, MatchPos, MatchLen, Diags);
   std::string Message = formatv("{0}: {1} string found in input",
                                 Pat.getCheckTy().getDescription(Prefix),
                                 (ExpectedMatch ? "expected" : "excluded"))
                             .str();
   if (Pat.getCount() > 1)
     Message += formatv(" ({0} out of {1})", MatchedCount, Pat.getCount()).str();
if (!IsLabelScanMode) {
     StringRef MatchBuffer = Buffer.substr(LastPos);
     StringRef SkippedRegion = Buffer.substr(LastPos, MatchPos);
 
     // If this check is a "CHECK-NEXT", verify that the previous match was on
     // the previous line (i.e. that there is one newline between them).
     if (CheckNext(SM, SkippedRegion)) {
       ProcessMatchResult(FileCheckDiag::MatchFinalButWrongLine, SM, Loc,
                          Pat.getCheckTy(), MatchBuffer, MatchPos, MatchLen,
-                         Diags);
+                         Diags, Req.Verbose);
       return StringRef::npos;
     }
 
     // If this check is a "CHECK-SAME", verify that the previous match was on
     // the same line (i.e. that there is no newline between them).
     if (CheckSame(SM, SkippedRegion)) {
       ProcessMatchResult(FileCheckDiag::MatchFinalButWrongLine, SM, Loc,
                          Pat.getCheckTy(), MatchBuffer, MatchPos, MatchLen,
-                         Diags);
+                         Diags, Req.Verbose);
       return StringRef::npos;
     }
 
     // If this match had "not strings", verify that they don't exist in the
     // skipped region.
     if (CheckNot(SM, SkippedRegion, NotStrings, VariableTable, Req, Diags))
       return StringRef::npos;
   }
for (auto MI = MatchRanges.begin(), ME = MatchRanges.end(); true; ++MI) {
       }
       if (Req.VerboseVerbose) {
         SMLoc OldStart = SMLoc::getFromPointer(Buffer.data() + MI->Pos);
         SMLoc OldEnd = SMLoc::getFromPointer(Buffer.data() + MI->End);
         SMRange OldRange(OldStart, OldEnd);
         SM.PrintMessage(OldStart, SourceMgr::DK_Note,
                         "match discarded, overlaps earlier DAG match here",
                         {OldRange});
+        if (Diags)
+          Diags->pop_back();
       }
       MatchPos = MI->End;
     }
     if (!Req.VerboseVerbose)
       PrintMatch(true, SM, Prefix, Pat.getLoc(), Pat, 1, Buffer, VariableTable,
                  MatchPos, MatchLen, Req, Diags);
 
     // Handle the end of a CHECK-DAG group.

llvm/trunk/test/FileCheck/dump-input-annotations.txt

 ; RUN: echo 'CHECK: universe' >> %t.chk
 
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -v 2>&1 \
 ; RUN: | FileCheck -strict-whitespace -match-full-lines -check-prefix=ALIGN %s
 
 ; ALIGN:Full input was:
 ; ALIGN-NEXT:<<<<<<
 ; ALIGN-NEXT:           1: hello world
+; ALIGN-NEXT:check:1       ^~~~~
 ; ALIGN-NEXT:check:2'0           X~~~~
 ; ALIGN-NEXT:           2: goodbye
 ; ALIGN-NEXT:check:2'0     ~~~~~~~
 ; ALIGN-NEXT:           3: world
 ; ALIGN-NEXT:check:2'0     ~~~~~
 ; ALIGN-NEXT:           4: unicorn
 ; ALIGN-NEXT:check:2'0     ~~~~~~~ error: no match found
 ; ALIGN-NEXT:check:2'1     ?       possible intended match

 ; RUN: | FileCheck -match-full-lines %s -check-prefix=CHK
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -v 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=CHK,CHK-V
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -vv 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=CHK,CHK-V
 
 ; CHK:        <<<<<<
 ; CHK-NEXT:              1: hello
+; CHK-V-NEXT: check:1       ^~~~~
 ; CHK-NEXT:              2: again
 ; CHK-NEXT:   check:2'0     X~~~~
 ; CHK-NEXT:              3: whirled
 ; CHK-NEXT:   check:2'0     ~~~~~~~ error: no match found
 ; CHK-NEXT:   check:2'1     ?       possible intended match
 ; CHK-NEXT:   >>>>>>
 ; CHK-NOT:    {{.}}
 
 ;--------------------------------------------------
 ; CHECK-COUNT-<num>
 ;--------------------------------------------------
 
 ; Good match and no match.
 
 ; RUN: echo 'pete'   > %t.in
 ; RUN: echo 'repete' >> %t.in
 ; RUN: echo 'repeat' >> %t.in
 
 ; RUN: echo 'CHECK-COUNT-3: pete' > %t.chk
 
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk 2>&1 \
-; RUN: | FileCheck -match-full-lines %s -check-prefix=CNT
+; RUN: | FileCheck -match-full-lines %s -check-prefixes=CNT,CNT-Q
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -v 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=CNT,CNT-V
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -vv 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=CNT,CNT-V
 
-; CNT:        <<<<<<
-; CNT-NEXT:            1: pete
-; CNT-NEXT:            2: repete
-; CNT-NEXT:            3: repeat
-; CNT-NEXT:   count:1     X~~~~~ error: no match found
-; CNT-NEXT:   >>>>>>
-; CNT-NOT:    {{.}}
+; CNT:          <<<<<<
+; CNT-NEXT:                1: pete
+; CNT-V-NEXT:   count:1'0     ^~~~
+; CNT-NEXT:                2: repete
+; CNT-V-NEXT:   count:1'1       ^~~~
+; CNT-NEXT:                3: repeat
+; CNT-Q-NEXT:   count:1       X~~~~~ error: no match found
+; CNT-V-NEXT:   count:1'2     X~~~~~ error: no match found
+; CNT-NEXT:     >>>>>>
+; CNT-NOT:      {{.}}
 
 ;--------------------------------------------------
 ; CHECK-NEXT (also: EOF search-range, wrong-line match)
 ;--------------------------------------------------
 
 ; Good match and no match.
 
 ; RUN: echo 'hello' > %t.in
 ; RUN: echo 'again' >> %t.in
 
 ; RUN: echo 'CHECK: hello' > %t.chk
 ; RUN: echo 'CHECK-NEXT: again' >> %t.chk
 ; RUN: echo 'CHECK-NEXT: world' >> %t.chk
 
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefix=NXT
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -v 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=NXT,NXT-V
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -vv 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=NXT,NXT-V,NXT-VV
 
 ; NXT:        <<<<<<
-; NXT-NEXT:           1: hello
-; NXT-NEXT:           2: again
-; NXT-NEXT:           3:
-; NXT-NEXT:   next:3     X error: no match found
+; NXT-NEXT:            1: hello
+; NXT-V-NEXT: check:1     ^~~~~
+; NXT-NEXT:            2: again
+; NXT-V-NEXT: next:2      ^~~~~
+; NXT-NEXT:            3:
+; NXT-NEXT:   next:3      X error: no match found
 ; NXT-NEXT:   >>>>>>
 ; NXT-NOT:    {{.}}
 
 ; Wrong-line match.
 
 ; RUN: echo 'yonder' >> %t.in
 ; RUN: echo 'world' >> %t.in
 
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefix=NXT2
 
 ; NXT2:      <<<<<<
 ; NXT2-NEXT:         1: hello
 ; NXT2-NEXT:         2: again
 ; NXT2-NEXT:         3: yonder
 ; NXT2-NEXT:         4: world
 ; NXT2-NEXT: next:3     !~~~~ error: match on wrong line
 ; NXT2-NEXT: >>>>>>
 ; NXT2-NOT:  {{.}}
 
 ;--------------------------------------------------
-; CHECK-SAME (also: single-char search range, wrong-line match)
+; CHECK-SAME (also: multiple annotations per line, single-char search range,
+; wrong-line match)
 ;--------------------------------------------------
 
 ; Good match and no match.
 
 ; RUN: echo 'hello world!' > %t.in
 
 ; RUN: echo 'CHECK: hello' > %t.chk
 ; RUN: echo 'CHECK-SAME: world' >> %t.chk
 ; RUN: echo 'CHECK-SAME: again' >> %t.chk
 
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefix=SAM
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -v 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=SAM,SAM-V
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -vv 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=SAM,SAM-V,SAM-VV
 
 ; SAM:        <<<<<<
-; SAM-NEXT:           1: hello world!
-; SAM-NEXT:   same:3                X error: no match found
+; SAM-NEXT:            1: hello world!
+; SAM-V-NEXT: check:1     ^~~~~
+; SAM-V-NEXT: same:2            ^~~~~
+; SAM-NEXT:   same:3                 X error: no match found
 ; SAM-NEXT:   >>>>>>
 ; SAM-NOT:    {{.}}
 
 ; Wrong-line match.
 
 ; RUN: echo 'again' >> %t.in
 
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -v 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=SAM2
 
 ; SAM2:      <<<<<<
 ; SAM2-NEXT:          1: hello world!
+; SAM2-NEXT: check:1     ^~~~~
+; SAM2-NEXT: same:2            ^~~~~
 ; SAM2-NEXT:          2: again
 ; SAM2-NEXT: same:3      !~~~~ error: match on wrong line
 ; SAM2-NEXT: >>>>>>
 ; SAM2-NOT:  {{.}}
 
 ;--------------------------------------------------
-; CHECK-EMPTY (also: search range ends at label, wrong-line match)
+; CHECK-EMPTY (also: search range ends at label, single-char match, wrong-line
+; match)
 ;--------------------------------------------------
 
 ; Good match and no match.
 ;
 ; CHECK-EMPTY always seems to match an empty line at EOF (illegally when it's
 ; not the next line) unless either (1) the last line is non-empty and has no
 ; newline or (2) there's a CHECK-LABEL to end the search range before EOF.  We
 ; choose scenario 2 to check the case of no match.

 ; RUN: | FileCheck -match-full-lines %s -check-prefix=EMP
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -v 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=EMP,EMP-V
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -vv 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=EMP,EMP-V,EMP-VV
 
 ; EMP:        <<<<<<
 ; EMP-NEXT:            1: hello
+; EMP-V-NEXT: check:1     ^~~~~
 ; EMP-NEXT:            2:
+; EMP-V-NEXT: empty:2     ^
 ; EMP-NEXT:            3: world
 ; EMP-NEXT:   empty:3     X~~~~
 ; EMP-NEXT:            4: label
 ; EMP-NEXT:   empty:3     ~~~~~ error: no match found
+; EMP-V-NEXT: label:4     ^~~~~
 ; EMP-NEXT:   >>>>>>
 ; EMP-NOT:    {{.}}
 
 ; Wrong-line match.
 
 ; RUN: echo 'hello' > %t.in
 ; RUN: echo 'world' >> %t.in
 
 ; RUN: echo 'CHECK: hello' > %t.chk
 ; RUN: echo 'CHECK-EMPTY:' >> %t.chk
 
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefix=EMP2
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -v 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=EMP2,EMP2-V
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -vv 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=EMP2,EMP2-V,EMP2-VV
 
 ; EMP2:        <<<<<<
 ; EMP2-NEXT:            1: hello
+; EMP2-V-NEXT: check:1     ^~~~~
 ; EMP2-NEXT:            2: world
 ; EMP2-NEXT:            3:
 ; EMP2-NEXT:   empty:2     !     error: match on wrong line
 ; EMP2-NEXT:   >>>>>>
 ; EMP2-NOT:    {{.}}
 
 ;--------------------------------------------------
-; CHECK-NOT
+; CHECK-NOT (also: EOF pattern)
 ;--------------------------------------------------
 
 ; No match (success) and unexpected match (error).
 
 ; RUN: echo 'hello' > %t.in
 ; RUN: echo 'world' >> %t.in
 ; RUN: echo 'again' >> %t.in
 
 ; RUN: echo 'CHECK-NOT: goodbye' > %t.chk
 ; RUN: echo 'CHECK-NOT: world' >> %t.chk
 
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefix=NOT
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -v 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=NOT,NOT-V
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -vv 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=NOT,NOT-V,NOT-VV
 
 ; NOT:         <<<<<<
 ; NOT-NEXT:           1: hello
 ; NOT-NEXT:           2: world
 ; NOT-NEXT:    not:2     !~~~~ error: no match expected
 ; NOT-NEXT:           3: again
+; NOT-VV-NEXT:        4:
+; NOT-VV-NEXT: eof:2     ^
 ; NOT-NEXT:    >>>>>>
 ; NOT-NOT:     {{.}}
 
 ; Again, but with a CHECK instead of EOF as search range end.
 
 ; RUN: echo 'CHECK: ain' >> %t.chk
 
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefix=NOT2
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -v 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=NOT2,NOT2-V
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -vv 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=NOT2,NOT2-V,NOT2-VV
 
 ; NOT2:         <<<<<<
 ; NOT2-NEXT:             1: hello
 ; NOT2-NEXT:             2: world
 ; NOT2-NEXT:    not:2       !~~~~ error: no match expected
 ; NOT2-NEXT:             3: again
+; NOT2-V-NEXT:  check:3       ^~~
 ; NOT2-NEXT:    >>>>>>
 ; NOT2-NOT:     {{.}}
 
 ;--------------------------------------------------
-; CHECK-DAG
+; CHECK-DAG (also: matches in different order than directives)
 ;--------------------------------------------------
 
 ; Good match, discarded match plus good match, and no match.
 
 ; RUN: echo 'abc' > %t.in
 ; RUN: echo 'def' >> %t.in
 ; RUN: echo 'abc' >> %t.in
 
 ; RUN: echo 'CHECK-DAG: def' > %t.chk
 ; RUN: echo 'CHECK-DAG: abc' >> %t.chk
 ; RUN: echo 'CHECK-DAG: abc' >> %t.chk
 ; RUN: echo 'CHECK-DAG: def' >> %t.chk
 
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk 2>&1 \
-; RUN: | FileCheck -match-full-lines %s -check-prefixes=DAG
+; RUN: | FileCheck -match-full-lines %s -check-prefixes=DAG,DAG-Q
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -v 2>&1 \
-; RUN: | FileCheck -match-full-lines %s -check-prefixes=DAG
+; RUN: | FileCheck -match-full-lines %s -check-prefixes=DAG,DAG-V
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -vv 2>&1 \
-; RUN: | FileCheck -match-full-lines %s -check-prefixes=DAG
+; RUN: | FileCheck -match-full-lines %s -check-prefixes=DAG,DAG-V,DAG-VV
 
 ; DAG:         <<<<<<
-; DAG-NEXT:           1: abc
-; DAG-NEXT:           2: def
-; DAG-NEXT:           3: abc
-; DAG-NEXT:    dag:4     X~~ error: no match found
+; DAG-NEXT:             1: abc
+; DAG-V-NEXT:  dag:2       ^~~
+; DAG-NEXT:             2: def
+; DAG-V-NEXT:  dag:1       ^~~
+; DAG-NEXT:             3: abc
+; DAG-V-NEXT:  dag:3       ^~~
+; DAG-NEXT:    dag:4       X~~ error: no match found
 ; DAG-NEXT:    >>>>>>
 ; DAG-NOT:     {{.}}
 
 ;--------------------------------------------------
 ; CHECK-LABEL
+;
+; FIXME: Labels sometimes produce redundant diagnostics for good matches.
+; That bug is independent of but affects -dump-input.
 ;--------------------------------------------------
 
 ; Good match and no match.
 
 ; RUN: echo 'lab0' > %t.in
 ; RUN: echo 'foo' >> %t.in
 ; RUN: echo 'lab1' >> %t.in
 ; RUN: echo 'bar' >> %t.in
 
 ; RUN: echo 'CHECK-LABEL: lab0' > %t.chk
 ; RUN: echo 'CHECK: foo' >> %t.chk
 ; RUN: echo 'CHECK-LABEL: lab2' >> %t.chk
 
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=LAB
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -v 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=LAB,LAB-V
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -vv 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=LAB,LAB-V,LAB-VV
 
 ; LAB:         <<<<<<
 ; LAB-NEXT:               1: lab0
+; LAB-V-NEXT:  label:1'0     ^~~~
+; LAB-V-NEXT:  label:1'1     ^~~~
 ; LAB-NEXT:               2: foo
 ; LAB-NEXT:    label:3'0     X~~
 ; LAB-NEXT:               3: lab1
 ; LAB-NEXT:    label:3'0     ~~~~
 ; LAB-NEXT:    label:3'1     ?    possible intended match
 ; LAB-NEXT:               4: bar
 ; LAB-NEXT:    label:3'0     ~~~ error: no match found
 ; LAB-NEXT:    >>>>>>
 ; LAB-NOT:     {{.}}
 
 

llvm/trunk/test/FileCheck/dump-input-enable.txt

 ; RUN:               -match-full-lines -dump-input=always 2>&1 \
 ; RUN: | FileCheck %s -match-full-lines -check-prefix=CHECK-ERR
 
 ; END.
 
 ; CHECK-GOOD: Full input was:
 ; CHECK-GOOD-NEXT: <<<<<<
 ; CHECK-GOOD-NEXT:          1: ciao
+; CHECK-GOOD-NEXT: check:1     ^~~~
 ; CHECK-GOOD-NEXT:          2: world
+; CHECK-GOOD-NEXT: next:2      ^~~~~
 ; CHECK-GOOD-NEXT: >>>>>>
 
 ; CHECK-ERR: Full input was:
 ; CHECK-ERR-NEXT: <<<<<<
 ; CHECK-ERR-NEXT:          1: hello
 ; CHECK-ERR-NEXT: check:1     X~~~~
 ; CHECK-ERR-NEXT:          2: world
 ; CHECK-ERR-NEXT: check:1     ~~~~~ error: no match found
 ; CHECK-ERR-NEXT: >>>>>>
 
 ; CHECK-NODUMP-NOT: <<<<<<

llvm/trunk/utils/FileCheck/FileCheck.cpp

 struct MarkerStyle {
   /// The starting char (before tildes) for marking the line.
   char Lead;
   /// What color to use for this annotation.
   raw_ostream::Colors Color;
   /// A note to follow the marker, or empty string if none.
   std::string Note;
   MarkerStyle() {}
-  MarkerStyle(char Lead, raw_ostream::Colors Color, const std::string &Note)
+  MarkerStyle(char Lead, raw_ostream::Colors Color,
+              const std::string &Note = "")
       : Lead(Lead), Color(Color), Note(Note) {}
 };
 
 static MarkerStyle GetMarker(FileCheckDiag::MatchType MatchTy) {
   switch (MatchTy) {
+  case FileCheckDiag::MatchFinalAndExpected:
+    return MarkerStyle('^', raw_ostream::GREEN);
   case FileCheckDiag::MatchFinalButExcluded:
     return MarkerStyle('!', raw_ostream::RED, "error: no match expected");
   case FileCheckDiag::MatchFinalButWrongLine:
     return MarkerStyle('!', raw_ostream::RED, "error: match on wrong line");
   case FileCheckDiag::MatchNoneButExpected:
     return MarkerStyle('X', raw_ostream::RED, "error: no match found");
   case FileCheckDiag::MatchFuzzy:
     return MarkerStyle('?', raw_ostream::MAGENTA, "possible intended match");
static void DumpInputAnnotationHelp(raw_ostream &OS) {
   OS << "  - ";
   WithColor(OS, raw_ostream::SAVEDCOLOR, true) << "T:L'N";
   OS << "  labels the Nth match result for a pattern of type T from line "
      << "L of\n"
      << "           the check file\n";
 
   // Markers on annotation lines.
   OS << "  - ";
+  WithColor(OS, raw_ostream::SAVEDCOLOR, true) << "^~~";
+  OS << "    marks good match (reported if -v)\n"
+     << "  - ";
   WithColor(OS, raw_ostream::SAVEDCOLOR, true) << "!~~";
   OS << "    marks bad match, such as:\n"
      << "           - CHECK-NEXT on same line as previous match (error)\n"
      << "           - CHECK-NOT found (error)\n"
      << "  - ";
   WithColor(OS, raw_ostream::SAVEDCOLOR, true) << "X~~";
   OS << "    marks search range when no match is found, such as:\n"
      << "           - CHECK-NEXT not found (error)\n"
      << "  - ";
   WithColor(OS, raw_ostream::SAVEDCOLOR, true) << "?";
   OS << "      marks fuzzy match when no match is found\n";
 
   // Colors.
   OS << "  - colors ";
+  WithColor(OS, raw_ostream::GREEN, true) << "success";
+  OS << ", ";
   WithColor(OS, raw_ostream::RED, true) << "error";
   OS << ", ";
   WithColor(OS, raw_ostream::MAGENTA, true) << "fuzzy match";
+  OS << ", ";
+  WithColor(OS, raw_ostream::CYAN, true, true) << "unmatched input";
   OS << "\n\n"
      << "If you are not seeing color above or in input dumps, try: -color\n";
 }
 
 /// An annotation for a single input line.
 struct InputAnnotation {
   /// The check file line (one-origin indexing) where the directive that
   /// produced this annotation is located.
   unsigned CheckLine;
   /// The index of the match result for this check.
   unsigned CheckDiagIndex;
   /// The label for this annotation.
   std::string Label;
   /// What input line (one-origin indexing) this annotation marks.  This might
   /// be different from the starting line of the original diagnostic if this is
   /// a non-initial fragment of a diagnostic that has been broken across
   /// multiple lines.
   unsigned InputLine;
   /// The column range (one-origin indexing, open end) in which to to mark the
   /// input line.  If InputEndCol is UINT_MAX, treat it as the last column
   /// before the newline.
   unsigned InputStartCol, InputEndCol;
   /// The marker to use.
   MarkerStyle Marker;
+  /// Whether this annotation represents a final match for an expected pattern.
+  bool FinalAndExpectedMatch;
 };
 
 /// Get an abbreviation for the check type.
 std::string GetCheckTypeAbbreviation(Check::FileCheckType Ty) {
   switch (Ty) {
   case Check::CheckPlain:
     if (Ty.getCount() > 1)
       return "count";
if (A.CheckDiagIndex != UINT_MAX)
       Label << "'" << A.CheckDiagIndex;
     else
       A.CheckDiagIndex = 0;
     Label.flush();
     LabelWidth = std::max((std::string::size_type)LabelWidth, A.Label.size());
 
     MarkerStyle Marker = GetMarker(DiagItr->MatchTy);
     A.Marker = Marker;
+    A.FinalAndExpectedMatch =
+        DiagItr->MatchTy == FileCheckDiag::MatchFinalAndExpected;
 
     // Compute the mark location, and break annotation into multiple
     // annotations if it spans multiple lines.
     A.InputLine = DiagItr->InputStartLine;
     A.InputStartCol = DiagItr->InputStartCol;
     if (DiagItr->InputStartLine == DiagItr->InputEndLine) {
       // Sometimes ranges are empty in order to indicate a specific point, but
       // that would mean nothing would be marked, so adjust the range to
if (DiagItr->InputStartLine == DiagItr->InputEndLine) {
         B.Marker = Marker;
         B.Marker.Lead = '~';
         B.InputStartCol = 1;
         if (L != E) {
           B.InputEndCol = UINT_MAX;
           B.Marker.Note = "";
         } else
           B.InputEndCol = DiagItr->InputEndCol;
+        B.FinalAndExpectedMatch = A.FinalAndExpectedMatch;
         Annotations.push_back(B);
       }
     }
   }
 }
 
-static void DumpAnnotatedInput(
-    raw_ostream &OS, StringRef InputFileText,
-    std::vector<InputAnnotation> &Annotations, unsigned LabelWidth) {
+static void DumpAnnotatedInput(raw_ostream &OS, const FileCheckRequest &Req,
+                               StringRef InputFileText,
+                               std::vector<InputAnnotation> &Annotations,
+                               unsigned LabelWidth) {
   OS << "Full input was:\n<<<<<<\n";
 
   // Sort annotations.
   //
   // First, sort in the order of input lines to make it easier to find relevant
   // annotations while iterating input lines in the implementation below.
   // FileCheck diagnostics are not always reported and recorded in the order of
   // input lines due to, for example, CHECK-DAG and CHECK-NOT.
   //
   // Second, for annotations for the same input line, sort in the order of the
   // FileCheck directive's line in the check file (where there's at most one
   // directive per line) and then by the index of the match result for that
   // directive.  The rationale of this choice is that, for any input line, this
   // sort establishes a total order of annotations that, with respect to match
   // results, is consistent across multiple lines, thus making match results
   // easier to track from one line to the next when they span multiple lines.
   std::sort(Annotations.begin(), Annotations.end(),
             [](const InputAnnotation &A, const InputAnnotation &B) {
               if (A.InputLine != B.InputLine)
                 return A.InputLine < B.InputLine;
               if (A.CheckLine != B.CheckLine)
                 return A.CheckLine < B.CheckLine;
-              assert(A.CheckDiagIndex != B.CheckDiagIndex &&
-                     "expected diagnostic indices to be unique within a "
-                     " check line");
+              // FIXME: Sometimes CHECK-LABEL reports its match twice with
+              // other diagnostics in between, and then diag index incrementing
+              // fails to work properly, and then this assert fails.  We should
+              // suppress one of those diagnostics or do a better job of
+              // computing this index.  For now, we just produce a redundant
+              // CHECK-LABEL annotation.
+              // assert(A.CheckDiagIndex != B.CheckDiagIndex &&
+              //        "expected diagnostic indices to be unique within a "
+              //        " check line");
               return A.CheckDiagIndex < B.CheckDiagIndex;
             });
 
   // Compute the width of the label column.
   const unsigned char *InputFilePtr = InputFileText.bytes_begin(),
                       *InputFileEnd = InputFileText.bytes_end();
   unsigned LineCount = InputFileText.count('\n');
   if (InputFileEnd[-1] != '\n')
for (unsigned Line = 1;
        InputFilePtr != InputFileEnd || AnnotationItr != AnnotationEnd;
        ++Line) {
     const unsigned char *InputFileLine = InputFilePtr;
 
     // Print right-aligned line number.
     WithColor(OS, raw_ostream::BLACK, true)
         << format_decimal(Line, LabelWidth) << ": ";
 
-    // Print numbered line.
+    // For case where -v and colors are enabled, find the annotations for final
+    // matches for expected patterns in order to highlight everything else in
+    // the line.  There are no such annotations if -v is disabled.
+    std::vector<InputAnnotation> FinalAndExpectedMatches;
+    if (Req.Verbose && WithColor(OS).colorsEnabled()) {
+      for (auto I = AnnotationItr; I != AnnotationEnd && I->InputLine == Line;
+           ++I) {
+        if (I->FinalAndExpectedMatch)
+          FinalAndExpectedMatches.push_back(*I);
+      }
+    }
+
+    // Print numbered line with highlighting where there are no matches for
+    // expected patterns.
     bool Newline = false;
-    while (InputFilePtr != InputFileEnd && !Newline) {
+    {
+      WithColor COS(OS);
+      bool InMatch = false;
+      if (Req.Verbose)
+        COS.changeColor(raw_ostream::CYAN, true, true);
+      for (unsigned Col = 1; InputFilePtr != InputFileEnd && !Newline; ++Col) {
+        bool WasInMatch = InMatch;
+        InMatch = false;
+        for (auto M : FinalAndExpectedMatches) {
+          if (M.InputStartCol <= Col && Col < M.InputEndCol) {
+            InMatch = true;
+            break;
+          }
+        }
+        if (!WasInMatch && InMatch)
+          COS.resetColor();
+        else if (WasInMatch && !InMatch)
+          COS.changeColor(raw_ostream::CYAN, true, true);
         if (*InputFilePtr == '\n')
           Newline = true;
         else
-        OS << *InputFilePtr;
+          COS << *InputFilePtr;
         ++InputFilePtr;
       }
+    }
     OS << '\n';
     unsigned InputLineWidth = InputFilePtr - InputFileLine - Newline;
 
     // Print any annotations.
     while (AnnotationItr != AnnotationEnd &&
            AnnotationItr->InputLine == Line) {
       WithColor COS(OS, AnnotationItr->Marker.Color, true);
       // The two spaces below are where the ": " appears on input lines.
errs() << "\n"
            << "\n"
            << "Check file: " << CheckFilename << "\n"
            << "\n"
            << "-dump-input=help describes the format of the following dump.\n"
            << "\n";
     std::vector<InputAnnotation> Annotations;
     unsigned LabelWidth;
     BuildInputAnnotations(Diags, Annotations, LabelWidth);
-    DumpAnnotatedInput(errs(), InputFileText, Annotations, LabelWidth);
+    DumpAnnotatedInput(errs(), Req, InputFileText, Annotations, LabelWidth);
   }
 
   return ExitCode;
 }