[FileCheck] Annotate input dump (7/7)

[jdenny]

Summary

This patch implements annotations for diagnostics reporting CHECK-NOT
failed matches.  These diagnostics are enabled by -vv.  As for
diagnostics reporting failed matches for other directives, these
annotations mark the search ranges using `X~~`.  The difference here
is that failed matches for CHECK-NOT are successes not errors, so they
are green not red when colors are enabled.

For example:

```
$ FileCheck -dump-input=help
The following description was requested by -dump-input=help to
explain the input annotations printed by -dump-input=always and
-dump-input=fail:

  - L:     labels line number L of the input file
  - T:L    labels the only match result for a pattern of type T from line L of
           the check file
  - T:L'N  labels the Nth match result for a pattern of type T from line L of
           the check file
  - ^~~    marks good match (reported if -v)
  - !~~    marks bad match, such as:
           - CHECK-NEXT on same line as previous match (error)
           - CHECK-NOT found (error)
           - CHECK-DAG overlapping match (discarded, reported if -vv)
  - X~~    marks search range when no match is found, such as:
           - CHECK-NEXT not found (error)
           - CHECK-NOT not found (success, reported if -vv)
           - CHECK-DAG not found after discarded matches (error)
  - ?      marks fuzzy match when no match is found
  - colors success, error, fuzzy match, discarded match, unmatched input

If you are not seeing color above or in input dumps, try: -color

$ FileCheck -vv -dump-input=always check5 < input5 |& sed -n '/^<<<</,$p'
<<<<<<
         1: abcdef
check:1     ^~~
not:2          X~~
         2: ghijkl
not:2       ~~~
check:3        ^~~
         3: mnopqr
not:4       X~~~~~
         4: stuvwx
not:4       ~~~~~~
         5:
eof:4       ^
>>>>>>

$ cat check5
CHECK: abc
CHECK-NOT: foobar
CHECK: jkl
CHECK-NOT: foobar

$ cat input5
abcdef
ghijkl
mnopqr
stuvwx
```

[jdenny]

Rebased onto updated patches earlier in series.

[jdenny]

Update for changes earlier in patch series.

[jdenny]

Propagate changes started earlier in patch series.

[jdenny]

Continue changes started earlier in patch series.

[jdenny]

Adjust comment, as suggested by probinson.

Remove MatchTypeCount, which is unused as of this patch.

[probinson]

In D53899#1328955, @jdenny wrote:

Remove MatchTypeCount, which is unused as of this patch.

What do we see if a Counted match doesn't have enough matches? I'm not seeing a test that covers that case.

[jdenny]

llvm/test/FileCheck/dump-input-annotations.txt

63–64

In D53899#1331576, @probinson wrote:

In D53899#1328955, @jdenny wrote:

Remove MatchTypeCount, which is unused as of this patch.

To be clear, I originally meant MatchTypeCount to be a count of the MatchType members, so it's not actually related to your question below. In this version of the patch series, that wasn't even it's role anymore. I should have named it something more like MatchTypeUndefined. Sorry about that. Anyway, this final patch removes it.

What do we see if a Counted match doesn't have enough matches? I'm not seeing a test that covers that case.

I'm responding as an inline comment in the test that covers it. (Don't be misled: the error isn't reported twice as CNT-Q and CNT-V are not active at the same time.)

Do you feel we need a separate MatchType and a more specific note for this case?

[probinson]

llvm/test/FileCheck/dump-input-annotations.txt

63–64

Thanks for pointing it out, and no I think what you have is fine.

[probinson]

LGTM

llvm/include/llvm/Support/FileCheck.h

struct FileCheckDiag {
   /// example, there might be several discards before either a final or fail,
   /// and there might be a fuzzy match after a fail.
   ///
   /// We iterate these types, so they must have contiguous values in
   /// [0, MatchTypeCount).  Moreover, keep match types together if they use the
   /// same mark in annotated input dumps or else printing of the annotation key
   /// will malfunction.
   enum MatchType {
-    // TODO: More members will appear with later patches in this series.
     MatchFinalAndExpected, //< the final match for an expected pattern
     MatchTypeFirst = MatchFinalAndExpected,
     MatchFinalButExcluded, //< the final match for an excluded pattern
     MatchFinalButIllegal,  //< the final but illegal match for an expected pattern
     MatchDiscard,          //< a discarded match for an expected pattern
+    MatchNoneAndExcluded,  //< no match for an excluded pattern
     MatchNoneButExpected,  //< no match for an expected pattern
     MatchFuzzy,            //< a fuzzy match (because no perfect match)
     MatchTypeCount,
   } MatchTy;
-  /// The match range if MatchTy is not MatchNoneButExpected, or the search
-  /// range otherwise.
+  /// The match range if MatchTy is not MatchNoneAndExcluded or
+  /// MatchNoneButExpected, or the search range otherwise.
   unsigned InputStartLine, InputStartCol, InputEndLine, InputEndCol;
   FileCheckDiag(const SourceMgr &SM, const Check::FileCheckType &CheckTy,
                 SMLoc CheckLoc, MatchType MatchTy, SMRange InputRange);
 };
 
 //===----------------------------------------------------------------------===//
 // Check Strings.
 //===----------------------------------------------------------------------===//

llvm/lib/Support/FileCheck.cpp

static SMRange ProcessMatchResult(FileCheckDiag::MatchType MatchTy,
                                   const SourceMgr &SM, SMLoc Loc,
                                   Check::FileCheckType CheckTy,
                                   StringRef Buffer, size_t Pos, size_t Len,
                                   std::list<FileCheckDiag> *Diags,
                                   bool AdjustPrevDiag = false) {
   SMLoc Start = SMLoc::getFromPointer(Buffer.data() + Pos);
   SMLoc End = SMLoc::getFromPointer(Buffer.data() + Pos + Len);
   SMRange Range(Start, End);
-  // TODO: The second condition will disappear when we extend this to handle
-  // more match types.
-  if (Diags && MatchTy != FileCheckDiag::MatchTypeCount) {
+  if (Diags) {
     if (AdjustPrevDiag)
       Diags->rbegin()->MatchTy = MatchTy;
     else
       Diags->emplace_back(SM, CheckTy, Loc, MatchTy, Range);
   }
   return Range;
 }
 
SM.PrintMessage(Loc,
                       (ExpectedMatch ? "expected" : "excluded") +
                       " string not found in input");
 
   // Print the "scanning from here" line.  If the current position is at the
   // end of a line, advance to the start of the next line.
   Buffer = Buffer.substr(Buffer.find_first_not_of(" \t\n\r"));
   SMRange SearchRange = ProcessMatchResult(
       ExpectedMatch ? FileCheckDiag::MatchNoneButExpected
-                    : FileCheckDiag::MatchTypeCount,
+                    : FileCheckDiag::MatchNoneAndExcluded,
       SM, Loc, Pat.getCheckTy(), Buffer, 0, Buffer.size(), Diags);
   SM.PrintMessage(SearchRange.Start, SourceMgr::DK_Note, "scanning from here");
 
   // Allow the pattern to print additional information if desired.
   Pat.PrintVariableUses(SM, Buffer, VariableTable);
+
   if (ExpectedMatch)
     Pat.PrintFuzzyMatch(SM, Buffer, VariableTable, Diags);
 }
 
 static void PrintNoMatch(bool ExpectedMatch, const SourceMgr &SM,
                          const FileCheckString &CheckStr, StringRef Buffer,
                          StringMap<StringRef> &VariableTable,
                          bool VerboseVerbose,
bool FileCheckString::CheckNot(
   for (const FileCheckPattern *Pat : NotStrings) {
     assert((Pat->getCheckTy() == Check::CheckNot) && "Expect CHECK-NOT!");
 
     size_t MatchLen = 0;
     size_t Pos = Pat->Match(Buffer, MatchLen, VariableTable);
 
     if (Pos == StringRef::npos) {
       PrintNoMatch(false, SM, Prefix, Pat->getLoc(), *Pat, Buffer,
-                   VariableTable, Req.VerboseVerbose, nullptr);
+                   VariableTable, Req.VerboseVerbose, Diags);
       continue;
     }
 
     PrintMatch(false, SM, Prefix, Pat->getLoc(), *Pat, Buffer, VariableTable,
                Pos, MatchLen, Req, Diags);
 
     return true;
   }

llvm/test/FileCheck/dump-input-annotations.txt

 ; CHK-NEXT:              2: again
 ; CHK-NEXT:   check:2'0     X~~~~
 ; CHK-NEXT:              3: whirled
 ; CHK-NEXT:   check:2'0     ~~~~~~~
 ; CHK-NEXT:   check:2'1     ?
 ; CHK-NEXT:   >>>>>>
 ; CHK-NOT:    {{.}}
 
 ;--------------------------------------------------
 ; CHECK-NEXT (also: EOF search-range, illegal match)
 ;--------------------------------------------------
 
 ; Good match and no match.
 
 ; RUN: echo 'hello' > %t.in
 ; RUN: echo 'again' >> %t.in
 
 ; RUN: echo 'CHECK: hello' > %t.chk

 ; EMP2-V-NEXT: check:1     ^~~~~
 ; EMP2-NEXT:            2: world
 ; EMP2-NEXT:            3:
 ; EMP2-NEXT:   empty:2     !
 ; EMP2-NEXT:   >>>>>>
 ; EMP2-NOT:    {{.}}
 
 ;--------------------------------------------------
-; CHECK-NOT (also: EOF pattern)
+; CHECK-NOT (also: EOF pattern, and multiline range that ends before EOL)
 ;--------------------------------------------------
 
 ; No match (success) and illegal match.
 
 ; RUN: echo 'hello' > %t.in
 ; RUN: echo 'world' >> %t.in
 ; RUN: echo 'again' >> %t.in
 
 ; RUN: echo 'CHECK-NOT: goodbye' > %t.chk
 ; RUN: echo 'CHECK-NOT: world' >> %t.chk
 
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefix=NOT
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -v 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=NOT,NOT-V
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -vv 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=NOT,NOT-V,NOT-VV
 
 ; NOT:         <<<<<<
 ; NOT-NEXT:           1: hello
+; NOT-VV-NEXT: not:1     X~~~~
 ; NOT-NEXT:           2: world
+; NOT-VV-NEXT: not:1     ~~~~~
 ; NOT-NEXT:    not:2     !~~~~
 ; NOT-NEXT:           3: again
+; NOT-VV-NEXT: not:1     ~~~~~
 ; NOT-VV-NEXT:        4:
 ; NOT-VV-NEXT: eof:2     ^
 ; NOT-NEXT:    >>>>>>
 ; NOT-NOT:     {{.}}
 
 ; Again, but with a CHECK instead of EOF as search range end.
 
 ; RUN: echo 'CHECK: ain' >> %t.chk
 
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefix=NOT2
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -v 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=NOT2,NOT2-V
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -vv 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=NOT2,NOT2-V,NOT2-VV
 
 ; NOT2:         <<<<<<
 ; NOT2-NEXT:             1: hello
+; NOT2-VV-NEXT: not:1       X~~~~
 ; NOT2-NEXT:             2: world
+; NOT2-VV-NEXT: not:1       ~~~~~
 ; NOT2-NEXT:    not:2       !~~~~
 ; NOT2-NEXT:             3: again
+; NOT2-VV-NEXT: not:1       ~~
 ; NOT2-V-NEXT:  check:3       ^~~
 ; NOT2-NEXT:    >>>>>>
 ; NOT2-NOT:     {{.}}
 
 ;--------------------------------------------------
 ; CHECK-DAG (also: matches in different order than directives, discarded match)
 ;--------------------------------------------------
 

llvm/utils/FileCheck/FileCheck.cpp

case FileCheckDiag::MatchFinalButIllegal:
     return MatchTypeStyle('!', true, raw_ostream::RED, MatchTypeStyle::Quiet,
                           "the final but illegal match for an expected "
                           "pattern (e.g., CHECK-NEXT)");
   case FileCheckDiag::MatchDiscard:
     return MatchTypeStyle('!', true, raw_ostream::CYAN,
                           MatchTypeStyle::VerboseVerbose,
                           "a discarded match for an expected pattern (e.g., "
                           "CHECK-DAG)");
+  case FileCheckDiag::MatchNoneAndExcluded:
+    return MatchTypeStyle('X', true, raw_ostream::GREEN,
+                          MatchTypeStyle::VerboseVerbose,
+                          "the search range for an unmatched excluded "
+                          "pattern (e.g., CHECK-NOT)");
   case FileCheckDiag::MatchNoneButExpected:
     return MatchTypeStyle('X', true, raw_ostream::RED, MatchTypeStyle::Quiet,
                           "the search range for an unmatched expected "
                           "pattern (e.g., CHECK)");
   case FileCheckDiag::MatchFuzzy:
     return MatchTypeStyle('?', false, raw_ostream::MAGENTA,
                           MatchTypeStyle::Quiet,
                           "a fuzzy match start for an otherwise unmatched "