[FileCheck] Annotate input dump (7/7)

[jdenny]

Summary

This patch implements annotations for diagnostics reporting CHECK-NOT
failed matches.  These diagnostics are enabled by -vv.  As for
diagnostics reporting failed matches for other directives, these
annotations mark the search ranges using `X~~`.  The difference here
is that failed matches for CHECK-NOT are successes not errors, so they
are green not red when colors are enabled.

For example:

```
$ FileCheck -dump-input=help
The following description was requested by -dump-input=help to
explain the input annotations printed by -dump-input=always and
-dump-input=fail:

  - L:     labels line number L of the input file
  - T:L    labels the only match result for a pattern of type T from line L of
           the check file
  - T:L'N  labels the Nth match result for a pattern of type T from line L of
           the check file
  - ^~~    marks good match (reported if -v)
  - !~~    marks bad match, such as:
           - CHECK-NEXT on same line as previous match (error)
           - CHECK-NOT found (error)
           - CHECK-DAG overlapping match (discarded, reported if -vv)
  - X~~    marks search range when no match is found, such as:
           - CHECK-NEXT not found (error)
           - CHECK-NOT not found (success, reported if -vv)
           - CHECK-DAG not found after discarded matches (error)
  - ?      marks fuzzy match when no match is found
  - colors success, error, fuzzy match, discarded match, unmatched input

If you are not seeing color above or in input dumps, try: -color

$ FileCheck -vv -dump-input=always check5 < input5 |& sed -n '/^<<<</,$p'
<<<<<<
         1: abcdef
check:1     ^~~
not:2          X~~
         2: ghijkl
not:2       ~~~
check:3        ^~~
         3: mnopqr
not:4       X~~~~~
         4: stuvwx
not:4       ~~~~~~
         5:
eof:4       ^
>>>>>>

$ cat check5
CHECK: abc
CHECK-NOT: foobar
CHECK: jkl
CHECK-NOT: foobar

$ cat input5
abcdef
ghijkl
mnopqr
stuvwx
```

[jdenny]

Rebased onto updated patches earlier in series.

[jdenny]

Update for changes earlier in patch series.

[jdenny]

Propagate changes started earlier in patch series.

[jdenny]

Continue changes started earlier in patch series.

[jdenny]

Adjust comment, as suggested by probinson.

Remove MatchTypeCount, which is unused as of this patch.

[probinson]

In D53899#1328955, @jdenny wrote:

Remove MatchTypeCount, which is unused as of this patch.

What do we see if a Counted match doesn't have enough matches? I'm not seeing a test that covers that case.

[jdenny]

llvm/test/FileCheck/dump-input-annotations.txt

93 ↗

(On Diff #177948)

In D53899#1331576, @probinson wrote:

In D53899#1328955, @jdenny wrote:

Remove MatchTypeCount, which is unused as of this patch.

To be clear, I originally meant MatchTypeCount to be a count of the MatchType members, so it's not actually related to your question below. In this version of the patch series, that wasn't even it's role anymore. I should have named it something more like MatchTypeUndefined. Sorry about that. Anyway, this final patch removes it.

What do we see if a Counted match doesn't have enough matches? I'm not seeing a test that covers that case.

I'm responding as an inline comment in the test that covers it. (Don't be misled: the error isn't reported twice as CNT-Q and CNT-V are not active at the same time.)

Do you feel we need a separate MatchType and a more specific note for this case?

[probinson]

llvm/test/FileCheck/dump-input-annotations.txt

93 ↗

(On Diff #177948)

Thanks for pointing it out, and no I think what you have is fine.

[probinson]

LGTM

llvm/trunk/include/llvm/Support/FileCheck.h

enum MatchType {
     MatchFinalAndExpected,
     /// Indicates the final match for an excluded pattern.
     MatchFinalButExcluded,
     /// Indicates the final match for an expected pattern, but the match is on
     /// the wrong line.
     MatchFinalButWrongLine,
     /// Indicates a discarded match for an expected pattern.
     MatchDiscard,
+    /// Indicates no match for an excluded pattern.
+    MatchNoneAndExcluded,
     /// Indicates no match for an expected pattern.
     MatchNoneButExpected,
     /// Indicates a possible intended match because there's no perfect match.
     MatchFuzzy,
-    MatchTypeCount,
   } MatchTy;
-  /// The match range if MatchTy is not MatchNoneButExpected, or the search
-  /// range otherwise.
+  /// The match range if MatchTy is not MatchNoneAndExcluded or
+  /// MatchNoneButExpected, or the search range otherwise.
   unsigned InputStartLine, InputStartCol, InputEndLine, InputEndCol;
   FileCheckDiag(const SourceMgr &SM, const Check::FileCheckType &CheckTy,
                 SMLoc CheckLoc, MatchType MatchTy, SMRange InputRange);
 };
 
 //===----------------------------------------------------------------------===//
 // Check Strings.
 //===----------------------------------------------------------------------===//

llvm/trunk/lib/Support/FileCheck.cpp

static SMRange ProcessMatchResult(FileCheckDiag::MatchType MatchTy,
                                   const SourceMgr &SM, SMLoc Loc,
                                   Check::FileCheckType CheckTy,
                                   StringRef Buffer, size_t Pos, size_t Len,
                                   std::vector<FileCheckDiag> *Diags,
                                   bool AdjustPrevDiag = false) {
   SMLoc Start = SMLoc::getFromPointer(Buffer.data() + Pos);
   SMLoc End = SMLoc::getFromPointer(Buffer.data() + Pos + Len);
   SMRange Range(Start, End);
-  // TODO: The second condition will disappear when we extend this to handle
-  // more match types.
-  if (Diags && MatchTy != FileCheckDiag::MatchTypeCount) {
+  if (Diags) {
     if (AdjustPrevDiag)
       Diags->rbegin()->MatchTy = MatchTy;
     else
       Diags->emplace_back(SM, CheckTy, Loc, MatchTy, Range);
   }
   return Range;
 }
 
static void PrintNoMatch(bool ExpectedMatch, const SourceMgr &SM,
   SM.PrintMessage(
       Loc, ExpectedMatch ? SourceMgr::DK_Error : SourceMgr::DK_Remark, Message);
 
   // Print the "scanning from here" line.  If the current position is at the
   // end of a line, advance to the start of the next line.
   Buffer = Buffer.substr(Buffer.find_first_not_of(" \t\n\r"));
   SMRange SearchRange = ProcessMatchResult(
       ExpectedMatch ? FileCheckDiag::MatchNoneButExpected
-                    : FileCheckDiag::MatchTypeCount,
+                    : FileCheckDiag::MatchNoneAndExcluded,
       SM, Loc, Pat.getCheckTy(), Buffer, 0, Buffer.size(), Diags);
   SM.PrintMessage(SearchRange.Start, SourceMgr::DK_Note, "scanning from here");
 
   // Allow the pattern to print additional information if desired.
   Pat.PrintVariableUses(SM, Buffer, VariableTable);
+
   if (ExpectedMatch)
     Pat.PrintFuzzyMatch(SM, Buffer, VariableTable, Diags);
 }
 
 static void PrintNoMatch(bool ExpectedMatch, const SourceMgr &SM,
                          const FileCheckString &CheckStr, int MatchedCount,
                          StringRef Buffer, StringMap<StringRef> &VariableTable,
                          bool VerboseVerbose,
bool FileCheckString::CheckNot(
   for (const FileCheckPattern *Pat : NotStrings) {
     assert((Pat->getCheckTy() == Check::CheckNot) && "Expect CHECK-NOT!");
 
     size_t MatchLen = 0;
     size_t Pos = Pat->Match(Buffer, MatchLen, VariableTable);
 
     if (Pos == StringRef::npos) {
       PrintNoMatch(false, SM, Prefix, Pat->getLoc(), *Pat, 1, Buffer,
-                   VariableTable, Req.VerboseVerbose, nullptr);
+                   VariableTable, Req.VerboseVerbose, Diags);
       continue;
     }
 
     PrintMatch(false, SM, Prefix, Pat->getLoc(), *Pat, 1, Buffer, VariableTable,
                Pos, MatchLen, Req, Diags);
 
     return true;
   }

llvm/trunk/test/FileCheck/dump-input-annotations.txt

 ; EMP2-V-NEXT: check:1     ^~~~~
 ; EMP2-NEXT:            2: world
 ; EMP2-NEXT:            3:
 ; EMP2-NEXT:   empty:2     !     error: match on wrong line
 ; EMP2-NEXT:   >>>>>>
 ; EMP2-NOT:    {{.}}
 
 ;--------------------------------------------------
-; CHECK-NOT (also: EOF pattern)
+; CHECK-NOT (also: EOF pattern, and multiline range that ends before EOL)
 ;--------------------------------------------------
 
 ; No match (success) and unexpected match (error).
 
 ; RUN: echo 'hello' > %t.in
 ; RUN: echo 'world' >> %t.in
 ; RUN: echo 'again' >> %t.in
 
 ; RUN: echo 'CHECK-NOT: goodbye' > %t.chk
 ; RUN: echo 'CHECK-NOT: world' >> %t.chk
 
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefix=NOT
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -v 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=NOT,NOT-V
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -vv 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=NOT,NOT-V,NOT-VV
 
 ; NOT:         <<<<<<
 ; NOT-NEXT:           1: hello
+; NOT-VV-NEXT: not:1     X~~~~
 ; NOT-NEXT:           2: world
+; NOT-VV-NEXT: not:1     ~~~~~
 ; NOT-NEXT:    not:2     !~~~~ error: no match expected
 ; NOT-NEXT:           3: again
+; NOT-VV-NEXT: not:1     ~~~~~
 ; NOT-VV-NEXT:        4:
 ; NOT-VV-NEXT: eof:2     ^
 ; NOT-NEXT:    >>>>>>
 ; NOT-NOT:     {{.}}
 
 ; Again, but with a CHECK instead of EOF as search range end.
 
 ; RUN: echo 'CHECK: ain' >> %t.chk
 
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefix=NOT2
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -v 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=NOT2,NOT2-V
 ; RUN: not FileCheck -dump-input=always -input-file %t.in %t.chk -vv 2>&1 \
 ; RUN: | FileCheck -match-full-lines %s -check-prefixes=NOT2,NOT2-V,NOT2-VV
 
 ; NOT2:         <<<<<<
 ; NOT2-NEXT:             1: hello
+; NOT2-VV-NEXT: not:1       X~~~~
 ; NOT2-NEXT:             2: world
+; NOT2-VV-NEXT: not:1       ~~~~~
 ; NOT2-NEXT:    not:2       !~~~~ error: no match expected
 ; NOT2-NEXT:             3: again
+; NOT2-VV-NEXT: not:1       ~~
 ; NOT2-V-NEXT:  check:3       ^~~
 ; NOT2-NEXT:    >>>>>>
 ; NOT2-NOT:     {{.}}
 
 ;--------------------------------------------------
 ; CHECK-DAG (also: matches in different order than directives, discarded match)
 ;--------------------------------------------------
 

llvm/trunk/utils/FileCheck/FileCheck.cpp

case FileCheckDiag::MatchFinalAndExpected:
     return MarkerStyle('^', raw_ostream::GREEN);
   case FileCheckDiag::MatchFinalButExcluded:
     return MarkerStyle('!', raw_ostream::RED, "error: no match expected");
   case FileCheckDiag::MatchFinalButWrongLine:
     return MarkerStyle('!', raw_ostream::RED, "error: match on wrong line");
   case FileCheckDiag::MatchDiscard:
     return MarkerStyle('!', raw_ostream::CYAN,
                        "discard: overlaps earlier match");
+  case FileCheckDiag::MatchNoneAndExcluded:
+    return MarkerStyle('X', raw_ostream::GREEN);
   case FileCheckDiag::MatchNoneButExpected:
     return MarkerStyle('X', raw_ostream::RED, "error: no match found");
   case FileCheckDiag::MatchFuzzy:
     return MarkerStyle('?', raw_ostream::MAGENTA, "possible intended match");
-  case FileCheckDiag::MatchTypeCount:
-    llvm_unreachable_internal("unexpected match type");
   }
   llvm_unreachable_internal("unexpected match type");
 }
 
 static void DumpInputAnnotationHelp(raw_ostream &OS) {
   OS << "The following description was requested by -dump-input=help to\n"
      << "explain the input annotations printed by -dump-input=always and\n"
      << "-dump-input=fail:\n\n";
OS << "    marks bad match, such as:\n"
      << "           - CHECK-NEXT on same line as previous match (error)\n"
      << "           - CHECK-NOT found (error)\n"
      << "           - CHECK-DAG overlapping match (discarded, reported if "
      << "-vv)\n"
      << "  - ";
   WithColor(OS, raw_ostream::SAVEDCOLOR, true) << "X~~";
   OS << "    marks search range when no match is found, such as:\n"
      << "           - CHECK-NEXT not found (error)\n"
+     << "           - CHECK-NOT not found (success, reported if -vv)\n"
      << "           - CHECK-DAG not found after discarded matches (error)\n"
      << "  - ";
   WithColor(OS, raw_ostream::SAVEDCOLOR, true) << "?";
   OS << "      marks fuzzy match when no match is found\n";
 
   // Colors.
   OS << "  - colors ";
   WithColor(OS, raw_ostream::GREEN, true) << "success";