Add FileCheck -implicit-check-not option to allow stricter tests without adding too many CHECK-NOTs manually.

[alexfh]

Summary

Add FileCheck -implicit-check-not option which allows specifying a
pattern that should only occur in the input when explicitly matched by a
positive check. This feature allows checking tool diagnostics in a way
clang -verify does it for compiler diagnostics.

The option has been tested on a number of clang-tidy checks, I'll post a link to
the clang-tidy patch to this thread.

Once there's an agreement on the general direction, I can add tests and
documentation.

[alexfh]

The change in clang-tidy tests used to test this patch is posted as D4463.

[djasper]

I don't know much about FileCheck's code, so I might not be the best reviewer. However, shouldn't you add at least one test making use of the new feature?

[djasper]

I don't think it is sufficient to have a test in clang-tools-extra...

[alexfh]

In D4462#6, @djasper wrote:

I don't think it is sufficient to have a test in clang-tools-extra...

Sure, this is why I wrote "Once there's an agreement on the general direction, I can add tests and documentation.", where by "tests" I meant standalone tests for FileCheck.

[alexfh]

Added tests and documentation for -implicit-check-not.

[bkramer]

lg, with a minor test suggestion.

test/FileCheck/implicit-check-not.txt

1–4 ↗

(On Diff #11291)

Can we also have a test with multiple implicit-check-not patterns?

[alexfh]

Added test for multiple -implicit-check-not directives and for regular expressions in -implicit-check-not.

utils/FileCheck/FileCheck.cpp

 static cl::list<std::string>
 CheckPrefixes("check-prefix",
               cl::desc("Prefix to use from check file (defaults to 'CHECK')"));
 
 static cl::opt<bool>
 NoCanonicalizeWhiteSpace("strict-whitespace",
               cl::desc("Do not treat all horizontal whitespace as equivalent"));
 
+static cl::list<std::string> ImplicitCheckNot(
+    "implicit-check-not",
+    cl::desc("Add an implicit negative check with this pattern to every\n"
+             "positive check. This can be used to ensure that no instances of\n"
+             "this pattern occur which are not matched by a positive pattern"),
+    cl::value_desc("pattern"));
+
 typedef cl::list<std::string>::const_iterator prefix_iterator;
 
 //===----------------------------------------------------------------------===//
 // Pattern Handling Code.
 //===----------------------------------------------------------------------===//
 
 namespace Check {
   enum CheckType {
static bool ReadCheckFile(SourceMgr &SM,
   // buffer containing the CHECK lines. Remove DOS style line endings.
   MemoryBuffer *F = CanonicalizeInputFile(FileOrErr.get().release(),
                                           NoCanonicalizeWhiteSpace);
 
   SM.AddNewSourceBuffer(F, SMLoc());
 
   // Find all instances of CheckPrefix followed by : in the file.
   StringRef Buffer = F->getBuffer();
-  std::vector<Pattern> DagNotMatches;
+
+  std::vector<Pattern> ImplicitNegativeChecks;
+  for (const auto &PatternString : ImplicitCheckNot) {
+    // Create a buffer with fake command line content in order to display the
+    // command line option responsible for the specific implicit CHECK-NOT.
+    std::string Prefix = std::string("-") + ImplicitCheckNot.ArgStr + "='";
+    std::string Suffix = "'";
+    MemoryBuffer *CmdLine = MemoryBuffer::getMemBufferCopy(
+        Prefix + PatternString + Suffix, "command line");
+    StringRef PatternInBuffer =
+        CmdLine->getBuffer().substr(Prefix.size(), PatternString.size());
+    SM.AddNewSourceBuffer(CmdLine, SMLoc());
+
+    ImplicitNegativeChecks.push_back(Pattern(Check::CheckNot));
+    ImplicitNegativeChecks.back().ParsePattern(PatternInBuffer,
+                                               "IMPLICIT-CHECK", SM, 0);
+  }
+
+
+  std::vector<Pattern> DagNotMatches = ImplicitNegativeChecks;
 
   // LineNumber keeps track of the line on which CheckPrefix instances are
   // found.
   unsigned LineNumber = 1;
 
   while (1) {
     Check::CheckType CheckTy;
     size_t PrefixLoc;
while (1) {
     }
 
     // Okay, add the string we captured to the output vector and move on.
     CheckStrings.push_back(CheckString(P,
                                        UsedPrefix,
                                        PatternLoc,
                                        CheckTy));
     std::swap(DagNotMatches, CheckStrings.back().DagNotStrings);
+    DagNotMatches = ImplicitNegativeChecks;
   }
 
   // Add an EOF pattern for any trailing CHECK-DAG/-NOTs, and use the first
   // prefix as a filler for the error message.
   if (!DagNotMatches.empty()) {
     CheckStrings.push_back(CheckString(Pattern(Check::CheckEOF),
                                        CheckPrefixes[0],
                                        SMLoc::getFromPointer(Buffer.data()),