[FileCheck] Add directive for checking for blank lines

[jhenderson]

Summary

There is currently no reliable way in FileCheck to check that a line is completely empty. The expected way of using "CHECK: {{^$}}" does not work because the '^' matches the end of the previous match (this behaviour may be desirable in certain instances). For the same reason, "CHECK-NEXT: {{^$}}" will fail when the previous match was at the end of the line, as the pattern will match there. Using the recommended [[:space:]] to match an explicit new line could also match a space, and thus is not always desired. Literal '\n' matches also do not seem to work.

This change adds a new directive that behaves the same as CHECK-NEXT, except that it only matches against empty lines. As with CHECK-NEXT, it will fail if more than one newline occurs before the next blank line. Example usage:
; test.txt
foo

bar
; CHECK: foo
; CHECK-EMPTY:
; CHECK-NEXT: bar

[probinson]

I spent some time experimenting with different approaches, and my inner hacker is pleased to find something unobvious that works. Admittedly it is not as cleanly self-documenting as the new directive.

CHECK:foo{{$}}
CHECK-SAME:{{[[:space:]]$}}
CHECK-NEXT:bar

which must be run using the --strict-whitespace option to FileCheck, so that the :space: cannot match anything other than the newline between the two $ metacharacters. Also note the second directive is SAME not NEXT. Without --strict-whitespace then additional spaces on the "empty" line would be allowed. That option is really for extra-picky cases, but if you are caring about blank lines then it's pretty clear you already have an extra-picky case. :-) Let me know if it doesn't work for you, or if you have other cases where this tactic would not be usable.

Alternatively, we could figure out how to get the regex package to recognize \n (and for that matter other backslash-escape sequences). This has come up before, so I think that would be more valuable in the long run than adding another FileCheck directive.

[arichardson]

I just ran into this problem writing a test case and for now I used the workaround by @probinson but having a CHECK-EMPTY directive would make it much cleaner.

[jeroen.dobbelaere]

How about using an empty 'CHECK-NEXT:' statement for this ?

; CHECK:foo
; CHECK-NEXT:  
; CHECK-NEXT:bar

That would make it much easier to copy-paste a reference piece, add 'CHECK-NEXT:' in front, except for the first line, and be done with it.

[jhenderson]

In D28896#1138983, @jeroen.dobbelaere wrote:

How about using an empty 'CHECK-NEXT:' statement for this ?

; CHECK:foo
; CHECK-NEXT:  
; CHECK-NEXT:bar

That would make it much easier to copy-paste a reference piece, add 'CHECK-NEXT:' in front, except for the first line, and be done with it.

What would be the meaning of CHECK-NEXT in this situation? Because that would to me, match an empty string, expected to be on the next line. The next empty string after foo is trivially on the same line as foo. Note that CHECK-NEXT searches for the next instance of the specified string, and fails if it finds it anywhere other than on the next line, rather than just searching the next line for the specified string.

[gbedwell]

This change would be really really useful for addressing http://llvm.org/PR37871 so I'd like to have this.

The llvm-mca output typically consists of logical "blocks" of info delimited by empty lines:

Foo
Bar
Baz

Wibble
Wobble

so our tests model this like:

CHECK:      Foo
CHECK-NEXT: Bar
CHECK-NEXT: Baz

CHECK:      Wibble
CHECK-NEXT: Wobble

which is fine for making sure that the contents of each of these blocks is correct but will miss the case where there is something unexpected in between those blocks. Being able to instead say:

CHECK:      Foo
CHECK-NEXT: Bar
CHECK-NEXT: Baz

CHECK-EMPTY:

CHECK-NEXT: Wibble
CHECK-NEXT: Wobble

would be very useful!

[probinson]

Bowing to popular demand.

test/FileCheck/check-empty-tag.txt

13 ↗

(On Diff #84952)

I believe this CHECK2A: line will match itself, and the "next line" therefore has a CHECK2A-EMPTY directive on it. If you run FileCheck manually I think you'll see that from the line number reported by the error.
I see that you are intending to match a line at the end of this file. To achieve that you want to add some kind of regex to the CHECK2A line. Something like the {{next}} line has spaces would work.

utils/FileCheck/FileCheck.cpp

470 ↗

(On Diff #84952)

Spurious whitespace difference.

755 ↗

(On Diff #84952)

Disallow EMPTY-NOT and NOT-EMPTY.

867 ↗

(On Diff #84952)

Spurious reformatting?

[jhenderson]

Address review comments:

• Rebased (no actual changes needed as a result, but it updates the diff context).
• Removed unrelated whitespace changes.
• Move CHECK2 to be the last case in the test, so that it is closer to the strings to be checked.
• Added checking for CHECK-EMPTY-NOT and CHECK-NOT-EMPTY patterns.

test/FileCheck/check-empty-tag.txt

13 ↗

(On Diff #84952)

I believe this CHECK2A: line will match itself, and the "next line" therefore has a CHECK2A-EMPTY directive on it. If you run FileCheck manually I think you'll see that from the line number reported by the error.

I thought of that at the time I originally wrote this, but it doesn't because of the "--match-full-lines" switch, since the "CHECK2A:" prefix prevents matching there.

I'm not sure I follow the need for a regex?

utils/FileCheck/FileCheck.cpp

867 ↗

(On Diff #84952)

This came about because that's what my clang-format did, but you're right, it's not related to this change.

[probinson]

One doc question, otherwise LGTM.

docs/CommandGuide/FileCheck.rst

247 ↗

(On Diff #152881)

Hm. Is whitespace allowed on the "blank" line? Even if --match-full-lines is not in effect? If whitespace is always disallowed, then maybe instead of "is blank" this should say something like "has nothing on it, not even whitespace" ?

test/FileCheck/check-empty-tag.txt

13 ↗

(On Diff #84952)

A regex is a way for a CHECK line to avoid matching itself; you see this in some of the other FileCheck tests. But with --match-full-lines you don't need that trick; sorry I missed that.

[jhenderson]

docs/CommandGuide/FileCheck.rst

247 ↗

(On Diff #152881)

At the moment, whitespace isn't allowed (see case 6 in the test). This could probably be changed, if there's call for it, but I figured that it was simpler this way.

I'll change the wording as suggested.

llvm/trunk/docs/CommandGuide/FileCheck.rst

.. code-block:: llvm
    ; CHECK:       !DILocation(line: 5,
    ; CHECK-NOT:               column:
    ; CHECK-SAME:              scope: ![[SCOPE:[0-9]+]]
 
 "``CHECK-SAME:``" directives reject the input if there are any newlines between
 it and the previous directive.  A "``CHECK-SAME:``" cannot be the first
 directive in a file.
 
+The "CHECK-EMPTY:" directive
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If you need to check that the next line has nothing on it, not even whitespace,
+you can use the "``CHECK-EMPTY:``" directive.
+
+.. code-block:: llvm
+
+   foo
+
+   bar
+   ; CHECK: foo
+   ; CHECK-EMPTY:
+   ; CHECK-NEXT: bar
+
+Just like "``CHECK-NEXT:``" the directive will fail if there is more than one
+newline before it finds the next blank line, and it cannot be the first
+directive in a file.
+
 The "CHECK-NOT:" directive
 ~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 The "``CHECK-NOT:``" directive is used to verify that a string doesn't occur
 between two matches (or before the first match, or after the last match).  For
 example, to verify that a load is removed by a transformation, a test like this
 can be used:
 

llvm/trunk/test/FileCheck/check-empty-tag.txt

+; basic functionality
+; RUN: FileCheck %s --input-file %s --check-prefix=CHECK1
+foo
+
+bar
+CHECK1: foo
+CHECK1-EMPTY:
+CHECK1-NEXT: bar
+
+; next line must be blank
+; RUN: not FileCheck %s --input-file %s --check-prefix=CHECK2A 2>&1 | FileCheck %s --check-prefix=CHECK2B
+badger
+CHECK2A: badger
+CHECK2A-EMPTY:
+CHECK2B: CHECK2A-EMPTY: is not on the line after the previous match
+
+; CHECK-EMPTY must have empty pattern
+; RUN: not FileCheck %s --input-file %s --check-prefix=CHECK3A 2>&1 | FileCheck %s --check-prefix=CHECK3B
+CHECK3A: foo
+CHECK3A-EMPTY: this is not empty
+CHECK3B: found non-empty check string for empty check with prefix 'CHECK3A:'
+
+; CHECK-EMPTY cannot be the first check
+; RUN: not FileCheck %s --input-file %s --check-prefix=CHECK4A 2>&1 | FileCheck %s --check-prefix=CHECK4B
+CHECK4A-EMPTY:
+CHECK4B: found 'CHECK4A-EMPTY' without previous 'CHECK4A: line
+
+; CHECK-EMPTY-NOT and CHECK-NOT-EMPTY rejected
+; RUN: not FileCheck %s --input-file %s --check-prefixes=CHECK5A 2>&1 | FileCheck %s --check-prefix=CHECK5C
+; RUN: not FileCheck %s --input-file %s --check-prefixes=CHECK5B 2>&1 | FileCheck %s --check-prefix=CHECK5C
+CHECK5A-EMPTY-NOT:
+CHECK5B-NOT-EMPTY:
+CHECK5C: unsupported -NOT combo on prefix 'CHECK5{{A|B}}'
+
+; whitespace does not count as empty
+; RUN: not FileCheck %s --input-file %s --check-prefix=CHECK6A --match-full-lines 2>&1 | FileCheck %s --check-prefix=CHECK6B
+CHECK6A: the next line has spaces
+CHECK6A-EMPTY:
+CHECK6B: expected string not found in input
+
+; ***don't add any further blank lines after this point***
+; CHECK-EMPTY, like CHECK-NEXT, will report an error if the first matching
+; line is not the line immediately following the previous check.
+the next line has spaces
+  
 No newline at end of file

llvm/trunk/utils/FileCheck/FileCheck.cpp

 enum CheckType {
   CheckNone = 0,
   CheckPlain,
   CheckNext,
   CheckSame,
   CheckNot,
   CheckDAG,
   CheckLabel,
+  CheckEmpty,
 
   /// Indicates the pattern only matches the end of file. This is used for
   /// trailing CHECK-NOTs.
   CheckEOF,
 
   /// Marks when parsing found a -NOT check combined with another CHECK suffix.
   CheckBadNot
 };
bool Pattern::ParsePattern(StringRef PatternStr, StringRef Prefix,
 
   if (!(NoCanonicalizeWhiteSpace && MatchFullLines))
     // Ignore trailing whitespace.
     while (!PatternStr.empty() &&
            (PatternStr.back() == ' ' || PatternStr.back() == '\t'))
       PatternStr = PatternStr.substr(0, PatternStr.size() - 1);
 
   // Check that there is something on the line.
-  if (PatternStr.empty()) {
+  if (PatternStr.empty() && CheckTy != Check::CheckEmpty) {
     SM.PrintMessage(PatternLoc, SourceMgr::DK_Error,
                     "found empty check string with prefix '" + Prefix + ":'");
     return true;
   }
 
+  if (!PatternStr.empty() && CheckTy == Check::CheckEmpty) {
+    SM.PrintMessage(
+        PatternLoc, SourceMgr::DK_Error,
+        "found non-empty check string for empty check with prefix '" + Prefix +
+            ":'");
+    return true;
+  }
+
+  if (CheckTy == Check::CheckEmpty) {
+    RegExStr = "(\n$)";
+    return false;
+  }
+
   // Check to see if this is a fixed string, or if it has regex pieces.
   if (!MatchFullLinesHere &&
       (PatternStr.size() < 2 || (PatternStr.find("{{") == StringRef::npos &&
                                  PatternStr.find("[[") == StringRef::npos))) {
     FixedStr = PatternStr;
     return false;
   }
 
case Check::CheckNot:
     return sizeof("-NOT:") - 1;
 
   case Check::CheckDAG:
     return sizeof("-DAG:") - 1;
 
   case Check::CheckLabel:
     return sizeof("-LABEL:") - 1;
 
+  case Check::CheckEmpty:
+    return sizeof("-EMPTY:") - 1;
+
   case Check::CheckEOF:
     llvm_unreachable("Should not be using EOF size");
   }
 
   llvm_unreachable("Bad check type");
 }
 
 static Check::CheckType FindCheckType(StringRef Buffer, StringRef Prefix) {
if (Rest.startswith("NOT:"))
     return Check::CheckNot;
 
   if (Rest.startswith("DAG:"))
     return Check::CheckDAG;
 
   if (Rest.startswith("LABEL:"))
     return Check::CheckLabel;
 
+  if (Rest.startswith("EMPTY:"))
+    return Check::CheckEmpty;
+
   // You can't combine -NOT with another suffix.
   if (Rest.startswith("DAG-NOT:") || Rest.startswith("NOT-DAG:") ||
       Rest.startswith("NEXT-NOT:") || Rest.startswith("NOT-NEXT:") ||
-      Rest.startswith("SAME-NOT:") || Rest.startswith("NOT-SAME:"))
+      Rest.startswith("SAME-NOT:") || Rest.startswith("NOT-SAME:") ||
+      Rest.startswith("EMPTY-NOT:") || Rest.startswith("NOT-EMPTY:"))
     return Check::CheckBadNot;
 
   return Check::CheckNone;
 }
 
 // From the given position, find the next character after the word.
 static size_t SkipWord(StringRef Str, size_t Loc) {
   while (Loc < Str.size() && IsPartOfWord(Str[Loc]))
if ((CheckTy == Check::CheckLabel) && P.hasVariable()) {
           SMLoc::getFromPointer(UsedPrefixStart), SourceMgr::DK_Error,
           "found '" + UsedPrefix + "-LABEL:'"
                                    " with variable definition or use");
       return true;
     }
 
     Buffer = Buffer.substr(EOL);
 
-    // Verify that CHECK-NEXT lines have at least one CHECK line before them.
-    if ((CheckTy == Check::CheckNext || CheckTy == Check::CheckSame) &&
+    // Verify that CHECK-NEXT/SAME/EMPTY lines have at least one CHECK line before them.
+    if ((CheckTy == Check::CheckNext || CheckTy == Check::CheckSame ||
+         CheckTy == Check::CheckEmpty) &&
         CheckStrings.empty()) {
-      StringRef Type = CheckTy == Check::CheckNext ? "NEXT" : "SAME";
+      StringRef Type = CheckTy == Check::CheckNext
+                           ? "NEXT"
+                           : CheckTy == Check::CheckEmpty ? "EMPTY" : "SAME";
       SM.PrintMessage(SMLoc::getFromPointer(UsedPrefixStart),
                       SourceMgr::DK_Error,
                       "found '" + UsedPrefix + "-" + Type +
                           "' without previous '" + UsedPrefix + ": line");
       return true;
     }
 
     // Handle CHECK-DAG/-NOT.
if (CheckNot(SM, SkippedRegion, NotStrings, VariableTable))
       return StringRef::npos;
   }
 
   return LastPos + MatchPos;
 }
 
 /// Verify there is a single line in the given buffer.
 bool CheckString::CheckNext(const SourceMgr &SM, StringRef Buffer) const {
-  if (Pat.getCheckTy() != Check::CheckNext)
+  if (Pat.getCheckTy() != Check::CheckNext &&
+      Pat.getCheckTy() != Check::CheckEmpty)
     return false;
 
+  Twine CheckName =
+      Prefix +
+      Twine(Pat.getCheckTy() == Check::CheckEmpty ? "-EMPTY" : "-NEXT");
+
   // Count the number of newlines between the previous match and this one.
   assert(Buffer.data() !=
              SM.getMemoryBuffer(SM.FindBufferContainingLoc(
                                     SMLoc::getFromPointer(Buffer.data())))
                  ->getBufferStart() &&
-         "CHECK-NEXT can't be the first check in a file");
+         "CHECK-NEXT and CHECK-EMPTY can't be the first check in a file");
 
   const char *FirstNewLine = nullptr;
   unsigned NumNewLines = CountNumNewlinesBetween(Buffer, FirstNewLine);
 
+  // For CHECK-EMPTY, the preceding new line is consumed by the pattern, so
+  // this needs to be re-added.
+  if (Pat.getCheckTy() == Check::CheckEmpty)
+    ++NumNewLines;
+
   if (NumNewLines == 0) {
     SM.PrintMessage(Loc, SourceMgr::DK_Error,
-                    Prefix + "-NEXT: is on the same line as previous match");
+                    CheckName + ": is on the same line as previous match");
     SM.PrintMessage(SMLoc::getFromPointer(Buffer.end()), SourceMgr::DK_Note,
                     "'next' match was here");
     SM.PrintMessage(SMLoc::getFromPointer(Buffer.data()), SourceMgr::DK_Note,
                     "previous match ended here");
     return true;
   }
 
   if (NumNewLines != 1) {
     SM.PrintMessage(Loc, SourceMgr::DK_Error,
-                    Prefix +
-                        "-NEXT: is not on the line after the previous match");
+                    CheckName +
+                        ": is not on the line after the previous match");
     SM.PrintMessage(SMLoc::getFromPointer(Buffer.end()), SourceMgr::DK_Note,
                     "'next' match was here");
     SM.PrintMessage(SMLoc::getFromPointer(Buffer.data()), SourceMgr::DK_Note,
                     "previous match ended here");
     SM.PrintMessage(SMLoc::getFromPointer(FirstNewLine), SourceMgr::DK_Note,
                     "non-matching line after previous match is here");
     return true;
   }