FileCheck Enhancement - Including files

[eklepilkina]

Summary

This patch adds new directive CHECK-INCLUDE - directive to include other file with checks to another.

[vsk]

Looks super useful :). I have some initial comments -- please clang-format and upload a diff with more context and I'll take a closer look.

utils/FileCheck/FileCheck.cpp

38

This seems a bit arbitrary. Why not use a DFS? E.g;

containsIncludeCycle(MainFile, IncludeFile) {
  if (Includes[IncludeFile].contains(MainFile)) return true
  for (Include in Includes[IncludeFile]) if (containsIncludeCycle(Include, MainFile)) return true
  return false
}

990

const std::string &Directory to avoid a copy.

991

Use llvm::sys::path::append. Also note that in this line, you're taking a reference to a stack temporary (the result of the std::string concatenation should be destroyed after this line). This looks like a use-after-free.

1003

Is there some way to avoid moving the code into this else branch? If not that's OK. Otherwise, it would really help to keep unmodified code out of this diff (makes it easier to inspect the diff).

[eklepilkina]

utils/FileCheck/FileCheck.cpp

38

May be I didn't understand something, but as I understood next situation will fail. Am I wrong?

1->2;
1->3;
3->3;
2->4;
2->5;
4->1;
5->2;

1003

I understood that it'll be easier to merge, but I don't see correct way to avoid branching.

docs/CommandGuide/FileCheck.rst

 failures to be detected in a single invocation.
 
 There is no requirement that ``CHECK-LABEL:`` directives contain strings that
 correspond to actual syntactic labels in a source or output language: they must
 simply uniquely match a single line in the file being verified.
 
 ``CHECK-LABEL:`` directives cannot contain variable definitions or uses.
 
+The "CHECK-LABEL:" directive
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Some checks may be used several times. Then these checks may be described in
+separate file. Then this file can be included by directive // CHECK-INCLUDE: filename.
+To add directory where included files would be searching in, user should use –I option.
+
+.. code-block:: llvm
+    // RUN: FileCheck -I=$LLVM_SRC_ROOT/test -input-file %s %s
+
+
 FileCheck Pattern Matching Syntax
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 All FileCheck directives take a pattern to match.
 For most uses of FileCheck, fixed string matching is perfectly sufficient.  For
 some things, a more flexible form of matching is desired.  To support this,
 FileCheck allows you to specify regular expressions in matching strings,
 surrounded by double braces: ``{{yourregex}}``.  Because we want to use fixed

test/FileCheck/check-include.txt

+// RUN: FileCheck -I=$LLVM_SRC_ROOT/test -input-file %s %s
+
+op1;
+op2;
+op33;
+
+first and
+This is second line
+
+one two three
+
+four!
+dog
+line
+op0xA1;
+op0xc2;
+op0x33;
+reg0x0
+// CHECK-INCLUDE: FileCheck/FileCheckImprovement/included.txt
+ADD r0xA, reg0x10
+// CHECK: {{ADD r0xA, reg0x10}}
+

test/FileCheck/included.txt

+// CHECK: op{{[0-9]}};
+// CHECK-NEXT: op{{[0-9]}};
+// CHECK-NOT: op1{{[0-9]}}
+// CHECK: first
+// CHECK-SAME: and
+// CHECK: second
+// CHECK-DAG: two
+// CHECK-DAG: three
+// CHECK-DAG: one
+// CHECK: four
+// CHECK-NOT: cat
+// CHECK: line

utils/FileCheck/FileCheck.cpp

 //===----------------------------------------------------------------------===//
 
 #include "llvm/ADT/SmallString.h"
 #include "llvm/ADT/StringExtras.h"
 #include "llvm/ADT/StringMap.h"
 #include "llvm/ADT/StringSet.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/MemoryBuffer.h"
+#include "llvm/Support/Path.h"
 #include "llvm/Support/PrettyStackTrace.h"
 #include "llvm/Support/Regex.h"
 #include "llvm/Support/Signals.h"
 #include "llvm/Support/SourceMgr.h"
 #include "llvm/Support/raw_ostream.h"
 #include <algorithm>
 #include <cctype>
 #include <map>
 #include <string>
 #include <system_error>
 #include <vector>
+
+#define MAX_INCLUDING_DEPTH 20 // maximum depth for including files.
+
 using namespace llvm;
 
 static cl::opt<std::string>
 CheckFilename(cl::Positional, cl::desc("<check-file>"), cl::Required);
 
 static cl::opt<std::string>
 InputFilename("input-file", cl::desc("File to check (defaults to stdin)"),
               cl::init("-"), cl::value_desc("filename"));
cl::desc("Allow the input file to be empty. This is useful when making\n"
              "checks that some error message does not occur, for example."));
 
 static cl::opt<bool> MatchFullLines(
     "match-full-lines", cl::init(false),
     cl::desc("Require all positive matches to cover an entire input line.\n"
              "Allows leading and trailing whitespace if --strict-whitespace\n"
              "is not also passed."));
 
+static cl::list<std::string> IncludedDirectories(
+    "I",
+    cl::desc("Add directories for searching included files in CHECK-INCLUDE"));
+
 typedef cl::list<std::string>::const_iterator prefix_iterator;
 
 //===----------------------------------------------------------------------===//
 // Pattern Handling Code.
 //===----------------------------------------------------------------------===//
 
 namespace Check {
   enum CheckType {
     CheckNone = 0,
     CheckPlain,
     CheckNext,
     CheckSame,
     CheckNot,
     CheckDAG,
     CheckLabel,
+    CheckInclude,
 
     /// MatchEOF - When set, this pattern only matches the end of file. This is
     /// used for trailing CHECK-NOTs.
     CheckEOF,
     /// CheckBadNot - Found -NOT combined with another CHECK suffix.
     CheckBadNot
   };
 }

 public:
 
   Pattern(Check::CheckType Ty)
     : CheckTy(Ty) { }
 
   /// getLoc - Return the location in source code.
   SMLoc getLoc() const { return PatternLoc; }
 
+  StringRef getFixedString() const { return FixedStr; }
+
   /// ParsePattern - Parse the given string into the Pattern. Prefix provides
   /// which prefix is being matched, SM provides the SourceMgr used for error
   /// reports, and LineNumber is the line number in the input file from which
   /// the pattern string was read.  Returns true in case of an error, false
   /// otherwise.
   bool ParsePattern(StringRef PatternStr,
                     StringRef Prefix,
                     SourceMgr &SM,
case Check::CheckNot:
     return sizeof("-NOT:") - 1;
 
   case Check::CheckDAG:
     return sizeof("-DAG:") - 1;
 
   case Check::CheckLabel:
     return sizeof("-LABEL:") - 1;
 
+  case Check::CheckInclude:
+    return sizeof("-INCLUDE:") - 1;
+
   case Check::CheckEOF:
     llvm_unreachable("Should not be using EOF size");
   }
 
   llvm_unreachable("Bad check type");
 }
 
 static Check::CheckType FindCheckType(StringRef Buffer, StringRef Prefix) {
if (Rest.startswith("NOT:"))
     return Check::CheckNot;
 
   if (Rest.startswith("DAG:"))
     return Check::CheckDAG;
 
   if (Rest.startswith("LABEL:"))
     return Check::CheckLabel;
 
+  if (Rest.startswith("INCLUDE:"))
+    return Check::CheckInclude;
+
   // You can't combine -NOT with another suffix.
   if (Rest.startswith("DAG-NOT:") || Rest.startswith("NOT-DAG:") ||
       Rest.startswith("NEXT-NOT:") || Rest.startswith("NOT-NEXT:") ||
       Rest.startswith("SAME-NOT:") || Rest.startswith("NOT-SAME:"))
     return Check::CheckBadNot;
 
   return Check::CheckNone;
 }
static StringRef FindFirstMatchingPrefix(StringRef &Buffer,
 
   return StringRef();
 }
 
 /// ReadCheckFile - Read the check file, which specifies the sequence of
 /// expected strings.  The strings are added to the CheckStrings vector.
 /// Returns true in case of an error, false otherwise.
 static bool ReadCheckFile(SourceMgr &SM,
-                          std::vector<CheckString> &CheckStrings) {
+                          std::vector<CheckString> &CheckStrings,
+                          std::string Filename = CheckFilename, 
+                          int IncludingLevel = 0) {
   ErrorOr<std::unique_ptr<MemoryBuffer>> FileOrErr =
-      MemoryBuffer::getFileOrSTDIN(CheckFilename);
+      MemoryBuffer::getFileOrSTDIN(Filename);
   if (std::error_code EC = FileOrErr.getError()) {
-    errs() << "Could not open check file '" << CheckFilename
+    errs() << "Could not open check file '" << Filename
            << "': " << EC.message() << '\n';
     return true;
   }
 
   // If we want to canonicalize whitespace, strip excess whitespace from the
   // buffer containing the CHECK lines. Remove DOS style line endings.
   std::unique_ptr<MemoryBuffer> F = CanonicalizeInputFile(
       std::move(FileOrErr.get()), NoCanonicalizeWhiteSpace);
while (1) {
     // Remember the location of the start of the pattern, for diagnostics.
     SMLoc PatternLoc = SMLoc::getFromPointer(Buffer.data());
 
     // Parse the pattern.
     Pattern P(CheckTy);
     if (P.ParsePattern(Buffer.substr(0, EOL), UsedPrefix, SM, LineNumber))
       return true;
 
+    // if included another file
+    if (CheckTy == Check::CheckInclude) {
+      // Check for escaping infinite including
+      if (IncludingLevel > MAX_INCLUDING_DEPTH) {
+        SM.PrintMessage(SMLoc::getFromPointer(Buffer.data()),
+                        SourceMgr::DK_Error,
+                        "Maximum including level!");
+        return true;
+      }
+      // Get included file name.
+      StringRef IncludedFileName = P.getFixedString();
+
+      std::vector<CheckString> IncludedCheckStrings;
+      // Try find file in included directories.
+      ErrorOr<std::unique_ptr<MemoryBuffer>> FileOrErr =
+          MemoryBuffer::getFileOrSTDIN(StringRef(IncludedFileName));
+      if (FileOrErr.getError()) {
+        for (const std::string &Directory : IncludedDirectories) {
+          SmallString<32> FullIncludedFileName;
+          FullIncludedFileName.append(Directory);
+          sys::path::append(FullIncludedFileName, IncludedFileName.data());
+          FileOrErr = MemoryBuffer::getFileOrSTDIN(FullIncludedFileName);
+          if (!FileOrErr.getError()) {
+            IncludedFileName = FullIncludedFileName.str();
+            break;
+          }
+        }
+      }
+      
+      // Read it and get check strings from it.
+      ReadCheckFile(SM, IncludedCheckStrings, IncludedFileName,
+                    IncludingLevel + 1);
+      CheckStrings.insert(CheckStrings.end(), IncludedCheckStrings.begin(),
+                          IncludedCheckStrings.end());
+    } else {
+
       // Verify that CHECK-LABEL lines do not define or use variables
       if ((CheckTy == Check::CheckLabel) && P.hasVariable()) {
         SM.PrintMessage(SMLoc::getFromPointer(UsedPrefixStart),
                         SourceMgr::DK_Error,
                         "found '" + UsedPrefix + "-LABEL:'"
                         " with variable definition or use");
         return true;
       }
 
       Buffer = Buffer.substr(EOL);
 
       // Verify that CHECK-NEXT lines have at least one CHECK line before them.
       if ((CheckTy == Check::CheckNext || CheckTy == Check::CheckSame) &&
           CheckStrings.empty()) {
         StringRef Type = CheckTy == Check::CheckNext ? "NEXT" : "SAME";
         SM.PrintMessage(SMLoc::getFromPointer(UsedPrefixStart),
                         SourceMgr::DK_Error,
                         "found '" + UsedPrefix + "-" + Type + "' without previous '"
                         + UsedPrefix + ": line");
         return true;
       }
 
       // Handle CHECK-DAG/-NOT.
       if (CheckTy == Check::CheckDAG || CheckTy == Check::CheckNot) {
         DagNotMatches.push_back(P);
         continue;
       }
 
       // Okay, add the string we captured to the output vector and move on.
       CheckStrings.emplace_back(P, UsedPrefix, PatternLoc);
       std::swap(DagNotMatches, CheckStrings.back().DagNotStrings);
       DagNotMatches = ImplicitNegativeChecks;
     }
+  }
 
   // Add an EOF pattern for any trailing CHECK-DAG/-NOTs, and use the first
   // prefix as a filler for the error message.
   if (!DagNotMatches.empty()) {
     CheckStrings.emplace_back(Pattern(Check::CheckEOF), *CheckPrefixes.begin(),
                               SMLoc::getFromPointer(Buffer.data()));
     std::swap(DagNotMatches, CheckStrings.back().DagNotStrings);
   }