[Tablegen/DAG Debug] - Instrumenting table gen DAGGenISelDAG to allow printing selected patterns(and their sources) during ISEL

[aditya_nandakumar]

Summary

To help assist in debugging ISEL or to prioritize GlobalISel backend work, this patch adds two more tables to <Target>GenISelDAGISel.inc - one which contains the patterns that are used during selection (The text comment Src and Dst comment you see in MatcherTable for each of the MorphNode and EmitNode cases.
It adds another Tablegen OpcodeType called OPC_Coverage. We encode the offset of the pattern (in the table) with OPC_Coverage.
It also modifies the CheckComplexPattern to also print the ComplexPattern that returned true.
Sometimes, due to several layers of defm instantiations, to make it easier to identify where a particular pattern came from, the source locations have also been encoded in another table. This would get printed as

COMPLEX_PATTERN: SelectNegArithImmed
MORPHTO: (add:i32 GPR32:i32:$Rn, neg_addsub_shifted_imm32:i32:$imm) -> (SUBSWri:i32:i32 GPR32:i32:$Rn, neg_addsub_shifted_imm32:i32:$imm)
INCLUDED: /Volumes/Data/workspace/gpu/compiler_checkouts/open_source/llvm/lib/Target/AArch64/AArch64InstrInfo.td:659
.....

It's currently enabled through CMAKE variable DAG_PRINT_USED_ISEL_PATTERNS which passes a -instrument-coverage flag to table gen for emitting the *GenDAGISel.inc file.
This information can also be used (with a python script which greps/counts patterns) for building a test-suite(such as llvm-test-suite) and see which instructions/patterns need to be selected (say for global isel work prioritization) and can also print the tablegen emitted patterns that are unused during the compilation of the llvm-test-suite.

Looking forward to feedback.

[ab]

A few minor nitpicks, but this seems like a nice feature!

CMakeLists.txt

177

I'd still prefix the name with 'LLVM_ENABLE_': it makes it easier to figure out where individual options come from in ccmake or CMakeCache.txt.

Maybe LLVM_ENABLE_DAGISEL_COV or something?

179

Maybe do this directly in TableGen.cmake, appending to LLVM_TABLEGEN_FLAGS? That way we don't need to change AArch64/CMakeLists.txt.

In theory, we should only add it to -gen-dagisel invocation, but in practice, there's only one binary, so the flag will just be useless on the other invocations.
You can fix that too by only adding -instrument-coverage to the flags inside tablegen(), only if ${ARGN} contains '-gen-dag-isel'.

utils/TableGen/DAGISelMatcherEmitter.cpp

62

If we never look into the components, maybe just concatenate the strings into one when you create them and avoid the std::pair?

68

While this isn't a huge deal (because it's a debug feature), the complexity of std::find makes me twitchy.

Maybe use a UniqueVector or std::lower_bound?

142

Print -> Emit

Also, can you declare it with the other Emit* functions above, and define the body elsewhere in the file?

148

Can you use the standard naming style? (index -> Index; here and elsewhere)

149

"char*" -> "char *" (here and elsewhere)

163

Since these tables are defined inside the functions, maybe remove the Prefix and give them a regular name?

700

NumCoverageBytes or something? Gets rid of the magic '* 3' later.

808

Give it a name? Maybe "Succeeded" or something?

[dsanders]

It will LGTM with some nits fixed in addition to the ones Ahmed mentioned.

CMakeLists.txt

177

I had a similar comment typed but the name you suggest is better.

include/llvm/CodeGen/SelectionDAGISel.h

218–226

SelectionDAGISel is already abstract (because of Select()) so I think we should use make these pure virtual with '= 0'.

lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp

3500

'Morph' has a particular meaning so I'm not sure we should use it for both OPC_EmitNode* and OPC_MorphNodeTo*. I'd suggest 'COVERED: ' or similar.

utils/TableGen/DAGISelMatcherEmitter.cpp

62

If we do need to keep the std::pair<> then I think there ought to be a comment here indicating that the first string is the src pattern and the second is the dst patten.

68

Do we really need to de-dupe them? I wouldn't expect there to be duplicates since this indicates redundant patterns in the tablegen files and the MatcherTable entries for one of them will be unreachable.

144

isUInt<16>(VecPatternsCovered.size())

148

Could you also add the missing whitespace before the '{' and emit indentation to the tablegen-erated file?

154–155

I'd make this unconditional since trailing commas are ok

167–168

Similarly, I'd make this unconditional.

622

Personally I prefer underscored names but we should use CamelCase. Similarly below.

[ab]

utils/TableGen/DAGISelMatcherEmitter.cpp

68

You're absolutely right, seems like this should just be a vector.

[aditya_nandakumar]

Will post the updated diff shortly.

CMakeLists.txt

177

Thanks. That sounds much better than what I picked.

179

Makes sense (I was initially hesitating to add to that function) . I'll go ahead and add this.

include/llvm/CodeGen/SelectionDAGISel.h

218–226

Currently Tablegen emits definitions for those two functions only when -instrument-coverage is passed in. This means if we make it pure virtual, then those two methods would be unimplemented by the <Target>DAGISels when -instrument-coverage is not passed in.

utils/TableGen/DAGISelMatcherEmitter.cpp

68

The problem I saw was that EmitMatcherList was called several times (since we need child and offset of failure code before emitting either of them, they buffer the output to a temporary string). I noticed w/o deduplication - the same patterns show up several times.
Not sure if lower_bound will work as ordering by '<' for strings seem weird?

There's no reason to use pair currently - I'll just create the concatenated strings directly and have a vector<string>

[aditya_nandakumar]

Updated based on feedback.

[dsanders]

include/llvm/CodeGen/SelectionDAGISel.h

218–226

Good point. Tablegen could emit the llvm_unreachable version when not given -instrument-coverage but there's no strong reason to to that over what you have here.

utils/TableGen/DAGISelMatcherEmitter.cpp

68

The problem I saw was that EmitMatcherList was called several times (since we need child
and offset of failure code before emitting either of them, they buffer the output to a
temporary string). I noticed w/o deduplication - the same patterns show up several times.

Ah ok, I see where the dupes come from now. In that case I'd suggest having some kind of map/set to make the dupe check less expensive for large numbers of patterns. MapVector might be a good choice given that we need to refer to the strings by their index. Using something else would mean we have to maintain the vector and map/set separately.

Not sure if lower_bound will work as ordering by '<' for strings seem weird?

The assembly parser uses lower_bound for the mnemonic but the catch is that the vector has to be sorted for it to work correctly.

[aditya_nandakumar]

Updated based on feedback. Used MapVector

[dsanders]

LGTM

CMakeLists.txt

if(LLVM_DEPENDENCY_DEBUGGING)
   endif()
 endif()
 
 option(LLVM_BUILD_GLOBAL_ISEL "Experimental: Build GlobalISel" ON)
 if(LLVM_BUILD_GLOBAL_ISEL)
   add_definitions(-DLLVM_BUILD_GLOBAL_ISEL)
 endif()
 
+option(LLVM_ENABLE_DAGISEL_COV "Debug: Prints tablegen patterns that were used for selecting" OFF)
+
 # Add path for custom modules
 set(CMAKE_MODULE_PATH
   ${CMAKE_MODULE_PATH}
   "${CMAKE_CURRENT_SOURCE_DIR}/cmake"
   "${CMAKE_CURRENT_SOURCE_DIR}/cmake/modules"
   )
 
 # Generate a CompilationDatabase (compile_commands.json file) for our build,
 # for use by clang_complete, YouCompleteMe, etc.

cmake/modules/TableGen.cmake

function(tablegen project ofn)
   file(GLOB_RECURSE global_tds "${LLVM_MAIN_INCLUDE_DIR}/llvm/*.td")
 
   if (IS_ABSOLUTE ${LLVM_TARGET_DEFINITIONS})
     set(LLVM_TARGET_DEFINITIONS_ABSOLUTE ${LLVM_TARGET_DEFINITIONS})
   else()
     set(LLVM_TARGET_DEFINITIONS_ABSOLUTE
       ${CMAKE_CURRENT_SOURCE_DIR}/${LLVM_TARGET_DEFINITIONS})
   endif()
+  if (LLVM_ENABLE_DAGISEL_COV)
+    list(FIND ARGN "-gen-dag-isel" idx)
+    if( NOT idx EQUAL -1 )
+      list(APPEND LLVM_TABLEGEN_FLAGS "-instrument-coverage")
+    endif()
+  endif()
+
   add_custom_command(OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/${ofn}.tmp
     # Generate tablegen output in a temporary file.
     COMMAND ${${project}_TABLEGEN_EXE} ${ARGN} -I ${CMAKE_CURRENT_SOURCE_DIR}
     ${LLVM_TABLEGEN_FLAGS} 
     ${LLVM_TARGET_DEFINITIONS_ABSOLUTE}
     -o ${CMAKE_CURRENT_BINARY_DIR}/${ofn}.tmp
     # The file in LLVM_TARGET_DEFINITIONS may be not in the current
     # directory and local_tds may not contain it, so we must

include/llvm/CodeGen/SelectionDAGISel.h

enum BuiltinOpcodes {
     OPC_EmitCopyToReg,
     OPC_EmitNodeXForm,
     OPC_EmitNode,
     // Space-optimized forms that implicitly encode number of result VTs.
     OPC_EmitNode0, OPC_EmitNode1, OPC_EmitNode2,
     OPC_MorphNodeTo,
     // Space-optimized forms that implicitly encode number of result VTs.
     OPC_MorphNodeTo0, OPC_MorphNodeTo1, OPC_MorphNodeTo2,
-    OPC_CompleteMatch
+    OPC_CompleteMatch,
+    // Contains offset in table for pattern being selected
+    OPC_Coverage
   };
 
   enum {
     OPFL_None       = 0,  // Node has no chain or glue input and isn't variadic.
     OPFL_Chain      = 1,     // Node has a chain input.
     OPFL_GlueInput  = 2,     // Node has a glue input.
     OPFL_GlueOutput = 4,     // Node has a glue output.
     OPFL_MemRefs    = 8,     // Node gets accumulated MemRefs.
void ReplaceNode(SDNode *F, SDNode *T) {
     CurDAG->RemoveDeadNode(F);
   }
 
   /// SelectInlineAsmMemoryOperands - Calls to this are automatically generated
   /// by tblgen.  Others should not call it.
   void SelectInlineAsmMemoryOperands(std::vector<SDValue> &Ops,
                                      const SDLoc &DL);
 
+  /// getPatternForIndex - Patterns selected by tablegen during ISEL
+  virtual StringRef getPatternForIndex(unsigned index) {
+    llvm_unreachable("Tblgen should generate the implementation of this!");
+  }
+
+  /// getIncludePathForIndex - get the td source location of pattern instantiation
+  virtual StringRef getIncludePathForIndex(unsigned index) {
+    llvm_unreachable("Tblgen should generate the implementation of this!");
+  }
 public:
   // Calls to these predicates are generated by tblgen.
   bool CheckAndMask(SDValue LHS, ConstantSDNode *RHS,
                     int64_t DesiredMaskS) const;
   bool CheckOrMask(SDValue LHS, ConstantSDNode *RHS,
                     int64_t DesiredMaskS) const;
 
 

lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp

#endif
     case OPC_EmitNodeXForm: {
       unsigned XFormNo = MatcherTable[MatcherIndex++];
       unsigned RecNo = MatcherTable[MatcherIndex++];
       assert(RecNo < RecordedNodes.size() && "Invalid EmitNodeXForm");
       SDValue Res = RunSDNodeXForm(RecordedNodes[RecNo].first, XFormNo);
       RecordedNodes.push_back(std::pair<SDValue,SDNode*>(Res, nullptr));
       continue;
     }
+    case OPC_Coverage: {
+      // This is emitted right before MorphNode/EmitNode.
+      // So it should be safe to assume that this node has been selected
+      unsigned index = MatcherTable[MatcherIndex++];
+      index |= (MatcherTable[MatcherIndex++] << 8);
+      dbgs() << "COVERED: " << getPatternForIndex(index) << "\n";
+      dbgs() << "INCLUDED: " << getIncludePathForIndex(index) << "\n";
+      continue;
+    }
 
     case OPC_EmitNode:     case OPC_MorphNodeTo:
     case OPC_EmitNode0:    case OPC_EmitNode1:    case OPC_EmitNode2:
     case OPC_MorphNodeTo0: case OPC_MorphNodeTo1: case OPC_MorphNodeTo2: {
       uint16_t TargetOpc = MatcherTable[MatcherIndex++];
       TargetOpc |= (unsigned short)MatcherTable[MatcherIndex++] << 8;
       unsigned EmitNodeInfo = MatcherTable[MatcherIndex++];
       // Get the result VT list.

utils/TableGen/DAGISelMatcherEmitter.cpp

 //===- DAGISelMatcherEmitter.cpp - Matcher Emitter ------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // This file contains code to generate C++ code for a matcher.
 //
 //===----------------------------------------------------------------------===//
 
-#include "DAGISelMatcher.h"
 #include "CodeGenDAGPatterns.h"
+#include "DAGISelMatcher.h"
 #include "llvm/ADT/DenseMap.h"
 #include "llvm/ADT/SmallString.h"
 #include "llvm/ADT/StringMap.h"
 #include "llvm/ADT/TinyPtrVector.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/FormattedStream.h"
+#include "llvm/Support/SourceMgr.h"
+#include "llvm/TableGen/Error.h"
 #include "llvm/TableGen/Record.h"
 using namespace llvm;
 
 enum {
   CommentIndent = 30
 };
 
 // To reduce generated source code size.
 static cl::opt<bool>
 OmitComments("omit-comments", cl::desc("Do not generate comments"),
              cl::init(false));
 
+static cl::opt<bool> InstrumentCoverage(
+    "instrument-coverage",
+    cl::desc("Generates tables to help identify patterns matched"),
+    cl::init(false));
+
 namespace {
 class MatcherTableEmitter {
   const CodeGenDAGPatterns &CGP;
   
   DenseMap<TreePattern *, unsigned> NodePredicateMap;
   std::vector<TreePredicateFn> NodePredicates;
 
   // We de-duplicate the predicates by code string, and use this map to track
   // all the patterns with "identical" predicates.
   StringMap<TinyPtrVector<TreePattern *>> NodePredicatesByCodeToRun;
   
   StringMap<unsigned> PatternPredicateMap;
   std::vector<std::string> PatternPredicates;
 
   DenseMap<const ComplexPattern*, unsigned> ComplexPatternMap;
   std::vector<const ComplexPattern*> ComplexPatterns;
 
 
   DenseMap<Record*, unsigned> NodeXFormMap;
   std::vector<Record*> NodeXForms;
 
+  std::vector<std::string> VecPatternsCovered;
+  std::vector<std::string> VecIncludeStrings;
+
+  unsigned getPatternIdxFromTable(std::string &&P, std::string &&include_loc) {
+    auto It =
+        std::find(VecPatternsCovered.begin(), VecPatternsCovered.end(), P);
+    if (It == VecPatternsCovered.end()) {
+      VecPatternsCovered.push_back(std::move(P));
+      VecIncludeStrings.push_back(std::move(include_loc));
+      return VecPatternsCovered.size() - 1;
+    }
+    return std::distance(VecPatternsCovered.begin(), It);
+  }
+
 public:
   MatcherTableEmitter(const CodeGenDAGPatterns &cgp)
     : CGP(cgp) {}
 
   unsigned EmitMatcherList(const Matcher *N, unsigned Indent,
                            unsigned StartIdx, formatted_raw_ostream &OS);
 
   void EmitPredicateFunctions(formatted_raw_ostream &OS);
 
   void EmitHistogram(const Matcher *N, formatted_raw_ostream &OS);
+
+  void EmitPatternMatchTable(raw_ostream &OS);
+
 private:
   unsigned EmitMatcher(const Matcher *N, unsigned Indent, unsigned CurrentIdx,
                        formatted_raw_ostream &OS);
 
   unsigned getNodePredicate(TreePredicateFn Pred) {
     TreePattern *TP = Pred.getOrigPatFragRecord();
     unsigned &Entry = NodePredicateMap[TP];
     if (Entry == 0) {
unsigned getNodeXFormID(Record *Rec) {
     if (Entry == 0) {
       NodeXForms.push_back(Rec);
       Entry = NodeXForms.size();
     }
     return Entry-1;
   }
 
 };
 } // end anonymous namespace.
 
+static std::string GetPatFromTreePatternNode(const TreePatternNode *N) {
+  std::string str;
+  raw_string_ostream Stream(str);
+  Stream << *N;
+  Stream.str();
+  return str;
+}
+
 static unsigned GetVBRSize(unsigned Val) {
   if (Val <= 127) return 1;
 
   unsigned NumBytes = 0;
   while (Val >= 128) {
     Val >>= 7;
     ++NumBytes;
   }
   return NumBytes+1;
 }
 
 /// EmitVBRValue - Emit the specified value as a VBR, returning the number of
 /// bytes emitted.
 static uint64_t EmitVBRValue(uint64_t Val, raw_ostream &OS) {
   if (Val <= 127) {
     OS << Val << ", ";
     return 1;
   }
 
   uint64_t InVal = Val;
   unsigned NumBytes = 0;
   while (Val >= 128) {
     OS << (Val&127) << "|128,";
     Val >>= 7;
     ++NumBytes;
   }
   OS << Val;
   if (!OmitComments)
     OS << "/*" << InVal << "*/";
   OS << ", ";
   return NumBytes+1;
 }
 
+// This is expensive and slow.
+static std::string getIncludePath(const Record *R) {
+  std::string str;
+  raw_string_ostream Stream(str);
+  auto Locs = R->getLoc();
+  SMLoc L;
+  if (Locs.size() > 1) {
+    // Get where the pattern prototype was instantiated
+    L = Locs[1];
+  } else if (Locs.size() == 1) {
+    L = Locs[0];
+  }
+  unsigned CurBuf = SrcMgr.FindBufferContainingLoc(L);
+  assert(CurBuf && "Invalid or unspecified location!");
+
+  Stream << SrcMgr.getBufferInfo(CurBuf).Buffer->getBufferIdentifier() << ":"
+         << SrcMgr.FindLineNumber(L, CurBuf);
+  Stream.str();
+  return str;
+}
+
+void MatcherTableEmitter::EmitPatternMatchTable(raw_ostream &OS) {
+
+  assert(isUInt<16>(VecPatternsCovered.size()) &&
+         "Using only 16 bits to encode offset into Pattern Table");
+  assert(VecPatternsCovered.size() == VecIncludeStrings.size() &&
+         "The sizes of Pattern and include vectors should be the same");
+  OS << "StringRef getPatternForIndex(unsigned Index) override {\n";
+  OS << "static const char * PATTERN_MATCH_TABLE[] = {\n";
+
+  for (auto &It : VecPatternsCovered) {
+    OS << "\"" << It << "\",\n";
+  }
+
+  OS << "\n};";
+  OS << "\nreturn StringRef(PATTERN_MATCH_TABLE[Index]);";
+  OS << "\n}";
+
+  OS << "\nStringRef getIncludePathForIndex(unsigned Index) override {\n";
+  OS << "static const char * INCLUDE_PATH_TABLE[] = {\n";
+
+  for (auto &It : VecIncludeStrings) {
+    OS << "\"" << It << "\",\n";
+  }
+
+  OS << "\n};";
+  OS << "\nreturn StringRef(INCLUDE_PATH_TABLE[Index]);";
+  OS << "\n}";
+}
+
 /// EmitMatcher - Emit bytes for the specified matcher and return
 /// the number of bytes emitted.
 unsigned MatcherTableEmitter::
 EmitMatcher(const Matcher *N, unsigned Indent, unsigned CurrentIdx,
             formatted_raw_ostream &OS) {
   OS.PadToColumn(Indent*2);
 
   switch (N->getKind()) {
case Matcher::EmitNodeXForm: {
     if (!OmitComments)
       OS.PadToColumn(CommentIndent) << "// "<<XF->getNodeXForm()->getName();
     OS <<'\n';
     return 3;
   }
 
   case Matcher::EmitNode:
   case Matcher::MorphNodeTo: {
+    auto NumCoveredBytes = 0;
+    if (InstrumentCoverage) {
+      if (const MorphNodeToMatcher *SNT = dyn_cast<MorphNodeToMatcher>(N)) {
+        NumCoveredBytes = 3;
+        OS << "OPC_Coverage, ";
+        std::string src =
+            GetPatFromTreePatternNode(SNT->getPattern().getSrcPattern());
+        std::string dst =
+            GetPatFromTreePatternNode(SNT->getPattern().getDstPattern());
+        Record *PatRecord = SNT->getPattern().getSrcRecord();
+        std::string include_src = getIncludePath(PatRecord);
+        unsigned Offset =
+            getPatternIdxFromTable(src + " -> " + dst, std::move(include_src));
+        OS << "TARGET_VAL(" << Offset << "),\n";
+        OS.PadToColumn(Indent * 2);
+      }
+    }
     const EmitNodeMatcherCommon *EN = cast<EmitNodeMatcherCommon>(N);
     OS << (isa<EmitNodeMatcher>(EN) ? "OPC_EmitNode" : "OPC_MorphNodeTo");
     bool CompressVTs = EN->getNumVTs() < 3;
     if (CompressVTs)
       OS << EN->getNumVTs();
 
     OS << ", TARGET_VAL(" << EN->getOpcodeName() << "), 0";
 
if (!OmitComments) {
           << *SNT->getPattern().getSrcPattern() << " - Complexity = "
           << SNT->getPattern().getPatternComplexity(CGP) << '\n';
         OS.PadToColumn(Indent*2) << "// Dst: "
           << *SNT->getPattern().getDstPattern() << '\n';
       }
     } else
       OS << '\n';
 
-    return 5 + !CompressVTs + EN->getNumVTs() + NumOperandBytes;
+    return 5 + !CompressVTs + EN->getNumVTs() + NumOperandBytes +
+           NumCoveredBytes;
   }
   case Matcher::CompleteMatch: {
     const CompleteMatchMatcher *CM = cast<CompleteMatchMatcher>(N);
+    auto NumCoveredBytes = 0;
+    if (InstrumentCoverage) {
+      NumCoveredBytes = 3;
+      OS << "OPC_Coverage, ";
+      std::string src =
+          GetPatFromTreePatternNode(CM->getPattern().getSrcPattern());
+      std::string dst =
+          GetPatFromTreePatternNode(CM->getPattern().getDstPattern());
+      Record *PatRecord = CM->getPattern().getSrcRecord();
+      std::string include_src = getIncludePath(PatRecord);
+      unsigned Offset =
+          getPatternIdxFromTable(src + " -> " + dst, std::move(include_src));
+      OS << "TARGET_VAL(" << Offset << "),\n";
+      OS.PadToColumn(Indent * 2);
+    }
     OS << "OPC_CompleteMatch, " << CM->getNumResults() << ", ";
     unsigned NumResultBytes = 0;
     for (unsigned i = 0, e = CM->getNumResults(); i != e; ++i)
       NumResultBytes += EmitVBRValue(CM->getResult(i), OS);
     OS << '\n';
     if (!OmitComments) {
       OS.PadToColumn(Indent*2) << "// Src: "
         << *CM->getPattern().getSrcPattern() << " - Complexity = "
         << CM->getPattern().getPatternComplexity(CGP) << '\n';
       OS.PadToColumn(Indent*2) << "// Dst: "
         << *CM->getPattern().getDstPattern();
     }
     OS << '\n';
-    return 2 + NumResultBytes;
+    return 2 + NumResultBytes + NumCoveredBytes;
   }
   }
   llvm_unreachable("Unreachable");
 }
 
 /// EmitMatcherList - Emit the bytes for the specified matcher subtree.
 unsigned MatcherTableEmitter::
 EmitMatcherList(const Matcher *N, unsigned Indent, unsigned CurrentIdx,
if (!ComplexPatterns.empty()) {
     for (unsigned i = 0, e = ComplexPatterns.size(); i != e; ++i) {
       const ComplexPattern &P = *ComplexPatterns[i];
       unsigned NumOps = P.getNumOperands();
 
       if (P.hasProperty(SDNPHasChain))
         ++NumOps;  // Get the chained node too.
 
       OS << "  case " << i << ":\n";
+      if (InstrumentCoverage)
+        OS << "  {\n";
       OS << "    Result.resize(NextRes+" << NumOps << ");\n";
-      OS << "    return "  << P.getSelectFunc();
+      if (InstrumentCoverage)
+        OS << "    bool Succeeded = " << P.getSelectFunc();
+      else
+        OS << "  return " << P.getSelectFunc();
 
       OS << "(";
       // If the complex pattern wants the root of the match, pass it in as the
       // first argument.
       if (P.hasProperty(SDNPWantRoot))
         OS << "Root, ";
 
       // If the complex pattern wants the parent of the operand being matched,
       // pass it in as the next argument.
       if (P.hasProperty(SDNPWantParent))
         OS << "Parent, ";
 
       OS << "N";
       for (unsigned i = 0; i != NumOps; ++i)
         OS << ", Result[NextRes+" << i << "].first";
       OS << ");\n";
+      if (InstrumentCoverage) {
+        OS << "    if (Succeeded)\n";
+        OS << "       dbgs() << \"\\nCOMPLEX_PATTERN: " << P.getSelectFunc()
+           << "\\n\" ;\n";
+        OS << "    return Succeeded;\n";
+        OS << "    }\n";
+      }
     }
     OS << "  }\n";
     OS << "}\n\n";
   }
 
 
   // Emit SDNodeXForm handlers.
   // FIXME: This should be const.
void llvm::EmitMatcherTable(const Matcher *TheMatcher,
 
   OS << "  #undef TARGET_VAL\n";
   OS << "  SelectCodeCommon(N, MatcherTable,sizeof(MatcherTable));\n";
   OS << "  return nullptr;\n";
   OS << "}\n";
 
   // Next up, emit the function for node and pattern predicates:
   MatcherEmitter.EmitPredicateFunctions(OS);
+
+  if (InstrumentCoverage)
+    MatcherEmitter.EmitPatternMatchTable(OS);
 }