[globalisel][tablegen] Add support for EXTRACT_SUBREG.

[dsanders]

Summary

After this patch, we finally have test cases that require multiple
instruction emission.

Depends on D33590

[dsanders]

After re-testing this on current trunk, there's a bug in X86 due to GR64 not fully supporting the 8bit subregister index. I'll fix this shortly using failedImport() for this case since we don't have multi-insn emission quite yet. The following patches introduce the infrastructure for it.

[dsanders]

Fix the X86 machine verifier bug where EXTRACT_SUBREG could emit a subregister
copy with classes that didn't quite support it. TableGen now finds a suitable
pair of classes. However, if this results in a copy of the operand into another
class, tablegen will currently refuse to import SelectionDAG because we lack
multi-insn emission at the moment. Support for that will start to be introduced
as part of the state-machine patches.

[qcolombet]

Could you re-upload the patch without the diffs from https://reviews.llvm.org/D33590?

test/CodeGen/AArch64/GlobalISel/select-trunc.mir

19 ↗

(On Diff #101196)

Do you know why we end up with 'all' in one case and 'sp' in the other?
I would have expected to both have 'all' or 'sp'.

utils/TableGen/CodeGenRegisters.h

368 ↗

(On Diff #101196)

The "as far as possible" part doesn't sound right. I thought tablegen was generating the missing register classes for that.

[dsanders]

Resubmit the patch minus the bits that were already in D33590. Sorry about that.

[dsanders]

test/CodeGen/AArch64/GlobalISel/select-trunc.mir

19 ↗

(On Diff #101196)

The gpr32sp is indirectly specified by the rule:

// (trunc:i32 GPR64sp:i64:$src)  =>  (EXTRACT_SUBREG:i32 GPR64sp:i64:$src, sub_32:i32)

$src is specified to be GPR64sp and the best fit for the sub_32 subregister index is GPR32sp. At this point, $src is left as GPR (which sounds wrong to me now that I've noticed it, I think it should constrain it to GPR64sp, I'll take another look at it). The GPR64all comes from the '%vreg0<def>(s64) = COPY %X0; GPR:%vreg0'

utils/TableGen/CodeGenRegisters.h

368 ↗

(On Diff #101196)

It tries to find the best class (often one of the generated classes) but I'm not sure I can guarantee that it will always find it for all targets. AMDGPU's unusual use of subregister indices is what I had in mind since I think that may end up picking a subreg register class that contains additional registers. Additional registers in the subreg register class isn't a problem if it happens.

[dsanders]

Constrain the source operand too. This fixes the odd GPR64all, it's now GPR64sp.

[ab]

Can you add a tablegen testcase?

lib/Target/AArch64/AArch64InstructionSelector.cpp

943–944 ↗

(On Diff #101662)

Remove the:

} else if (DstRC == &AArch64::GPR32RegClass &&
           SrcRC == &AArch64::GPR64RegClass) {

below instead.

utils/TableGen/CodeGenRegisters.cpp

923 ↗

(On Diff #101490)

Looks like we already have an escape hatch in TargetRegisterInfo::getSubClassWithSubReg. Should we just use that?

utils/TableGen/CodeGenRegisters.h

374 ↗

(On Diff #101662)

Return an Optional<std::pair> ?

utils/TableGen/GlobalISelEmitter.cpp

871 ↗

(On Diff #101662)

Triple-slash.

1625–1630 ↗

(On Diff #101662)

At this point, we don't need to compute SrcRC/DstRC, right? Can you check this when emitting the constraining code?

1817 ↗

(On Diff #101662)

Split the 'if' from the define and early-exit?

1825 ↗

(On Diff #101662)

Stale text?

[dsanders]

lib/Target/AArch64/AArch64InstructionSelector.cpp

953 ↗

(On Diff #101662)

This code is also used by G_PTRTOINT. I'll fix these comments.

976–977 ↗

(On Diff #101662)

I've updated this to account for the change you requested but I couldn't make that exact change due to a couple problems.

It turns out this is needed for G_PTRTOINT to prevent select-int-ptr-casts.mir failing. This is because a suitable rule isn't imported. As far as I can see there are no suitable rules to import.

It also turned out to be needed for G_TRUNC to support s1/s8/s16 to avoid errors like:

LLVM ERROR: cannot select: %vreg1<def>(s8) = G_TRUNC %vreg0; GPR32:%vreg1 GPR64:%vreg0 (in function: trunc_s8_s64)

This happens because getRegClassForTypeOnBank() returns GPR32 for integer types smaller than 32 bit. To fix this, I've done the equivalent with DstTy and SrcTy.

utils/TableGen/CodeGenRegisters.cpp

923 ↗

(On Diff #101490)

That only selects a class suitable for the source of the copy. We also need to find a suitable class for the destination otherwise the MachineVerifier will complain that some destination registers for the subreg COPY are not subregisters of a register from the source class.

utils/TableGen/GlobalISelEmitter.cpp

871 ↗

(On Diff #101662)

Well spotted

1625–1630 ↗

(On Diff #101662)

We need to compute it to determine whether we need to copy the source to a suitable register class or not. The current code deals with the case where the extra copy isn't needed and rejects the one that needs it. To support the case that needs it we'll need to refactor to support multiple insn emission.

[dsanders]

Fix the nits ( -> /, Optional<std::pair<...>>, etc.)

Made the change requested for AArch64InstructionSelector.cpp as far as
possible. There are a few issues with the exact change as mentioned in my
comment.

[ab]

Can you add a tablegen testcase? If nothing else, it's a good way to document the tablegen backend.

llvm/trunk/utils/TableGen/CodeGenRegisters.cpp

919

Looks like we already have an escape hatch in TargetRegisterInfo::getSubClassWithSubReg. Should we just use that?

That only selects a class suitable for the source of the copy. We also need to find a suitable class for the destination otherwise the MachineVerifier will complain that some destination registers for the subreg COPY are not subregisters of a register from the source class.

Huh, OK; I thought we did have a helper somewhere (that computes the class of all subregisters of a superregister class via a given index).

llvm/trunk/utils/TableGen/GlobalISelEmitter.cpp

1669

There are multiple 80-col violations, btw

1679

We need to compute it to determine whether we need to copy the source to a suitable register class or not. The current code deals with the case where the extra copy isn't needed and rejects the one that needs it. To support the case that needs it we'll need to refactor to support multiple insn emission.

Sure, but wouldn't it work to only compute it once, later, when emitting, and check whether we need the copy then? Here's a concrete patch:

 utils/TableGen/GlobalISelEmitter.cpp | 12 ++----------
 1 file changed, 2 insertions(+), 10 deletions(-)

diff --git i/utils/TableGen/GlobalISelEmitter.cpp w/utils/TableGen/GlobalISelEmitter.cpp
index 50da9085c21..edb772e5c26 100644
--- i/utils/TableGen/GlobalISelEmitter.cpp
+++ w/utils/TableGen/GlobalISelEmitter.cpp
@@ -1667,18 +1667,8 @@ Expected<BuildMIAction &> GlobalISelEmitter::createAndImportInstructionRenderer(
       return failedImport("EXTRACT_SUBREG child #1 is not a leaf");

     if (DefInit *SubRegInit = dyn_cast<DefInit>(Dst->getChild(1)->getLeafValue())) {
-      CodeGenRegisterClass *RC = CGRegs.getRegClass(
-          getInitValueAsRegClass(Dst->getChild(0)->getLeafValue()));
       CodeGenSubRegIndex *SubIdx = CGRegs.getSubRegIdx(SubRegInit->getDef());

-      const auto &SrcRCDstRCPair =
-          RC->getMatchingSubClassWithSubRegs(CGRegs, SubIdx);
-      if (SrcRCDstRCPair.hasValue()) {
-        assert(SrcRCDstRCPair->second && "Couldn't find a matching subclass");
-        if (SrcRCDstRCPair->first != RC)
-          return failedImport("EXTRACT_SUBREG requires an additional COPY");
-      }
-
       DstMIBuilder.addRenderer<CopySubRegRenderer>(
           InsnMatcher, Dst->getChild(0)->getName(), SubIdx);
       return DstMIBuilder;
@@ -1898,6 +1888,8 @@ Expected<RuleMatcher> GlobalISelEmitter::runOnPattern(const PatternToMatch &P) {
       const auto &SrcRCDstRCPair =
           SrcRC->getMatchingSubClassWithSubRegs(CGRegs, SubIdx);
       assert(SrcRCDstRCPair->second && "Couldn't find a matching subclass");
+      if (SrcRCDstRCPair->first != SrcRC)
+        return failedImport("EXTRACT_SUBREG requires an additional COPY");
       M.addAction<ConstrainOperandToRegClassAction>("NewI", 0, *SrcRCDstRCPair->second);
       M.addAction<ConstrainOperandToRegClassAction>("NewI", 1, *SrcRCDstRCPair->first);

1878

This was where I was suggesting an early-exit, not L1866: it might help to drop the if around SubRegInit?

1884

The operand number is also wrong, no? (here and elsewhere)

1896

Add a message?

[dsanders]

In D33596#792538, @ab wrote:

Can you add a tablegen testcase? If nothing else, it's a good way to document the tablegen backend.

Sure.

llvm/trunk/utils/TableGen/GlobalISelEmitter.cpp

1669

Ok, I'll fix these. Hopefully, I'll also find time soon to get the commit hook I used to have working again.

1679

That looks like it will work at the moment but I'm not sure it fits with the emission of nested instructions. For example:

(EXTRACT_SUBREG (ADD x:s64, y:s64), sub_32)

could select to:

%2(A64) = ADD %0(A64), %1(A64)
%3(B64) = COPY %2(A64)
%4(B32) = COPY %3(B64), sub_32

where A64 and B64 are disjoint register classes and B32 is a subregister class of B64. Handling the extra copy outside the tree walking code would need a way to insert the copy into the right part of the emitted list. If we handle it during the tree walking code we can just append to the list as we go.

Here's a particularly complicated example from Mips:

def : MSAPat<
  (i32 (vextract_zext_i8 v16i8:$ws, i64:$idx)),
  (SRL (COPY_TO_REGCLASS
         (i32 (EXTRACT_SUBREG
                 (SPLAT_B v16i8:$ws,
                   (COPY_TO_REGCLASS
                     (i32 (EXTRACT_SUBREG i64:$idx, sub_32)), GPR32)),
                 sub_lo)),
         GPR32),
       (i32 24))>;

1878

Ah I see. I don't mind either way, I'll make the change.

1884

It depends how you view the data. The gMIR view is:

%0 = EXTRACT_SUBREG %1, subidx

in which case it's operand 1, but the DAG view is

(EXTRACT_SUBREG %1, subidx)

in which case it's child 1. I've been using 'operand' for the gMIR view and 'child' for the DAG view.

1896

Ok

llvm/trunk/lib/Target/AArch64/AArch64InstructionSelector.cpp

case TargetOpcode::G_TRUNC: {
 
     const unsigned DstReg = I.getOperand(0).getReg();
     const unsigned SrcReg = I.getOperand(1).getReg();
 
     const RegisterBank &DstRB = *RBI.getRegBank(DstReg, MRI, TRI);
     const RegisterBank &SrcRB = *RBI.getRegBank(SrcReg, MRI, TRI);
 
     if (DstRB.getID() != SrcRB.getID()) {
-      DEBUG(dbgs() << "G_TRUNC input/output on different banks\n");
+      DEBUG(dbgs() << "G_TRUNC/G_PTRTOINT input/output on different banks\n");
       return false;
     }
 
     if (DstRB.getID() == AArch64::GPRRegBankID) {
       const TargetRegisterClass *DstRC =
           getRegClassForTypeOnBank(DstTy, DstRB, RBI);
       if (!DstRC)
         return false;
 
       const TargetRegisterClass *SrcRC =
           getRegClassForTypeOnBank(SrcTy, SrcRB, RBI);
       if (!SrcRC)
         return false;
 
       if (!RBI.constrainGenericRegister(SrcReg, *SrcRC, MRI) ||
           !RBI.constrainGenericRegister(DstReg, *DstRC, MRI)) {
-        DEBUG(dbgs() << "Failed to constrain G_TRUNC\n");
+        DEBUG(dbgs() << "Failed to constrain G_TRUNC/G_PTRTOINT\n");
         return false;
       }
 
       if (DstRC == SrcRC) {
         // Nothing to be done
+      } else if (Opcode == TargetOpcode::G_TRUNC && DstTy == LLT::scalar(32) &&
+                 SrcTy == LLT::scalar(64)) {
+        llvm_unreachable("TableGen can import this case");
+        return false;
       } else if (DstRC == &AArch64::GPR32RegClass &&
                  SrcRC == &AArch64::GPR64RegClass) {
         I.getOperand(1).setSubReg(AArch64::sub_32);
       } else {
+        DEBUG(dbgs() << "Unhandled mismatched classes in G_TRUNC/G_PTRTOINT\n");
         return false;
       }
 
       I.setDesc(TII.get(TargetOpcode::COPY));
       return true;
     } else if (DstRB.getID() == AArch64::FPRRegBankID) {
       if (DstTy == LLT::vector(4, 16) && SrcTy == LLT::vector(4, 32)) {
         I.setDesc(TII.get(AArch64::XTNv4i16));

llvm/trunk/test/CodeGen/AArch64/GlobalISel/select-trunc.mir

 
 ---
 # CHECK-LABEL: name: trunc_s32_s64
 name:            trunc_s32_s64
 legalized:       true
 regBankSelected: true
 
 # CHECK:      registers:
-# CHECK-NEXT:  - { id: 0, class: gpr64, preferred-register: '' }
-# CHECK-NEXT:  - { id: 1, class: gpr32, preferred-register: '' }
+# CHECK-NEXT:  - { id: 0, class: gpr64sp, preferred-register: '' }
+# CHECK-NEXT:  - { id: 1, class: gpr32sp, preferred-register: '' }
 registers:
   - { id: 0, class: gpr }
   - { id: 1, class: gpr }
 
 # CHECK:  body:
 # CHECK:    %1 = COPY %0.sub_32
 body:             |
   bb.0:

llvm/trunk/utils/TableGen/CodeGenRegisters.h

public:
 
     // Return the Record that defined this class, or NULL if the class was
     // created by TableGen.
     Record *getDef() const { return TheDef; }
 
     const std::string &getName() const { return Name; }
     std::string getQualifiedName() const;
     ArrayRef<MVT::SimpleValueType> getValueTypes() const {return VTs;}
+    bool hasValueType(MVT::SimpleValueType VT) const {
+      return std::find(VTs.begin(), VTs.end(), VT) != VTs.end();
+    }
     unsigned getNumValueTypes() const { return VTs.size(); }
 
     MVT::SimpleValueType getValueTypeNum(unsigned VTNum) const {
       if (VTNum < VTs.size())
         return VTs[VTNum];
       llvm_unreachable("VTNum greater than number of ValueTypes in RegClass!");
     }
 
public:
 
     // getSubClassWithSubReg - Returns the largest sub-class where all
     // registers have a SubIdx sub-register.
     CodeGenRegisterClass *
     getSubClassWithSubReg(const CodeGenSubRegIndex *SubIdx) const {
       return SubClassWithSubReg.lookup(SubIdx);
     }
 
+    /// Find largest subclass where all registers have SubIdx subregisters in
+    /// SubRegClass and the largest subregister class that contains those
+    /// subregisters without (as far as possible) also containing additional registers.
+    ///
+    /// This can be used to find a suitable pair of classes for subregister copies.
+    /// \return std::pair<SubClass, SubRegClass> where SubClass is a SubClass is
+    /// a class where every register has SubIdx and SubRegClass is a class where
+    /// every register is covered by the SubIdx subregister of SubClass.
+    Optional<std::pair<CodeGenRegisterClass *, CodeGenRegisterClass *>>
+    getMatchingSubClassWithSubRegs(CodeGenRegBank &RegBank,
+                                   const CodeGenSubRegIndex *SubIdx) const;
+
     void setSubClassWithSubReg(const CodeGenSubRegIndex *SubIdx,
                                CodeGenRegisterClass *SubRC) {
       SubClassWithSubReg[SubIdx] = SubRC;
     }
 
     // getSuperRegClasses - Returns a bit vector of all register classes
     // containing only SubIdx super-registers of this class.
     void getSuperRegClasses(const CodeGenSubRegIndex *SubIdx,
                             BitVector &Out) const;
 
-    // addSuperRegClass - Add a class containing only SudIdx super-registers.
+    // addSuperRegClass - Add a class containing only SubIdx super-registers.
     void addSuperRegClass(CodeGenSubRegIndex *SubIdx,
                           CodeGenRegisterClass *SuperRC) {
       SuperRegClasses[SubIdx].insert(SuperRC);
     }
 
     // getSubClasses - Returns a constant BitVector of subclasses indexed by
     // EnumValue.
     // The SubClasses vector includes an entry for this class.

llvm/trunk/utils/TableGen/CodeGenRegisters.cpp

void CodeGenRegisterClass::computeSubClasses(CodeGenRegBank &RegBank) {
   // With the class hierarchy in place, let synthesized register classes inherit
   // properties from their closest super-class. The iteration order here can
   // propagate properties down multiple levels.
   for (auto &RC : RegClasses)
     if (!RC.getDef())
       RC.inheritProperties(RegBank);
 }
 
+Optional<std::pair<CodeGenRegisterClass *, CodeGenRegisterClass *>>
+CodeGenRegisterClass::getMatchingSubClassWithSubRegs(
+    CodeGenRegBank &RegBank, const CodeGenSubRegIndex *SubIdx) const {
+  auto SizeOrder = [](const CodeGenRegisterClass *A,
+                      const CodeGenRegisterClass *B) {
+    return A->getMembers().size() >= B->getMembers().size();
+  };
+
+  auto &RegClasses = RegBank.getRegClasses();
+
+  // Find all the subclasses of this one that fully support the sub-register
+  // index and order them by size. BiggestSuperRC should always be first.
+  CodeGenRegisterClass *BiggestSuperRegRC = getSubClassWithSubReg(SubIdx);
+  if (!BiggestSuperRegRC)
+    return None;
+  BitVector SuperRegRCsBV = BiggestSuperRegRC->getSubClasses();
+  std::vector<CodeGenRegisterClass *> SuperRegRCs;
+  for (auto &RC : RegClasses)
+    if (SuperRegRCsBV[RC.EnumValue])
+      SuperRegRCs.emplace_back(&RC);
+  std::sort(SuperRegRCs.begin(), SuperRegRCs.end(), SizeOrder);
+  assert(SuperRegRCs.front() == BiggestSuperRegRC && "Biggest class wasn't first");
+
+  // Find all the subreg classes and order them by size too.
+  std::vector<std::pair<CodeGenRegisterClass *, BitVector>> SuperRegClasses;
+  for (auto &RC: RegClasses) {
+    BitVector SuperRegClassesBV(RegClasses.size());
+    RC.getSuperRegClasses(SubIdx, SuperRegClassesBV);
+    if (SuperRegClassesBV.any())
+      SuperRegClasses.push_back(std::make_pair(&RC, SuperRegClassesBV));
+  }
+  std::sort(SuperRegClasses.begin(), SuperRegClasses.end(),
+            [&](const std::pair<CodeGenRegisterClass *, BitVector> &A,
+                const std::pair<CodeGenRegisterClass *, BitVector> &B) {
+              return SizeOrder(A.first, B.first);
+            });
+
+  // Find the biggest subclass and subreg class such that R:subidx is in the
+  // subreg class for all R in subclass.
+  //
+  // For example:
+  // All registers in X86's GR64 have a sub_32bit subregister but no class
+  // exists that contains all the 32-bit subregisters because GR64 contains RIP
+  // but GR32 does not contain EIP. Instead, we constrain SuperRegRC to
+  // GR32_with_sub_8bit (which is identical to GR32_with_sub_32bit) and then,
+  // having excluded RIP, we are able to find a SubRegRC (GR32).
+  CodeGenRegisterClass *ChosenSuperRegClass = nullptr;
+  CodeGenRegisterClass *SubRegRC = nullptr;
+  for (auto *SuperRegRC : SuperRegRCs) {
+    for (const auto &SuperRegClassPair : SuperRegClasses) {
+      const BitVector &SuperRegClassBV = SuperRegClassPair.second;
+      if (SuperRegClassBV[SuperRegRC->EnumValue]) {
+        SubRegRC = SuperRegClassPair.first;
+        ChosenSuperRegClass = SuperRegRC;
+
+        // If SubRegRC is bigger than SuperRegRC then there are members of
+        // SubRegRC that don't have super registers via SubIdx. Keep looking to
+        // find a better fit and fall back on this one if there isn't one.
+        //
+        // This is intended to prevent X86 from making odd choices such as
+        // picking LOW32_ADDR_ACCESS_RBP instead of GR32 in the example above.
+        // LOW32_ADDR_ACCESS_RBP is a valid choice but contains registers that
+        // aren't subregisters of SuperRegRC whereas GR32 has a direct 1:1
+        // mapping.
+        if (SuperRegRC->getMembers().size() >= SubRegRC->getMembers().size())
+          return std::make_pair(ChosenSuperRegClass, SubRegRC);
+      }
+    }
+
+    // If we found a fit but it wasn't quite ideal because SubRegRC had excess
+    // registers, then we're done.
+    if (ChosenSuperRegClass)
+      return std::make_pair(ChosenSuperRegClass, SubRegRC);
+  }
+
+  return None;
+}
+
 void CodeGenRegisterClass::getSuperRegClasses(const CodeGenSubRegIndex *SubIdx,
                                               BitVector &Out) const {
   auto FindI = SuperRegClasses.find(SubIdx);
   if (FindI == SuperRegClasses.end())
     return;
   for (CodeGenRegisterClass *RC : FindI->second)
     Out.set(RC->EnumValue);
 }

llvm/trunk/utils/TableGen/GlobalISelEmitter.cpp

void emitCxxPredicateExpr(raw_ostream &OS, RuleMatcher &Rule,
     InsnMatcher->emitCxxPredicateExpr(OS, Rule, OperandExpr);
     OS << ")\n";
   }
 };
 
 //===- Actions ------------------------------------------------------------===//
 class OperandRenderer {
 public:
-  enum RendererKind { OR_Copy, OR_Imm, OR_Register, OR_ComplexPattern };
+  enum RendererKind {
+    OR_Copy,
+    OR_CopySubReg,
+    OR_Imm,
+    OR_Register,
+    OR_ComplexPattern
+  };
 
 protected:
   RendererKind Kind;
 
 public:
   OperandRenderer(RendererKind Kind) : Kind(Kind) {}
   virtual ~OperandRenderer() {}
 
void emitCxxRenderStmts(raw_ostream &OS, RuleMatcher &Rule) const override {
     const OperandMatcher &Operand = Matched.getOperand(SymbolicName);
     StringRef InsnVarName =
         Rule.getInsnVarName(Operand.getInstructionMatcher());
     std::string OperandExpr = Operand.getOperandExpr(InsnVarName);
     OS << "    MIB.add(" << OperandExpr << "/*" << SymbolicName << "*/);\n";
   }
 };
 
+/// A CopySubRegRenderer emits code to copy a single register operand from an
+/// existing instruction to the one being built and indicate that only a
+/// subregister should be copied.
+class CopySubRegRenderer : public OperandRenderer {
+protected:
+  /// The matcher for the instruction that this operand is copied from.
+  /// This provides the facility for looking up an a operand by it's name so
+  /// that it can be used as a source for the instruction being built.
+  const InstructionMatcher &Matched;
+  /// The name of the operand.
+  const StringRef SymbolicName;
+  /// The subregister to extract.
+  const CodeGenSubRegIndex *SubReg;
+
+public:
+  CopySubRegRenderer(const InstructionMatcher &Matched, StringRef SymbolicName,
+                     const CodeGenSubRegIndex *SubReg)
+      : OperandRenderer(OR_CopySubReg), Matched(Matched),
+        SymbolicName(SymbolicName), SubReg(SubReg) {}
+
+  static bool classof(const OperandRenderer *R) {
+    return R->getKind() == OR_CopySubReg;
+  }
+
+  const StringRef getSymbolicName() const { return SymbolicName; }
+
+  void emitCxxRenderStmts(raw_ostream &OS, RuleMatcher &Rule) const override {
+    const OperandMatcher &Operand = Matched.getOperand(SymbolicName);
+    StringRef InsnVarName =
+        Rule.getInsnVarName(Operand.getInstructionMatcher());
+    std::string OperandExpr = Operand.getOperandExpr(InsnVarName);
+    OS << "    MIB.addReg(" << OperandExpr << ".getReg() /*" << SymbolicName
+       << "*/, 0, " << SubReg->EnumValue << ");\n";
+  }
+};
+
 /// Adds a specific physical register to the instruction being built.
 /// This is typically useful for WZR/XZR on AArch64.
 class AddRegisterRenderer : public OperandRenderer {
 protected:
   const Record *RegisterDef;
 
 public:
   AddRegisterRenderer(const Record *RegisterDef)

 public:
   explicit GlobalISelEmitter(RecordKeeper &RK);
   void run(raw_ostream &OS);
 
 private:
   const RecordKeeper &RK;
   const CodeGenDAGPatterns CGP;
   const CodeGenTarget &Target;
+  CodeGenRegBank CGRegs;
 
   /// Keep track of the equivalence between SDNodes and Instruction.
   /// This is defined using 'GINodeEquiv' in the target description.
   DenseMap<Record *, const CodeGenInstruction *> NodeEquivs;
 
   /// Keep track of the equivalence between ComplexPattern's and
   /// GIComplexOperandMatcher. Map entries are specified by subclassing
   /// GIComplexPatternEquiv.
   DenseMap<const Record *, const Record *> ComplexPatternEquivs;
 
   // Map of predicates to their subtarget features.
   SubtargetFeatureInfoMap SubtargetFeatures;
 
   void gatherNodeEquivs();
   const CodeGenInstruction *findNodeEquiv(Record *N) const;
 
   Error importRulePredicates(RuleMatcher &M, ArrayRef<Init *> Predicates);
   Expected<InstructionMatcher &>
   createAndImportSelDAGMatcher(InstructionMatcher &InsnMatcher,
                                const TreePatternNode *Src) const;
   Error importChildMatcher(InstructionMatcher &InsnMatcher,
                            const TreePatternNode *SrcChild, unsigned OpIdx,
                            unsigned &TempOpIdx) const;
-  Expected<BuildMIAction &> createAndImportInstructionRenderer(
-      RuleMatcher &M, const TreePatternNode *Dst,
-      const InstructionMatcher &InsnMatcher) const;
+  Expected<BuildMIAction &>
+  createAndImportInstructionRenderer(RuleMatcher &M, const TreePatternNode *Dst,
+                                     const InstructionMatcher &InsnMatcher);
   Error importExplicitUseRenderer(BuildMIAction &DstMIBuilder,
                                   TreePatternNode *DstChild,
                                   const InstructionMatcher &InsnMatcher) const;
   Error importDefaultOperandRenderers(BuildMIAction &DstMIBuilder,
                                       DagInit *DefaultOps) const;
   Error
   importImplicitDefRenderers(BuildMIAction &DstMIBuilder,
                              const std::vector<Record *> &ImplicitDefs) const;

  }
 }
 
 const CodeGenInstruction *GlobalISelEmitter::findNodeEquiv(Record *N) const {
   return NodeEquivs.lookup(N);
 }
 
 GlobalISelEmitter::GlobalISelEmitter(RecordKeeper &RK)
-    : RK(RK), CGP(RK), Target(CGP.getTargetInfo()) {}
+    : RK(RK), CGP(RK), Target(CGP.getTargetInfo()), CGRegs(RK) {}
 
 //===- Emitter ------------------------------------------------------------===//
 
 Error
 GlobalISelEmitter::importRulePredicates(RuleMatcher &M,
                                         ArrayRef<Init *> Predicates) {
   for (const Init *Predicate : Predicates) {
     const DefInit *PredicateDef = static_cast<const DefInit *>(Predicate);
return failedImport(
         "Dst pattern child def is an unsupported tablegen class");
   }
 
   return failedImport("Dst pattern child is an unsupported kind");
 }
 
 Expected<BuildMIAction &> GlobalISelEmitter::createAndImportInstructionRenderer(
     RuleMatcher &M, const TreePatternNode *Dst,
-    const InstructionMatcher &InsnMatcher) const {
+    const InstructionMatcher &InsnMatcher) {
   Record *DstOp = Dst->getOperator();
   if (!DstOp->isSubClassOf("Instruction")) {
     if (DstOp->isSubClassOf("ValueType"))
       return failedImport(
           "Pattern operator isn't an instruction (it's a ValueType)");
     return failedImport("Pattern operator isn't an instruction");
   }
   CodeGenInstruction *DstI = &Target.getInstruction(DstOp);
 
   unsigned DstINumUses = DstI->Operands.size() - DstI->Operands.NumDefs;
   unsigned ExpectedDstINumUses = Dst->getNumChildren();
+  bool IsExtractSubReg = false;
 
   // COPY_TO_REGCLASS is just a copy with a ConstrainOperandToRegClassAction
-  // attached.
+  // attached. Similarly for EXTRACT_SUBREG except that's a subregister copy.
   if (DstI->TheDef->getName() == "COPY_TO_REGCLASS") {
     DstI = &Target.getInstruction(RK.getDef("COPY"));
     DstINumUses--; // Ignore the class constraint.
     ExpectedDstINumUses--;
+  } else if (DstI->TheDef->getName() == "EXTRACT_SUBREG") {
+    DstI = &Target.getInstruction(RK.getDef("COPY"));
+    IsExtractSubReg = true;
   }
 
   auto &DstMIBuilder = M.addAction<BuildMIAction>("NewI", DstI, InsnMatcher);
 
   // Render the explicit defs.
   for (unsigned I = 0; I < DstI->Operands.NumDefs; ++I) {
     const CGIOperandList::OperandInfo &DstIOperand = DstI->Operands[I];
     DstMIBuilder.addRenderer<CopyRenderer>(InsnMatcher, DstIOperand.Name);
   }
 
+  // EXTRACT_SUBREG needs to use a subregister COPY.
+  if (IsExtractSubReg) {
+    if (!Dst->getChild(0)->isLeaf())
+      return failedImport("EXTRACT_SUBREG child #1 is not a leaf");
+
+    if (DefInit *SubRegInit = dyn_cast<DefInit>(Dst->getChild(1)->getLeafValue())) {
+      CodeGenRegisterClass *RC = CGRegs.getRegClass(
+          getInitValueAsRegClass(Dst->getChild(0)->getLeafValue()));
+      CodeGenSubRegIndex *SubIdx = CGRegs.getSubRegIdx(SubRegInit->getDef());
+
+      const auto &SrcRCDstRCPair =
+          RC->getMatchingSubClassWithSubRegs(CGRegs, SubIdx);
+      if (SrcRCDstRCPair.hasValue()) {
+        assert(SrcRCDstRCPair->second && "Couldn't find a matching subclass");
+        if (SrcRCDstRCPair->first != RC)
+          return failedImport("EXTRACT_SUBREG requires an additional COPY");
+      }
+
+      DstMIBuilder.addRenderer<CopySubRegRenderer>(
+          InsnMatcher, Dst->getChild(0)->getName(), SubIdx);
+      return DstMIBuilder;
+    }
+
+    return failedImport("EXTRACT_SUBREG child #1 is not a subreg index");
+  }
+
   // Render the explicit uses.
   unsigned Child = 0;
   unsigned NumDefaultOps = 0;
   for (unsigned I = 0; I != DstINumUses; ++I) {
     const CGIOperandList::OperandInfo &DstIOperand =
         DstI->Operands[DstI->Operands.NumDefs + I];
 
     // If the operand has default values, introduce them now.
for (const EEVT::TypeSet &Ty : Src->getExtTypes()) {
     const auto &DstIOperand = DstI.Operands[OpIdx];
     Record *DstIOpRec = DstIOperand.Rec;
     if (DstI.TheDef->getName() == "COPY_TO_REGCLASS") {
       DstIOpRec = getInitValueAsRegClass(Dst->getChild(1)->getLeafValue());
 
       if (DstIOpRec == nullptr)
         return failedImport(
             "COPY_TO_REGCLASS operand #1 isn't a register class");
+    } else if (DstI.TheDef->getName() == "EXTRACT_SUBREG") {
+      if (!Dst->getChild(0)->isLeaf())
+        return failedImport("EXTRACT_SUBREG operand #0 isn't a leaf");
+
+      // We can assume that a subregister is in the same bank as it's super register.
+      DstIOpRec = getInitValueAsRegClass(Dst->getChild(0)->getLeafValue());
+
+      if (DstIOpRec == nullptr)
+        return failedImport(
+            "EXTRACT_SUBREG operand #0 isn't a register class");
     } else if (DstIOpRec->isSubClassOf("RegisterOperand"))
       DstIOpRec = DstIOpRec->getValueAsDef("RegClass");
     else if (!DstIOpRec->isSubClassOf("RegisterClass"))
       return failedImport("Dst MI def isn't a register class" + to_string(*Dst));
 
     OperandMatcher &OM = InsnMatcher.getOperand(OpIdx);
     OM.setSymbolicName(DstIOperand.Name);
     OM.addPredicate<RegisterBankOperandMatcher>(
if (DstI.TheDef->getName() == "COPY_TO_REGCLASS") {
     Record *DstIOpRec =
         getInitValueAsRegClass(Dst->getChild(1)->getLeafValue());
 
     if (DstIOpRec == nullptr)
       return failedImport("COPY_TO_REGCLASS operand #1 isn't a register class");
 
     M.addAction<ConstrainOperandToRegClassAction>(
         "NewI", 0, Target.getRegisterClass(DstIOpRec));
-  } else
-    M.addAction<ConstrainOperandsToDefinitionAction>("NewI");
+
+    // We're done with this pattern!  It's eligible for GISel emission; return
+    // it.
+    ++NumPatternImported;
+    return std::move(M);
+  }
+
+  if (DstI.TheDef->getName() == "EXTRACT_SUBREG") {
+    // EXTRACT_SUBREG selects into a subregister COPY but unlike most
+    // instructions, the result register class is controlled by the
+    // subregisters of the operand. As a result, we must constrain the result
+    // class rather than check that it's already the right one.
+    if (!Dst->getChild(0)->isLeaf())
+      return failedImport("EXTRACT_SUBREG child #1 is not a leaf");
+
+    if (DefInit *SubRegInit =
+            dyn_cast<DefInit>(Dst->getChild(1)->getLeafValue())) {
+      // Constrain the result to the same register bank as the operand.
+      Record *DstIOpRec =
+          getInitValueAsRegClass(Dst->getChild(0)->getLeafValue());
+
+      if (DstIOpRec == nullptr)
+        return failedImport("EXTRACT_SUBREG operand #1 isn't a register class");
+
+      CodeGenSubRegIndex *SubIdx = CGRegs.getSubRegIdx(SubRegInit->getDef());
+      CodeGenRegisterClass *SrcRC = CGRegs.getRegClass(
+          getInitValueAsRegClass(Dst->getChild(0)->getLeafValue()));
+
+      // It would be nice to leave this constraint implicit but we're required
+      // to pick a register class so constrain the result to a register class
+      // that can hold the correct MVT.
+      //
+      // FIXME: This may introduce an extra copy if the chosen class doesn't
+      //        actually contain the subregisters.
+      assert(Src->getExtTypes().size() == 1);
+
+      const auto &SrcRCDstRCPair =
+          SrcRC->getMatchingSubClassWithSubRegs(CGRegs, SubIdx);
+      assert(SrcRCDstRCPair->second && "Couldn't find a matching subclass");
+      M.addAction<ConstrainOperandToRegClassAction>("NewI", 0, *SrcRCDstRCPair->second);
+      M.addAction<ConstrainOperandToRegClassAction>("NewI", 1, *SrcRCDstRCPair->first);
+
+      // We're done with this pattern!  It's eligible for GISel emission; return
+      // it.
+      ++NumPatternImported;
+      return std::move(M);
+    }
+
+    return failedImport("EXTRACT_SUBREG child #1 is not a subreg index");
+  }
+
 
   // We're done with this pattern!  It's eligible for GISel emission; return it.
   ++NumPatternImported;
   return std::move(M);
 }
 
 void GlobalISelEmitter::run(raw_ostream &OS) {
   // Track the GINodeEquiv definitions.