[globalisel][tablegen] Add support for EXTRACT_SUBREG.

[dsanders]

Summary

After this patch, we finally have test cases that require multiple
instruction emission.

Depends on D33590

[dsanders]

After re-testing this on current trunk, there's a bug in X86 due to GR64 not fully supporting the 8bit subregister index. I'll fix this shortly using failedImport() for this case since we don't have multi-insn emission quite yet. The following patches introduce the infrastructure for it.

[dsanders]

Fix the X86 machine verifier bug where EXTRACT_SUBREG could emit a subregister
copy with classes that didn't quite support it. TableGen now finds a suitable
pair of classes. However, if this results in a copy of the operand into another
class, tablegen will currently refuse to import SelectionDAG because we lack
multi-insn emission at the moment. Support for that will start to be introduced
as part of the state-machine patches.

[qcolombet]

Could you re-upload the patch without the diffs from https://reviews.llvm.org/D33590?

test/CodeGen/AArch64/GlobalISel/select-trunc.mir

19

Do you know why we end up with 'all' in one case and 'sp' in the other?
I would have expected to both have 'all' or 'sp'.

utils/TableGen/CodeGenRegisters.h

368

The "as far as possible" part doesn't sound right. I thought tablegen was generating the missing register classes for that.

[dsanders]

Resubmit the patch minus the bits that were already in D33590. Sorry about that.

[dsanders]

test/CodeGen/AArch64/GlobalISel/select-trunc.mir

19

The gpr32sp is indirectly specified by the rule:

// (trunc:i32 GPR64sp:i64:$src)  =>  (EXTRACT_SUBREG:i32 GPR64sp:i64:$src, sub_32:i32)

$src is specified to be GPR64sp and the best fit for the sub_32 subregister index is GPR32sp. At this point, $src is left as GPR (which sounds wrong to me now that I've noticed it, I think it should constrain it to GPR64sp, I'll take another look at it). The GPR64all comes from the '%vreg0<def>(s64) = COPY %X0; GPR:%vreg0'

utils/TableGen/CodeGenRegisters.h

368

It tries to find the best class (often one of the generated classes) but I'm not sure I can guarantee that it will always find it for all targets. AMDGPU's unusual use of subregister indices is what I had in mind since I think that may end up picking a subreg register class that contains additional registers. Additional registers in the subreg register class isn't a problem if it happens.

[dsanders]

Constrain the source operand too. This fixes the odd GPR64all, it's now GPR64sp.

[ab]

Can you add a tablegen testcase?

lib/Target/AArch64/AArch64InstructionSelector.cpp

943–944

Remove the:

} else if (DstRC == &AArch64::GPR32RegClass &&
           SrcRC == &AArch64::GPR64RegClass) {

below instead.

utils/TableGen/CodeGenRegisters.cpp

923

Looks like we already have an escape hatch in TargetRegisterInfo::getSubClassWithSubReg. Should we just use that?

utils/TableGen/CodeGenRegisters.h

374

Return an Optional<std::pair> ?

utils/TableGen/GlobalISelEmitter.cpp

871

Triple-slash.

1621–1626

At this point, we don't need to compute SrcRC/DstRC, right? Can you check this when emitting the constraining code?

1799

Split the 'if' from the define and early-exit?

1807

Stale text?

[dsanders]

lib/Target/AArch64/AArch64InstructionSelector.cpp

953

This code is also used by G_PTRTOINT. I'll fix these comments.

976–977

I've updated this to account for the change you requested but I couldn't make that exact change due to a couple problems.

It turns out this is needed for G_PTRTOINT to prevent select-int-ptr-casts.mir failing. This is because a suitable rule isn't imported. As far as I can see there are no suitable rules to import.

It also turned out to be needed for G_TRUNC to support s1/s8/s16 to avoid errors like:

LLVM ERROR: cannot select: %vreg1<def>(s8) = G_TRUNC %vreg0; GPR32:%vreg1 GPR64:%vreg0 (in function: trunc_s8_s64)

This happens because getRegClassForTypeOnBank() returns GPR32 for integer types smaller than 32 bit. To fix this, I've done the equivalent with DstTy and SrcTy.

utils/TableGen/CodeGenRegisters.cpp

923

That only selects a class suitable for the source of the copy. We also need to find a suitable class for the destination otherwise the MachineVerifier will complain that some destination registers for the subreg COPY are not subregisters of a register from the source class.

utils/TableGen/GlobalISelEmitter.cpp

871

Well spotted

1621–1626

We need to compute it to determine whether we need to copy the source to a suitable register class or not. The current code deals with the case where the extra copy isn't needed and rejects the one that needs it. To support the case that needs it we'll need to refactor to support multiple insn emission.

[dsanders]

Fix the nits ( -> /, Optional<std::pair<...>>, etc.)

Made the change requested for AArch64InstructionSelector.cpp as far as
possible. There are a few issues with the exact change as mentioned in my
comment.

[ab]

Can you add a tablegen testcase? If nothing else, it's a good way to document the tablegen backend.

llvm/trunk/utils/TableGen/CodeGenRegisters.cpp

919 ↗

(On Diff #104119)

Looks like we already have an escape hatch in TargetRegisterInfo::getSubClassWithSubReg. Should we just use that?

That only selects a class suitable for the source of the copy. We also need to find a suitable class for the destination otherwise the MachineVerifier will complain that some destination registers for the subreg COPY are not subregisters of a register from the source class.

Huh, OK; I thought we did have a helper somewhere (that computes the class of all subregisters of a superregister class via a given index).

llvm/trunk/utils/TableGen/GlobalISelEmitter.cpp

1669 ↗

(On Diff #104119)

There are multiple 80-col violations, btw

1679 ↗

(On Diff #104119)

We need to compute it to determine whether we need to copy the source to a suitable register class or not. The current code deals with the case where the extra copy isn't needed and rejects the one that needs it. To support the case that needs it we'll need to refactor to support multiple insn emission.

Sure, but wouldn't it work to only compute it once, later, when emitting, and check whether we need the copy then? Here's a concrete patch:

 utils/TableGen/GlobalISelEmitter.cpp | 12 ++----------
 1 file changed, 2 insertions(+), 10 deletions(-)

diff --git i/utils/TableGen/GlobalISelEmitter.cpp w/utils/TableGen/GlobalISelEmitter.cpp
index 50da9085c21..edb772e5c26 100644
--- i/utils/TableGen/GlobalISelEmitter.cpp
+++ w/utils/TableGen/GlobalISelEmitter.cpp
@@ -1667,18 +1667,8 @@ Expected<BuildMIAction &> GlobalISelEmitter::createAndImportInstructionRenderer(
       return failedImport("EXTRACT_SUBREG child #1 is not a leaf");

     if (DefInit *SubRegInit = dyn_cast<DefInit>(Dst->getChild(1)->getLeafValue())) {
-      CodeGenRegisterClass *RC = CGRegs.getRegClass(
-          getInitValueAsRegClass(Dst->getChild(0)->getLeafValue()));
       CodeGenSubRegIndex *SubIdx = CGRegs.getSubRegIdx(SubRegInit->getDef());

-      const auto &SrcRCDstRCPair =
-          RC->getMatchingSubClassWithSubRegs(CGRegs, SubIdx);
-      if (SrcRCDstRCPair.hasValue()) {
-        assert(SrcRCDstRCPair->second && "Couldn't find a matching subclass");
-        if (SrcRCDstRCPair->first != RC)
-          return failedImport("EXTRACT_SUBREG requires an additional COPY");
-      }
-
       DstMIBuilder.addRenderer<CopySubRegRenderer>(
           InsnMatcher, Dst->getChild(0)->getName(), SubIdx);
       return DstMIBuilder;
@@ -1898,6 +1888,8 @@ Expected<RuleMatcher> GlobalISelEmitter::runOnPattern(const PatternToMatch &P) {
       const auto &SrcRCDstRCPair =
           SrcRC->getMatchingSubClassWithSubRegs(CGRegs, SubIdx);
       assert(SrcRCDstRCPair->second && "Couldn't find a matching subclass");
+      if (SrcRCDstRCPair->first != SrcRC)
+        return failedImport("EXTRACT_SUBREG requires an additional COPY");
       M.addAction<ConstrainOperandToRegClassAction>("NewI", 0, *SrcRCDstRCPair->second);
       M.addAction<ConstrainOperandToRegClassAction>("NewI", 1, *SrcRCDstRCPair->first);

1878 ↗

(On Diff #104119)

This was where I was suggesting an early-exit, not L1866: it might help to drop the if around SubRegInit?

1884 ↗

(On Diff #104119)

The operand number is also wrong, no? (here and elsewhere)

1896 ↗

(On Diff #104119)

Add a message?

[dsanders]

In D33596#792538, @ab wrote:

Can you add a tablegen testcase? If nothing else, it's a good way to document the tablegen backend.

Sure.

llvm/trunk/utils/TableGen/GlobalISelEmitter.cpp

1669 ↗

(On Diff #104119)

Ok, I'll fix these. Hopefully, I'll also find time soon to get the commit hook I used to have working again.

1679 ↗

(On Diff #104119)

That looks like it will work at the moment but I'm not sure it fits with the emission of nested instructions. For example:

(EXTRACT_SUBREG (ADD x:s64, y:s64), sub_32)

could select to:

%2(A64) = ADD %0(A64), %1(A64)
%3(B64) = COPY %2(A64)
%4(B32) = COPY %3(B64), sub_32

where A64 and B64 are disjoint register classes and B32 is a subregister class of B64. Handling the extra copy outside the tree walking code would need a way to insert the copy into the right part of the emitted list. If we handle it during the tree walking code we can just append to the list as we go.

Here's a particularly complicated example from Mips:

def : MSAPat<
  (i32 (vextract_zext_i8 v16i8:$ws, i64:$idx)),
  (SRL (COPY_TO_REGCLASS
         (i32 (EXTRACT_SUBREG
                 (SPLAT_B v16i8:$ws,
                   (COPY_TO_REGCLASS
                     (i32 (EXTRACT_SUBREG i64:$idx, sub_32)), GPR32)),
                 sub_lo)),
         GPR32),
       (i32 24))>;

1878 ↗

(On Diff #104119)

Ah I see. I don't mind either way, I'll make the change.

1884 ↗

(On Diff #104119)

It depends how you view the data. The gMIR view is:

%0 = EXTRACT_SUBREG %1, subidx

in which case it's operand 1, but the DAG view is

(EXTRACT_SUBREG %1, subidx)

in which case it's child 1. I've been using 'operand' for the gMIR view and 'child' for the DAG view.

1896 ↗

(On Diff #104119)

Ok

include/llvm/CodeGen/GlobalISel/InstructionSelector.h

 namespace llvm {
 class MachineInstr;
 class MachineInstrBuilder;
 class MachineFunction;
 class MachineOperand;
 class MachineRegisterInfo;
 class RegisterBankInfo;
 class TargetInstrInfo;
+class TargetRegisterClass;
 class TargetRegisterInfo;
 
 /// Container class for CodeGen predicate results.
 /// This is convenient because std::bitset does not have a constructor
 /// with an initializer list of set bits.
 ///
 /// Each InstructionSelector subclass should define a PredicateBitset class with:
 ///   const unsigned MAX_SUBTARGET_PREDICATES = 192;
public:
   ///
   virtual bool select(MachineInstr &I) const = 0;
 
 protected:
   typedef std::function<void(MachineInstrBuilder &)> ComplexRendererFn;
 
   InstructionSelector();
 
+  bool constrainOperandRegToRegClass(MachineInstr &I, unsigned OpIdx,
+                                     const TargetRegisterClass *RC,
+                                     const TargetInstrInfo &TII,
+                                     const TargetRegisterInfo &TRI,
+                                     const RegisterBankInfo &RBI) const;
+
   /// Mutate the newly-selected instruction \p I to constrain its (possibly
   /// generic) virtual register operands to the instruction's register class.
   /// This could involve inserting COPYs before (for uses) or after (for defs).
   /// This requires the number of operands to match the instruction description.
   /// \returns whether operand regclass constraining succeeded.
   ///
   // FIXME: Not all instructions have the same number of operands. We should
   // probably expose a constrain helper per operand and let the target selector

include/llvm/CodeGen/GlobalISel/Utils.h

 class MachineOptimizationRemarkEmitter;
 class MachineOptimizationRemarkMissed;
 class MachineRegisterInfo;
 class MCInstrDesc;
 class RegisterBankInfo;
 class TargetInstrInfo;
 class TargetPassConfig;
 class TargetRegisterInfo;
+class TargetRegisterClass;
 class Twine;
 class ConstantFP;
 
+unsigned constrainRegToClass(MachineRegisterInfo &MRI,
+                             const TargetInstrInfo &TII,
+                             const RegisterBankInfo &RBI,
+                             MachineInstr &InsertPt, unsigned Reg,
+                             const TargetRegisterClass *RegClass);
+
 /// Try to constrain Reg so that it is usable by argument OpIdx of the
 /// provided MCInstrDesc \p II. If this fails, create a new virtual
 /// register in the correct class and insert a COPY before \p InsertPt.
 /// The debug location of \p InsertPt is used for the new copy.
 ///
 /// \return The virtual register constrained to the right register class.
 unsigned constrainOperandRegClass(const MachineFunction &MF,
                                   const TargetRegisterInfo &TRI,

lib/CodeGen/GlobalISel/InstructionSelector.cpp

 #include "llvm/Target/TargetRegisterInfo.h"
 
 #define DEBUG_TYPE "instructionselector"
 
 using namespace llvm;
 
 InstructionSelector::InstructionSelector() {}
 
+bool InstructionSelector::constrainOperandRegToRegClass(
+    MachineInstr &I, unsigned OpIdx, const TargetRegisterClass *RC,
+    const TargetInstrInfo &TII, const TargetRegisterInfo &TRI,
+    const RegisterBankInfo &RBI) const {
+  MachineBasicBlock &MBB = *I.getParent();
+  MachineFunction &MF = *MBB.getParent();
+  MachineRegisterInfo &MRI = MF.getRegInfo();
+
+  return llvm::constrainRegToClass(MRI, TII, RBI, I,
+                                   I.getOperand(OpIdx).getReg(), RC);
+}
+
 bool InstructionSelector::constrainSelectedInstRegOperands(
     MachineInstr &I, const TargetInstrInfo &TII, const TargetRegisterInfo &TRI,
     const RegisterBankInfo &RBI) const {
   MachineBasicBlock &MBB = *I.getParent();
   MachineFunction &MF = *MBB.getParent();
   MachineRegisterInfo &MRI = MF.getRegInfo();
 
   for (unsigned OpI = 0, OpE = I.getNumExplicitOperands(); OpI != OpE; ++OpI) {

lib/CodeGen/GlobalISel/Utils.cpp

 #include "llvm/IR/Constants.h"
 #include "llvm/Target/TargetInstrInfo.h"
 #include "llvm/Target/TargetRegisterInfo.h"
 
 #define DEBUG_TYPE "globalisel-utils"
 
 using namespace llvm;
 
+unsigned llvm::constrainRegToClass(MachineRegisterInfo &MRI,
+                                   const TargetInstrInfo &TII,
+                                   const RegisterBankInfo &RBI,
+                                   MachineInstr &InsertPt, unsigned Reg,
+                                   const TargetRegisterClass *RegClass) {
+  if (!RBI.constrainGenericRegister(Reg, *RegClass, MRI)) {
+    unsigned NewReg = MRI.createVirtualRegister(RegClass);
+    BuildMI(*InsertPt.getParent(), InsertPt, InsertPt.getDebugLoc(),
+            TII.get(TargetOpcode::COPY), NewReg)
+        .addReg(Reg);
+    return NewReg;
+  }
+
+  return Reg;
+}
+
+
 unsigned llvm::constrainOperandRegClass(
     const MachineFunction &MF, const TargetRegisterInfo &TRI,
     MachineRegisterInfo &MRI, const TargetInstrInfo &TII,
     const RegisterBankInfo &RBI, MachineInstr &InsertPt, const MCInstrDesc &II,
     unsigned Reg, unsigned OpIdx) {
   // Assume physical registers are properly constrained.
   assert(TargetRegisterInfo::isVirtualRegister(Reg) &&
          "PhysReg not implemented");
 
   const TargetRegisterClass *RegClass = TII.getRegClass(II, OpIdx, &TRI, MF);
-
-  if (!RBI.constrainGenericRegister(Reg, *RegClass, MRI)) {
-    unsigned NewReg = MRI.createVirtualRegister(RegClass);
-    BuildMI(*InsertPt.getParent(), InsertPt, InsertPt.getDebugLoc(),
-            TII.get(TargetOpcode::COPY), NewReg)
-        .addReg(Reg);
-    return NewReg;
-  }
-
-  return Reg;
+  return constrainRegToClass(MRI, TII, RBI, InsertPt, Reg, RegClass);
 }
 
 bool llvm::isTriviallyDead(const MachineInstr &MI,
                            const MachineRegisterInfo &MRI) {
   // If we can move an instruction, we can remove it.  Otherwise, it has
   // a side-effect of some sort.
   bool SawStore = false;
   if (!MI.isSafeToMove(/*AA=*/nullptr, SawStore))

lib/Target/AArch64/AArch64InstructionSelector.cpp

case TargetOpcode::G_PTR_MASK: {
 
     return constrainSelectedInstRegOperands(I, TII, TRI, RBI);
   }
   case TargetOpcode::G_PTRTOINT:
   case TargetOpcode::G_TRUNC: {
     const LLT DstTy = MRI.getType(I.getOperand(0).getReg());
     const LLT SrcTy = MRI.getType(I.getOperand(1).getReg());
 
+    if (DstTy == LLT::scalar(32) && SrcTy == LLT::scalar(64))
+      llvm_unreachable("Tablegen code can import this");
+
     const unsigned DstReg = I.getOperand(0).getReg();
     const unsigned SrcReg = I.getOperand(1).getReg();
 
     const RegisterBank &DstRB = *RBI.getRegBank(DstReg, MRI, TRI);
     const RegisterBank &SrcRB = *RBI.getRegBank(SrcReg, MRI, TRI);
 
     if (DstRB.getID() != SrcRB.getID()) {
       DEBUG(dbgs() << "G_TRUNC input/output on different banks\n");
       return false;
     }
 
     if (DstRB.getID() == AArch64::GPRRegBankID) {
       const TargetRegisterClass *DstRC =
           getRegClassForTypeOnBank(DstTy, DstRB, RBI);
       if (!DstRC)
         return false;
 
       const TargetRegisterClass *SrcRC =
           getRegClassForTypeOnBank(SrcTy, SrcRB, RBI);
       if (!SrcRC)
         return false;
 
       if (!RBI.constrainGenericRegister(SrcReg, *SrcRC, MRI) ||
           !RBI.constrainGenericRegister(DstReg, *DstRC, MRI)) {
         DEBUG(dbgs() << "Failed to constrain G_TRUNC\n");
         return false;
       }
 
       if (DstRC == SrcRC) {
         // Nothing to be done
       } else if (DstRC == &AArch64::GPR32RegClass &&
                  SrcRC == &AArch64::GPR64RegClass) {
         I.getOperand(1).setSubReg(AArch64::sub_32);
       } else {
         return false;
       }
 
       I.setDesc(TII.get(TargetOpcode::COPY));
       return true;
     } else if (DstRB.getID() == AArch64::FPRRegBankID) {

test/CodeGen/AArch64/GlobalISel/select-bitcast.mir

 ---
 # CHECK-LABEL: name: bitcast_s32_fpr_gpr
 name:            bitcast_s32_fpr_gpr
 legalized:       true
 regBankSelected: true
 
 # CHECK:      registers:
 # CHECK-NEXT:  - { id: 0, class: fpr32 }
-# CHECK-NEXT:  - { id: 1, class: gpr32all }
+# CHECK-NEXT:  - { id: 1, class: gpr32 }
 registers:
   - { id: 0, class: fpr }
   - { id: 1, class: gpr }
 
 # CHECK:  body:
 # CHECK:    %0 = COPY %s0
 # CHECK:    %1 = COPY %0
 body:             |

 ---
 # CHECK-LABEL: name: bitcast_s64_fpr_gpr
 name:            bitcast_s64_fpr_gpr
 legalized:       true
 regBankSelected: true
 
 # CHECK:      registers:
 # CHECK-NEXT:  - { id: 0, class: fpr64 }
-# CHECK-NEXT:  - { id: 1, class: gpr64all }
+# CHECK-NEXT:  - { id: 1, class: gpr64 }
 registers:
   - { id: 0, class: fpr }
   - { id: 1, class: gpr }
 
 # CHECK:  body:
 # CHECK:    %0 = COPY %d0
 # CHECK:    %1 = COPY %0
 body:             |
   bb.0:
     liveins: %d0
 
     %0(s64) = COPY %d0
     %1(s64) = G_BITCAST %0
     %x0 = COPY %1(s64)
 ...

test/CodeGen/AArch64/GlobalISel/select-trunc.mir

 
 ---
 # CHECK-LABEL: name: trunc_s32_s64
 name:            trunc_s32_s64
 legalized:       true
 regBankSelected: true
 
 # CHECK:      registers:
-# CHECK-NEXT:  - { id: 0, class: gpr64 }
-# CHECK-NEXT:  - { id: 1, class: gpr32 }
+# CHECK-NEXT:  - { id: 0, class: gpr64all }
+# CHECK-NEXT:  - { id: 1, class: gpr32sp }
 registers:
   - { id: 0, class: gpr }
   - { id: 1, class: gpr }
 
 # CHECK:  body:
 # CHECK:    %1 = COPY %0.sub_32
 body:             |
   bb.0:

test/TableGen/GlobalISelEmitter.td

 // RUN: llvm-tblgen -gen-global-isel -I %p/../../include %s | FileCheck %s
 
 include "llvm/Target/Target.td"
 
 //===- Define the necessary boilerplate for our test target. --------------===//
 
 def MyTargetISA : InstrInfo;
 def MyTarget : Target { let InstructionSet = MyTargetISA; }
 
 def R0 : Register<"r0"> { let Namespace = "MyTarget"; }
 def GPR32 : RegisterClass<"MyTarget", [i32], 32, (add R0)>;
 def GPR32Op : RegisterOperand<GPR32>;
+def F0 : Register<"f0"> { let Namespace = "MyTarget"; }
+def FPR32 : RegisterClass<"MyTarget", [f32], 32, (add F0)>;
 
 class I<dag OOps, dag IOps, list<dag> Pat>
   : Instruction {
   let Namespace = "MyTarget";
   let OutOperandList = OOps;
   let InOperandList = IOps;
   let Pattern = Pat;
 }

 // CHECK-NEXT:      return true;
 // CHECK-NEXT:    }
 // CHECK-NEXT:    return false;
 // CHECK-NEXT:  }()) { return true; }
 
 def ORN : I<(outs GPR32:$dst), (ins GPR32:$src1, GPR32:$src2), []>;
 def : Pat<(not GPR32:$Wm), (ORN R0, GPR32:$Wm)>;
 
+//===- Test a COPY_TO_REGCLASS --------------------------------------------===//
+//
+
+// CHECK-LABEL: if ([&]() {
+// CHECK-NEXT:    MachineInstr &MI0 = I;
+// CHECK-NEXT:    if (MI0.getNumOperands() < 2)
+// CHECK-NEXT:      return false;
+// CHECK-NEXT:    if ((MI0.getOpcode() == TargetOpcode::G_BITCAST) &&
+// CHECK-NEXT:        ((/* dst */ (MRI.getType(MI0.getOperand(0).getReg()) == (LLT::scalar(32))) &&
+// CHECK-NEXT:         ((&RBI.getRegBankFromRegClass(MyTarget::GPR32RegClass) == RBI.getRegBank(MI0.getOperand(0).getReg(), MRI, TRI))))) &&
+// CHECK-NEXT:        ((/* src1 */ (MRI.getType(MI0.getOperand(1).getReg()) == (LLT::scalar(32))) &&
+// CHECK-NEXT:         ((&RBI.getRegBankFromRegClass(MyTarget::FPR32RegClass) == RBI.getRegBank(MI0.getOperand(1).getReg(), MRI, TRI))))))
+// CHECK-NEXT:      // (bitconvert:i32 FPR32:f32:$src1) => (COPY_TO_REGCLASS:i32 FPR32:f32:$src1, GPR32:i32)
+// CHECK-NEXT:      I.setDesc(TII.get(TargetOpcode::COPY));
+// CHECK-NEXT:      MachineInstr &NewI = I;
+// CHECK-NEXT:      constrainOperandRegToRegClass(NewI, 0, &MyTarget::GPR32RegClass, TII, TRI, RBI);
+// CHECK-NEXT:      return true;
+// CHECK-NEXT:    }
+// CHECK-NEXT:    return false;
+// CHECK-NEXT:  }()) { return true; }
+
+// The -6 is just to distinguish it from the other 'not' cases.
+def : Pat<(i32 (bitconvert FPR32:$src1)),
+          (COPY_TO_REGCLASS FPR32:$src1, GPR32)>;
+
 //===- Test a pattern with an MBB operand. --------------------------------===//
 
 // CHECK-LABEL: if ([&]() {
 // CHECK-NEXT:      MachineInstr &MI0 = I;
 // CHECK-NEXT:      if (MI0.getNumOperands() < 1)
 // CHECK-NEXT:        return false;
 // CHECK-NEXT:      if ((MI0.getOpcode() == TargetOpcode::G_BR) &&
 // CHECK-NEXT:          ((/* target */ (MI0.getOperand(0).isMBB())))) {

utils/TableGen/CodeGenRegisters.h

public:
 
     // Return the Record that defined this class, or NULL if the class was
     // created by TableGen.
     Record *getDef() const { return TheDef; }
 
     const std::string &getName() const { return Name; }
     std::string getQualifiedName() const;
     ArrayRef<MVT::SimpleValueType> getValueTypes() const {return VTs;}
+    bool hasValueType(MVT::SimpleValueType VT) const {
+      return std::find(VTs.begin(), VTs.end(), VT) != VTs.end();
+    }
     unsigned getNumValueTypes() const { return VTs.size(); }
 
     MVT::SimpleValueType getValueTypeNum(unsigned VTNum) const {
       if (VTNum < VTs.size())
         return VTs[VTNum];
       llvm_unreachable("VTNum greater than number of ValueTypes in RegClass!");
     }
 
public:
 
     // getSubClassWithSubReg - Returns the largest sub-class where all
     // registers have a SubIdx sub-register.
     CodeGenRegisterClass *
     getSubClassWithSubReg(const CodeGenSubRegIndex *SubIdx) const {
       return SubClassWithSubReg.lookup(SubIdx);
     }
 
+    /// Find largest subclass where all registers have SubIdx subregisters in
+    /// SubRegClass and the largest subregister class that contains those
+    /// subregisters without (as far as possible) also containing additional registers.
+    ///
+    /// This can be used to find a suitable pair of classes for subregister copies.
+    CodeGenRegisterClass *
+    getMatchingSubClassWithSubRegs(CodeGenRegBank &RegBank,
+                                   const CodeGenSubRegIndex *SubIdx,
+                                   CodeGenRegisterClass **SubRegClass) const;
+
     void setSubClassWithSubReg(const CodeGenSubRegIndex *SubIdx,
                                CodeGenRegisterClass *SubRC) {
       SubClassWithSubReg[SubIdx] = SubRC;
     }
 
     // getSuperRegClasses - Returns a bit vector of all register classes
     // containing only SubIdx super-registers of this class.
     void getSuperRegClasses(const CodeGenSubRegIndex *SubIdx,
                             BitVector &Out) const;
 
-    // addSuperRegClass - Add a class containing only SudIdx super-registers.
+    // addSuperRegClass - Add a class containing only SubIdx super-registers.
     void addSuperRegClass(CodeGenSubRegIndex *SubIdx,
                           CodeGenRegisterClass *SuperRC) {
       SuperRegClasses[SubIdx].insert(SuperRC);
     }
 
     // getSubClasses - Returns a constant BitVector of subclasses indexed by
     // EnumValue.
     // The SubClasses vector includes an entry for this class.

utils/TableGen/CodeGenRegisters.cpp

void CodeGenRegisterClass::computeSubClasses(CodeGenRegBank &RegBank) {
   // With the class hierarchy in place, let synthesized register classes inherit
   // properties from their closest super-class. The iteration order here can
   // propagate properties down multiple levels.
   for (auto &RC : RegClasses)
     if (!RC.getDef())
       RC.inheritProperties(RegBank);
 }
 
+CodeGenRegisterClass *CodeGenRegisterClass::getMatchingSubClassWithSubRegs(
+    CodeGenRegBank &RegBank, const CodeGenSubRegIndex *SubIdx,
+    CodeGenRegisterClass **SubRegRC) const {
+  auto SizeOrder = [](const CodeGenRegisterClass *A,
+                      const CodeGenRegisterClass *B) {
+    return A->getMembers().size() >= B->getMembers().size();
+  };
+
+  auto &RegClasses = RegBank.getRegClasses();
+
+  // Find all the subclasses of this one that fully support the sub-register
+  // index and order them by size. BiggestSuperRC should always be first.
+  CodeGenRegisterClass *BiggestSuperRegRC = getSubClassWithSubReg(SubIdx);
+  if (!BiggestSuperRegRC)
+    return nullptr;
+  BitVector SuperRegRCsBV = BiggestSuperRegRC->getSubClasses();
+  std::vector<CodeGenRegisterClass *> SuperRegRCs;
+  for (auto &RC : RegClasses)
+    if (SuperRegRCsBV[RC.EnumValue])
+      SuperRegRCs.emplace_back(&RC);
+  std::sort(SuperRegRCs.begin(), SuperRegRCs.end(), SizeOrder);
+  assert(SuperRegRCs.front() == BiggestSuperRegRC && "Biggest class wasn't first");
+
+  // Find all the subreg classes and order them by size too.
+  std::vector<std::pair<CodeGenRegisterClass *, BitVector>> SuperRegClasses;
+  for (auto &RC: RegClasses) {
+    BitVector SuperRegClassesBV(RegClasses.size());
+    RC.getSuperRegClasses(SubIdx, SuperRegClassesBV);
+    if (SuperRegClassesBV.any())
+      SuperRegClasses.push_back(std::make_pair(&RC, SuperRegClassesBV));
+  }
+  std::sort(SuperRegClasses.begin(), SuperRegClasses.end(),
+            [&](const std::pair<CodeGenRegisterClass *, BitVector> &A,
+                const std::pair<CodeGenRegisterClass *, BitVector> &B) {
+              return SizeOrder(A.first, B.first);
+            });
+
+  // Find the biggest subclass and subreg class such that R:subidx is in the
+  // subreg class for all R in subclass.
+  //
+  // For example:
+  // All registers in X86's GR64 have a sub_32bit subregister but no class
+  // exists that contains all the 32-bit subregisters because GR64 contains RIP
+  // but GR32 does not contain EIP. Instead, we constrain SuperRegRC to
+  // GR32_with_sub_8bit (which is identical to GR32_with_sub_32bit) and then,
+  // having excluded RIP, we are able to find a SubRegRC (GR32).
+  CodeGenRegisterClass *ChosenSuperRegClass = nullptr;
+  for (auto *SuperRegRC : SuperRegRCs) {
+    for (const auto &SuperRegClassPair : SuperRegClasses) {
+      const BitVector &SuperRegClassBV = SuperRegClassPair.second;
+      if (SuperRegClassBV[SuperRegRC->EnumValue]) {
+        *SubRegRC = SuperRegClassPair.first;
+        ChosenSuperRegClass = SuperRegRC;
+
+        // If SubRegRC is bigger than SuperRegRC then there are members of
+        // SubRegRC that don't have super registers via SubIdx. Keep looking to
+        // find a better fit and fall back on this one if there isn't one.
+        //
+        // This is intended to prevent X86 from making odd choices such as
+        // picking LOW32_ADDR_ACCESS_RBP instead of GR32 in the example above.
+        // LOW32_ADDR_ACCESS_RBP is a valid choice but contains registers that
+        // aren't subregisters of SuperRegRC whereas GR32 has a direct 1:1
+        // mapping.
+        if (SuperRegRC->getMembers().size() >= (*SubRegRC)->getMembers().size())
+          return ChosenSuperRegClass;
+      }
+    }
+
+    // If we found a fit but it wasn't quite ideal because SubRegRC had excess
+    // registers, then we're done.
+    if (ChosenSuperRegClass)
+      return ChosenSuperRegClass;
+  }
+
+  return nullptr;
+}
+
 void CodeGenRegisterClass::getSuperRegClasses(const CodeGenSubRegIndex *SubIdx,
                                               BitVector &Out) const {
   auto FindI = SuperRegClasses.find(SubIdx);
   if (FindI == SuperRegClasses.end())
     return;
   for (CodeGenRegisterClass *RC : FindI->second)
     Out.set(RC->EnumValue);
 }

utils/TableGen/GlobalISelEmitter.cpp

static Error isTrivialOperatorNode(const TreePatternNode *N) {
   }
 
   if (!N->isLeaf() && !N->hasAnyPredicate() && !N->getTransformFn())
     return Error::success();
 
   return failedImport(Explanation);
 }
 
+static Record *getInitValueAsRegClass(Init *V) {
+  if (DefInit *VDefInit = dyn_cast<DefInit>(V)) {
+    if (VDefInit->getDef()->isSubClassOf("RegisterOperand"))
+      return VDefInit->getDef()->getValueAsDef("RegClass");
+    if (VDefInit->getDef()->isSubClassOf("RegisterClass"))
+      return VDefInit->getDef();
+  }
+  return nullptr;
+}
+
 //===- Matchers -----------------------------------------------------------===//
 
 class OperandMatcher;
 class MatchAction;
 
 /// Generates code to check that a match rule matches.
 class RuleMatcher {
   /// A list of matchers that all need to succeed for the current rule to match.
void emitCxxPredicateExpr(raw_ostream &OS, RuleMatcher &Rule,
     InsnMatcher->emitCxxPredicateExpr(OS, Rule, OperandExpr);
     OS << ")\n";
   }
 };
 
 //===- Actions ------------------------------------------------------------===//
 class OperandRenderer {
 public:
-  enum RendererKind { OR_Copy, OR_Imm, OR_Register, OR_ComplexPattern };
+  enum RendererKind {
+    OR_Copy,
+    OR_CopySubReg,
+    OR_Imm,
+    OR_Register,
+    OR_ComplexPattern
+  };
 
 protected:
   RendererKind Kind;
 
 public:
   OperandRenderer(RendererKind Kind) : Kind(Kind) {}
   virtual ~OperandRenderer() {}
 
void emitCxxRenderStmts(raw_ostream &OS, RuleMatcher &Rule) const override {
     const OperandMatcher &Operand = Matched.getOperand(SymbolicName);
     StringRef InsnVarName =
         Rule.getInsnVarName(Operand.getInstructionMatcher());
     std::string OperandExpr = Operand.getOperandExpr(InsnVarName);
     OS << "    MIB.add(" << OperandExpr << "/*" << SymbolicName << "*/);\n";
   }
 };
 
+/// A CopySubRegRenderer emits code to copy a single register operand from an
+/// existing instruction to the one being built and indicate that only a
+/// subregister should be copied.
+class CopySubRegRenderer : public OperandRenderer {
+protected:
+  /// The matcher for the instruction that this operand is copied from.
+  /// This provides the facility for looking up an a operand by it's name so
+  /// that it can be used as a source for the instruction being built.
+  const InstructionMatcher &Matched;
+  /// The name of the operand.
+  const StringRef SymbolicName;
+  // The subregister to extract.
+  const CodeGenSubRegIndex *SubReg;
+
+public:
+  CopySubRegRenderer(const InstructionMatcher &Matched, StringRef SymbolicName,
+                     const CodeGenSubRegIndex *SubReg)
+      : OperandRenderer(OR_CopySubReg), Matched(Matched),
+        SymbolicName(SymbolicName), SubReg(SubReg) {}
+
+  static bool classof(const OperandRenderer *R) {
+    return R->getKind() == OR_CopySubReg;
+  }
+
+  const StringRef getSymbolicName() const { return SymbolicName; }
+
+  void emitCxxRenderStmts(raw_ostream &OS, RuleMatcher &Rule) const override {
+    const OperandMatcher &Operand = Matched.getOperand(SymbolicName);
+    StringRef InsnVarName =
+        Rule.getInsnVarName(Operand.getInstructionMatcher());
+    std::string OperandExpr = Operand.getOperandExpr(InsnVarName);
+    OS << "    MIB.addReg(" << OperandExpr << ".getReg() /*" << SymbolicName
+       << "*/, 0, " << SubReg->EnumValue << ");\n";
+  }
+};
+
 /// Adds a specific physical register to the instruction being built.
 /// This is typically useful for WZR/XZR on AArch64.
 class AddRegisterRenderer : public OperandRenderer {
 protected:
   const Record *RegisterDef;
 
 public:
   AddRegisterRenderer(const Record *RegisterDef)
void emitCxxActionStmts(raw_ostream &OS, RuleMatcher &Rule,
     OS << "// " << *P.getSrcPattern() << "  =>  " << *P.getDstPattern() << "\n";
   }
 };
 
 /// Generates code to build an instruction or mutate an existing instruction
 /// into the desired instruction when this is possible.
 class BuildMIAction : public MatchAction {
 private:
+  std::string Name;
   const CodeGenInstruction *I;
   const InstructionMatcher &Matched;
   std::vector<std::unique_ptr<OperandRenderer>> OperandRenderers;
 
   /// True if the instruction can be built solely by mutating the opcode.
   bool canMutate() const {
     if (OperandRenderers.size() != Matched.getNumOperands())
       return false;
 
     for (const auto &Renderer : enumerate(OperandRenderers)) {
       if (const auto *Copy = dyn_cast<CopyRenderer>(&*Renderer.value())) {
         const OperandMatcher &OM = Matched.getOperand(Copy->getSymbolicName());
         if (&Matched != &OM.getInstructionMatcher() ||
             OM.getOperandIndex() != Renderer.index())
           return false;
       } else
         return false;
     }
 
     return true;
   }
 
 public:
-  BuildMIAction(const CodeGenInstruction *I, const InstructionMatcher &Matched)
-      : I(I), Matched(Matched) {}
+  BuildMIAction(const StringRef Name, const CodeGenInstruction *I,
+                const InstructionMatcher &Matched)
+      : Name(Name), I(I), Matched(Matched) {}
 
   template <class Kind, class... Args>
   Kind &addRenderer(Args&&... args) {
     OperandRenderers.emplace_back(
         llvm::make_unique<Kind>(std::forward<Args>(args)...));
     return *static_cast<Kind *>(OperandRenderers.back().get());
   }
 
if (canMutate()) {
           auto Namespace = Use->getValue("Namespace")
                                ? Use->getValueAsString("Namespace")
                                : "";
           OS << "    MIB.addUse(" << Namespace << "::" << Use->getName()
              << ", RegState::Implicit);\n";
         }
       }
 
-      OS << "    MachineInstr &NewI = " << RecycleVarName << ";\n";
+      OS << "    MachineInstr &" << Name << " = " << RecycleVarName << ";\n";
       return;
     }
 
     // TODO: Simple permutation looks like it could be almost as common as
     //       mutation due to commutative operations.
 
     OS << "MachineInstrBuilder MIB = BuildMI(*I.getParent(), I, "
           "I.getDebugLoc(), TII.get("
        << I->Namespace << "::" << I->TheDef->getName() << "));\n";
     for (const auto &Renderer : OperandRenderers)
       Renderer->emitCxxRenderStmts(OS, Rule);
     OS << "    for (const auto *FromMI : ";
     Rule.emitCxxCapturedInsnList(OS);
     OS << ")\n";
     OS << "      for (const auto &MMO : FromMI->memoperands())\n";
     OS << "        MIB.addMemOperand(MMO);\n";
     OS << "    " << RecycleVarName << ".eraseFromParent();\n";
-    OS << "    MachineInstr &NewI = *MIB;\n";
+    OS << "    MachineInstr &" << Name << " = *MIB;\n";
+  }
+};
+
+class ConstrainOperandsToDefinitionAction : public MatchAction {
+private:
+  std::string Name;
+
+public:
+  ConstrainOperandsToDefinitionAction(const StringRef Name) : Name(Name) {}
+
+  void emitCxxActionStmts(raw_ostream &OS, RuleMatcher &Rule,
+                          StringRef RecycleVarName) const override {
+    OS << "      constrainSelectedInstRegOperands(" << Name << ", TII, TRI, RBI);\n";
+  }
+};
+
+class ConstrainOperandToRegClassAction : public MatchAction {
+private:
+  std::string Name;
+  unsigned OpIdx;
+  const CodeGenRegisterClass &RC;
+
+public:
+  ConstrainOperandToRegClassAction(const StringRef Name, unsigned OpIdx,
+                                   const CodeGenRegisterClass &RC)
+      : Name(Name), OpIdx(OpIdx), RC(RC) {}
+
+  void emitCxxActionStmts(raw_ostream &OS, RuleMatcher &Rule,
+                          StringRef RecycleVarName) const override {
+    OS << "      constrainOperandRegToRegClass(" << Name << ", " << OpIdx
+       << ", &" << RC.getQualifiedName() << "RegClass, TII, TRI, RBI);\n";
   }
 };
 
 InstructionMatcher &RuleMatcher::addInstructionMatcher() {
   Matchers.emplace_back(new InstructionMatcher());
   return *Matchers.back();
 }
 
for (const auto &Pair : InsnVariableNames) {
       //          by rejecting all loads.
     }
   }
 
   for (const auto &MA : Actions) {
     MA->emitCxxActionStmts(OS, *this, "I");
   }
 
-  OS << "      constrainSelectedInstRegOperands(NewI, TII, TRI, RBI);\n";
   OS << "      return true;\n";
   OS << "    }\n";
   OS << "    return false;\n";
   OS << "  }()) { return true; }\n\n";
 }
 
 bool RuleMatcher::isHigherPriorityThan(const RuleMatcher &B) const {
   // Rules involving more match roots have higher priority.

 public:
   explicit GlobalISelEmitter(RecordKeeper &RK);
   void run(raw_ostream &OS);
 
 private:
   const RecordKeeper &RK;
   const CodeGenDAGPatterns CGP;
   const CodeGenTarget &Target;
+  CodeGenRegBank CGRegs;
 
   /// Keep track of the equivalence between SDNodes and Instruction.
   /// This is defined using 'GINodeEquiv' in the target description.
   DenseMap<Record *, const CodeGenInstruction *> NodeEquivs;
 
   /// Keep track of the equivalence between ComplexPattern's and
   /// GIComplexOperandMatcher. Map entries are specified by subclassing
   /// GIComplexPatternEquiv.
   DenseMap<const Record *, const Record *> ComplexPatternEquivs;
 
   // Map of predicates to their subtarget features.
   SubtargetFeatureInfoMap SubtargetFeatures;
 
   void gatherNodeEquivs();
   const CodeGenInstruction *findNodeEquiv(Record *N) const;
 
   Error importRulePredicates(RuleMatcher &M, ArrayRef<Init *> Predicates);
   Expected<InstructionMatcher &>
   createAndImportSelDAGMatcher(InstructionMatcher &InsnMatcher,
                                const TreePatternNode *Src) const;
   Error importChildMatcher(InstructionMatcher &InsnMatcher,
                            TreePatternNode *SrcChild, unsigned OpIdx,
                            unsigned &TempOpIdx) const;
-  Expected<BuildMIAction &> createAndImportInstructionRenderer(
-      RuleMatcher &M, const TreePatternNode *Dst,
-      const InstructionMatcher &InsnMatcher) const;
+  Expected<BuildMIAction &>
+  createAndImportInstructionRenderer(RuleMatcher &M, const TreePatternNode *Dst,
+                                     const InstructionMatcher &InsnMatcher);
   Error importExplicitUseRenderer(BuildMIAction &DstMIBuilder,
                                   TreePatternNode *DstChild,
                                   const InstructionMatcher &InsnMatcher) const;
   Error importDefaultOperandRenderers(BuildMIAction &DstMIBuilder,
                                       DagInit *DefaultOps) const;
   Error
   importImplicitDefRenderers(BuildMIAction &DstMIBuilder,
                              const std::vector<Record *> &ImplicitDefs) const;

  }
 }
 
 const CodeGenInstruction *GlobalISelEmitter::findNodeEquiv(Record *N) const {
   return NodeEquivs.lookup(N);
 }
 
 GlobalISelEmitter::GlobalISelEmitter(RecordKeeper &RK)
-    : RK(RK), CGP(RK), Target(CGP.getTargetInfo()) {}
+    : RK(RK), CGP(RK), Target(CGP.getTargetInfo()), CGRegs(RK) {}
 
 //===- Emitter ------------------------------------------------------------===//
 
 Error
 GlobalISelEmitter::importRulePredicates(RuleMatcher &M,
                                         ArrayRef<Init *> Predicates) {
   for (const Init *Predicate : Predicates) {
     const DefInit *PredicateDef = static_cast<const DefInit *>(Predicate);
if (auto *ChildInt = dyn_cast<IntInit>(SrcChild->getLeafValue())) {
     return Error::success();
   }
 
   // Check for def's like register classes or ComplexPattern's.
   if (auto *ChildDefInit = dyn_cast<DefInit>(SrcChild->getLeafValue())) {
     auto *ChildRec = ChildDefInit->getDef();
 
     // Check for register classes.
-    if (ChildRec->isSubClassOf("RegisterClass")) {
-      OM.addPredicate<RegisterBankOperandMatcher>(
-          Target.getRegisterClass(ChildRec));
-      return Error::success();
-    }
-
-    if (ChildRec->isSubClassOf("RegisterOperand")) {
+    if (ChildRec->isSubClassOf("RegisterClass") ||
+        ChildRec->isSubClassOf("RegisterOperand")) {
       OM.addPredicate<RegisterBankOperandMatcher>(
-          Target.getRegisterClass(ChildRec->getValueAsDef("RegClass")));
+          Target.getRegisterClass(getInitValueAsRegClass(ChildDefInit)));
       return Error::success();
     }
 
     // Check for ComplexPattern's.
     if (ChildRec->isSubClassOf("ComplexPattern")) {
       const auto &ComplexPattern = ComplexPatternEquivs.find(ChildRec);
       if (ComplexPattern == ComplexPatternEquivs.end())
         return failedImport("SelectionDAG ComplexPattern (" +
return failedImport(
         "Dst pattern child def is an unsupported tablegen class");
   }
 
   return failedImport("Dst pattern child is an unsupported kind");
 }
 
 Expected<BuildMIAction &> GlobalISelEmitter::createAndImportInstructionRenderer(
     RuleMatcher &M, const TreePatternNode *Dst,
-    const InstructionMatcher &InsnMatcher) const {
+    const InstructionMatcher &InsnMatcher) {
   Record *DstOp = Dst->getOperator();
   if (!DstOp->isSubClassOf("Instruction")) {
     if (DstOp->isSubClassOf("ValueType"))
       return failedImport(
           "Pattern operator isn't an instruction (it's a ValueType)");
     return failedImport("Pattern operator isn't an instruction");
   }
-  auto &DstI = Target.getInstruction(DstOp);
+  auto *DstI = &Target.getInstruction(DstOp);
 
-  auto &DstMIBuilder = M.addAction<BuildMIAction>(&DstI, InsnMatcher);
+  unsigned DstINumUses = DstI->Operands.size() - DstI->Operands.NumDefs;
+  unsigned ExpectedDstINumUses = Dst->getNumChildren();
+  bool IsExtractSubReg = false;
+
+  // COPY_TO_REGCLASS is just a copy with a ConstrainOperandToRegClassAction
+  // attached. Similarly for EXTRACT_SUBREG except that's a subregister copy.
+  if (DstI->TheDef->getName() == "COPY_TO_REGCLASS") {
+    DstI = &Target.getInstruction(RK.getDef("COPY"));
+    DstINumUses--; // Ignore the class constraint.
+    ExpectedDstINumUses--;
+  } else if (DstI->TheDef->getName() == "EXTRACT_SUBREG") {
+    DstI = &Target.getInstruction(RK.getDef("COPY"));
+    IsExtractSubReg = true;
+  }
+
+  auto &DstMIBuilder = M.addAction<BuildMIAction>("NewI", DstI, InsnMatcher);
 
   // Render the explicit defs.
-  for (unsigned I = 0; I < DstI.Operands.NumDefs; ++I) {
-    const auto &DstIOperand = DstI.Operands[I];
+  for (unsigned I = 0; I < DstI->Operands.NumDefs; ++I) {
+    const auto &DstIOperand = DstI->Operands[I];
     DstMIBuilder.addRenderer<CopyRenderer>(InsnMatcher, DstIOperand.Name);
   }
 
+  // EXTRACT_SUBREG needs to use a subregister COPY.
+  if (IsExtractSubReg) {
+    if (!Dst->getChild(0)->isLeaf())
+      return failedImport("EXTRACT_SUBREG child #1 is not a leaf");
+
+    if (DefInit *SubRegInit = dyn_cast<DefInit>(Dst->getChild(1)->getLeafValue())) {
+      CodeGenRegisterClass *RC = CGRegs.getRegClass(
+          getInitValueAsRegClass(Dst->getChild(0)->getLeafValue()));
+      CodeGenSubRegIndex *SubIdx = CGRegs.getSubRegIdx(SubRegInit->getDef());
+
+      CodeGenRegisterClass *DstRC = nullptr;
+      CodeGenRegisterClass *SrcRC =
+          RC->getMatchingSubClassWithSubRegs(CGRegs, SubIdx, &DstRC);
+      assert(DstRC && "Couldn't find a matching subclass");
+      if (SrcRC != RC)
+        return failedImport("EXTRACT_SUBREG requires an additional COPY");
+
+      DstMIBuilder.addRenderer<CopySubRegRenderer>(
+          InsnMatcher, Dst->getChild(0)->getName(), SubIdx);
+      return DstMIBuilder;
+    }
+
+    return failedImport("EXTRACT_SUBREG child #1 is not a subreg index");
+  }
+
   // Render the explicit uses.
   unsigned Child = 0;
-  unsigned DstINumUses = DstI.Operands.size() - DstI.Operands.NumDefs;
   unsigned NumDefaultOps = 0;
   for (unsigned I = 0; I != DstINumUses; ++I) {
-    const auto &DstIOperand = DstI.Operands[DstI.Operands.NumDefs + I];
+    const auto &DstIOperand = DstI->Operands[DstI->Operands.NumDefs + I];
 
     // If the operand has default values, introduce them now.
     // FIXME: Until we have a decent test case that dictates we should do
     // otherwise, we're going to assume that operands with default values cannot
     // be specified in the patterns. Therefore, adding them will not cause us to
     // end up with too many rendered operands.
     if (DstIOperand.Rec->isSubClassOf("OperandWithDefaultOps")) {
       DagInit *DefaultOps = DstIOperand.Rec->getValueAsDag("DefaultOps");
       if (auto Error = importDefaultOperandRenderers(DstMIBuilder, DefaultOps))
         return std::move(Error);
       ++NumDefaultOps;
       continue;
     }
 
     if (auto Error = importExplicitUseRenderer(
             DstMIBuilder, Dst->getChild(Child), InsnMatcher))
       return std::move(Error);
     ++Child;
   }
 
-  if (NumDefaultOps + Dst->getNumChildren() != DstINumUses)
+  if (NumDefaultOps + ExpectedDstINumUses != DstINumUses)
     return failedImport("Expected " + llvm::to_string(DstINumUses) +
                         " used operands but found " +
-                        llvm::to_string(Dst->getNumChildren()) +
+                        llvm::to_string(ExpectedDstINumUses) +
                         " explicit ones and " + llvm::to_string(NumDefaultOps) +
                         " default ones");
 
   return DstMIBuilder;
 }
 
 Error GlobalISelEmitter::importDefaultOperandRenderers(
     BuildMIAction &DstMIBuilder, DagInit *DefaultOps) const {
Expected<RuleMatcher> GlobalISelEmitter::runOnPattern(const PatternToMatch &P) {
   // The root of the match also has constraints on the register bank so that it
   // matches the result instruction.
   unsigned OpIdx = 0;
   for (const EEVT::TypeSet &Ty : Src->getExtTypes()) {
     (void)Ty;
 
     const auto &DstIOperand = DstI.Operands[OpIdx];
     Record *DstIOpRec = DstIOperand.Rec;
-    if (DstIOpRec->isSubClassOf("RegisterOperand"))
+    if (DstI.TheDef->getName() == "COPY_TO_REGCLASS") {
+      DstIOpRec = getInitValueAsRegClass(Dst->getChild(1)->getLeafValue());
+
+      if (DstIOpRec == nullptr)
+        return failedImport(
+            "COPY_TO_REGCLASS operand #1 isn't a register class");
+    } else if (DstI.TheDef->getName() == "EXTRACT_SUBREG") {
+      if (!Dst->getChild(0)->isLeaf())
+        return failedImport("EXTRACT_SUBREG operand #0 isn't a leaf");
+
+      // We can assume that a subregister is in the same bank as it's super register.
+      DstIOpRec = getInitValueAsRegClass(Dst->getChild(0)->getLeafValue());
+
+      if (DstIOpRec == nullptr)
+        return failedImport(
+            "EXTRACT_SUBREG operand #0 isn't a register class");
+    } else if (DstIOpRec->isSubClassOf("RegisterOperand"))
       DstIOpRec = DstIOpRec->getValueAsDef("RegClass");
-    if (!DstIOpRec->isSubClassOf("RegisterClass"))
-      return failedImport("Dst MI def isn't a register class");
+    else if (!DstIOpRec->isSubClassOf("RegisterClass"))
+      return failedImport("Dst MI def isn't a register class" + to_string(*Dst));
 
     OperandMatcher &OM = InsnMatcher.getOperand(OpIdx);
     OM.setSymbolicName(DstIOperand.Name);
     OM.addPredicate<RegisterBankOperandMatcher>(
         Target.getRegisterClass(DstIOpRec));
     ++OpIdx;
   }
 
   auto DstMIBuilderOrError =
       createAndImportInstructionRenderer(M, Dst, InsnMatcher);
   if (auto Error = DstMIBuilderOrError.takeError())
     return std::move(Error);
   BuildMIAction &DstMIBuilder = DstMIBuilderOrError.get();
 
   // Render the implicit defs.
   // These are only added to the root of the result.
   if (auto Error = importImplicitDefRenderers(DstMIBuilder, P.getDstRegs()))
     return std::move(Error);
 
+  // Constrain the registers to classes. This is normally derived from the
+  // emitted instruction but a few instructions require special handling.
+  if (DstI.TheDef->getName() == "COPY_TO_REGCLASS") {
+    // COPY_TO_REGCLASS does not provide operand constraints itself but the
+    // result is constrained to the class given by the second child.
+    Record *DstIOpRec =
+        getInitValueAsRegClass(Dst->getChild(1)->getLeafValue());
+
+    if (DstIOpRec == nullptr)
+      return failedImport("COPY_TO_REGCLASS operand #1 isn't a register class");
+
+    M.addAction<ConstrainOperandToRegClassAction>(
+        "NewI", 0, Target.getRegisterClass(DstIOpRec));
+  } else if (DstI.TheDef->getName() == "EXTRACT_SUBREG") {
+    // EXTRACT_SUBREG selects into a subregister COPY but unlike most
+    // instructions, the result register class is controlled by the
+    // subregisters of the operand. As a result, we must constrain the result
+    // class rather than check that it's already the right one.
+    if (!Dst->getChild(0)->isLeaf())
+      return failedImport("EXTRACT_SUBREG child #1 is not a leaf");
+
+    if (DefInit *SubRegInit =
+            dyn_cast<DefInit>(Dst->getChild(1)->getLeafValue())) {
+      // Constrain the result to the same register bank as the operand.
+      Record *DstIOpRec =
+          getInitValueAsRegClass(Dst->getChild(0)->getLeafValue());
+
+      if (DstIOpRec == nullptr)
+        return failedImport(
+            "COPY_TO_REGCLASS operand #1 isn't a register class");
+
+      CodeGenSubRegIndex *SubIdx = CGRegs.getSubRegIdx(SubRegInit->getDef());
+      CodeGenRegisterClass *SrcRC = CGRegs.getRegClass(
+          getInitValueAsRegClass(Dst->getChild(0)->getLeafValue()));
+
+      // It would be nice to leave this constraint implicit but we're required
+      // to pick a register class so constrain the result to a register class
+      // that can hold the correct MVT.
+      //
+      // FIXME: This may introduce an extra copy if the chosen class doesn't
+      //        actually contain the subregisters.
+      assert(Src->getExtTypes().size() == 1);
+
+      CodeGenRegisterClass *DstRC = nullptr;
+      SrcRC = SrcRC->getMatchingSubClassWithSubRegs(CGRegs, SubIdx, &DstRC);
+      assert(DstRC && "Couldn't find a matching subclass");
+      M.addAction<ConstrainOperandToRegClassAction>("NewI", 0, *DstRC);
+
+      return M;
+    }
+
+    return failedImport("EXTRACT_SUBREG child #1 is not a subreg index");
+  } else
+    M.addAction<ConstrainOperandsToDefinitionAction>("NewI");
+
   // We're done with this pattern!  It's eligible for GISel emission; return it.
   ++NumPatternImported;
   return std::move(M);
 }
 
 void GlobalISelEmitter::run(raw_ostream &OS) {
   // Track the GINodeEquiv definitions.
   gatherNodeEquivs();