[globalisel][tablegen] Add support for COPY_TO_REGCLASS.

[dsanders]

Summary

As part of this

• Emitted instructions now have named MachineInstr variables associated with them. This isn't particularly important yet but it's a small step towards multiple-insn emission.
• constrainSelectedInstRegOperands() is no longer hardcoded. It's now added as the ConstrainOperandsToDefinitionAction() action. COPY_TO_REGCLASS uses an alternate constraint mechanism ConstrainOperandToRegClassAction() which supports arbitrary constraints such as that defined by COPY_TO_REGCLASS.

[qcolombet]

Hi Daniel,

Looks globally fine. Mainly two remarks:

• Put more comment on the APIs and classes definition
• Double check why tablegen gives us gpr32 instead of gpr32all

Thanks,
-Quentin

include/llvm/CodeGen/GlobalISel/Utils.h

36 ↗

(On Diff #100380)

Add comments on what this is doing.
Don't hesitate to move the comment from the other helper and cross reference to this one.

lib/CodeGen/GlobalISel/InstructionSelector.cpp

29 ↗

(On Diff #100380)

TargetRegisterClass &

lib/CodeGen/GlobalISel/Utils.cpp

33 ↗

(On Diff #100380)

Use TargetRegisterClass &

test/CodeGen/AArch64/GlobalISel/select-bitcast.mir

98 ↗

(On Diff #100380)

That change is strange. How does tablegen come up with this constraint?
I would have expected that we get the less constrained regclass (gpr32all in that case)

197 ↗

(On Diff #100380)

Ditto

test/TableGen/GlobalISelEmitter.td

476 ↗

(On Diff #100380)

Remark: Long term we should be able to use the right regbank directly.

utils/TableGen/GlobalISelEmitter.cpp

1043 ↗

(On Diff #100380)

Comment on the class' purpose

1056 ↗

(On Diff #100380)

Ditto

1543 ↗

(On Diff #100380)

Could you use explicit types here?

1560 ↗

(On Diff #100380)

Ditto

1568 ↗

(On Diff #100380)

Ditto

[qcolombet]

test/TableGen/GlobalISelEmitter.td

488 ↗

(On Diff #100380)

I didn't get that BTW

[dsanders]

Fix all the nits.

Only one issue remains. I'll comment on this one in phabricator

[dsanders]

The remaining issue I mentioned in the update is this one.

• Double check why tablegen gives us gpr32 instead of gpr32all

This happens because the SelectionDAG rules specify gpr32. For example:

// (bitconvert:i32 FPR32:f32:$Xn) => (COPY_TO_REGCLASS:i32 FPR32:f32:$Xn, GPR32:i32)

We can probably change the SelectionDAG rules. I'll give this a try

test/CodeGen/AArch64/GlobalISel/select-bitcast.mir

98 ↗

(On Diff #100380)

It's explicitly stated to be GPR32 in the SelectionDAG rule. Here's the pattern comment from the imported rules:
// (bitconvert:i32 FPR32:f32:$Xn) => (COPY_TO_REGCLASS:i32 FPR32:f32:$Xn, GPR32:i32)

197 ↗

(On Diff #100380)

This one has the same cause:
// (bitconvert:i64 FPR64:f64:$Xn) => (COPY_TO_REGCLASS:i64 FPR64:f64:$Xn, GPR64:i32)

test/TableGen/GlobalISelEmitter.td

476 ↗

(On Diff #100380)

I agree. I manually experimented with this a bit while fixing the compile-time regression. To do it properly I'll need to refactor some of the RegisterBankEmitter to be able to re-use this information from other tablegen emitters.

488 ↗

(On Diff #100380)

This is a bad copy/paste. It's referring to something that only matters to the xor rules.

[dsanders]

Update the SelectionDAG rules for FPR->GPR bitcasts to use GPR32all and GPR64all

[ab]

LGTM without the pattern changes.

include/llvm/CodeGen/GlobalISel/InstructionSelector.h

83 ↗

(On Diff #101485)

Comment? Or, alternatively, inline the one-line body in the emitted code?

lib/Target/AArch64/AArch64InstrInfo.td

5588–5592 ↗

(On Diff #101485)

I don't think this is correct: FMOV really does take GPR32/GPR64, not GPR32all/GPR64all, and that's what copyPhysReg looks for.

And the c++ selector actually also looks wrong. I'm guessing it happens to work because we never allocate SP, so anything we allocate out of GPR32all is in GPR32 as well anyway.

utils/TableGen/GlobalISelEmitter.cpp

1046 ↗

(On Diff #101485)

Unnecessary 'private', here and below.

[dsanders]

Made all requested changes in the commit rL305791

llvm/trunk/include/llvm/CodeGen/GlobalISel/InstructionSelector.h

 namespace llvm {
 class MachineInstr;
 class MachineInstrBuilder;
 class MachineFunction;
 class MachineOperand;
 class MachineRegisterInfo;
 class RegisterBankInfo;
 class TargetInstrInfo;
+class TargetRegisterClass;
 class TargetRegisterInfo;
 
 /// Container class for CodeGen predicate results.
 /// This is convenient because std::bitset does not have a constructor
 /// with an initializer list of set bits.
 ///
 /// Each InstructionSelector subclass should define a PredicateBitset class with:
 ///   const unsigned MAX_SUBTARGET_PREDICATES = 192;
public:
   ///
   virtual bool select(MachineInstr &I) const = 0;
 
 protected:
   typedef std::function<void(MachineInstrBuilder &)> ComplexRendererFn;
 
   InstructionSelector();
 
+  /// Constrain a register operand of an instruction \p I to a specified
+  /// register class. This could involve inserting COPYs before (for uses) or
+  /// after (for defs) and may replace the operand of \p I.
+  /// \returns whether operand regclass constraining succeeded.
+  bool constrainOperandRegToRegClass(MachineInstr &I, unsigned OpIdx,
+                                     const TargetRegisterClass &RC,
+                                     const TargetInstrInfo &TII,
+                                     const TargetRegisterInfo &TRI,
+                                     const RegisterBankInfo &RBI) const;
+
   /// Mutate the newly-selected instruction \p I to constrain its (possibly
   /// generic) virtual register operands to the instruction's register class.
   /// This could involve inserting COPYs before (for uses) or after (for defs).
   /// This requires the number of operands to match the instruction description.
   /// \returns whether operand regclass constraining succeeded.
   ///
   // FIXME: Not all instructions have the same number of operands. We should
   // probably expose a constrain helper per operand and let the target selector

llvm/trunk/include/llvm/CodeGen/GlobalISel/Utils.h

 class MachineOptimizationRemarkEmitter;
 class MachineOptimizationRemarkMissed;
 class MachineRegisterInfo;
 class MCInstrDesc;
 class RegisterBankInfo;
 class TargetInstrInfo;
 class TargetPassConfig;
 class TargetRegisterInfo;
+class TargetRegisterClass;
 class Twine;
 class ConstantFP;
 
+/// Try to constrain Reg to the specified register class. If this fails,
+/// create a new virtual register in the correct class and insert a COPY before
+/// \p InsertPt. The debug location of \p InsertPt is used for the new copy.
+///
+/// \return The virtual register constrained to the right register class.
+unsigned constrainRegToClass(MachineRegisterInfo &MRI,
+                             const TargetInstrInfo &TII,
+                             const RegisterBankInfo &RBI,
+                             MachineInstr &InsertPt, unsigned Reg,
+                             const TargetRegisterClass &RegClass);
+
 /// Try to constrain Reg so that it is usable by argument OpIdx of the
 /// provided MCInstrDesc \p II. If this fails, create a new virtual
 /// register in the correct class and insert a COPY before \p InsertPt.
-/// The debug location of \p InsertPt is used for the new copy.
+/// This is equivalent to constrainRegToClass() with RegClass obtained from the
+/// MCInstrDesc. The debug location of \p InsertPt is used for the new copy.
 ///
 /// \return The virtual register constrained to the right register class.
 unsigned constrainOperandRegClass(const MachineFunction &MF,
                                   const TargetRegisterInfo &TRI,
                                   MachineRegisterInfo &MRI,
                                   const TargetInstrInfo &TII,
                                   const RegisterBankInfo &RBI,
                                   MachineInstr &InsertPt, const MCInstrDesc &II,

llvm/trunk/lib/CodeGen/GlobalISel/InstructionSelector.cpp

 #include "llvm/Target/TargetRegisterInfo.h"
 
 #define DEBUG_TYPE "instructionselector"
 
 using namespace llvm;
 
 InstructionSelector::InstructionSelector() {}
 
+bool InstructionSelector::constrainOperandRegToRegClass(
+    MachineInstr &I, unsigned OpIdx, const TargetRegisterClass &RC,
+    const TargetInstrInfo &TII, const TargetRegisterInfo &TRI,
+    const RegisterBankInfo &RBI) const {
+  MachineBasicBlock &MBB = *I.getParent();
+  MachineFunction &MF = *MBB.getParent();
+  MachineRegisterInfo &MRI = MF.getRegInfo();
+
+  return llvm::constrainRegToClass(MRI, TII, RBI, I,
+                                   I.getOperand(OpIdx).getReg(), RC);
+}
+
 bool InstructionSelector::constrainSelectedInstRegOperands(
     MachineInstr &I, const TargetInstrInfo &TII, const TargetRegisterInfo &TRI,
     const RegisterBankInfo &RBI) const {
   MachineBasicBlock &MBB = *I.getParent();
   MachineFunction &MF = *MBB.getParent();
   MachineRegisterInfo &MRI = MF.getRegInfo();
 
   for (unsigned OpI = 0, OpE = I.getNumExplicitOperands(); OpI != OpE; ++OpI) {

llvm/trunk/lib/CodeGen/GlobalISel/Utils.cpp

 #include "llvm/IR/Constants.h"
 #include "llvm/Target/TargetInstrInfo.h"
 #include "llvm/Target/TargetRegisterInfo.h"
 
 #define DEBUG_TYPE "globalisel-utils"
 
 using namespace llvm;
 
+unsigned llvm::constrainRegToClass(MachineRegisterInfo &MRI,
+                                   const TargetInstrInfo &TII,
+                                   const RegisterBankInfo &RBI,
+                                   MachineInstr &InsertPt, unsigned Reg,
+                                   const TargetRegisterClass &RegClass) {
+  if (!RBI.constrainGenericRegister(Reg, RegClass, MRI)) {
+    unsigned NewReg = MRI.createVirtualRegister(&RegClass);
+    BuildMI(*InsertPt.getParent(), InsertPt, InsertPt.getDebugLoc(),
+            TII.get(TargetOpcode::COPY), NewReg)
+        .addReg(Reg);
+    return NewReg;
+  }
+
+  return Reg;
+}
+
+
 unsigned llvm::constrainOperandRegClass(
     const MachineFunction &MF, const TargetRegisterInfo &TRI,
     MachineRegisterInfo &MRI, const TargetInstrInfo &TII,
     const RegisterBankInfo &RBI, MachineInstr &InsertPt, const MCInstrDesc &II,
     unsigned Reg, unsigned OpIdx) {
   // Assume physical registers are properly constrained.
   assert(TargetRegisterInfo::isVirtualRegister(Reg) &&
          "PhysReg not implemented");
 
   const TargetRegisterClass *RegClass = TII.getRegClass(II, OpIdx, &TRI, MF);
-
-  if (!RBI.constrainGenericRegister(Reg, *RegClass, MRI)) {
-    unsigned NewReg = MRI.createVirtualRegister(RegClass);
-    BuildMI(*InsertPt.getParent(), InsertPt, InsertPt.getDebugLoc(),
-            TII.get(TargetOpcode::COPY), NewReg)
-        .addReg(Reg);
-    return NewReg;
-  }
-
-  return Reg;
+  return constrainRegToClass(MRI, TII, RBI, InsertPt, Reg, *RegClass);
 }
 
 bool llvm::isTriviallyDead(const MachineInstr &MI,
                            const MachineRegisterInfo &MRI) {
   // If we can move an instruction, we can remove it.  Otherwise, it has
   // a side-effect of some sort.
   bool SawStore = false;
   if (!MI.isSafeToMove(/*AA=*/nullptr, SawStore))

llvm/trunk/test/CodeGen/AArch64/GlobalISel/select-bitcast.mir

 ---
 # CHECK-LABEL: name: bitcast_s32_fpr_gpr
 name:            bitcast_s32_fpr_gpr
 legalized:       true
 regBankSelected: true
 
 # CHECK:      registers:
 # CHECK-NEXT:  - { id: 0, class: fpr32, preferred-register: '' }
-# CHECK-NEXT:  - { id: 1, class: gpr32all, preferred-register: '' }
+# CHECK-NEXT:  - { id: 1, class: gpr32, preferred-register: '' }
 registers:
   - { id: 0, class: fpr }
   - { id: 1, class: gpr }
 
 # CHECK:  body:
 # CHECK:    %0 = COPY %s0
 # CHECK:    %1 = COPY %0
 body:             |

 ---
 # CHECK-LABEL: name: bitcast_s64_fpr_gpr
 name:            bitcast_s64_fpr_gpr
 legalized:       true
 regBankSelected: true
 
 # CHECK:      registers:
 # CHECK-NEXT:  - { id: 0, class: fpr64, preferred-register: '' }
-# CHECK-NEXT:  - { id: 1, class: gpr64all, preferred-register: '' }
+# CHECK-NEXT:  - { id: 1, class: gpr64, preferred-register: '' }
 registers:
   - { id: 0, class: fpr }
   - { id: 1, class: gpr }
 
 # CHECK:  body:
 # CHECK:    %0 = COPY %d0
 # CHECK:    %1 = COPY %0
 body:             |
   bb.0:
     liveins: %d0
 
     %0(s64) = COPY %d0
     %1(s64) = G_BITCAST %0
     %x0 = COPY %1(s64)
 ...

llvm/trunk/test/TableGen/GlobalISelEmitter.td

 // RUN: llvm-tblgen -gen-global-isel -I %p/../../include %s | FileCheck %s
 
 include "llvm/Target/Target.td"
 
 //===- Define the necessary boilerplate for our test target. --------------===//
 
 def MyTargetISA : InstrInfo;
 def MyTarget : Target { let InstructionSet = MyTargetISA; }
 
 def R0 : Register<"r0"> { let Namespace = "MyTarget"; }
 def GPR32 : RegisterClass<"MyTarget", [i32], 32, (add R0)>;
 def GPR32Op : RegisterOperand<GPR32>;
+def F0 : Register<"f0"> { let Namespace = "MyTarget"; }
+def FPR32 : RegisterClass<"MyTarget", [f32], 32, (add F0)>;
 
 class I<dag OOps, dag IOps, list<dag> Pat>
   : Instruction {
   let Namespace = "MyTarget";
   let OutOperandList = OOps;
   let InOperandList = IOps;
   let Pattern = Pat;
 }

 // CHECK-NEXT:      return true;
 // CHECK-NEXT:    }
 // CHECK-NEXT:    return false;
 // CHECK-NEXT:  }()) { return true; }
 
 def ORN : I<(outs GPR32:$dst), (ins GPR32:$src1, GPR32:$src2), []>;
 def : Pat<(not GPR32:$Wm), (ORN R0, GPR32:$Wm)>;
 
+//===- Test a COPY_TO_REGCLASS --------------------------------------------===//
+//
+
+// CHECK-LABEL: if ([&]() {
+// CHECK-NEXT:    MachineInstr &MI0 = I;
+// CHECK-NEXT:    if (MI0.getNumOperands() < 2)
+// CHECK-NEXT:      return false;
+// CHECK-NEXT:    if ((MI0.getOpcode() == TargetOpcode::G_BITCAST) &&
+// CHECK-NEXT:        ((/* dst */ (MRI.getType(MI0.getOperand(0).getReg()) == (LLT::scalar(32))) &&
+// CHECK-NEXT:         ((&RBI.getRegBankFromRegClass(MyTarget::GPR32RegClass) == RBI.getRegBank(MI0.getOperand(0).getReg(), MRI, TRI))))) &&
+// CHECK-NEXT:        ((/* src1 */ (MRI.getType(MI0.getOperand(1).getReg()) == (LLT::scalar(32))) &&
+// CHECK-NEXT:         ((&RBI.getRegBankFromRegClass(MyTarget::FPR32RegClass) == RBI.getRegBank(MI0.getOperand(1).getReg(), MRI, TRI))))))
+// CHECK-NEXT:      // (bitconvert:i32 FPR32:f32:$src1) => (COPY_TO_REGCLASS:i32 FPR32:f32:$src1, GPR32:i32)
+// CHECK-NEXT:      I.setDesc(TII.get(TargetOpcode::COPY));
+// CHECK-NEXT:      MachineInstr &NewI = I;
+// CHECK-NEXT:      constrainOperandRegToRegClass(NewI, 0, MyTarget::GPR32RegClass, TII, TRI, RBI);
+// CHECK-NEXT:      return true;
+// CHECK-NEXT:    }
+// CHECK-NEXT:    return false;
+// CHECK-NEXT:  }()) { return true; }
+
+def : Pat<(i32 (bitconvert FPR32:$src1)),
+          (COPY_TO_REGCLASS FPR32:$src1, GPR32)>;
+
 //===- Test a simple pattern with just a leaf immediate. ------------------===//
 
 // CHECK-LABEL: if ([&]() {
 // CHECK-NEXT:    MachineInstr &MI0 = I;
 // CHECK-NEXT:    if (MI0.getNumOperands() < 2)
 // CHECK-NEXT:      return false;
 // CHECK-NEXT:    if ((MI0.getOpcode() == TargetOpcode::G_CONSTANT) &&
 // CHECK-NEXT:        ((/* dst */ (MRI.getType(MI0.getOperand(0).getReg()) == (LLT::scalar(32))) &&

llvm/trunk/utils/TableGen/GlobalISelEmitter.cpp

static Error isTrivialOperatorNode(const TreePatternNode *N) {
   }
 
   if (!N->isLeaf() && !N->hasAnyPredicate() && !N->getTransformFn())
     return Error::success();
 
   return failedImport(Explanation);
 }
 
+static Record *getInitValueAsRegClass(Init *V) {
+  if (DefInit *VDefInit = dyn_cast<DefInit>(V)) {
+    if (VDefInit->getDef()->isSubClassOf("RegisterOperand"))
+      return VDefInit->getDef()->getValueAsDef("RegClass");
+    if (VDefInit->getDef()->isSubClassOf("RegisterClass"))
+      return VDefInit->getDef();
+  }
+  return nullptr;
+}
+
 //===- Matchers -----------------------------------------------------------===//
 
 class OperandMatcher;
 class MatchAction;
 
 /// Generates code to check that a match rule matches.
 class RuleMatcher {
   /// A list of matchers that all need to succeed for the current rule to match.
void emitCxxActionStmts(raw_ostream &OS, RuleMatcher &Rule,
     OS << "// " << *P.getSrcPattern() << "  =>  " << *P.getDstPattern() << "\n";
   }
 };
 
 /// Generates code to build an instruction or mutate an existing instruction
 /// into the desired instruction when this is possible.
 class BuildMIAction : public MatchAction {
 private:
+  std::string Name;
   const CodeGenInstruction *I;
   const InstructionMatcher &Matched;
   std::vector<std::unique_ptr<OperandRenderer>> OperandRenderers;
 
   /// True if the instruction can be built solely by mutating the opcode.
   bool canMutate() const {
     if (OperandRenderers.size() != Matched.getNumOperands())
       return false;
 
     for (const auto &Renderer : enumerate(OperandRenderers)) {
       if (const auto *Copy = dyn_cast<CopyRenderer>(&*Renderer.value())) {
         const OperandMatcher &OM = Matched.getOperand(Copy->getSymbolicName());
         if (&Matched != &OM.getInstructionMatcher() ||
             OM.getOperandIndex() != Renderer.index())
           return false;
       } else
         return false;
     }
 
     return true;
   }
 
 public:
-  BuildMIAction(const CodeGenInstruction *I, const InstructionMatcher &Matched)
-      : I(I), Matched(Matched) {}
+  BuildMIAction(const StringRef Name, const CodeGenInstruction *I,
+                const InstructionMatcher &Matched)
+      : Name(Name), I(I), Matched(Matched) {}
 
   template <class Kind, class... Args>
   Kind &addRenderer(Args&&... args) {
     OperandRenderers.emplace_back(
         llvm::make_unique<Kind>(std::forward<Args>(args)...));
     return *static_cast<Kind *>(OperandRenderers.back().get());
   }
 
if (canMutate()) {
           auto Namespace = Use->getValue("Namespace")
                                ? Use->getValueAsString("Namespace")
                                : "";
           OS << "    MIB.addUse(" << Namespace << "::" << Use->getName()
              << ", RegState::Implicit);\n";
         }
       }
 
-      OS << "    MachineInstr &NewI = " << RecycleVarName << ";\n";
+      OS << "    MachineInstr &" << Name << " = " << RecycleVarName << ";\n";
       return;
     }
 
     // TODO: Simple permutation looks like it could be almost as common as
     //       mutation due to commutative operations.
 
     OS << "MachineInstrBuilder MIB = BuildMI(*I.getParent(), I, "
           "I.getDebugLoc(), TII.get("
        << I->Namespace << "::" << I->TheDef->getName() << "));\n";
     for (const auto &Renderer : OperandRenderers)
       Renderer->emitCxxRenderStmts(OS, Rule);
     OS << "    for (const auto *FromMI : ";
     Rule.emitCxxCapturedInsnList(OS);
     OS << ")\n";
     OS << "      for (const auto &MMO : FromMI->memoperands())\n";
     OS << "        MIB.addMemOperand(MMO);\n";
     OS << "    " << RecycleVarName << ".eraseFromParent();\n";
-    OS << "    MachineInstr &NewI = *MIB;\n";
+    OS << "    MachineInstr &" << Name << " = *MIB;\n";
+  }
+};
+
+/// Generates code to constrain the operands of an output instruction to the
+/// register classes specified by the definition of that instruction.
+class ConstrainOperandsToDefinitionAction : public MatchAction {
+  std::string Name;
+
+public:
+  ConstrainOperandsToDefinitionAction(const StringRef Name) : Name(Name) {}
+
+  void emitCxxActionStmts(raw_ostream &OS, RuleMatcher &Rule,
+                          StringRef RecycleVarName) const override {
+    OS << "      constrainSelectedInstRegOperands(" << Name << ", TII, TRI, RBI);\n";
+  }
+};
+
+/// Generates code to constrain the specified operand of an output instruction
+/// to the specified register class.
+class ConstrainOperandToRegClassAction : public MatchAction {
+  std::string Name;
+  unsigned OpIdx;
+  const CodeGenRegisterClass &RC;
+
+public:
+  ConstrainOperandToRegClassAction(const StringRef Name, unsigned OpIdx,
+                                   const CodeGenRegisterClass &RC)
+      : Name(Name), OpIdx(OpIdx), RC(RC) {}
+
+  void emitCxxActionStmts(raw_ostream &OS, RuleMatcher &Rule,
+                          StringRef RecycleVarName) const override {
+    OS << "      constrainOperandRegToRegClass(" << Name << ", " << OpIdx
+       << ", " << RC.getQualifiedName() << "RegClass, TII, TRI, RBI);\n";
   }
 };
 
 InstructionMatcher &RuleMatcher::addInstructionMatcher() {
   Matchers.emplace_back(new InstructionMatcher());
   return *Matchers.back();
 }
 
for (const auto &Name : Names) {
       //          by rejecting all loads.
     }
   }
 
   for (const auto &MA : Actions) {
     MA->emitCxxActionStmts(OS, *this, "I");
   }
 
-  OS << "      constrainSelectedInstRegOperands(NewI, TII, TRI, RBI);\n";
   OS << "      return true;\n";
   OS << "    }\n";
   OS << "    return false;\n";
   OS << "  }()) { return true; }\n\n";
 }
 
 bool RuleMatcher::isHigherPriorityThan(const RuleMatcher &B) const {
   // Rules involving more match roots have higher priority.
if (auto *ChildInt = dyn_cast<IntInit>(SrcChild->getLeafValue())) {
     return Error::success();
   }
 
   // Check for def's like register classes or ComplexPattern's.
   if (auto *ChildDefInit = dyn_cast<DefInit>(SrcChild->getLeafValue())) {
     auto *ChildRec = ChildDefInit->getDef();
 
     // Check for register classes.
-    if (ChildRec->isSubClassOf("RegisterClass")) {
-      OM.addPredicate<RegisterBankOperandMatcher>(
-          Target.getRegisterClass(ChildRec));
-      return Error::success();
-    }
-
-    if (ChildRec->isSubClassOf("RegisterOperand")) {
+    if (ChildRec->isSubClassOf("RegisterClass") ||
+        ChildRec->isSubClassOf("RegisterOperand")) {
       OM.addPredicate<RegisterBankOperandMatcher>(
-          Target.getRegisterClass(ChildRec->getValueAsDef("RegClass")));
+          Target.getRegisterClass(getInitValueAsRegClass(ChildDefInit)));
       return Error::success();
     }
 
     // Check for ComplexPattern's.
     if (ChildRec->isSubClassOf("ComplexPattern")) {
       const auto &ComplexPattern = ComplexPatternEquivs.find(ChildRec);
       if (ComplexPattern == ComplexPatternEquivs.end())
         return failedImport("SelectionDAG ComplexPattern (" +
Expected<BuildMIAction &> GlobalISelEmitter::createAndImportInstructionRenderer(
     const InstructionMatcher &InsnMatcher) const {
   Record *DstOp = Dst->getOperator();
   if (!DstOp->isSubClassOf("Instruction")) {
     if (DstOp->isSubClassOf("ValueType"))
       return failedImport(
           "Pattern operator isn't an instruction (it's a ValueType)");
     return failedImport("Pattern operator isn't an instruction");
   }
-  auto &DstI = Target.getInstruction(DstOp);
+  CodeGenInstruction *DstI = &Target.getInstruction(DstOp);
 
-  auto &DstMIBuilder = M.addAction<BuildMIAction>(&DstI, InsnMatcher);
+  unsigned DstINumUses = DstI->Operands.size() - DstI->Operands.NumDefs;
+  unsigned ExpectedDstINumUses = Dst->getNumChildren();
+
+  // COPY_TO_REGCLASS is just a copy with a ConstrainOperandToRegClassAction
+  // attached.
+  if (DstI->TheDef->getName() == "COPY_TO_REGCLASS") {
+    DstI = &Target.getInstruction(RK.getDef("COPY"));
+    DstINumUses--; // Ignore the class constraint.
+    ExpectedDstINumUses--;
+  }
+
+  auto &DstMIBuilder = M.addAction<BuildMIAction>("NewI", DstI, InsnMatcher);
 
   // Render the explicit defs.
-  for (unsigned I = 0; I < DstI.Operands.NumDefs; ++I) {
-    const auto &DstIOperand = DstI.Operands[I];
+  for (unsigned I = 0; I < DstI->Operands.NumDefs; ++I) {
+    const CGIOperandList::OperandInfo &DstIOperand = DstI->Operands[I];
     DstMIBuilder.addRenderer<CopyRenderer>(InsnMatcher, DstIOperand.Name);
   }
 
   // Render the explicit uses.
   unsigned Child = 0;
-  unsigned DstINumUses = DstI.Operands.size() - DstI.Operands.NumDefs;
   unsigned NumDefaultOps = 0;
   for (unsigned I = 0; I != DstINumUses; ++I) {
-    const auto &DstIOperand = DstI.Operands[DstI.Operands.NumDefs + I];
+    const CGIOperandList::OperandInfo &DstIOperand =
+        DstI->Operands[DstI->Operands.NumDefs + I];
 
     // If the operand has default values, introduce them now.
     // FIXME: Until we have a decent test case that dictates we should do
     // otherwise, we're going to assume that operands with default values cannot
     // be specified in the patterns. Therefore, adding them will not cause us to
     // end up with too many rendered operands.
     if (DstIOperand.Rec->isSubClassOf("OperandWithDefaultOps")) {
       DagInit *DefaultOps = DstIOperand.Rec->getValueAsDag("DefaultOps");
       if (auto Error = importDefaultOperandRenderers(DstMIBuilder, DefaultOps))
         return std::move(Error);
       ++NumDefaultOps;
       continue;
     }
 
     if (auto Error = importExplicitUseRenderer(
             DstMIBuilder, Dst->getChild(Child), InsnMatcher))
       return std::move(Error);
     ++Child;
   }
 
-  if (NumDefaultOps + Dst->getNumChildren() != DstINumUses)
+  if (NumDefaultOps + ExpectedDstINumUses != DstINumUses)
     return failedImport("Expected " + llvm::to_string(DstINumUses) +
                         " used operands but found " +
-                        llvm::to_string(Dst->getNumChildren()) +
+                        llvm::to_string(ExpectedDstINumUses) +
                         " explicit ones and " + llvm::to_string(NumDefaultOps) +
                         " default ones");
 
   return DstMIBuilder;
 }
 
 Error GlobalISelEmitter::importDefaultOperandRenderers(
     BuildMIAction &DstMIBuilder, DagInit *DefaultOps) const {
Expected<RuleMatcher> GlobalISelEmitter::runOnPattern(const PatternToMatch &P) {
   // The root of the match also has constraints on the register bank so that it
   // matches the result instruction.
   unsigned OpIdx = 0;
   for (const EEVT::TypeSet &Ty : Src->getExtTypes()) {
     (void)Ty;
 
     const auto &DstIOperand = DstI.Operands[OpIdx];
     Record *DstIOpRec = DstIOperand.Rec;
-    if (DstIOpRec->isSubClassOf("RegisterOperand"))
+    if (DstI.TheDef->getName() == "COPY_TO_REGCLASS") {
+      DstIOpRec = getInitValueAsRegClass(Dst->getChild(1)->getLeafValue());
+
+      if (DstIOpRec == nullptr)
+        return failedImport(
+            "COPY_TO_REGCLASS operand #1 isn't a register class");
+    } else if (DstIOpRec->isSubClassOf("RegisterOperand"))
       DstIOpRec = DstIOpRec->getValueAsDef("RegClass");
-    if (!DstIOpRec->isSubClassOf("RegisterClass"))
-      return failedImport("Dst MI def isn't a register class");
+    else if (!DstIOpRec->isSubClassOf("RegisterClass"))
+      return failedImport("Dst MI def isn't a register class" + to_string(*Dst));
 
     OperandMatcher &OM = InsnMatcher.getOperand(OpIdx);
     OM.setSymbolicName(DstIOperand.Name);
     OM.addPredicate<RegisterBankOperandMatcher>(
         Target.getRegisterClass(DstIOpRec));
     ++OpIdx;
   }
 
   auto DstMIBuilderOrError =
       createAndImportInstructionRenderer(M, Dst, InsnMatcher);
   if (auto Error = DstMIBuilderOrError.takeError())
     return std::move(Error);
   BuildMIAction &DstMIBuilder = DstMIBuilderOrError.get();
 
   // Render the implicit defs.
   // These are only added to the root of the result.
   if (auto Error = importImplicitDefRenderers(DstMIBuilder, P.getDstRegs()))
     return std::move(Error);
 
+  // Constrain the registers to classes. This is normally derived from the
+  // emitted instruction but a few instructions require special handling.
+  if (DstI.TheDef->getName() == "COPY_TO_REGCLASS") {
+    // COPY_TO_REGCLASS does not provide operand constraints itself but the
+    // result is constrained to the class given by the second child.
+    Record *DstIOpRec =
+        getInitValueAsRegClass(Dst->getChild(1)->getLeafValue());
+
+    if (DstIOpRec == nullptr)
+      return failedImport("COPY_TO_REGCLASS operand #1 isn't a register class");
+
+    M.addAction<ConstrainOperandToRegClassAction>(
+        "NewI", 0, Target.getRegisterClass(DstIOpRec));
+  } else
+    M.addAction<ConstrainOperandsToDefinitionAction>("NewI");
+
   // We're done with this pattern!  It's eligible for GISel emission; return it.
   ++NumPatternImported;
   return std::move(M);
 }
 
 void GlobalISelEmitter::run(raw_ostream &OS) {
   // Track the GINodeEquiv definitions.
   gatherNodeEquivs();