[globalisel][tablegen] Revise API for ComplexPattern operands to improve flexibility.

[dsanders]

Summary

Some targets need to be able to do more complex rendering than just adding an
operand or two to an instruction. For example, it may need to insert an
instruction to extract a subreg first, or it may need to perform an operation
on the operand.

In SelectionDAG, targets would create SDNode's to achieve the desired effect
during the complex pattern predicate. This worked because SelectionDAG had a
form of garbage collection that would take care of SDNode's that were created
but not used due to a later predicate rejecting a match. This doesn't translate
well to GlobalISel and the churn was wasteful.

The API changes in this patch enable GlobalISel to accomplish the same thing
without the waste. The API is now:
InstructionSelector::OptionalComplexRendererFn selectArithImmed(MachineOperand &Root) const;
where Root is the root of the match. The return value can be omitted to
indicate that the predicate failed to match, or a function with the signature
ComplexRendererFn can be returned. For example:
return OptionalComplexRendererFn(

	       [=](MachineInstrBuilder &MIB) { MIB.addImm(Immed).addImm(ShVal); });

adds two immediate operands to the rendered instruction. Immed and ShVal are
captured from the predicate function.

As an added bonus, this also reduces the amount of information we need to
provide to GIComplexOperandMatcher.

Depends on D31418

[ab]

I'm slightly worried that this makes the selector functions a bit harder to read, but this seems like a nice simplification otherwise.

One question: does this prevent us from ever selecting operands in-place? The previous implementation seems amenable to that (by only assigning the complex-pattern MO instead of adding them all to a new MI). But now we'd be forced to always append operands, right?

This also means that we can remove MO_Placeholder, yes?

include/llvm/CodeGen/GlobalISel/InstructionSelector.h

82

The Optional seems a bit redundant: std::function is already optional. Can we return a {} default?

test/TableGen/GlobalISelEmitter-GIRule.td

70 ↗

(On Diff #94367)

What about:

((Renderer0 = selectComplexPattern(MI0.getOperand(2))))

I found the ',' more surprising than the '='.

[dsanders]

In D31761#722903, @ab wrote:

I'm slightly worried that this makes the selector functions a bit harder to read, but this seems like a nice simplification otherwise.

One question: does this prevent us from ever selecting operands in-place? The previous implementation seems amenable to that (by only assigning the complex-pattern MO instead of adding them all to a new MI). But now we'd be forced to always append operands, right?

I think it's still possible to handle the mutation cases we could potentially support before this patch. Supporting one matched operand being rendered as one operand can be supported with a little additional information to specify which operand to replace* and a tablegen field to indicate that the renderer supports mutation. However, it's quite common for one matched operand to be rendered as multiple operands (e.g. for addressing modes like addr+offset, or for ARM's immediates). I'm not sure this case is efficiently supportable before or after this patch since we'd need to make room for the additional operands and the underlying storage is an array.

*This gets trickier when multiple instructions are emitted by a rule though. We'd need to decide which instruction to mutate.

This also means that we can remove MO_Placeholder, yes?

That's right. I'll update the patch to do that.

---

I've been using this mechanism more thoroughly on an out-of-tree target over the last few days and there's an issue that I need to find a solution to. At the moment, we can create new instructions in the renderer but there's no way to have the instruction selector process them. For example, we can emit a target instruction to load an immediate but we can't create a legal G_CONSTANT and ask the instruction selector to figure out some code for it. I'm currently thinking the instruction selector should have a work list we can add new instructions to and only when the work list is empty would it move on to the next instruction in the BB.

It would also be nice to be able to have the legalizer run on newly-created instructions so targets don't end up implementing parts of legalization in the instruction selector like we've seen in SelectionDAG.

[dsanders]

Forgot to reply to the inline comments

include/llvm/CodeGen/GlobalISel/InstructionSelector.h

82

I think that will work. I'll give it a try.

test/TableGen/GlobalISelEmitter-GIRule.td

70 ↗

(On Diff #94367)

Assuming the std::function change above works out, this change will be easy to make.

[dsanders]

Remove placeholder operands since they're no longer needed.
Drop the Optional<> around ComplexRendererFn in favour of nullptr.
Remove the dependency on D31329.

[ab]

Nice, LGTM!

include/llvm/CodeGen/GlobalISel/InstructionSelector.h

 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_CODEGEN_GLOBALISEL_INSTRUCTIONSELECTOR_H
 #define LLVM_CODEGEN_GLOBALISEL_INSTRUCTIONSELECTOR_H
 
 #include "llvm/ADT/Optional.h"
 #include <cstdint>
 #include <bitset>
+#include <functional>
 
 namespace llvm {
 class MachineInstr;
+class MachineInstrBuilder;
 class MachineFunction;
 class MachineOperand;
 class MachineRegisterInfo;
 class RegisterBankInfo;
 class TargetInstrInfo;
 class TargetRegisterInfo;
 
 /// Container class for CodeGen predicate results.
public:
   /// \post
   ///   if returns true:
   ///     for I in all mutated/inserted instructions:
   ///       !isPreISelGenericOpcode(I.getOpcode())
   ///
   virtual bool select(MachineInstr &I) const = 0;
 
 protected:
+  typedef std::function<void(MachineInstrBuilder &)> ComplexRendererFn;
+
   InstructionSelector();
 
   /// Mutate the newly-selected instruction \p I to constrain its (possibly
   /// generic) virtual register operands to the instruction's register class.
   /// This could involve inserting COPYs before (for uses) or after (for defs).
   /// This requires the number of operands to match the instruction description.
   /// \returns whether operand regclass constraining succeeded.
   ///

include/llvm/CodeGen/MachineOperand.h

enum MachineOperandType : unsigned char {
     MO_BlockAddress,      ///< Address of a basic block
     MO_RegisterMask,      ///< Mask of preserved registers.
     MO_RegisterLiveOut,   ///< Mask of live-out registers.
     MO_Metadata,          ///< Metadata reference (for debug info)
     MO_MCSymbol,          ///< MCSymbol reference (for debug/eh info)
     MO_CFIIndex,          ///< MCCFIInstruction index.
     MO_IntrinsicID,       ///< Intrinsic ID for ISel
     MO_Predicate,         ///< Generic predicate for ISel
-    MO_Placeholder,       ///< Placeholder for GlobalISel ComplexPattern result.
   };
 
 private:
   /// OpKind - Specify what kind of operand this is.  This discriminates the
   /// union.
   MachineOperandType OpKind : 8;
 
   /// Subregister number for MO_Register.  A value of 0 indicates the
public:
   }
 
   static MachineOperand CreatePredicate(unsigned Pred) {
     MachineOperand Op(MachineOperand::MO_Predicate);
     Op.Contents.Pred = Pred;
     return Op;
   }
 
-  static MachineOperand CreatePlaceholder() {
-    MachineOperand Op(MachineOperand::MO_Placeholder);
-    return Op;
-  }
-
   friend class MachineInstr;
   friend class MachineRegisterInfo;
 private:
   void removeRegFromUses();
 
   //===--------------------------------------------------------------------===//
   // Methods for handling register use/def lists.
   //===--------------------------------------------------------------------===//

include/llvm/Target/GlobalISel/Target.td

 def s64 : LLT;
 
 // Defines a matcher for complex operands. This is analogous to ComplexPattern
 // from SelectionDAG.
 //
 // Definitions that inherit from this may also inherit from
 // GIComplexPatternEquiv to enable the import of SelectionDAG patterns involving
 // those ComplexPatterns.
-class GIComplexOperandMatcher<LLT type, dag operands, string matcherfn> {
+class GIComplexOperandMatcher<LLT type, string matcherfn> {
   // The expected type of the root of the match.
   //
   // TODO: We should probably support, any-type, any-scalar, and multiple types
   //       in the future.
   LLT Type = type;
 
-  // The operands that result from a successful match
-  // Should be of the form '(ops ty1, ty2, ...)' where ty1/ty2 are definitions
-  // that inherit from Operand.
-  //
-  // FIXME: Which definition is used for ty1/ty2 doesn't actually matter at the
-  //        moment. Only the number of operands is used.
-  dag Operands = operands;
-
   // The function that determines whether the operand matches. It should be of
   // the form:
   //   bool select(const MatchOperand &Root, MatchOperand &Result1)
   // and should have the same number of ResultX arguments as the number of
   // result operands. It must return true on successful match and false
   // otherwise. If it returns true, then all the ResultX arguments must be
   // overwritten.
   string MatcherFn = matcherfn;
 }

lib/CodeGen/MIRPrinter.cpp

case MachineOperand::MO_IntrinsicID: {
     break;
   }
   case MachineOperand::MO_Predicate: {
     auto Pred = static_cast<CmpInst::Predicate>(Op.getPredicate());
     OS << (CmpInst::isIntPredicate(Pred) ? "int" : "float") << "pred("
        << CmpInst::getPredicateName(Pred) << ')';
     break;
   }
-  case MachineOperand::MO_Placeholder:
-    OS << "<placeholder>";
-    break;
   }
 }
 
 void MIPrinter::print(const MachineMemOperand &Op) {
   OS << '(';
   // TODO: Print operand's target specific flags.
   if (Op.isVolatile())
     OS << "volatile ";

lib/CodeGen/MachineInstr.cpp

bool MachineOperand::isIdenticalTo(const MachineOperand &Other) const {
   case MachineOperand::MO_CFIIndex:
     return getCFIIndex() == Other.getCFIIndex();
   case MachineOperand::MO_Metadata:
     return getMetadata() == Other.getMetadata();
   case MachineOperand::MO_IntrinsicID:
     return getIntrinsicID() == Other.getIntrinsicID();
   case MachineOperand::MO_Predicate:
     return getPredicate() == Other.getPredicate();
-  case MachineOperand::MO_Placeholder:
-    return true;
   }
   llvm_unreachable("Invalid machine operand type");
 }
 
 // Note: this must stay exactly in sync with isIdenticalTo above.
 hash_code llvm::hash_value(const MachineOperand &MO) {
   switch (MO.getType()) {
   case MachineOperand::MO_Register:
hash_code llvm::hash_value(const MachineOperand &MO) {
   case MachineOperand::MO_MCSymbol:
     return hash_combine(MO.getType(), MO.getTargetFlags(), MO.getMCSymbol());
   case MachineOperand::MO_CFIIndex:
     return hash_combine(MO.getType(), MO.getTargetFlags(), MO.getCFIIndex());
   case MachineOperand::MO_IntrinsicID:
     return hash_combine(MO.getType(), MO.getTargetFlags(), MO.getIntrinsicID());
   case MachineOperand::MO_Predicate:
     return hash_combine(MO.getType(), MO.getTargetFlags(), MO.getPredicate());
-  case MachineOperand::MO_Placeholder:
-    return hash_combine();
   }
   llvm_unreachable("Invalid machine operand type");
 }
 
 void MachineOperand::print(raw_ostream &OS, const TargetRegisterInfo *TRI,
                            const TargetIntrinsicInfo *IntrinsicInfo) const {
   ModuleSlotTracker DummyMST(nullptr);
   print(OS, DummyMST, TRI, IntrinsicInfo);
case MachineOperand::MO_IntrinsicID: {
     break;
   }
   case MachineOperand::MO_Predicate: {
     auto Pred = static_cast<CmpInst::Predicate>(getPredicate());
     OS << '<' << (CmpInst::isIntPredicate(Pred) ? "intpred" : "floatpred")
        << CmpInst::getPredicateName(Pred) << '>';
     break;
   }
-  case MachineOperand::MO_Placeholder:
-    OS << "<placeholder>";
-    break;
   }
   if (unsigned TF = getTargetFlags())
     OS << "[TF=" << TF << ']';
 }
 
 #if !defined(NDEBUG) || defined(LLVM_ENABLE_DUMP)
 LLVM_DUMP_METHOD void MachineOperand::dump() const {
   dbgs() << *this << '\n';

lib/Target/AArch64/AArch64InstrFormats.td

 }
 
 def addsub_shifted_imm32 : addsub_shifted_imm<i32>;
 def addsub_shifted_imm64 : addsub_shifted_imm<i64>;
 def addsub_shifted_imm32_neg : addsub_shifted_imm_neg<i32>;
 def addsub_shifted_imm64_neg : addsub_shifted_imm_neg<i64>;
 
 def gi_addsub_shifted_imm32 :
-    GIComplexOperandMatcher<s32, (ops i32imm, i32imm), "selectArithImmed">,
+    GIComplexOperandMatcher<s32, "selectArithImmed">,
     GIComplexPatternEquiv<addsub_shifted_imm32>;
 
 def gi_addsub_shifted_imm64 :
-    GIComplexOperandMatcher<s64, (ops i32imm, i32imm), "selectArithImmed">,
+    GIComplexOperandMatcher<s64, "selectArithImmed">,
     GIComplexPatternEquiv<addsub_shifted_imm64>;
 
 class neg_addsub_shifted_imm<ValueType Ty>
     : Operand<Ty>, ComplexPattern<Ty, 2, "SelectNegArithImmed", [imm]> {
   let PrintMethod = "printAddSubImm";
   let EncoderMethod = "getAddSubImmOpValue";
   let ParserMatchClass = AddSubImmOperand;
   let MIOperandInfo = (ops i32imm, i32imm);

lib/Target/AArch64/AArch64InstructionSelector.cpp

private:
   bool selectVaStartAAPCS(MachineInstr &I, MachineFunction &MF,
                           MachineRegisterInfo &MRI) const;
   bool selectVaStartDarwin(MachineInstr &I, MachineFunction &MF,
                            MachineRegisterInfo &MRI) const;
 
   bool selectCompareBranch(MachineInstr &I, MachineFunction &MF,
                            MachineRegisterInfo &MRI) const;
 
-  bool selectArithImmed(MachineOperand &Root, MachineOperand &Result1,
-                        MachineOperand &Result2) const;
+  ComplexRendererFn selectArithImmed(MachineOperand &Root) const;
 
   const AArch64TargetMachine &TM;
   const AArch64Subtarget &STI;
   const AArch64InstrInfo &TII;
   const AArch64RegisterInfo &TRI;
   const AArch64RegisterBankInfo &RBI;
   bool ForCodeSize;
 
#endif
   }
 
   return false;
 }
 
 /// SelectArithImmed - Select an immediate value that can be represented as
 /// a 12-bit value shifted left by either 0 or 12.  If so, return true with
 /// Val set to the 12-bit value and Shift set to the shifter operand.
-bool AArch64InstructionSelector::selectArithImmed(
-    MachineOperand &Root, MachineOperand &Result1,
-    MachineOperand &Result2) const {
+InstructionSelector::ComplexRendererFn
+AArch64InstructionSelector::selectArithImmed(MachineOperand &Root) const {
   MachineInstr &MI = *Root.getParent();
   MachineBasicBlock &MBB = *MI.getParent();
   MachineFunction &MF = *MBB.getParent();
   MachineRegisterInfo &MRI = MF.getRegInfo();
 
   // This function is called from the addsub_shifted_imm ComplexPattern,
   // which lists [imm] as the list of opcode it's interested in, however
   // we still need to check whether the operand is actually an immediate
   // here because the ComplexPattern opcode list is only used in
   // root-level opcode matching.
   uint64_t Immed;
   if (Root.isImm())
     Immed = Root.getImm();
   else if (Root.isCImm())
     Immed = Root.getCImm()->getZExtValue();
   else if (Root.isReg()) {
     MachineInstr *Def = MRI.getVRegDef(Root.getReg());
     if (Def->getOpcode() != TargetOpcode::G_CONSTANT)
-      return false;
+      return nullptr;
     MachineOperand &Op1 = Def->getOperand(1);
     if (!Op1.isCImm() || Op1.getCImm()->getBitWidth() > 64)
-      return false;
+      return nullptr;
     Immed = Op1.getCImm()->getZExtValue();
   } else
-    return false;
+    return nullptr;
 
   unsigned ShiftAmt;
 
   if (Immed >> 12 == 0) {
     ShiftAmt = 0;
   } else if ((Immed & 0xfff) == 0 && Immed >> 24 == 0) {
     ShiftAmt = 12;
     Immed = Immed >> 12;
   } else
-    return false;
+    return nullptr;
 
   unsigned ShVal = AArch64_AM::getShifterImm(AArch64_AM::LSL, ShiftAmt);
-  Result1.ChangeToImmediate(Immed);
-  Result1.clearParent();
-  Result2.ChangeToImmediate(ShVal);
-  Result2.clearParent();
-  return true;
+  return [=](MachineInstrBuilder &MIB) { MIB.addImm(Immed).addImm(ShVal); };
 }
 
 namespace llvm {
 InstructionSelector *
 createAArch64InstructionSelector(const AArch64TargetMachine &TM,
                                  AArch64Subtarget &Subtarget,
                                  AArch64RegisterBankInfo &RBI) {
   return new AArch64InstructionSelector(TM, Subtarget, RBI);
 }
 }

lib/Target/ARM/ARMExpandPseudoInsts.cpp

case MachineOperand::MO_RegisterLiveOut:
     return false;
   case MachineOperand::MO_Metadata:
   case MachineOperand::MO_MCSymbol:
     return true;
   case MachineOperand::MO_CFIIndex:
     return false;
   case MachineOperand::MO_IntrinsicID:
   case MachineOperand::MO_Predicate:
-  case MachineOperand::MO_Placeholder:
     llvm_unreachable("should not exist post-isel");
   }
   llvm_unreachable("unhandled machine operand type");
 }
 
 void ARMExpandPseudo::ExpandMOV32BitImm(MachineBasicBlock &MBB,
                                         MachineBasicBlock::iterator &MBBI) {
   MachineInstr &MI = *MBBI;

test/TableGen/GlobalISelEmitter.td

class I<dag OOps, dag IOps, list<dag> Pat>
   let InOperandList = IOps;
   let Pattern = Pat;
 }
 
 def complex : Operand<i32>, ComplexPattern<i32, 2, "SelectComplexPattern", []> {
   let MIOperandInfo = (ops i32imm, i32imm);
 }
 def gi_complex :
-    GIComplexOperandMatcher<s32, (ops i32imm, i32imm), "selectComplexPattern">,
+    GIComplexOperandMatcher<s32, "selectComplexPattern">,
     GIComplexPatternEquiv<complex>;
 
 def m1 : OperandWithDefaultOps <i32, (ops (i32 -1))>;
 def Z : OperandWithDefaultOps <i32, (ops R0)>;
 def m1Z : OperandWithDefaultOps <i32, (ops (i32 -1), R0)>;
 
 def HasA : Predicate<"Subtarget->hasA()">;
 def HasB : Predicate<"Subtarget->hasB()">;

 // CHECK-NEXT:    if (MI0.getNumOperands() < 4)
 // CHECK-NEXT:      return false;
 // CHECK-NEXT:    if ((MI0.getOpcode() == TargetOpcode::G_SELECT) &&
 // CHECK-NEXT:        ((/* dst */ (MRI.getType(MI0.getOperand(0).getReg()) == (LLT::scalar(32))) &&
 // CHECK-NEXT:         ((&RBI.getRegBankFromRegClass(MyTarget::GPR32RegClass) == RBI.getRegBank(MI0.getOperand(0).getReg(), MRI, TRI))))) &&
 // CHECK-NEXT:        ((/* src1 */ (MRI.getType(MI0.getOperand(1).getReg()) == (LLT::scalar(32))) &&
 // CHECK-NEXT:         ((&RBI.getRegBankFromRegClass(MyTarget::GPR32RegClass) == RBI.getRegBank(MI0.getOperand(1).getReg(), MRI, TRI))))) &&
 // CHECK-NEXT:        ((/* src2 */ (MRI.getType(MI0.getOperand(2).getReg()) == (LLT::scalar(32))) &&
-// CHECK-NEXT:        (selectComplexPattern(MI0.getOperand(2), TempOp0, TempOp1)))) &&
+// CHECK-NEXT:        ((Renderer0 = selectComplexPattern(MI0.getOperand(2)))))) &&
 // CHECK-NEXT:        ((/* src3 */ (MRI.getType(MI0.getOperand(3).getReg()) == (LLT::scalar(32))) &&
-// CHECK-NEXT:        (selectComplexPattern(MI0.getOperand(3), TempOp2, TempOp3))))) {
+// CHECK-NEXT:        ((Renderer1 = selectComplexPattern(MI0.getOperand(3))))))) {
 // CHECK-NEXT:          // (select:i32 GPR32:i32:$src1, complex:i32:$src2, complex:i32:$src3) => (INSN2:i32 GPR32:i32:$src1, complex:i32:$src3, complex:i32:$src2)
 // CHECK-NEXT:          MachineInstrBuilder MIB = BuildMI(*I.getParent(), I, I.getDebugLoc(), TII.get(MyTarget::INSN2));
 // CHECK-NEXT:          MIB.add(MI0.getOperand(0)/*dst*/);
 // CHECK-NEXT:          MIB.add(MI0.getOperand(1)/*src1*/);
-// CHECK-NEXT:          MIB.add(TempOp2);
-// CHECK-NEXT:          MIB.add(TempOp3);
-// CHECK-NEXT:          MIB.add(TempOp0);
-// CHECK-NEXT:          MIB.add(TempOp1);
+// CHECK-NEXT:          Renderer1(MIB);
+// CHECK-NEXT:          Renderer0(MIB);
 // CHECK-NEXT:          for (const auto *FromMI : {&MI0, })
 // CHECK-NEXT:            for (const auto &MMO : FromMI->memoperands())
 // CHECK-NEXT:              MIB.addMemOperand(MMO);
 // CHECK-NEXT:          I.eraseFromParent();
 // CHECK-NEXT:          MachineInstr &NewI = *MIB;
 // CHECK-NEXT:          constrainSelectedInstRegOperands(NewI, TII, TRI, RBI);
 // CHECK-NEXT:          return true;
 // CHECK-NEXT:        }

 // CHECK-NEXT:    if (MI0.getNumOperands() < 3)
 // CHECK-NEXT:      return false;
 // CHECK-NEXT:    if ((MI0.getOpcode() == TargetOpcode::G_SUB) &&
 // CHECK-NEXT:        ((/* dst */ (MRI.getType(MI0.getOperand(0).getReg()) == (LLT::scalar(32))) &&
 // CHECK-NEXT:         ((&RBI.getRegBankFromRegClass(MyTarget::GPR32RegClass) == RBI.getRegBank(MI0.getOperand(0).getReg(), MRI, TRI))))) &&
 // CHECK-NEXT:        ((/* src1 */ (MRI.getType(MI0.getOperand(1).getReg()) == (LLT::scalar(32))) &&
 // CHECK-NEXT:         ((&RBI.getRegBankFromRegClass(MyTarget::GPR32RegClass) == RBI.getRegBank(MI0.getOperand(1).getReg(), MRI, TRI))))) &&
 // CHECK-NEXT:        ((/* src2 */ (MRI.getType(MI0.getOperand(2).getReg()) == (LLT::scalar(32))) &&
-// CHECK-NEXT:        (selectComplexPattern(MI0.getOperand(2), TempOp0, TempOp1))))) {
+// CHECK-NEXT:        ((Renderer0 = selectComplexPattern(MI0.getOperand(2))))))) {
 // CHECK-NEXT:          // (sub:i32 GPR32:i32:$src1, complex:i32:$src2) => (INSN1:i32 GPR32:i32:$src1, complex:i32:$src2)
 // CHECK-NEXT:          MachineInstrBuilder MIB = BuildMI(*I.getParent(), I, I.getDebugLoc(), TII.get(MyTarget::INSN1));
 // CHECK-NEXT:          MIB.add(MI0.getOperand(0)/*dst*/);
 // CHECK-NEXT:          MIB.add(MI0.getOperand(1)/*src1*/);
-// CHECK-NEXT:          MIB.add(TempOp0);
-// CHECK-NEXT:          MIB.add(TempOp1);
+// CHECK-NEXT:          Renderer0(MIB);
 // CHECK-NEXT:          for (const auto *FromMI : {&MI0, })
 // CHECK-NEXT:            for (const auto &MMO : FromMI->memoperands())
 // CHECK-NEXT:              MIB.addMemOperand(MMO);
 // CHECK-NEXT:          I.eraseFromParent();
 // CHECK-NEXT:          MachineInstr &NewI = *MIB;
 // CHECK-NEXT:          constrainSelectedInstRegOperands(NewI, TII, TRI, RBI);
 // CHECK-NEXT:          return true;
 // CHECK-NEXT:        }

utils/TableGen/GlobalISelEmitter.cpp

void emitCxxConstructorCall(raw_ostream &OS) const {
     }
     llvm_unreachable("Unhandled LLT");
   }
 
   const LLT &get() const { return Ty; }
 };
 
 class InstructionMatcher;
-class OperandPlaceholder {
-private:
-  enum PlaceholderKind {
-    OP_MatchReference,
-    OP_Temporary,
-  } Kind;
-
-  struct MatchReferenceData {
-    InstructionMatcher *InsnMatcher;
-    StringRef InsnVarName;
-    StringRef SymbolicName;
-  };
-
-  struct TemporaryData {
-    unsigned OpIdx;
-  };
-
-  union {
-    struct MatchReferenceData MatchReference;
-    struct TemporaryData Temporary;
-  };
-
-  OperandPlaceholder(PlaceholderKind Kind) : Kind(Kind) {}
-
-public:
-  ~OperandPlaceholder() {}
-
-  static OperandPlaceholder
-  CreateMatchReference(InstructionMatcher *InsnMatcher,
-                       StringRef InsnVarName, StringRef SymbolicName) {
-    OperandPlaceholder Result(OP_MatchReference);
-    Result.MatchReference.InsnMatcher = InsnMatcher;
-    Result.MatchReference.InsnVarName = InsnVarName;
-    Result.MatchReference.SymbolicName = SymbolicName;
-    return Result;
-  }
-
-  static OperandPlaceholder CreateTemporary(unsigned OpIdx) {
-    OperandPlaceholder Result(OP_Temporary);
-    Result.Temporary.OpIdx = OpIdx;
-    return Result;
-  }
-
-  void emitCxxValueExpr(raw_ostream &OS) const;
-};
-
 /// Convert an MVT to an equivalent LLT if possible, or the invalid LLT() for
 /// MVTs that don't map cleanly to an LLT (e.g., iPTR, *any, ...).
 static Optional<LLTCodeGen> MVTToLLT(MVT::SimpleValueType SVT) {
   MVT VT(SVT);
   if (VT.isVector() && VT.getVectorNumElements() != 1)
     return LLTCodeGen(LLT::vector(VT.getVectorNumElements(), VT.getScalarSizeInBits()));
   if (VT.isInteger() || VT.isFloatingPoint())
     return LLTCodeGen(LLT::scalar(VT.getSizeInBits()));
public:
 
   /// Compare the priority of this object and B.
   ///
   /// Returns true if this object is more important than B.
   bool isHigherPriorityThan(const RuleMatcher &B) const;
 
   /// Report the maximum number of temporary operands needed by the rule
   /// matcher.
-  unsigned countTemporaryOperands() const;
+  unsigned countRendererFns() const;
+
+  // FIXME: Remove this as soon as possible
+  InstructionMatcher &insnmatcher_front() const { return *Matchers.front(); }
 };
 
 template <class PredicateTy> class PredicateListMatcher {
 private:
   typedef std::vector<std::unique_ptr<PredicateTy>> PredicateVec;
   PredicateVec Predicates;
 
 public:
public:
   ///
   /// Returns true if this object is more important than B.
   virtual bool isHigherPriorityThan(const OperandPredicateMatcher &B) const {
     return Kind < B.Kind;
   };
 
   /// Report the maximum number of temporary operands needed by the predicate
   /// matcher.
-  virtual unsigned countTemporaryOperands() const { return 0; }
+  virtual unsigned countRendererFns() const { return 0; }
 };
 
 /// Generates code to check that an operand is a particular LLT.
 class LLTOperandMatcher : public OperandPredicateMatcher {
 protected:
   LLTCodeGen Ty;
 
 public:

 };
 
 /// Generates code to check that an operand is a particular target constant.
 class ComplexPatternOperandMatcher : public OperandPredicateMatcher {
 protected:
   const OperandMatcher &Operand;
   const Record &TheDef;
 
-  unsigned getNumOperands() const {
-    return TheDef.getValueAsDag("Operands")->getNumArgs();
-  }
-
   unsigned getAllocatedTemporariesBaseID() const;
 
 public:
   ComplexPatternOperandMatcher(const OperandMatcher &Operand,
                                const Record &TheDef)
       : OperandPredicateMatcher(OPM_ComplexPattern), Operand(Operand),
         TheDef(TheDef) {}
 
   static bool classof(const OperandPredicateMatcher *P) {
     return P->getKind() == OPM_ComplexPattern;
   }
 
   void emitCxxPredicateExpr(raw_ostream &OS, RuleMatcher &Rule,
                             StringRef OperandExpr) const override {
-    OS << TheDef.getValueAsString("MatcherFn") << "(" << OperandExpr;
-    for (unsigned I = 0; I < getNumOperands(); ++I) {
-      OS << ", ";
-      OperandPlaceholder::CreateTemporary(getAllocatedTemporariesBaseID() + I)
-          .emitCxxValueExpr(OS);
-    }
-    OS << ")";
+    unsigned ID = getAllocatedTemporariesBaseID();
+    OS << "(Renderer" << ID << " = " << TheDef.getValueAsString("MatcherFn")
+       << "(" << OperandExpr << "))";
   }
 
-  unsigned countTemporaryOperands() const override {
-    return getNumOperands();
+  unsigned countRendererFns() const override {
+    return 1;
   }
 };
 
 /// Generates code to check that an operand is in a particular register bank.
 class RegisterBankOperandMatcher : public OperandPredicateMatcher {
 protected:
   const CodeGenRegisterClass &RC;
 

 class OperandMatcher : public PredicateListMatcher<OperandPredicateMatcher> {
 protected:
   InstructionMatcher &Insn;
   unsigned OpIdx;
   std::string SymbolicName;
 
   /// The index of the first temporary variable allocated to this operand. The
   /// number of allocated temporaries can be found with
-  /// countTemporaryOperands().
+  /// countRendererFns().
   unsigned AllocatedTemporariesBaseID;
 
 public:
   OperandMatcher(InstructionMatcher &Insn, unsigned OpIdx,
                  const std::string &SymbolicName,
                  unsigned AllocatedTemporariesBaseID)
       : Insn(Insn), OpIdx(OpIdx), SymbolicName(SymbolicName),
         AllocatedTemporariesBaseID(AllocatedTemporariesBaseID) {}
for (const auto &Predicate : zip(predicates(), B.predicates())) {
         return false;
     }
 
     return false;
   };
 
   /// Report the maximum number of temporary operands needed by the operand
   /// matcher.
-  unsigned countTemporaryOperands() const {
+  unsigned countRendererFns() const {
     return std::accumulate(
         predicates().begin(), predicates().end(), 0,
         [](unsigned A,
            const std::unique_ptr<OperandPredicateMatcher> &Predicate) {
-          return A + Predicate->countTemporaryOperands();
+          return A + Predicate->countRendererFns();
         });
   }
 
   unsigned getAllocatedTemporariesBaseID() const {
     return AllocatedTemporariesBaseID;
   }
 };
 
public:
   ///
   /// Returns true if this object is more important than B.
   virtual bool isHigherPriorityThan(const InstructionPredicateMatcher &B) const {
     return Kind < B.Kind;
   };
 
   /// Report the maximum number of temporary operands needed by the predicate
   /// matcher.
-  virtual unsigned countTemporaryOperands() const { return 0; }
+  virtual unsigned countRendererFns() const { return 0; }
 };
 
 /// Generates code to check the opcode of an instruction.
 class InstructionOpcodeMatcher : public InstructionPredicateMatcher {
 protected:
   const CodeGenInstruction *I;
 
 public:
for (const auto &Operand : zip(Operands, B.Operands)) {
         return false;
     }
 
     return false;
   };
 
   /// Report the maximum number of temporary operands needed by the instruction
   /// matcher.
-  unsigned countTemporaryOperands() const {
+  unsigned countRendererFns() const {
     return std::accumulate(predicates().begin(), predicates().end(), 0,
                            [](unsigned A,
                               const std::unique_ptr<InstructionPredicateMatcher>
                                   &Predicate) {
-                             return A + Predicate->countTemporaryOperands();
+                             return A + Predicate->countRendererFns();
                            }) +
            std::accumulate(
                Operands.begin(), Operands.end(), 0,
                [](unsigned A, const std::unique_ptr<OperandMatcher> &Operand) {
-                 return A + Operand->countTemporaryOperands();
+                 return A + Operand->countRendererFns();
                });
   }
 };
 
 /// Generates code to check that the operand is a register defined by an
 /// instruction that matches the given instruction matcher.
 ///
 /// For example, the pattern:
void emitCxxPredicateExpr(raw_ostream &OS, RuleMatcher &Rule,
     OperandExpr = Rule.getInsnVarName(*InsnMatcher);
     OS << "(";
     InsnMatcher->emitCxxPredicateExpr(OS, Rule, OperandExpr);
     OS << ")\n";
   }
 };
 
 //===- Actions ------------------------------------------------------------===//
-void OperandPlaceholder::emitCxxValueExpr(raw_ostream &OS) const {
-  switch (Kind) {
-  case OP_MatchReference:
-    OS << MatchReference.InsnMatcher->getOperand(MatchReference.SymbolicName)
-              .getOperandExpr(MatchReference.InsnVarName);
-    break;
-  case OP_Temporary:
-    OS << "TempOp" << Temporary.OpIdx;
-    break;
-  }
-}
-
 class OperandRenderer {
 public:
   enum RendererKind { OR_Copy, OR_Imm, OR_Register, OR_ComplexPattern };
 
 protected:
   RendererKind Kind;
 
 public:
static bool classof(const OperandRenderer *R) {
     return R->getKind() == OR_Imm;
   }
 
   void emitCxxRenderStmts(raw_ostream &OS, RuleMatcher &Rule) const override {
     OS << "    MIB.addImm(" << Imm << ");\n";
   }
 };
 
+/// Adds operands by calling a renderer function supplied by the ComplexPattern
+/// matcher function.
 class RenderComplexPatternOperand : public OperandRenderer {
 private:
   const Record &TheDef;
-  std::vector<OperandPlaceholder> Sources;
+  /// The name of the operand.
+  const StringRef SymbolicName;
+  /// The renderer number. This must be unique within a rule since it's used to
+  /// identify a temporary variable to hold the renderer function.
+  unsigned RendererID;
 
   unsigned getNumOperands() const {
     return TheDef.getValueAsDag("Operands")->getNumArgs();
   }
 
 public:
-  RenderComplexPatternOperand(const Record &TheDef,
-                              const ArrayRef<OperandPlaceholder> Sources)
-      : OperandRenderer(OR_ComplexPattern), TheDef(TheDef), Sources(Sources) {}
+  RenderComplexPatternOperand(const Record &TheDef, StringRef SymbolicName,
+                              unsigned RendererID)
+      : OperandRenderer(OR_ComplexPattern), TheDef(TheDef),
+        SymbolicName(SymbolicName), RendererID(RendererID) {}
 
   static bool classof(const OperandRenderer *R) {
     return R->getKind() == OR_ComplexPattern;
   }
 
   void emitCxxRenderStmts(raw_ostream &OS, RuleMatcher &Rule) const override {
-    assert(Sources.size() == getNumOperands() && "Inconsistent number of operands");
-    for (const auto &Source : Sources) {
-      OS << "MIB.add(";
-      Source.emitCxxValueExpr(OS);
-      OS << ");\n";
-    }
+    OS << "Renderer" << RendererID << "(MIB);\n";
   }
 };
 
 /// An action taken when all Matcher predicates succeeded for a parent rule.
 ///
 /// Typical actions include:
 /// * Changing the opcode of an instruction.
 /// * Adding an operand to an instruction.
if (std::get<0>(Matcher)->isHigherPriorityThan(*std::get<1>(Matcher)))
       return true;
     if (std::get<1>(Matcher)->isHigherPriorityThan(*std::get<0>(Matcher)))
       return false;
   }
 
   return false;
 }
 
-unsigned RuleMatcher::countTemporaryOperands() const {
+unsigned RuleMatcher::countRendererFns() const {
   return std::accumulate(
       Matchers.begin(), Matchers.end(), 0,
       [](unsigned A, const std::unique_ptr<InstructionMatcher> &Matcher) {
-        return A + Matcher->countTemporaryOperands();
+        return A + Matcher->countRendererFns();
       });
 }
 
 //===- GlobalISelEmitter class --------------------------------------------===//
 
 class GlobalISelEmitter {
 public:
   explicit GlobalISelEmitter(RecordKeeper &RK);
if (auto *ChildDefInit = dyn_cast<DefInit>(SrcChild->getLeafValue())) {
 
     // Check for ComplexPattern's.
     if (ChildRec->isSubClassOf("ComplexPattern")) {
       const auto &ComplexPattern = ComplexPatternEquivs.find(ChildRec);
       if (ComplexPattern == ComplexPatternEquivs.end())
         return failedImport("SelectionDAG ComplexPattern (" +
                             ChildRec->getName() + ") not mapped to GlobalISel");
 
-      const auto &Predicate = OM.addPredicate<ComplexPatternOperandMatcher>(
-          OM, *ComplexPattern->second);
-      TempOpIdx += Predicate.countTemporaryOperands();
+      OM.addPredicate<ComplexPatternOperandMatcher>(OM,
+                                                    *ComplexPattern->second);
+      TempOpIdx++;
       return Error::success();
     }
 
     if (ChildRec->isSubClassOf("ImmLeaf")) {
       return failedImport(
           "Src pattern child def is an unsupported tablegen class (ImmLeaf)");
     }
 
if (auto *ChildDefInit = dyn_cast<DefInit>(DstChild->getLeafValue())) {
     }
 
     if (ChildRec->isSubClassOf("ComplexPattern")) {
       const auto &ComplexPattern = ComplexPatternEquivs.find(ChildRec);
       if (ComplexPattern == ComplexPatternEquivs.end())
         return failedImport(
             "SelectionDAG ComplexPattern not mapped to GlobalISel");
 
-      SmallVector<OperandPlaceholder, 2> RenderedOperands;
       const OperandMatcher &OM = InsnMatcher.getOperand(DstChild->getName());
-      for (unsigned I = 0; I < OM.countTemporaryOperands(); ++I)
-        RenderedOperands.push_back(OperandPlaceholder::CreateTemporary(
-            OM.getAllocatedTemporariesBaseID() + I));
       DstMIBuilder.addRenderer<RenderComplexPatternOperand>(
-          *ComplexPattern->second, RenderedOperands);
+          *ComplexPattern->second, DstChild->getName(),
+          OM.getAllocatedTemporariesBaseID());
       return Error::success();
     }
 
     if (ChildRec->isSubClassOf("SDNodeXForm"))
       return failedImport("Dst pattern child def is an unsupported tablegen "
                           "class (SDNodeXForm)");
 
     return failedImport(
std::stable_sort(Rules.begin(), Rules.end(),
                                                      "the same time");
                 return true;
               }
               return false;
             });
 
   unsigned MaxTemporaries = 0;
   for (const auto &Rule : Rules)
-    MaxTemporaries = std::max(MaxTemporaries, Rule.countTemporaryOperands());
+    MaxTemporaries = std::max(MaxTemporaries, Rule.countRendererFns());
 
   OS << "#ifdef GET_GLOBALISEL_PREDICATE_BITSET\n"
      << "const unsigned MAX_SUBTARGET_PREDICATES = " << SubtargetFeatures.size()
      << ";\n"
      << "using PredicateBitset = "
         "PredicateBitsetImpl<MAX_SUBTARGET_PREDICATES>;\n"
      << "#endif // ifdef GET_GLOBALISEL_PREDICATE_BITSET\n\n";
 
   OS << "#ifdef GET_GLOBALISEL_TEMPORARIES_DECL\n";
   for (unsigned I = 0; I < MaxTemporaries; ++I)
-    OS << "  mutable MachineOperand TempOp" << I << ";\n";
+    OS << "  mutable ComplexRendererFn Renderer" << I << ";\n";
   OS << "#endif // ifdef GET_GLOBALISEL_TEMPORARIES_DECL\n\n";
 
   OS << "#ifdef GET_GLOBALISEL_TEMPORARIES_INIT\n";
   for (unsigned I = 0; I < MaxTemporaries; ++I)
-    OS << ", TempOp" << I << "(MachineOperand::CreatePlaceholder())\n";
+    OS << ", Renderer" << I << "(nullptr)\n";
   OS << "#endif // ifdef GET_GLOBALISEL_TEMPORARIES_INIT\n\n";
 
   OS << "#ifdef GET_GLOBALISEL_IMPL\n";
   SubtargetFeatureInfo::emitSubtargetFeatureBitEnumeration(SubtargetFeatures,
                                                            OS);
   SubtargetFeatureInfo::emitNameTable(SubtargetFeatures, OS);
   SubtargetFeatureInfo::emitComputeAvailableFeatures(
       Target.getName(), "InstructionSelector", "computeAvailableFeatures",