[TableGen] Allow DAG isel patterns to override default operands.

[simon_tatham]

Summary

When a Tablegen instruction description uses OperandWithDefaultOps,
isel patterns for that instruction don't have to fill in the default
value for the operand in question. But the flip side is that they
actually can't override the defaults even if they want to.

This will be very inconvenient for the Arm backend, when we start
wanting to write isel patterns that generate the many MVE predicated
vector instructions, in the form with predication actually enabled. So
this small Tablegen fix makes it possible to write an isel pattern
either with or without values for a defaulted operand, and have the
default values filled in only if they are not overridden.

If all the defaulted operands come at the end of the instruction's
operand list, there's a natural way to match them up to the arguments
supplied in the pattern: consume pattern arguments until you run out,
then fill in any missing instruction operands with their default
values. But if defaulted and non-defaulted operands are interleaved,
it's less clear what to do. This does happen in existing targets (the
first example I came across was KILLGT, in the AMDGPU/R600 backend),
and of course they expect the previous behaviour (that the default for
those operands is used and a pattern argument is not consumed), so for
backwards compatibility I've stuck with that.

[simon_tatham]

Changed my mind about the opt-in system: instead of using a subclass of OperandWithDefaultOps, I've switched to using a flag field inside the existing class.

That should make it easier for a back end to define some flags one way and some another. And if people ever want to switch the default over to permitting overrides, and have a small number of back ends unset the same flag, it will be easier this way.

[nhaehnle]

Would it be possible to make default operands overridable automatically iff they are at the end of the operand list? I.e., if you have a suffix of default operands, then those can be overridden?

[simon_tatham]

Apparently it would: trying it that way, it does look simpler, and as far as I can tell it doesn't change the output of any currently checked-in Tablegen input.

[hfinkel]

LGTM

llvm/test/TableGen/DAGDefaultOps.td

+// RUN: llvm-tblgen -gen-dag-isel -I %p/../../include %s -o %t
+// RUN: FileCheck --check-prefix=ADD %s < %t
+// RUN: FileCheck --check-prefix=ADDINT %s < %t
+// RUN: FileCheck --check-prefix=SUB %s < %t
+// RUN: FileCheck --check-prefix=MULINT %s < %t
+
+include "llvm/Target/Target.td"
+
+def TestInstrInfo : InstrInfo;
+def TestTarget : Target {
+    let InstructionSet = TestInstrInfo;
+}
+
+class TestEncoding : Instruction {
+  field bits<32> Inst;
+}
+
+class TestReg<int index> : Register<"R"#index, []> {
+    let HWEncoding{15-4} = 0;
+    let HWEncoding{3-0} = !cast<bits<4>>(index);
+}
+foreach i = 0-15 in
+  def "R"#i : TestReg<i>;
+
+def Reg : RegisterClass<"TestTarget", [i32], 32, (sequence "R%d", 0, 15)>;
+
+def IntOperand: Operand<i32>;
+def OptionalIntOperand: OperandWithDefaultOps<i32, (ops (i32 0))>;
+
+class RRI<string Mnemonic, bits<4> Opcode> : TestEncoding {
+  dag OutOperandList = (outs Reg:$dest);
+  dag InOperandList = (ins Reg:$src1, Reg:$src2, OptionalIntOperand:$imm);
+  string AsmString = Mnemonic # " $dest1, $src1, $src2, #$imm";
+  string AsmVariantName = "";
+  field bits<4> dest;
+  field bits<4> src1;
+  field bits<4> src2;
+  field bits<16> imm;
+  let Inst{31-28} = Opcode;
+  let Inst{27-24} = dest;
+  let Inst{23-20} = src1;
+  let Inst{19-16} = src2;
+  let Inst{15-0} = imm;
+}
+
+def AddRRI : RRI<"add", 0b0001>;
+
+// I define one of these intrinsics with IntrNoMem and the other
+// without it, so that they'll match different top-level DAG opcodes
+// (INTRINSIC_WO_CHAIN and INTRINSIC_W_CHAIN), which makes the
+// FileCheck-herding easier, because every case I want to detect
+// should show up as a separate top-level switch element.
+def int_addplus1 : Intrinsic<
+    [llvm_i32_ty], [llvm_i32_ty, llvm_i32_ty], [IntrNoMem]>;
+def int_mul3 : Intrinsic<
+    [llvm_i32_ty], [llvm_i32_ty, llvm_i32_ty, llvm_i32_ty]>;
+
+def AddPat  : Pat<(add i32:$x, i32:$y),
+                  (AddRRI Reg:$x, Reg:$y)>;
+def Add1Pat : Pat<(int_addplus1 i32:$x, i32:$y),
+                  (AddRRI Reg:$x, Reg:$y, (i32 1))>;
+
+def SubRRI : RRI<"sub", 0b0010> {
+  let Pattern = [(set Reg:$dest, (sub Reg:$src1, Reg:$src2))];
+}
+
+def MulRRI : RRI<"mul", 0b0011> {
+  let Pattern = [(set Reg:$dest, (int_mul3 Reg:$src1, Reg:$src2, i32:$imm))];
+}
+
+def MulIRR : RRI<"mul2", 0b0100> {
+  let InOperandList = (ins OptionalIntOperand:$imm, Reg:$src1, Reg:$src2);
+}
+def MulIRRPat : Pat<(mul i32:$x, i32:$y), (MulIRR Reg:$x, Reg:$y)>;
+
+// ADD: SwitchOpcode{{.*}}TARGET_VAL(ISD::ADD)
+// ADD-NEXT: OPC_RecordChild0
+// ADD-NEXT: OPC_RecordChild1
+// ADD-NEXT: OPC_EmitInteger, MVT::i32, 0
+// ADD-NEXT: OPC_MorphNodeTo1, TARGET_VAL(::AddRRI)
+
+// ADDINT: SwitchOpcode{{.*}}TARGET_VAL(ISD::INTRINSIC_WO_CHAIN)
+// ADDINT-NEXT: OPC_CheckChild0Integer
+// ADDINT-NEXT: OPC_RecordChild1
+// ADDINT-NEXT: OPC_RecordChild2
+// ADDINT-NEXT: OPC_EmitInteger, MVT::i32, 1
+// ADDINT-NEXT: OPC_MorphNodeTo1, TARGET_VAL(::AddRRI)
+
+// SUB: SwitchOpcode{{.*}}TARGET_VAL(ISD::SUB)
+// SUB-NEXT: OPC_RecordChild0
+// SUB-NEXT: OPC_RecordChild1
+// SUB-NEXT: OPC_EmitInteger, MVT::i32, 0
+// SUB-NEXT: OPC_MorphNodeTo1, TARGET_VAL(::SubRRI)
+
+// MULINT: SwitchOpcode{{.*}}TARGET_VAL(ISD::INTRINSIC_W_CHAIN)
+// MULINT-NEXT: OPC_RecordNode
+// MULINT-NEXT: OPC_CheckChild1Integer
+// MULINT-NEXT: OPC_RecordChild2
+// MULINT-NEXT: OPC_RecordChild3
+// MULINT-NEXT: OPC_RecordChild4
+// MULINT-NEXT: OPC_EmitMergeInputChains
+// MULINT-NEXT: OPC_MorphNodeTo1, TARGET_VAL(::MulRRI)
+
+// MUL: SwitchOpcode{{.*}}TARGET_VAL(ISD::MUL)
+// MUL-NEXT: OPC_EmitInteger, MVT::i32, 0
+// MUL-NEXT: OPC_RecordChild0
+// MUL-NEXT: OPC_RecordChild1
+// MUL-NEXT: OPC_MorphNodeTo1, TARGET_VAL(::MulRRI)

llvm/utils/TableGen/CodeGenDAGPatterns.h

public:
   Record *get_intrinsic_wo_chain_sdnode() const {
     return intrinsic_wo_chain_sdnode;
   }
 
   bool hasTargetIntrinsics() { return !TgtIntrinsics.empty(); }
 
   unsigned allocateScope() { return ++NumScopes; }
 
+  bool operandHasDefault(Record *Op) const {
+    return Op->isSubClassOf("OperandWithDefaultOps") &&
+      !getDefaultOperand(Op).DefaultOps.empty();
+  }
+
 private:
   void ParseNodeInfo();
   void ParseNodeTransforms();
   void ParseComplexPatterns();
   void ParsePatternFragments(bool OutFrags = false);
   void ParseDefaultOperands();
   void ParseInstructions();
   void ParsePatterns();

llvm/utils/TableGen/CodeGenDAGPatterns.cpp

if (getOperator()->getName() == "INSERT_SUBREG") {
         if (!isOperandClass(SubIdxChild, "SubRegIndex")) {
           TP.error("REG_SEQUENCE requires a SubRegIndex for operand " +
                    Twine(I + 1) + "!");
           return false;
         }
       }
     }
 
+    // If one or more operands with a default value appear at the end of the
+    // formal operand list for an instruction, we allow them to be overridden
+    // by optional operands provided in the pattern.
+    //
+    // But if an operand B without a default appears at any point after an
+    // operand A with a default, then we don't allow A to be overridden,
+    // because there would be no way to specify whether the next operand in
+    // the pattern was intended to override A or skip it.
+    unsigned NonOverridableOperands = Inst.getNumOperands();
+    while (NonOverridableOperands > 0 &&
+           CDP.operandHasDefault(Inst.getOperand(NonOverridableOperands-1)))
+      --NonOverridableOperands;
+
     unsigned ChildNo = 0;
     for (unsigned i = 0, e = Inst.getNumOperands(); i != e; ++i) {
       Record *OperandNode = Inst.getOperand(i);
 
-      // If the instruction expects a predicate or optional def operand, we
-      // codegen this by setting the operand to it's default value if it has a
-      // non-empty DefaultOps field.
-      if (OperandNode->isSubClassOf("OperandWithDefaultOps") &&
-          !CDP.getDefaultOperand(OperandNode).DefaultOps.empty())
+      // If the operand has a default value, do we use it? We must use the
+      // default if we've run out of children of the pattern DAG to consume,
+      // or if the operand is followed by a non-defaulted one.
+      if (CDP.operandHasDefault(OperandNode) &&
+          (i < NonOverridableOperands || ChildNo >= getNumChildren()))
         continue;
 
-      // Verify that we didn't run out of provided operands.
+      // If we have run out of child nodes and there _isn't_ a default
+      // value we can use for the next operand, give an error.
       if (ChildNo >= getNumChildren()) {
         emitTooFewOperandsError(TP, getOperator()->getName(), getNumChildren());
         return false;
       }
 
       TreePatternNode *Child = getChild(ChildNo++);
       unsigned ChildResNo = 0;  // Instructions always use res #0 of their op.
 

llvm/utils/TableGen/DAGISelMatcherGen.cpp

EmitResultInstructionAsOperand(const TreePatternNode *N,
 
   // Loop over all of the fixed operands of the instruction pattern, emitting
   // code to fill them all in. The node 'N' usually has number children equal to
   // the number of input operands of the instruction.  However, in cases where
   // there are predicate operands for an instruction, we need to fill in the
   // 'execute always' values. Match up the node operands to the instruction
   // operands to do this.
   unsigned ChildNo = 0;
+
+  // Similarly to the code in TreePatternNode::ApplyTypeConstraints, count the
+  // number of operands at the end of the list which have default values.
+  // Those can come from the pattern if it provides enough arguments, or be
+  // filled in with the default if the pattern hasn't provided them. But any
+  // operand with a default value _before_ the last mandatory one will be
+  // filled in with their defaults unconditionally.
+  unsigned NonOverridableOperands = NumFixedOperands;
+  while (NonOverridableOperands > NumResults &&
+         CGP.operandHasDefault(II.Operands[NonOverridableOperands-1].Rec))
+    --NonOverridableOperands;
+
   for (unsigned InstOpNo = NumResults, e = NumFixedOperands;
        InstOpNo != e; ++InstOpNo) {
     // Determine what to emit for this operand.
     Record *OperandNode = II.Operands[InstOpNo].Rec;
-    if (OperandNode->isSubClassOf("OperandWithDefaultOps") &&
-        !CGP.getDefaultOperand(OperandNode).DefaultOps.empty()) {
-      // This is a predicate or optional def operand; emit the
-      // 'default ops' operands.
+    if (CGP.operandHasDefault(OperandNode) &&
+        (InstOpNo < NonOverridableOperands || ChildNo >= N->getNumChildren())) {
+      // This is a predicate or optional def operand which the pattern has not
+      // overridden, or which we aren't letting it override; emit the 'default
+      // ops' operands.
       const DAGDefaultOperand &DefaultOp
         = CGP.getDefaultOperand(OperandNode);
       for (unsigned i = 0, e = DefaultOp.DefaultOps.size(); i != e; ++i)
         EmitResultOperand(DefaultOp.DefaultOps[i].get(), InstOps);
       continue;
     }
 
     // Otherwise this is a normal operand or a predicate operand without