diff --git a/llvm/docs/GlobalISel/MIRPatterns.rst b/llvm/docs/GlobalISel/MIRPatterns.rst new file mode 100644 --- /dev/null +++ b/llvm/docs/GlobalISel/MIRPatterns.rst @@ -0,0 +1,304 @@ + +.. _tblgen-mirpats: + +======================== +MIR Patterns in TableGen +======================== + +.. contents:: + :local: + + +User's Guide +============ + +This section is intended for developers who want to use MIR patterns in their +TableGen files. + +``NOTE``: +This feature is still in active development. This document may become outdated +over time. If you see something that's incorrect, please update it. + +Use Cases +--------- + +MIR patterns are supported in the following places: + +* GlobalISel ``GICombineRule`` +* GlobalISel ``GICombinePatFrag`` + +Syntax +------ + +MIR patterns use the DAG datatype in TableGen. + +.. code-block:: text + + (inst operand0, operand1, ...) + +``inst`` must be a def which inherits from ``Instruction`` (e.g. ``G_FADD``) +or ``GICombinePatFrag``. + +Operands essentially fall into one of two categories: + +* immediates + + * untyped, unnamed: ``0`` + * untyped, named: ``0:$y`` + * typed, unnamed: ``(i32 0)`` + * typed, named: ``(i32 0):$y`` + +* machine operands + + * untyped: ``$x`` + * typed: ``i32:$x`` + +Semantics: + +* A typed operand always adds an operand type check to the matcher. +* There is a trivial type inference system to propagate types. + + * e.g. You only need to use ``i32:$x`` once in any pattern, then all + other patterns can simply use ``$x`` (``i32:$x`` is redundant). + +* A nammed operand's behavior depends on whether the name has been seen before. + + * For match patterns, reusing an operand name checks that the operands + are identical (see example 2 below). + * For apply patterns, reusing an operand name simply copies that operand into + the new instruction (see example 2 below). + +Operands are ordered just like they would be in a MachineInstr: the defs (outs) +come first, then the uses (ins). + +Patterns are generally grouped into another DAG datatype with a dummy operator +such as ``match``, ``apply`` or ``pattern``. + +Finally, any DAG datatype in TableGen can be named. This also holds for +patterns. e.g. the following is valid: ``(G_FOO $root, (i32 0):$cst):$mypat``. +This may also be helpful to debug issues. Patterns are *always* named, and if +they don't have a name, an "anonymous" one is given to them. If you're trying +to debug an error related to a MIR pattern, but the error mentions an anonymous +pattern, you can try naming your patterns to see exactly where the issue is. + +.. code-block:: text + :caption: Pattern Example 1 + + // Match + // %imp = G_IMPLICIT_DEF + // %root = G_MUL %x, %imp + (match (G_IMPLICIT_DEF $imp), + (G_MUL $root, $x, $imp)) + +.. code-block:: text + :caption: Pattern Example 2 + + // using $x twice here checks that the operand 1 and 2 of the G_AND are + // identical. + (match (G_AND $root, $x, $x)) + // using $x again here copies operand 1 from G_AND into the new inst. + (apply (COPY $root, $x)) + + +Limitations +----------- + +This a non-exhaustive list of known issues with MIR patterns at this time. + +* Matching intrinsics is not yet possible. +* Using ``GICombinePatFrag`` within another ``GICombinePatFrag`` is not + supported. +* Using ``GICombinePatFrag`` in the ``apply`` pattern of a ``GICombineRule`` + is not supported. +* Immediates must always be typed in the ``apply`` pattern of a + ``GICombineRule``. +* Immediates always emit a CImm in the ``apply`` pattern of a + ``GICombineRule``. There is no way to implicitly emit a ``G_CONSTANT``, + or emit an simple immediate operand. +* Matching a constant >64 bits is not supported (see comment in + ``GIM_CheckConstantInt```) + +GICombineRule +---------------------- + +MIR patterns can appear in the ``match`` or ``apply`` patterns of a +``GICombineRule``. + +The ``root`` of the rule can either be a def of an instruction, or a +named pattern. The latter is helpful when the instruction you want +to match has no defs. The former is generally preferred because +it's less verbose. + +.. code-block:: text + :caption: Combine Rule root is a def + + // Fold x op 1 -> x + def right_identity_one: GICombineRule< + (defs root:$dst), + (match (G_MUL $dst, $x, 1)), + // Note: Patterns always need to create something, we can't just replace $dst with $x, so we need a COPY. + (apply (COPY $dst, $x)) + >; + +.. code-block:: text + :caption: Combine Rule root is a named pattern + + def Foo : GICombineRule< + (defs root:$root), + (match (G_ZEXT $tmp, (i32 0)), + (G_STORE $tmp, $ptr):$root), + (apply (G_STORE (i32 0), $ptr):$root)>; + + +Combine Rules also allow mixing C++ code with MIR patterns, so that you +may perform additional checks when matching, or run additional code after +rewriting a pattern. + +The following expansions are available for MIR patterns: + +* operand names (``MachineOperand &``) +* pattern names (``MachineInstr *`` for ``match``, + ``MachineInstrBuilder &`` for apply) + +GICombinePatFrag +---------------- + +``GICombinePatFrag`` is an equivalent of ``PatFrags`` for MIR patterns. +They have two main usecases: + +* Reduce repetition by creating a ``GICombinePatFrag`` for common + patterns (see example 1). +* Implicitly duplicate a CombineRule for multiple variants of a + pattern (see example 2). + +A ``GICombinePatFrag`` is composed of three elements: + +* zero or more ``in`` (def) parameter +* zero or more ``out`` parameter +* A list of MIR patterns that can match. + + * When a ``GICombinePatFrag`` is used within a pattern, the pattern is + cloned once for each alternative that can match. + +Parameters can have the following types: + +* ``gi_mo``, which is the implicit default (no type = ``gi_mo``). + + * Refers to any operand of an instruction (register, BB ref, imm, etc.). + * Can be used in both ``in`` and ``out`` parameters. + * Users of the PatFrag can only use an operand name for this + parameter (e.g. ``(my_pat_frag $foo)``). + +* ``root`` + + * This is identical to ``gi_mo``. + * Can only be used in ``out`` parameters to declare the root of the + pattern. + * Non-empty ``out`` parameter lists must always have exactly one ``root``. + +* ``gi_imm`` + + * Refers to an (potentially typed) immediate. + * Can only be used in ``in`` parameters. + * Users of the PatFrag can only use an immediate for this parameter + (e.g. ``(my_pat_frag 0)`` or ``(my_pat_frag (i32 0))``) + +``out`` operands can only be empty if the ``GICombinePatFrag`` only contains +C++ code. If the fragment contains instruction patterns, it has to have at +least one ``out`` operand of type ``root``. + +``in`` operands are less restricted, but there is one important concept to +remember: you can pass "unbound" operand names, but only if the +``GICombinePatFrag`` binds it. See example 3 below. + +``GICombinePatFrag`` are used just like any other instructions. +Note that the ``out`` operands are defs, so they come first in the list +of operands. + +.. code-block:: text + :caption: Example 1: Reduce Repetition + + def zext_cst : GICombinePatFrag<(outs root:$dst, $cst), (ins gi_imm:$val), + [(pattern (G_CONSTANT $cst, $val), + (G_ZEXT $dst, $cst))] + >; + + def foo_to_impdef : GICombineRule< + (defs root:$dst), + (match (zext_cst $y, $cst, (i32 0)) + (G_FOO $dst, $y)), + (apply (G_IMPLICIT_DEF $dst))>; + + def store_ext_zero : GICombineRule< + (defs root:$root), + (match (zext_cst $y, $cst, (i32 0)) + (G_STORE $y, $ptr):$root), + (apply (G_STORE $cst, $ptr):$root)>; + +.. code-block:: text + :caption: Example 2: Generate Multiple Rules at Once + + // Fold (freeze (freeze x)) -> (freeze x). + // Fold (fabs (fabs x)) -> (fabs x). + // Fold (fcanonicalize (fcanonicalize x)) -> (fcanonicalize x). + def idempotent_prop_frags : GICombinePatFrag<(outs root:$dst, $src), (ins), + [ + (pattern (G_FREEZE $dst, $src), (G_FREEZE $src, $x)), + (pattern (G_FABS $dst, $src), (G_FABS $src, $x)), + (pattern (G_FCANONICALIZE $dst, $src), (G_FCANONICALIZE $src, $x)) + ] + >; + + def idempotent_prop : GICombineRule< + (defs root:$dst), + (match (idempotent_prop_frags $dst, $src)), + (apply (COPY $dst, $src))>; + + + +.. code-block:: text + :caption: Example 3: Unbound Operand Names + + // This fragment binds $x to an operand in all of its + // alternative patterns. + def always_binds : GICombinePatFrag< + (outs root:$dst), (ins $x), + [ + (pattern (G_FREEZE $dst, $x)), + (pattern (G_FABS $dst, $x)), + ] + >; + + // This fragment does not bind $x to an operand in any + // all of its alternative patterns. + def does_not_bind : GICombinePatFrag< + (outs root:$dst), (ins $x), + [ + (pattern (G_FREEZE $dst, $x)), // binds $x + (pattern (G_FOO $dst (i32 0))), // does not bind $x + (pattern "return myCheck(${x}.getReg())"), // does not bind $x + ] + >; + + // Here we pass $x, which is unbound, to always_binds. + // This works because if $x is unbound, always_binds will bind it for us. + def test0 : GICombineRule< + (defs root:$dst), + (match (always_binds $dst, $x)), + (apply (COPY $dst, $x))>; + + // Here we pass $src, which is unbound, to does_not_bind. + // This cannot work because $x may not have been initialized in 'apply'. + // error: operand 'x' (for parameter 'src' of 'does_not_bind') cannot be unbound + def test1 : GICombineRule< + (defs root:$dst), + (match (does_not_bind $dst, $x)), + (apply (COPY $dst, $x))>; + + // Here we pass $src, which is bound, to does_not_bind. + // This is fine because $x will always be bound when emitting does_not_bind + def test2 : GICombineRule< + (defs root:$dst), + (match (does_not_bind $tmp, $x) + (G_MUL $dst, $x, $tmp)), + (apply (COPY $dst, $x))>; diff --git a/llvm/docs/GlobalISel/index.rst b/llvm/docs/GlobalISel/index.rst --- a/llvm/docs/GlobalISel/index.rst +++ b/llvm/docs/GlobalISel/index.rst @@ -51,6 +51,7 @@ GMIR GenericOpcode + MIRPatterns Pipeline Porting Resources diff --git a/llvm/docs/UserGuides.rst b/llvm/docs/UserGuides.rst --- a/llvm/docs/UserGuides.rst +++ b/llvm/docs/UserGuides.rst @@ -35,6 +35,7 @@ FatLTO ExtendingLLVM GoldPlugin + GlobalISel/MIRPatterns HowToBuildOnARM HowToBuildWithPGO HowToBuildWindowsItaniumPrograms @@ -190,6 +191,13 @@ Describes the TableGen tool, which is used heavily by the LLVM code generator. +========== +GlobalISel +========== + +:doc:`MIRPatterns ` + Describes the design of MIR Patterns and how to use them. + === JIT === @@ -261,4 +269,3 @@ :doc:`RISCVUsage` This document describes using the RISCV-V target. - diff --git a/llvm/include/llvm/CodeGen/GlobalISel/GIMatchTableExecutor.h b/llvm/include/llvm/CodeGen/GlobalISel/GIMatchTableExecutor.h --- a/llvm/include/llvm/CodeGen/GlobalISel/GIMatchTableExecutor.h +++ b/llvm/include/llvm/CodeGen/GlobalISel/GIMatchTableExecutor.h @@ -360,6 +360,12 @@ /// - Imm - The immediate to add GIR_AddImm, + /// Add an CImm to the specified instruction + /// - InsnID - Instruction ID to modify + /// - Ty - Type of the constant immediate. + /// - Imm - The immediate to add + GIR_AddCImm, + /// Render complex operands to the specified instruction /// - InsnID - Instruction ID to modify /// - RendererID - The renderer to call @@ -386,7 +392,8 @@ /// Calls a C++ function to perform an action when a match is complete. /// The MatcherState is passed to the function to allow it to modify /// instructions. - /// This is less constrained than a custom renderer and can update instructions + /// This is less constrained than a custom renderer and can update + /// instructions /// in the state. /// - FnID - The function to call. /// TODO: Remove this at some point when combiners aren't reliant on it. It's @@ -553,7 +560,8 @@ const int64_t *MatchTable, const TargetInstrInfo &TII, MachineRegisterInfo &MRI, const TargetRegisterInfo &TRI, const RegisterBankInfo &RBI, const PredicateBitset &AvailableFeatures, - CodeGenCoverage *CoverageInfo) const; + CodeGenCoverage *CoverageInfo, + GISelChangeObserver *Observer = nullptr) const; virtual const int64_t *getMatchTable() const { llvm_unreachable("Should have been overridden by tablegen if used"); @@ -581,12 +589,14 @@ llvm_unreachable("Subclass does not implement testSimplePredicate!"); } - virtual void runCustomAction(unsigned, const MatcherState &State) const { + virtual void runCustomAction(unsigned, const MatcherState &State, + NewMIVector &OutMIs) const { llvm_unreachable("Subclass does not implement runCustomAction!"); } bool isOperandImmEqual(const MachineOperand &MO, int64_t Value, - const MachineRegisterInfo &MRI) const; + const MachineRegisterInfo &MRI, + bool Splat = false) const; /// Return true if the specified operand is a G_PTR_ADD with a G_CONSTANT on /// the right-hand side. GlobalISel's separation of pointer and integer types diff --git a/llvm/include/llvm/CodeGen/GlobalISel/GIMatchTableExecutorImpl.h b/llvm/include/llvm/CodeGen/GlobalISel/GIMatchTableExecutorImpl.h --- a/llvm/include/llvm/CodeGen/GlobalISel/GIMatchTableExecutorImpl.h +++ b/llvm/include/llvm/CodeGen/GlobalISel/GIMatchTableExecutorImpl.h @@ -17,6 +17,7 @@ #include "llvm/ADT/SmallVector.h" #include "llvm/CodeGen/GlobalISel/GIMatchTableExecutor.h" +#include "llvm/CodeGen/GlobalISel/GISelChangeObserver.h" #include "llvm/CodeGen/GlobalISel/Utils.h" #include "llvm/CodeGen/MachineInstrBuilder.h" #include "llvm/CodeGen/MachineOperand.h" @@ -27,6 +28,7 @@ #include "llvm/CodeGen/TargetRegisterInfo.h" #include "llvm/IR/Constants.h" #include "llvm/IR/DataLayout.h" +#include "llvm/IR/Type.h" #include "llvm/Support/CodeGenCoverage.h" #include "llvm/Support/Debug.h" #include "llvm/Support/ErrorHandling.h" @@ -46,7 +48,7 @@ const int64_t *MatchTable, const TargetInstrInfo &TII, MachineRegisterInfo &MRI, const TargetRegisterInfo &TRI, const RegisterBankInfo &RBI, const PredicateBitset &AvailableFeatures, - CodeGenCoverage *CoverageInfo) const { + CodeGenCoverage *CoverageInfo, GISelChangeObserver *Observer) const { uint64_t CurrentIdx = 0; SmallVector OnFailResumeAt; @@ -299,8 +301,7 @@ assert(State.MIs[InsnID] != nullptr && "Used insn before defined"); assert(State.MIs[InsnID]->getOpcode() == TargetOpcode::G_CONSTANT && "Expected G_CONSTANT"); - assert(Predicate > GICXXPred_Invalid && - "Expected a valid predicate"); + assert(Predicate > GICXXPred_Invalid && "Expected a valid predicate"); if (!State.MIs[InsnID]->getOperand(1).isCImm()) llvm_unreachable("Expected Imm or CImm operand"); @@ -323,8 +324,7 @@ "Expected G_FCONSTANT"); assert(State.MIs[InsnID]->getOperand(1).isFPImm() && "Expected FPImm operand"); - assert(Predicate > GICXXPred_Invalid && - "Expected a valid predicate"); + assert(Predicate > GICXXPred_Invalid && "Expected a valid predicate"); const APFloat &Value = State.MIs[InsnID]->getOperand(1).getFPImm()->getValueAPF(); @@ -727,9 +727,16 @@ if (MO.isReg()) { // isOperandImmEqual() will sign-extend to 64-bits, so should we. LLT Ty = MRI.getType(MO.getReg()); - Value = SignExtend64(Value, Ty.getSizeInBits()); + // If the type is > 64 bits, it can't be a constant int, so we bail + // early because SignExtend64 will assert otherwise. + if (Ty.getScalarSizeInBits() > 64) { + if (handleReject() == RejectAndGiveUp) + return false; + break; + } - if (!isOperandImmEqual(MO, Value, MRI)) { + Value = SignExtend64(Value, Ty.getScalarSizeInBits()); + if (!isOperandImmEqual(MO, Value, MRI, /*Splat=*/true)) { if (handleReject() == RejectAndGiveUp) return false; } @@ -1010,6 +1017,23 @@ break; } + case GIR_AddCImm: { + int64_t InsnID = MatchTable[CurrentIdx++]; + int64_t TypeID = MatchTable[CurrentIdx++]; + int64_t Imm = MatchTable[CurrentIdx++]; + assert(OutMIs[InsnID] && "Attempted to add to undefined instruction"); + + unsigned Width = ExecInfo.TypeObjects[TypeID].getScalarSizeInBits(); + LLVMContext &Ctx = MF->getFunction().getContext(); + OutMIs[InsnID].addCImm( + ConstantInt::get(IntegerType::get(Ctx, Width), Imm, /*signed*/ true)); + DEBUG_WITH_TYPE(TgtExecutor::getName(), + dbgs() << CurrentIdx << ": GIR_AddCImm(OutMIs[" << InsnID + << "], TypeID=" << TypeID << ", Imm=" << Imm + << ")\n"); + break; + } + case GIR_ComplexRenderer: { int64_t InsnID = MatchTable[CurrentIdx++]; int64_t RendererID = MatchTable[CurrentIdx++]; @@ -1109,7 +1133,7 @@ dbgs() << CurrentIdx << ": GIR_CustomAction(FnID=" << FnID << ")\n"); assert(FnID > GICXXCustomAction_Invalid && "Expected a valid FnID"); - runCustomAction(FnID, State); + runCustomAction(FnID, State, OutMIs); break; } case GIR_CustomOperandRenderer: { @@ -1179,12 +1203,14 @@ case GIR_EraseFromParent: { int64_t InsnID = MatchTable[CurrentIdx++]; - assert(State.MIs[InsnID] && - "Attempted to erase an undefined instruction"); - State.MIs[InsnID]->eraseFromParent(); + MachineInstr *MI = State.MIs[InsnID]; + assert(MI && "Attempted to erase an undefined instruction"); DEBUG_WITH_TYPE(TgtExecutor::getName(), dbgs() << CurrentIdx << ": GIR_EraseFromParent(MIs[" << InsnID << "])\n"); + if (Observer) + Observer->erasingInstr(*MI); + MI->eraseFromParent(); break; } @@ -1214,6 +1240,10 @@ case GIR_Done: DEBUG_WITH_TYPE(TgtExecutor::getName(), dbgs() << CurrentIdx << ": GIR_Done\n"); + if (Observer) { + for (MachineInstr *MI : OutMIs) + Observer->createdInstr(*MI); + } propagateFlags(OutMIs); return true; default: diff --git a/llvm/include/llvm/Target/GlobalISel/Combine.td b/llvm/include/llvm/Target/GlobalISel/Combine.td --- a/llvm/include/llvm/Target/GlobalISel/Combine.td +++ b/llvm/include/llvm/Target/GlobalISel/Combine.td @@ -66,10 +66,11 @@ /// is called. Targets can use this to, for instance, check Subtarget /// features. list Predicates = []; -} -/// The operator at the root of a GICombineRule.Defs dag. -def defs; + // Maximum number of permutations of this rule that can be emitted. + // Set to -1 to disable the limit. + int MaxPermutations = 16; +} /// All arguments of the defs operator must be subclasses of GIDefKind or /// sub-dags whose operator is GIDefKindWithArgs. @@ -82,6 +83,30 @@ /// is incorrect. def root : GIDefKind; +def gi_mo; +def gi_imm; +def pattern; + +// This is an equivalent of PatFrags but for MIR Patterns. +// +// GICombinePatFrags can be used in place of instructions for 'match' patterns. +// Much like normal instructions, the defs (outs) come first, and the ins second +// +// Out operands can only be of type "root" or "gi_mo", and they must be defined +// by an instruction pattern in all alternatives. +// +// In operands can be gi_imm or gi_mo. They cannot be redefined in any alternative +// pattern and may only appear in the C++ code, or in the output operand of an +// instruction pattern. +class GICombinePatFrag alts> { + dag InOperands = ins; + dag OutOperands = outs; + list Alternatives = alts; +} + +/// The operator at the root of a GICombineRule.Defs dag. +def defs; + /// Declares data that is passed from the match stage to the apply stage. class GIDefMatchData : GIDefKind { /// A C++ type name indicating the storage type. diff --git a/llvm/lib/CodeGen/GlobalISel/GIMatchTableExecutor.cpp b/llvm/lib/CodeGen/GlobalISel/GIMatchTableExecutor.cpp --- a/llvm/lib/CodeGen/GlobalISel/GIMatchTableExecutor.cpp +++ b/llvm/lib/CodeGen/GlobalISel/GIMatchTableExecutor.cpp @@ -26,12 +26,19 @@ GIMatchTableExecutor::GIMatchTableExecutor() = default; -bool GIMatchTableExecutor::isOperandImmEqual( - const MachineOperand &MO, int64_t Value, - const MachineRegisterInfo &MRI) const { - if (MO.isReg() && MO.getReg()) +bool GIMatchTableExecutor::isOperandImmEqual(const MachineOperand &MO, + int64_t Value, + const MachineRegisterInfo &MRI, + bool Splat) const { + if (MO.isReg() && MO.getReg()) { if (auto VRegVal = getIConstantVRegValWithLookThrough(MO.getReg(), MRI)) return VRegVal->Value.getSExtValue() == Value; + + if (Splat) { + if (auto VRegVal = getIConstantSplatVal(MO.getReg(), MRI)) + return VRegVal->getSExtValue() == Value; + } + } return false; } diff --git a/llvm/test/TableGen/GlobalISelCombinerMatchTableEmitter/match-table-imms.td b/llvm/test/TableGen/GlobalISelCombinerMatchTableEmitter/match-table-imms.td new file mode 100644 --- /dev/null +++ b/llvm/test/TableGen/GlobalISelCombinerMatchTableEmitter/match-table-imms.td @@ -0,0 +1,70 @@ +// RUN: llvm-tblgen -I %p/../../../include -gen-global-isel-combiner-matchtable \ +// RUN: -combiners=MyCombiner %s | \ +// RUN: FileCheck %s + +include "llvm/Target/Target.td" +include "llvm/Target/GlobalISel/Combine.td" + +def MyTargetISA : InstrInfo; +def MyTarget : Target { let InstructionSet = MyTargetISA; } + +def InstTest0 : GICombineRule< + (defs root:$a), + (match (COPY $a, (i32 0))), + (apply [{ APPLY }])>; + +def InstTest1 : GICombineRule< + (defs root:$a), + (match (G_ZEXT $a, 0)), + (apply [{ APPLY }])>; + +def CImmInstTest1 : GICombineRule< + (defs root:$a), + (match (G_CONSTANT $a, (i32 0))), + (apply [{ APPLY }])>; + +def MyCombiner: GICombinerHelper<"GenMyCombiner", [ + InstTest0, + InstTest1, + CImmInstTest1 +]>; + +// CHECK: const int64_t *GenMyCombiner::getMatchTable() const { +// CHECK-NEXT: constexpr static int64_t MatchTable0[] = { +// CHECK-NEXT: GIM_Try, /*On fail goto*//*Label 0*/ 18, // Rule ID 0 // +// CHECK-NEXT: GIM_CheckSimplePredicate, GICXXPred_Simple_IsRule0Enabled, +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/0, TargetOpcode::COPY, +// CHECK-NEXT: GIM_CheckType, /*MI*/0, /*Op*/1, /*Type*/GILLT_s32, +// CHECK-NEXT: // MIs[0] a +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: GIM_CheckConstantInt, /*MI*/0, /*Op*/1, 0, +// CHECK-NEXT: // Combiner Rule #0: InstTest0 +// CHECK-NEXT: GIR_CustomAction, GICXXCustomAction_CombineApplyGICombiner0, +// CHECK-NEXT: GIR_Done, +// CHECK-NEXT: // Label 0: @18 +// CHECK-NEXT: GIM_Try, /*On fail goto*//*Label 1*/ 32, // Rule ID 1 // +// CHECK-NEXT: GIM_CheckSimplePredicate, GICXXPred_Simple_IsRule1Enabled, +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/0, TargetOpcode::G_ZEXT, +// CHECK-NEXT: // MIs[0] a +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // MIs[0] Operand 1 +// CHECK-NEXT: GIM_CheckConstantInt, /*MI*/0, /*Op*/1, 0, +// CHECK-NEXT: // Combiner Rule #1: InstTest1 +// CHECK-NEXT: GIR_CustomAction, GICXXCustomAction_CombineApplyGICombiner0, +// CHECK-NEXT: GIR_Done, +// CHECK-NEXT: // Label 1: @32 +// CHECK-NEXT: GIM_Try, /*On fail goto*//*Label 2*/ 50, // Rule ID 2 // +// CHECK-NEXT: GIM_CheckSimplePredicate, GICXXPred_Simple_IsRule2Enabled, +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/0, TargetOpcode::G_CONSTANT, +// CHECK-NEXT: GIM_CheckType, /*MI*/0, /*Op*/1, /*Type*/GILLT_s32, +// CHECK-NEXT: // MIs[0] a +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: GIM_CheckLiteralInt, /*MI*/0, /*Op*/1, 0, +// CHECK-NEXT: // Combiner Rule #2: CImmInstTest1 +// CHECK-NEXT: GIR_CustomAction, GICXXCustomAction_CombineApplyGICombiner0, +// CHECK-NEXT: GIR_Done, +// CHECK-NEXT: // Label 2: @50 +// CHECK-NEXT: GIM_Reject, +// CHECK-NEXT: }; +// CHECK-NEXT: return MatchTable0; +// CHECK-NEXT: } diff --git a/llvm/test/TableGen/GlobalISelCombinerMatchTableEmitter/match-table-operand-types.td b/llvm/test/TableGen/GlobalISelCombinerMatchTableEmitter/match-table-operand-types.td new file mode 100644 --- /dev/null +++ b/llvm/test/TableGen/GlobalISelCombinerMatchTableEmitter/match-table-operand-types.td @@ -0,0 +1,52 @@ +// RUN: llvm-tblgen -I %p/../../../include -gen-global-isel-combiner-matchtable \ +// RUN: -combiners=MyCombiner %s | \ +// RUN: FileCheck %s + +include "llvm/Target/Target.td" +include "llvm/Target/GlobalISel/Combine.td" + +def MyTargetISA : InstrInfo; +def MyTarget : Target { let InstructionSet = MyTargetISA; } + +def InstTest0 : GICombineRule< + (defs root:$a), + (match (G_MUL i32:$x, i32:$b, i32:$c), + (G_MUL $a, i32:$b, i32:$x)), + (apply (G_ADD i64:$tmp, $b, i32:$c), + (G_ADD i8:$a, $b, i64:$tmp))>; + +def MyCombiner: GICombinerHelper<"GenMyCombiner", [ + InstTest0, +]>; + +// CHECK: const int64_t *GenMyCombiner::getMatchTable() const { +// CHECK-NEXT: constexpr static int64_t MatchTable0[] = { +// CHECK-NEXT: GIM_Try, /*On fail goto*//*Label 0*/ 73, // Rule ID 0 // +// CHECK-NEXT: GIM_CheckSimplePredicate, GICXXPred_Simple_IsRule0Enabled, +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/0, TargetOpcode::G_MUL, +// CHECK-NEXT: GIM_CheckType, /*MI*/0, /*Op*/0, /*Type*/GILLT_s8, +// CHECK-NEXT: GIM_CheckType, /*MI*/0, /*Op*/1, /*Type*/GILLT_s32, +// CHECK-NEXT: GIM_CheckType, /*MI*/0, /*Op*/2, /*Type*/GILLT_s32, +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/1, /*MI*/0, /*OpIdx*/2, // MIs[1] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/1, TargetOpcode::G_MUL, +// CHECK-NEXT: GIM_CheckType, /*MI*/1, /*Op*/2, /*Type*/GILLT_s32, +// CHECK-NEXT: // MIs[1] b +// CHECK-NEXT: GIM_CheckIsSameOperandIgnoreCopies, /*MI*/1, /*OpIdx*/1, /*OtherMI*/0, /*OtherOpIdx*/1, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/1, +// CHECK-NEXT: // Combiner Rule #0: InstTest0 +// CHECK-NEXT: GIR_MakeTempReg, /*TempRegID*/0, /*TypeID*/GILLT_s64, +// CHECK-NEXT: GIR_BuildMI, /*InsnID*/0, /*Opcode*/TargetOpcode::G_ADD, +// CHECK-NEXT: GIR_AddTempRegister, /*InsnID*/0, /*TempRegID*/0, /*TempRegFlags*/0, +// CHECK-NEXT: GIR_Copy, /*NewInsnID*/0, /*OldInsnID*/0, /*OpIdx*/1, // b +// CHECK-NEXT: GIR_Copy, /*NewInsnID*/0, /*OldInsnID*/1, /*OpIdx*/2, // c +// CHECK-NEXT: GIR_EraseFromParent, /*InsnID*/0, +// CHECK-NEXT: GIR_BuildMI, /*InsnID*/1, /*Opcode*/TargetOpcode::G_ADD, +// CHECK-NEXT: GIR_Copy, /*NewInsnID*/1, /*OldInsnID*/0, /*OpIdx*/0, // a +// CHECK-NEXT: GIR_Copy, /*NewInsnID*/1, /*OldInsnID*/0, /*OpIdx*/1, // b +// CHECK-NEXT: GIR_AddTempRegister, /*InsnID*/1, /*TempRegID*/0, /*TempRegFlags*/0, +// CHECK-NEXT: GIR_Done, +// CHECK-NEXT: // Label 0: @73 +// CHECK-NEXT: GIM_Reject, +// CHECK-NEXT: }; +// CHECK-NEXT: return MatchTable0; +// CHECK-NEXT: } diff --git a/llvm/test/TableGen/GlobalISelCombinerMatchTableEmitter/match-table-patfrag-root.td b/llvm/test/TableGen/GlobalISelCombinerMatchTableEmitter/match-table-patfrag-root.td new file mode 100644 --- /dev/null +++ b/llvm/test/TableGen/GlobalISelCombinerMatchTableEmitter/match-table-patfrag-root.td @@ -0,0 +1,79 @@ +// RUN: llvm-tblgen -I %p/../../../include -gen-global-isel-combiner-matchtable \ +// RUN: -combiners=MyCombiner -gicombiner-matchtable-debug-cxxpreds %s | \ +// RUN: FileCheck %s + +include "llvm/Target/Target.td" +include "llvm/Target/GlobalISel/Combine.td" + +def MyTargetISA : InstrInfo; +def MyTarget : Target { let InstructionSet = MyTargetISA; } + +def MatchFooPerms: GICombinePatFrag< + (outs root:$foo), + (ins gi_imm:$cst), + [ + (pattern (G_ZEXT $foo, $b), (G_TRUNC $b, $x):$dbg0), + (pattern (G_TRUNC $foo, $z):$dbg1), + (pattern (G_FPEXT $foo, $z):$dbg1) + ]>; + +def Test0 : GICombineRule< + (defs root:$root), + (match (MatchFooPerms $root, (i32 10))), + (apply (COPY $root, (i32 0)))>; + +def MyCombiner: GICombinerHelper<"GenMyCombiner", [ + Test0 +]>; +// CHECK: const int64_t *GenMyCombiner::getMatchTable() const { +// CHECK-NEXT: constexpr static int64_t MatchTable0[] = { +// CHECK-NEXT: GIM_Try, /*On fail goto*//*Label 0*/ 30, // Rule ID 0 // +// CHECK-NEXT: GIM_CheckSimplePredicate, GICXXPred_Simple_IsRule0Enabled, +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/0, TargetOpcode::G_ZEXT, +// CHECK-NEXT: // MIs[0] root +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // MIs[0] __anon_pat_match_0_0.b +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/1, /*MI*/0, /*OpIdx*/1, // MIs[1] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/1, TargetOpcode::G_TRUNC, +// CHECK-NEXT: // MIs[1] __anon_pat_match_0_0.x +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/1, +// CHECK-NEXT: // Combiner Rule #0: Test0 @ [__anon_pat_match_0_0[0]] +// CHECK-NEXT: GIR_BuildMI, /*InsnID*/0, /*Opcode*/TargetOpcode::COPY, +// CHECK-NEXT: GIR_Copy, /*NewInsnID*/0, /*OldInsnID*/0, /*OpIdx*/0, // root +// CHECK-NEXT: GIR_AddCImm, /*InsnID*/0, /*Type*/GILLT_s32, /*Imm*/0, +// CHECK-NEXT: GIR_EraseFromParent, /*InsnID*/0, +// CHECK-NEXT: GIR_Done, +// CHECK-NEXT: // Label 0: @30 +// CHECK-NEXT: GIM_Try, /*On fail goto*//*Label 1*/ 51, // Rule ID 1 // +// CHECK-NEXT: GIM_CheckSimplePredicate, GICXXPred_Simple_IsRule0Enabled, +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/0, TargetOpcode::G_TRUNC, +// CHECK-NEXT: // MIs[0] root +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // MIs[0] __anon_pat_match_0_0.z +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // Combiner Rule #0: Test0 @ [__anon_pat_match_0_0[1]] +// CHECK-NEXT: GIR_BuildMI, /*InsnID*/0, /*Opcode*/TargetOpcode::COPY, +// CHECK-NEXT: GIR_Copy, /*NewInsnID*/0, /*OldInsnID*/0, /*OpIdx*/0, // root +// CHECK-NEXT: GIR_AddCImm, /*InsnID*/0, /*Type*/GILLT_s32, /*Imm*/0, +// CHECK-NEXT: GIR_EraseFromParent, /*InsnID*/0, +// CHECK-NEXT: GIR_Done, +// CHECK-NEXT: // Label 1: @51 +// CHECK-NEXT: GIM_Try, /*On fail goto*//*Label 2*/ 72, // Rule ID 2 // +// CHECK-NEXT: GIM_CheckSimplePredicate, GICXXPred_Simple_IsRule0Enabled, +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/0, TargetOpcode::G_FPEXT, +// CHECK-NEXT: // MIs[0] root +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // MIs[0] __anon_pat_match_0_0.z +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // Combiner Rule #0: Test0 @ [__anon_pat_match_0_0[2]] +// CHECK-NEXT: GIR_BuildMI, /*InsnID*/0, /*Opcode*/TargetOpcode::COPY, +// CHECK-NEXT: GIR_Copy, /*NewInsnID*/0, /*OldInsnID*/0, /*OpIdx*/0, // root +// CHECK-NEXT: GIR_AddCImm, /*InsnID*/0, /*Type*/GILLT_s32, /*Imm*/0, +// CHECK-NEXT: GIR_EraseFromParent, /*InsnID*/0, +// CHECK-NEXT: GIR_Done, +// CHECK-NEXT: // Label 2: @72 +// CHECK-NEXT: GIM_Reject, +// CHECK-NEXT: }; +// CHECK-NEXT: return MatchTable0; +// CHECK-NEXT: } diff --git a/llvm/test/TableGen/GlobalISelCombinerMatchTableEmitter/match-table-permutations.td b/llvm/test/TableGen/GlobalISelCombinerMatchTableEmitter/match-table-permutations.td new file mode 100644 --- /dev/null +++ b/llvm/test/TableGen/GlobalISelCombinerMatchTableEmitter/match-table-permutations.td @@ -0,0 +1,513 @@ +// RUN: llvm-tblgen -I %p/../../../include -gen-global-isel-combiner-matchtable \ +// RUN: -combiners=MyCombiner -gicombiner-matchtable-debug-cxxpreds %s | \ +// RUN: FileCheck %s + +include "llvm/Target/Target.td" +include "llvm/Target/GlobalISel/Combine.td" + +def MyTargetISA : InstrInfo; +def MyTarget : Target { let InstructionSet = MyTargetISA; } + +def MatchFooPerms: GICombinePatFrag< + (outs root:$foo), + (ins gi_imm:$cst), + [ + (pattern (G_ZEXT $foo, $b), (G_TRUNC $b, $x):$dbg0, "return foo(${x}, ${cst})"), + (pattern (G_TRUNC $foo, $z):$dbg1, "return bar(${foo}, ${cst})") + ]>; + +def Test0 : GICombineRule< + (defs root:$dst), + (match (G_AND $dst, $cst0, $tmp), + (G_AND $tmp, $cst1, $cst2), + (MatchFooPerms $cst0, (i32 10)):$a, + (MatchFooPerms $cst1, (i32 20)):$b, + (MatchFooPerms $cst2, (i32 30)):$c + ), + (apply (COPY $dst, (i32 0)), "APPLY ${cst0}")>; + +def MyCombiner: GICombinerHelper<"GenMyCombiner", [ + Test0 +]>; + +// CHECK: bool GenMyCombiner::testMIPredicate_MI(unsigned PredicateID, const MachineInstr & MI, const MatcherState &State) const { +// CHECK-NEXT: switch (PredicateID) { +// CHECK-NEXT: case GICXXPred_MI_Predicate_GICombiner0: { +// CHECK-NEXT: // Pattern Alternatives: [a[0], b[0], c[0]] +// CHECK-NEXT: return foo(State.MIs[2]->getOperand(1), 10) +// CHECK-NEXT: llvm_unreachable("GICombiner0 should have returned"); +// CHECK-NEXT: } +// CHECK-NEXT: case GICXXPred_MI_Predicate_GICombiner1: { +// CHECK-NEXT: // Pattern Alternatives: [a[0], b[0], c[0]] +// CHECK-NEXT: return foo(State.MIs[5]->getOperand(1), 20) +// CHECK-NEXT: llvm_unreachable("GICombiner1 should have returned"); +// CHECK-NEXT: } +// CHECK-NEXT: case GICXXPred_MI_Predicate_GICombiner2: { +// CHECK-NEXT: // Pattern Alternatives: [a[0], b[0], c[0]] +// CHECK-NEXT: return foo(State.MIs[7]->getOperand(1), 30) +// CHECK-NEXT: llvm_unreachable("GICombiner2 should have returned"); +// CHECK-NEXT: } +// CHECK-NEXT: case GICXXPred_MI_Predicate_GICombiner3: { +// CHECK-NEXT: // Pattern Alternatives: [a[0], b[0], c[1]] +// CHECK-NEXT: return foo(State.MIs[2]->getOperand(1), 10) +// CHECK-NEXT: llvm_unreachable("GICombiner3 should have returned"); +// CHECK-NEXT: } +// CHECK-NEXT: case GICXXPred_MI_Predicate_GICombiner4: { +// CHECK-NEXT: // Pattern Alternatives: [a[0], b[0], c[1]] +// CHECK-NEXT: return foo(State.MIs[5]->getOperand(1), 20) +// CHECK-NEXT: llvm_unreachable("GICombiner4 should have returned"); +// CHECK-NEXT: } +// CHECK-NEXT: case GICXXPred_MI_Predicate_GICombiner5: { +// CHECK-NEXT: // Pattern Alternatives: [a[0], b[0], c[1]] +// CHECK-NEXT: return bar(State.MIs[3]->getOperand(2), 30) +// CHECK-NEXT: llvm_unreachable("GICombiner5 should have returned"); +// CHECK-NEXT: } +// CHECK-NEXT: case GICXXPred_MI_Predicate_GICombiner6: { +// CHECK-NEXT: // Pattern Alternatives: [a[0], b[1], c[0]] +// CHECK-NEXT: return foo(State.MIs[2]->getOperand(1), 10) +// CHECK-NEXT: llvm_unreachable("GICombiner6 should have returned"); +// CHECK-NEXT: } +// CHECK-NEXT: case GICXXPred_MI_Predicate_GICombiner7: { +// CHECK-NEXT: // Pattern Alternatives: [a[0], b[1], c[0]] +// CHECK-NEXT: return bar(State.MIs[3]->getOperand(1), 20) +// CHECK-NEXT: llvm_unreachable("GICombiner7 should have returned"); +// CHECK-NEXT: } +// CHECK-NEXT: case GICXXPred_MI_Predicate_GICombiner8: { +// CHECK-NEXT: // Pattern Alternatives: [a[0], b[1], c[0]] +// CHECK-NEXT: return foo(State.MIs[6]->getOperand(1), 30) +// CHECK-NEXT: llvm_unreachable("GICombiner8 should have returned"); +// CHECK-NEXT: } +// CHECK-NEXT: case GICXXPred_MI_Predicate_GICombiner9: { +// CHECK-NEXT: // Pattern Alternatives: [a[0], b[1], c[1]] +// CHECK-NEXT: return foo(State.MIs[2]->getOperand(1), 10) +// CHECK-NEXT: llvm_unreachable("GICombiner9 should have returned"); +// CHECK-NEXT: } +// CHECK-NEXT: case GICXXPred_MI_Predicate_GICombiner10: { +// CHECK-NEXT: // Pattern Alternatives: [a[0], b[1], c[1]] +// CHECK-NEXT: return bar(State.MIs[3]->getOperand(1), 20) +// CHECK-NEXT: llvm_unreachable("GICombiner10 should have returned"); +// CHECK-NEXT: } +// CHECK-NEXT: case GICXXPred_MI_Predicate_GICombiner11: { +// CHECK-NEXT: // Pattern Alternatives: [a[0], b[1], c[1]] +// CHECK-NEXT: return bar(State.MIs[3]->getOperand(2), 30) +// CHECK-NEXT: llvm_unreachable("GICombiner11 should have returned"); +// CHECK-NEXT: } +// CHECK-NEXT: case GICXXPred_MI_Predicate_GICombiner12: { +// CHECK-NEXT: // Pattern Alternatives: [a[1], b[0], c[0]] +// CHECK-NEXT: return bar(State.MIs[0]->getOperand(1), 10) +// CHECK-NEXT: llvm_unreachable("GICombiner12 should have returned"); +// CHECK-NEXT: } +// CHECK-NEXT: case GICXXPred_MI_Predicate_GICombiner13: { +// CHECK-NEXT: // Pattern Alternatives: [a[1], b[0], c[0]] +// CHECK-NEXT: return foo(State.MIs[4]->getOperand(1), 20) +// CHECK-NEXT: llvm_unreachable("GICombiner13 should have returned"); +// CHECK-NEXT: } +// CHECK-NEXT: case GICXXPred_MI_Predicate_GICombiner14: { +// CHECK-NEXT: // Pattern Alternatives: [a[1], b[0], c[0]] +// CHECK-NEXT: return foo(State.MIs[6]->getOperand(1), 30) +// CHECK-NEXT: llvm_unreachable("GICombiner14 should have returned"); +// CHECK-NEXT: } +// CHECK-NEXT: case GICXXPred_MI_Predicate_GICombiner15: { +// CHECK-NEXT: // Pattern Alternatives: [a[1], b[0], c[1]] +// CHECK-NEXT: return bar(State.MIs[0]->getOperand(1), 10) +// CHECK-NEXT: llvm_unreachable("GICombiner15 should have returned"); +// CHECK-NEXT: } +// CHECK-NEXT: case GICXXPred_MI_Predicate_GICombiner16: { +// CHECK-NEXT: // Pattern Alternatives: [a[1], b[0], c[1]] +// CHECK-NEXT: return foo(State.MIs[4]->getOperand(1), 20) +// CHECK-NEXT: llvm_unreachable("GICombiner16 should have returned"); +// CHECK-NEXT: } +// CHECK-NEXT: case GICXXPred_MI_Predicate_GICombiner17: { +// CHECK-NEXT: // Pattern Alternatives: [a[1], b[0], c[1]] +// CHECK-NEXT: return bar(State.MIs[3]->getOperand(2), 30) +// CHECK-NEXT: llvm_unreachable("GICombiner17 should have returned"); +// CHECK-NEXT: } +// CHECK-NEXT: case GICXXPred_MI_Predicate_GICombiner18: { +// CHECK-NEXT: // Pattern Alternatives: [a[1], b[1], c[0]] +// CHECK-NEXT: return bar(State.MIs[0]->getOperand(1), 10) +// CHECK-NEXT: llvm_unreachable("GICombiner18 should have returned"); +// CHECK-NEXT: } +// CHECK-NEXT: case GICXXPred_MI_Predicate_GICombiner19: { +// CHECK-NEXT: // Pattern Alternatives: [a[1], b[1], c[0]] +// CHECK-NEXT: return bar(State.MIs[3]->getOperand(1), 20) +// CHECK-NEXT: llvm_unreachable("GICombiner19 should have returned"); +// CHECK-NEXT: } +// CHECK-NEXT: case GICXXPred_MI_Predicate_GICombiner20: { +// CHECK-NEXT: // Pattern Alternatives: [a[1], b[1], c[0]] +// CHECK-NEXT: return foo(State.MIs[5]->getOperand(1), 30) +// CHECK-NEXT: llvm_unreachable("GICombiner20 should have returned"); +// CHECK-NEXT: } +// CHECK-NEXT: case GICXXPred_MI_Predicate_GICombiner21: { +// CHECK-NEXT: // Pattern Alternatives: [a[1], b[1], c[1]] +// CHECK-NEXT: return bar(State.MIs[0]->getOperand(1), 10) +// CHECK-NEXT: llvm_unreachable("GICombiner21 should have returned"); +// CHECK-NEXT: } +// CHECK-NEXT: case GICXXPred_MI_Predicate_GICombiner22: { +// CHECK-NEXT: // Pattern Alternatives: [a[1], b[1], c[1]] +// CHECK-NEXT: return bar(State.MIs[3]->getOperand(1), 20) +// CHECK-NEXT: llvm_unreachable("GICombiner22 should have returned"); +// CHECK-NEXT: } +// CHECK-NEXT: case GICXXPred_MI_Predicate_GICombiner23: { +// CHECK-NEXT: // Pattern Alternatives: [a[1], b[1], c[1]] +// CHECK-NEXT: return bar(State.MIs[3]->getOperand(2), 30) +// CHECK-NEXT: llvm_unreachable("GICombiner23 should have returned"); +// CHECK-NEXT: } +// CHECK-NEXT: } +// CHECK-NEXT: llvm_unreachable("Unknown predicate"); +// CHECK-NEXT: return false; +// CHECK-NEXT: } + +// CHECK: const int64_t *GenMyCombiner::getMatchTable() const { +// CHECK-NEXT: constexpr static int64_t MatchTable0[] = { +// CHECK-NEXT: GIM_Try, /*On fail goto*//*Label 0*/ 634, +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/0, TargetOpcode::G_AND, +// CHECK-NEXT: GIM_Try, /*On fail goto*//*Label 1*/ 97, // Rule ID 0 // +// CHECK-NEXT: GIM_CheckSimplePredicate, GICXXPred_Simple_IsRule0Enabled, +// CHECK-NEXT: // MIs[0] dst +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // MIs[0] cst0 +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/1, /*MI*/0, /*OpIdx*/1, // MIs[1] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/1, TargetOpcode::G_ZEXT, +// CHECK-NEXT: // MIs[1] a.b +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/2, /*MI*/1, /*OpIdx*/1, // MIs[2] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/2, TargetOpcode::G_TRUNC, +// CHECK-NEXT: // MIs[2] a.x +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // MIs[0] tmp +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/3, /*MI*/0, /*OpIdx*/2, // MIs[3] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/3, TargetOpcode::G_AND, +// CHECK-NEXT: // MIs[3] cst1 +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/4, /*MI*/3, /*OpIdx*/1, // MIs[4] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/4, TargetOpcode::G_ZEXT, +// CHECK-NEXT: // MIs[4] b.b +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/5, /*MI*/4, /*OpIdx*/1, // MIs[5] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/5, TargetOpcode::G_TRUNC, +// CHECK-NEXT: // MIs[5] b.x +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // MIs[3] cst2 +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/6, /*MI*/3, /*OpIdx*/2, // MIs[6] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/6, TargetOpcode::G_ZEXT, +// CHECK-NEXT: // MIs[6] c.b +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/7, /*MI*/6, /*OpIdx*/1, // MIs[7] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/7, TargetOpcode::G_TRUNC, +// CHECK-NEXT: // MIs[7] c.x +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: GIM_CheckCxxInsnPredicate, /*MI*/0, /*FnId*/GICXXPred_MI_Predicate_GICombiner0, +// CHECK-NEXT: GIM_CheckCxxInsnPredicate, /*MI*/0, /*FnId*/GICXXPred_MI_Predicate_GICombiner1, +// CHECK-NEXT: GIM_CheckCxxInsnPredicate, /*MI*/0, /*FnId*/GICXXPred_MI_Predicate_GICombiner2, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/1, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/2, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/3, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/4, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/5, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/6, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/7, +// CHECK-NEXT: // Combiner Rule #0: Test0 @ [a[0], b[0], c[0]] +// CHECK-NEXT: GIR_BuildMI, /*InsnID*/0, /*Opcode*/TargetOpcode::COPY, +// CHECK-NEXT: GIR_Copy, /*NewInsnID*/0, /*OldInsnID*/0, /*OpIdx*/0, // dst +// CHECK-NEXT: GIR_AddCImm, /*InsnID*/0, /*Type*/GILLT_s32, /*Imm*/0, +// CHECK-NEXT: GIR_EraseFromParent, /*InsnID*/0, +// CHECK-NEXT: GIR_CustomAction, GICXXCustomAction_CombineApplyGICombiner0, +// CHECK-NEXT: GIR_Done, +// CHECK-NEXT: // Label 1: @97 +// CHECK-NEXT: GIM_Try, /*On fail goto*//*Label 2*/ 180, // Rule ID 1 // +// CHECK-NEXT: GIM_CheckSimplePredicate, GICXXPred_Simple_IsRule0Enabled, +// CHECK-NEXT: // MIs[0] dst +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // MIs[0] cst0 +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/1, /*MI*/0, /*OpIdx*/1, // MIs[1] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/1, TargetOpcode::G_ZEXT, +// CHECK-NEXT: // MIs[1] a.b +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/2, /*MI*/1, /*OpIdx*/1, // MIs[2] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/2, TargetOpcode::G_TRUNC, +// CHECK-NEXT: // MIs[2] a.x +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // MIs[0] tmp +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/3, /*MI*/0, /*OpIdx*/2, // MIs[3] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/3, TargetOpcode::G_AND, +// CHECK-NEXT: // MIs[3] cst1 +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/4, /*MI*/3, /*OpIdx*/1, // MIs[4] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/4, TargetOpcode::G_ZEXT, +// CHECK-NEXT: // MIs[4] b.b +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/5, /*MI*/4, /*OpIdx*/1, // MIs[5] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/5, TargetOpcode::G_TRUNC, +// CHECK-NEXT: // MIs[5] b.x +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // MIs[3] cst2 +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/6, /*MI*/3, /*OpIdx*/2, // MIs[6] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/6, TargetOpcode::G_TRUNC, +// CHECK-NEXT: // MIs[6] c.z +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: GIM_CheckCxxInsnPredicate, /*MI*/0, /*FnId*/GICXXPred_MI_Predicate_GICombiner3, +// CHECK-NEXT: GIM_CheckCxxInsnPredicate, /*MI*/0, /*FnId*/GICXXPred_MI_Predicate_GICombiner4, +// CHECK-NEXT: GIM_CheckCxxInsnPredicate, /*MI*/0, /*FnId*/GICXXPred_MI_Predicate_GICombiner5, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/1, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/2, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/3, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/4, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/5, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/6, +// CHECK-NEXT: // Combiner Rule #0: Test0 @ [a[0], b[0], c[1]] +// CHECK-NEXT: GIR_BuildMI, /*InsnID*/0, /*Opcode*/TargetOpcode::COPY, +// CHECK-NEXT: GIR_Copy, /*NewInsnID*/0, /*OldInsnID*/0, /*OpIdx*/0, // dst +// CHECK-NEXT: GIR_AddCImm, /*InsnID*/0, /*Type*/GILLT_s32, /*Imm*/0, +// CHECK-NEXT: GIR_EraseFromParent, /*InsnID*/0, +// CHECK-NEXT: GIR_CustomAction, GICXXCustomAction_CombineApplyGICombiner0, +// CHECK-NEXT: GIR_Done, +// CHECK-NEXT: // Label 2: @180 +// CHECK-NEXT: GIM_Try, /*On fail goto*//*Label 3*/ 263, // Rule ID 2 // +// CHECK-NEXT: GIM_CheckSimplePredicate, GICXXPred_Simple_IsRule0Enabled, +// CHECK-NEXT: // MIs[0] dst +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // MIs[0] cst0 +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/1, /*MI*/0, /*OpIdx*/1, // MIs[1] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/1, TargetOpcode::G_ZEXT, +// CHECK-NEXT: // MIs[1] a.b +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/2, /*MI*/1, /*OpIdx*/1, // MIs[2] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/2, TargetOpcode::G_TRUNC, +// CHECK-NEXT: // MIs[2] a.x +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // MIs[0] tmp +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/3, /*MI*/0, /*OpIdx*/2, // MIs[3] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/3, TargetOpcode::G_AND, +// CHECK-NEXT: // MIs[3] cst1 +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/4, /*MI*/3, /*OpIdx*/1, // MIs[4] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/4, TargetOpcode::G_TRUNC, +// CHECK-NEXT: // MIs[4] b.z +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // MIs[3] cst2 +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/5, /*MI*/3, /*OpIdx*/2, // MIs[5] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/5, TargetOpcode::G_ZEXT, +// CHECK-NEXT: // MIs[5] c.b +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/6, /*MI*/5, /*OpIdx*/1, // MIs[6] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/6, TargetOpcode::G_TRUNC, +// CHECK-NEXT: // MIs[6] c.x +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: GIM_CheckCxxInsnPredicate, /*MI*/0, /*FnId*/GICXXPred_MI_Predicate_GICombiner6, +// CHECK-NEXT: GIM_CheckCxxInsnPredicate, /*MI*/0, /*FnId*/GICXXPred_MI_Predicate_GICombiner7, +// CHECK-NEXT: GIM_CheckCxxInsnPredicate, /*MI*/0, /*FnId*/GICXXPred_MI_Predicate_GICombiner8, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/1, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/2, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/3, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/4, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/5, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/6, +// CHECK-NEXT: // Combiner Rule #0: Test0 @ [a[0], b[1], c[0]] +// CHECK-NEXT: GIR_BuildMI, /*InsnID*/0, /*Opcode*/TargetOpcode::COPY, +// CHECK-NEXT: GIR_Copy, /*NewInsnID*/0, /*OldInsnID*/0, /*OpIdx*/0, // dst +// CHECK-NEXT: GIR_AddCImm, /*InsnID*/0, /*Type*/GILLT_s32, /*Imm*/0, +// CHECK-NEXT: GIR_EraseFromParent, /*InsnID*/0, +// CHECK-NEXT: GIR_CustomAction, GICXXCustomAction_CombineApplyGICombiner0, +// CHECK-NEXT: GIR_Done, +// CHECK-NEXT: // Label 3: @263 +// CHECK-NEXT: GIM_Try, /*On fail goto*//*Label 4*/ 337, // Rule ID 3 // +// CHECK-NEXT: GIM_CheckSimplePredicate, GICXXPred_Simple_IsRule0Enabled, +// CHECK-NEXT: // MIs[0] dst +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // MIs[0] cst0 +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/1, /*MI*/0, /*OpIdx*/1, // MIs[1] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/1, TargetOpcode::G_ZEXT, +// CHECK-NEXT: // MIs[1] a.b +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/2, /*MI*/1, /*OpIdx*/1, // MIs[2] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/2, TargetOpcode::G_TRUNC, +// CHECK-NEXT: // MIs[2] a.x +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // MIs[0] tmp +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/3, /*MI*/0, /*OpIdx*/2, // MIs[3] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/3, TargetOpcode::G_AND, +// CHECK-NEXT: // MIs[3] cst1 +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/4, /*MI*/3, /*OpIdx*/1, // MIs[4] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/4, TargetOpcode::G_TRUNC, +// CHECK-NEXT: // MIs[4] b.z +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // MIs[3] cst2 +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/5, /*MI*/3, /*OpIdx*/2, // MIs[5] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/5, TargetOpcode::G_TRUNC, +// CHECK-NEXT: // MIs[5] c.z +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: GIM_CheckCxxInsnPredicate, /*MI*/0, /*FnId*/GICXXPred_MI_Predicate_GICombiner9, +// CHECK-NEXT: GIM_CheckCxxInsnPredicate, /*MI*/0, /*FnId*/GICXXPred_MI_Predicate_GICombiner10, +// CHECK-NEXT: GIM_CheckCxxInsnPredicate, /*MI*/0, /*FnId*/GICXXPred_MI_Predicate_GICombiner11, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/1, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/2, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/3, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/4, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/5, +// CHECK-NEXT: // Combiner Rule #0: Test0 @ [a[0], b[1], c[1]] +// CHECK-NEXT: GIR_BuildMI, /*InsnID*/0, /*Opcode*/TargetOpcode::COPY, +// CHECK-NEXT: GIR_Copy, /*NewInsnID*/0, /*OldInsnID*/0, /*OpIdx*/0, // dst +// CHECK-NEXT: GIR_AddCImm, /*InsnID*/0, /*Type*/GILLT_s32, /*Imm*/0, +// CHECK-NEXT: GIR_EraseFromParent, /*InsnID*/0, +// CHECK-NEXT: GIR_CustomAction, GICXXCustomAction_CombineApplyGICombiner0, +// CHECK-NEXT: GIR_Done, +// CHECK-NEXT: // Label 4: @337 +// CHECK-NEXT: GIM_Try, /*On fail goto*//*Label 5*/ 420, // Rule ID 4 // +// CHECK-NEXT: GIM_CheckSimplePredicate, GICXXPred_Simple_IsRule0Enabled, +// CHECK-NEXT: // MIs[0] dst +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // MIs[0] cst0 +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/1, /*MI*/0, /*OpIdx*/1, // MIs[1] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/1, TargetOpcode::G_TRUNC, +// CHECK-NEXT: // MIs[1] a.z +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // MIs[0] tmp +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/2, /*MI*/0, /*OpIdx*/2, // MIs[2] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/2, TargetOpcode::G_AND, +// CHECK-NEXT: // MIs[2] cst1 +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/3, /*MI*/2, /*OpIdx*/1, // MIs[3] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/3, TargetOpcode::G_ZEXT, +// CHECK-NEXT: // MIs[3] b.b +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/4, /*MI*/3, /*OpIdx*/1, // MIs[4] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/4, TargetOpcode::G_TRUNC, +// CHECK-NEXT: // MIs[4] b.x +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // MIs[2] cst2 +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/5, /*MI*/2, /*OpIdx*/2, // MIs[5] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/5, TargetOpcode::G_ZEXT, +// CHECK-NEXT: // MIs[5] c.b +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/6, /*MI*/5, /*OpIdx*/1, // MIs[6] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/6, TargetOpcode::G_TRUNC, +// CHECK-NEXT: // MIs[6] c.x +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: GIM_CheckCxxInsnPredicate, /*MI*/0, /*FnId*/GICXXPred_MI_Predicate_GICombiner12, +// CHECK-NEXT: GIM_CheckCxxInsnPredicate, /*MI*/0, /*FnId*/GICXXPred_MI_Predicate_GICombiner13, +// CHECK-NEXT: GIM_CheckCxxInsnPredicate, /*MI*/0, /*FnId*/GICXXPred_MI_Predicate_GICombiner14, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/1, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/2, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/3, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/4, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/5, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/6, +// CHECK-NEXT: // Combiner Rule #0: Test0 @ [a[1], b[0], c[0]] +// CHECK-NEXT: GIR_BuildMI, /*InsnID*/0, /*Opcode*/TargetOpcode::COPY, +// CHECK-NEXT: GIR_Copy, /*NewInsnID*/0, /*OldInsnID*/0, /*OpIdx*/0, // dst +// CHECK-NEXT: GIR_AddCImm, /*InsnID*/0, /*Type*/GILLT_s32, /*Imm*/0, +// CHECK-NEXT: GIR_EraseFromParent, /*InsnID*/0, +// CHECK-NEXT: GIR_CustomAction, GICXXCustomAction_CombineApplyGICombiner0, +// CHECK-NEXT: GIR_Done, +// CHECK-NEXT: // Label 5: @420 +// CHECK-NEXT: GIM_Try, /*On fail goto*//*Label 6*/ 494, // Rule ID 5 // +// CHECK-NEXT: GIM_CheckSimplePredicate, GICXXPred_Simple_IsRule0Enabled, +// CHECK-NEXT: // MIs[0] dst +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // MIs[0] cst0 +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/1, /*MI*/0, /*OpIdx*/1, // MIs[1] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/1, TargetOpcode::G_TRUNC, +// CHECK-NEXT: // MIs[1] a.z +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // MIs[0] tmp +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/2, /*MI*/0, /*OpIdx*/2, // MIs[2] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/2, TargetOpcode::G_AND, +// CHECK-NEXT: // MIs[2] cst1 +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/3, /*MI*/2, /*OpIdx*/1, // MIs[3] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/3, TargetOpcode::G_ZEXT, +// CHECK-NEXT: // MIs[3] b.b +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/4, /*MI*/3, /*OpIdx*/1, // MIs[4] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/4, TargetOpcode::G_TRUNC, +// CHECK-NEXT: // MIs[4] b.x +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // MIs[2] cst2 +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/5, /*MI*/2, /*OpIdx*/2, // MIs[5] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/5, TargetOpcode::G_TRUNC, +// CHECK-NEXT: // MIs[5] c.z +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: GIM_CheckCxxInsnPredicate, /*MI*/0, /*FnId*/GICXXPred_MI_Predicate_GICombiner15, +// CHECK-NEXT: GIM_CheckCxxInsnPredicate, /*MI*/0, /*FnId*/GICXXPred_MI_Predicate_GICombiner16, +// CHECK-NEXT: GIM_CheckCxxInsnPredicate, /*MI*/0, /*FnId*/GICXXPred_MI_Predicate_GICombiner17, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/1, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/2, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/3, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/4, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/5, +// CHECK-NEXT: // Combiner Rule #0: Test0 @ [a[1], b[0], c[1]] +// CHECK-NEXT: GIR_BuildMI, /*InsnID*/0, /*Opcode*/TargetOpcode::COPY, +// CHECK-NEXT: GIR_Copy, /*NewInsnID*/0, /*OldInsnID*/0, /*OpIdx*/0, // dst +// CHECK-NEXT: GIR_AddCImm, /*InsnID*/0, /*Type*/GILLT_s32, /*Imm*/0, +// CHECK-NEXT: GIR_EraseFromParent, /*InsnID*/0, +// CHECK-NEXT: GIR_CustomAction, GICXXCustomAction_CombineApplyGICombiner0, +// CHECK-NEXT: GIR_Done, +// CHECK-NEXT: // Label 6: @494 +// CHECK-NEXT: GIM_Try, /*On fail goto*//*Label 7*/ 568, // Rule ID 6 // +// CHECK-NEXT: GIM_CheckSimplePredicate, GICXXPred_Simple_IsRule0Enabled, +// CHECK-NEXT: // MIs[0] dst +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // MIs[0] cst0 +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/1, /*MI*/0, /*OpIdx*/1, // MIs[1] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/1, TargetOpcode::G_TRUNC, +// CHECK-NEXT: // MIs[1] a.z +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // MIs[0] tmp +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/2, /*MI*/0, /*OpIdx*/2, // MIs[2] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/2, TargetOpcode::G_AND, +// CHECK-NEXT: // MIs[2] cst1 +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/3, /*MI*/2, /*OpIdx*/1, // MIs[3] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/3, TargetOpcode::G_TRUNC, +// CHECK-NEXT: // MIs[3] b.z +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // MIs[2] cst2 +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/4, /*MI*/2, /*OpIdx*/2, // MIs[4] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/4, TargetOpcode::G_ZEXT, +// CHECK-NEXT: // MIs[4] c.b +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/5, /*MI*/4, /*OpIdx*/1, // MIs[5] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/5, TargetOpcode::G_TRUNC, +// CHECK-NEXT: // MIs[5] c.x +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: GIM_CheckCxxInsnPredicate, /*MI*/0, /*FnId*/GICXXPred_MI_Predicate_GICombiner18, +// CHECK-NEXT: GIM_CheckCxxInsnPredicate, /*MI*/0, /*FnId*/GICXXPred_MI_Predicate_GICombiner19, +// CHECK-NEXT: GIM_CheckCxxInsnPredicate, /*MI*/0, /*FnId*/GICXXPred_MI_Predicate_GICombiner20, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/1, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/2, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/3, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/4, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/5, +// CHECK-NEXT: // Combiner Rule #0: Test0 @ [a[1], b[1], c[0]] +// CHECK-NEXT: GIR_BuildMI, /*InsnID*/0, /*Opcode*/TargetOpcode::COPY, +// CHECK-NEXT: GIR_Copy, /*NewInsnID*/0, /*OldInsnID*/0, /*OpIdx*/0, // dst +// CHECK-NEXT: GIR_AddCImm, /*InsnID*/0, /*Type*/GILLT_s32, /*Imm*/0, +// CHECK-NEXT: GIR_EraseFromParent, /*InsnID*/0, +// CHECK-NEXT: GIR_CustomAction, GICXXCustomAction_CombineApplyGICombiner0, +// CHECK-NEXT: GIR_Done, +// CHECK-NEXT: // Label 7: @568 +// CHECK-NEXT: GIM_Try, /*On fail goto*//*Label 8*/ 633, // Rule ID 7 // +// CHECK-NEXT: GIM_CheckSimplePredicate, GICXXPred_Simple_IsRule0Enabled, +// CHECK-NEXT: // MIs[0] dst +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // MIs[0] cst0 +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/1, /*MI*/0, /*OpIdx*/1, // MIs[1] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/1, TargetOpcode::G_TRUNC, +// CHECK-NEXT: // MIs[1] a.z +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // MIs[0] tmp +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/2, /*MI*/0, /*OpIdx*/2, // MIs[2] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/2, TargetOpcode::G_AND, +// CHECK-NEXT: // MIs[2] cst1 +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/3, /*MI*/2, /*OpIdx*/1, // MIs[3] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/3, TargetOpcode::G_TRUNC, +// CHECK-NEXT: // MIs[3] b.z +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // MIs[2] cst2 +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/4, /*MI*/2, /*OpIdx*/2, // MIs[4] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/4, TargetOpcode::G_TRUNC, +// CHECK-NEXT: // MIs[4] c.z +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: GIM_CheckCxxInsnPredicate, /*MI*/0, /*FnId*/GICXXPred_MI_Predicate_GICombiner21, +// CHECK-NEXT: GIM_CheckCxxInsnPredicate, /*MI*/0, /*FnId*/GICXXPred_MI_Predicate_GICombiner22, +// CHECK-NEXT: GIM_CheckCxxInsnPredicate, /*MI*/0, /*FnId*/GICXXPred_MI_Predicate_GICombiner23, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/1, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/2, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/3, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/4, +// CHECK-NEXT: // Combiner Rule #0: Test0 @ [a[1], b[1], c[1]] +// CHECK-NEXT: GIR_BuildMI, /*InsnID*/0, /*Opcode*/TargetOpcode::COPY, +// CHECK-NEXT: GIR_Copy, /*NewInsnID*/0, /*OldInsnID*/0, /*OpIdx*/0, // dst +// CHECK-NEXT: GIR_AddCImm, /*InsnID*/0, /*Type*/GILLT_s32, /*Imm*/0, +// CHECK-NEXT: GIR_EraseFromParent, /*InsnID*/0, +// CHECK-NEXT: GIR_CustomAction, GICXXCustomAction_CombineApplyGICombiner0, +// CHECK-NEXT: GIR_Done, +// CHECK-NEXT: // Label 8: @633 +// CHECK-NEXT: GIM_Reject, +// CHECK-NEXT: // Label 0: @634 +// CHECK-NEXT: GIM_Reject, +// CHECK-NEXT: }; +// CHECK-NEXT: return MatchTable0; +// CHECK-NEXT: } diff --git a/llvm/test/TableGen/GlobalISelCombinerMatchTableEmitter/match-table-variadics.td b/llvm/test/TableGen/GlobalISelCombinerMatchTableEmitter/match-table-variadics.td new file mode 100644 --- /dev/null +++ b/llvm/test/TableGen/GlobalISelCombinerMatchTableEmitter/match-table-variadics.td @@ -0,0 +1,103 @@ +// RUN: llvm-tblgen -I %p/../../../include -gen-global-isel-combiner-matchtable \ +// RUN: -combiners=MyCombiner %s | \ +// RUN: FileCheck %s + +include "llvm/Target/Target.td" +include "llvm/Target/GlobalISel/Combine.td" + +def MyTargetISA : InstrInfo; +def MyTarget : Target { let InstructionSet = MyTargetISA; } + +def InstTest0 : GICombineRule< + (defs root:$a), + (match (G_BUILD_VECTOR $a, $b, $c, $d)), + (apply [{ APPLY }])>; + +def InstTest1 : GICombineRule< + (defs root:$a), + (match (G_BUILD_VECTOR $a, $b)), + (apply [{ APPLY }])>; + +def InstTest2 : GICombineRule< + (defs root:$a), + (match (G_UNMERGE_VALUES $a, $b)), + (apply [{ APPLY }])>; + +def InstTest3 : GICombineRule< + (defs root:$a), + (match (G_UNMERGE_VALUES $a, $b, $c, $d)), + (apply [{ APPLY }])>; + +def MyCombiner: GICombinerHelper<"GenMyCombiner", [ + InstTest0, + InstTest1, + InstTest2, + InstTest3 +]>; + +// CHECK: const int64_t *GenMyCombiner::getMatchTable() const { +// CHECK-NEXT: constexpr static int64_t MatchTable0[] = { +// CHECK-NEXT: GIM_Try, /*On fail goto*//*Label 0*/ 26, +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/0, TargetOpcode::G_BUILD_VECTOR, +// CHECK-NEXT: GIM_Try, /*On fail goto*//*Label 1*/ 15, // Rule ID 1 // +// CHECK-NEXT: GIM_CheckSimplePredicate, GICXXPred_Simple_IsRule1Enabled, +// CHECK-NEXT: GIM_CheckNumOperands, /*MI*/0, /*Expected*/2, +// CHECK-NEXT: // MIs[0] a +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // MIs[0] b +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // Combiner Rule #1: InstTest1 +// CHECK-NEXT: GIR_CustomAction, GICXXCustomAction_CombineApplyGICombiner0, +// CHECK-NEXT: GIR_Done, +// CHECK-NEXT: // Label 1: @15 +// CHECK-NEXT: GIM_Try, /*On fail goto*//*Label 2*/ 25, // Rule ID 0 // +// CHECK-NEXT: GIM_CheckSimplePredicate, GICXXPred_Simple_IsRule0Enabled, +// CHECK-NEXT: GIM_CheckNumOperands, /*MI*/0, /*Expected*/4, +// CHECK-NEXT: // MIs[0] a +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // MIs[0] b +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // MIs[0] c +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // MIs[0] d +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // Combiner Rule #0: InstTest0 +// CHECK-NEXT: GIR_CustomAction, GICXXCustomAction_CombineApplyGICombiner0, +// CHECK-NEXT: GIR_Done, +// CHECK-NEXT: // Label 2: @25 +// CHECK-NEXT: GIM_Reject, +// CHECK-NEXT: // Label 0: @26 +// CHECK-NEXT: GIM_Try, /*On fail goto*//*Label 3*/ 52, +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/0, TargetOpcode::G_UNMERGE_VALUES, +// CHECK-NEXT: GIM_Try, /*On fail goto*//*Label 4*/ 41, // Rule ID 2 // +// CHECK-NEXT: GIM_CheckSimplePredicate, GICXXPred_Simple_IsRule2Enabled, +// CHECK-NEXT: GIM_CheckNumOperands, /*MI*/0, /*Expected*/2, +// CHECK-NEXT: // MIs[0] a +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // MIs[0] b +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // Combiner Rule #2: InstTest2 +// CHECK-NEXT: GIR_CustomAction, GICXXCustomAction_CombineApplyGICombiner0, +// CHECK-NEXT: GIR_Done, +// CHECK-NEXT: // Label 4: @41 +// CHECK-NEXT: GIM_Try, /*On fail goto*//*Label 5*/ 51, // Rule ID 3 // +// CHECK-NEXT: GIM_CheckSimplePredicate, GICXXPred_Simple_IsRule3Enabled, +// CHECK-NEXT: GIM_CheckNumOperands, /*MI*/0, /*Expected*/4, +// CHECK-NEXT: // MIs[0] a +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // MIs[0] b +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // MIs[0] c +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // MIs[0] d +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // Combiner Rule #3: InstTest3 +// CHECK-NEXT: GIR_CustomAction, GICXXCustomAction_CombineApplyGICombiner0, +// CHECK-NEXT: GIR_Done, +// CHECK-NEXT: // Label 5: @51 +// CHECK-NEXT: GIM_Reject, +// CHECK-NEXT: // Label 3: @52 +// CHECK-NEXT: GIM_Reject, +// CHECK-NEXT: }; +// CHECK-NEXT: return MatchTable0; +// CHECK-NEXT: } diff --git a/llvm/test/TableGen/GlobalISelCombinerMatchTableEmitter/match-table.td b/llvm/test/TableGen/GlobalISelCombinerMatchTableEmitter/match-table.td --- a/llvm/test/TableGen/GlobalISelCombinerMatchTableEmitter/match-table.td +++ b/llvm/test/TableGen/GlobalISelCombinerMatchTableEmitter/match-table.td @@ -10,52 +10,68 @@ def dummy; -def R0 : Register<"r0"> { let Namespace = "MyTarget"; } -def GPR32 : RegisterClass<"MyTarget", [i32], 32, (add R0)>; -class I Pat> - : Instruction { - let Namespace = "MyTarget"; - let OutOperandList = OOps; - let InOperandList = IOps; - let Pattern = Pat; -} -def MOV : I<(outs GPR32:$dst), (ins GPR32:$src1), []>; -def TRUNC : I<(outs GPR32:$dst), (ins GPR32:$src1), []>; -def ZEXT : I<(outs GPR32:$dst), (ins GPR32:$src1), []>; -def SEXT : I<(outs GPR32:$dst), (ins GPR32:$src1), []>; - def HasAnswerToEverything : Predicate<"Subtarget->getAnswerToUniverse() == 42 && Subtarget->getAnswerToLife() == 42">; def WipOpcodeTest0 : GICombineRule< (defs root:$d), - (match (wip_match_opcode TRUNC):$d), + (match (wip_match_opcode G_TRUNC):$d), (apply [{ APPLY }])>; def WipOpcodeTest1 : GICombineRule< (defs root:$d), - (match (wip_match_opcode TRUNC, SEXT):$d), + (match (wip_match_opcode G_TRUNC, G_SEXT):$d), (apply [{ APPLY }])>; // Note: also checks that spaces in the type name are removed. def reg_matchinfo : GIDefMatchData<"Register ">; def InstTest0 : GICombineRule< (defs root:$d, reg_matchinfo:$r0, reg_matchinfo:$r1), - (match (MOV $a, $b):$d), + (match (COPY $a, $b):$d), (apply [{ APPLY ${r0}, ${r1} }])>; let Predicates = [HasAnswerToEverything] in def InstTest1 : GICombineRule< (defs root:$d, reg_matchinfo:$r0), - (match (MOV $a, $b):$d, - (ZEXT $b, $c), + (match (COPY $a, $b):$d, + (G_ZEXT $b, $c), [{ return CHECK ${a}, ${b}, ${c}, ${d} }]), (apply [{ APPLY }])>; +def InOutInstTest0 : GICombineRule< + (defs root:$root), + (match (G_ZEXT $tmp, $ext), + (G_STORE $tmp, $ptr):$root), + (apply (G_STORE $ext, $ptr):$root, "APPLY ${ext} ${ptr} ${root}")>; + +// Imm operand of G_CONSTANT should match a literal int, while the second +// should match a constant. +def InOutInstTest1 : GICombineRule< + (defs root:$dst), + (match + (G_CONSTANT $x, -42:$z), + (G_AND $dst, $x, (i32 43))), + (apply (G_TRUNC $dst, $z))>; + +def MatchICst: GICombinePatFrag< + (outs), + (ins gi_mo:$foo, gi_imm:$cst), + [(pattern "return matchIConstant(${foo}, ${cst});")]>; + +def PatFragTest0 : GICombineRule< + (defs root:$dst), + (match (G_ZEXT $dst, $cst), (MatchICst $cst, (i32 0))), + (apply (COPY $dst, (i32 0)))>; + +// TODO: add test with temp reg use + def MyCombiner: GICombinerHelper<"GenMyCombiner", [ WipOpcodeTest0, WipOpcodeTest1, InstTest0, - InstTest1 + InstTest1, + InOutInstTest0, + InOutInstTest1, + PatFragTest0 ]>; // We have at most 2 registers used by one rule at a time, so we should only have 2 registers MDInfos. @@ -69,6 +85,9 @@ // CHECK-NEXT: case GICXXPred_MI_Predicate_GICombiner0: { // CHECK-NEXT: return CHECK State.MIs[0]->getOperand(0), State.MIs[0]->getOperand(1), State.MIs[1]->getOperand(1), State.MIs[0] // CHECK-NEXT: } +// CHECK-NEXT: case GICXXPred_MI_Predicate_GICombiner1: { +// CHECK-NEXT: return matchIConstant(State.MIs[0]->getOperand(1), 0); +// CHECK-NEXT: } // Verify we reset MatchData on each tryCombineAll // CHECK: bool GenMyCombiner::tryCombineAll(MachineInstr &I) const { @@ -79,28 +98,51 @@ // CHECK-NEXT: State.MIs.push_back(&I); // CHECK-NEXT: MatchInfos = MatchInfosTy(); // CHECK-EMPTY: -// CHECK-NEXT: if (executeMatchTable(*this, OutMIs, State, ExecInfo, getMatchTable(), *ST.getInstrInfo(), MRI, *MRI.getTargetRegisterInfo(), *ST.getRegBankInfo(), AvailableFeatures, /*CoverageInfo*/ nullptr)) +// CHECK-NEXT: if (executeMatchTable(*this, OutMIs, State, ExecInfo, getMatchTable(), *ST.getInstrInfo(), MRI, *MRI.getTargetRegisterInfo(), *ST.getRegBankInfo(), AvailableFeatures, /*CoverageInfo*/ nullptr, &Observer)) // CHECK-NEXT: return true; // CHECK-NEXT: } // CHECK-EMPTY: // CHECK-NEXT: return false; // CHECK-NEXT: } +// Check apply +// CHECK: enum { +// CHECK-NEXT: GICXXCustomAction_CombineApplyGICombiner0 = GICXXCustomAction_Invalid + 1, +// CHECK-NEXT: GICXXCustomAction_CombineApplyGICombiner1, +// CHECK-NEXT: GICXXCustomAction_CombineApplyGICombiner2, +// CHECK-NEXT: }; +// CHECK-NEXT: void GenMyCombiner::runCustomAction(unsigned ApplyID, const MatcherState &State, NewMIVector &OutMIs) const { +// CHECK-NEXT: switch(ApplyID) { +// CHECK-NEXT: case GICXXCustomAction_CombineApplyGICombiner0:{ +// CHECK-NEXT: APPLY +// CHECK-NEXT: return; +// CHECK-NEXT: } +// CHECK-NEXT: case GICXXCustomAction_CombineApplyGICombiner1:{ +// CHECK-NEXT: APPLY MatchInfos.MDInfo0, MatchInfos.MDInfo1 +// CHECK-NEXT: return; +// CHECK-NEXT: } +// CHECK-NEXT: case GICXXCustomAction_CombineApplyGICombiner2:{ +// CHECK-NEXT: APPLY State.MIs[1]->getOperand(1) State.MIs[0]->getOperand(1) OutMIs[0] +// CHECK-NEXT: return; +// CHECK-NEXT: } +// CHECK-NEXT: } +// CHECK-NEXT: llvm_unreachable("Unknown Apply Action"); +// CHECK-NEXT: } // Verify match table. // CHECK: const int64_t *GenMyCombiner::getMatchTable() const { // CHECK-NEXT: constexpr static int64_t MatchTable0[] = { // CHECK-NEXT: GIM_Try, /*On fail goto*//*Label 0*/ 20, -// CHECK-NEXT: GIM_CheckOpcode, /*MI*/0, MyTarget::TRUNC, +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/0, TargetOpcode::G_TRUNC, // CHECK-NEXT: GIM_Try, /*On fail goto*//*Label 1*/ 12, // Rule ID 0 // // CHECK-NEXT: GIM_CheckSimplePredicate, GICXXPred_Simple_IsRule0Enabled, -// CHECK-NEXT: // Combiner Rule #0: WipOpcodeTest0; wip_match_opcode alternative 'TRUNC' +// CHECK-NEXT: // Combiner Rule #0: WipOpcodeTest0; wip_match_opcode 'G_TRUNC' // CHECK-NEXT: GIR_CustomAction, GICXXCustomAction_CombineApplyGICombiner0, // CHECK-NEXT: GIR_Done, // CHECK-NEXT: // Label 1: @12 // CHECK-NEXT: GIM_Try, /*On fail goto*//*Label 2*/ 19, // Rule ID 1 // // CHECK-NEXT: GIM_CheckSimplePredicate, GICXXPred_Simple_IsRule1Enabled, -// CHECK-NEXT: // Combiner Rule #1: WipOpcodeTest1; wip_match_opcode alternative 'TRUNC' +// CHECK-NEXT: // Combiner Rule #1: WipOpcodeTest1; wip_match_opcode 'G_TRUNC' // CHECK-NEXT: GIR_CustomAction, GICXXCustomAction_CombineApplyGICombiner0, // CHECK-NEXT: GIR_Done, // CHECK-NEXT: // Label 2: @19 @@ -108,13 +150,13 @@ // CHECK-NEXT: // Label 0: @20 // CHECK-NEXT: GIM_Try, /*On fail goto*//*Label 3*/ 30, // Rule ID 2 // // CHECK-NEXT: GIM_CheckSimplePredicate, GICXXPred_Simple_IsRule1Enabled, -// CHECK-NEXT: GIM_CheckOpcode, /*MI*/0, MyTarget::SEXT, -// CHECK-NEXT: // Combiner Rule #1: WipOpcodeTest1; wip_match_opcode alternative 'SEXT' +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/0, TargetOpcode::G_SEXT, +// CHECK-NEXT: // Combiner Rule #1: WipOpcodeTest1; wip_match_opcode 'G_SEXT' // CHECK-NEXT: GIR_CustomAction, GICXXCustomAction_CombineApplyGICombiner0, // CHECK-NEXT: GIR_Done, // CHECK-NEXT: // Label 3: @30 // CHECK-NEXT: GIM_Try, /*On fail goto*//*Label 4*/ 64, -// CHECK-NEXT: GIM_CheckOpcode, /*MI*/0, MyTarget::MOV, +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/0, TargetOpcode::COPY, // CHECK-NEXT: GIM_Try, /*On fail goto*//*Label 5*/ 42, // Rule ID 3 // // CHECK-NEXT: GIM_CheckSimplePredicate, GICXXPred_Simple_IsRule2Enabled, // CHECK-NEXT: // MIs[0] a @@ -131,8 +173,8 @@ // CHECK-NEXT: // MIs[0] a // CHECK-NEXT: // No operand predicates // CHECK-NEXT: // MIs[0] b -// CHECK-NEXT: GIM_RecordInsn, /*DefineMI*/1, /*MI*/0, /*OpIdx*/1, // MIs[1] -// CHECK-NEXT: GIM_CheckOpcode, /*MI*/1, MyTarget::ZEXT, +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/1, /*MI*/0, /*OpIdx*/1, // MIs[1] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/1, TargetOpcode::G_ZEXT, // CHECK-NEXT: // MIs[1] c // CHECK-NEXT: // No operand predicates // CHECK-NEXT: GIM_CheckCxxInsnPredicate, /*MI*/0, /*FnId*/GICXXPred_MI_Predicate_GICombiner0, @@ -143,6 +185,61 @@ // CHECK-NEXT: // Label 6: @63 // CHECK-NEXT: GIM_Reject, // CHECK-NEXT: // Label 4: @64 +// CHECK-NEXT: GIM_Try, /*On fail goto*//*Label 7*/ 101, // Rule ID 5 // +// CHECK-NEXT: GIM_CheckSimplePredicate, GICXXPred_Simple_IsRule4Enabled, +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/0, TargetOpcode::G_STORE, +// CHECK-NEXT: // MIs[0] tmp +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/1, /*MI*/0, /*OpIdx*/0, // MIs[1] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/1, TargetOpcode::G_ZEXT, +// CHECK-NEXT: // MIs[1] ext +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // MIs[0] ptr +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/1, +// CHECK-NEXT: // Combiner Rule #4: InOutInstTest0 +// CHECK-NEXT: GIR_BuildMI, /*InsnID*/0, /*Opcode*/TargetOpcode::G_STORE, +// CHECK-NEXT: GIR_Copy, /*NewInsnID*/0, /*OldInsnID*/1, /*OpIdx*/1, // ext +// CHECK-NEXT: GIR_Copy, /*NewInsnID*/0, /*OldInsnID*/0, /*OpIdx*/1, // ptr +// CHECK-NEXT: GIR_MergeMemOperands, /*InsnID*/0, /*MergeInsnID's*/0, 1, GIU_MergeMemOperands_EndOfList, +// CHECK-NEXT: GIR_EraseFromParent, /*InsnID*/0, +// CHECK-NEXT: GIR_CustomAction, GICXXCustomAction_CombineApplyGICombiner2, +// CHECK-NEXT: GIR_Done, +// CHECK-NEXT: // Label 7: @101 +// CHECK-NEXT: GIM_Try, /*On fail goto*//*Label 8*/ 143, // Rule ID 6 // +// CHECK-NEXT: GIM_CheckSimplePredicate, GICXXPred_Simple_IsRule5Enabled, +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/0, TargetOpcode::G_AND, +// CHECK-NEXT: GIM_CheckType, /*MI*/0, /*Op*/2, /*Type*/GILLT_s32, +// CHECK-NEXT: // MIs[0] dst +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // MIs[0] x +// CHECK-NEXT: GIM_RecordInsnIgnoreCopies, /*DefineMI*/1, /*MI*/0, /*OpIdx*/1, // MIs[1] +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/1, TargetOpcode::G_CONSTANT, +// CHECK-NEXT: // MIs[1] z +// CHECK-NEXT: GIM_CheckLiteralInt, /*MI*/1, /*Op*/1, -42, +// CHECK-NEXT: GIM_CheckConstantInt, /*MI*/0, /*Op*/2, 43, +// CHECK-NEXT: GIM_CheckIsSafeToFold, /*InsnID*/1, +// CHECK-NEXT: // Combiner Rule #5: InOutInstTest1 +// CHECK-NEXT: GIR_BuildMI, /*InsnID*/0, /*Opcode*/TargetOpcode::G_TRUNC, +// CHECK-NEXT: GIR_Copy, /*NewInsnID*/0, /*OldInsnID*/0, /*OpIdx*/0, // dst +// CHECK-NEXT: GIR_Copy, /*NewInsnID*/0, /*OldInsnID*/1, /*OpIdx*/1, // z +// CHECK-NEXT: GIR_EraseFromParent, /*InsnID*/0, +// CHECK-NEXT: GIR_Done, +// CHECK-NEXT: // Label 8: @143 +// CHECK-NEXT: GIM_Try, /*On fail goto*//*Label 9*/ 167, // Rule ID 7 // +// CHECK-NEXT: GIM_CheckSimplePredicate, GICXXPred_Simple_IsRule6Enabled, +// CHECK-NEXT: GIM_CheckOpcode, /*MI*/0, TargetOpcode::G_ZEXT, +// CHECK-NEXT: // MIs[0] dst +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: // MIs[0] cst +// CHECK-NEXT: // No operand predicates +// CHECK-NEXT: GIM_CheckCxxInsnPredicate, /*MI*/0, /*FnId*/GICXXPred_MI_Predicate_GICombiner1, +// CHECK-NEXT: // Combiner Rule #6: PatFragTest0 @ [__anon_pat_match_6_1[0]] +// CHECK-NEXT: GIR_BuildMI, /*InsnID*/0, /*Opcode*/TargetOpcode::COPY, +// CHECK-NEXT: GIR_Copy, /*NewInsnID*/0, /*OldInsnID*/0, /*OpIdx*/0, // dst +// CHECK-NEXT: GIR_AddCImm, /*InsnID*/0, /*Type*/GILLT_s32, /*Imm*/0, +// CHECK-NEXT: GIR_EraseFromParent, /*InsnID*/0, +// CHECK-NEXT: GIR_Done, +// CHECK-NEXT: // Label 9: @167 // CHECK-NEXT: GIM_Reject, // CHECK-NEXT: }; // CHECK-NEXT: return MatchTable0; diff --git a/llvm/test/TableGen/GlobalISelCombinerMatchTableEmitter/operand-types.td b/llvm/test/TableGen/GlobalISelCombinerMatchTableEmitter/operand-types.td new file mode 100644 --- /dev/null +++ b/llvm/test/TableGen/GlobalISelCombinerMatchTableEmitter/operand-types.td @@ -0,0 +1,77 @@ +// RUN: llvm-tblgen -I %p/../../../include -gen-global-isel-combiner-matchtable \ +// RUN: -gicombiner-stop-after-parse -combiners=MyCombiner %s | \ +// RUN: FileCheck %s + +include "llvm/Target/Target.td" +include "llvm/Target/GlobalISel/Combine.td" + +def MyTargetISA : InstrInfo; +def MyTarget : Target { let InstructionSet = MyTargetISA; } + +// CHECK: (CombineRule name:InstTest0 id:0 root:x +// CHECK-NEXT: (MatchPats +// CHECK-NEXT: __anon_pat_match_0_0:(CodeGenInstructionPattern G_BUILD_VECTOR operands:[i64:$z, i32:$y, i16:$w]) +// CHECK-NEXT: __anon_pat_match_0_1:(CodeGenInstructionPattern G_BUILD_VECTOR operands:[i64:$x, i32:$y, i64:$z]) +// CHECK-NEXT: ) +// CHECK-NEXT: (ApplyPats +// CHECK-NEXT: __anon_pat_apply_0_2:(CodeGenInstructionPattern G_MUL operands:[i64:$x, i32:$y, i16:$w]) +// CHECK-NEXT: ) +// CHECK-NEXT: (OperandTable +// CHECK-NEXT: [z match_pat:__anon_pat_match_0_0] +// CHECK-NEXT: [y match-live-in] +// CHECK-NEXT: [w match-live-in] +// CHECK-NEXT: [x match_pat:__anon_pat_match_0_1 apply_pat:__anon_pat_apply_0_2] +// CHECK-NEXT: ) +// CHECK-NEXT: ) +def InstTest0 : GICombineRule< + (defs root:$x), + (match + (G_BUILD_VECTOR i64:$z, $y, $w), + (G_BUILD_VECTOR $x, i32:$y, $z)), + (apply (G_MUL i64:$x, $y, i16:$w))>; + +// CHECK: (CombineRule name:PatFragTest0 id:1 root:dst +// CHECK-NEXT: (PatFrags +// CHECK-NEXT: (PatFrag name:FooPF +// CHECK-NEXT: (outs [cst:root]) +// CHECK-NEXT: (alternatives [ +// CHECK-NEXT: [ +// CHECK-NEXT: (CodeGenInstructionPattern name:__anon_pat_pattern_1_1 G_BUILD_VECTOR operands:[i64:$cst, i8:$y, $x]), +// CHECK-NEXT: (CodeGenInstructionPattern name:__anon_pat_pattern_1_2 G_BUILD_VECTOR operands:[$x, i8:$y, $w]), +// CHECK-NEXT: ], +// CHECK-NEXT: [ +// CHECK-NEXT: (CodeGenInstructionPattern name:__anon_pat_pattern_1_3 G_BUILD_VECTOR operands:[i32:$cst, i32:$y, i16:$x]), +// CHECK-NEXT: (CodeGenInstructionPattern name:__anon_pat_pattern_1_4 G_BUILD_VECTOR operands:[i16:$x, i32:$y, $w]), +// CHECK-NEXT: ], +// CHECK-NEXT: ]) +// CHECK-NEXT: ) +// CHECK-NEXT: ) +// CHECK-NEXT: (MatchPats +// CHECK-NEXT: __anon_pat_match_1_0:(PatFragPattern FooPF operands:[$dst]) +// CHECK-NEXT: ) +// CHECK-NEXT: (ApplyPats +// CHECK-NEXT: __anon_pat_apply_1_5:(CodeGenInstructionPattern COPY operands:[$dst, (i32 0)]) +// CHECK-NEXT: ) +// CHECK-NEXT: (OperandTable +// CHECK-NEXT: [dst match_pat:__anon_pat_match_1_0 apply_pat:__anon_pat_apply_1_5] +// CHECK-NEXT: ) +// CHECK-NEXT: (PermutationsToEmit +// CHECK-NEXT: [__anon_pat_match_1_0[0]], +// CHECK-NEXT: [__anon_pat_match_1_0[1]], +// CHECK-NEXT: ) +// CHECK-NEXT: ) +def FooPF: GICombinePatFrag< + (outs root:$cst),(ins), + [ + (pattern (G_BUILD_VECTOR i64:$cst, $y, $x), (G_BUILD_VECTOR $x, i8:$y, $w)), + (pattern (G_BUILD_VECTOR i32:$cst, $y, i16:$x), (G_BUILD_VECTOR $x, i32:$y, $w)), + ]>; +def PatFragTest0 : GICombineRule< + (defs root:$dst), + (match (FooPF $dst)), + (apply (COPY $dst, (i32 0)))>; + +def MyCombiner: GICombinerHelper<"GenMyCombiner", [ + InstTest0, + PatFragTest0 +]>; diff --git a/llvm/test/TableGen/GlobalISelCombinerMatchTableEmitter/patfrag-errors.td b/llvm/test/TableGen/GlobalISelCombinerMatchTableEmitter/patfrag-errors.td new file mode 100644 --- /dev/null +++ b/llvm/test/TableGen/GlobalISelCombinerMatchTableEmitter/patfrag-errors.td @@ -0,0 +1,282 @@ +// RUN: not llvm-tblgen -I %p/../../../include -gen-global-isel-combiner-matchtable \ +// RUN: -combiners=MyCombiner %s 2>&1| \ +// RUN: FileCheck %s -implicit-check-not=error: + +include "llvm/Target/Target.td" +include "llvm/Target/GlobalISel/Combine.td" + +def MyTargetISA : InstrInfo; +def MyTarget : Target { let InstructionSet = MyTargetISA; } + +def dummy; + +def MatchFooPerms: GICombinePatFrag< + (outs), + (ins gi_mo:$foo, gi_imm:$cst), + [ + (pattern "return foo(${foo}, ${cst})"), + (pattern "return bar(${foo}, ${cst})"), + (pattern "return bux(${foo}, ${cst})"), + ]>; + +// CHECK: :[[@LINE+2]]:{{[0-9]+}}: error: cannot emit rule 'too_many_perms'; 27 permutations would be emitted, but the max is 16 +let MaxPermutations = 16 in +def too_many_perms : GICombineRule< + (defs root:$dst), + (match (G_ZEXT $dst, $cst), + (MatchFooPerms $cst, (i32 0)):$a, + (MatchFooPerms $cst, (i32 0)):$b, + (MatchFooPerms $cst, (i32 0)):$c + ), + (apply (COPY $dst, (i32 0)), "APPLY ${src}")>; + +def CXXUsesLiveIn: GICombinePatFrag< + (outs), + (ins gi_mo:$in), + [ + (pattern "return foo(${in})"), + ]>; +// CHECK: :[[@LINE+2]]:{{[0-9]+}}: error: operand 'foo' (for parameter 'in' of 'CXXUsesLiveIn') cannot be unbound +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: note: one or more alternatives of 'CXXUsesLiveIn' do not bind 'in' to an instruction operand; either use a bound operand or ensure 'CXXUsesLiveIn' binds 'in' in all alternatives +def undef_livein : GICombineRule< + (defs root:$dst), + (match (G_ZEXT $dst, $bar), + (CXXUsesLiveIn $foo) + ), + (apply (COPY $dst, (i32 0)))>; + +// CHECK: :[[@LINE+2]]:{{[0-9]+}}: error: GICombinePatFrag must have one root in its 'out' operands +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Could not parse GICombinePatFrag 'OutMustBeRoot' +def OutMustBeRoot: GICombinePatFrag< + (outs $foo, $bar), + (ins), + [ + (pattern (G_ZEXT $foo, $bar), (G_FPEXT $bar, $y)), + ]>; +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Failed to parse pattern: '(OutMustBeRoot ?:$bar, ?:$foo)' +def out_must_be_root : GICombineRule< + (defs root:$dst), + (match (G_ZEXT $dst, $bar), + (OutMustBeRoot $bar, $foo) + ), + (apply (COPY $dst, (i32 0)))>; + +// CHECK: :[[@LINE+2]]:{{[0-9]+}}: error: output parameter 'bar' must be 'root' or 'gi_mo' +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Could not parse GICombinePatFrag 'BadOutType' +def BadOutType: GICombinePatFrag< + (outs root:$foo, gi_imm:$bar), + (ins), + [ + (pattern (G_ZEXT $foo, $bar), (G_FPEXT $bar, $y)), + ]>; +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Failed to parse pattern: '(BadOutType ?:$bar, ?:$foo)' +def bad_out_type : GICombineRule< + (defs root:$dst), + (match (G_ZEXT $dst, $bar), + (BadOutType $bar, $foo) + ), + (apply (COPY $dst, (i32 0)))>; + +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: pattern 'dbg' ('G_FPEXT') is unreachable from the pattern root! +def UnreachablePat: GICombinePatFrag< + (outs root:$foo, $bar), + (ins), + [ + (pattern (G_ZEXT $foo, $x), (G_FPEXT $bar, $y):$dbg), + ]>; +def unreachable_pat : GICombineRule< + (defs root:$dst), + (match (G_ZEXT $dst, $bar), + (UnreachablePat $bar, $foo) + ), + (apply (COPY $dst, (i32 0)))>; + +// CHECK: :[[@LINE+2]]:{{[0-9]+}}: error: wip_match_opcode cannot be used in GICombinePatFrag +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Could not parse GICombinePatFrag 'WipMatchOpcodePatFrag' +def WipMatchOpcodePatFrag: GICombinePatFrag< + (outs), + (ins), + [ + (pattern (wip_match_opcode G_ZEXT)), + ]>; +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Failed to parse pattern: '(WipMatchOpcodePatFrag)' +def wip_match_opcode_patfrag : GICombineRule< + (defs root:$dst), + (match (WipMatchOpcodePatFrag)), + (apply (COPY $dst, (i32 0)))>; + +def DummyPF: GICombinePatFrag<(outs),(ins),[]>; +// CHECK: :[[@LINE+2]]:{{[0-9]+}}: error: nested GICombinePatFrag are not supported +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Could not parse GICombinePatFrag 'NestingPatFrag' +def NestingPatFrag: GICombinePatFrag< + (outs), + (ins), + [ + (pattern (DummyPF)), + ]>; +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Failed to parse pattern: '(NestingPatFrag)' +def nest_pat_frag : GICombineRule< + (defs root:$dst), + (match (NestingPatFrag)), + (apply (COPY $dst, (i32 0)))>; + +// CHECK: :[[@LINE+2]]:{{[0-9]+}}: error: input parameter 'k' cannot be redefined! +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Could not parse GICombinePatFrag 'DupParamIn' +def DupParamIn: GICombinePatFrag< + (outs), + (ins gi_mo:$k, gi_mo:$k), + [ + (pattern (G_ZEXT $k, $x)), + ]>; +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Failed to parse pattern: '(DupParamIn ?:$dst, ?:$bar)' +def dup_params_in : GICombineRule< + (defs root:$dst), + (match (DupParamIn $dst, $bar)), + (apply (COPY $dst, (i32 0)))>; + +// CHECK: :[[@LINE+2]]:{{[0-9]+}}: error: duplicate parameter 'k' +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Could not parse GICombinePatFrag 'DupParamOut' +def DupParamOut: GICombinePatFrag< + (outs root:$k, root:$k), + (ins), + [ + (pattern (G_ZEXT $k, $x)), + ]>; +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Failed to parse pattern: '(DupParamOut ?:$dst, ?:$bar)' +def dup_params_out : GICombineRule< + (defs root:$dst), + (match (DupParamOut $dst, $bar)), + (apply (COPY $dst, (i32 0)))>; + +// CHECK: :[[@LINE+2]]:{{[0-9]+}}: error: input parameter 'k' cannot be redefined! +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Could not parse GICombinePatFrag 'DupParamInOut' +def DupParamInOut: GICombinePatFrag< + (outs root:$k), + (ins gi_mo:$k), + [ + (pattern (G_ZEXT $k, $x)), + ]>; +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Failed to parse pattern: '(DupParamInOut ?:$dst, ?:$bar)' +def dup_params_inout : GICombineRule< + (defs root:$dst), + (match (DupParamInOut $dst, $bar)), + (apply (COPY $dst, (i32 0)))>; + +// CHECK: :[[@LINE+2]]:{{[0-9]+}}: error: output parameter 'k' must be defined by all alternative patterns in 'DefByAllAlts' +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Could not parse GICombinePatFrag 'DefByAllAlts' +def DefByAllAlts: GICombinePatFrag< + (outs root:$k), + (ins), + [ + (pattern (G_ZEXT $k, $x)), + (pattern (G_FPEXT $z, $k)) + ]>; +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Failed to parse pattern: '(DefByAllAlts ?:$dst)' +def def_by_all_alts : GICombineRule< + (defs root:$dst), + (match (DefByAllAlts $dst)), + (apply (COPY $dst, (i32 0)))>; + +// CHECK: :[[@LINE+2]]:{{[0-9]+}}: error: Operand 'x' is defined multiple times in patterns of alternative #1 +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Could not parse GICombinePatFrag 'MultiDefInPat' +def MultiDefInPat: GICombinePatFrag< + (outs root:$k), + (ins), + [ + (pattern (G_ZEXT $k, $a)), + (pattern (G_ZEXT $k, $x), (G_ZEXT $x, $foo), (G_FPEXT $x, $foo)), + ]>; +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Failed to parse pattern: '(MultiDefInPat ?:$dst)' +def multi_def_in_pat : GICombineRule< + (defs root:$dst), + (match (MultiDefInPat $dst)), + (apply (COPY $dst, (i32 0)))>; + +def ExpectedImm: GICombinePatFrag< + (outs root:$k), + (ins gi_imm:$i), + [ + (pattern (G_ZEXT $k, $i)), + ]>; +// CHECK: :[[@LINE+2]]:{{[0-9]+}}: error: expected operand 1 of 'ExpectedImm' to be an immediate; got MachineOperand $z +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Failed to parse pattern: '(ExpectedImm ?:$dst, ?:$z)' +def expected_imm : GICombineRule< + (defs root:$dst), + (match (ExpectedImm $dst, $z)), + (apply (COPY $dst, (i32 0)))>; + +// CHECK: :[[@LINE+2]]:{{[0-9]+}}: error: operand 1 of 'ExpectedImm' cannot be a named immediate +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Failed to parse pattern: '(ExpectedImm ?:$dst, (i32 0):$z)' +def expected_imm_namedimm : GICombineRule< + (defs root:$dst), + (match (ExpectedImm $dst, (i32 0):$z)), + (apply (COPY $dst, (i32 0)))>; + +def ExpectedMO: GICombinePatFrag< + (outs root:$k), + (ins gi_mo:$i), + [ + (pattern (G_ZEXT $k, $i)), + ]>; +// CHECK: :[[@LINE+2]]:{{[0-9]+}}: error: expected operand 1 of 'ExpectedMO' to be a MachineOperand; got imm 0 +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Failed to parse pattern: '(ExpectedMO ?:$dst, (i32 0))' +def expected_mo : GICombineRule< + (defs root:$dst), + (match (ExpectedMO $dst, (i32 0))), + (apply (COPY $dst, (i32 0)))>; +// CHECK: :[[@LINE+2]]:{{[0-9]+}}: error: expected operand 1 of 'ExpectedMO' to be a MachineOperand; got imm 0:$z +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Failed to parse pattern: '(ExpectedMO ?:$dst, (i32 0):$z)' +def expected_mo_namedimm : GICombineRule< + (defs root:$dst), + (match (ExpectedMO $dst, (i32 0):$z)), + (apply (COPY $dst, (i32 0)))>; + +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: 'dum': using GICombinePatFrag is not supported in apply patterns +def patfrag_in_apply : GICombineRule< + (defs root:$dst), + (match (COPY $dst, (i32 0))), + (apply (DummyPF):$dum)>; + +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: cannot use 'DummyPF as match root +def patfrag_cannot_be_root : GICombineRule< + (defs root:$root), + (match (DummyPF):$root), + (apply (COPY $dst, (i32 0)):$root)>; + +def TypedParams: GICombinePatFrag< + (outs root:$k), + (ins gi_mo:$i), + [ + (pattern (G_ZEXT $k, i32:$i)), + ]>; +// CHECK: :[[@LINE+3]]:{{[0-9]+}}: warning: impossible type constraints: operand 1 of 'broken' has type 'i64', but 'TypedParams' constrains it to 'i32' +// CHECK: :[[@LINE+2]]:{{[0-9]+}}: note: operand 1 of 'broken' is 'k' +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: note: argument 1 of 'TypedParams' is 'i' +def inconsistent_arg_type : GICombineRule< + (defs root:$dst), + (match (TypedParams $dst, i64:$k):$broken), + (apply (COPY $dst, (i32 0)))>; + +// CHECK: error: Failed to parse one or more rules + +def MyCombiner: GICombinerHelper<"GenMyCombiner", [ + too_many_perms, + undef_livein, + out_must_be_root, + bad_out_type, + unreachable_pat, + wip_match_opcode_patfrag, + nest_pat_frag, + dup_params_in, + dup_params_out, + dup_params_inout, + def_by_all_alts, + multi_def_in_pat, + expected_imm, + expected_imm_namedimm, + expected_mo, + expected_mo_namedimm, + patfrag_in_apply, + patfrag_cannot_be_root, + inconsistent_arg_type +]>; diff --git a/llvm/test/TableGen/GlobalISelCombinerMatchTableEmitter/pattern-errors.td b/llvm/test/TableGen/GlobalISelCombinerMatchTableEmitter/pattern-errors.td new file mode 100644 --- /dev/null +++ b/llvm/test/TableGen/GlobalISelCombinerMatchTableEmitter/pattern-errors.td @@ -0,0 +1,256 @@ +// RUN: not llvm-tblgen -I %p/../../../include -gen-global-isel-combiner-matchtable \ +// RUN: -combiners=MyCombiner %s 2>&1| \ +// RUN: FileCheck %s -implicit-check-not=error: + +include "llvm/Target/Target.td" +include "llvm/Target/GlobalISel/Combine.td" + +def MyTargetISA : InstrInfo; +def MyTarget : Target { let InstructionSet = MyTargetISA; } + +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Cannot find root 'missing' in match patterns! +def root_not_found : GICombineRule< + (defs root:$missing), + (match (COPY $a, $b):$d), + (apply [{ APPLY }])>; + +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: apply pattern 'd' is supposed to be a root but it does not redefine any of the defs of the match root +def misleading_root : GICombineRule< + (defs root:$d), + (match (COPY $a, $b):$d), + (apply (COPY $tmp, $b):$d, + (COPY $a, $tmp))>; + +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: C++ code cannot be the root of a rule! +def cxx_root : GICombineRule< + (defs root:$a), + (match "return MATCH":$a), + (apply [{ APPLY }]:$z)>; + +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Cannot use live-in operand 'b' as match pattern root! +def livein_root : GICombineRule< + (defs root:$b), + (match (COPY $a, $b)), + (apply [{ APPLY }])>; + +// CHECK: :[[@LINE+2]]:{{[0-9]+}}: error: 'COPY' expected 2 operands, got 1 +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Failed to parse pattern: '(COPY ?:$a)' +def not_enough_operands : GICombineRule< + (defs root:$d), + (match (COPY $a):$d), + (apply [{ APPLY }])>; + +// CHECK: :[[@LINE+2]]:{{[0-9]+}}: error: 'COPY' expected 2 operands, got 3 +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Failed to parse pattern: '(COPY ?:$a, ?:$b, ?:$c)' +def too_many_operands : GICombineRule< + (defs root:$d), + (match (COPY $a, $b, $c):$d), + (apply [{ APPLY }])>; + +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Operand 'd' is defined multiple times in the 'match' patterns +def multi_defs : GICombineRule< + (defs root:$d), + (match (COPY $d, $b), (COPY $d, $x)), + (apply [{ APPLY }])>; + +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Operand 'd' is defined multiple times in the 'match' patterns +def multi_defs_2 : GICombineRule< + (defs root:$d), + (match (G_UNMERGE_VALUES $d, $d, $b)), + (apply [{ APPLY }])>; + +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: pattern 'foo' ('COPY') is unreachable from the pattern root! +def unreachable_pat : GICombineRule< + (defs root:$d), + (match (COPY $a, $b):$d, (COPY $z, $k):$foo), + (apply [{ APPLY }])>; + +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: 'applytest': wip_match_opcode is not supported in apply patterns +def wip_match_opcode_in_apply : GICombineRule< + (defs root:$d), + (match (COPY $a, $b):$d, (wip_match_opcode G_ZEXT)), + (apply (wip_match_opcode G_ZEXT):$applytest)>; + +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: wip_match_opcode can not be used with instruction patterns! +def wip_match_opcode_with_inst_pat : GICombineRule< + (defs root:$d), + (match (COPY $a, $b):$d, (wip_match_opcode G_ZEXT)), + (apply [{ APPLY }])>; + +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: wip_opcode_match can only be present once +def multiple_wip_match_opcode : GICombineRule< + (defs root:$d), + (match (wip_match_opcode COPY):$d, (wip_match_opcode G_ZEXT)), + (apply [{ APPLY }])>; + +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Operand 'a' is defined multiple times in the 'apply' patterns +def multiple_def_in_apply : GICombineRule< + (defs root:$d), + (match (COPY $a, $b):$d), + (apply (COPY $a, $b), (COPY $a, $b))>; + +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: 'd' match pattern defined more than once! +def redef_match : GICombineRule< + (defs root:$d), + (match (COPY $a, $b):$d, (COPY $b, $z):$d), + (apply (COPY $a, $b))>; + +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: 'o' apply pattern defined more than once! +def redef_apply: GICombineRule< + (defs root:$d), + (match (COPY $a, $b):$d), + (apply (COPY $a, $x):$o, (COPY $x, $b):$o)>; + +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: redefining an instruction other than the root is not supported (operand 'b' +def redef_nonroot : GICombineRule< + (defs root:$a), + (match (COPY $a, $b), (COPY $b, $z)), + (apply (COPY $a, $b), (G_ZEXT $b, (i32 0)))>; + +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Cannot define live-in operand 'b' in the 'apply' pattern +def cannot_def_match_livein : GICombineRule< + (defs root:$d), + (match (COPY $a, $b):$d), + (apply (COPY $a, $b), (COPY $b, $b))>; + +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: invalid output operand 'x': operand is not a live-in of the match pattern, and it has no definition +def undef_in_apply : GICombineRule< + (defs root:$d), + (match (COPY $a, $b):$d), + (apply (COPY $a, $x))>; + +// CHECK: :[[@LINE+2]]:{{[0-9]+}}: error: def of pattern root 'a' is not redefined in the apply pattern! +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: note: match pattern root is 'foo' +def no_redef_in_apply : GICombineRule< + (defs root:$a), + (match (COPY $a, $b):$foo), + (apply (COPY $x, $b))>; + +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: def of pattern root 'b' is not redefined in the apply pattern! +def no_redef_in_apply_multidefroot : GICombineRule< + (defs root:$a), + (match (G_UNMERGE_VALUES $a, $b, $c)), + (apply (COPY $a, $c))>; + +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: cannot use wip_match_opcode in combination with apply instruction patterns +def instpat_with_wipmatch : GICombineRule< + (defs root:$a), + (match (wip_match_opcode COPY):$d), + (apply (COPY $x, $b):$d)>; + +// CHECK: :[[@LINE+2]]:{{[0-9]+}}: error: cannot parse operand '(i32)' +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Failed to parse pattern: '(COPY ?:$x, (i32)) +def bad_imm_noarg : GICombineRule< + (defs root:$a), + (match (COPY $x, (i32)):$d), + (apply (COPY $x, $b):$d)>; + +// CHECK: :[[@LINE+2]]:{{[0-9]+}}: error: cannot parse operand '(i32 0, 0)' +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Failed to parse pattern: '(COPY ?:$x, (i32 0, 0)) +def bad_imm_too_many_args : GICombineRule< + (defs root:$a), + (match (COPY $x, (i32 0, 0)):$d), + (apply (COPY $x, $b):$d)>; + +// CHECK: :[[@LINE+2]]:{{[0-9]+}}: error: cannot parse immediate '(COPY 0)', 'COPY' is not a ValueType +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Failed to parse pattern: '(COPY ?:$x, (COPY 0)) +def bad_imm_not_a_valuetype : GICombineRule< + (defs root:$a), + (match (COPY $x, (COPY 0)):$d), + (apply (COPY $x, $b):$d)>; + +// CHECK: :[[@LINE+2]]:{{[0-9]+}}: error: invalid output operand 'imm': output immediates cannot be named +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: note: while emitting pattern 'd' (COPY) +def output_imm_cannot_be_named : GICombineRule< + (defs root:$x), + (match (COPY $x, (i32 0)):$d), + (apply (COPY $x, (i32 0):$imm):$d)>; + +// CHECK: :[[@LINE+2]]:{{[0-9]+}}: error: output immediates must be typed! +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: note: while emitting pattern 'd' (COPY) +def output_imm_must_be_typed : GICombineRule< + (defs root:$x), + (match (COPY $x, (i32 0)):$d), + (apply (COPY $x, 0):$d)>; + +// CHECK: :[[@LINE+2]]:{{[0-9]+}}: error: 'G_BUILD_VECTOR' expected at least 2 operands, got 1 +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Failed to parse pattern: '(G_BUILD_VECTOR ?:$x)' +def too_few_ops_for_variadic : GICombineRule< + (defs root:$x), + (match (G_BUILD_VECTOR $x)), + (apply (COPY $x, 0))>; + +// CHECK: :[[@LINE+2]]:{{[0-9]+}}: error: expected an operand name after 'i32' +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Failed to parse pattern: '(G_FNEG ?:$x, i32)' +def expected_op_name : GICombineRule< + (defs root:$x), + (match (G_FNEG $x, i32)), + (apply (COPY $x, (i32 0)))>; + +// CHECK: :[[@LINE+3]]:{{[0-9]+}}: error: invalid operand type: 'not_a_type' is not a ValueType +// CHECK: :[[@LINE+2]]:{{[0-9]+}}: error: Failed to parse pattern: '(G_FNEG ?:$x, not_a_type:$y)' +def not_a_type; +def bad_mo_type_not_a_valuetype : GICombineRule< + (defs root:$x), + (match (G_FNEG $x, not_a_type:$y)), + (apply (COPY $x, (i32 0)))>; + +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: def of a new register 'newreg' in the apply patterns must have a type +def untyped_new_reg_in_apply : GICombineRule< + (defs root:$x), + (match (G_FNEG $x, $y)), + (apply (COPY $newreg, (i32 0)), + (COPY $x, $newreg))>; + +// CHECK: :[[@LINE+2]]:{{[0-9]+}}: error: 'y' is a named immediate, it cannot be defined by another instruction +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: note: 'y' is defined by 'foo' +def def_named_imm_match : GICombineRule< + (defs root:$x), + (match (G_SEXT $y, $z):$foo, + (G_FNEG $x, (i32 0):$y)), + (apply (COPY $x, (i32 0)))>; + +// CHECK: :[[@LINE+2]]:{{[0-9]+}}: error: invalid output operand 'tmp': output immediates cannot be named +// CHECK: :[[@LINE+1]]:{{[0-9]+}}: note: while emitting pattern 'foo' (COPY) +def def_named_imm_apply : GICombineRule< + (defs root:$x), + (match (G_FNEG $x, $y)), + (apply (COPY i32:$tmp, $y), + (COPY $x, (i32 0):$tmp):$foo)>; + +// CHECK: error: Failed to parse one or more rules + +def MyCombiner: GICombinerHelper<"GenMyCombiner", [ + root_not_found, + misleading_root, + cxx_root, + livein_root, + not_enough_operands, + too_many_operands, + multi_defs, + multi_defs_2, + unreachable_pat, + wip_match_opcode_in_apply, + wip_match_opcode_with_inst_pat, + multiple_wip_match_opcode, + multiple_def_in_apply, + redef_match, + redef_apply, + redef_nonroot, + cannot_def_match_livein, + undef_in_apply, + no_redef_in_apply, + no_redef_in_apply_multidefroot, + instpat_with_wipmatch, + bad_imm_noarg, + bad_imm_too_many_args, + bad_imm_not_a_valuetype, + output_imm_cannot_be_named, + output_imm_must_be_typed, + too_few_ops_for_variadic, + expected_op_name, + bad_mo_type_not_a_valuetype, + untyped_new_reg_in_apply, + def_named_imm_match, + def_named_imm_apply +]>; diff --git a/llvm/test/TableGen/GlobalISelCombinerMatchTableEmitter/pattern-parsing-errors.td b/llvm/test/TableGen/GlobalISelCombinerMatchTableEmitter/pattern-parsing-errors.td deleted file mode 100644 --- a/llvm/test/TableGen/GlobalISelCombinerMatchTableEmitter/pattern-parsing-errors.td +++ /dev/null @@ -1,92 +0,0 @@ -// RUN: not llvm-tblgen -I %p/../../../include -gen-global-isel-combiner-matchtable \ -// RUN: -combiners=MyCombiner %s 2>&1| \ -// RUN: FileCheck %s -implicit-check-not=error: - -include "llvm/Target/Target.td" -include "llvm/Target/GlobalISel/Combine.td" - -def MyTargetISA : InstrInfo; -def MyTarget : Target { let InstructionSet = MyTargetISA; } - -def dummy; - -def R0 : Register<"r0"> { let Namespace = "MyTarget"; } -def GPR32 : RegisterClass<"MyTarget", [i32], 32, (add R0)>; -class I Pat> - : Instruction { - let Namespace = "MyTarget"; - let OutOperandList = OOps; - let InOperandList = IOps; - let Pattern = Pat; -} -def MOV : I<(outs GPR32:$dst), (ins GPR32:$src1), []>; -def ADD : I<(outs GPR32:$dst), (ins GPR32:$src1, GPR32:$src2), []>; -def SUB : I<(outs GPR32:$dst), (ins GPR32:$src1, GPR32:$src2), []>; -def MUL : I<(outs GPR32:$dst), (ins GPR32:$src1, GPR32:$src2), []>; -def TRUNC : I<(outs GPR32:$dst), (ins GPR32:$src1), []>; -def SEXT : I<(outs GPR32:$dst), (ins GPR32:$src1), []>; -def ZEXT : I<(outs GPR32:$dst), (ins GPR32:$src1), []>; -def ICMP : I<(outs GPR32:$dst), (ins GPR32:$tst, GPR32:$src1, GPR32:$src2), []>; - -// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Cannot find root 'missing' in match patterns! -def root_not_found : GICombineRule< - (defs root:$missing), - (match (MOV $a, $b):$d), - (apply [{ APPLY }])>; - -// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Cannot use live-in operand 'b' as match pattern root! -def livein_root : GICombineRule< - (defs root:$b), - (match (MOV $a, $b)), - (apply [{ APPLY }])>; - -// CHECK: :[[@LINE+2]]:{{[0-9]+}}: error: 'MOV' expected 2 operands, got 1 -// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Expected a subclass of GIMatchKind or a sub-dag whose operator is either of a GIMatchKindWithArgs or Instruction -def not_enough_operands : GICombineRule< - (defs root:$d), - (match (MOV $a):$d), - (apply [{ APPLY }])>; - -// CHECK: :[[@LINE+2]]:{{[0-9]+}}: error: 'MOV' expected 2 operands, got 3 -// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Expected a subclass of GIMatchKind or a sub-dag whose operator is either of a GIMatchKindWithArgs or Instruction -def too_many_operands : GICombineRule< - (defs root:$d), - (match (MOV $a, $b, $c):$d), - (apply [{ APPLY }])>; - -// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Operand 'd' is defined multiple times in the 'match' patterns -def multi_defs : GICombineRule< - (defs root:$d), - (match (MOV $d, $b), (MOV $d, $x)), - (apply [{ APPLY }])>; - -// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: Instruction pattern '__anon_pat_match_5_0' is unreachable from the pattern root! -def unreachable_pat : GICombineRule< - (defs root:$d), - (match (MOV $a, $b):$d, (MOV $z, $k)), - (apply [{ APPLY }])>; - -// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: wip_match_opcode can not be used with instruction patterns! -def wip_match_opcode_with_inst_pat : GICombineRule< - (defs root:$d), - (match (MOV $a, $b):$d, (wip_match_opcode SEXT)), - (apply [{ APPLY }])>; - -// CHECK: :[[@LINE+1]]:{{[0-9]+}}: error: wip_opcode_match can only be present once -def multiple_wip_match_opcode : GICombineRule< - (defs root:$d), - (match (wip_match_opcode MOV):$d, (wip_match_opcode SEXT)), - (apply [{ APPLY }])>; - -// CHECK: error: Failed to parse one or more rules - -def MyCombiner: GICombinerHelper<"GenMyCombiner", [ - root_not_found, - livein_root, - not_enough_operands, - too_many_operands, - multi_defs, - unreachable_pat, - wip_match_opcode_with_inst_pat, - multiple_wip_match_opcode -]>; diff --git a/llvm/test/TableGen/GlobalISelCombinerMatchTableEmitter/pattern-parsing.td b/llvm/test/TableGen/GlobalISelCombinerMatchTableEmitter/pattern-parsing.td --- a/llvm/test/TableGen/GlobalISelCombinerMatchTableEmitter/pattern-parsing.td +++ b/llvm/test/TableGen/GlobalISelCombinerMatchTableEmitter/pattern-parsing.td @@ -10,27 +10,12 @@ def dummy; -def R0 : Register<"r0"> { let Namespace = "MyTarget"; } -def GPR32 : RegisterClass<"MyTarget", [i32], 32, (add R0)>; -class I Pat> - : Instruction { - let Namespace = "MyTarget"; - let OutOperandList = OOps; - let InOperandList = IOps; - let Pattern = Pat; -} -def MOV : I<(outs GPR32:$dst), (ins GPR32:$src1), []>; -def TRUNC : I<(outs GPR32:$dst), (ins GPR32:$src1), []>; -def ZEXT : I<(outs GPR32:$dst), (ins GPR32:$src1), []>; -def SEXT : I<(outs GPR32:$dst), (ins GPR32:$src1), []>; - def HasAnswerToEverything : Predicate<"Subtarget->getAnswerToUniverse() == 42 && Subtarget->getAnswerToLife() == 42">; def reg_matchinfo : GIDefMatchData<"Register">; // CHECK: (CombineRule name:WipOpcodeTest0 id:0 root:d -// CHECK-NEXT: (MatchDatas ) // CHECK-NEXT: (MatchPats -// CHECK-NEXT: d:(AnyOpcodePattern [TRUNC]) +// CHECK-NEXT: d:(AnyOpcodePattern [G_TRUNC]) // CHECK-NEXT: ) // CHECK-NEXT: (ApplyPats // CHECK-NEXT: __anon_pat_apply_0_0:(CXXPattern apply code:"APPLY") @@ -39,13 +24,12 @@ // CHECK-NEXT: ) def WipOpcodeTest0 : GICombineRule< (defs root:$d), - (match (wip_match_opcode TRUNC):$d), + (match (wip_match_opcode G_TRUNC):$d), (apply [{ APPLY }])>; // CHECK: (CombineRule name:WipOpcodeTest1 id:1 root:d -// CHECK-NEXT: (MatchDatas ) // CHECK-NEXT: (MatchPats -// CHECK-NEXT: d:(AnyOpcodePattern [TRUNC, SEXT]) +// CHECK-NEXT: d:(AnyOpcodePattern [G_TRUNC, G_SEXT]) // CHECK-NEXT: ) // CHECK-NEXT: (ApplyPats // CHECK-NEXT: __anon_pat_apply_1_0:(CXXPattern apply code:"APPLY") @@ -54,25 +38,24 @@ // CHECK-NEXT: ) def WipOpcodeTest1 : GICombineRule< (defs root:$d), - (match (wip_match_opcode TRUNC, SEXT):$d), + (match (wip_match_opcode G_TRUNC, G_SEXT):$d), (apply [{ APPLY }])>; // CHECK: (CombineRule name:InstTest0 id:2 root:d -// CHECK-NEXT: (MatchDatas ) // CHECK-NEXT: (MatchPats -// CHECK-NEXT: d:(InstructionPattern inst:MOV operands:[a, b]) +// CHECK-NEXT: d:(CodeGenInstructionPattern COPY operands:[$a, $b]) // CHECK-NEXT: ) // CHECK-NEXT: (ApplyPats // CHECK-NEXT: __anon_pat_apply_2_0:(CXXPattern apply code:"APPLY") // CHECK-NEXT: ) // CHECK-NEXT: (OperandTable // CHECK-NEXT: [a match_pat:d] -// CHECK-NEXT: [b live-in] +// CHECK-NEXT: [b match-live-in] // CHECK-NEXT: ) // CHECK-NEXT: ) def InstTest0 : GICombineRule< (defs root:$d), - (match (MOV $a, $b):$d), + (match (COPY $a, $b):$d), (apply [{ APPLY }])>; // CHECK: (CombineRule name:InstTest1 id:3 root:d @@ -80,28 +63,227 @@ // CHECK-NEXT: (MatchDataInfo pattern_symbol:r0 type:'Register' var_name:MDInfo0) // CHECK-NEXT: ) // CHECK-NEXT: (MatchPats -// CHECK-NEXT: d:(InstructionPattern inst:MOV operands:[a, b]) -// CHECK-NEXT: __anon_pat_match_3_0:(InstructionPattern inst:ZEXT operands:[x, a]) +// CHECK-NEXT: d:(CodeGenInstructionPattern COPY operands:[$a, i32:$b]) +// CHECK-NEXT: __anon_pat_match_3_0:(CodeGenInstructionPattern G_ZEXT operands:[$x, 0]) // CHECK-NEXT: ) // CHECK-NEXT: (ApplyPats // CHECK-NEXT: __anon_pat_apply_3_1:(CXXPattern apply code:"APPLY") // CHECK-NEXT: ) // CHECK-NEXT: (OperandTable // CHECK-NEXT: [a match_pat:d] -// CHECK-NEXT: [b live-in] +// CHECK-NEXT: [b match-live-in] // CHECK-NEXT: [x match_pat:__anon_pat_match_3_0] // CHECK-NEXT: ) // CHECK-NEXT: ) let Predicates = [HasAnswerToEverything] in def InstTest1 : GICombineRule< (defs root:$d, reg_matchinfo:$r0), - (match (MOV $a, $b):$d, - (ZEXT $x, $a)), + (match (COPY $a, i32:$b):$d, + (G_ZEXT $x, 0)), (apply [{ APPLY }])>; +// CHECK: (CombineRule name:InstTest2 id:4 root:d +// CHECK-NEXT: (MatchDatas +// CHECK-NEXT: (MatchDataInfo pattern_symbol:r0 type:'Register' var_name:MDInfo0) +// CHECK-NEXT: ) +// CHECK-NEXT: (MatchPats +// CHECK-NEXT: __anon_pat_match_4_0:(CodeGenInstructionPattern COPY operands:[$d, (i32 0):$x]) +// CHECK-NEXT: ) +// CHECK-NEXT: (ApplyPats +// CHECK-NEXT: __anon_pat_apply_4_1:(CXXPattern apply code:"APPLY") +// CHECK-NEXT: ) +// CHECK-NEXT: (OperandTable +// CHECK-NEXT: [d match_pat:__anon_pat_match_4_0] +// CHECK-NEXT: [x match-live-in] +// CHECK-NEXT: ) +// CHECK-NEXT: ) +def InstTest2 : GICombineRule< + (defs root:$d, reg_matchinfo:$r0), + (match (COPY $d, (i32 0):$x)), + (apply [{ APPLY }])>; + +// CHECK: (CombineRule name:InOutInstTest0 id:5 root:dst +// CHECK-NEXT: (MatchPats +// CHECK-NEXT: __anon_pat_match_5_0:(CodeGenInstructionPattern COPY operands:[$dst, $tmp]) +// CHECK-NEXT: __anon_pat_match_5_1:(CodeGenInstructionPattern G_ZEXT operands:[$tmp, $src]) +// CHECK-NEXT: ) +// CHECK-NEXT: (ApplyPats +// CHECK-NEXT: __anon_pat_apply_5_2:(CodeGenInstructionPattern G_TRUNC operands:[$dst, $src]) +// CHECK-NEXT: __anon_pat_apply_5_3:(CXXPattern apply code:"APPLY ${src}") +// CHECK-NEXT: ) +// CHECK-NEXT: (OperandTable +// CHECK-NEXT: [dst match_pat:__anon_pat_match_5_0 apply_pat:__anon_pat_apply_5_2] +// CHECK-NEXT: [tmp match_pat:__anon_pat_match_5_1] +// CHECK-NEXT: [src match-live-in] +// CHECK-NEXT: ) +// CHECK-NEXT: ) +def InOutInstTest0 : GICombineRule< + (defs root:$dst), + (match (COPY $dst, $tmp), + (G_ZEXT $tmp, $src)), + (apply (G_TRUNC $dst, $src), "APPLY ${src}")>; + +def MatchICst: GICombinePatFrag< + (outs), + (ins gi_mo:$foo, gi_imm:$cst), + [(pattern "return matchIConstant(${foo}, ${cst})")]>; + +// CHECK: (CombineRule name:PatFragTest0 id:6 root:dst +// CHECK-NEXT: (PatFrags +// CHECK-NEXT: (PatFrag name:MatchICst +// CHECK-NEXT: (ins [foo:machine_operand, cst:imm]) +// CHECK-NEXT: (alternatives [ +// CHECK-NEXT: [ +// CHECK-NEXT: (CXXPattern name:__anon_pat_pattern_6_2 match code:"return matchIConstant(${foo}, ${cst})"), +// CHECK-NEXT: ], +// CHECK-NEXT: ]) +// CHECK-NEXT: ) +// CHECK-NEXT: ) +// CHECK-NEXT: (MatchPats +// CHECK-NEXT: __anon_pat_match_6_0:(CodeGenInstructionPattern G_ZEXT operands:[$dst, $cst]) +// CHECK-NEXT: __anon_pat_match_6_1:(PatFragPattern MatchICst operands:[$cst, (i32 0)]) +// CHECK-NEXT: ) +// CHECK-NEXT: (ApplyPats +// CHECK-NEXT: __anon_pat_apply_6_3:(CodeGenInstructionPattern COPY operands:[$dst, (i32 0)]) +// CHECK-NEXT: __anon_pat_apply_6_4:(CXXPattern apply code:"APPLY ${src}") +// CHECK-NEXT: ) +// CHECK-NEXT: (OperandTable +// CHECK-NEXT: [dst match_pat:__anon_pat_match_6_0 apply_pat:__anon_pat_apply_6_3] +// CHECK-NEXT: [cst match-live-in] +// CHECK-NEXT: ) +// CHECK-NEXT: ) +def PatFragTest0 : GICombineRule< + (defs root:$dst), + (match (G_ZEXT $dst, $cst), (MatchICst $cst, (i32 0))), + (apply (COPY $dst, (i32 0)), "APPLY ${src}")>; + +def MatchFooPerms: GICombinePatFrag< + (outs), + (ins gi_mo:$foo, gi_imm:$cst), + [ + (pattern "return foo(${foo}, ${cst})"), + (pattern "return bar(${foo}, ${cst})"), + (pattern "return bux(${foo}, ${cst})"), + ]>; + +// CHECK: (CombineRule name:PatFragTest1 id:7 root:dst +// CHECK-NEXT: (PatFrags +// CHECK-NEXT: (PatFrag name:MatchFooPerms +// CHECK-NEXT: (ins [foo:machine_operand, cst:imm]) +// CHECK-NEXT: (alternatives [ +// CHECK-NEXT: [ +// CHECK-NEXT: (CXXPattern name:__anon_pat_pattern_7_1 match code:"return foo(${foo}, ${cst})"), +// CHECK-NEXT: ], +// CHECK-NEXT: [ +// CHECK-NEXT: (CXXPattern name:__anon_pat_pattern_7_2 match code:"return bar(${foo}, ${cst})"), +// CHECK-NEXT: ], +// CHECK-NEXT: [ +// CHECK-NEXT: (CXXPattern name:__anon_pat_pattern_7_3 match code:"return bux(${foo}, ${cst})"), +// CHECK-NEXT: ], +// CHECK-NEXT: ]) +// CHECK-NEXT: ) +// CHECK-NEXT: ) +// CHECK-NEXT: (MatchPats +// CHECK-NEXT: __anon_pat_match_7_0:(CodeGenInstructionPattern G_ZEXT operands:[$dst, $cst]) +// CHECK-NEXT: a:(PatFragPattern MatchFooPerms operands:[$cst, (i32 0)]) +// CHECK-NEXT: b:(PatFragPattern MatchFooPerms operands:[$cst, (i32 0)]) +// CHECK-NEXT: c:(PatFragPattern MatchFooPerms operands:[$cst, (i32 0)]) +// CHECK-NEXT: ) +// CHECK-NEXT: (ApplyPats +// CHECK-NEXT: __anon_pat_apply_7_4:(CodeGenInstructionPattern COPY operands:[$dst, (i32 0)]) +// CHECK-NEXT: __anon_pat_apply_7_5:(CXXPattern apply code:"APPLY ${src}") +// CHECK-NEXT: ) +// CHECK-NEXT: (OperandTable +// CHECK-NEXT: [dst match_pat:__anon_pat_match_7_0 apply_pat:__anon_pat_apply_7_4] +// CHECK-NEXT: [cst match-live-in] +// CHECK-NEXT: ) +// CHECK-NEXT: (PermutationsToEmit +// CHECK-NEXT: [a[0], b[0], c[0]], +// CHECK-NEXT: [a[0], b[0], c[1]], +// CHECK-NEXT: [a[0], b[0], c[2]], +// CHECK-NEXT: [a[0], b[1], c[0]], +// CHECK-NEXT: [a[0], b[1], c[1]], +// CHECK-NEXT: [a[0], b[1], c[2]], +// CHECK-NEXT: [a[0], b[2], c[0]], +// CHECK-NEXT: [a[0], b[2], c[1]], +// CHECK-NEXT: [a[0], b[2], c[2]], +// CHECK-NEXT: [a[1], b[0], c[0]], +// CHECK-NEXT: [a[1], b[0], c[1]], +// CHECK-NEXT: [a[1], b[0], c[2]], +// CHECK-NEXT: [a[1], b[1], c[0]], +// CHECK-NEXT: [a[1], b[1], c[1]], +// CHECK-NEXT: [a[1], b[1], c[2]], +// CHECK-NEXT: [a[1], b[2], c[0]], +// CHECK-NEXT: [a[1], b[2], c[1]], +// CHECK-NEXT: [a[1], b[2], c[2]], +// CHECK-NEXT: [a[2], b[0], c[0]], +// CHECK-NEXT: [a[2], b[0], c[1]], +// CHECK-NEXT: [a[2], b[0], c[2]], +// CHECK-NEXT: [a[2], b[1], c[0]], +// CHECK-NEXT: [a[2], b[1], c[1]], +// CHECK-NEXT: [a[2], b[1], c[2]], +// CHECK-NEXT: [a[2], b[2], c[0]], +// CHECK-NEXT: [a[2], b[2], c[1]], +// CHECK-NEXT: [a[2], b[2], c[2]], +// CHECK-NEXT: ) +// CHECK-NEXT: ) +let MaxPermutations = -1 in +def PatFragTest1 : GICombineRule< + (defs root:$dst), + (match (G_ZEXT $dst, $cst), + (MatchFooPerms $cst, (i32 0)):$a, + (MatchFooPerms $cst, (i32 0)):$b, + (MatchFooPerms $cst, (i32 0)):$c + ), + (apply (COPY $dst, (i32 0)), "APPLY ${src}")>; + +// CHECK: (CombineRule name:VariadicsInTest id:8 root:dst +// CHECK-NEXT: (MatchPats +// CHECK-NEXT: __anon_pat_match_8_0:(CodeGenInstructionPattern G_BUILD_VECTOR operands:[$dst, $a, $b]) +// CHECK-NEXT: ) +// CHECK-NEXT: (ApplyPats +// CHECK-NEXT: __anon_pat_apply_8_1:(CodeGenInstructionPattern COPY operands:[$dst, (i32 0)]) +// CHECK-NEXT: ) +// CHECK-NEXT: (OperandTable +// CHECK-NEXT: [dst match_pat:__anon_pat_match_8_0 apply_pat:__anon_pat_apply_8_1] +// CHECK-NEXT: [a match-live-in] +// CHECK-NEXT: [b match-live-in] +// CHECK-NEXT: ) +// CHECK-NEXT: ) +def VariadicsInTest : GICombineRule< + (defs root:$dst), + (match (G_BUILD_VECTOR $dst, $a, $b)), + (apply (COPY $dst, (i32 0)))>; + +// CHECK: (CombineRule name:VariadicsOutTest id:9 root:a +// CHECK-NEXT: (MatchPats +// CHECK-NEXT: __anon_pat_match_9_0:(CodeGenInstructionPattern G_UNMERGE_VALUES operands:[$a, $b, $src]) +// CHECK-NEXT: ) +// CHECK-NEXT: (ApplyPats +// CHECK-NEXT: __anon_pat_apply_9_1:(CodeGenInstructionPattern COPY operands:[$a, (i32 0)]) +// CHECK-NEXT: __anon_pat_apply_9_2:(CodeGenInstructionPattern COPY operands:[$b, (i32 0)]) +// CHECK-NEXT: ) +// CHECK-NEXT: (OperandTable +// CHECK-NEXT: [a match_pat:__anon_pat_match_9_0 apply_pat:__anon_pat_apply_9_1] +// CHECK-NEXT: [b match_pat:__anon_pat_match_9_0 apply_pat:__anon_pat_apply_9_2] +// CHECK-NEXT: [src match-live-in] +// CHECK-NEXT: ) +// CHECK-NEXT: ) +def VariadicsOutTest : GICombineRule< + (defs root:$a), + (match (G_UNMERGE_VALUES $a, $b, $src)), + (apply (COPY $a, (i32 0)), + (COPY $b, (i32 0)))>; + def MyCombiner: GICombinerHelper<"GenMyCombiner", [ WipOpcodeTest0, WipOpcodeTest1, InstTest0, - InstTest1 + InstTest1, + InstTest2, + InOutInstTest0, + PatFragTest0, + PatFragTest1, + VariadicsInTest, + VariadicsOutTest ]>; diff --git a/llvm/test/TableGen/GlobalISelEmitter.td b/llvm/test/TableGen/GlobalISelEmitter.td --- a/llvm/test/TableGen/GlobalISelEmitter.td +++ b/llvm/test/TableGen/GlobalISelEmitter.td @@ -82,7 +82,7 @@ // CHECK-NEXT: const int64_t *getMatchTable() const override; // CHECK-NEXT: bool testMIPredicate_MI(unsigned PredicateID, const MachineInstr &MI, const MatcherState &State) const override; // CHECK-NEXT: bool testSimplePredicate(unsigned PredicateID) const override; -// CHECK-NEXT: void runCustomAction(unsigned FnID, const MatcherState &State) const override; +// CHECK-NEXT: void runCustomAction(unsigned FnID, const MatcherState &State, NewMIVector &OutMIs) const override; // CHECK-NEXT: #endif // ifdef GET_GLOBALISEL_TEMPORARIES_DECL // CHECK-LABEL: #ifdef GET_GLOBALISEL_TEMPORARIES_INIT diff --git a/llvm/utils/TableGen/CodeGenTarget.h b/llvm/utils/TableGen/CodeGenTarget.h --- a/llvm/utils/TableGen/CodeGenTarget.h +++ b/llvm/utils/TableGen/CodeGenTarget.h @@ -43,7 +43,7 @@ /// getValueType - Return the MVT::SimpleValueType that the specified TableGen /// record corresponds to. -MVT::SimpleValueType getValueType(Record *Rec); +MVT::SimpleValueType getValueType(const Record *Rec); StringRef getName(MVT::SimpleValueType T); StringRef getEnumName(MVT::SimpleValueType T); diff --git a/llvm/utils/TableGen/CodeGenTarget.cpp b/llvm/utils/TableGen/CodeGenTarget.cpp --- a/llvm/utils/TableGen/CodeGenTarget.cpp +++ b/llvm/utils/TableGen/CodeGenTarget.cpp @@ -43,7 +43,7 @@ /// getValueType - Return the MVT::SimpleValueType that the specified TableGen /// record corresponds to. -MVT::SimpleValueType llvm::getValueType(Record *Rec) { +MVT::SimpleValueType llvm::getValueType(const Record *Rec) { return (MVT::SimpleValueType)Rec->getValueAsInt("Value"); } diff --git a/llvm/utils/TableGen/GlobalISel/CodeExpansions.h b/llvm/utils/TableGen/GlobalISel/CodeExpansions.h --- a/llvm/utils/TableGen/GlobalISel/CodeExpansions.h +++ b/llvm/utils/TableGen/GlobalISel/CodeExpansions.h @@ -29,6 +29,10 @@ Expansions.try_emplace(Name, Expansion); } + void redeclare(StringRef Name, StringRef Expansion) { + Expansions[Name] = Expansion; + } + std::string lookup(StringRef Variable) const { return Expansions.lookup(Variable); } diff --git a/llvm/utils/TableGen/GlobalISel/CombinerUtils.h b/llvm/utils/TableGen/GlobalISel/CombinerUtils.h --- a/llvm/utils/TableGen/GlobalISel/CombinerUtils.h +++ b/llvm/utils/TableGen/GlobalISel/CombinerUtils.h @@ -61,10 +61,9 @@ inline const DagInit *getDagWithOperatorOfSubClass(const Init &N, StringRef Cls) { if (const DagInit *I = dyn_cast(&N)) - if (I->getNumArgs() > 0) - if (const DefInit *OpI = dyn_cast(I->getOperator())) - if (OpI->getDef()->isSubClassOf(Cls)) - return I; + if (const DefInit *OpI = dyn_cast(I->getOperator())) + if (OpI->getDef()->isSubClassOf(Cls)) + return I; return nullptr; } } // namespace llvm diff --git a/llvm/utils/TableGen/GlobalISelCombinerMatchTableEmitter.cpp b/llvm/utils/TableGen/GlobalISelCombinerMatchTableEmitter.cpp --- a/llvm/utils/TableGen/GlobalISelCombinerMatchTableEmitter.cpp +++ b/llvm/utils/TableGen/GlobalISelCombinerMatchTableEmitter.cpp @@ -9,6 +9,21 @@ /// \file Generate a combiner implementation for GlobalISel from a declarative /// syntax using GlobalISelMatchTable. /// +/// Usually, TableGen backends use "assert is an error" as a means to report +/// invalid input. They try to diagnose common case but don't try very hard and +/// crashes can be common. This backend aims to behave closer to how a language +/// compiler frontend would behave: we try extra hard to diagnose invalid inputs +/// early, and any crash should be considered a bug (= a feature or diagnostic +/// is missing). +/// +/// While this can make the backend a bit more complex than it needs to be, it +/// pays off because MIR patterns can get complicated. Giving useful error +/// messages to combine writers can help boost their productivity. +/// +/// As with anything, a good balance has to be found. We also don't want to +/// write hundreds of lines of code to detect edge cases. In practice, crashing +/// very occasionally, or giving poor errors in some rare instances, is fine. +/// //===----------------------------------------------------------------------===// #include "CodeGenInstruction.h" @@ -19,12 +34,14 @@ #include "GlobalISelMatchTable.h" #include "GlobalISelMatchTableExecutorEmitter.h" #include "SubtargetFeatureInfo.h" +#include "llvm/ADT/APInt.h" #include "llvm/ADT/Hashing.h" #include "llvm/ADT/MapVector.h" #include "llvm/ADT/Statistic.h" #include "llvm/ADT/StringSet.h" #include "llvm/Support/CommandLine.h" #include "llvm/Support/Debug.h" +#include "llvm/Support/PrettyStackTrace.h" #include "llvm/Support/ScopedPrinter.h" #include "llvm/TableGen/Error.h" #include "llvm/TableGen/Record.h" @@ -39,26 +56,54 @@ extern cl::list SelectedCombiners; extern cl::opt StopAfterParse; +extern cl::OptionCategory GICombinerEmitterCat; +static cl::opt DebugCXXPreds( + "gicombiner-matchtable-debug-cxxpreds", + cl::desc("Add Contextual/Debug comments to all C++ predicates"), + cl::cat(GICombinerEmitterCat)); namespace { constexpr StringLiteral CXXApplyPrefix = "GICXXCustomAction_CombineApply"; constexpr StringLiteral CXXPredPrefix = "GICXXPred_MI_Predicate_"; +constexpr StringLiteral PatFragClassName = "GICombinePatFrag"; std::string getIsEnabledPredicateEnumName(unsigned CombinerRuleID) { return "GICXXPred_Simple_IsRule" + to_string(CombinerRuleID) + "Enabled"; } +/// Copies a StringRef into a static pool to make sure it has a static lifetime. +StringRef insertStrRef(StringRef S) { + if (S.empty()) + return {}; + + static StringSet<> Pool; + auto [It, Inserted] = Pool.insert(S); + return It->getKey(); +} + void declareInstExpansion(CodeExpansions &CE, const InstructionMatcher &IM, StringRef Name) { CE.declare(Name, "State.MIs[" + to_string(IM.getInsnVarID()) + "]"); } +void declareInstExpansion(CodeExpansions &CE, const BuildMIAction &A, + StringRef Name) { + // Note: we use redeclare here because this may overwrite a matcher inst + // expansion. + CE.redeclare(Name, "OutMIs[" + to_string(A.getInsnID()) + "]"); +} + void declareOperandExpansion(CodeExpansions &CE, const OperandMatcher &OM, StringRef Name) { CE.declare(Name, "State.MIs[" + to_string(OM.getInsnVarID()) + "]->getOperand(" + to_string(OM.getOpIdx()) + ")"); } +void declareTempRegExpansion(CodeExpansions &CE, unsigned TempRegID, + StringRef Name) { + CE.declare(Name, "State.TempRegisters[" + to_string(TempRegID) + "]"); +} + template auto keys(Container &&C) { return map_range(C, [](auto &Entry) -> auto & { return Entry.first; }); } @@ -67,6 +112,11 @@ return map_range(C, [](auto &Entry) -> auto & { return Entry.second; }); } +LLTCodeGen getLLTCodeGenFromRecord(const Record *Ty) { + assert(Ty->isSubClassOf("ValueType")); + return LLTCodeGen(*MVTToLLT(getValueType(Ty))); +} + //===- MatchData Handling -------------------------------------------------===// /// Represents MatchData defined by the match stage and required by the apply @@ -148,19 +198,71 @@ //===- C++ Predicates Handling --------------------------------------------===// -/// Entry into the static pool of all CXX Predicate code. This contains the +/// Entry into the static pool of all CXX Predicate code. This contains /// fully expanded C++ code. /// -/// Each CXXPattern creates a new entry in the pool to store its data, even -/// after the pattern is destroyed. +/// The static pool is hidden inside the object and can be accessed through +/// getAllMatchCode/getAllApplyCode /// /// Note that CXXPattern trims C++ code, so the Code is already expected to be /// free of leading/trailing whitespace. -struct CXXPredicateCode { +class CXXPredicateCode { + using CXXPredicateCodePool = + DenseMap>; + static CXXPredicateCodePool AllCXXMatchCode; + static CXXPredicateCodePool AllCXXApplyCode; + + /// Sorts a `CXXPredicateCodePool` by their IDs and returns it. + static std::vector + getSorted(const CXXPredicateCodePool &Pool) { + std::vector Out; + std::transform(Pool.begin(), Pool.end(), std::back_inserter(Out), + [&](auto &Elt) { return Elt.second.get(); }); + sort(Out, [](const auto *A, const auto *B) { return A->ID < B->ID; }); + return Out; + } + + /// Gets an instance of `CXXPredicateCode` for \p Code, or returns an already + /// existing one. + static const CXXPredicateCode &get(CXXPredicateCodePool &Pool, + std::string Code) { + // Check if we already have an identical piece of code, if not, create an + // entry in the pool. + const auto CodeHash = hash_value(Code); + if (auto It = Pool.find(CodeHash); It != Pool.end()) + return *It->second; + + const auto ID = Pool.size(); + auto OwnedData = std::unique_ptr( + new CXXPredicateCode(std::move(Code), ID)); + const auto &DataRef = *OwnedData; + Pool[CodeHash] = std::move(OwnedData); + return DataRef; + } + CXXPredicateCode(std::string Code, unsigned ID) : Code(Code), ID(ID), BaseEnumName("GICombiner" + to_string(ID)) { - assert(StringRef(Code).trim() == Code && - "Code was expected to be trimmed!"); + // Don't assert if ErrorsPrinted is set. This may mean CodeExpander failed, + // and it may add spaces in such cases. + assert(ErrorsPrinted || StringRef(Code).trim() == Code && + "Code was expected to be trimmed!"); + } + +public: + static const CXXPredicateCode &getMatchCode(std::string Code) { + return get(AllCXXMatchCode, std::move(Code)); + } + + static const CXXPredicateCode &getApplyCode(std::string Code) { + return get(AllCXXApplyCode, std::move(Code)); + } + + static std::vector getAllMatchCode() { + return getSorted(AllCXXMatchCode); + } + + static std::vector getAllApplyCode() { + return getSorted(AllCXXApplyCode); } const std::string Code; @@ -176,48 +278,29 @@ } }; -using CXXPredicateCodePool = - DenseMap>; -CXXPredicateCodePool AllCXXMatchCode; -CXXPredicateCodePool AllCXXApplyCode; - -/// Gets an instance of `CXXPredicateCode` for \p Code, or returns an already -/// existing one. -const CXXPredicateCode &getOrInsert(CXXPredicateCodePool &Pool, - std::string Code) { - // Check if we already have an identical piece of code, if not, create an - // entry in the pool. - const auto CodeHash = hash_value(Code); - if (auto It = Pool.find(CodeHash); It != Pool.end()) - return *It->second; - - const auto ID = Pool.size(); - auto OwnedData = std::make_unique(std::move(Code), ID); - const auto &DataRef = *OwnedData; - Pool[CodeHash] = std::move(OwnedData); - return DataRef; -} - -/// Sorts a `CXXPredicateCodePool` by their IDs and returns it. -std::vector -getSorted(const CXXPredicateCodePool &Pool) { - std::vector Out; - std::transform(Pool.begin(), Pool.end(), std::back_inserter(Out), - [&](auto &Elt) { return Elt.second.get(); }); - sort(Out, [](const auto *A, const auto *B) { return A->ID < B->ID; }); - return Out; -} +CXXPredicateCode::CXXPredicateCodePool CXXPredicateCode::AllCXXMatchCode; +CXXPredicateCode::CXXPredicateCodePool CXXPredicateCode::AllCXXApplyCode; //===- Pattern Base Class -------------------------------------------------===// -// An abstract pattern found in a combine rule. This can be an apply or match -// pattern. +/// Base class for all patterns that can be written in an `apply`, `match` or +/// `pattern` DAG operator. +/// +/// For example: +/// +/// (apply (G_ZEXT $x, $y), (G_ZEXT $y, $z), "return isFoo(${z})") +/// +/// Creates 3 Pattern objects: +/// - Two CodeGenInstruction Patterns +/// - A CXXPattern class Pattern { public: enum { K_AnyOpcode, - K_Inst, K_CXX, + + K_CodeGenInstruction, + K_PatFrag, }; virtual ~Pattern() = default; @@ -232,7 +315,8 @@ void dump() const { return print(dbgs()); } protected: - Pattern(unsigned Kind, StringRef Name) : Kind(Kind), Name(Name.str()) { + Pattern(unsigned Kind, StringRef Name) + : Kind(Kind), Name(insertStrRef(Name)) { assert(!Name.empty() && "unnamed pattern!"); } @@ -241,21 +325,19 @@ private: unsigned Kind; - - // Note: if this ever changes to a StringRef (e.g. allocated in a pool or - // something), CombineRuleBuilder::verify() needs to be updated as well. - // It currently checks that the StringRef in the PatternMap references this. - std::string Name; + StringRef Name; }; const char *Pattern::getKindName() const { switch (Kind) { case K_AnyOpcode: return "AnyOpcodePattern"; - case K_Inst: - return "InstructionPattern"; case K_CXX: return "CXXPattern"; + case K_CodeGenInstruction: + return "CodeGenInstructionPattern"; + case K_PatFrag: + return "PatFragPattern"; } llvm_unreachable("unknown pattern kind!"); @@ -275,6 +357,9 @@ /// `wip_match_opcode` patterns. /// This matches one or more opcodes, and does not check any operands /// whatsoever. +/// +/// TODO: Long-term, this needs to be removed. It's a hack around MIR +/// pattern matching limitations. class AnyOpcodePattern : public Pattern { public: AnyOpcodePattern(StringRef Name) : Pattern(K_AnyOpcode, Name) {} @@ -300,77 +385,9 @@ }); } -//===- InstructionPattern -------------------------------------------------===// - -/// Matches an instruction, e.g. `G_ADD $x, $y, $z`. -/// -/// This pattern is simply CodeGenInstruction + a list of operands. -class InstructionPattern : public Pattern { -public: - struct Operand { - std::string Name; - bool IsDef = false; - }; - - InstructionPattern(const CodeGenInstruction &I, StringRef Name) - : Pattern(K_Inst, Name), I(I) {} - - static bool classof(const Pattern *P) { return P->getKind() == K_Inst; } - - const auto &operands() const { return Operands; } - void addOperand(StringRef Name); - unsigned getNumDefs() const { return I.Operands.NumDefs; } - - const CodeGenInstruction &getInst() const { return I; } - StringRef getInstName() const { return I.TheDef->getName(); } - - void reportUnreachable(ArrayRef Locs) const; - bool checkSemantics(ArrayRef Loc) const; - - void print(raw_ostream &OS, bool PrintName = true) const override; - -private: - const CodeGenInstruction &I; - SmallVector Operands; -}; - -void InstructionPattern::addOperand(StringRef Name) { - const bool IsDef = Operands.size() < getNumDefs(); - Operands.emplace_back(Operand{Name.str(), IsDef}); -} - -void InstructionPattern::reportUnreachable(ArrayRef Locs) const { - PrintError(Locs, "Instruction pattern '" + getName() + - "' is unreachable from the pattern root!"); -} - -bool InstructionPattern::checkSemantics(ArrayRef Loc) const { - unsigned NumExpectedOperands = I.Operands.size(); - if (NumExpectedOperands != Operands.size()) { - - PrintError(Loc, "'" + getInstName() + "' expected " + - Twine(NumExpectedOperands) + " operands, got " + - Twine(Operands.size())); - return false; - } - return true; -} - -void InstructionPattern::print(raw_ostream &OS, bool PrintName) const { - printImpl(OS, PrintName, [&OS, this]() { - OS << "inst:" << I.TheDef->getName() << " operands:[" - << join(map_range(Operands, - [](const auto &O) { - return (O.IsDef ? "" : "") + O.Name; - }), - ", ") - << "]"; - }); -} - //===- CXXPattern ---------------------------------------------------------===// -/// Raw C++ code which may need some expansions. +/// Represents raw C++ code which may need some expansions. /// /// e.g. [{ return isFooBux(${src}.getReg()); }] /// @@ -392,15 +409,15 @@ /// `.startswith("return")` trivially without worrying about spaces. class CXXPattern : public Pattern { public: - CXXPattern(const StringInit &Code, StringRef Name, bool IsApply) - : CXXPattern(Code.getAsUnquotedString(), Name, IsApply) {} + CXXPattern(const StringInit &Code, StringRef Name) + : CXXPattern(Code.getAsUnquotedString(), Name) {} - CXXPattern(StringRef Code, StringRef Name, bool IsApply) - : Pattern(K_CXX, Name), IsApply(IsApply), RawCode(Code.trim().str()) {} + CXXPattern(StringRef Code, StringRef Name) + : Pattern(K_CXX, Name), RawCode(Code.trim().str()) {} static bool classof(const Pattern *P) { return P->getKind() == K_CXX; } - bool isApply() const { return IsApply; } + void setIsApply(bool Value = true) { IsApply = Value; } StringRef getRawCode() const { return RawCode; } /// Expands raw code, replacing things such as `${foo}` with their @@ -409,27 +426,37 @@ /// \param CE Map of Code Expansions /// \param Locs SMLocs for the Code Expander, in case it needs to emit /// diagnostics. + /// \param AddComment If DebugCXXPreds is enabled, this is called to emit a + /// comment before the expanded code. + /// /// \return A CXXPredicateCode object that contains the expanded code. Note /// that this may or may not insert a new object. All CXXPredicateCode objects /// are held in a set to avoid emitting duplicate C++ code. - const CXXPredicateCode &expandCode(const CodeExpansions &CE, - ArrayRef Locs) const; + const CXXPredicateCode & + expandCode(const CodeExpansions &CE, ArrayRef Locs, + function_ref AddComment = {}) const; void print(raw_ostream &OS, bool PrintName = true) const override; private: - bool IsApply; + bool IsApply = false; std::string RawCode; }; -const CXXPredicateCode &CXXPattern::expandCode(const CodeExpansions &CE, - ArrayRef Locs) const { +const CXXPredicateCode & +CXXPattern::expandCode(const CodeExpansions &CE, ArrayRef Locs, + function_ref AddComment) const { std::string Result; raw_string_ostream OS(Result); + + if (DebugCXXPreds && AddComment) + AddComment(OS); + CodeExpander Expander(RawCode, CE, Locs, /*ShowExpansions*/ false); Expander.emit(OS); - return getOrInsert(IsApply ? AllCXXApplyCode : AllCXXMatchCode, - std::move(Result)); + if (IsApply) + return CXXPredicateCode::getApplyCode(std::move(Result)); + return CXXPredicateCode::getMatchCode(std::move(Result)); } void CXXPattern::print(raw_ostream &OS, bool PrintName) const { @@ -440,233 +467,1230 @@ }); } -//===- CombineRuleBuilder -------------------------------------------------===// +//===- InstructionPattern ---------------------------------------------===// -/// Helper for CombineRuleBuilder. -/// -/// Represents information about an operand. -/// Operands with no MatchPat are considered live-in to the pattern. -struct OperandTableEntry { - // The matcher pattern that defines this operand. - // null for live-ins. - InstructionPattern *MatchPat = nullptr; - // The apply pattern that (re)defines this operand. - // This can only be non-null if MatchPat is. - InstructionPattern *ApplyPat = nullptr; +/// Base class for CodeGenInstructionPattern & PatFragPattern, which handles all +/// the boilerplate for patterns that have a list of operands for some (pseudo) +/// instruction. +class InstructionPattern : public Pattern { +public: + class Operand; - bool isLiveIn() const { return !MatchPat; } -}; + virtual ~InstructionPattern(); -/// Parses combine rule and builds a small intermediate representation to tie -/// patterns together and emit RuleMatchers to match them. This may emit more -/// than one RuleMatcher, e.g. for `wip_match_opcode`. -/// -/// Memory management for `Pattern` objects is done through `std::unique_ptr`. -/// In most cases, there are two stages to a pattern's lifetime: -/// - Creation in a `parse` function -/// - The unique_ptr is stored in a variable, and may be destroyed if the -/// pattern is found to be semantically invalid. -/// - Ownership transfer into a `PatternMap` -/// - Once a pattern is moved into either the map of Match or Apply -/// patterns, it is known to be valid and it never moves back. -class CombineRuleBuilder { -public: - using PatternMap = MapVector>; + static bool classof(const Pattern *P) { + return P->getKind() == K_CodeGenInstruction || P->getKind() == K_PatFrag; + } - CombineRuleBuilder(const CodeGenTarget &CGT, - SubtargetFeatureInfoMap &SubtargetFeatures, - Record &RuleDef, unsigned ID, - std::vector &OutRMs) - : CGT(CGT), SubtargetFeatures(SubtargetFeatures), RuleDef(RuleDef), - RuleID(ID), OutRMs(OutRMs) {} + template void addOperand(Ty &&...Init) { + Operands.emplace_back(std::forward(Init)...); + } - /// Parses all fields in the RuleDef record. - bool parseAll(); + auto &operands() { return Operands; } + const auto &operands() const { return Operands; } + unsigned operands_size() const { return Operands.size(); } + Operand &getOperand(unsigned K) { return Operands[K]; } + const Operand &getOperand(unsigned K) const { return Operands[K]; } - /// Emits all RuleMatchers into the vector of RuleMatchers passed in the - /// constructor. - bool emitRuleMatchers(); + virtual bool isVariadic() const { return false; } + virtual unsigned getNumInstOperands() const = 0; + virtual unsigned getNumInstDefs() const = 0; - void print(raw_ostream &OS) const; - void dump() const { print(dbgs()); } + bool hasAllDefs() const { return operands_size() >= getNumInstDefs(); } - /// Debug-only verification of invariants. - void verify() const; + virtual StringRef getInstName() const = 0; -private: - void PrintError(Twine Msg) const { ::PrintError(RuleDef.getLoc(), Msg); } + void reportUnreachable(ArrayRef Locs) const; + virtual bool checkSemantics(ArrayRef Loc); - /// Adds the expansions from \see MatchDatas to \p CE. - void declareAllMatchDatasExpansions(CodeExpansions &CE) const; + void print(raw_ostream &OS, bool PrintName = true) const override; - /// Adds \p P to \p IM, expanding its code using \p CE. - void addCXXPredicate(InstructionMatcher &IM, const CodeExpansions &CE, - const CXXPattern &P); +protected: + InstructionPattern(unsigned K, StringRef Name) : Pattern(K, Name) {} - /// Generates a name for anonymous patterns. - /// - /// e.g. (G_ADD $x, $y, $z):$foo is a pattern named "foo", but if ":$foo" is - /// absent, then the pattern is anonymous and this is used to assign it a - /// name. - std::string makeAnonPatName(StringRef Prefix) const; - mutable unsigned AnonIDCnt = 0; + // std::vector is used here because we can instantiate it using a forward + // declaration. + std::vector Operands; +}; - /// Creates a new RuleMatcher with some boilerplate - /// settings/actions/predicates, and and adds it to \p OutRMs. - /// \see addFeaturePredicates too. - /// - /// \param AdditionalComment Comment string to be added to the - /// `DebugCommentAction`. - RuleMatcher &addRuleMatcher(Twine AdditionalComment = ""); - bool addFeaturePredicates(RuleMatcher &M); +/// An operand for an InstructionPattern. +/// +/// Operands are composed of three elements: +/// - (Optional) Value +/// - (Optional) Name +/// - (Optional) Type +/// +/// Some examples: +/// (i32 0):$x -> V=int(0), Name='x', Type=i32 +/// 0:$x -> V=int(0), Name='x' +/// $x -> Name='x' +/// i32:$x -> Name='x', Type = i32 +class InstructionPattern::Operand { +public: + using IntImmTy = int64_t; - bool findRoots(); - bool buildOperandsTable(); + Operand(IntImmTy Imm, StringRef Name, const Record *Type) + : Value(Imm), Name(insertStrRef(Name)), Type(Type) { + assert(!Type || Type->isSubClassOf("ValueType")); + } - bool parseDefs(DagInit &Def); - bool parseMatch(DagInit &Match); - bool parseApply(DagInit &Apply); + Operand(StringRef Name, const Record *Type) + : Name(insertStrRef(Name)), Type(Type) {} - std::unique_ptr parseInstructionMatcher(const Init &Arg, - StringRef PatName); - std::unique_ptr parseWipMatchOpcodeMatcher(const Init &Arg, - StringRef PatName); + bool isNamedImmediate() const { return hasImmValue() && isNamedOperand(); } - bool emitMatchPattern(CodeExpansions &CE, const InstructionPattern &IP); - bool emitMatchPattern(CodeExpansions &CE, const AnyOpcodePattern &AOP); + bool hasImmValue() const { return Value.has_value(); } + IntImmTy getImmValue() const { return *Value; } - bool emitApplyPatterns(CodeExpansions &CE, RuleMatcher &M); + bool isNamedOperand() const { return !Name.empty(); } + StringRef getOperandName() const { + assert(isNamedOperand() && "Operand is unnamed"); + return Name; + } - // Recursively visits InstructionPattern from P to build up the - // RuleMatcher/InstructionMatcher. May create new InstructionMatchers as - // needed. - bool emitInstructionMatchPattern(CodeExpansions &CE, RuleMatcher &M, - InstructionMatcher &IM, - const InstructionPattern &P, - DenseSet &SeenPats); + Operand withNewName(StringRef NewName) const { + Operand Result = *this; + Result.Name = insertStrRef(NewName); + return Result; + } - const CodeGenTarget &CGT; - SubtargetFeatureInfoMap &SubtargetFeatures; - Record &RuleDef; - const unsigned RuleID; - std::vector &OutRMs; + void setIsDef(bool Value = true) { Def = Value; } + bool isDef() const { return Def; } - // For InstructionMatcher::addOperand - unsigned AllocatedTemporariesBaseID = 0; + void setType(const Record *R) { + assert((!Type || (Type == R)) && "Overwriting type!"); + Type = R; + } + const Record *getType() const { return Type; } + + std::string describe() const { + if (!hasImmValue()) + return "MachineOperand $" + getOperandName().str() + ""; + std::string Str = "imm " + to_string(getImmValue()); + if (isNamedImmediate()) + Str += ":$" + getOperandName().str() + ""; + return Str; + } - /// The root of the pattern. - StringRef RootName; + void print(raw_ostream &OS) const { + if (isDef()) + OS << ""; + + bool NeedsColon = true; + if (const Record *Ty = getType()) { + if (hasImmValue()) + OS << "(" << Ty->getName() << " " << getImmValue() << ")"; + else + OS << Ty->getName(); + } else if (hasImmValue()) + OS << getImmValue(); + else + NeedsColon = false; - /// These maps have ownership of the actual Pattern objects. - /// They both map a Pattern's name to the Pattern instance. - PatternMap MatchPats; - PatternMap ApplyPats; + if (isNamedOperand()) + OS << (NeedsColon ? ":" : "") << "$" << getOperandName(); + } - /// Set by findRoots. - Pattern *MatchRoot = nullptr; + void dump() const { return print(dbgs()); } - MapVector OperandTable; - SmallVector MatchDatas; +private: + std::optional Value; + StringRef Name; + const Record *Type = nullptr; + bool Def = false; }; -bool CombineRuleBuilder::parseAll() { - if (!parseDefs(*RuleDef.getValueAsDag("Defs"))) - return false; - if (!parseMatch(*RuleDef.getValueAsDag("Match"))) - return false; - if (!parseApply(*RuleDef.getValueAsDag("Apply"))) - return false; - if (!buildOperandsTable()) - return false; - if (!findRoots()) - return false; - LLVM_DEBUG(verify()); - return true; +InstructionPattern::~InstructionPattern() = default; + +void InstructionPattern::reportUnreachable(ArrayRef Locs) const { + PrintError(Locs, "pattern '" + getName() + "' ('" + getInstName() + + "') is unreachable from the pattern root!"); } -bool CombineRuleBuilder::emitRuleMatchers() { - assert(MatchRoot); - CodeExpansions CE; - declareAllMatchDatasExpansions(CE); +bool InstructionPattern::checkSemantics(ArrayRef Loc) { + unsigned NumExpectedOperands = getNumInstOperands(); - switch (MatchRoot->getKind()) { - case Pattern::K_AnyOpcode: { - if (!emitMatchPattern(CE, *cast(MatchRoot))) - return false; - break; - } - case Pattern::K_Inst: - if (!emitMatchPattern(CE, *cast(MatchRoot))) + if (isVariadic()) { + if (Operands.size() < NumExpectedOperands) { + PrintError(Loc, +"'" + getInstName() + "' expected at least " + + Twine(NumExpectedOperands) + " operands, got " + + Twine(Operands.size())); return false; - break; - case Pattern::K_CXX: - PrintError("C++ code cannot be the root of a pattern!"); + } + } else if (NumExpectedOperands != Operands.size()) { + PrintError(Loc, +"'" + getInstName() + "' expected " + + Twine(NumExpectedOperands) + " operands, got " + + Twine(Operands.size())); return false; - default: - llvm_unreachable("unknown pattern kind!"); } + unsigned OpIdx = 0; + unsigned NumDefs = getNumInstDefs(); + for (auto &O : Operands) + O.setIsDef(OpIdx++ < NumDefs); + return true; } -void CombineRuleBuilder::print(raw_ostream &OS) const { - OS << "(CombineRule name:" << RuleDef.getName() << " id:" << RuleID - << " root:" << RootName << "\n"; - - OS << " (MatchDatas "; - if (MatchDatas.empty()) - OS << ")\n"; - else { - OS << "\n"; - for (const auto &MD : MatchDatas) { - OS << " "; - MD.print(OS); - OS << "\n"; +void InstructionPattern::print(raw_ostream &OS, bool PrintName) const { + printImpl(OS, PrintName, [&OS, this] { + OS << getInstName() << " operands:["; + StringRef Sep = ""; + for (const auto &O : Operands) { + OS << Sep; + O.print(OS); + Sep = ", "; } - OS << " )\n"; + OS << "]"; + }); +} + +//===- CodeGenInstructionPattern ------------------------------------------===// + +/// Matches an instruction, e.g. `G_ADD $x, $y, $z`. +class CodeGenInstructionPattern : public InstructionPattern { +public: + CodeGenInstructionPattern(const CodeGenInstruction &I, StringRef Name) + : InstructionPattern(K_CodeGenInstruction, Name), I(I) {} + + static bool classof(const Pattern *P) { + return P->getKind() == K_CodeGenInstruction; } - const auto DumpPats = [&](StringRef Name, const PatternMap &Pats) { - OS << " (" << Name << " "; - if (Pats.empty()) { - OS << ")\n"; - return; - } + bool hasVariadicDefs() const; + bool isVariadic() const override { return I.Operands.isVariadic; } + unsigned getNumInstDefs() const override; + unsigned getNumInstOperands() const override; - OS << "\n"; - for (const auto &[Name, Pat] : Pats) { - OS << " "; - if (Pat.get() == MatchRoot) - OS << ""; - OS << Name << ":"; - Pat->print(OS, /*PrintName=*/false); - OS << "\n"; - } - OS << " )\n"; - }; + const CodeGenInstruction &getInst() const { return I; } + StringRef getInstName() const override { return I.TheDef->getName(); } - DumpPats("MatchPats", MatchPats); - DumpPats("ApplyPats", ApplyPats); +private: + const CodeGenInstruction &I; +}; - OS << " (OperandTable "; - if (OperandTable.empty()) - OS << ")\n"; - else { - OS << "\n"; - for (const auto &[Key, Val] : OperandTable) { - OS << " [" << Key; - if (const auto *P = Val.MatchPat) - OS << " match_pat:" << P->getName(); - if (const auto *P = Val.ApplyPat) - OS << " apply_pat:" << P->getName(); - if (Val.isLiveIn()) - OS << " live-in"; - OS << "]\n"; - } - OS << " )\n"; - } +bool CodeGenInstructionPattern::hasVariadicDefs() const { + // Note: we cannot use variadicOpsAreDefs, it's not set for + // GenericInstructions. + if (!isVariadic()) + return false; - OS << ")\n"; + if (I.variadicOpsAreDefs) + return true; + + DagInit *OutOps = I.TheDef->getValueAsDag("OutOperandList"); + if (OutOps->arg_empty()) + return false; + + auto *LastArgTy = dyn_cast(OutOps->getArg(OutOps->arg_size() - 1)); + return LastArgTy && LastArgTy->getDef()->getName() == "variable_ops"; +} + +unsigned CodeGenInstructionPattern::getNumInstDefs() const { + if (!isVariadic() || !hasVariadicDefs()) + return I.Operands.NumDefs; + unsigned NumOuts = I.Operands.size() - I.Operands.NumDefs; + assert(Operands.size() > NumOuts); + return std::max(I.Operands.NumDefs, Operands.size() - NumOuts); +} + +unsigned CodeGenInstructionPattern::getNumInstOperands() const { + unsigned NumCGIOps = I.Operands.size(); + return isVariadic() ? std::max(NumCGIOps, Operands.size()) + : NumCGIOps; +} + +//===- OperandTable -------------------------------------------------------===// + +/// Represents information about an operand. +/// Operands with no MatchPat are considered live-in to the pattern. +/// 'live-in' means that we're not sure what defines this operand/what it is. +/// We just know its name to refer to it. +/// +/// e.g. (G_SEXT $x, $y) -> $y is a live-in. $y could be MBB ref, an immediate, +/// a flying squirrel or just a register, we don't know. All we know is that +/// when we refer to $y, we refer to that G_SEXT's second operand. +struct OperandTableEntry { + // The matcher pattern that defines this operand. + // null for live-ins. + InstructionPattern *MatchPat = nullptr; + // The apply pattern that (re)defines this operand. + // This can only be non-null if MatchPat is. + CodeGenInstructionPattern *ApplyPat = nullptr; + // If MatchPat is null, whether this is a live-in of the match pattern. + bool IsMatchLiveIn = false; +}; + +/// The Operand Table holds information about the operands in a set of Patterns. +/// This is only useful for Instruction patterns. +/// +/// This is a simple Key -> Value map where the Key is an operand's name, and +/// the Value is an OperandTableEntry, a small struct holding information about +/// the operand. +/// +/// MapVector is used to have a deterministic iteration order for dump-parse. +class OperandTable : public MapVector { +public: + bool addMatchPattern(ArrayRef DiagLoc, InstructionPattern *P, + StringRef ErrSuffix); + bool addApplyPattern(ArrayRef DiagLoc, CodeGenInstructionPattern *P); + + const InstructionPattern *getMatchDefPattern(StringRef OpName) const { + if (auto It = find(OpName); It != end()) + return It->second.MatchPat; + return nullptr; + } + + const CodeGenInstructionPattern *getApplyDefPattern(StringRef OpName) const { + if (auto It = find(OpName); It != end()) + return It->second.ApplyPat; + return nullptr; + } + + bool isMatchLiveInOrDef(StringRef OpName) const; + + void print(raw_ostream &OS, StringRef Indent = "") const; +}; + +bool OperandTable::addMatchPattern(ArrayRef DiagLoc, + InstructionPattern *P, StringRef ErrSuffix) { + for (const auto &Operand : P->operands()) { + if (!Operand.isNamedOperand()) + continue; + + StringRef OpName = Operand.getOperandName(); + + // Create an entry, no matter if it's a use or a def. + auto &Entry = (*this)[OpName]; + + if (Operand.isDef()) { + if (Entry.MatchPat) { + PrintError(DiagLoc, "Operand '" + OpName + + "' is defined multiple times " + ErrSuffix); + return false; + } + Entry.MatchPat = P; + Entry.IsMatchLiveIn = false; + } else if (!Entry.MatchPat) { + Entry.IsMatchLiveIn = true; + } + } + + return true; +} + +bool OperandTable::addApplyPattern(ArrayRef DiagLoc, + CodeGenInstructionPattern *P) { + for (const auto &Operand : P->operands()) { + if (!Operand.isNamedOperand()) + continue; + + StringRef OpName = Operand.getOperandName(); + // Create an entry, no matter if it's a use or a def. + auto &Entry = (*this)[OpName]; + + if (Operand.isNamedImmediate()) + continue; + + if (Operand.isDef()) { + if (!Entry.MatchPat && Entry.IsMatchLiveIn) { + PrintError(DiagLoc, "Cannot define live-in operand '" + OpName + + "' in the 'apply' pattern"); + return false; + } + if (Entry.ApplyPat) { + PrintError(DiagLoc, + "Operand '" + OpName + + "' is defined multiple times in the 'apply' patterns"); + return false; + } + Entry.ApplyPat = P; + } + } + + return true; +} + +bool OperandTable::isMatchLiveInOrDef(StringRef OpName) const { + const auto It = find(OpName); + if (It == end()) + return false; + const auto &Entry = It->second; + return Entry.MatchPat || Entry.IsMatchLiveIn; +} + +void OperandTable::print(raw_ostream &OS, StringRef Indent) const { + OS << Indent << "(OperandTable "; + if (empty()) { + OS << ")\n"; + return; + } + + OS << "\n"; + for (const auto &[Name, Val] : *this) { + OS << " [" << Name; + if (const auto *P = Val.MatchPat) + OS << " match_pat:" << P->getName(); + if (const auto *P = Val.ApplyPat) + OS << " apply_pat:" << P->getName(); + if (Val.IsMatchLiveIn) + OS << " match-live-in"; + OS << "]\n"; + } + OS << " )\n"; +} + +//===- OperandTypeChecker -------------------------------------------------===// + +/// This is a trivial type checker for all operands in a set of +/// InstructionPatterns. +/// +/// It infers the type of each operand, check it's consistent with the known +/// type of the operand, and then sets all of the types in all operands in +/// setAllOperandTypes. +class OperandTypeChecker { +public: + OperandTypeChecker(ArrayRef DiagLoc) : DiagLoc(DiagLoc) {} + + bool check(InstructionPattern *P); + + void setAllOperandTypes(); + +private: + struct OpTypeInfo { + const Record *Type = nullptr; + InstructionPattern *TypeSrc = nullptr; + }; + + ArrayRef DiagLoc; + StringMap Types; + + SmallVector Pats; +}; + +bool OperandTypeChecker::check(InstructionPattern *P) { + Pats.push_back(P); + + for (auto &O : P->operands()) { + const Record *Ty = O.getType(); + if (!O.isNamedOperand() || !Ty) + continue; + + auto &Info = Types[O.getOperandName()]; + + if (!Info.Type) { + Info.Type = Ty; + Info.TypeSrc = P; + continue; + } + + if (Info.Type != Ty) { + PrintError(DiagLoc, "conflicting types for operand '" + + O.getOperandName() + "': first seen with '" + + Info.Type->getName() + "' in '" + + Info.TypeSrc->getName() + ", now seen with '" + + Ty->getName() + "' in '" + P->getName() + "'"); + return false; + } + } + + return true; +} + +void OperandTypeChecker::setAllOperandTypes() { + for (auto *P : Pats) { + for (auto &O : P->operands()) { + if (!O.isNamedOperand()) + continue; + + if (auto &Info = Types[O.getOperandName()]; Info.Type) + O.setType(Info.Type); + } + } +} + +//===- PatFrag ------------------------------------------------------------===// + +/// Represents a parsed GICombinePatFrag. This can be thought of as the +/// equivalent of a CodeGenInstruction, but for PatFragPatterns. +/// +/// PatFrags are made of 3 things: +/// - Out parameters (defs) +/// - In parameters +/// - A set of pattern lists (alternatives). +/// +/// If the PatFrag uses instruction patterns, the root must be one of the defs. +/// +/// Note that this DOES NOT represent the use of the PatFrag, only its +/// definition. The use of the PatFrag in a Pattern is represented by +/// PatFragPattern. +/// +/// PatFrags use the term "parameter" instead of operand because they're +/// essentially macros, and using that name avoids confusion. Other than that, +/// they're structured similarly to a MachineInstruction - all parameters +/// (operands) are in the same list, with defs at the start. This helps mapping +/// parameters to values, because, param N of a PatFrag is always operand N of a +/// PatFragPattern. +class PatFrag { +public: + enum ParamKind { + PK_Root, + PK_MachineOperand, + PK_Imm, + }; + + struct Param { + StringRef Name; + ParamKind Kind; + }; + + using ParamVec = SmallVector; + using ParamIt = ParamVec::const_iterator; + + /// Represents an alternative of the PatFrag. When parsing a GICombinePatFrag, + /// this is created from its "Alternatives" list. Each alternative is a list + /// of patterns written wrapped in a `(pattern ...)` dag init. + /// + /// Each argument to the `pattern` DAG operator is parsed into a Pattern + /// instance. + struct Alternative { + OperandTable OpTable; + SmallVector, 4> Pats; + }; + + PatFrag(const Record &Def) : Def(Def) {} + + static StringRef getParamKindStr(ParamKind OK); + + StringRef getName() const { return Def.getName(); } + + const Record &getDef() const { return Def; } + ArrayRef getLoc() const { return Def.getLoc(); } + + Alternative &addAlternative() { return Alts.emplace_back(); } + const Alternative &getAlternative(unsigned K) const { return Alts[K]; } + unsigned num_alternatives() const { return Alts.size(); } + + void addInParam(StringRef Name, ParamKind Kind); + iterator_range in_params() const; + unsigned num_in_params() const { return Params.size() - NumOutParams; } + + void addOutParam(StringRef Name, ParamKind Kind); + iterator_range out_params() const; + unsigned num_out_params() const { return NumOutParams; } + + unsigned num_roots() const; + unsigned num_params() const { return num_in_params() + num_out_params(); } + + /// Finds the operand \p Name and returns its index or -1 if not found. + /// Remember that all params are part of the same list, with out params at the + /// start. This means that the index returned can be used to access operands + /// of InstructionPatterns. + unsigned getParamIdx(StringRef Name) const; + const Param &getParam(unsigned K) const { return Params[K]; } + + bool canBeMatchRoot() const { return num_roots() == 1; } + + void print(raw_ostream &OS, StringRef Indent = "") const; + void dump() const { print(dbgs()); } + + /// Checks if the in-param \p ParamName can be unbound or not. + /// \p ArgName is the name of the argument passed to the PatFrag. + /// + /// An argument can be unbound only if, for all alternatives: + /// - There is no CXX pattern, OR: + /// - There is an InstructionPattern that binds the parameter. + /// + /// e.g. in (MyPatFrag $foo), if $foo has never been seen before (= it's + /// unbound), this checks if MyPatFrag supports it or not. + bool handleUnboundInParam(StringRef ParamName, StringRef ArgName, + ArrayRef DiagLoc) const; + + bool checkSemantics(); + bool buildOperandsTables(); + +private: + static void printParamsList(raw_ostream &OS, iterator_range Params); + + void PrintError(Twine Msg) const { ::PrintError(&Def, Msg); } + + const Record &Def; + unsigned NumOutParams = 0; + ParamVec Params; + SmallVector Alts; +}; + +StringRef PatFrag::getParamKindStr(ParamKind OK) { + switch (OK) { + case PK_Root: + return "root"; + case PK_MachineOperand: + return "machine_operand"; + case PK_Imm: + return "imm"; + } + + llvm_unreachable("Unknown operand kind!"); +} + +void PatFrag::addInParam(StringRef Name, ParamKind Kind) { + Params.emplace_back(Param{insertStrRef(Name), Kind}); +} + +iterator_range PatFrag::in_params() const { + return {Params.begin() + NumOutParams, Params.end()}; +} + +void PatFrag::addOutParam(StringRef Name, ParamKind Kind) { + assert(NumOutParams == Params.size() && + "Adding out-param after an in-param!"); + Params.emplace_back(Param{insertStrRef(Name), Kind}); + ++NumOutParams; +} + +iterator_range PatFrag::out_params() const { + return {Params.begin(), Params.begin() + NumOutParams}; +} + +unsigned PatFrag::num_roots() const { + return count_if(out_params(), + [&](const auto &P) { return P.Kind == PK_Root; }); +} + +unsigned PatFrag::getParamIdx(StringRef Name) const { + for (const auto &[Idx, O] : enumerate(Params)) { + if (O.Name == Name) + return Idx; + } + + return -1; +} + +bool PatFrag::checkSemantics() { + for (const auto &A : Alts) { + for (const auto &Pat : A.Pats) { + switch (Pat->getKind()) { + case Pattern::K_AnyOpcode: + PrintError("wip_match_opcode cannot be used in " + PatFragClassName); + return false; + case Pattern::K_CXX: + case Pattern::K_CodeGenInstruction: + continue; + case Pattern::K_PatFrag: + // TODO: It's just that the emitter doesn't handle it but technically + // there is no reason why we can't. We just have to be careful with + // operand mappings, it could get complex. + PrintError("nested " + PatFragClassName + " are not supported"); + return false; + } + } + } + + StringSet<> SeenOps; + for (const auto &O : in_params()) { + if (SeenOps.count(O.Name)) { + PrintError("duplicate parameter '" + O.Name + "'"); + return false; + } + + // Check this operand is NOT defined in any alternative's patterns. + for (const auto &A : Alts) { + if (A.OpTable.getMatchDefPattern(O.Name)) { + PrintError("input parameter '" + O.Name + "' cannot be redefined!"); + return false; + } + } + + if (O.Kind == PK_Root) { + PrintError("input parameterr '" + O.Name + "' cannot be a root!"); + return false; + } + + SeenOps.insert(O.Name); + } + + for (const auto &O : out_params()) { + if (O.Kind != PK_Root && O.Kind != PK_MachineOperand) { + PrintError("output parameter '" + O.Name + "' must be 'root' or 'gi_mo'"); + return false; + } + + if (SeenOps.count(O.Name)) { + PrintError("duplicate parameter '" + O.Name + "'"); + return false; + } + + // Check this operand is defined in all alternative's patterns. + for (const auto &A : Alts) { + if (!A.OpTable.getMatchDefPattern(O.Name)) { + PrintError("output parameter '" + O.Name + + "' must be defined by all alternative patterns in '" + + Def.getName() + "'"); + return false; + } + } + + SeenOps.insert(O.Name); + } + + if (num_out_params() != 0 && num_roots() == 0) { + PrintError(PatFragClassName + " must have one root in its 'out' operands"); + return false; + } + + if (num_roots() > 1) { + PrintError(PatFragClassName + " can only have one root"); + return false; + } + + // TODO: find unused params + + // Now, typecheck all alternatives. + for (auto &A : Alts) { + OperandTypeChecker OTC(Def.getLoc()); + for (auto &Pat : A.Pats) { + if (auto *IP = dyn_cast(Pat.get())) { + if (!OTC.check(IP)) + return false; + } + } + OTC.setAllOperandTypes(); + } + + return true; +} + +bool PatFrag::handleUnboundInParam(StringRef ParamName, StringRef ArgName, + ArrayRef DiagLoc) const { + // The parameter must be a live-in of all alternatives for this to work. + // Otherwise, we risk having unbound parameters being used (= crashes). + // + // Examples: + // + // in (ins $y), (patterns (G_FNEG $dst, $y), "return matchFnegOp(${y})") + // even if $y is unbound, we'll lazily bind it when emitting the G_FNEG. + // + // in (ins $y), (patterns "return matchFnegOp(${y})") + // if $y is unbound when this fragment is emitted, C++ code expansion will + // fail. + for (const auto &A : Alts) { + if (!A.OpTable.isMatchLiveInOrDef(ParamName) && + any_of(A.Pats, + [](const auto &P) { return isa(P.get()); })) { + ::PrintError(DiagLoc, "operand '" + ArgName + "' (for parameter '" + + ParamName + "' of '" + getName() + + "') cannot be unbound"); + PrintNote( + DiagLoc, + "one or more alternatives of '" + getName() + "' do not bind '" + + ParamName + + "' to an instruction operand; either use a bound operand or " + "ensure '" + + Def.getName() + "' binds '" + ParamName + + "' in all alternatives"); + return false; + } + } + + return true; +} + +bool PatFrag::buildOperandsTables() { + // enumerate(...) doesn't seem to allow lvalues so we need to count the old + // way. + unsigned Idx = 0; + for (auto &A : Alts) { + for (auto &Pat : A.Pats) { + if (auto *IP = dyn_cast(Pat.get())) { + if (!A.OpTable.addMatchPattern(Def.getLoc(), IP, + "in patterns of alternative #" + + to_string(Idx))) { + return false; + } + } + } + + ++Idx; + } + + return true; +} + +void PatFrag::print(raw_ostream &OS, StringRef Indent) const { + OS << Indent << "(PatFrag name:" << getName() << "\n"; + if (!in_params().empty()) { + OS << Indent << " (ins "; + printParamsList(OS, in_params()); + OS << ")\n"; + } + + if (!out_params().empty()) { + OS << Indent << " (outs "; + printParamsList(OS, out_params()); + OS << ")\n"; + } + + // TODO: Dump OperandTable as well. + OS << Indent << " (alternatives [\n"; + for (const auto &A : Alts) { + OS << Indent << " [\n"; + for (const auto &P : A.Pats) { + OS << Indent << " "; + P->print(OS, /*PrintName=*/true); + OS << ",\n"; + } + OS << Indent << " ],\n"; + } + OS << Indent << " ])\n"; + + OS << Indent << ")"; +} + +void PatFrag::printParamsList(raw_ostream &OS, iterator_range Params) { + OS << "[" + << join(map_range(Params, + [](auto &O) { + return (O.Name + ":" + getParamKindStr(O.Kind)).str(); + }), + ", ") + << "]"; +} + +//===- PatFragPattern -----------------------------------------------------===// + +class PatFragPattern : public InstructionPattern { +public: + PatFragPattern(const PatFrag &PF, StringRef Name) + : InstructionPattern(K_PatFrag, Name), PF(PF) {} + + static bool classof(const Pattern *P) { return P->getKind() == K_PatFrag; } + + const PatFrag &getPatFrag() const { return PF; } + StringRef getInstName() const override { return PF.getName(); } + + unsigned getNumInstDefs() const override { return PF.num_out_params(); } + unsigned getNumInstOperands() const override { return PF.num_params(); } + + bool checkSemantics(ArrayRef DiagLoc) override; + + bool mapInputCodeExpansions(const CodeExpansions &ParentCEs, + CodeExpansions &PatFragCEs, + ArrayRef DiagLoc) const; + +private: + const PatFrag &PF; +}; + +bool PatFragPattern::checkSemantics(ArrayRef DiagLoc) { + InstructionPattern::checkSemantics(DiagLoc); + + for (const auto &[Idx, Op] : enumerate(Operands)) { + switch (PF.getParam(Idx).Kind) { + case PatFrag::PK_Imm: + if (!Op.hasImmValue()) { + PrintError(DiagLoc, "expected operand " + to_string(Idx) + " of '" + + getInstName() + "' to be an immediate; got " + + Op.describe()); + return false; + } + if (Op.isNamedImmediate()) { + PrintError(DiagLoc, "operand " + to_string(Idx) + " of '" + + getInstName() + + "' cannot be a named immediate"); + return false; + } + break; + case PatFrag::PK_Root: + case PatFrag::PK_MachineOperand: + if (!Op.isNamedOperand() || Op.isNamedImmediate()) { + PrintError(DiagLoc, "expected operand " + to_string(Idx) + " of '" + + getInstName() + + "' to be a MachineOperand; got " + + Op.describe()); + return false; + } + break; + } + } + + return true; +} + +bool PatFragPattern::mapInputCodeExpansions(const CodeExpansions &ParentCEs, + CodeExpansions &PatFragCEs, + ArrayRef DiagLoc) const { + for (const auto &[Idx, O] : enumerate(operands())) { + StringRef ParamName = PF.getParam(Idx).Name; + + // Operands to a PFP can only be named, or be an immediate, but not a named + // immediate. + assert(!O.isNamedImmediate()); + + if (O.isNamedOperand()) { + StringRef ArgName = O.getOperandName(); + // Map it only if it's been defined. + auto It = ParentCEs.find(ArgName); + if (It == ParentCEs.end()) { + if (!PF.handleUnboundInParam(ParamName, ArgName, DiagLoc)) + return false; + } else + PatFragCEs.declare(ParamName, It->second); + continue; + } + + if (O.hasImmValue()) { + PatFragCEs.declare(ParamName, to_string(O.getImmValue())); + continue; + } + + llvm_unreachable("Unknown Operand Type!"); + } + + return true; +} + +//===- PrettyStackTrace Helpers ------------------------------------------===// + +class PrettyStackTraceParse : public PrettyStackTraceEntry { + const Record &Def; + +public: + PrettyStackTraceParse(const Record &Def) : Def(Def) {} + + void print(raw_ostream &OS) const override { + if (Def.isSubClassOf("GICombineRule")) + OS << "Parsing GICombineRule '" << Def.getName() << "'"; + else if (Def.isSubClassOf(PatFragClassName)) + OS << "Parsing " << PatFragClassName << " '" << Def.getName() << "'"; + else + OS << "Parsing '" << Def.getName() << "'"; + OS << "\n"; + } +}; + +class PrettyStackTraceEmit : public PrettyStackTraceEntry { + const Record &Def; + const Pattern *Pat = nullptr; + +public: + PrettyStackTraceEmit(const Record &Def, const Pattern *Pat = nullptr) + : Def(Def), Pat(Pat) {} + + void print(raw_ostream &OS) const override { + if (Def.isSubClassOf("GICombineRule")) + OS << "Emitting GICombineRule '" << Def.getName() << "'"; + else if (Def.isSubClassOf(PatFragClassName)) + OS << "Emitting " << PatFragClassName << " '" << Def.getName() << "'"; + else + OS << "Emitting '" << Def.getName() << "'"; + + if (Pat) + OS << " [" << Pat->getKindName() << " '" << Pat->getName() << "']"; + OS << "\n"; + } +}; + +//===- CombineRuleBuilder -------------------------------------------------===// + +/// Parses combine rule and builds a small intermediate representation to tie +/// patterns together and emit RuleMatchers to match them. This may emit more +/// than one RuleMatcher, e.g. for `wip_match_opcode`. +/// +/// Memory management for `Pattern` objects is done through `std::unique_ptr`. +/// In most cases, there are two stages to a pattern's lifetime: +/// - Creation in a `parse` function +/// - The unique_ptr is stored in a variable, and may be destroyed if the +/// pattern is found to be semantically invalid. +/// - Ownership transfer into a `PatternMap` +/// - Once a pattern is moved into either the map of Match or Apply +/// patterns, it is known to be valid and it never moves back. +class CombineRuleBuilder { +public: + using PatternMap = MapVector>; + using PatternAlternatives = DenseMap; + + CombineRuleBuilder(const CodeGenTarget &CGT, + SubtargetFeatureInfoMap &SubtargetFeatures, + Record &RuleDef, unsigned ID, + std::vector &OutRMs) + : CGT(CGT), SubtargetFeatures(SubtargetFeatures), RuleDef(RuleDef), + RuleID(ID), OutRMs(OutRMs) {} + + /// Parses all fields in the RuleDef record. + bool parseAll(); + + /// Emits all RuleMatchers into the vector of RuleMatchers passed in the + /// constructor. + bool emitRuleMatchers(); + + void print(raw_ostream &OS) const; + void dump() const { print(dbgs()); } + + /// Debug-only verification of invariants. + void verify() const; + +private: + void PrintError(Twine Msg) const { ::PrintError(&RuleDef, Msg); } + void PrintWarning(Twine Msg) const { ::PrintWarning(RuleDef.getLoc(), Msg); } + void PrintNote(Twine Msg) const { ::PrintNote(RuleDef.getLoc(), Msg); } + + void print(raw_ostream &OS, const PatternAlternatives &Alts) const; + + bool addApplyPattern(std::unique_ptr Pat); + bool addMatchPattern(std::unique_ptr Pat); + + /// Adds the expansions from \see MatchDatas to \p CE. + void declareAllMatchDatasExpansions(CodeExpansions &CE) const; + + /// Adds a matcher \p P to \p IM, expanding its code using \p CE. + /// Note that the predicate is added on the last InstructionMatcher. + /// + /// \p Alts is only used if DebugCXXPreds is enabled. + void addCXXPredicate(RuleMatcher &M, const CodeExpansions &CE, + const CXXPattern &P, const PatternAlternatives &Alts); + + /// Adds an apply \p P to \p IM, expanding its code using \p CE. + void addCXXAction(RuleMatcher &M, const CodeExpansions &CE, + const CXXPattern &P); + + bool hasOnlyCXXApplyPatterns() const; + bool hasAnyOpcodeMatchPattern() const { + return any_of(MatchPats, [&](const auto &E) { + return isa(E.second.get()); + }); + } + + // Infer machine operand types and check their consistency. + bool typecheckPatterns(); + + /// For all PatFragPatterns, add a new entry in PatternAlternatives for each + /// PatternList it contains. This is multiplicative, so if we have 2 + /// PatFrags with 3 alternatives each, we get 2*3 permutations added to + /// PermutationsToEmit. The "MaxPermutations" field controls how many + /// permutations are allowed before an error is emitted and this function + /// returns false. This is a simple safeguard to prevent combination of + /// PatFrags from generating enormous amounts of rules. + bool buildPermutationsToEmit(); + + /// Generates a name for anonymous patterns. + /// + /// e.g. (G_ADD $x, $y, $z):$foo is a pattern named "foo", but if ":$foo" is + /// absent, then the pattern is anonymous and this is used to assign it a + /// name. + std::string makeAnonPatName(StringRef Prefix) const; + mutable unsigned AnonIDCnt = 0; + + /// Creates a new RuleMatcher with some boilerplate + /// settings/actions/predicates, and and adds it to \p OutRMs. + /// \see addFeaturePredicates too. + /// + /// \param Alts Current set of alternatives, for debug comment. + /// \param AdditionalComment Comment string to be added to the + /// `DebugCommentAction`. + RuleMatcher &addRuleMatcher(const PatternAlternatives &Alts, + Twine AdditionalComment = ""); + bool addFeaturePredicates(RuleMatcher &M); + + bool findRoots(); + bool buildRuleOperandsTable(); + + bool parseDefs(const DagInit &Def); + bool + parsePatternList(const DagInit &List, + function_ref)> ParseAction, + StringRef Operator, ArrayRef DiagLoc) const; + + std::unique_ptr parseInstructionPattern(const Init &Arg, + StringRef PatName) const; + std::unique_ptr parseWipMatchOpcodeMatcher(const Init &Arg, + StringRef PatName) const; + bool parseInstructionPatternOperand(InstructionPattern &IP, + const Init *OpInit, + const StringInit *OpName) const; + std::unique_ptr parsePatFragImpl(const Record *Def) const; + bool parsePatFragParamList( + ArrayRef DiagLoc, const DagInit &OpsList, + function_ref ParseAction) const; + const PatFrag *parsePatFrag(const Record *Def) const; + + bool emitMatchPattern(CodeExpansions &CE, const PatternAlternatives &Alts, + const InstructionPattern &IP); + bool emitMatchPattern(CodeExpansions &CE, const PatternAlternatives &Alts, + const AnyOpcodePattern &AOP); + + bool emitPatFragMatchPattern(CodeExpansions &CE, + const PatternAlternatives &Alts, RuleMatcher &RM, + InstructionMatcher *IM, + const PatFragPattern &PFP, + DenseSet &SeenPats); + + bool emitApplyPatterns(CodeExpansions &CE, RuleMatcher &M); + bool emitTempRegisterActions(CodeExpansions &CE, RuleMatcher &M, + StringMap &OperandToTempRegID) const; + + // Recursively visits CodeGenInstructionPattern from P to build up the + // RuleMatcher actions. + bool emitCodeGenInstructionApplyPattern( + CodeExpansions &CE, RuleMatcher &M, const CodeGenInstructionPattern &P, + DenseSet &SeenPats, + const StringMap &OperandToTempRegID); + + // Recursively visits CodeGenInstructionPattern from P to build up the + // RuleMatcher/InstructionMatcher. May create new InstructionMatchers as + // needed. + using OperandMapperFnRef = function_ref; + bool emitCodeGenInstructionMatchPattern( + CodeExpansions &CE, const PatternAlternatives &Alts, + const OperandTable &OpTable, RuleMatcher &M, InstructionMatcher &IM, + const CodeGenInstructionPattern &P, DenseSet &SeenPats, + OperandMapperFnRef OperandMapper = [](const auto &O) { return O; }); + + const CodeGenTarget &CGT; + SubtargetFeatureInfoMap &SubtargetFeatures; + Record &RuleDef; + const unsigned RuleID; + std::vector &OutRMs; + + // For InstructionMatcher::addOperand + unsigned AllocatedTemporariesBaseID = 0; + + /// The root of the pattern. + StringRef RootName; + + /// These maps have ownership of the actual Pattern objects. + /// They both map a Pattern's name to the Pattern instance. + PatternMap MatchPats; + PatternMap ApplyPats; + + /// Set by findRoots. + Pattern *MatchRoot = nullptr; + SmallVector ApplyRoots; + + // Operand table for the whole rule. + OperandTable RuleOpTable; + + SmallVector MatchDatas; + SmallVector PermutationsToEmit; + + // print()/debug-only members. + mutable SmallPtrSet SeenPatFrags; +}; + +bool CombineRuleBuilder::parseAll() { + auto StackTrace = PrettyStackTraceParse(RuleDef); + + if (!parseDefs(*RuleDef.getValueAsDag("Defs"))) + return false; + + if (!parsePatternList( + *RuleDef.getValueAsDag("Match"), + [this](auto Pat) { return addMatchPattern(std::move(Pat)); }, "match", + RuleDef.getLoc())) + return false; + + if (!parsePatternList( + *RuleDef.getValueAsDag("Apply"), + [this](auto Pat) { return addApplyPattern(std::move(Pat)); }, "apply", + RuleDef.getLoc())) + return false; + + if (!buildRuleOperandsTable() || !typecheckPatterns() || !findRoots() || + !buildPermutationsToEmit()) + return false; + LLVM_DEBUG(verify()); + return true; +} + +bool CombineRuleBuilder::emitRuleMatchers() { + auto StackTrace = PrettyStackTraceEmit(RuleDef); + + assert(MatchRoot); + CodeExpansions CE; + declareAllMatchDatasExpansions(CE); + + assert(!PermutationsToEmit.empty()); + for (const auto &Alts : PermutationsToEmit) { + switch (MatchRoot->getKind()) { + case Pattern::K_AnyOpcode: { + if (!emitMatchPattern(CE, Alts, *cast(MatchRoot))) + return false; + break; + } + case Pattern::K_PatFrag: + case Pattern::K_CodeGenInstruction: + if (!emitMatchPattern(CE, Alts, *cast(MatchRoot))) + return false; + break; + case Pattern::K_CXX: + PrintError("C++ code cannot be the root of a rule!"); + return false; + default: + llvm_unreachable("unknown pattern kind!"); + } + } + + return true; +} + +void CombineRuleBuilder::print(raw_ostream &OS) const { + OS << "(CombineRule name:" << RuleDef.getName() << " id:" << RuleID + << " root:" << RootName << "\n"; + + if (!MatchDatas.empty()) { + OS << " (MatchDatas\n"; + for (const auto &MD : MatchDatas) { + OS << " "; + MD.print(OS); + OS << "\n"; + } + OS << " )\n"; + } + + if (!SeenPatFrags.empty()) { + OS << " (PatFrags\n"; + for (const auto *PF : SeenPatFrags) { + PF->print(OS, /*Indent=*/" "); + OS << "\n"; + } + OS << " )\n"; + } + + const auto DumpPats = [&](StringRef Name, const PatternMap &Pats) { + OS << " (" << Name << " "; + if (Pats.empty()) { + OS << ")\n"; + return; + } + + OS << "\n"; + for (const auto &[Name, Pat] : Pats) { + OS << " "; + if (Pat.get() == MatchRoot) + OS << ""; + OS << Name << ":"; + Pat->print(OS, /*PrintName=*/false); + OS << "\n"; + } + OS << " )\n"; + }; + + DumpPats("MatchPats", MatchPats); + DumpPats("ApplyPats", ApplyPats); + + RuleOpTable.print(OS, /*Indent=*/" "); + + if (PermutationsToEmit.size() > 1) { + OS << " (PermutationsToEmit\n"; + for (const auto &P : PermutationsToEmit) { + OS << " "; + print(OS, P); + OS << ",\n"; + } + OS << " )\n"; + } + + OS << ")\n"; } void CombineRuleBuilder::verify() const { @@ -681,9 +1705,8 @@ ", Pat name: " + Pat->getName()); } - // As an optimization, the PatternMaps don't re-allocate the PatternName - // string. They simply reference the std::string inside Pattern. Ensure - // this is the case to avoid memory issues. + // Sanity check: the map should point to the same data as the Pattern. + // Both strings are allocated in the pool using insertStrRef. if (Name.data() != Pat->getName().data()) { dbgs() << "Map StringRef: '" << Name << "' @ " << (const void *)Name.data() << "\n"; @@ -698,40 +1721,99 @@ VerifyPats(MatchPats); VerifyPats(ApplyPats); - for (const auto &[Name, Op] : OperandTable) { + // Check there are no wip_match_opcode patterns in the "apply" patterns. + if (any_of(ApplyPats, + [&](auto &E) { return isa(E.second.get()); })) { + dump(); + PrintFatalError( + "illegal wip_match_opcode pattern in the 'apply' patterns!"); + } + + for (const auto &[Name, Op] : RuleOpTable) { if (Op.ApplyPat && !Op.MatchPat) { dump(); PrintFatalError("Operand " + Name + " has an apply pattern, but no match pattern!"); } + + if (Op.MatchPat && Op.IsMatchLiveIn) { + dump(); + PrintFatalError( + "Operand " + Name + + " is defined in match pattern but marked as match-live-in"); + } } } -bool CombineRuleBuilder::addFeaturePredicates(RuleMatcher &M) { - if (!RuleDef.getValue("Predicates")) - return true; +void CombineRuleBuilder::print(raw_ostream &OS, + const PatternAlternatives &Alts) const { + SmallVector Strings( + map_range(Alts, [](const auto &PatAndPerm) { + return PatAndPerm.first->getName().str() + "[" + + to_string(PatAndPerm.second) + "]"; + })); + // Sort so output is deterministic for tests. Otherwise it's sorted by pointer + // values. + sort(Strings); + OS << "[" << join(Strings, ", ") << "]"; +} - ListInit *Preds = RuleDef.getValueAsListInit("Predicates"); - for (Init *I : Preds->getValues()) { - if (DefInit *Pred = dyn_cast(I)) { - Record *Def = Pred->getDef(); - if (!Def->isSubClassOf("Predicate")) { - ::PrintError(Def->getLoc(), "Unknown 'Predicate' Type"); - return false; - } +bool CombineRuleBuilder::addApplyPattern(std::unique_ptr Pat) { + StringRef Name = Pat->getName(); + if (ApplyPats.contains(Name)) { + PrintError("'" + Name + "' apply pattern defined more than once!"); + return false; + } - if (Def->getValueAsString("CondString").empty()) - continue; + if (isa(Pat.get())) { + PrintError("'" + Name + + "': wip_match_opcode is not supported in apply patterns"); + return false; + } - if (SubtargetFeatures.count(Def) == 0) { - SubtargetFeatures.emplace( - Def, SubtargetFeatureInfo(Def, SubtargetFeatures.size())); - } + if (isa(Pat.get())) { + if (hasAnyOpcodeMatchPattern()) { + PrintError("cannot use wip_match_opcode in combination with apply " + "instruction patterns!"); + return false; + } - M.addRequiredFeature(Def); + if (isa(Pat.get())) { + PrintError("'" + Name + "': using " + PatFragClassName + + " is not supported in apply patterns"); + return false; + } + } + + if (auto *CXXPat = dyn_cast(Pat.get())) + CXXPat->setIsApply(); + + ApplyPats[Name] = std::move(Pat); + return true; +} + +bool CombineRuleBuilder::addMatchPattern(std::unique_ptr Pat) { + StringRef Name = Pat->getName(); + if (MatchPats.contains(Name)) { + PrintError("'" + Name + "' match pattern defined more than once!"); + return false; + } + + // TODO: Could move this to some "checkSemantics" method? + if (isa(Pat.get())) { + if (hasAnyOpcodeMatchPattern()) { + PrintError("wip_opcode_match can only be present once"); + return false; + } + } + + if (const auto *CXXPat = dyn_cast(Pat.get())) { + if (!CXXPat->getRawCode().contains("return ")) { + PrintWarning("'match' C++ code does not seem to return!"); } } + MatchPats[Name] = std::move(Pat); return true; } @@ -741,109 +1823,270 @@ CE.declare(MD.getPatternSymbol(), MD.getQualifiedVariableName()); } -void CombineRuleBuilder::addCXXPredicate(InstructionMatcher &IM, +void CombineRuleBuilder::addCXXPredicate(RuleMatcher &M, const CodeExpansions &CE, - const CXXPattern &P) { - const auto &ExpandedCode = P.expandCode(CE, RuleDef.getLoc()); - IM.addPredicate( + const CXXPattern &P, + const PatternAlternatives &Alts) { + // FIXME: Hack so C++ code is executed last. May not work for more complex + // patterns. + auto &IM = *std::prev(M.insnmatchers().end()); + const auto &ExpandedCode = + P.expandCode(CE, RuleDef.getLoc(), [&](raw_ostream &OS) { + OS << "// Pattern Alternatives: "; + print(OS, Alts); + OS << "\n"; + }); + IM->addPredicate( ExpandedCode.getEnumNameWithPrefix(CXXPredPrefix)); } +void CombineRuleBuilder::addCXXAction(RuleMatcher &M, const CodeExpansions &CE, + const CXXPattern &P) { + const auto &ExpandedCode = P.expandCode(CE, RuleDef.getLoc()); + M.addAction( + ExpandedCode.getEnumNameWithPrefix(CXXApplyPrefix)); +} + +bool CombineRuleBuilder::hasOnlyCXXApplyPatterns() const { + return all_of(ApplyPats, [&](auto &Entry) { + return isa(Entry.second.get()); + }); +} + +bool CombineRuleBuilder::typecheckPatterns() { + OperandTypeChecker OTC(RuleDef.getLoc()); + + for (auto &Pat : values(MatchPats)) { + if (auto *IP = dyn_cast(Pat.get())) { + if (!OTC.check(IP)) + return false; + } + } + + for (auto &Pat : values(ApplyPats)) { + if (auto *IP = dyn_cast(Pat.get())) { + if (!OTC.check(IP)) + return false; + } + } + + OTC.setAllOperandTypes(); + return true; +} + +bool CombineRuleBuilder::buildPermutationsToEmit() { + PermutationsToEmit.clear(); + + // Start with one empty set of alternatives. + PermutationsToEmit.emplace_back(); + for (const auto &Pat : values(MatchPats)) { + unsigned NumAlts = 0; + // Note: technically, AnyOpcodePattern also needs permutations, but: + // - We only allow a single one of them in the root. + // - They cannot be mixed with any other pattern other than C++ code. + // So we don't really need to take them into account here. We could, but + // that pattern is a hack anyway and the less it's involved, the better. + if (const auto *PFP = dyn_cast(Pat.get())) + NumAlts = PFP->getPatFrag().num_alternatives(); + else + continue; + + // For each pattern that needs permutations, multiply the current set of + // alternatives. + auto CurPerms = PermutationsToEmit; + PermutationsToEmit.clear(); + + for (const auto &Perm : CurPerms) { + assert(!Perm.count(Pat.get()) && "Pattern already emitted?"); + for (unsigned K = 0; K < NumAlts; ++K) { + PatternAlternatives NewPerm = Perm; + NewPerm[Pat.get()] = K; + PermutationsToEmit.emplace_back(std::move(NewPerm)); + } + } + } + + if (int64_t MaxPerms = RuleDef.getValueAsInt("MaxPermutations"); + MaxPerms > 0) { + if ((int64_t)PermutationsToEmit.size() > MaxPerms) { + PrintError("cannot emit rule '" + RuleDef.getName() + "'; " + + Twine(PermutationsToEmit.size()) + + " permutations would be emitted, but the max is " + + Twine(MaxPerms)); + return false; + } + } + + // Ensure we always have a single empty entry, it simplifies the emission + // logic so it doesn't need to handle the case where there are no perms. + if (PermutationsToEmit.empty()) { + PermutationsToEmit.emplace_back(); + return true; + } + + return true; +} + std::string CombineRuleBuilder::makeAnonPatName(StringRef Prefix) const { return to_string("__anon_pat_" + Prefix + "_" + to_string(RuleID) + "_" + to_string(AnonIDCnt++)); } -RuleMatcher &CombineRuleBuilder::addRuleMatcher(Twine AdditionalComment) { +RuleMatcher &CombineRuleBuilder::addRuleMatcher(const PatternAlternatives &Alts, + Twine AdditionalComment) { auto &RM = OutRMs.emplace_back(RuleDef.getLoc()); addFeaturePredicates(RM); + RM.setPermanentGISelFlags(GISF_IgnoreCopies); RM.addRequiredSimplePredicate(getIsEnabledPredicateEnumName(RuleID)); const std::string AdditionalCommentStr = AdditionalComment.str(); - RM.addAction( - "Combiner Rule #" + to_string(RuleID) + ": " + RuleDef.getName().str() + - (AdditionalCommentStr.empty() ? "" : "; " + AdditionalCommentStr)); + + std::string Comment; + raw_string_ostream CommentOS(Comment); + CommentOS << "Combiner Rule #" << RuleID << ": " << RuleDef.getName(); + if (!Alts.empty()) { + CommentOS << " @ "; + print(CommentOS, Alts); + } + if (!AdditionalCommentStr.empty()) + CommentOS << "; " << AdditionalCommentStr; + RM.addAction(Comment); return RM; } +bool CombineRuleBuilder::addFeaturePredicates(RuleMatcher &M) { + if (!RuleDef.getValue("Predicates")) + return true; + + ListInit *Preds = RuleDef.getValueAsListInit("Predicates"); + for (Init *I : Preds->getValues()) { + if (DefInit *Pred = dyn_cast(I)) { + Record *Def = Pred->getDef(); + if (!Def->isSubClassOf("Predicate")) { + ::PrintError(Def, "Unknown 'Predicate' Type"); + return false; + } + + if (Def->getValueAsString("CondString").empty()) + continue; + + if (SubtargetFeatures.count(Def) == 0) { + SubtargetFeatures.emplace( + Def, SubtargetFeatureInfo(Def, SubtargetFeatures.size())); + } + + M.addRequiredFeature(Def); + } + } + + return true; +} + bool CombineRuleBuilder::findRoots() { + const auto Finish = [&]() { + assert(MatchRoot); + + if (hasOnlyCXXApplyPatterns()) + return true; + + auto *IPRoot = dyn_cast(MatchRoot); + if (!IPRoot) + return true; + + if (IPRoot->getNumInstDefs() == 0) { + // No defs to work with -> find the root using the pattern name. + auto It = ApplyPats.find(RootName); + if (It == ApplyPats.end()) { + PrintError("Cannot find root '" + RootName + "' in apply patterns!"); + return false; + } + + auto *ApplyRoot = dyn_cast(It->second.get()); + if (!ApplyRoot) { + PrintError("apply pattern root '" + RootName + + "' must be an instruction pattern"); + return false; + } + dbgs() << "ApplyRoot is " << ApplyRoot->getName() << "\n"; + ApplyRoots.push_back(ApplyRoot); + return true; + } + + // Collect all redefinitions of the MatchRoot's defs and put them in + // ApplyRoots. + for (unsigned K = 0; K < IPRoot->getNumInstDefs(); ++K) { + auto &O = IPRoot->getOperand(K); + assert(O.isDef() && O.isNamedOperand()); + StringRef Name = O.getOperandName(); + + auto *ApplyRedef = RuleOpTable.lookup(Name).ApplyPat; + if (!ApplyRedef) { + PrintError("def of pattern root '" + Name + + "' is not redefined in the apply pattern!"); + PrintNote("match pattern root is '" + MatchRoot->getName() + "'"); + return false; + } + + ApplyRoots.push_back(ApplyRedef); + } + + if (auto It = ApplyPats.find(RootName); It != ApplyPats.end()) { + if (find(ApplyRoots, It->second.get()) == ApplyRoots.end()) { + PrintError("apply pattern '" + RootName + + "' is supposed to be a root but it does not redefine any of " + "the defs of the match root"); + return false; + } + } + + return true; + }; + // Look by pattern name, e.g. // (G_FNEG $x, $y):$root - if (auto It = MatchPats.find(RootName); It != MatchPats.end()) { - MatchRoot = It->second.get(); - return true; + if (auto MatchPatIt = MatchPats.find(RootName); + MatchPatIt != MatchPats.end()) { + MatchRoot = MatchPatIt->second.get(); + return Finish(); } // Look by def: // (G_FNEG $root, $y) - auto It = OperandTable.find(RootName); - if (It == OperandTable.end()) { + auto OpIt = RuleOpTable.find(RootName); + if (OpIt == RuleOpTable.end()) { PrintError("Cannot find root '" + RootName + "' in match patterns!"); return false; } - if (!It->second.MatchPat) { + if (!OpIt->second.MatchPat) { PrintError("Cannot use live-in operand '" + RootName + "' as match pattern root!"); return false; } - MatchRoot = It->second.MatchPat; - return true; -} - -bool CombineRuleBuilder::buildOperandsTable() { - // Walk each instruction pattern - for (auto &P : values(MatchPats)) { - auto *IP = dyn_cast(P.get()); - if (!IP) - continue; - for (const auto &Operand : IP->operands()) { - // Create an entry, no matter if it's a use or a def. - auto &Entry = OperandTable[Operand.Name]; - - // We only need to do additional checking on defs, though. - if (!Operand.IsDef) - continue; - - if (Entry.MatchPat) { - PrintError("Operand '" + Operand.Name + - "' is defined multiple times in the 'match' patterns"); - return false; - } - Entry.MatchPat = IP; - } - } - - for (auto &P : values(ApplyPats)) { - auto *IP = dyn_cast(P.get()); - if (!IP) - continue; - for (const auto &Operand : IP->operands()) { - // Create an entry, no matter if it's a use or a def. - auto &Entry = OperandTable[Operand.Name]; + MatchRoot = OpIt->second.MatchPat; + return Finish(); +} - // We only need to do additional checking on defs, though. - if (!Operand.IsDef) - continue; +bool CombineRuleBuilder::buildRuleOperandsTable() { + // Walk each instruction pattern + for (auto &P : values(MatchPats)) { + auto *IP = dyn_cast(P.get()); + if (IP && !RuleOpTable.addMatchPattern(RuleDef.getLoc(), IP, + "in the 'match' patterns")) + return false; + } - if (!Entry.MatchPat) { - PrintError("Cannot define live-in operand '" + Operand.Name + - "' in the 'apply' pattern"); - return false; - } - if (Entry.ApplyPat) { - PrintError("Operand '" + Operand.Name + - "' is defined multiple times in the 'apply' patterns"); - return false; - } - Entry.ApplyPat = IP; - } + for (auto &P : values(ApplyPats)) { + // Only CodeGenInstructionPatterns should be in the "apply" pattern. + auto *IP = dyn_cast(P.get()); + if (IP && !RuleOpTable.addApplyPattern(RuleDef.getLoc(), IP)) + return false; } return true; } -bool CombineRuleBuilder::parseDefs(DagInit &Def) { +bool CombineRuleBuilder::parseDefs(const DagInit &Def) { if (Def.getOperatorAsDef(RuleDef.getLoc())->getName() != "defs") { PrintError("Expected defs operator"); return false; @@ -889,98 +2132,82 @@ return true; } -bool CombineRuleBuilder::parseMatch(DagInit &Match) { - if (Match.getOperatorAsDef(RuleDef.getLoc())->getName() != "match") { - PrintError("Expected match operator"); +bool CombineRuleBuilder::parsePatternList( + const DagInit &List, + function_ref)> ParseAction, + StringRef Operator, ArrayRef DiagLoc) const { + if (List.getOperatorAsDef(RuleDef.getLoc())->getName() != Operator) { + ::PrintError(DiagLoc, "Expected " + Operator + " operator"); return false; } - if (Match.getNumArgs() == 0) { - PrintError("Matcher is empty"); + if (List.getNumArgs() == 0) { + ::PrintError(DiagLoc, Operator + " pattern list is empty"); return false; } // The match section consists of a list of matchers and predicates. Parse each // one and add the equivalent GIMatchDag nodes, predicates, and edges. - bool HasOpcodeMatcher = false; - for (unsigned I = 0; I < Match.getNumArgs(); ++I) { - Init *Arg = Match.getArg(I); - std::string Name = Match.getArgName(I) - ? Match.getArgName(I)->getValue().str() - : makeAnonPatName("match"); - - if (MatchPats.contains(Name)) { - PrintError("'" + Name + "' match pattern defined more than once!"); - return false; - } + for (unsigned I = 0; I < List.getNumArgs(); ++I) { + Init *Arg = List.getArg(I); + std::string Name = List.getArgName(I) ? List.getArgName(I)->getValue().str() + : makeAnonPatName(Operator); - if (auto Pat = parseInstructionMatcher(*Arg, Name)) { - MatchPats[Pat->getName()] = std::move(Pat); + if (auto Pat = parseInstructionPattern(*Arg, Name)) { + if (!ParseAction(std::move(Pat))) + return false; continue; } if (auto Pat = parseWipMatchOpcodeMatcher(*Arg, Name)) { - if (HasOpcodeMatcher) { - PrintError("wip_opcode_match can only be present once"); + if (!ParseAction(std::move(Pat))) return false; - } - HasOpcodeMatcher = true; - MatchPats[Pat->getName()] = std::move(Pat); continue; } // Parse arbitrary C++ code if (const auto *StringI = dyn_cast(Arg)) { - auto CXXPat = - std::make_unique(*StringI, Name, /*IsApply*/ false); - if (!CXXPat->getRawCode().contains("return ")) { - PrintWarning(RuleDef.getLoc(), - "'match' C++ code does not seem to return!"); - } - MatchPats[CXXPat->getName()] = std::move(CXXPat); + auto CXXPat = std::make_unique(*StringI, Name); + if (!ParseAction(std::move(CXXPat))) + return false; continue; } - // TODO: don't print this on, e.g. bad operand count in inst pat - PrintError("Expected a subclass of GIMatchKind or a sub-dag whose " - "operator is either of a GIMatchKindWithArgs or Instruction"); - PrintNote("Pattern was `" + Arg->getAsString() + "'"); - return false; - } - - return true; -} - -bool CombineRuleBuilder::parseApply(DagInit &Apply) { - // Currently we only support C++ :( - if (Apply.getOperatorAsDef(RuleDef.getLoc())->getName() != "apply") { - PrintError("Expected 'apply' operator in Apply DAG"); - return false; - } - - if (Apply.getNumArgs() != 1) { - PrintError("Expected exactly 1 argument in 'apply'"); + ::PrintError(DiagLoc, + "Failed to parse pattern: '" + Arg->getAsString() + "'"); return false; } - const StringInit *Code = dyn_cast(Apply.getArg(0)); - auto Pat = std::make_unique(*Code, makeAnonPatName("apply"), - /*IsApply*/ true); - ApplyPats[Pat->getName()] = std::move(Pat); return true; } std::unique_ptr -CombineRuleBuilder::parseInstructionMatcher(const Init &Arg, StringRef Name) { - const DagInit *Matcher = getDagWithOperatorOfSubClass(Arg, "Instruction"); - if (!Matcher) +CombineRuleBuilder::parseInstructionPattern(const Init &Arg, + StringRef Name) const { + const DagInit *DagPat = dyn_cast(&Arg); + if (!DagPat) return nullptr; - auto &Instr = CGT.getInstruction(Matcher->getOperatorAsDef(RuleDef.getLoc())); - auto Pat = std::make_unique(Instr, Name); + std::unique_ptr Pat; + if (const DagInit *IP = getDagWithOperatorOfSubClass(Arg, "Instruction")) { + auto &Instr = CGT.getInstruction(IP->getOperatorAsDef(RuleDef.getLoc())); + Pat = std::make_unique(Instr, Name); + } else if (const DagInit *PFP = + getDagWithOperatorOfSubClass(Arg, PatFragClassName)) { + const Record *Def = PFP->getOperatorAsDef(RuleDef.getLoc()); + const PatFrag *PF = parsePatFrag(Def); + if (!PF) + return nullptr; // Already diagnosed by parsePatFrag + Pat = std::make_unique(*PF, Name); + } else { + return nullptr; + } - for (const auto &NameInit : Matcher->getArgNames()) - Pat->addOperand(NameInit->getAsUnquotedString()); + for (unsigned K = 0; K < DagPat->getNumArgs(); ++K) { + if (!parseInstructionPatternOperand(*Pat, DagPat->getArg(K), + DagPat->getArgName(K))) + return nullptr; + } if (!Pat->checkSemantics(RuleDef.getLoc())) return nullptr; @@ -990,7 +2217,7 @@ std::unique_ptr CombineRuleBuilder::parseWipMatchOpcodeMatcher(const Init &Arg, - StringRef Name) { + StringRef Name) const { const DagInit *Matcher = getDagWithSpecificOperator(Arg, "wip_match_opcode"); if (!Matcher) return nullptr; @@ -1016,15 +2243,221 @@ return std::move(Result); } +bool CombineRuleBuilder::parseInstructionPatternOperand( + InstructionPattern &IP, const Init *OpInit, + const StringInit *OpName) const { + const auto ParseErr = [&]() { + PrintError("cannot parse operand '" + OpInit->getAsUnquotedString() + "' "); + if (OpName) + PrintNote("operand name is '" + OpName->getAsUnquotedString() + "'"); + return false; + }; + + // untyped immediate, e.g. 0 + if (const auto *IntImm = dyn_cast(OpInit)) { + std::string Name = OpName ? OpName->getAsUnquotedString() : ""; + IP.addOperand(IntImm->getValue(), Name, /*Type=*/nullptr); + return true; + } + + // typed immediate, e.g. (i32 0) + if (const auto *DagOp = dyn_cast(OpInit)) { + if (DagOp->getNumArgs() != 1) + return ParseErr(); + + Record *ImmTy = DagOp->getOperatorAsDef(RuleDef.getLoc()); + if (!ImmTy->isSubClassOf("ValueType")) { + PrintError("cannot parse immediate '" + OpInit->getAsUnquotedString() + + "', '" + ImmTy->getName() + "' is not a ValueType!"); + return false; + } + + if (!IP.hasAllDefs()) { + PrintError("out operand of '" + IP.getInstName() + + "' cannot be an immediate"); + return false; + } + + const auto *Val = dyn_cast(DagOp->getArg(0)); + if (!Val) + return ParseErr(); + + std::string Name = OpName ? OpName->getAsUnquotedString() : ""; + IP.addOperand(Val->getValue(), Name, ImmTy); + return true; + } + + // Typed operand e.g. $x/$z in (G_FNEG $x, $z) + if (auto *DefI = dyn_cast(OpInit)) { + if (!OpName) { + PrintError("expected an operand name after '" + OpInit->getAsString() + + "'"); + return false; + } + const Record *Def = DefI->getDef(); + if (!Def->isSubClassOf("ValueType")) { + PrintError("invalid operand type: '" + Def->getName() + + "' is not a ValueType"); + return false; + } + IP.addOperand(OpName->getAsUnquotedString(), Def); + return true; + } + + // Untyped operand e.g. $x/$z in (G_FNEG $x, $z) + if (isa(OpInit)) { + assert(OpName && "Unset w/ no OpName?"); + IP.addOperand(OpName->getAsUnquotedString(), /*Type=*/nullptr); + return true; + } + + return ParseErr(); +} + +std::unique_ptr +CombineRuleBuilder::parsePatFragImpl(const Record *Def) const { + auto StackTrace = PrettyStackTraceParse(*Def); + if (!Def->isSubClassOf(PatFragClassName)) + return nullptr; + + const DagInit *Ins = Def->getValueAsDag("InOperands"); + if (Ins->getOperatorAsDef(Def->getLoc())->getName() != "ins") { + ::PrintError(Def, "expected 'ins' operator for " + PatFragClassName + + " in operands list"); + return nullptr; + } + + const DagInit *Outs = Def->getValueAsDag("OutOperands"); + if (Outs->getOperatorAsDef(Def->getLoc())->getName() != "outs") { + ::PrintError(Def, "expected 'outs' operator for " + PatFragClassName + + " out operands list"); + return nullptr; + } + + auto Result = std::make_unique(*Def); + if (!parsePatFragParamList(Def->getLoc(), *Outs, + [&](StringRef Name, PatFrag::ParamKind Kind) { + Result->addOutParam(Name, Kind); + return true; + })) + return nullptr; + + if (!parsePatFragParamList(Def->getLoc(), *Ins, + [&](StringRef Name, PatFrag::ParamKind Kind) { + Result->addInParam(Name, Kind); + return true; + })) + return nullptr; + + const ListInit *Alts = Def->getValueAsListInit("Alternatives"); + for (const Init *Alt : *Alts) { + const auto *PatDag = dyn_cast(Alt); + if (!PatDag) { + ::PrintError(Def, "expected dag init for PatFrag pattern alternative"); + return nullptr; + } + + PatFrag::Alternative &A = Result->addAlternative(); + const auto AddPat = [&](std::unique_ptr Pat) { + A.Pats.push_back(std::move(Pat)); + return true; + }; + + if (!parsePatternList(*PatDag, AddPat, "pattern", Def->getLoc())) + return nullptr; + } + + if (!Result->buildOperandsTables() || !Result->checkSemantics()) + return nullptr; + + return Result; +} + +bool CombineRuleBuilder::parsePatFragParamList( + ArrayRef DiagLoc, const DagInit &OpsList, + function_ref ParseAction) const { + for (unsigned K = 0; K < OpsList.getNumArgs(); ++K) { + const StringInit *Name = OpsList.getArgName(K); + const Init *Ty = OpsList.getArg(K); + + if (!Name) { + ::PrintError(DiagLoc, "all operands must be named'"); + return false; + } + const std::string NameStr = Name->getAsUnquotedString(); + + PatFrag::ParamKind OpKind; + if (isSpecificDef(*Ty, "gi_imm")) + OpKind = PatFrag::PK_Imm; + else if (isSpecificDef(*Ty, "root")) + OpKind = PatFrag::PK_Root; + else if (isa(Ty) || + isSpecificDef(*Ty, "gi_mo")) // no type = gi_mo. + OpKind = PatFrag::PK_MachineOperand; + else { + ::PrintError( + DiagLoc, + "'" + NameStr + + "' operand type was expected to be 'root', 'gi_imm' or 'gi_mo'"); + return false; + } + + if (!ParseAction(NameStr, OpKind)) + return false; + } + + return true; +} + +const PatFrag *CombineRuleBuilder::parsePatFrag(const Record *Def) const { + // Cache already parsed PatFrags to avoid doing extra work. + static DenseMap> ParsedPatFrags; + + auto It = ParsedPatFrags.find(Def); + if (It != ParsedPatFrags.end()) { + SeenPatFrags.insert(It->second.get()); + return It->second.get(); + } + + std::unique_ptr NewPatFrag = parsePatFragImpl(Def); + if (!NewPatFrag) { + ::PrintError(Def, "Could not parse " + PatFragClassName + " '" + + Def->getName() + "'"); + // Put a nullptr in the map so we don't attempt parsing this again. + ParsedPatFrags[Def] = nullptr; + return nullptr; + } + + const auto *Res = NewPatFrag.get(); + ParsedPatFrags[Def] = std::move(NewPatFrag); + SeenPatFrags.insert(Res); + return Res; +} + bool CombineRuleBuilder::emitMatchPattern(CodeExpansions &CE, + const PatternAlternatives &Alts, const InstructionPattern &IP) { - auto &M = addRuleMatcher(); + auto StackTrace = PrettyStackTraceEmit(RuleDef, &IP); + + auto &M = addRuleMatcher(Alts); InstructionMatcher &IM = M.addInstructionMatcher("root"); declareInstExpansion(CE, IM, IP.getName()); DenseSet SeenPats; - if (!emitInstructionMatchPattern(CE, M, IM, IP, SeenPats)) - return false; + if (const auto *CGP = dyn_cast(&IP)) { + if (!emitCodeGenInstructionMatchPattern(CE, Alts, RuleOpTable, M, IM, *CGP, + SeenPats)) + return false; + } else if (const auto *PFP = dyn_cast(&IP)) { + if (!PFP->getPatFrag().canBeMatchRoot()) { + PrintError("cannot use '" + PFP->getInstName() + " as match root"); + return false; + } + + if (!emitPatFragMatchPattern(CE, Alts, M, &IM, *PFP, SeenPats)) + return false; + } else + llvm_unreachable("Unknown kind of InstructionPattern!"); // Emit remaining patterns for (auto &Pat : values(MatchPats)) { @@ -1035,11 +2468,17 @@ case Pattern::K_AnyOpcode: PrintError("wip_match_opcode can not be used with instruction patterns!"); return false; - case Pattern::K_Inst: + case Pattern::K_PatFrag: { + if (!emitPatFragMatchPattern(CE, Alts, M, /*IM*/ nullptr, + *cast(Pat.get()), SeenPats)) + return false; + continue; + } + case Pattern::K_CodeGenInstruction: cast(Pat.get())->reportUnreachable(RuleDef.getLoc()); return false; case Pattern::K_CXX: { - addCXXPredicate(IM, CE, *cast(Pat.get())); + addCXXPredicate(M, CE, *cast(Pat.get()), Alts); continue; } default: @@ -1051,11 +2490,13 @@ } bool CombineRuleBuilder::emitMatchPattern(CodeExpansions &CE, + const PatternAlternatives &Alts, const AnyOpcodePattern &AOP) { + auto StackTrace = PrettyStackTraceEmit(RuleDef, &AOP); for (const CodeGenInstruction *CGI : AOP.insts()) { - auto &M = addRuleMatcher("wip_match_opcode alternative '" + - CGI->TheDef->getName() + "'"); + auto &M = addRuleMatcher(Alts, "wip_match_opcode '" + + CGI->TheDef->getName() + "'"); InstructionMatcher &IM = M.addInstructionMatcher(AOP.getName()); declareInstExpansion(CE, IM, AOP.getName()); @@ -1074,12 +2515,20 @@ case Pattern::K_AnyOpcode: PrintError("wip_match_opcode can only be present once!"); return false; - case Pattern::K_Inst: + case Pattern::K_PatFrag: { + DenseSet SeenPats; + if (!emitPatFragMatchPattern(CE, Alts, M, /*IM*/ nullptr, + *cast(Pat.get()), + SeenPats)) + return false; + continue; + } + case Pattern::K_CodeGenInstruction: cast(Pat.get())->reportUnreachable( RuleDef.getLoc()); return false; case Pattern::K_CXX: { - addCXXPredicate(IM, CE, *cast(Pat.get())); + addCXXPredicate(M, CE, *cast(Pat.get()), Alts); break; } default: @@ -1094,63 +2543,433 @@ return true; } +bool CombineRuleBuilder::emitPatFragMatchPattern( + CodeExpansions &CE, const PatternAlternatives &Alts, RuleMatcher &RM, + InstructionMatcher *IM, const PatFragPattern &PFP, + DenseSet &SeenPats) { + auto StackTrace = PrettyStackTraceEmit(RuleDef, &PFP); + + if (SeenPats.contains(&PFP)) + return true; + SeenPats.insert(&PFP); + + const auto &PF = PFP.getPatFrag(); + + if (!IM) { + // When we don't have an IM, this means this PatFrag isn't reachable from + // the root. This is only acceptable if it doesn't define anything (e.g. a + // pure C++ PatFrag). + if (PF.num_out_params() != 0) { + PFP.reportUnreachable(RuleDef.getLoc()); + return false; + } + } else { + // When an IM is provided, this is reachable from the root, and we're + // expecting to have output operands. + // TODO: If we want to allow for multiple roots we'll need a map of IMs + // then, and emission becomes a bit more complicated. + assert(PF.num_roots() == 1); + } + + CodeExpansions PatFragCEs; + if (!PFP.mapInputCodeExpansions(CE, PatFragCEs, RuleDef.getLoc())) + return false; + + // List of {ParamName, ArgName}. + // When all patterns have been emitted, find expansions in PatFragCEs named + // ArgName and add their expansion to CE using ParamName as the key. + SmallVector, 4> CEsToImport; + + // Map parameter names to the actual argument. + const auto OperandMapper = + [&](const InstructionPattern::Operand &O) -> InstructionPattern::Operand { + if (!O.isNamedOperand()) + return O; + + StringRef ParamName = O.getOperandName(); + + // Not sure what to do with those tbh. They should probably never be here. + assert(!O.isNamedImmediate() && "TODO: handle named imms"); + unsigned PIdx = PF.getParamIdx(ParamName); + + // Map parameters to the argument values. + if (PIdx == (unsigned)-1) { + // This is a temp of the PatFragPattern, prefix the name to avoid + // conflicts. + return O.withNewName((PFP.getName() + "." + ParamName).str()); + } + + // The operand will be added to PatFragCEs's code expansions using the + // parameter's name. If it's bound to some operand during emission of the + // patterns, we'll want to add it to CE. + auto ArgOp = PFP.getOperand(PIdx); + if (ArgOp.isNamedOperand()) + CEsToImport.emplace_back(ArgOp.getOperandName().str(), ParamName); + + if (ArgOp.getType() && O.getType() && ArgOp.getType() != O.getType()) { + StringRef PFName = PF.getName(); + PrintWarning("impossible type constraints: operand " + Twine(PIdx) + + " of '" + PFP.getName() + "' has type '" + + ArgOp.getType()->getName() + "', but '" + PFName + + "' constrains it to '" + O.getType()->getName() + "'"); + if (ArgOp.isNamedOperand()) + PrintNote("operand " + Twine(PIdx) + " of '" + PFP.getName() + + "' is '" + ArgOp.getOperandName() + "'"); + if (O.isNamedOperand()) + PrintNote("argument " + Twine(PIdx) + " of '" + PFName + "' is '" + + ParamName + "'"); + } + + return ArgOp; + }; + + // PatFragPatterns are only made of InstructionPatterns or CXXPatterns. + // Emit instructions from the root. + const auto &FragAlt = PF.getAlternative(Alts.lookup(&PFP)); + DenseSet PatFragSeenPats; + for (const auto &[Idx, InOp] : enumerate(PF.out_params())) { + if (InOp.Kind != PatFrag::PK_Root) + continue; + + StringRef ParamName = InOp.Name; + const auto *Def = FragAlt.OpTable.getMatchDefPattern(ParamName); + assert(Def && "PatFrag::checkSemantics should have emitted an error if " + "an out operand isn't defined!"); + assert(isa(Def) && + "Nested PatFrags not supported yet"); + + if (!emitCodeGenInstructionMatchPattern( + PatFragCEs, Alts, FragAlt.OpTable, RM, *IM, + *cast(Def), PatFragSeenPats, + OperandMapper)) + return false; + } + + // Emit leftovers. + for (const auto &Pat : FragAlt.Pats) { + if (PatFragSeenPats.contains(Pat.get())) + continue; + + if (const auto *CXXPat = dyn_cast(Pat.get())) { + addCXXPredicate(RM, PatFragCEs, *CXXPat, Alts); + continue; + } + + if (const auto *IP = dyn_cast(Pat.get())) { + IP->reportUnreachable(PF.getLoc()); + return false; + } + + llvm_unreachable("Unexpected pattern kind in PatFrag"); + } + + for (const auto &[ParamName, ArgName] : CEsToImport) { + // Note: we're find if ParamName already exists. It just means it's been + // bound before, so we prefer to keep the first binding. + CE.declare(ParamName, PatFragCEs.lookup(ArgName)); + } + + return true; +} + bool CombineRuleBuilder::emitApplyPatterns(CodeExpansions &CE, RuleMatcher &M) { + if (hasOnlyCXXApplyPatterns()) { + for (auto &Pat : values(ApplyPats)) + addCXXAction(M, CE, *cast(Pat.get())); + return true; + } + + DenseSet SeenPats; + + // Preallocate all of the temporary registers we need. + // This needs to be done first so all of the MakeTempRegisterActions are + // emitted first. + StringMap OperandToTempRegID; + if (!emitTempRegisterActions(CE, M, OperandToTempRegID)) + return false; + + for (auto *ApplyRoot : ApplyRoots) { + assert(isa(ApplyRoot) && + "PatFragPatterns are not supported in apply patterns yet!"); + if (!emitCodeGenInstructionApplyPattern( + CE, M, *cast(ApplyRoot), SeenPats, + OperandToTempRegID)) + return false; + } + for (auto &Pat : values(ApplyPats)) { + if (SeenPats.contains(Pat.get())) + continue; + switch (Pat->getKind()) { case Pattern::K_AnyOpcode: - case Pattern::K_Inst: - llvm_unreachable("Unsupported pattern kind in output pattern!"); + llvm_unreachable("Unexpected pattern in apply!"); + case Pattern::K_PatFrag: + // TODO: We could support pure C++ PatFrags as a temporary thing. + llvm_unreachable("Unexpected pattern in apply!"); + case Pattern::K_CodeGenInstruction: + cast(Pat.get())->reportUnreachable( + RuleDef.getLoc()); + return false; case Pattern::K_CXX: { - CXXPattern *CXXPat = cast(Pat.get()); - const auto &ExpandedCode = CXXPat->expandCode(CE, RuleDef.getLoc()); - M.addAction( - ExpandedCode.getEnumNameWithPrefix(CXXApplyPrefix)); + addCXXAction(M, CE, *cast(Pat.get())); continue; } default: - llvm_unreachable("Unknown pattern kind!"); + llvm_unreachable("unknown pattern kind!"); + } + } + + return true; +} + +bool CombineRuleBuilder::emitTempRegisterActions( + CodeExpansions &CE, RuleMatcher &M, + StringMap &OperandToTempRegID) const { + for (const auto &Pat : values(ApplyPats)) { + const auto *IP = dyn_cast(Pat.get()); + if (!IP) + continue; + + for (const auto &O : IP->operands()) { + // Skip non-defs & rewrites of matched instructions. + if (!O.isDef() || RuleOpTable.getMatchDefPattern(O.getOperandName())) + continue; + + // This is a brand new register. + StringRef OpName = O.getOperandName(); + assert(OperandToTempRegID.count(OpName) == 0 && + "Register has multiple defs?!"); + const unsigned TempRegID = M.allocateTempRegID(); + OperandToTempRegID[OpName] = TempRegID; + const Record *Ty = O.getType(); + if (!Ty) { + PrintError("def of a new register '" + OpName + + "' in the apply patterns must have a type"); + return false; + } + declareTempRegExpansion(CE, TempRegID, OpName); + M.addAction(getLLTCodeGenFromRecord(Ty), + TempRegID); } } return true; } -bool CombineRuleBuilder::emitInstructionMatchPattern( - CodeExpansions &CE, RuleMatcher &M, InstructionMatcher &IM, - const InstructionPattern &P, DenseSet &SeenPats) { +bool CombineRuleBuilder::emitCodeGenInstructionApplyPattern( + CodeExpansions &CE, RuleMatcher &M, const CodeGenInstructionPattern &P, + DenseSet &SeenPats, + const StringMap &OperandToTempRegID) { + auto StackTrace = PrettyStackTraceEmit(RuleDef, &P); + if (SeenPats.contains(&P)) return true; SeenPats.insert(&P); - IM.addPredicate(&P.getInst()); - declareInstExpansion(CE, IM, P.getName()); + // First, render the uses. + for (auto &O : P.operands()) { + if (O.isDef() || !O.isNamedOperand()) + continue; + + StringRef OpName = O.getOperandName(); + if (const auto *DefPat = RuleOpTable.getApplyDefPattern(OpName)) { + if (!emitCodeGenInstructionApplyPattern(CE, M, *DefPat, SeenPats, + OperandToTempRegID)) + return false; + } else { + // If we have no def, check this exists in the MatchRoot. + if (!O.isNamedImmediate() && !RuleOpTable.isMatchLiveInOrDef(OpName)) { + PrintError("invalid output operand '" + OpName + + "': operand is not a live-in of the match pattern, and it " + "has no definition"); + return false; + } + } + } + + // Now render this inst. + auto &DstMI = + M.addAction(M.allocateOutputInsnID(), &P.getInst()); - unsigned OpIdx = 0; for (auto &O : P.operands()) { - auto &OpTableEntry = OperandTable.find(O.Name)->second; + if (O.isNamedImmediate()) { + PrintError("invalid output operand '" + O.getOperandName() + + "': output immediates cannot be named"); + PrintNote("while emitting pattern '" + P.getName() + "' (" + + P.getInstName() + ")"); + return false; + } - OperandMatcher &OM = - IM.addOperand(OpIdx++, O.Name, AllocatedTemporariesBaseID++); - declareOperandExpansion(CE, OM, O.Name); + if (O.hasImmValue()) { + if (!O.getType()) { + PrintError("output immediates must be typed!"); + PrintNote("while emitting pattern '" + P.getName() + "' (" + + P.getInstName() + ")"); + return false; + } + + DstMI.addRenderer(O.getImmValue(), + getLLTCodeGenFromRecord(O.getType())); + continue; + } - if (O.IsDef) + StringRef OpName = O.getOperandName(); + if (!O.isDef()) { + if (auto It = OperandToTempRegID.find(OpName); + It != OperandToTempRegID.end()) + DstMI.addRenderer(It->second); + else { + // This should be a match live in or a redef of a matched instr. + // If not, it means the register hasn't been allocated. + assert(RuleOpTable.isMatchLiveInOrDef(OpName) && + !RuleOpTable.getApplyDefPattern(OpName) && + "Temp reg not emitted yet!"); + DstMI.addRenderer(OpName); + } continue; + } - if (InstructionPattern *DefPat = OpTableEntry.MatchPat) { - auto InstOpM = OM.addPredicate(M, O.Name); - if (!InstOpM) { - // TODO: copy-pasted from GlobalISelEmitter.cpp. Is it still relevant - // here? - PrintError("Nested instruction '" + DefPat->getName() + - "' cannot be the same as another operand '" + O.Name + "'"); + if (const auto *MatchDef = RuleOpTable.getMatchDefPattern(OpName)) { + // TODO: Handle this. We need to mutate the instr, or delete the old + // one. + // Likewise, we also need to ensure we redef everything, if the + // instr has more than one def, we need to redef all or nothing. + if (MatchDef != MatchRoot) { + PrintError("redefining an instruction other than the root is not " + "supported (operand '" + + OpName + "')"); return false; } + // redef of a match + DstMI.addRenderer(OpName); + } else + DstMI.addRenderer(OperandToTempRegID.lookup(OpName)); + } + + // TODO: works? + DstMI.chooseInsnToMutate(M); + declareInstExpansion(CE, DstMI, P.getName()); + + return true; +} + +bool isLiteralImm(const InstructionPattern &P, unsigned OpIdx) { + if (const auto *CGP = dyn_cast(&P)) { + StringRef InstName = CGP->getInst().TheDef->getName(); + return (InstName == "G_CONSTANT" || InstName == "G_FCONSTANT") && + OpIdx == 1; + } + + llvm_unreachable("TODO"); +} + +bool CombineRuleBuilder::emitCodeGenInstructionMatchPattern( + CodeExpansions &CE, const PatternAlternatives &Alts, + const OperandTable &OpTable, RuleMatcher &M, InstructionMatcher &IM, + const CodeGenInstructionPattern &P, DenseSet &SeenPats, + OperandMapperFnRef OperandMapper) { + auto StackTrace = PrettyStackTraceEmit(RuleDef, &P); + + if (SeenPats.contains(&P)) + return true; + + SeenPats.insert(&P); + + IM.addPredicate(&P.getInst()); + declareInstExpansion(CE, IM, P.getName()); + + for (const auto &[Idx, OriginalO] : enumerate(P.operands())) { + // Remap the operand. This is used when emitting InstructionPatterns inside + // PatFrags, so it can remap them to the arguments passed to the pattern. + // + // We use the remapped operand to emit immediates, and for the symbolic + // operand names (in IM.addOperand). CodeExpansions and OperandTable lookups + // still use the original name. + // + // The "def" flag on the remapped operand is always ignored. + auto RemappedO = OperandMapper(OriginalO); + assert(RemappedO.isNamedOperand() == OriginalO.isNamedOperand() && + "Cannot remap an unnamed operand to a named one!"); + + const auto OpName = + RemappedO.isNamedOperand() ? RemappedO.getOperandName().str() : ""; + OperandMatcher &OM = + IM.addOperand(Idx, OpName, AllocatedTemporariesBaseID++); + if (!OpName.empty()) + declareOperandExpansion(CE, OM, OriginalO.getOperandName()); + + // Handle immediates. + if (RemappedO.hasImmValue()) { + if (isLiteralImm(P, Idx)) + OM.addPredicate(RemappedO.getImmValue()); + else + OM.addPredicate(RemappedO.getImmValue()); + } + + // Handle typed operands, but only bother to check if it hasn't been done + // before. + // + // getOperandMatcher will always return the first OM to have been created + // for that Operand. "OM" here is always a new OperandMatcher. + // + // Always emit a check for unnamed operands. + if (OpName.empty() || + !M.getOperandMatcher(OpName).contains()) { + if (const Record *Ty = RemappedO.getType()) + OM.addPredicate(getLLTCodeGenFromRecord(Ty)); + } + + // Stop here if the operand is a def, or if it had no name. + if (OriginalO.isDef() || !OriginalO.isNamedOperand()) + continue; + + const auto *DefPat = OpTable.getMatchDefPattern(OriginalO.getOperandName()); + if (!DefPat) + continue; + + if (OriginalO.hasImmValue()) { + assert(!OpName.empty()); + // This is a named immediate that also has a def, that's not okay. + // e.g. + // (G_SEXT $y, (i32 0)) + // (COPY $x, 42:$y) + PrintError("'" + OpName + + "' is a named immediate, it cannot be defined by another " + "instruction"); + PrintNote("'" + OpName + "' is defined by '" + DefPat->getName() + "'"); + return false; + } + + // From here we know that the operand defines an instruction, and we need to + // emit it. + auto InstOpM = + OM.addPredicate(M, DefPat->getName()); + if (!InstOpM) { + // TODO: copy-pasted from GlobalISelEmitter.cpp. Is it still relevant + // here? + PrintError("Nested instruction '" + DefPat->getName() + + "' cannot be the same as another operand '" + + OriginalO.getOperandName() + "'"); + return false; + } + + auto &IM = (*InstOpM)->getInsnMatcher(); + if (const auto *CGIDef = dyn_cast(DefPat)) { + if (!emitCodeGenInstructionMatchPattern(CE, Alts, OpTable, M, IM, *CGIDef, + SeenPats, OperandMapper)) + return false; + continue; + } - if (!emitInstructionMatchPattern(CE, M, (*InstOpM)->getInsnMatcher(), - *DefPat, SeenPats)) + if (const auto *PFPDef = dyn_cast(DefPat)) { + if (!emitPatFragMatchPattern(CE, Alts, M, &IM, *PFPDef, SeenPats)) return false; + continue; } + + llvm_unreachable("unknown type of InstructionPattern"); } return true; @@ -1158,10 +2977,12 @@ //===- GICombinerEmitter --------------------------------------------------===// -/// This class is essentially the driver. It fetches all TableGen records, calls -/// CombineRuleBuilder to build the MatchTable's RuleMatchers, then creates the -/// MatchTable & emits it. It also handles emitting all the supporting code such -/// as the list of LLTs, the CXXPredicates, etc. +/// Main implementation class. This emits the tablegenerated output. +/// +/// It collects rules, uses `CombineRuleBuilder` to parse them and accumulate +/// RuleMatchers, then takes all the necessary state/data from the various +/// static storage pools and wires them together to emit the match table & +/// associated function/data structures. class GICombinerEmitter final : public GlobalISelMatchTableExecutorEmitter { RecordKeeper &Records; StringRef Name; @@ -1334,7 +3155,7 @@ << " if (executeMatchTable(*this, OutMIs, State, ExecInfo" << ", getMatchTable(), *ST.getInstrInfo(), MRI, " "*MRI.getTargetRegisterInfo(), *ST.getRegBankInfo(), AvailableFeatures" - << ", /*CoverageInfo*/ nullptr)) {\n" + << ", /*CoverageInfo*/ nullptr, &Observer)) {\n" << " return true;\n" << " }\n\n" << " return false;\n" @@ -1342,7 +3163,7 @@ } void GICombinerEmitter::emitMIPredicateFns(raw_ostream &OS) { - auto MatchCode = getSorted(AllCXXMatchCode); + auto MatchCode = CXXPredicateCode::getAllMatchCode(); emitMIPredicateFnsImpl( OS, "", ArrayRef(MatchCode), [](const CXXPredicateCode *C) -> StringRef { return C->BaseEnumName; }, @@ -1396,7 +3217,7 @@ } void GICombinerEmitter::emitRunCustomAction(raw_ostream &OS) { - const auto ApplyCode = getSorted(AllCXXApplyCode); + const auto ApplyCode = CXXPredicateCode::getAllApplyCode(); if (!ApplyCode.empty()) { OS << "enum {\n"; @@ -1410,13 +3231,14 @@ } OS << "void " << getClassName() - << "::runCustomAction(unsigned ApplyID, const MatcherState &State) const " + << "::runCustomAction(unsigned ApplyID, const MatcherState &State, " + "NewMIVector &OutMIs) const " "{\n"; if (!ApplyCode.empty()) { OS << " switch(ApplyID) {\n"; for (const auto &Apply : ApplyCode) { OS << " case " << Apply->getEnumNameWithPrefix(CXXApplyPrefix) << ":{\n" - << " " << Apply->Code << "\n" + << " " << join(split(Apply->Code, "\n"), "\n ") << "\n" << " return;\n"; OS << " }\n"; } @@ -1492,16 +3314,20 @@ CombineRuleBuilder CRB(Target, SubtargetFeatures, *R, NextRuleID++, ActiveRules); - if (!CRB.parseAll()) + if (!CRB.parseAll()) { + assert(ErrorsPrinted && "Parsing failed without errors!"); continue; + } if (StopAfterParse) { CRB.print(outs()); continue; } - if (!CRB.emitRuleMatchers()) + if (!CRB.emitRuleMatchers()) { + assert(ErrorsPrinted && "Emission failed without errors!"); continue; + } } else gatherRules(ActiveRules, R->getValueAsListOfDefs("Rules")); } @@ -1527,11 +3353,17 @@ // Unused std::vector CustomRendererFns; - // Unused, but hack to avoid empty declarator - std::vector TypeObjects = {LLTCodeGen(LLT::scalar(1))}; // Unused std::vector ComplexPredicates; + std::vector TypeObjects; + append_range(TypeObjects, KnownTypes); + llvm::sort(TypeObjects); + + // Hack: Avoid empty declarator. + if (TypeObjects.empty()) + TypeObjects.push_back(LLT::scalar(1)); + // GET_GICOMBINER_DEPS, which pulls in extra dependencies. OS << "#ifdef GET_GICOMBINER_DEPS\n" << "#include \"llvm/ADT/SparseBitVector.h\"\n" @@ -1566,6 +3398,7 @@ //===----------------------------------------------------------------------===// static void EmitGICombiner(RecordKeeper &RK, raw_ostream &OS) { + EnablePrettyStackTrace(); CodeGenTarget Target(RK); if (SelectedCombiners.empty()) diff --git a/llvm/utils/TableGen/GlobalISelEmitter.cpp b/llvm/utils/TableGen/GlobalISelEmitter.cpp --- a/llvm/utils/TableGen/GlobalISelEmitter.cpp +++ b/llvm/utils/TableGen/GlobalISelEmitter.cpp @@ -2357,7 +2357,8 @@ void GlobalISelEmitter::emitRunCustomAction(raw_ostream &OS) { OS << "void " << getClassName() - << "::runCustomAction(unsigned, const MatcherState&) const {\n" + << "::runCustomAction(unsigned, const MatcherState&, NewMIVector &) const " + "{\n" << " llvm_unreachable(\"" + getClassName() + " does not support custom C++ actions!\");\n" << "}\n"; diff --git a/llvm/utils/TableGen/GlobalISelMatchTable.h b/llvm/utils/TableGen/GlobalISelMatchTable.h --- a/llvm/utils/TableGen/GlobalISelMatchTable.h +++ b/llvm/utils/TableGen/GlobalISelMatchTable.h @@ -526,6 +526,8 @@ std::make_unique(std::forward(args)...)); } + void setPermanentGISelFlags(GISelFlags V) { Flags = V; } + // Update the active GISelFlags based on the GISelFlags Record R. // A SaveAndRestore object is returned so the old GISelFlags are restored // at the end of the scope. @@ -648,6 +650,10 @@ } bool predicates_empty() const { return Predicates.empty(); } + template bool contains() const { + return any_of(Predicates, [&](auto &P) { return isa(P.get()); }); + } + std::unique_ptr predicates_pop_front() { std::unique_ptr Front = std::move(Predicates.front()); Predicates.pop_front(); @@ -1930,23 +1936,41 @@ }; /// Adds a specific immediate to the instruction being built. +/// If a LLT is passed, a ConstantInt immediate is created instead. class ImmRenderer : public OperandRenderer { protected: unsigned InsnID; int64_t Imm; + std::optional CImmLLT; public: ImmRenderer(unsigned InsnID, int64_t Imm) : OperandRenderer(OR_Imm), InsnID(InsnID), Imm(Imm) {} + ImmRenderer(unsigned InsnID, int64_t Imm, const LLTCodeGen &CImmLLT) + : OperandRenderer(OR_Imm), InsnID(InsnID), Imm(Imm), CImmLLT(CImmLLT) { + KnownTypes.insert(CImmLLT); + } + static bool classof(const OperandRenderer *R) { return R->getKind() == OR_Imm; } void emitRenderOpcodes(MatchTable &Table, RuleMatcher &Rule) const override { - Table << MatchTable::Opcode("GIR_AddImm") << MatchTable::Comment("InsnID") - << MatchTable::IntValue(InsnID) << MatchTable::Comment("Imm") - << MatchTable::IntValue(Imm) << MatchTable::LineBreak; + if (CImmLLT) { + assert(Table.isCombiner() && + "ConstantInt immediate are only for combiners!"); + Table << MatchTable::Opcode("GIR_AddCImm") + << MatchTable::Comment("InsnID") << MatchTable::IntValue(InsnID) + << MatchTable::Comment("Type") + << MatchTable::NamedValue(CImmLLT->getCxxEnumValue()) + << MatchTable::Comment("Imm") << MatchTable::IntValue(Imm) + << MatchTable::LineBreak; + } else { + Table << MatchTable::Opcode("GIR_AddImm") << MatchTable::Comment("InsnID") + << MatchTable::IntValue(InsnID) << MatchTable::Comment("Imm") + << MatchTable::IntValue(Imm) << MatchTable::LineBreak; + } } }; diff --git a/llvm/utils/TableGen/GlobalISelMatchTableExecutorEmitter.h b/llvm/utils/TableGen/GlobalISelMatchTableExecutorEmitter.h --- a/llvm/utils/TableGen/GlobalISelMatchTableExecutorEmitter.h +++ b/llvm/utils/TableGen/GlobalISelMatchTableExecutorEmitter.h @@ -105,7 +105,8 @@ if (!Predicates.empty()) { OS << " switch (PredicateID) {\n"; for (const auto &Pred : Predicates) { - const auto Code = GetPredCode(Pred); + // Ensure all code is indented. + const auto Code = join(split(GetPredCode(Pred).str(), "\n"), "\n "); OS << " case GICXXPred_" << TypeIdentifier << "_Predicate_" << GetPredEnumName(Pred) << ": {\n" << " " << Code << "\n"; diff --git a/llvm/utils/TableGen/GlobalISelMatchTableExecutorEmitter.cpp b/llvm/utils/TableGen/GlobalISelMatchTableExecutorEmitter.cpp --- a/llvm/utils/TableGen/GlobalISelMatchTableExecutorEmitter.cpp +++ b/llvm/utils/TableGen/GlobalISelMatchTableExecutorEmitter.cpp @@ -222,7 +222,8 @@ ", const MatcherState &State) " "const override;\n" << " bool testSimplePredicate(unsigned PredicateID) const override;\n" - << " void runCustomAction(unsigned FnID, const MatcherState &State) " + << " void runCustomAction(unsigned FnID, const MatcherState &State, " + "NewMIVector &OutMIs) " "const override;\n"; emitAdditionalTemporariesDecl(OS, " "); OS << "#endif // ifdef " << IfDefName << "\n\n";