[TableGen][SubtargetEmitter] Add the ability for processor models to describe dependency breaking instructions.

[andreadb]

Summary

This patch adds the ability for processor models to describe dependency breaking instructions.

Different processors may specify a different set of dependency-breaking instructions.
That means, we cannot assume that all processors of the same target would use the same rules to classify dependency breaking instructions.

The main goal of this patch is to provide the means to describe dependency breaking instructions directly via tablegen, and have the following TargetSubtargetInfo hooks redefined in overrides by tabegen'd XXXGenSubtargetInfo classes (here, XXX is a Target name).

virtual bool isZeroIdiom(const MachineInstr *MI, APInt &Mask) const {
  return false;
}

virtual bool isDependencyBreaking(const MachineInstr *MI, APInt &Mask) const {
  return isZeroIdiom(MI);
}

An instruction MI is a dependency-breaking instruction if a call to method isDependencyBreaking(MI) on the STI (TargetSubtargetInfo object) evaluates to true. Similarly, an instruction MI is a special case of zero-idiom dependency breaking instruction if a call to STI.isZeroIdiom(MI) returns true.
The extra APInt is used for those targets that may want to select which machine operands have their dependency broken (see comments in code).
Note that by default, subtargets don't know about the existence of dependency-breaking. In the absence of external information, those method calls would always return false.

A new tablegen class named STIPredicate has been added by this patch to let processor models classify instructions that have properties in common. The idea is that, a MCInstrPredicate definition can be used to "generate" an instruction equivalence class, with the idea that instructions of a same class all have a property in common.

STIPredicate definitions are essentially a collection of instruction equivalence classes.
Also, different processor models can specify a different variant of the same STIPredicate with different rules (i.e. predicates) to classify instructions. Tablegen backends (in this particular case, the SubtargetEmitter) will be able to process STIPredicate definitions, and automatically generate functions in XXXGenSubtargetInfo.

This patch introduces two special kind of STIPredicate classes named IsZeroIdiomFunction and IsDepBreakingFunction in tablegen. It also adds a definition for those in the BtVer2 scheduling model only.

The definition of zero-idioms in BtVer2 is quite big. For simplicity, in the example below I only reported GPR and AVX zero-idioms variants:

def : IsZeroIdiomFunction<[
  // GPR Zero-idioms.
  DepBreakingClass<[ SUB32rr, SUB64rr, XOR32rr, XOR64rr ], ZeroIdiomPredicate>,

  // AVX Zero-idioms.
  DepBreakingClass<[
    VPXORrr, VPANDNrr, VXORPSrr, VXORPDrr,
    VXORPSYrr, VXORPDYrr, VANDNPSrr, VANDNPDrr,
    VPSUBBrr, VPSUBDrr, VPSUBQrr, VPSUBWrr,
    VPCMPGTBrr, VPCMPGTDrr, VPCMPGTQrr, VPCMPGTWrr
  ], ZeroIdiomPredicate>
>;

This is what the SubtargetEmitter generates for those variants:

bool X86GenSubtargetInfo::isZeroIdiom(const MachineInstr *MI, APInt &Mask) const {
  unsigned ProcessorID = getSchedModel().getProcessorID();
  switch(MI->getOpcode()) {
  default:
    break;
  case X86::SUB32rr:
  case X86::SUB64rr:
  case X86::XOR32rr:
  case X86::XOR64rr:
  case X86::VPXORrr:
  case X86::VPANDNrr:
  case X86::VXORPSrr:
  case X86::VXORPDrr:
  case X86::VXORPSYrr:
  case X86::VXORPDYrr:
  case X86::VANDNPSrr:
  case X86::VANDNPDrr:
  case X86::VPSUBBrr:
  case X86::VPSUBDrr:
  case X86::VPSUBQrr:
  case X86::VPSUBWrr:
  case X86::VPCMPGTBrr:
  case X86::VPCMPGTDrr:
  case X86::VPCMPGTQrr:
  case X86::VPCMPGTWrr:
    if (ProcessorID == 4) {
      Mask.clearAllBits();
      return MI->getOperand(1).getReg() == MI->getOperand(2).getReg();
    }
    break;
  }

  return false;
} // X86GenSubtargetInfo::isZeroIdiom

Rules for different zero-idioms are discriminated based on the processor identifier which comes from the scheduling model.
Note that a similar definition can be generated for MCInst. This patch shows how to do it, and those extra definitions are currently expanded into X86MCInstrDesc.

This patch supersedes the one committed at r338372 for D49310.

The main advantages are:

• we can describe subtarget predicates via tablegen using STIPredicates.
• we can describe zero-idioms / dep-breaking instructions directly via tablegen in the scheduling models.

In future, the STIPredicates framework can be used for solving other problems. For example:

• teach how to identify optimizable register-register moves
• teach how to identify slow LEA instructions (each subtarget defining its own concept of "slow" LEA).
• teach how to identify instructions that have undocumented false dependencies on the output registers on some processors only.
• etc.

It is also (in my opinion) an elegant way to expose knowledge to both external tools like llvm-mca, and codegen passes.
For example, machine schedulers in LLVM could reuse that information when internally constructing the data dependency graph for a code region.

This new design feature is also an "opt-in" feature. Processor models don't have to use the new STIPredicates. It has all been designed to be as unintrusive as possible.

Please let me know what you think.

Thanks,
Andrea

[RKSimon]

include/llvm/CodeGen/TargetSubtargetInfo.h

152

relationship

include/llvm/MC/MCInstrAnalysis.h

94

\param MI

114

Please can you make it clear that Mask's bitwidth is total number of operands or just register operands?

[RKSimon]

Some minors

lib/Target/X86/X86ScheduleBtVer2.td

716

VANDNPSYrr/VANDNPDYrr?

test/tools/llvm-mca/X86/BtVer2/zero-idioms-avx-256.s

68

Shouldn't this only take a single resource cycle (0.5 rtp)? IIRC dep-breaking 256-bit ops only needs to process the upper half

tools/llvm-mca/include/Instruction.h

319

Comments?

tools/llvm-mca/lib/Stages/DispatchStage.cpp

115

Does this comment need updating to be zero-idiom specific?

utils/TableGen/CodeGenSchedule.cpp

412

Pass by const ref?

utils/TableGen/CodeGenSchedule.h

311

Why SmallVector here - the other added classes use std::vector

[andreadb]

lib/Target/X86/X86ScheduleBtVer2.td

716

Interestingly, those were missing in the original implementation of X86InstrAnalysis::isDependencyBreaking().

I have added them to the set.
I have also added two extra tests for VANDNPSYrr/VANDNPDYrr in test/tools/llvm-mca/BtVer2/zero-idioms-avx-256.s

test/tools/llvm-mca/X86/BtVer2/zero-idioms-avx-256.s

68

Nice catch.

I think the latency/throughput of instructions should be fixed by a separate patch. This patch should only help to identify independent operands of an instruction.
I will add a TODO to this test.

utils/TableGen/CodeGenSchedule.h

311

That vector is expected to be small. In the worst case scenario, it has exactly one element per processor model. It is initialized to 8, because most targets define less than 8 predicates. X86 is the target with most models (9).

If you want, I can use a vector here for consistency.

[andreadb]

Patch updated.

Addressed review comments.

Added a TODO comment to test zero-idioms-avx-256.s. Processor resource cycles consumed by VXORPSYrr and VXORPDYrr are wrongly set to 2cy (instead of 1cy).

[mattd]

I like this change. I have a few nits in the comments, but this looks good to me, caveat: I am not a tablegen expert.

include/llvm/Target/TargetInstrPredicate.td

245

*instructions

266

*remove one of the 'operands'

[andreadb]

Patch updated.

Addressed review comments.

[andreadb]

Patch rebased.

[courbet]

Nice! I only have one comment on the schema, the rest is cosmetic.

include/llvm/Target/TargetInstrPredicate.td

292

This feels like a double negation to me. What about "ExpandForMC" ? It would also make it more future-rpoof if we want to expand to more stuff.

tools/llvm-mca/lib/InstrBuilder.cpp

457

This could use two temporary bool variables for readability.

utils/TableGen/CodeGenSchedule.cpp

257

*processSTIPredicate

[andreadb]

include/llvm/Target/TargetInstrPredicate.td

292

Sure, I will change it. Thanks for the feedback!

306

I will update this comment too.

tools/llvm-mca/lib/InstrBuilder.cpp

457

I will change it.
I tried to expand that if-stmt a bit, adding extra code comments. Hopefully, it is more readable with code comments. Let me know if you prefer it done in a different way.

utils/TableGen/CodeGenSchedule.cpp

257

I will fix it. Thanks.

[andreadb]

Address review comments.

Also:

• Improved the description of class PredicateInfo.
• Removed two unused fields from tablegen class STIPredicate.
• Added extra verification checks to: a) explicitly disallow InstructionEquivalenceClass definitions with an empty set of opcodes; b) avoid that an instruction opcode is used by multiple equivalence classes of a same STIPredicate.

[RKSimon]

LGTM - the vandps/vandpd btver2 zero-idiom cases need cleaning up but have confirmed with @andreadb offline that he'll do this as a follow-up

lib/Target/X86/X86ScheduleBtVer2.td

716

Possibly sort all these instructions so its easier to find a specific one?

include/llvm/CodeGen/TargetSubtargetInfo.h

 //===- llvm/CodeGen/TargetSubtargetInfo.h - Target Information --*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // This file describes the subtarget options of a Target machine.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_CODEGEN_TARGETSUBTARGETINFO_H
 #define LLVM_CODEGEN_TARGETSUBTARGETINFO_H
 
+#include "llvm/ADT/APInt.h"
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/CodeGen/PBQPRAConstraint.h"
 #include "llvm/CodeGen/ScheduleDAGMutation.h"
 #include "llvm/CodeGen/SchedulerRegistry.h"
 #include "llvm/MC/MCSubtargetInfo.h"
 #include "llvm/Support/CodeGen.h"
public:
   /// another variant SchedClass, but repeated invocation must quickly terminate
   /// in a nonvariant SchedClass.
   virtual unsigned resolveSchedClass(unsigned SchedClass,
                                      const MachineInstr *MI,
                                      const TargetSchedModel *SchedModel) const {
     return 0;
   }
 
+  /// Returns true if \param MI is a dependency breaking zero-idiom instruction
+  /// for the subtarget.
+  ///
+  /// This function also sets bits in \param Mask related to input operands that
+  /// are not in a data dependency relatioship.  There is one bit for each
+  /// machine operand; implicit operands follow explicit operands in the bit
+  /// representation used for \param Mask.  An empty \param Mask (i.e. a mask
+  /// with all bits cleared) means: data dependencies are "broken" for all the
+  /// explicit input machine operands of \param MI.
+  virtual bool isZeroIdiom(const MachineInstr *MI, APInt &Mask) const {
+    return false;
+  }
+
+  /// Returns true if \param MI is a dependency breaking instruction for the
+  /// subtarget.
+  ///
+  /// Similar in behavior to `isZeroIdiom`. However, it knows how to identify
+  /// all dependency breaking instructions (i.e. not just zero-idioms).
+  /// 
+  /// As for `isZeroIdiom`, this method returns a mask of "broken" dependencies.
+  /// (See method `isZeroIdiom` for a detailed description of \param Mask).
+  virtual bool isDependencyBreaking(const MachineInstr *MI, APInt &Mask) const {
+    return isZeroIdiom(MI, Mask);
+  }
+
   /// True if the subtarget should run MachineScheduler after aggressive
   /// coalescing.
   ///
   /// This currently replaces the SelectionDAG scheduler with the "source" order
   /// scheduler (though see below for an option to turn this off and use the
   /// TargetLowering preference). It does not yet disable the postRA scheduler.
   virtual bool enableMachineScheduler() const;
 

include/llvm/MC/MCInstrAnalysis.h

public:
   ///
   /// The assumption is that the bit-width of the APInt is correctly set by
   /// the caller. The default implementation conservatively assumes that none of
   /// the writes clears the upper portion of a super-register.
   virtual bool clearsSuperRegisters(const MCRegisterInfo &MRI,
                                     const MCInst &Inst,
                                     APInt &Writes) const;
 
-  /// Returns true if \param Inst is a dependency breaking instruction for the
+  /// Returns true if \param Inst is a dependency breaking zero-idiom for the
   /// given subtarget.
   ///
+  /// If \param Inst is a dependency breaking zero-idiom instruction for
+  /// processor \param CPUID, then bits of \param Mask which correspond to
+  /// independent input register operands are set. Other bits of the mask are
+  /// cleared. A mask with all bits cleared means: dependencies are broken for
+  /// all explicit input register operands of \param MI.
+  virtual bool isZeroIdiom(const MCInst &MI, APInt &Mask,
+                           unsigned CPUID) const {
+    return false;
+  }
+
+  /// Returns true if \param Inst is a dependency breaking instruction for the
+  /// subtarget associated with \param CPUID.
+  ///
   /// The value computed by a dependency breaking instruction is not dependent
   /// on the inputs. An example of dependency breaking instruction on X86 is
   /// `XOR %eax, %eax`.
-  /// TODO: In future, we could implement an alternative approach where this
-  /// method returns `true` if the input instruction is not dependent on
-  /// some/all of its input operands. An APInt mask could then be used to
-  /// identify independent operands.
-  virtual bool isDependencyBreaking(const MCSubtargetInfo &STI,
-                                    const MCInst &Inst) const;
+  ///
+  /// If \param Inst is a dependency breaking instruction for subtarget
+  /// \param CPUID, then bits of \param Mask which correspond to independent
+  /// input register operands are set. Other bits of the mask are cleared.
+  virtual bool isDependencyBreaking(const MCInst &MI, APInt &Mask,
+                                    unsigned CPUID) const {
+    return isZeroIdiom(MI, Mask, CPUID);
+  }
 
   /// Given a branch instruction try to get the address the branch
   /// targets. Return true on success, and the address in Target.
   virtual bool
   evaluateBranch(const MCInst &Inst, uint64_t Addr, uint64_t Size,
                  uint64_t &Target) const;
 
   /// Returns (PLT virtual address, GOT virtual address) pairs for PLT entries.

include/llvm/Target/TargetInstrPredicate.td

 //
 // New MCInstPredicate classes must be added to this file. For each new class
 // XYZ, an "expandXYZ" method must be added to the PredicateExpander.
 //
 //===----------------------------------------------------------------------===//
 
 // Forward declarations.
 class Instruction;
+class SchedMachineModel;
 
 // A generic machine instruction predicate.
 class MCInstPredicate;
 
 class MCTrue  : MCInstPredicate;   // A predicate that always evaluates to True.
 class MCFalse : MCInstPredicate;   // A predicate that always evaluates to False.
 def TruePred  : MCTrue;
 def FalsePred : MCFalse;

 // a call to `MCInstFn`, or into a call to`MachineInstrFn` depending on whether
 // it is lowering predicates for MCInst or MachineInstr.
 //
 // In this context, `MCInstFn` and `MachineInstrFn` are both function names.
 class CheckFunctionPredicate<string MCInstFn, string MachineInstrFn> : MCInstPredicate {
   string MCInstFnName = MCInstFn;
   string MachineInstrFnName = MachineInstrFn;
 }
+
+// Used to classify machine instructions based on a machine instruction
+// predicate.
+//
+// Let IC be an InstructionEquivalenceClass definition, and MI a machine
+// instruction.  We say that MI belongs to the equivalence class described by IC
+// if and only if the following two conditions are met:
+//  a) MI's opcode is in the `opcodes` set, and
+//  b) `Predicate` evaluates to true when applied to MI.
+//
+// Instances of this class can be used by processor scheduling models to
+// describe instruction that have a property in common.  For example,
+// InstructionEquivalenceClass definitions can be used to identify the set of
+// dependency breaking instructions for a processor model.
+//
+// An (optional) list of operand indices can be used to further describe
+// properties that apply to instruction operands. For example, it can be used to
+// identify register uses of a dependency breaking instructions that are not in
+// a RAW dependency.
+class InstructionEquivalenceClass<list<Instruction> opcodes,
+                                  MCInstPredicate pred,
+                                  list<int> operands = []> {
+  list<Instruction> Opcodes = opcodes;
+  MCInstPredicate Predicate = pred;
+  list<int> OperandIndices = operands;
+}
+
+// Used by processor models to describe dependency breaking instructions.
+//
+// This is mainly an alias for InstructionEquivalenceClass.  Input operand
+// `BrokenDeps` identifies the set of "broken dependencies". There is one bit
+// per each implicit and explicit input operand.  An empty set of broken
+// dependencies means: "explicit input register operands operands are
+// independent."
+class DepBreakingClass<list<Instruction> opcodes, MCInstPredicate pred,
+                       list<int> BrokenDeps = []>
+    : InstructionEquivalenceClass<opcodes, pred, BrokenDeps>;
+
+// A function descriptor used to describe the signature of a predicate methods
+// which will be expanded by the STIPredicateExpander into a tablegen'd
+// XXXGenSubtargetInfo class member definition (here, XXX is a target name).
+//
+// It describes the signature of a TargetSubtarget hook, as well as a few extra
+// properties. Examples of extra properties are:
+//  - The default return value for the auto-generate function hook.
+//  - A list of subtarget hooks (Delegates) that are called from this function.
+//
+class STIPredicateDecl<string name, MCInstPredicate default = FalsePred,
+                       bit overrides = 1, bit machineInstrOnly = 0,
+                       bit updatesOpcodeMask = 0,
+                       list<STIPredicateDecl> delegates = []> {
+  string Name = name;
+  MCInstPredicate DefaultReturnValue = default;
+
+  // True if this method is declared as virtual in class TargetSubtargetInfo.
+  bit OverridesBaseClassMember = overrides;
+
+  // True if this information is only meant to be used by CodeGen passes, and
+  // not exposed to the MC layer.
+  bit ExpandForMachineInstrOnly = machineInstrOnly;
+
+  // True if the autogenerated method has a extra in/out APInt param used as a
+  // mask of operands.
+  bit UpdatesOpcodeMask = 0;
+
+  // A list of STIPredicates used by this definition to delegate part of the
+  // computation. For example, STIPredicateFunction `isDependencyBreaking()`
+  // delegates to `isZeroIdiom()` part of its computation.
+  list<STIPredicateDecl> Delegates = delegates;
+}
+
+// A predicate function definition member of class `XXXGenSubtargetInfo`.
+//
+// If `Declaration.ExpandForMachineInstrOnly` is false, then SubtargetEmitter
+// will also expand another definition of this method that accepts a MCInst.
+class STIPredicate<STIPredicateDecl declaration,
+                   list<InstructionEquivalenceClass> classes,
+                   MCInstPredicate default = FalsePred,
+                   list<STIPredicateDecl> delegates = []> {
+  STIPredicateDecl Declaration = declaration;
+  list<InstructionEquivalenceClass> Classes = classes;
+  SchedMachineModel SchedModel = ?;
+}
+
+// Convenience classes and definitions used by processor scheduling models to
+// describe dependency breaking instructions.
+let UpdatesOpcodeMask = 1 in {
+
+def IsZeroIdiomDecl : STIPredicateDecl<"isZeroIdiom">;
+
+let Delegates = [IsZeroIdiomDecl] in
+def IsDepBreakingDecl : STIPredicateDecl<"isDependencyBreaking">;
+
+} // UpdatesOpcodeMask
+
+class IsZeroIdiomFunction<list<DepBreakingClass> classes>
+    : STIPredicate<IsZeroIdiomDecl, classes>;
+
+class IsDepBreakingFunction<list<DepBreakingClass> classes>
+    : STIPredicate<IsDepBreakingDecl, classes>;

lib/MC/MCInstrAnalysis.cpp

 
 bool MCInstrAnalysis::clearsSuperRegisters(const MCRegisterInfo &MRI,
                                            const MCInst &Inst,
                                            APInt &Writes) const {
   Writes.clearAllBits();
   return false;
 }
 
-bool MCInstrAnalysis::isDependencyBreaking(const MCSubtargetInfo &STI,
-                                           const MCInst &Inst) const {
-  return false;
-}
-
 bool MCInstrAnalysis::evaluateBranch(const MCInst &Inst, uint64_t Addr,
                                      uint64_t Size, uint64_t &Target) const {
   if (Inst.getNumOperands() == 0 ||
       Info->get(Inst.getOpcode()).OpInfo[0].OperandType != MCOI::OPERAND_PCREL)
     return false;
 
   int64_t Imm = Inst.getOperand(0).getImm();
   Target = Addr+Size+Imm;
   return true;
 }

lib/Target/X86/MCTargetDesc/X86MCTargetDesc.cpp

 class X86MCInstrAnalysis : public MCInstrAnalysis {
   X86MCInstrAnalysis(const X86MCInstrAnalysis &) = delete;
   X86MCInstrAnalysis &operator=(const X86MCInstrAnalysis &) = delete;
   virtual ~X86MCInstrAnalysis() = default;
 
 public:
   X86MCInstrAnalysis(const MCInstrInfo *MCII) : MCInstrAnalysis(MCII) {}
 
-  bool isDependencyBreaking(const MCSubtargetInfo &STI,
-                            const MCInst &Inst) const override;
+#define GET_STIPREDICATE_DECLS_FOR_MC_ANALYSIS
+#include "X86GenSubtargetInfo.inc"
+
   bool clearsSuperRegisters(const MCRegisterInfo &MRI, const MCInst &Inst,
                             APInt &Mask) const override;
   std::vector<std::pair<uint64_t, uint64_t>>
   findPltEntries(uint64_t PltSectionVA, ArrayRef<uint8_t> PltContents,
                  uint64_t GotSectionVA,
                  const Triple &TargetTriple) const override;
 };
 
-bool X86MCInstrAnalysis::isDependencyBreaking(const MCSubtargetInfo &STI,
-                                              const MCInst &Inst) const {
-  if (STI.getCPU() == "btver2") {
-    // Reference: Agner Fog's microarchitecture.pdf - Section 20 "AMD Bobcat and
-    // Jaguar pipeline", subsection 8 "Dependency-breaking instructions".
-    switch (Inst.getOpcode()) {
-    default:
-      return false;
-    case X86::SUB32rr:
-    case X86::SUB64rr:
-    case X86::SBB32rr:
-    case X86::SBB64rr:
-    case X86::XOR32rr:
-    case X86::XOR64rr:
-    case X86::XORPSrr:
-    case X86::XORPDrr:
-    case X86::VXORPSrr:
-    case X86::VXORPDrr:
-    case X86::ANDNPSrr:
-    case X86::VANDNPSrr:
-    case X86::ANDNPDrr:
-    case X86::VANDNPDrr:
-    case X86::PXORrr:
-    case X86::VPXORrr:
-    case X86::PANDNrr:
-    case X86::VPANDNrr:
-    case X86::PSUBBrr:
-    case X86::PSUBWrr:
-    case X86::PSUBDrr:
-    case X86::PSUBQrr:
-    case X86::VPSUBBrr:
-    case X86::VPSUBWrr:
-    case X86::VPSUBDrr:
-    case X86::VPSUBQrr:
-    case X86::PCMPEQBrr:
-    case X86::PCMPEQWrr:
-    case X86::PCMPEQDrr:
-    case X86::PCMPEQQrr:
-    case X86::VPCMPEQBrr:
-    case X86::VPCMPEQWrr:
-    case X86::VPCMPEQDrr:
-    case X86::VPCMPEQQrr:
-    case X86::PCMPGTBrr:
-    case X86::PCMPGTWrr:
-    case X86::PCMPGTDrr:
-    case X86::PCMPGTQrr:
-    case X86::VPCMPGTBrr:
-    case X86::VPCMPGTWrr:
-    case X86::VPCMPGTDrr:
-    case X86::VPCMPGTQrr:
-    case X86::MMX_PXORirr:
-    case X86::MMX_PANDNirr:
-    case X86::MMX_PSUBBirr:
-    case X86::MMX_PSUBDirr:
-    case X86::MMX_PSUBQirr:
-    case X86::MMX_PSUBWirr:
-    case X86::MMX_PCMPGTBirr:
-    case X86::MMX_PCMPGTDirr:
-    case X86::MMX_PCMPGTWirr:
-    case X86::MMX_PCMPEQBirr:
-    case X86::MMX_PCMPEQDirr:
-    case X86::MMX_PCMPEQWirr:
-      return Inst.getOperand(1).getReg() == Inst.getOperand(2).getReg();
-    case X86::CMP32rr:
-    case X86::CMP64rr:
-      return Inst.getOperand(0).getReg() == Inst.getOperand(1).getReg();
-    }
-  }
-
-  return false;
-}
+#define GET_STIPREDICATE_DEFS_FOR_MC_ANALYSIS
+#include "X86GenSubtargetInfo.inc"
 
 bool X86MCInstrAnalysis::clearsSuperRegisters(const MCRegisterInfo &MRI,
                                               const MCInst &Inst,
                                               APInt &Mask) const {
   const MCInstrDesc &Desc = Info->get(Inst.getOpcode());
   unsigned NumDefs = Desc.getNumDefs();
   unsigned NumImplicitDefs = Desc.getNumImplicitDefs();
   assert(Mask.getBitWidth() == NumDefs + NumImplicitDefs &&

lib/Target/X86/X86ScheduleBtVer2.td

 
 def JSlowLEA16r : SchedWriteRes<[JALU01]> {
   let Latency = 3;
   let ResourceCycles = [4];
 }
 
 def : InstRW<[JSlowLEA16r], (instrs LEA16r)>;
 
+///////////////////////////////////////////////////////////////////////////////
+// Dependency breaking instructions.
+///////////////////////////////////////////////////////////////////////////////
+
+def : IsZeroIdiomFunction<[
+  // GPR Zero-idioms.
+  DepBreakingClass<[ SUB32rr, SUB64rr, XOR32rr, XOR64rr ], ZeroIdiomPredicate>,
+
+  // MMX Zero-idioms.
+  DepBreakingClass<[
+    MMX_PXORirr, MMX_PANDNirr, MMX_PSUBBirr,
+    MMX_PSUBDirr, MMX_PSUBQirr, MMX_PSUBWirr,
+    MMX_PCMPGTBirr, MMX_PCMPGTDirr, MMX_PCMPGTWirr
+  ], ZeroIdiomPredicate>,
+
+  // SSE Zero-idioms.
+  DepBreakingClass<[
+    PXORrr, PANDNrr, XORPSrr, XORPDrr,
+    ANDNPSrr, ANDNPDrr, PSUBBrr, PSUBDrr,
+    PSUBQrr, PSUBWrr, PCMPGTBrr, PCMPGTDrr,
+    PCMPGTQrr, PCMPGTWrr
+  ], ZeroIdiomPredicate>,
+
+  // AVX Zero-idioms.
+  DepBreakingClass<[
+    VPXORrr, VPANDNrr, VXORPSrr, VXORPDrr,
+    VXORPSYrr, VXORPDYrr, VANDNPSrr, VANDNPDrr,
+    VPSUBBrr, VPSUBDrr, VPSUBQrr, VPSUBWrr,
+    VPCMPGTBrr, VPCMPGTDrr, VPCMPGTQrr, VPCMPGTWrr
+  ], ZeroIdiomPredicate>
+]>;
+
+def : IsDepBreakingFunction<[
+  // GPR
+  DepBreakingClass<[ SBB32rr, SBB64rr ], ZeroIdiomPredicate>,
+  DepBreakingClass<[ CMP32rr, CMP64rr ], CheckSameRegOperand<0, 1> >,
+
+  // MMX
+  DepBreakingClass<[
+    MMX_PCMPEQBirr, MMX_PCMPEQDirr, MMX_PCMPEQWirr
+  ], ZeroIdiomPredicate>,
+
+  // SSE
+  DepBreakingClass<[ 
+    PCMPEQBrr, PCMPEQWrr, PCMPEQDrr, PCMPEQQrr
+  ], ZeroIdiomPredicate>,
+
+  // AVX
+  DepBreakingClass<[
+    VPCMPEQBrr, VPCMPEQWrr, VPCMPEQDrr, VPCMPEQQrr
+  ], ZeroIdiomPredicate>
+]>;
+
 } // SchedModel

test/tools/llvm-mca/X86/BtVer2/zero-idioms-avx-256.s

+# NOTE: Assertions have been autogenerated by utils/update_mca_test_checks.py
+# RUN: llvm-mca -mtriple=x86_64-unknown-unknown -mcpu=btver2 -timeline -timeline-max-iterations=3 < %s | FileCheck %s
+
+# LLVM-MCA-BEGIN ZERO-IDIOM-1
+
+vaddps %ymm0, %ymm0, %ymm1
+vxorps %ymm1, %ymm1, %ymm1
+vblendps $2, %ymm1, %ymm2, %ymm3
+
+# LLVM-MCA-END
+
+# LLVM-MCA-BEGIN ZERO-IDIOM-2
+
+vaddpd %ymm0, %ymm0, %ymm1
+vxorpd %ymm1, %ymm1, %ymm1
+vblendpd $2, %ymm1, %ymm2, %ymm3
+
+# LLVM-MCA-END
+
+# CHECK:      [0] Code Region - ZERO-IDIOM-1
+
+# CHECK:      Iterations:        100
+# CHECK-NEXT: Instructions:      300
+# CHECK-NEXT: Total Cycles:      306
+# CHECK-NEXT: Total uOps:        600
+
+# CHECK:      Dispatch Width:    2
+# CHECK-NEXT: uOps Per Cycle:    1.96
+# CHECK-NEXT: IPC:               0.98
+# CHECK-NEXT: Block RThroughput: 3.0
+
+# CHECK:      Instruction Info:
+# CHECK-NEXT: [1]: #uOps
+# CHECK-NEXT: [2]: Latency
+# CHECK-NEXT: [3]: RThroughput
+# CHECK-NEXT: [4]: MayLoad
+# CHECK-NEXT: [5]: MayStore
+# CHECK-NEXT: [6]: HasSideEffects (U)
+
+# CHECK:      [1]    [2]    [3]    [4]    [5]    [6]    Instructions:
+# CHECK-NEXT:  2      3     2.00                        vaddps  %ymm0, %ymm0, %ymm1
+# CHECK-NEXT:  2      1     1.00                        vxorps  %ymm1, %ymm1, %ymm1
+# CHECK-NEXT:  2      1     1.00                        vblendps  $2, %ymm1, %ymm2, %ymm3
+
+# CHECK:      Resources:
+# CHECK-NEXT: [0]   - JALU0
+# CHECK-NEXT: [1]   - JALU1
+# CHECK-NEXT: [2]   - JDiv
+# CHECK-NEXT: [3]   - JFPA
+# CHECK-NEXT: [4]   - JFPM
+# CHECK-NEXT: [5]   - JFPU0
+# CHECK-NEXT: [6]   - JFPU1
+# CHECK-NEXT: [7]   - JLAGU
+# CHECK-NEXT: [8]   - JMul
+# CHECK-NEXT: [9]   - JSAGU
+# CHECK-NEXT: [10]  - JSTC
+# CHECK-NEXT: [11]  - JVALU0
+# CHECK-NEXT: [12]  - JVALU1
+# CHECK-NEXT: [13]  - JVIMUL
+
+# CHECK:      Resource pressure per iteration:
+# CHECK-NEXT: [0]    [1]    [2]    [3]    [4]    [5]    [6]    [7]    [8]    [9]    [10]   [11]   [12]   [13]
+# CHECK-NEXT:  -      -      -     3.00   3.00   3.00   3.00    -      -      -      -      -      -      -
+
+# CHECK:      Resource pressure by instruction:
+# CHECK-NEXT: [0]    [1]    [2]    [3]    [4]    [5]    [6]    [7]    [8]    [9]    [10]   [11]   [12]   [13]   Instructions:
+# CHECK-NEXT:  -      -      -     2.00    -     2.00    -      -      -      -      -      -      -      -     vaddps  %ymm0, %ymm0, %ymm1
+# CHECK-NEXT:  -      -      -      -     2.00    -     2.00    -      -      -      -      -      -      -     vxorps  %ymm1, %ymm1, %ymm1
+# CHECK-NEXT:  -      -      -     1.00   1.00   1.00   1.00    -      -      -      -      -      -      -     vblendps  $2, %ymm1, %ymm2, %ymm3
+
+# CHECK:      Timeline view:
+# CHECK-NEXT:                     012
+# CHECK-NEXT: Index     0123456789
+
+# CHECK:      [0,0]     DeeeER    . .   vaddps  %ymm0, %ymm0, %ymm1
+# CHECK-NEXT: [0,1]     .DeE-R    . .   vxorps  %ymm1, %ymm1, %ymm1
+# CHECK-NEXT: [0,2]     . DeE-R   . .   vblendps  $2, %ymm1, %ymm2, %ymm3
+# CHECK-NEXT: [1,0]     .  D=eeeER. .   vaddps  %ymm0, %ymm0, %ymm1
+# CHECK-NEXT: [1,1]     .   DeE--R. .   vxorps  %ymm1, %ymm1, %ymm1
+# CHECK-NEXT: [1,2]     .    D=eE-R .   vblendps  $2, %ymm1, %ymm2, %ymm3
+# CHECK-NEXT: [2,0]     .    .DeeeER.   vaddps  %ymm0, %ymm0, %ymm1
+# CHECK-NEXT: [2,1]     .    . D=eER.   vxorps  %ymm1, %ymm1, %ymm1
+# CHECK-NEXT: [2,2]     .    .  D=eER   vblendps  $2, %ymm1, %ymm2, %ymm3
+
+# CHECK:      Average Wait times (based on the timeline view):
+# CHECK-NEXT: [0]: Executions
+# CHECK-NEXT: [1]: Average time spent waiting in a scheduler's queue
+# CHECK-NEXT: [2]: Average time spent waiting in a scheduler's queue while ready
+# CHECK-NEXT: [3]: Average time elapsed from WB until retire stage
+
+# CHECK:            [0]    [1]    [2]    [3]
+# CHECK-NEXT: 0.     3     1.3    1.3    0.0       vaddps %ymm0, %ymm0, %ymm1
+# CHECK-NEXT: 1.     3     1.3    1.3    1.0       vxorps %ymm1, %ymm1, %ymm1
+# CHECK-NEXT: 2.     3     1.7    0.3    0.7       vblendps $2, %ymm1, %ymm2, %ymm3
+
+# CHECK:      [1] Code Region - ZERO-IDIOM-2
+
+# CHECK:      Iterations:        100
+# CHECK-NEXT: Instructions:      300
+# CHECK-NEXT: Total Cycles:      306
+# CHECK-NEXT: Total uOps:        600
+
+# CHECK:      Dispatch Width:    2
+# CHECK-NEXT: uOps Per Cycle:    1.96
+# CHECK-NEXT: IPC:               0.98
+# CHECK-NEXT: Block RThroughput: 3.0
+
+# CHECK:      Instruction Info:
+# CHECK-NEXT: [1]: #uOps
+# CHECK-NEXT: [2]: Latency
+# CHECK-NEXT: [3]: RThroughput
+# CHECK-NEXT: [4]: MayLoad
+# CHECK-NEXT: [5]: MayStore
+# CHECK-NEXT: [6]: HasSideEffects (U)
+
+# CHECK:      [1]    [2]    [3]    [4]    [5]    [6]    Instructions:
+# CHECK-NEXT:  2      3     2.00                        vaddpd  %ymm0, %ymm0, %ymm1
+# CHECK-NEXT:  2      1     1.00                        vxorpd  %ymm1, %ymm1, %ymm1
+# CHECK-NEXT:  2      1     1.00                        vblendpd  $2, %ymm1, %ymm2, %ymm3
+
+# CHECK:      Resources:
+# CHECK-NEXT: [0]   - JALU0
+# CHECK-NEXT: [1]   - JALU1
+# CHECK-NEXT: [2]   - JDiv
+# CHECK-NEXT: [3]   - JFPA
+# CHECK-NEXT: [4]   - JFPM
+# CHECK-NEXT: [5]   - JFPU0
+# CHECK-NEXT: [6]   - JFPU1
+# CHECK-NEXT: [7]   - JLAGU
+# CHECK-NEXT: [8]   - JMul
+# CHECK-NEXT: [9]   - JSAGU
+# CHECK-NEXT: [10]  - JSTC
+# CHECK-NEXT: [11]  - JVALU0
+# CHECK-NEXT: [12]  - JVALU1
+# CHECK-NEXT: [13]  - JVIMUL
+
+# CHECK:      Resource pressure per iteration:
+# CHECK-NEXT: [0]    [1]    [2]    [3]    [4]    [5]    [6]    [7]    [8]    [9]    [10]   [11]   [12]   [13]
+# CHECK-NEXT:  -      -      -     3.00   3.00   3.00   3.00    -      -      -      -      -      -      -
+
+# CHECK:      Resource pressure by instruction:
+# CHECK-NEXT: [0]    [1]    [2]    [3]    [4]    [5]    [6]    [7]    [8]    [9]    [10]   [11]   [12]   [13]   Instructions:
+# CHECK-NEXT:  -      -      -     2.00    -     2.00    -      -      -      -      -      -      -      -     vaddpd  %ymm0, %ymm0, %ymm1
+# CHECK-NEXT:  -      -      -      -     2.00    -     2.00    -      -      -      -      -      -      -     vxorpd  %ymm1, %ymm1, %ymm1
+# CHECK-NEXT:  -      -      -     1.00   1.00   1.00   1.00    -      -      -      -      -      -      -     vblendpd  $2, %ymm1, %ymm2, %ymm3
+
+# CHECK:      Timeline view:
+# CHECK-NEXT:                     012
+# CHECK-NEXT: Index     0123456789
+
+# CHECK:      [0,0]     DeeeER    . .   vaddpd  %ymm0, %ymm0, %ymm1
+# CHECK-NEXT: [0,1]     .DeE-R    . .   vxorpd  %ymm1, %ymm1, %ymm1
+# CHECK-NEXT: [0,2]     . DeE-R   . .   vblendpd  $2, %ymm1, %ymm2, %ymm3
+# CHECK-NEXT: [1,0]     .  D=eeeER. .   vaddpd  %ymm0, %ymm0, %ymm1
+# CHECK-NEXT: [1,1]     .   DeE--R. .   vxorpd  %ymm1, %ymm1, %ymm1
+# CHECK-NEXT: [1,2]     .    D=eE-R .   vblendpd  $2, %ymm1, %ymm2, %ymm3
+# CHECK-NEXT: [2,0]     .    .DeeeER.   vaddpd  %ymm0, %ymm0, %ymm1
+# CHECK-NEXT: [2,1]     .    . D=eER.   vxorpd  %ymm1, %ymm1, %ymm1
+# CHECK-NEXT: [2,2]     .    .  D=eER   vblendpd  $2, %ymm1, %ymm2, %ymm3
+
+# CHECK:      Average Wait times (based on the timeline view):
+# CHECK-NEXT: [0]: Executions
+# CHECK-NEXT: [1]: Average time spent waiting in a scheduler's queue
+# CHECK-NEXT: [2]: Average time spent waiting in a scheduler's queue while ready
+# CHECK-NEXT: [3]: Average time elapsed from WB until retire stage
+
+# CHECK:            [0]    [1]    [2]    [3]
+# CHECK-NEXT: 0.     3     1.3    1.3    0.0       vaddpd %ymm0, %ymm0, %ymm1
+# CHECK-NEXT: 1.     3     1.3    1.3    1.0       vxorpd %ymm1, %ymm1, %ymm1
+# CHECK-NEXT: 2.     3     1.7    0.3    0.7       vblendpd $2, %ymm1, %ymm2, %ymm3

tools/llvm-mca/include/Instruction.h

class ReadState {
   int CyclesLeft;
   // This field is updated on every writeStartEvent(). When the number of
   // dependent writes (i.e. field DependentWrite) is zero, this value is
   // propagated to field CyclesLeft.
   unsigned TotalCycles;
   // This field is set to true only if there are no dependent writes, and
   // there are no `CyclesLeft' to wait.
   bool IsReady;
+  // True if this register read is from a dependency-breaking instruction.
+  bool IndependentFromDef;
 
 public:
   ReadState(const ReadDescriptor &Desc, unsigned RegID)
       : RD(Desc), RegisterID(RegID), DependentWrites(0),
-        CyclesLeft(UNKNOWN_CYCLES), TotalCycles(0), IsReady(true) {}
+        CyclesLeft(UNKNOWN_CYCLES), TotalCycles(0), IsReady(true),
+        IndependentFromDef(false) {}
   ReadState(const ReadState &Other) = delete;
   ReadState &operator=(const ReadState &Other) = delete;
 
   const ReadDescriptor &getDescriptor() const { return RD; }
   unsigned getSchedClass() const { return RD.SchedClassID; }
   unsigned getRegisterID() const { return RegisterID; }
 
   bool isReady() const { return IsReady; }
   bool isImplicitRead() const { return RD.isImplicitRead(); }
 
+  bool isIndependentFromDef() const { return IndependentFromDef; }
+  void setIndependentFromDef() { IndependentFromDef = true; }
+
   void cycleEvent();
   void writeStartEvent(unsigned Cycles);
   void setDependentWrites(unsigned Writes) {
     DependentWrites = Writes;
     IsReady = !Writes;
   }
 };
 
class Instruction {
   // This value defaults to the instruction latency. This instruction is
   // considered executed when field CyclesLeft goes to zero.
   int CyclesLeft;
 
   // Retire Unit token ID for this instruction.
   unsigned RCUTokenID;
 
   bool IsDepBreaking;
+  bool IsZeroIdiom;
 
   using UniqueDef = std::unique_ptr<WriteState>;
   using UniqueUse = std::unique_ptr<ReadState>;
   using VecDefs = std::vector<UniqueDef>;
   using VecUses = std::vector<UniqueUse>;
 
   // Output dependencies.
   // One entry per each implicit and explicit register definition.
   VecDefs Defs;
 
   // Input dependencies.
   // One entry per each implicit and explicit register use.
   VecUses Uses;
 
 public:
   Instruction(const InstrDesc &D)
       : Desc(D), Stage(IS_INVALID), CyclesLeft(UNKNOWN_CYCLES), RCUTokenID(0),
-        IsDepBreaking(false) {}
+        IsDepBreaking(false), IsZeroIdiom(false) {}
   Instruction(const Instruction &Other) = delete;
   Instruction &operator=(const Instruction &Other) = delete;
 
   VecDefs &getDefs() { return Defs; }
   const VecDefs &getDefs() const { return Defs; }
   VecUses &getUses() { return Uses; }
   const VecUses &getUses() const { return Uses; }
   const InstrDesc &getDesc() const { return Desc; }
   unsigned getRCUTokenID() const { return RCUTokenID; }
   int getCyclesLeft() const { return CyclesLeft; }
 
   bool hasDependentUsers() const {
     return llvm::any_of(Defs, [](const UniqueDef &Def) {
       return Def->getNumUsers() > 0;
     });
   }
 
   bool isDependencyBreaking() const { return IsDepBreaking; }
+  bool isZeroIdiom() const { return IsZeroIdiom; }
   void setDependencyBreaking() { IsDepBreaking = true; }
+  void setZeroIdiom() { IsZeroIdiom = true; }
 
   unsigned getNumUsers() const {
     unsigned NumUsers = 0;
     for (const UniqueDef &Def : Defs)
       NumUsers += Def->getNumUsers();
     return NumUsers;
   }
 

tools/llvm-mca/lib/InstrBuilder.cpp

 Expected<std::unique_ptr<Instruction>>
 InstrBuilder::createInstruction(const MCInst &MCI) {
   Expected<const InstrDesc &> DescOrErr = getOrCreateInstrDesc(MCI);
   if (!DescOrErr)
     return DescOrErr.takeError();
   const InstrDesc &D = *DescOrErr;
   std::unique_ptr<Instruction> NewIS = llvm::make_unique<Instruction>(D);
 
+  // Check if this is a dependency breaking instruction.
+  APInt Mask;
+  unsigned ProcID = STI.getSchedModel().getProcessorID();
+  if (MCIA.isZeroIdiom(MCI, Mask, ProcID)) {
+    NewIS->setDependencyBreaking();
+    NewIS->setZeroIdiom();
+  } else if (MCIA.isDependencyBreaking(MCI, Mask, ProcID)) {
+    NewIS->setDependencyBreaking();
+  }
+
   // Initialize Reads first.
   for (const ReadDescriptor &RD : D.Reads) {
     int RegID = -1;
     if (!RD.isImplicitRead()) {
       // explicit read.
       const MCOperand &Op = MCI.getOperand(RD.OpIndex);
       // Skip non-register operands.
       if (!Op.isReg())
         continue;
       RegID = Op.getReg();
     } else {
       // Implicit read.
       RegID = RD.RegisterID;
     }
 
     // Skip invalid register operands.
     if (!RegID)
       continue;
 
     // Okay, this is a register operand. Create a ReadState for it.
     assert(RegID > 0 && "Invalid register ID found!");
-    NewIS->getUses().emplace_back(llvm::make_unique<ReadState>(RD, RegID));
+    auto RS = llvm::make_unique<ReadState>(RD, RegID);
+
+    if (NewIS->isDependencyBreaking()) {
+      if ((Mask.isNullValue() && !RD.isImplicitRead()) ||
+          ((Mask.getBitWidth() > RD.UseIndex) && Mask[RD.UseIndex])) {
+        RS->setIndependentFromDef();
+      }
+    }
+    NewIS->getUses().emplace_back(std::move(RS));
   }
 
   // Early exit if there are no writes.
   if (D.Writes.empty())
     return std::move(NewIS);
 
   // Track register writes that implicitly clear the upper portion of the
   // underlying super-registers using an APInt.
   APInt WriteMask(D.Writes.size(), 0);
 
   // Now query the MCInstrAnalysis object to obtain information about which
   // register writes implicitly clear the upper portion of a super-register.
   MCIA.clearsSuperRegisters(MRI, MCI, WriteMask);
 
-  // Check if this is a dependency breaking instruction.
-  if (MCIA.isDependencyBreaking(STI, MCI))
-    NewIS->setDependencyBreaking();
-
   // Initialize writes.
   unsigned WriteIndex = 0;
   for (const WriteDescriptor &WD : D.Writes) {
     unsigned RegID = WD.isImplicitWrite() ? WD.RegisterID
                                           : MCI.getOperand(WD.OpIndex).getReg();
     // Check if this is a optional definition that references NoReg.
     if (WD.IsOptionalDef && !RegID) {
       ++WriteIndex;

tools/llvm-mca/lib/Stages/DispatchStage.cpp

llvm::Error DispatchStage::dispatch(InstRef IR) {
   }
 
   // A dependency-breaking instruction doesn't have to wait on the register
   // input operands, and it is often optimized at register renaming stage.
   // Update RAW dependencies if this instruction is not a dependency-breaking
   // instruction. A dependency-breaking instruction is a zero-latency
   // instruction that doesn't consume hardware resources.
   // An example of dependency-breaking instruction on X86 is a zero-idiom XOR.
-  bool IsDependencyBreaking = IS.isDependencyBreaking();
   for (std::unique_ptr<ReadState> &RS : IS.getUses())
-    if (RS->isImplicitRead() || !IsDependencyBreaking)
+    if (!RS->isIndependentFromDef())
       updateRAWDependencies(*RS, STI);
 
   // By default, a dependency-breaking zero-latency instruction is expected to
   // be optimized at register renaming stage. That means, no physical register
   // is allocated to the instruction.
-  bool ShouldAllocateRegisters =
-      !(Desc.isZeroLatency() && IsDependencyBreaking);
+  bool ShouldAllocateRegisters = !IS.isZeroIdiom();
   SmallVector<unsigned, 4> RegisterFiles(PRF.getNumRegisterFiles());
   for (std::unique_ptr<WriteState> &WS : IS.getDefs()) {
     PRF.addRegisterWrite(WriteRef(IR.first, WS.get()), RegisterFiles,
                          ShouldAllocateRegisters);
   }
 
   // Reserve slots in the RCU, and notify the instruction that it has been
   // dispatched to the schedulers for execution.

tools/llvm-mca/lib/Stages/RetireStage.cpp

llvm::Error RetireStage::execute(InstRef &IR) {
   RCU.onInstructionExecuted(IR.getInstruction()->getRCUTokenID());
   return llvm::ErrorSuccess();
 }
 
 void RetireStage::notifyInstructionRetired(const InstRef &IR) {
   LLVM_DEBUG(llvm::dbgs() << "[E] Instruction Retired: #" << IR << '\n');
   llvm::SmallVector<unsigned, 4> FreedRegs(PRF.getNumRegisterFiles());
   const Instruction &Inst = *IR.getInstruction();
-  const InstrDesc &Desc = Inst.getDesc();
 
-  bool ShouldFreeRegs = !(Desc.isZeroLatency() && Inst.isDependencyBreaking());
+  bool ShouldFreeRegs = !Inst.isZeroIdiom();
   for (const std::unique_ptr<WriteState> &WS : Inst.getDefs())
     PRF.removeRegisterWrite(*WS.get(), FreedRegs, ShouldFreeRegs);
   notifyEvent<HWInstructionEvent>(HWInstructionRetiredEvent(IR, FreedRegs));
 }
 
 } // namespace mca

utils/TableGen/CodeGenSchedule.h

 // This file defines structures to encapsulate the machine model as described in
 // the target description.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_UTILS_TABLEGEN_CODEGENSCHEDULE_H
 #define LLVM_UTILS_TABLEGEN_CODEGENSCHEDULE_H
 
+#include "llvm/ADT/APInt.h"
 #include "llvm/ADT/DenseMap.h"
 #include "llvm/ADT/StringMap.h"
 #include "llvm/Support/ErrorHandling.h"
 #include "llvm/TableGen/Record.h"
 #include "llvm/TableGen/SetTheory.h"
 
 namespace llvm {
 
struct CodeGenProcModel {
 
   bool isUnsupported(const CodeGenInstruction &Inst) const;
 
 #ifndef NDEBUG
   void dump() const;
 #endif
 };
 
+/// Used to correlate instructions to predicates used by instruction equivalence
+/// classes contributed by different processor models.
+///
+/// Example: a XORrr where register operands are all the same, is a known
+/// zero-idiom in most modern X86 processors.
+///
+/// A PredicateInfo object can be used to identify a predicate used by
+/// processor models to identify XORrr instructions that are zero-idioms.
+/// If multiple processors use the same predicate for XORrr, then field
+/// `ProcModelMask` will have more than one bit set.
+///
+/// By construction, there can be multiple instances of PredicateInfo associated
+/// to a same instruction opcode. For example, different processors may define
+/// different constraints on the same opcode.
+///
+/// Field OperandMask can be used as an extra constraint when classifying Predicates.
+/// This may be used to describe conditions that appy only to a subset of the
+/// operands of a machine instruction, and the operands subset may not be the
+/// same for all processor models.
+struct PredicateInfo {
+  llvm::APInt ProcModelMask; // A set of processor model indices.
+  llvm::APInt OperandMask;   // An operand mask.
+  const Record *Predicate;   // MCInstrPredicate definition.
+  PredicateInfo(llvm::APInt CpuMask, llvm::APInt Operands, const Record *Pred)
+      : ProcModelMask(CpuMask), OperandMask(Operands), Predicate(Pred) {}
+
+  bool operator==(const PredicateInfo &Other) const {
+    return ProcModelMask == Other.ProcModelMask &&
+           OperandMask == Other.OperandMask && Predicate == Other.Predicate;
+  }
+};
+
+/// A collection of PredicateInfo objects.
+///
+/// There is at least one OpcodeInfo object for every opcode specified by a
+/// TIPredicate definition.
+class OpcodeInfo {
+  llvm::SmallVector<PredicateInfo, 8> Predicates;
+
+  OpcodeInfo(const OpcodeInfo &Other) = delete;
+  OpcodeInfo &operator=(const OpcodeInfo &Other) = delete;
+
+public:
+  OpcodeInfo() = default;
+  OpcodeInfo &operator=(OpcodeInfo &&Other) = default;
+  OpcodeInfo(OpcodeInfo &&Other) = default;
+
+  ArrayRef<PredicateInfo> getPredicates() const { return Predicates; }
+
+  void addPredicateForProcModel(llvm::APInt CpuMask, llvm::APInt OperandMask,
+                                const Record *Predicate);
+};
+
+/// Used to group together tablegen instruction definitions that are subject
+/// to a same set of constraints (identified by an instance of OpcodeInfo).
+class OpcodeGroup {
+  OpcodeInfo Info;
+  std::vector<const Record *> Opcodes;
+
+  OpcodeGroup(const OpcodeGroup &Other) = delete;
+  OpcodeGroup &operator=(const OpcodeGroup &Other) = delete;
+
+public:
+  OpcodeGroup(OpcodeInfo &&OpInfo) : Info(std::move(OpInfo)) {}
+  OpcodeGroup(OpcodeGroup &&Other) = default;
+
+  void addOpcode(const Record *Opcode) {
+    assert(std::find(Opcodes.begin(), Opcodes.end(), Opcode) == Opcodes.end() &&
+           "Opcode already in set!");
+    Opcodes.push_back(Opcode);
+  }
+
+  ArrayRef<const Record *> getOpcodes() const { return Opcodes; }
+  const OpcodeInfo &getOpcodeInfo() const { return Info; }
+};
+
+/// An STIPredicateFunction descriptor used by tablegen backends to
+/// auto-generate the body of a predicate function as a member of tablegen'd
+/// class XXXGenSubtargetInfo.
+class STIPredicateFunction {
+  const Record *FunctionDeclaration;
+
+  std::vector<const Record *> Definitions;
+  std::vector<OpcodeGroup> Groups;
+
+  STIPredicateFunction(const STIPredicateFunction &Other) = delete;
+  STIPredicateFunction &operator=(const STIPredicateFunction &Other) = delete;
+
+public:
+  STIPredicateFunction(const Record *Rec) : FunctionDeclaration(Rec) {}
+  STIPredicateFunction(STIPredicateFunction &&Other) = default;
+
+  bool isCompatibleWith(const STIPredicateFunction &Other) const {
+    return FunctionDeclaration == Other.FunctionDeclaration;
+  }
+
+  void addDefinition(const Record *Def) { Definitions.push_back(Def); }
+  void addOpcode(const Record *OpcodeRec, OpcodeInfo &&Info) {
+    if (Groups.empty() ||
+        Groups.back().getOpcodeInfo().getPredicates() != Info.getPredicates())
+      Groups.emplace_back(std::move(Info));
+    Groups.back().addOpcode(OpcodeRec);
+  }
+
+  StringRef getName() const {
+    return FunctionDeclaration->getValueAsString("Name");
+  }
+  const Record *getDefaultReturnPredicate() const {
+    return FunctionDeclaration->getValueAsDef("DefaultReturnValue");
+  }
+
+  const Record *getDeclaration() const { return FunctionDeclaration; }
+  ArrayRef<const Record *> getDefinitions() const { return Definitions; }
+  ArrayRef<OpcodeGroup> getGroups() const { return Groups; }
+};
+
 /// Top level container for machine model data.
 class CodeGenSchedModels {
   RecordKeeper &Records;
   const CodeGenTarget &Target;
 
   // Map dag expressions to Instruction lists.
   SetTheory Sets;
 
class CodeGenSchedModels {
   RecVec ProcResourceDefs;
   RecVec ProcResGroups;
 
   // Map each instruction to its unique SchedClass index considering the
   // combination of it's itinerary class, SchedRW list, and InstRW records.
   using InstClassMapTy = DenseMap<Record*, unsigned>;
   InstClassMapTy InstrClassMap;
 
+  std::vector<STIPredicateFunction> STIPredicates;
+
 public:
   CodeGenSchedModels(RecordKeeper& RK, const CodeGenTarget &TGT);
 
   // iterator access to the scheduling classes.
   using class_iterator = std::vector<CodeGenSchedClass>::iterator;
   using const_class_iterator = std::vector<CodeGenSchedClass>::const_iterator;
   class_iterator classes_begin() { return SchedClasses.begin(); }
   const_class_iterator classes_begin() const { return SchedClasses.begin(); }
unsigned addSchedClass(Record *ItinDef, ArrayRef<unsigned> OperWrites,
                          ArrayRef<unsigned> OperReads,
                          ArrayRef<unsigned> ProcIndices);
 
   unsigned findOrInsertRW(ArrayRef<unsigned> Seq, bool IsRead);
 
   Record *findProcResUnits(Record *ProcResKind, const CodeGenProcModel &PM,
                            ArrayRef<SMLoc> Loc) const;
 
+  ArrayRef<STIPredicateFunction> getSTIPredicates() const {
+    return STIPredicates;
+  }
 private:
   void collectProcModels();
 
   // Initialize a new processor model if it is unique.
   void addProcModel(Record *ProcDef);
 
   void collectSchedRW();
 
private:
   void collectProcItinRW();
 
   void collectProcUnsupportedFeatures();
 
   void inferSchedClasses();
 
   void checkMCInstPredicates() const;
 
+  void checkSTIPredicates() const;
+
+  void collectSTIPredicates();
+
   void checkCompleteness();
 
   void inferFromRW(ArrayRef<unsigned> OperWrites, ArrayRef<unsigned> OperReads,
                    unsigned FromClassIdx, ArrayRef<unsigned> ProcIndices);
   void inferFromItinClass(Record *ItinClassDef, unsigned FromClassIdx);
   void inferFromInstRWs(unsigned SCIdx);
 
   bool hasSuperGroup(RecVec &SubUnits, CodeGenProcModel &PM);

utils/TableGen/CodeGenSchedule.cpp

CodeGenSchedModels::CodeGenSchedModels(RecordKeeper &RK,
   collectProcResources();
 
   // Collect optional processor description.
   collectOptionalProcessorInfo();
 
   // Check MCInstPredicate definitions.
   checkMCInstPredicates();
 
+  // Check STIPredicate definitions.
+  checkSTIPredicates();
+
+  // Find STIPredicate definitions for each processor model, and construct
+  // STIPredicateFunction objects.
+  collectSTIPredicates();
+
   checkCompleteness();
 }
 
+void CodeGenSchedModels::checkSTIPredicates() const {
+  DenseMap<StringRef, const Record *> Declarations;
+
+  // There cannot be multiple declarations with the same name.
+  const RecVec Decls = Records.getAllDerivedDefinitions("STIPredicateDecl");
+  for (const Record *R : Decls) {
+    StringRef Name = R->getValueAsString("Name");
+    const auto It = Declarations.find(Name);
+    if (It == Declarations.end()) {
+      Declarations[Name] = R;
+      continue;
+    }
+
+    PrintError(R->getLoc(), "STIPredicate " + Name + " multiply declared.");
+    PrintNote(It->second->getLoc(), "Previous declaration was here.");
+    PrintFatalError(R->getLoc(), "Invalid STIPredicateDecl found.");
+  }
+}
+
+// Used by function `ProcessSTIPredicate` to construct a mask of machine
+// instruction operands.
+static APInt constructOperandMask(ArrayRef<int64_t> Indices) {
+  APInt OperandMask;
+  if (Indices.empty())
+    return OperandMask;
+
+  int64_t MaxIndex = *std::max_element(Indices.begin(), Indices.end());
+  assert(MaxIndex >= 0 && "Invalid negative indices in input!");
+  OperandMask = OperandMask.zext(MaxIndex + 1);
+  for (const int64_t Index : Indices) {
+    assert(Index >= 0 && "Invalid negative indices!");
+    OperandMask.setBit(Index);
+  }
+
+  return OperandMask;
+}
+
+static void
+processSTIPredicate(STIPredicateFunction &Fn,
+                    const DenseMap<Record *, unsigned> &ProcModelMap) {
+  DenseMap<const Record *, unsigned> Opcode2Index;
+  using OpcodeMapPair = std::pair<const Record *, OpcodeInfo>;
+  std::vector<OpcodeMapPair> OpcodeMappings;
+  std::vector<std::pair<APInt, APInt>> OpcodeMasks;
+
+  DenseMap<const Record *, unsigned> Predicate2Index;
+  unsigned NumUniquePredicates = 0;
+
+  // Number unique predicates and opcodes used by InstructionEquivalenceClass
+  // definitions. Each unique opcode will be associated with an OpcodeInfo
+  // object.
+  for (const Record *Def : Fn.getDefinitions()) {
+    RecVec Classes = Def->getValueAsListOfDefs("Classes");
+    for (const Record *EC : Classes) {
+      const Record *Pred = EC->getValueAsDef("Predicate");
+      if (Predicate2Index.find(Pred) == Predicate2Index.end())
+        Predicate2Index[Pred] = NumUniquePredicates++;
+
+      RecVec Opcodes = EC->getValueAsListOfDefs("Opcodes");
+      for (const Record *Opcode : Opcodes) {
+        if (Opcode2Index.find(Opcode) == Opcode2Index.end()) {
+          Opcode2Index[Opcode] = OpcodeMappings.size();
+          OpcodeMappings.emplace_back(Opcode, OpcodeInfo());
+        }
+      }
+    }
+  }
+
+  // Initialize vector `OpcodeMasks` with default values.  We want to keep track
+  // of which processors "use" which opcodes.  We also want to be able to
+  // identify predicates that are used by different processors for a same
+  // opcode.
+  // This information is used later on by this algorithm to sort OpcodeMapping
+  // elements based on their processor and predicate sets.
+  OpcodeMasks.resize(OpcodeMappings.size());
+  APInt DefaultProcMask(ProcModelMap.size(), 0);
+  APInt DefaultPredMask(NumUniquePredicates, 0);
+  for (std::pair<APInt, APInt> &MaskPair : OpcodeMasks)
+    MaskPair = std::make_pair(DefaultProcMask, DefaultPredMask);
+
+  // Construct a OpcodeInfo object for every unique opcode declared by an
+  // InstructionEquivalenceClass definition.
+  for (const Record *Def : Fn.getDefinitions()) {
+    RecVec Classes = Def->getValueAsListOfDefs("Classes");
+    const Record *SchedModel = Def->getValueAsDef("SchedModel");
+    unsigned ProcIndex = ProcModelMap.find(SchedModel)->second;
+    APInt ProcMask(ProcModelMap.size(), 0);
+    ProcMask.setBit(ProcIndex);
+
+    for (const Record *EC : Classes) {
+      RecVec Opcodes = EC->getValueAsListOfDefs("Opcodes");
+
+      std::vector<int64_t> OpIndices =
+          EC->getValueAsListOfInts("OperandIndices");
+      APInt OperandMask = constructOperandMask(OpIndices);
+
+      const Record *Pred = EC->getValueAsDef("Predicate");
+      APInt PredMask(NumUniquePredicates, 0);
+      PredMask.setBit(Predicate2Index[Pred]);
+
+      for (const Record *Opcode : Opcodes) {
+        unsigned OpcodeIdx = Opcode2Index[Opcode];
+        OpcodeMasks[OpcodeIdx].first |= ProcMask;
+        OpcodeMasks[OpcodeIdx].second |= PredMask;
+        OpcodeInfo &OI = OpcodeMappings[OpcodeIdx].second;
+        OI.addPredicateForProcModel(ProcMask, OperandMask, Pred);
+      }
+    }
+  }
+
+  // Sort OpcodeMappings elements based on their CPU and predicate masks.
+  // As a last resort, order elements by opcode identifier.
+  llvm::sort(OpcodeMappings.begin(), OpcodeMappings.end(),
+             [&](const OpcodeMapPair &Lhs, const OpcodeMapPair &Rhs) {
+               unsigned LhsIdx = Opcode2Index[Lhs.first];
+               unsigned RhsIdx = Opcode2Index[Rhs.first];
+               std::pair<APInt, APInt> &LhsMasks = OpcodeMasks[LhsIdx];
+               std::pair<APInt, APInt> &RhsMasks = OpcodeMasks[RhsIdx];
+
+               if (LhsMasks.first != RhsMasks.first) {
+                 if (LhsMasks.first.countPopulation() <
+                     RhsMasks.first.countPopulation())
+                   return true;
+                 return LhsMasks.first.countLeadingZeros() >
+                        RhsMasks.first.countLeadingZeros();
+               }
+
+               if (LhsMasks.second != RhsMasks.second) {
+                 if (LhsMasks.second.countPopulation() <
+                     RhsMasks.second.countPopulation())
+                   return true;
+                 return LhsMasks.second.countLeadingZeros() >
+                        RhsMasks.second.countLeadingZeros();
+               }
+
+               return LhsIdx < RhsIdx;
+             });
+
+  // Now construct opcode groups. Groups are used by the SubtargetEmitter when
+  // expanding the body of a STIPredicate function. In particular, each opcode
+  // group is expanded into a sequence of labels in a switch statement.
+  // It identifies opcodes for which different processors define same predicates
+  // and same opcode masks.
+  for (OpcodeMapPair &Info : OpcodeMappings)
+    Fn.addOpcode(Info.first, std::move(Info.second));
+}
+
+void CodeGenSchedModels::collectSTIPredicates() {
+  // Map STIPredicateDecl records to elements of vector
+  // CodeGenSchedModels::STIPredicates.
+  DenseMap<const Record *, unsigned> Decl2Index;
+
+  RecVec RV = Records.getAllDerivedDefinitions("STIPredicate");
+  for (const Record *R : RV) {
+    const Record *Decl = R->getValueAsDef("Declaration");
+
+    const auto It = Decl2Index.find(Decl);
+    if (It == Decl2Index.end()) {
+      Decl2Index[Decl] = STIPredicates.size();
+      STIPredicateFunction Predicate(Decl);
+      Predicate.addDefinition(R);
+      STIPredicates.emplace_back(std::move(Predicate));
+      continue;
+    }
+
+    STIPredicateFunction &PreviousDef = STIPredicates[It->second];
+    PreviousDef.addDefinition(R);
+  }
+
+  for (STIPredicateFunction &Fn : STIPredicates)
+    processSTIPredicate(Fn, ProcModelMap);
+}
+
+void OpcodeInfo::addPredicateForProcModel(llvm::APInt CpuMask,
+                                          llvm::APInt OperandMask,
+                                          const Record *Predicate) {
+  auto It = llvm::find_if(
+      Predicates, [&OperandMask, &Predicate](const PredicateInfo &P) {
+        return P.Predicate == Predicate && P.OperandMask == OperandMask;
+      });
+  if (It == Predicates.end()) {
+    Predicates.emplace_back(CpuMask, OperandMask, Predicate);
+    return;
+  }
+  It->ProcModelMask |= CpuMask;
+}
+
 void CodeGenSchedModels::checkMCInstPredicates() const {
   RecVec MCPredicates = Records.getAllDerivedDefinitions("TIIPredicate");
   if (MCPredicates.empty())
     return;
 
   // A target cannot have multiple TIIPredicate definitions with a same name.
   llvm::StringMap<const Record *> TIIPredicates(MCPredicates.size());
   for (const Record *TIIPred : MCPredicates) {

utils/TableGen/PredicateExpander.h

 public:
   PredicateExpander(StringRef Target)
       : EmitCallsByRef(true), NegatePredicate(false), ExpandForMC(false),
         IndentLevel(1U), TargetName(Target) {}
   bool isByRef() const { return EmitCallsByRef; }
   bool shouldNegate() const { return NegatePredicate; }
   bool shouldExpandForMC() const { return ExpandForMC; }
   unsigned getIndentLevel() const { return IndentLevel; }
+  StringRef getTargetName() const { return TargetName; }
 
   void setByRef(bool Value) { EmitCallsByRef = Value; }
   void flipNegatePredicate() { NegatePredicate = !NegatePredicate; }
   void setNegatePredicate(bool Value) { NegatePredicate = Value; }
   void setExpandForMC(bool Value) { ExpandForMC = Value; }
+  void setIndentLevel(unsigned Level) { IndentLevel = Level; }
   void increaseIndentLevel() { ++IndentLevel; }
   void decreaseIndentLevel() { --IndentLevel; }
-  void setIndentLevel(unsigned Level) { IndentLevel = Level; }
 
   using RecVec = std::vector<Record *>;
   void expandTrue(raw_ostream &OS);
   void expandFalse(raw_ostream &OS);
   void expandCheckImmOperand(raw_ostream &OS, int OpIndex, int ImmVal);
   void expandCheckImmOperand(raw_ostream &OS, int OpIndex, StringRef ImmVal);
   void expandCheckRegOperand(raw_ostream &OS, int OpIndex, const Record *Reg);
   void expandCheckSameRegOperand(raw_ostream &OS, int First, int Second);
public:
   void expandPredicate(raw_ostream &OS, const Record *Rec);
   void expandReturnStatement(raw_ostream &OS, const Record *Rec);
   void expandOpcodeSwitchCase(raw_ostream &OS, const Record *Rec);
   void expandOpcodeSwitchStatement(raw_ostream &OS, const RecVec &Cases,
                                    const Record *Default);
   void expandStatement(raw_ostream &OS, const Record *Rec);
 };
 
+// Forward declarations.
+class STIPredicateFunction;
+class OpcodeGroup;
+
+class STIPredicateExpander : public PredicateExpander {
+  StringRef ClassPrefix;
+  bool ExpandDefinition;
+
+  STIPredicateExpander(const PredicateExpander &) = delete;
+  STIPredicateExpander &operator=(const PredicateExpander &) = delete;
+
+  void expandHeader(raw_ostream &OS, const STIPredicateFunction &Fn);
+  void expandPrologue(raw_ostream &OS, const STIPredicateFunction &Fn);
+  void expandOpcodeGroup(raw_ostream &OS, const OpcodeGroup &Group,
+                         bool ShouldUpdateOpcodeMask);
+  void expandBody(raw_ostream &OS, const STIPredicateFunction &Fn);
+  void expandEpilogue(raw_ostream &OS, const STIPredicateFunction &Fn);
+
+public:
+  STIPredicateExpander(StringRef Target)
+      : PredicateExpander(Target), ClassPrefix(), ExpandDefinition(false) {}
+
+  bool shouldExpandDefinition() const { return ExpandDefinition; }
+  StringRef getClassPrefix() const { return ClassPrefix; }
+  void setClassPrefix(StringRef S) { ClassPrefix = S; }
+  void setExpandDefinition(bool Value) { ExpandDefinition = Value; }
+
+  void expandSTIPredicate(raw_ostream &OS, const STIPredicateFunction &Fn);
+};
+
 } // namespace llvm
 
 #endif

utils/TableGen/PredicateExpander.cpp

 //===--------------------- PredicateExpander.cpp --------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 /// \file
 /// Functionalities used by the Tablegen backends to expand machine predicates.
 //
 //===----------------------------------------------------------------------===//
 
 #include "PredicateExpander.h"
+#include "CodeGenSchedule.h" // Definition of STIPredicateFunction.
 
 namespace llvm {
 
 void PredicateExpander::expandTrue(raw_ostream &OS) { OS << "true"; }
 void PredicateExpander::expandFalse(raw_ostream &OS) { OS << "false"; }
 
 void PredicateExpander::expandCheckImmOperand(raw_ostream &OS, int OpIndex,
                                               int ImmVal) {
if (Rec->isSubClassOf("CheckNonPortable"))
     return expandCheckNonPortable(OS, Rec->getValueAsString("CodeBlock"));
 
   if (Rec->isSubClassOf("TIIPredicate"))
     return expandTIIFunctionCall(OS, Rec->getValueAsString("FunctionName"));
 
   llvm_unreachable("No known rules to expand this MCInstPredicate");
 }
 
+void STIPredicateExpander::expandHeader(raw_ostream &OS,
+                                        const STIPredicateFunction &Fn) {
+  const Record *Rec = Fn.getDeclaration();
+  StringRef FunctionName = Rec->getValueAsString("Name");
+
+  OS.indent(getIndentLevel() * 2);
+  OS << "bool ";
+  if (shouldExpandDefinition())
+    OS << getClassPrefix() << "::";
+  OS << FunctionName << "(";
+  if (shouldExpandForMC())
+    OS << "const MCInst " << (isByRef() ? "&" : "*") << "MI";
+  else
+    OS << "const MachineInstr " << (isByRef() ? "&" : "*") << "MI";
+  if (Rec->getValueAsBit("UpdatesOpcodeMask"))
+    OS << ", APInt &Mask";
+  OS << (shouldExpandForMC() ? ", unsigned ProcessorID) const " : ") const ");
+  if (shouldExpandDefinition()) {
+    OS << "{\n";
+    return;
+  }
+
+  if (Rec->getValueAsBit("OverridesBaseClassMember"))
+    OS << "override";
+  OS << ";\n";
+}
+
+void STIPredicateExpander::expandPrologue(raw_ostream &OS,
+                                          const STIPredicateFunction &Fn) {
+  RecVec Delegates = Fn.getDeclaration()->getValueAsListOfDefs("Delegates");
+  bool UpdatesOpcodeMask =
+      Fn.getDeclaration()->getValueAsBit("UpdatesOpcodeMask");
+
+  increaseIndentLevel();
+  unsigned IndentLevel = getIndentLevel();
+  for (const Record *Delegate : Delegates) {
+    OS.indent(IndentLevel * 2);
+    OS << "if (" << Delegate->getValueAsString("Name") << "(MI";
+    if (UpdatesOpcodeMask)
+      OS << ", Mask";
+    if (shouldExpandForMC())
+      OS << ", ProcessorID";
+    OS << "))\n";
+    OS.indent((1 + IndentLevel) * 2);
+    OS << "return true;\n\n";
+  }
+
+  if (shouldExpandForMC())
+    return;
+
+  OS.indent(IndentLevel * 2);
+  OS << "unsigned ProcessorID = getSchedModel().getProcessorID();\n";
+}
+
+void STIPredicateExpander::expandOpcodeGroup(raw_ostream &OS, const OpcodeGroup &Group,
+                                             bool ShouldUpdateOpcodeMask) {
+  const OpcodeInfo &OI = Group.getOpcodeInfo();
+  for (const PredicateInfo &PI : OI.getPredicates()) {
+    const APInt &ProcModelMask = PI.ProcModelMask;
+    bool FirstProcID = true;
+    for (unsigned I = 0, E = ProcModelMask.getActiveBits(); I < E; ++I) {
+      if (!ProcModelMask[I])
+        continue;
+
+      if (FirstProcID) {
+        OS.indent(getIndentLevel() * 2);
+        OS << "if (ProcessorID == " << I;
+      } else {
+        OS << " || ProcessorID == " << I;
+      }
+      FirstProcID = false;
+    }
+
+    OS << ") {\n";
+
+    increaseIndentLevel();
+    OS.indent(getIndentLevel() * 2);
+    if (ShouldUpdateOpcodeMask) {
+      if (PI.OperandMask.isNullValue())
+        OS << "Mask.clearAllBits();\n";
+      else
+        OS << "Mask = " << PI.OperandMask << ";\n";
+      OS.indent(getIndentLevel() * 2);
+    }
+    OS << "return ";
+    expandPredicate(OS, PI.Predicate);
+    OS << ";\n";
+    decreaseIndentLevel();
+    OS.indent(getIndentLevel() * 2);
+    OS << "}\n";
+  }
+}
+
+void STIPredicateExpander::expandBody(raw_ostream &OS,
+                                      const STIPredicateFunction &Fn) {
+  bool UpdatesOpcodeMask =
+      Fn.getDeclaration()->getValueAsBit("UpdatesOpcodeMask");
+
+  unsigned IndentLevel = getIndentLevel();
+  OS.indent(IndentLevel * 2);
+  OS << "switch(MI" << (isByRef() ? "." : "->") << "getOpcode()) {\n";
+  OS.indent(IndentLevel * 2);
+  OS << "default:\n";
+  OS.indent(IndentLevel * 2);
+  OS << "  break;";
+
+  for (const OpcodeGroup &Group : Fn.getGroups()) {
+    for (const Record *Opcode : Group.getOpcodes()) {
+      OS << '\n';
+      OS.indent(IndentLevel * 2);
+      OS << "case " << getTargetName() << "::" << Opcode->getName() << ":";
+    }
+
+    OS << '\n';
+    increaseIndentLevel();
+    expandOpcodeGroup(OS, Group, UpdatesOpcodeMask);
+
+    OS.indent(getIndentLevel() * 2);
+    OS << "break;\n";
+    decreaseIndentLevel();
+  }
+
+  OS.indent(IndentLevel * 2);
+  OS << "}\n";
+}
+
+void STIPredicateExpander::expandEpilogue(raw_ostream &OS,
+                                          const STIPredicateFunction &Fn) {
+  OS << '\n';
+  OS.indent(getIndentLevel() * 2);
+  OS << "return ";
+  expandPredicate(OS, Fn.getDefaultReturnPredicate());
+  OS << ";\n";
+
+  decreaseIndentLevel();
+  OS.indent(getIndentLevel() * 2);
+  StringRef FunctionName = Fn.getDeclaration()->getValueAsString("Name");
+  OS << "} // " << ClassPrefix << "::" << FunctionName << "\n\n";
+}
+
+void STIPredicateExpander::expandSTIPredicate(raw_ostream &OS,
+                                              const STIPredicateFunction &Fn) {
+  const Record *Rec = Fn.getDeclaration();
+  if (shouldExpandForMC() && Rec->getValueAsBit("ExpandForMachineInstrOnly"))
+    return;
+
+  expandHeader(OS, Fn);
+  if (shouldExpandDefinition()) {
+    expandPrologue(OS, Fn);
+    expandBody(OS, Fn);
+    expandEpilogue(OS, Fn);
+  }
+}
+
 } // namespace llvm

utils/TableGen/SubtargetEmitter.cpp

void GenSchedClassTables(const CodeGenProcModel &ProcModel,
                            SchedClassTables &SchedTables);
   void EmitSchedClassTables(SchedClassTables &SchedTables, raw_ostream &OS);
   void EmitProcessorModels(raw_ostream &OS);
   void EmitProcessorLookup(raw_ostream &OS);
   void EmitSchedModelHelpers(const std::string &ClassName, raw_ostream &OS);
   void emitSchedModelHelpersImpl(raw_ostream &OS,
                                  bool OnlyExpandMCInstPredicates = false);
   void emitGenMCSubtargetInfo(raw_ostream &OS);
+  void EmitMCInstrAnalysisPredicateFunctions(raw_ostream &OS);
 
   void EmitSchedModel(raw_ostream &OS);
   void EmitHwModeCheck(const std::string &ClassName, raw_ostream &OS);
   void ParseFeaturesFunction(raw_ostream &OS, unsigned NumFeatures,
                              unsigned NumProcs);
 
 public:
   SubtargetEmitter(RecordKeeper &R, CodeGenTarget &TGT)
void SubtargetEmitter::EmitSchedModelHelpers(const std::string &ClassName,
   
   OS << "} // " << ClassName << "::resolveSchedClass\n\n";
 
   OS << "unsigned " << ClassName
      << "\n::resolveVariantSchedClass(unsigned SchedClass, const MCInst *MI,"
      << " unsigned CPUID) const {\n"
      << "  return " << Target << "_MC"
      << "::resolveVariantSchedClassImpl(SchedClass, MI, CPUID);\n"
-     << "} // " << ClassName << "::resolveVariantSchedClass\n";
+     << "} // " << ClassName << "::resolveVariantSchedClass\n\n";
+
+  STIPredicateExpander PE(Target);
+  PE.setClassPrefix(ClassName);
+  PE.setExpandDefinition(true);
+  PE.setByRef(false);
+  PE.setIndentLevel(0);
+
+  for (const STIPredicateFunction &Fn : SchedModels.getSTIPredicates())
+    PE.expandSTIPredicate(OS, Fn);
 }
 
 void SubtargetEmitter::EmitHwModeCheck(const std::string &ClassName,
                                        raw_ostream &OS) {
   const CodeGenHwModes &CGH = TGT.getHwModes();
   assert(CGH.getNumModeIds() > 0);
   if (CGH.getNumModeIds() == 1)
     return;
OS << "  " << Target << "GenMCSubtargetInfo(const Triple &TT, \n"
      << "  unsigned resolveVariantSchedClass(unsigned SchedClass,\n"
      << "      const MCInst *MI, unsigned CPUID) const override {\n"
      << "    return " << Target << "_MC"
      << "::resolveVariantSchedClassImpl(SchedClass, MI, CPUID); \n";
   OS << "  }\n";
   OS << "};\n";
 }
 
+void SubtargetEmitter::EmitMCInstrAnalysisPredicateFunctions(raw_ostream &OS) {
+  OS << "\n#ifdef GET_STIPREDICATE_DECLS_FOR_MC_ANALYSIS\n";
+  OS << "#undef GET_STIPREDICATE_DECLS_FOR_MC_ANALYSIS\n\n";
+
+  STIPredicateExpander PE(Target);
+  PE.setExpandForMC(true);
+  PE.setByRef(true);
+  for (const STIPredicateFunction &Fn : SchedModels.getSTIPredicates())
+    PE.expandSTIPredicate(OS, Fn);
+
+  OS << "#endif // GET_STIPREDICATE_DECLS_FOR_MC_ANALYSIS\n\n";
+
+  OS << "\n#ifdef GET_STIPREDICATE_DEFS_FOR_MC_ANALYSIS\n";
+  OS << "#undef GET_STIPREDICATE_DEFS_FOR_MC_ANALYSIS\n\n";
+
+  std::string ClassPrefix = Target + "MCInstrAnalysis";
+  PE.setExpandDefinition(true);
+  PE.setClassPrefix(ClassPrefix);
+  PE.setIndentLevel(0);
+  for (const STIPredicateFunction &Fn : SchedModels.getSTIPredicates())
+    PE.expandSTIPredicate(OS, Fn);
+
+  OS << "#endif // GET_STIPREDICATE_DEFS_FOR_MC_ANALYSIS\n\n";
+}
+
 //
 // SubtargetEmitter::run - Main subtarget enumeration emitter.
 //
 void SubtargetEmitter::run(raw_ostream &OS) {
   emitSourceFileHeader("Subtarget Enumeration Source Fragment", OS);
 
   OS << "\n#ifdef GET_SUBTARGETINFO_ENUM\n";
   OS << "#undef GET_SUBTARGETINFO_ENUM\n\n";
OS << "struct " << ClassName << " : public TargetSubtargetInfo {\n"
      << " const MachineInstr *DefMI,"
      << " const TargetSchedModel *SchedModel) const override;\n"
      << "  unsigned resolveVariantSchedClass(unsigned SchedClass,"
      << " const MCInst *MI, unsigned CPUID) const override;\n"
      << "  DFAPacketizer *createDFAPacketizer(const InstrItineraryData *IID)"
      << " const;\n";
   if (TGT.getHwModes().getNumModeIds() > 1)
     OS << "  unsigned getHwMode() const override;\n";
+
+  STIPredicateExpander PE(Target);
+  PE.setByRef(false);
+  for (const STIPredicateFunction &Fn : SchedModels.getSTIPredicates())
+    PE.expandSTIPredicate(OS, Fn);
+
   OS << "};\n"
      << "} // end namespace llvm\n\n";
 
   OS << "#endif // GET_SUBTARGETINFO_HEADER\n\n";
 
   OS << "\n#ifdef GET_SUBTARGETINFO_CTOR\n";
   OS << "#undef GET_SUBTARGETINFO_CTOR\n\n";
 
#endif
   OS << ") {}\n\n";
 
   EmitSchedModelHelpers(ClassName, OS);
   EmitHwModeCheck(ClassName, OS);
 
   OS << "} // end namespace llvm\n\n";
 
   OS << "#endif // GET_SUBTARGETINFO_CTOR\n\n";
+
+  EmitMCInstrAnalysisPredicateFunctions(OS);
 }
 
 namespace llvm {
 
 void EmitSubtarget(RecordKeeper &RK, raw_ostream &OS) {
   CodeGenTarget CGTarget(RK);
   SubtargetEmitter(RK, CGTarget).run(OS);
 }
 
 } // end namespace llvm