[AArch64] Refactor the Exynos scheduling predicates

[evandro]

Summary

Refactor the scheduling predicates based on MCInstPredicate. In this case, for the Exynos processors.

[andreadb]

Hi Evandro,

Thanks for the patch.

Could you please update the diff?
I tried to apply your patch, but there were failures.
AArch64InstrInfo::isExynosShiftFast() is referenced by your patch, but it doesn't appear to exist upstream. In another case, the conflict was caused by a diff which appears to have been committed already.

Thanks,
Andrea

llvm/lib/Target/AArch64/AArch64.td

154–157

This change was already committed at r348398.

llvm/lib/Target/AArch64/AArch64InstrInfo.cpp

708

There is no isExynosShiftFast().

llvm/lib/Target/AArch64/AArch64InstrInfo.h

253–255

Not sure where isExynosShiftFast declaration comes from. It is not available at HEAD revision. So, this patch fails to apply.

[evandro]

Rebase the patch.

[andreadb]

In D55345#1321500, @evandro wrote:

Rebase the patch.

Unfortunally, I am still seeing several tablegen errors.

Tablegen complains about missing definitions. It looks like most (if not all) of those missing definitions will be added by D55375.
I reported only a few missing symbols. Is this patch meant to be dependent on D55375? If not, then this patch should be fixed. If it is going to be complicated, then I don't mind if you merge the two patches into a single diff. As long as I can build it, and check that the generated code is still semantically equivalent to what there was before, then I am happy.

Although this patch is supposed to be an NFC, I'd like to see at least one llvm-mca test. Ideally, a test that shows how we can now analyze instructions that llvm-mca was previously unable to analyze.

llvm/lib/Target/AArch64/AArch64InstrInfo.cpp

940–941

For the record: this method never existed in the original code. Not sure how it ended up in this diff.

llvm/lib/Target/AArch64/AArch64SchedPredExynos.td

25

IsArithExt32Op is not defined.

27

CheckExtBy0 is not defined.

35

IsArithExt64Op is not defined.

48

These two definitions are missing. I saw you added those in D55375.

[evandro]

Sorry, I was juggling too many patches out of order.

I'd appreciate your reviewing the new predicates and their application while I work on test cases.

Thank you.

[andreadb]

In D55345#1321718, @evandro wrote:

Sorry, I was juggling too many patches out of order.

I'd appreciate your reviewing the new predicates and their application while I work on test cases.

Thank you.

No problem! I am happy to review your patches :-).

[andreadb]

I went through the various changes, and I didn’t find anything that looks obviously wrong. The new predicates look fine to me.

For the tests, I recommend you to use flag -instruction-table. As an example, see the RUN line in test/tools/llvm-mca/X86/BtVer2/resources-avx1.s.

Flag -instruction-tables enables printing the theoretical resource distribution. In my experience, that’s the best way to check the resource consumption of every instruction.
That flag, plus the implicit instruction-info view, is the best combination for tests where you want to verify latency/throughput and resource usage.

It is particularly useful in the presence of instructions that require resolving write variants; you can basically check all the interesting variants.

As soon as tests are added, and you are happy with the latency/throughput/resource usage of instructions, I will be happy to accept this patch.

-Andrea

[andreadb]

LGTM.

Thanks Evandro.

llvm/include/llvm/Target/TargetInstrPredicate.td

 // MCInstPredicate definitions are typically used by scheduling models to
 // construct MCSchedPredicate definitions (see the definition of class
 // MCSchedPredicate in llvm/Target/TargetSchedule.td).
 // In particular, an MCSchedPredicate can be used instead of a SchedPredicate
 // when defining the set of SchedReadVariant and SchedWriteVariant of a
 // processor scheduling model.
 //
 // The `MCInstPredicateExample` definition above is equivalent (and therefore
-// could replace) the following definition from the ExynosM3 model (see
+// could replace) the following definition from a previous ExynosM3 model (see
 // AArch64SchedExynosM3.td):
 //
 // def M3BranchLinkFastPred  : SchedPredicate<[{
 //    MI->getOpcode() == AArch64::BLR &&
 //    MI->getOperand(0).isReg() &&
 //    MI->getOperand(0).getReg() != AArch64::LR}]>;
 //
 // The main advantage of using MCInstPredicate instead of SchedPredicate is

llvm/lib/Target/AArch64/AArch64.td

 def FeaturePredictableSelectIsExpensive : SubtargetFeature<
     "predictable-select-expensive", "PredictableSelectIsExpensive", "true",
     "Prefer likely predicted branches over selects">;
 
 def FeatureCustomCheapAsMoveHandling : SubtargetFeature<"custom-cheap-as-move",
     "CustomAsCheapAsMove", "true",
     "Use custom handling of cheap instructions">;
 
 def FeatureExynosCheapAsMoveHandling : SubtargetFeature<"exynos-cheap-as-move",
     "ExynosAsCheapAsMove", "true",
     "Use Exynos specific handling of cheap instructions",
     [FeatureCustomCheapAsMoveHandling]>;
 
 def FeaturePostRAScheduler : SubtargetFeature<"use-postra-scheduler",
     "UsePostRAScheduler", "true", "Schedule again after register allocation">;
 
 def FeatureSlowMisaligned128Store : SubtargetFeature<"slow-misaligned-128store",
     "Misaligned128StoreIsSlow", "true", "Misaligned 128 bit stores are slow">;
 
 def FeatureSlowPaired128 : SubtargetFeature<"slow-paired-128",

 
 //===----------------------------------------------------------------------===//
 // Instruction Descriptions
 //===----------------------------------------------------------------------===//
 
 include "AArch64Schedule.td"
 include "AArch64InstrInfo.td"
 include "AArch64SchedPredicates.td"
+include "AArch64SchedPredExynos.td"
 
 def AArch64InstrInfo : InstrInfo;
 
 //===----------------------------------------------------------------------===//
 // Named operands for MRS/MSR/TLBI/...
 //===----------------------------------------------------------------------===//
 
 include "AArch64SystemOperands.td"

llvm/lib/Target/AArch64/AArch64InstrInfo.h

bool isMBBSafeToOutlineFrom(MachineBasicBlock &MBB,
                               unsigned &Flags) const override;
   void buildOutlinedFrame(MachineBasicBlock &MBB, MachineFunction &MF,
                           const outliner::OutlinedFunction &OF) const override;
   MachineBasicBlock::iterator
   insertOutlinedCall(Module &M, MachineBasicBlock &MBB,
                      MachineBasicBlock::iterator &It, MachineFunction &MF,
                      const outliner::Candidate &C) const override;
   bool shouldOutlineFromFunctionByDefault(MachineFunction &MF) const override;
-  /// Returns true if the instruction sets a constant value that can be
-  /// executed more efficiently.
-  static bool isExynosResetFast(const MachineInstr &MI);
-  /// Returns true if the load or store has an extension that can be executed
-  /// more efficiently.
-  static bool isExynosLdStExtFast(const MachineInstr &MI);
-  /// Returns true if the instruction has a constant shift left or extension
-  /// that can be executed more efficiently.
-  static bool isExynosShiftExtFast(const MachineInstr &MI, unsigned Extra = 0);
-  /// Returns true if the instruction is a constant shift left that can be
-  /// executed more quickly.
-  static bool isExynosShiftFast(const MachineInstr &MI, unsigned Extra = 0);
   /// Returns true if the instruction has a shift by immediate that can be
   /// executed in one cycle less.
   static bool isFalkorShiftExtFast(const MachineInstr &MI);
   /// Returns true if the instruction is a long operation using the upper half
   /// of vectors.
   static bool isLongVectorUpperHalf(const MachineInstr &MI);
   /// Return true if the instructions is a SEH instruciton used for unwinding
   /// on Windows.

llvm/lib/Target/AArch64/AArch64InstrInfo.cpp

if (Opcode == TargetOpcode::COPY &&
         (MI.getOperand(1).getReg() == AArch64::WZR ||
          MI.getOperand(1).getReg() == AArch64::XZR))
       return true;
   }
 
   // Secondly, check cases specific to sub-targets.
 
   if (Subtarget.hasExynosCheapAsMoveHandling()) {
-    if (isExynosResetFast(MI) ||
-        isExynosShiftFast(MI) || isExynosShiftExtFast(MI))
+    if (isExynosCheapAsMove(MI))
       return true;
 
     return MI.isAsCheapAsAMove();
   }
 
   // Finally, check generic cases.
 
   switch (Opcode) {
case AArch64::MOVi32imm:
     return canBeExpandedToORR(MI, 32);
   case AArch64::MOVi64imm:
     return canBeExpandedToORR(MI, 64);
   }
 
   llvm_unreachable("Unknown opcode to check as cheap as a move!");
 }
 
-bool AArch64InstrInfo::isExynosResetFast(const MachineInstr &MI) {
-  unsigned Reg, Imm, Shift;
-
-  switch (MI.getOpcode()) {
-  default:
-    return false;
-
-  // MOV Rd, SP
-  case AArch64::ADDWri:
-  case AArch64::ADDXri:
-    if (!MI.getOperand(1).isReg() || !MI.getOperand(2).isImm())
-      return false;
-
-    Reg = MI.getOperand(1).getReg();
-    Imm = MI.getOperand(2).getImm();
-    return ((Reg == AArch64::WSP || Reg == AArch64::SP) && Imm == 0);
-
-  // Literal
-  case AArch64::ADR:
-  case AArch64::ADRP:
-    return true;
-
-  // MOVI Vd, #0
-  case AArch64::MOVID:
-  case AArch64::MOVIv8b_ns:
-  case AArch64::MOVIv2d_ns:
-  case AArch64::MOVIv16b_ns:
-    Imm = MI.getOperand(1).getImm();
-    return (Imm == 0);
-
-  // MOVI Vd, #0
-  case AArch64::MOVIv2i32:
-  case AArch64::MOVIv4i16:
-  case AArch64::MOVIv4i32:
-  case AArch64::MOVIv8i16:
-    Imm = MI.getOperand(1).getImm();
-    Shift = MI.getOperand(2).getImm();
-    return (Imm == 0 && Shift == 0);
-
-  // MOV Rd, Imm
-  case AArch64::MOVNWi:
-  case AArch64::MOVNXi:
-
-  // MOV Rd, Imm
-  case AArch64::MOVZWi:
-  case AArch64::MOVZXi:
-    return true;
-
-  // MOV Rd, Imm
-  case AArch64::ORRWri:
-  case AArch64::ORRXri:
-    if (!MI.getOperand(1).isReg())
-      return false;
-
-    Reg = MI.getOperand(1).getReg();
-    Imm = MI.getOperand(2).getImm();
-    return ((Reg == AArch64::WZR || Reg == AArch64::XZR) && Imm == 0);
-
-  // MOV Rd, Rm
-  case AArch64::ORRWrs:
-  case AArch64::ORRXrs:
-    if (!MI.getOperand(1).isReg())
-      return false;
-
-    Reg = MI.getOperand(1).getReg();
-    Imm = MI.getOperand(3).getImm();
-    Shift = AArch64_AM::getShiftValue(Imm);
-    return ((Reg == AArch64::WZR || Reg == AArch64::XZR) && Shift == 0);
-  }
-}
-
-bool AArch64InstrInfo::isExynosLdStExtFast(const MachineInstr &MI) {
-  unsigned Imm;
-  AArch64_AM::ShiftExtendType Ext;
-
-  switch (MI.getOpcode()) {
-  default:
-    return false;
-
-  // WriteLD
-  case AArch64::PRFMroW:
-  case AArch64::PRFMroX:
-
-  // WriteLDIdx
-  case AArch64::LDRBBroW:
-  case AArch64::LDRBBroX:
-  case AArch64::LDRHHroW:
-  case AArch64::LDRHHroX:
-  case AArch64::LDRSBWroW:
-  case AArch64::LDRSBWroX:
-  case AArch64::LDRSBXroW:
-  case AArch64::LDRSBXroX:
-  case AArch64::LDRSHWroW:
-  case AArch64::LDRSHWroX:
-  case AArch64::LDRSHXroW:
-  case AArch64::LDRSHXroX:
-  case AArch64::LDRSWroW:
-  case AArch64::LDRSWroX:
-  case AArch64::LDRWroW:
-  case AArch64::LDRWroX:
-  case AArch64::LDRXroW:
-  case AArch64::LDRXroX:
-
-  case AArch64::LDRBroW:
-  case AArch64::LDRBroX:
-  case AArch64::LDRDroW:
-  case AArch64::LDRDroX:
-  case AArch64::LDRHroW:
-  case AArch64::LDRHroX:
-  case AArch64::LDRSroW:
-  case AArch64::LDRSroX:
-
-  // WriteSTIdx
-  case AArch64::STRBBroW:
-  case AArch64::STRBBroX:
-  case AArch64::STRHHroW:
-  case AArch64::STRHHroX:
-  case AArch64::STRWroW:
-  case AArch64::STRWroX:
-  case AArch64::STRXroW:
-  case AArch64::STRXroX:
-
-  case AArch64::STRBroW:
-  case AArch64::STRBroX:
-  case AArch64::STRDroW:
-  case AArch64::STRDroX:
-  case AArch64::STRHroW:
-  case AArch64::STRHroX:
-  case AArch64::STRSroW:
-  case AArch64::STRSroX:
-    Imm = MI.getOperand(3).getImm();
-    Ext = AArch64_AM::getMemExtendType(Imm);
-    return (Ext == AArch64_AM::SXTX || Ext == AArch64_AM::UXTX);
-  }
-}
-
-bool AArch64InstrInfo::isExynosShiftExtFast(const MachineInstr &MI,
-                                            unsigned Extra) {
-  unsigned Imm, Shift;
-  AArch64_AM::ShiftExtendType Ext;
-
-  switch (MI.getOpcode()) {
-  default:
-    return false;
-
-  // WriteI
-  case AArch64::ADDSWri:
-  case AArch64::ADDSXri:
-  case AArch64::ADDWri:
-  case AArch64::ADDXri:
-  case AArch64::SUBSWri:
-  case AArch64::SUBSXri:
-  case AArch64::SUBWri:
-  case AArch64::SUBXri:
-    return true;
-
-  // WriteIEReg
-  case AArch64::ADDSWrx:
-  case AArch64::ADDSXrx:
-  case AArch64::ADDSXrx64:
-  case AArch64::ADDWrx:
-  case AArch64::ADDXrx:
-  case AArch64::ADDXrx64:
-  case AArch64::SUBSWrx:
-  case AArch64::SUBSXrx:
-  case AArch64::SUBSXrx64:
-  case AArch64::SUBWrx:
-  case AArch64::SUBXrx:
-  case AArch64::SUBXrx64:
-    Imm = MI.getOperand(3).getImm();
-    Shift = AArch64_AM::getArithShiftValue(Imm);
-    Ext = AArch64_AM::getArithExtendType(Imm);
-    return (Shift == 0 ||
-            (Ext == AArch64_AM::UXTX && (Shift <= 3 || Shift == Extra)));
-  }
-}
-
-bool AArch64InstrInfo::isExynosShiftFast(const MachineInstr &MI,
-                                         unsigned Extra) {
-  unsigned Imm, Shift;
-  AArch64_AM::ShiftExtendType Ext;
-
-  switch (MI.getOpcode()) {
-  default:
-    return false;
-
-  // WriteISReg
-  case AArch64::ADDSWrs:
-  case AArch64::ADDSXrs:
-  case AArch64::ADDWrs:
-  case AArch64::ADDXrs:
-  case AArch64::ANDSWrs:
-  case AArch64::ANDSXrs:
-  case AArch64::ANDWrs:
-  case AArch64::ANDXrs:
-  case AArch64::BICSWrs:
-  case AArch64::BICSXrs:
-  case AArch64::BICWrs:
-  case AArch64::BICXrs:
-  case AArch64::EONWrs:
-  case AArch64::EONXrs:
-  case AArch64::EORWrs:
-  case AArch64::EORXrs:
-  case AArch64::ORNWrs:
-  case AArch64::ORNXrs:
-  case AArch64::ORRWrs:
-  case AArch64::ORRXrs:
-  case AArch64::SUBSWrs:
-  case AArch64::SUBSXrs:
-  case AArch64::SUBWrs:
-  case AArch64::SUBXrs:
-    Imm = MI.getOperand(3).getImm();
-    Shift = AArch64_AM::getShiftValue(Imm);
-    Ext = AArch64_AM::getShiftType(Imm);
-    return (Shift == 0 ||
-            (Ext == AArch64_AM::LSL && (Shift <= 3 || Shift == Extra)));
-  }
-}
-
 bool AArch64InstrInfo::isFalkorShiftExtFast(const MachineInstr &MI) {
   switch (MI.getOpcode()) {
   default:
     return false;
 
   case AArch64::ADDWrs:
   case AArch64::ADDXrs:
   case AArch64::ADDSWrs:

llvm/lib/Target/AArch64/AArch64SchedExynosM1.td

 }
 
 def M1UnitALU  : ProcResGroup<[M1UnitA,
                                M1UnitC]>;    // All integer
 def M1UnitNALU : ProcResGroup<[M1UnitNAL0,
                                M1UnitNAL1]>; // All simple vector
 
 //===----------------------------------------------------------------------===//
-// Predicates.
-
-def M1BranchLinkPred : SchedPredicate<[{MI->getOpcode() == AArch64::BLR &&
-                                        MI->getOperand(0).getReg() != AArch64::LR}]>;
-def M1LdStExtPred    : SchedPredicate<[{TII->isExynosLdStExtFast(*MI)}]>;
-def M1ShiftExtPred   : SchedPredicate<[{TII->isExynosShiftExtFast(*MI)}]>;
-
-//===----------------------------------------------------------------------===//
 // Coarse scheduling model.
 
 def M1WriteA1 : SchedWriteRes<[M1UnitALU]> { let Latency = 1; }
 def M1WriteA2 : SchedWriteRes<[M1UnitALU]> { let Latency = 2; }
 def M1WriteAA : SchedWriteRes<[M1UnitALU]> { let Latency = 2;
                                              let ResourceCycles = [2]; }
 def M1WriteAB : SchedWriteRes<[M1UnitALU,
                                M1UnitC]>   { let Latency = 1;
                                              let NumMicroOps = 2; }
 def M1WriteAC : SchedWriteRes<[M1UnitALU,
                                M1UnitALU,
                                M1UnitC]>   { let Latency = 2;
                                              let NumMicroOps = 3; }
 def M1WriteAD : SchedWriteRes<[M1UnitALU,
                                M1UnitC]>   { let Latency = 2;
                                              let NumMicroOps = 2; }
-def M1WriteAX : SchedWriteVariant<[SchedVar<M1ShiftExtPred, [M1WriteA1]>,
-                                   SchedVar<NoSchedPred,    [M1WriteAA]>]>;
+def M1WriteAX : SchedWriteVariant<[SchedVar<ExynosShiftPred, [M1WriteA1]>,
+                                   SchedVar<NoSchedPred,     [M1WriteAA]>]>;
 def M1WriteC1 : SchedWriteRes<[M1UnitC]>   { let Latency = 1; }
 def M1WriteC2 : SchedWriteRes<[M1UnitC]>   { let Latency = 2; }
 
 def M1WriteB1 : SchedWriteRes<[M1UnitB]> { let Latency = 1; }
-def M1WriteBX : SchedWriteVariant<[SchedVar<M1BranchLinkPred, [M1WriteAB]>,
-                                   SchedVar<NoSchedPred,      [M1WriteAC]>]>;
+def M1WriteBX : SchedWriteVariant<[SchedVar<ExynosBranchLinkLRPred, [M1WriteAC]>,
+                                   SchedVar<NoSchedPred,            [M1WriteAB]>]>;
 
 def M1WriteL5 : SchedWriteRes<[M1UnitL]> { let Latency = 5; }
 def M1WriteL6 : SchedWriteRes<[M1UnitL]> { let Latency = 6; }
 def M1WriteLA : SchedWriteRes<[M1UnitL]> { let Latency = 6;
                                            let ResourceCycles = [2]; }
 def M1WriteLB : SchedWriteRes<[M1UnitL,
                                M1UnitA]> { let Latency = 4;
                                            let NumMicroOps = 2; }
 def M1WriteLC : SchedWriteRes<[M1UnitL,
                                M1UnitA]> { let Latency = 5;
                                            let NumMicroOps = 2; }
 def M1WriteLD : SchedWriteRes<[M1UnitL,
                                M1UnitA]> { let Latency = 6;
                                            let NumMicroOps = 2;
                                            let ResourceCycles = [2, 1]; }
 def M1WriteLH : SchedWriteRes<[]>        { let Latency = 5;
                                            let NumMicroOps = 0; }
-def M1WriteLX : SchedWriteVariant<[SchedVar<M1LdStExtPred, [M1WriteL5]>,
+def M1WriteLX : SchedWriteVariant<[SchedVar<ExynosLdStExtPred, [M1WriteL5]>,
                                    SchedVar<NoSchedPred,   [M1WriteLC]>]>;
-def M1WriteLY : SchedWriteVariant<[SchedVar<M1LdStExtPred, [M1WriteL5]>,
+def M1WriteLY : SchedWriteVariant<[SchedVar<ExynosLdStExtPred, [M1WriteL5]>,
                                    SchedVar<NoSchedPred,   [M1WriteLD]>]>;
 
 def M1WriteS1 : SchedWriteRes<[M1UnitS]>   { let Latency = 1; }
 def M1WriteS3 : SchedWriteRes<[M1UnitS]>   { let Latency = 3; }
 def M1WriteS4 : SchedWriteRes<[M1UnitS]>   { let Latency = 4; }
 def M1WriteSA : SchedWriteRes<[M1UnitS,
                                M1UnitFST,
                                M1UnitS,
def M1WriteSC : SchedWriteRes<[M1UnitS,
                                              let NumMicroOps = 3; }
 def M1WriteSD : SchedWriteRes<[M1UnitS,
                                M1UnitFST,
                                M1UnitA]>   { let Latency = 1;
                                              let NumMicroOps = 2; }
 def M1WriteSE : SchedWriteRes<[M1UnitS,
                                M1UnitA]>   { let Latency = 2;
                                              let NumMicroOps = 2; }
-def M1WriteSX : SchedWriteVariant<[SchedVar<M1LdStExtPred, [M1WriteS1]>,
-                                   SchedVar<NoSchedPred,   [M1WriteSE]>]>;
-def M1WriteSY : SchedWriteVariant<[SchedVar<M1LdStExtPred, [M1WriteS1]>,
-                                   SchedVar<NoSchedPred,   [M1WriteSB]>]>;
+def M1WriteSX : SchedWriteVariant<[SchedVar<ExynosLdStExtPred, [M1WriteS1]>,
+                                   SchedVar<NoSchedPred,       [M1WriteSE]>]>;
+def M1WriteSY : SchedWriteVariant<[SchedVar<ExynosLdStExtPred, [M1WriteS1]>,
+                                   SchedVar<NoSchedPred,       [M1WriteSB]>]>;
 
 def M1ReadAdrBase : SchedReadVariant<[SchedVar<ScaledIdxPred, [ReadDefault]>,
                                       SchedVar<NoSchedPred,   [ReadDefault]>]>;
 
 // Branch instructions.
 def : WriteRes<WriteBr,    []>        { let Latency = 0; }
 def : WriteRes<WriteBrReg, [M1UnitC]> { let Latency = 1; }
 
def M1WriteVSTG    : SchedWriteRes<[M1UnitNALU,
                                     M1UnitFST,
                                     M1UnitFST]>     { let Latency = 16;
                                                       let NumMicroOps = 6;
                                                       let ResourceCycles = [1, 7, 1, 7, 1, 1, 1, 1, 1]; }
 def M1WriteVSTH    : SchedWriteRes<[M1UnitNALU,
                                     M1UnitS,
                                     M1UnitFST,
                                     M1UnitFST,
-                                    M1UnitFST]>      { let Latency = 14;
-                                                       let NumMicroOps = 4;
-                                                       let ResourceCycles = [1, 7, 1, 7, 1]; }
+                                    M1UnitFST]>     { let Latency = 14;
 def M1WriteVSTI    : SchedWriteRes<[M1UnitNALU,
                                     M1UnitS,
                                     M1UnitFST,
                                     M1UnitS,
                                     M1UnitFST,
                                     M1UnitS,
                                     M1UnitFST,
                                     M1UnitS,
                                     M1UnitFST,
                                     M1UnitFST,
-                                    M1UnitFST]>      { let Latency = 17;
-                                                       let NumMicroOps = 7;
-                                                       let ResourceCycles = [1, 7, 1, 7, 1, 1, 1, 1, 1, 1, 1]; }
+                                    M1UnitFST]>     { let Latency = 17;
 
+// Special cases.
+def M1WriteAES  : SchedWriteRes<[M1UnitNCRYPT]>     { let Latency = 1; }
+def M1WriteCOPY : SchedWriteVariant<[SchedVar<ExynosFPPred, [M1WriteNALU1]>,
+                                     SchedVar<NoSchedPred,  [M1WriteA1]>]>;
+
+// Fast forwarding.
+def M1ReadAES : SchedReadAdvance<1, [M1WriteAES]>;
+
 // Branch instructions
 def : InstRW<[M1WriteB1], (instrs Bcc)>;
 def : InstRW<[M1WriteA1], (instrs BL)>;
 def : InstRW<[M1WriteBX], (instrs BLR)>;
 def : InstRW<[M1WriteC1], (instregex "^CBN?Z[WX]")>;
 def : InstRW<[M1WriteAD], (instregex "^TBN?Z[WX]")>;
 
 // Arithmetic and logical integer instructions.
-def : InstRW<[M1WriteA1], (instrs COPY)>;
 def : InstRW<[M1WriteAX], (instregex ".+r[sx](64)?$")>;
 
+// Move instructions.
+def : InstRW<[M1WriteCOPY], (instrs COPY)>;
+
 // Divide and multiply instructions.
 
 // Miscellaneous instructions.
 
 // Load instructions.
 def : InstRW<[M1WriteLB,
               WriteLDHi,
               WriteAdr],    (instregex "^LDP(SW|W|X)(post|pre)")>;

 def : InstRW<[M1WriteVSTI],   (instregex "ST4Fourv(16b|8h|4s)$")>;
 def : InstRW<[M1WriteVSTI,
               WriteAdr],      (instregex "ST4Fourv(16b|8h|4s)_POST$")>;
 def : InstRW<[M1WriteVSTI],   (instregex "ST4Fourv(2d)$")>;
 def : InstRW<[M1WriteVSTI,
               WriteAdr],      (instregex "ST4Fourv(2d)_POST$")>;
 
 // Cryptography instructions.
-def M1WriteAES : SchedWriteRes<[M1UnitNCRYPT]> { let Latency = 1; }
-def M1ReadAES  : SchedReadAdvance<1, [M1WriteAES]>;
 def : InstRW<[M1WriteAES], (instregex "^AES[DE]")>;
 def : InstRW<[M1WriteAES, M1ReadAES], (instregex "^AESI?MC")>;
 
 def : InstRW<[M1WriteNCRYPT1], (instregex "^PMUL")>;
 def : InstRW<[M1WriteNCRYPT1], (instregex "^SHA1(H|SU)")>;
 def : InstRW<[M1WriteNCRYPT5], (instregex "^SHA1[CMP]")>;
 def : InstRW<[M1WriteNCRYPT1], (instregex "^SHA256SU0")>;
 def : InstRW<[M1WriteNCRYPT5], (instregex "^SHA256(H|SU1)")>;
 
 // CRC instructions.
 def : InstRW<[M1WriteC2], (instregex "^CRC32")>;
 
 } // SchedModel = ExynosM1Model

llvm/lib/Target/AArch64/AArch64SchedExynosM3.td

 def M3UnitNSHT : ProcResGroup<[M3UnitNSHT0,
                                M3UnitNSHT1,
                                M3UnitNSHT2]>;
 def M3UnitNSHF : ProcResGroup<[M3UnitNSHF0,
                                M3UnitNSHF1,
                                M3UnitNSHF2]>;
 
 //===----------------------------------------------------------------------===//
-// Predicates.
-
-def M3BranchLinkPred : SchedPredicate<[{MI->getOpcode() == AArch64::BLR &&
-                                        MI->getOperand(0).isReg() &&
-                                        MI->getOperand(0).getReg() != AArch64::LR}]>;
-def M3ResetPred      : SchedPredicate<[{TII->isExynosResetFast(*MI)}]>;
-def M3RotatePred     : SchedPredicate<[{(MI->getOpcode() == AArch64::EXTRWrri ||
-                                         MI->getOpcode() == AArch64::EXTRXrri) &&
-                                        MI->getOperand(1).isReg() && MI->getOperand(2).isReg() &&
-                                        MI->getOperand(1).getReg() == MI->getOperand(2).getReg()}]>;
-def M3LdStExtPred    : SchedPredicate<[{TII->isExynosLdStExtFast(*MI)}]>;
-def M3ShiftExtPred   : SchedPredicate<[{TII->isExynosShiftExtFast(*MI)}]>;
-
-//===----------------------------------------------------------------------===//
 // Coarse scheduling model.
 
 def M3WriteZ0 : SchedWriteRes<[]> { let Latency = 0;
                                     let NumMicroOps = 1; }
 
 def M3WriteA1 : SchedWriteRes<[M3UnitALU]> { let Latency = 1; }
 def M3WriteAA : SchedWriteRes<[M3UnitALU]> { let Latency = 2;
                                              let ResourceCycles = [2]; }
 def M3WriteAB : SchedWriteRes<[M3UnitALU,
                                M3UnitC]>   { let Latency = 1;
                                              let NumMicroOps = 2; }
 def M3WriteAC : SchedWriteRes<[M3UnitALU,
                                M3UnitALU,
                                M3UnitC]>   { let Latency = 2;
                                              let NumMicroOps = 3; }
 def M3WriteAD : SchedWriteRes<[M3UnitALU,
                                M3UnitC]>   { let Latency = 2;
                                              let NumMicroOps = 2; }
 def M3WriteC1 : SchedWriteRes<[M3UnitC]>   { let Latency = 1; }
 def M3WriteC2 : SchedWriteRes<[M3UnitC]>   { let Latency = 2; }
-def M3WriteAX : SchedWriteVariant<[SchedVar<M3ResetPred,    [M3WriteZ0]>,
-                                   SchedVar<M3ShiftExtPred, [M3WriteA1]>,
-                                   SchedVar<NoSchedPred,    [M3WriteAA]>]>;
-def M3WriteAY : SchedWriteVariant<[SchedVar<M3RotatePred, [M3WriteA1]>,
-                                   SchedVar<NoSchedPred,  [M3WriteAA]>]>;
+def M3WriteAU : SchedWriteVariant<[SchedVar<IsCopyIdiomPred, [M3WriteZ0]>,
+                                   SchedVar<ExynosShiftPred, [M3WriteA1]>,
+                                   SchedVar<NoSchedPred,     [M3WriteAA]>]>;
+def M3WriteAV : SchedWriteVariant<[SchedVar<IsCopyIdiomPred, [M3WriteZ0]>,
+                                   SchedVar<NoSchedPred,     [M3WriteAA]>]>;
+def M3WriteAW : SchedWriteVariant<[SchedVar<IsZeroIdiomPred, [M3WriteZ0]>,
+                                   SchedVar<NoSchedPred,     [M3WriteAA]>]>;
+def M3WriteAX : SchedWriteVariant<[SchedVar<ExynosExtPred, [M3WriteA1]>,
+                                   SchedVar<NoSchedPred,   [M3WriteAA]>]>;
+def M3WriteAY : SchedWriteVariant<[SchedVar<ExynosRotateRightImmPred, [M3WriteA1]>,
+                                   SchedVar<NoSchedPred,              [M3WriteAA]>]>;
+def M3WriteAZ : SchedWriteVariant<[SchedVar<ExynosShiftPred, [M3WriteA1]>,
+                                   SchedVar<NoSchedPred,     [M3WriteAA]>]>;
 
 def M3WriteB1 : SchedWriteRes<[M3UnitB]> { let Latency = 1; }
-def M3WriteBX : SchedWriteVariant<[SchedVar<M3BranchLinkPred, [M3WriteAB]>,
-                                   SchedVar<NoSchedPred,      [M3WriteAC]>]>;
+def M3WriteBX : SchedWriteVariant<[SchedVar<ExynosBranchLinkLRPred, [M3WriteAC]>,
+                                   SchedVar<NoSchedPred,            [M3WriteAB]>]>;
 
 def M3WriteL4 : SchedWriteRes<[M3UnitL]> { let Latency = 4; }
 def M3WriteL5 : SchedWriteRes<[M3UnitL]> { let Latency = 5; }
 def M3WriteLA : SchedWriteRes<[M3UnitL,
                                M3UnitL]> { let Latency = 5;
                                            let NumMicroOps = 1; }
 def M3WriteLB : SchedWriteRes<[M3UnitA,
                                M3UnitL]> { let Latency = 5;
                                            let NumMicroOps = 2; }
 def M3WriteLC : SchedWriteRes<[M3UnitA,
                                M3UnitL,
                                M3UnitL]> { let Latency = 5;
                                            let NumMicroOps = 2; }
 def M3WriteLD : SchedWriteRes<[M3UnitA,
                                M3UnitL]> { let Latency = 4;
                                            let NumMicroOps = 2; }
 def M3WriteLH : SchedWriteRes<[]>        { let Latency = 5;
                                            let NumMicroOps = 0; }
 
-def M3WriteLX : SchedWriteVariant<[SchedVar<M3LdStExtPred, [M3WriteL5]>,
-                                   SchedVar<NoSchedPred,   [M3WriteLB]>]>;
+def M3WriteLX : SchedWriteVariant<[SchedVar<ExynosLdStExtPred, [M3WriteL5]>,
+                                   SchedVar<NoSchedPred,       [M3WriteLB]>]>;
 
 def M3WriteS1 : SchedWriteRes<[M3UnitS]>   { let Latency = 1; }
 def M3WriteSA : SchedWriteRes<[M3UnitA,
                                M3UnitS,
                                M3UnitFST]> { let Latency = 2;
                                              let NumMicroOps = 2; }
 def M3WriteSB : SchedWriteRes<[M3UnitA,
                                M3UnitS]>   { let Latency = 1;
                                              let NumMicroOps = 2; }
 def M3WriteSC : SchedWriteRes<[M3UnitA,
                                M3UnitS]>   { let Latency = 2;
                                              let NumMicroOps = 2; }
 
-def M3WriteSX : SchedWriteVariant<[SchedVar<M3LdStExtPred, [M3WriteS1]>,
-                                   SchedVar<NoSchedPred,   [M3WriteSB]>]>;
-def M3WriteSY : SchedWriteVariant<[SchedVar<M3LdStExtPred, [M3WriteS1]>,
-                                   SchedVar<NoSchedPred,   [M3WriteSC]>]>;
+def M3WriteSX : SchedWriteVariant<[SchedVar<ExynosLdStExtPred, [M3WriteS1]>,
+                                   SchedVar<NoSchedPred,       [M3WriteSB]>]>;
+def M3WriteSY : SchedWriteVariant<[SchedVar<ExynosLdStExtPred, [M3WriteS1]>,
+                                   SchedVar<NoSchedPred,       [M3WriteSC]>]>;
 
 def M3ReadAdrBase : SchedReadVariant<[SchedVar<ScaledIdxPred, [ReadDefault]>,
                                       SchedVar<NoSchedPred,   [ReadDefault]>]>;
 
 // Branch instructions.
 def : SchedAlias<WriteBr, M3WriteZ0>;
 def : WriteRes<WriteBrReg, [M3UnitC]> { let Latency = 1; }
 

 def : WriteRes<WriteID64, [M3UnitC,
                            M3UnitD]>  { let Latency = 21;
                                         let ResourceCycles = [1, 21]; }
 def : WriteRes<WriteIM32, [M3UnitC]>  { let Latency = 3; }
 def : WriteRes<WriteIM64, [M3UnitC]>  { let Latency = 4;
                                         let ResourceCycles = [2]; }
 
 // Miscellaneous instructions.
-def : WriteRes<WriteExtr, [M3UnitALU,
-                           M3UnitALU]> { let Latency = 1;
-                                         let NumMicroOps = 2; }
+def : SchedAlias<WriteExtr, M3WriteAY>;
 
 // Addressing modes.
 def : WriteRes<WriteAdr, []> { let Latency = 1;
                                let NumMicroOps = 0; }
 def : SchedAlias<ReadAdrBase, M3ReadAdrBase>;
 
 // Load instructions.
 def : SchedAlias<WriteLD, M3WriteL4>;
def M3WriteVSTI    : SchedWriteRes<[M3UnitNALU,
                                     M3UnitFST,
                                     M3UnitS,
                                     M3UnitFST]>   { let Latency = 17;
                                                     let NumMicroOps = 9;
                                                     let ResourceCycles = [1, 3, 3, 1, 3, 1, 3, 1, 3]; }
 
 // Special cases.
 def M3WriteAES     : SchedWriteRes<[M3UnitNCRY]>  { let Latency = 1; }
+def M3WriteCOPY    : SchedWriteVariant<[SchedVar<ExynosFPPred, [M3WriteNALU1]>,
+                                        SchedVar<NoSchedPred,  [M3WriteZ0]>]>;
+def M3WriteMOVI    : SchedWriteVariant<[SchedVar<IsZeroFPIdiomPred, [M3WriteZ0]>,
+                                        SchedVar<NoSchedPred,       [M3WriteNALU1]>]>;
+
+// Fast forwarding.
 def M3ReadAES      : SchedReadAdvance<1, [M3WriteAES]>;
 def M3ReadFMAC     : SchedReadAdvance<1, [M3WriteFMAC4,
                                           M3WriteFMAC5]>;
-def M3WriteMOVI    : SchedWriteVariant<[SchedVar<M3ResetPred, [M3WriteZ0]>,
-                                        SchedVar<NoSchedPred, [M3WriteNALU1]>]>;
 def M3ReadNMUL     : SchedReadAdvance<1, [M3WriteNMUL3]>;
 
 // Branch instructions
 def : InstRW<[M3WriteB1], (instrs Bcc)>;
 def : InstRW<[M3WriteA1], (instrs BL)>;
 def : InstRW<[M3WriteBX], (instrs BLR)>;
 def : InstRW<[M3WriteC1], (instregex "^CBN?Z[WX]")>;
 def : InstRW<[M3WriteAD], (instregex "^TBN?Z[WX]")>;
 
 // Arithmetic and logical integer instructions.
-def : InstRW<[M3WriteA1], (instrs COPY)>;
-def : InstRW<[M3WriteAX], (instregex "^(ADD|SUB)S?Xrx64")>;
-def : InstRW<[M3WriteAX], (instregex "^(ADD|AND|BIC|EON|EOR|ORN|ORR|SUB)[WX]r[sx]$")>;
-def : InstRW<[M3WriteAX], (instregex "^(ADD|BIC|SUB)S[WX]r[sx]$")>;
-def : InstRW<[M3WriteAX], (instregex "^(ADD|AND|EOR|ORR|SUB)[WX]ri")>;
+def : InstRW<[M3WriteAZ], (instregex "^(ADD|AND|BIC|EON|EOR|ORN|SUB)[WX]rs$")>;
+def : InstRW<[M3WriteAU], (instrs ORRWrs, ORRXrs)>;
+def : InstRW<[M3WriteAX], (instregex "^(ADD|SUB)S?[WX]rx")>;
+def : InstRW<[M3WriteAZ], (instregex "^(ADD|AND|BIC|SUB)S[WX]rs$")>;
+def : InstRW<[M3WriteAV], (instrs ADDWri, ADDXri)>;
+def : InstRW<[M3WriteAW], (instrs ORRWri, ORRXri)>;
 
 // Move instructions.
-def : InstRW<[M3WriteZ0], (instrs ADR, ADRP)>;
-def : InstRW<[M3WriteZ0], (instregex "^MOV[NZ][WX]i")>;
+def : InstRW<[M3WriteCOPY], (instrs COPY)>;
+def : InstRW<[M3WriteZ0],   (instrs ADR, ADRP)>;
+def : InstRW<[M3WriteZ0],   (instregex "^MOV[NZ][WX]i")>;
 
 // Divide and multiply instructions.
 
 // Miscellaneous instructions.
-def : InstRW<[M3WriteAY], (instrs EXTRWrri, EXTRXrri)>;
 
 // Load instructions.
 def : InstRW<[M3WriteLD,
               WriteLDHi,
               WriteAdr],    (instregex "^LDP(SW|W|X)(post|pre)")>;
 def : InstRW<[M3WriteLX,
               ReadAdrBase], (instregex "^PRFMro[WX]")>;
 

llvm/lib/Target/AArch64/AArch64SchedPredExynos.td

+//===- AArch64SchedPredExynos.td - AArch64 Sched Preds -----*- tablegen -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This file defines scheduling predicate definitions that are used by the
+// AArch64 Exynos processors.
+//
+//===----------------------------------------------------------------------===//
+
+// Identify BLR specifying the LR register as the indirect target register.
+def ExynosBranchLinkLRPred : MCSchedPredicate<
+                               CheckAll<[CheckOpcode<[BLR]>,
+                                         CheckRegOperand<0, LR>]>>;
+
+// Identify arithmetic and logic instructions without or with limited extension.
+def ExynosExtFn   : TIIPredicate<
+                      "isExynosExtFast",
+                      MCOpcodeSwitchStatement<
+                        [MCOpcodeSwitchCase<
+                           IsArithExt32Op.ValidOpcodes,
+                           MCReturnStatement<
+                             CheckAny<[CheckExtBy0,
+                                       CheckAll<
+                                         [CheckExtUXTW,
+                                          CheckAny<
+                                            [CheckExtBy1,
+                                             CheckExtBy2,
+                                             CheckExtBy3]>]>]>>>,
+                         MCOpcodeSwitchCase<
+                           IsArithExt64Op.ValidOpcodes,
+                           MCReturnStatement<
+                             CheckAny<[CheckExtBy0,
+                                       CheckAll<
+                                         [CheckExtUXTX,
+                                          CheckAny<
+                                            [CheckExtBy1,
+                                             CheckExtBy2,
+                                             CheckExtBy3]>]>]>>>],
+                        MCReturnStatement<FalsePred>>>;
+def ExynosExtPred : MCSchedPredicate<ExynosExtFn>;
+
+// Identify FP instructions.
+def ExynosFPPred : MCSchedPredicate<CheckAny<[CheckDForm, CheckQForm]>>;
+
+// Identify load and store instructions using the register offset
+// addressing mode efficiently.
+def ExynosLdStExtFn   : TIIPredicate<
+                          "isExynosLdStExtFast",
+                          MCOpcodeSwitchStatement<
+                            [MCOpcodeSwitchCase<
+                               IsLoadStoreRegOffsetOp.ValidOpcodes,
+                               MCReturnStatement<
+                                 CheckAny<
+                                   [CheckMemExtLSL,
+                                    CheckMemExtSXTX]>>>],
+                            MCReturnStatement<FalsePred>>>;
+def ExynosLdStExtPred : MCSchedPredicate<ExynosLdStExtFn>;
+
+// Identify whether an instruction whose result is a long vector
+// operates on the upper half of the input registers.
+def ExynosLongVectorUpperFn   : TIIPredicate<
+                                  "isExynosLongVectorUpper",
+                                  MCOpcodeSwitchStatement<
+                                  [MCOpcodeSwitchCase<
+                                    IsLongVectorUpperOp.ValidOpcodes,
+                                    MCReturnStatement<TruePred>>],
+                                  MCReturnStatement<FalsePred>>>;
+def ExynosLongVectorUpperPred : MCSchedPredicate<ExynosLongVectorUpperFn>;
+
+// Identify 128-bit NEON instructions.
+def ExynosQFormPred : MCSchedPredicate<CheckQForm>;
+
+// Identify instructions that reset a register efficiently.
+def ExynosResetFn   : TIIPredicate<
+                        "isExynosResetFast",
+                        MCOpcodeSwitchStatement<
+                          [MCOpcodeSwitchCase<
+                             [ADR, ADRP,
+                              MOVNWi, MOVNXi,
+                              MOVZWi, MOVZXi],
+                             MCReturnStatement<TruePred>>],
+                          MCReturnStatement<
+                            CheckAny<
+                              [IsCopyIdiomFn,
+                               IsZeroFPIdiomFn,
+                               IsZeroIdiomFn]>>>>;
+def ExynosResetPred : MCSchedPredicate<ExynosResetFn>;
+
+// Identify EXTR as the alias for ROR (immediate).
+def ExynosRotateRightImmPred : MCSchedPredicate<
+                                 CheckAll<[CheckOpcode<[EXTRWrri, EXTRXrri]>,
+                                           CheckSameRegOperand<1, 2>]>>;
+
+// Identify arithmetic and logic instructions without or with limited shift.
+def ExynosShiftFn   : TIIPredicate<
+                        "isExynosShiftFast",
+                        MCOpcodeSwitchStatement<
+                          [MCOpcodeSwitchCase<
+                             IsArithLogicShiftOp.ValidOpcodes,
+                             MCReturnStatement<
+                               CheckAny<[CheckShiftBy0,
+                                         CheckAll<
+                                           [CheckShiftLSL,
+                                            CheckAny<
+                                              [CheckShiftBy1,
+                                               CheckShiftBy2,
+                                               CheckShiftBy3]>]>]>>>], 
+                          MCReturnStatement<FalsePred>>>;
+def ExynosShiftPred : MCSchedPredicate<ExynosShiftFn>;
+
+// Identify more arithmetic and logic instructions without or limited shift.
+def ExynosShiftExFn   : TIIPredicate<
+                          "isExynosShiftExFast",
+                          MCOpcodeSwitchStatement<
+                            [MCOpcodeSwitchCase<
+                               IsArithLogicShiftOp.ValidOpcodes,
+                               MCReturnStatement<
+                                 CheckAll<
+                                   [CheckShiftLSL,
+                                    CheckShiftBy8]>>>],
+                            MCReturnStatement<ExynosShiftFn>>>;
+def ExynosShiftExPred : MCSchedPredicate<ExynosShiftExFn>;
+
+// Identify arithmetic and logic immediate instructions.
+def ExynosCheapFn : TIIPredicate<
+                      "isExynosCheapAsMove",
+                      MCOpcodeSwitchStatement<
+                        [MCOpcodeSwitchCase<
+                           IsArithLogicImmOp.ValidOpcodes,
+                           MCReturnStatement<TruePred>>],
+                        MCReturnStatement<
+                          CheckAny<
+                            [ExynosExtFn, ExynosResetFn, ExynosShiftFn]>>>>;