[PGO][PGSO] TargetInstrInfo part.

[hjyamauchi]

Summary

(Split of off D67120)

TargetInstrInfo changes for profile guided size optimization.

[arsenm]

llvm/include/llvm/CodeGen/TargetInstrInfo.h

136–138

Why is this needed for commuting?

[hjyamauchi]

llvm/include/llvm/CodeGen/TargetInstrInfo.h

136–138

There isn't anything particular about commuting, but wherever we make a size-vs-speed code gen choice (one in X86InstrInfo.cpp), and if we want to do it in a profile guided manner, we'd need to propagate profile (PSI/MBFI) down.

[arsenm]

llvm/include/llvm/CodeGen/TargetInstrInfo.h

136–138

I don’t think it’s appropriate to pass this in to commuteInstruction. It should just perform the commute and not concern itself with profitability. It would be the caller’s responsibility to determine if commuting is a good idea

[hjyamauchi]

llvm/include/llvm/CodeGen/TargetInstrInfo.h

136–138

Around X86InstrInfo.cpp:1576,

case X86::BLENDPDrri:
case X86::BLENDPDrri:
case X86::BLENDPSrri:
case X86::VBLENDPDrri:
case X86::VBLENDPSrri: {
  // If we're optimizing for size, try to use MOVSD/MOVSS.
  if (MI.getParent()->getParent()->getFunction().hasOptSize()) {
     ....
  }

this code tries to replace BLEND with MOVSD, if possible, under optsize.

This is unfortunately internal to Target/X86InstrInfo (TII) and isn't up to the caller, except X86InstrInfo internally checks for optsize on the function.

This patch would like to make it profile guided (eg. do this for cold code, even without optsize) and it needs access to the profile (PSI/MBFI) there in some way.

It doesn't seem like a good idea to pass the profile to TII per pass or function because it is TII is created once and stateless.

There doesn't seem a good way to access the profile directly from TII because the profile isn't attached to the function or the IR.

Do you see some other way to accomplish this?

We could consider instead passing "bool OptSize" to commuteInstruction. What do you think?

[arsenm]

llvm/include/llvm/CodeGen/TargetInstrInfo.h

136–138

This usage of OptSize seems broken in the first place. commuteInstruction is used, for example, in MachineCSE. This would mean optsize can potentially disable CSE for some operations, which will increase instruction count (and therefore code size) by blocking the commute.

AMDGPU also has situations for example where.a smaller encoding is possible, but we have a pass that uses this based on when it makes sense to switch encodings. commuteInstruction isn't making this decision.

[davidxl]

llvm/include/llvm/CodeGen/TargetInstrInfo.h

136–138

I agree with Matt. Can we remove this change for commute and see if it matters for size? The existing code is broken and we don't want this badness to be exposed at interface level.

[hjyamauchi]

llvm/include/llvm/CodeGen/TargetInstrInfo.h

136–138

Yeah, it looks like some CSE between blend and movsd/movss could indeed be missed depending on whether optsize.

This code came from https://reviews.llvm.org/rL336731. It may be alternatively fine to do a blend -> movsd/movss replacement in a separate pass if optsize.

Thoughts?

[hjyamauchi]

llvm/include/llvm/CodeGen/TargetInstrInfo.h

136–138

I measured the size impact for an internal app. Turned out to be very tiny 0.0002% :) I believe we can drop this patch.

llvm/include/llvm/CodeGen/TargetInstrInfo.h

 namespace llvm {
 
 class AAResults;
 class DFAPacketizer;
 class InstrItineraryData;
 class LiveIntervals;
 class LiveVariables;
 class MachineLoop;
+class MachineBlockFrequencyInfo;
 class MachineMemOperand;
 class MachineRegisterInfo;
 class MCAsmInfo;
 class MCInst;
 struct MCSchedModel;
 class Module;
+class ProfileSummaryInfo;
 class ScheduleDAG;
 class ScheduleHazardRecognizer;
 class SDNode;
 class SelectionDAG;
 class RegScavenger;
 class TargetRegisterClass;
 class TargetRegisterInfo;
 class TargetSchedModel;
protected:
   /// The default implementation simply swaps the commutable operands.
   ///
   /// If NewMI is false, MI is modified in place and returned; otherwise, a
   /// new machine instruction is created and returned.
   ///
   /// Do not call this method for a non-commutable instruction.
   /// Even though the instruction is commutable, the method may still
   /// fail to commute the operands, null pointer is returned in such cases.
-  virtual MachineInstr *commuteInstructionImpl(MachineInstr &MI, bool NewMI,
+  virtual MachineInstr *commuteInstructionImpl(MachineInstr &MI,
+                                               ProfileSummaryInfo *PSI,
+                                               const MachineBlockFrequencyInfo *MBFI,
+                                               bool NewMI,
                                                unsigned OpIdx1,
                                                unsigned OpIdx2) const;
 
   /// Assigns the (CommutableOpIdx1, CommutableOpIdx2) pair of commutable
   /// operand indices to (ResultIdx1, ResultIdx2).
   /// One or both input values of the pair: (ResultIdx1, ResultIdx2) may be
   /// predefined to some indices or be undefined (designated by the special
   /// value 'CommuteAnyOperandIndex').
public:
   /// If NewMI is false, MI is modified in place and returned; otherwise, a
   /// new machine instruction is created and returned.
   ///
   /// Do not call this method for a non-commutable instruction or
   /// for non-commuable operands.
   /// Even though the instruction is commutable, the method may still
   /// fail to commute the operands, null pointer is returned in such cases.
   MachineInstr *
-  commuteInstruction(MachineInstr &MI, bool NewMI = false,
+  commuteInstruction(MachineInstr &MI, ProfileSummaryInfo *PSI,
+                     const MachineBlockFrequencyInfo *MBFI,
+                     bool NewMI = false,
                      unsigned OpIdx1 = CommuteAnyOperandIndex,
                      unsigned OpIdx2 = CommuteAnyOperandIndex) const;
 
   /// Returns true iff the routine could find two commutable operands in the
   /// given machine instruction.
   /// The 'SrcOpIdx1' and 'SrcOpIdx2' are INPUT and OUTPUT arguments.
   /// If any of the INPUT values is set to the special value
   /// 'CommuteAnyOperandIndex' then the method arbitrarily picks a commutable
public:
   /// If this is possible, a new instruction is returned with the specified
   /// operand folded, otherwise NULL is returned.
   /// The new instruction is inserted before MI, and the client is responsible
   /// for removing the old instruction.
   /// If VRM is passed, the assigned physregs can be inspected by target to
   /// decide on using an opcode (note that those assignments can still change).
   MachineInstr *foldMemoryOperand(MachineInstr &MI, ArrayRef<unsigned> Ops,
                                   int FI,
+                                  ProfileSummaryInfo *PSI,
+                                  const MachineBlockFrequencyInfo *MBFI,
                                   LiveIntervals *LIS = nullptr,
                                   VirtRegMap *VRM = nullptr) const;
 
   /// Same as the previous version except it allows folding of any load and
   /// store from / to any address, not just from a specific stack slot.
   MachineInstr *foldMemoryOperand(MachineInstr &MI, ArrayRef<unsigned> Ops,
                                   MachineInstr &LoadMI,
+                                  ProfileSummaryInfo *PSI,
+                                  const MachineBlockFrequencyInfo *MBFI,
                                   LiveIntervals *LIS = nullptr) const;
 
   /// Return true when there is potentially a faster code sequence
   /// for an instruction chain ending in \p Root. All potential patterns are
   /// returned in the \p Pattern vector. Pattern should be sorted in priority
   /// order since the pattern evaluator stops checking as soon as it finds a
   /// faster sequence.
   /// \param Root - Instruction that could be combined with one of its operands
protected:
   /// Target-independent code in foldMemoryOperand will
   /// take care of adding a MachineMemOperand to the newly created instruction.
   /// The instruction and any auxiliary instructions necessary will be inserted
   /// at InsertPt.
   virtual MachineInstr *
   foldMemoryOperandImpl(MachineFunction &MF, MachineInstr &MI,
                         ArrayRef<unsigned> Ops,
                         MachineBasicBlock::iterator InsertPt, int FrameIndex,
+                        ProfileSummaryInfo *PSI,
+                        const MachineBlockFrequencyInfo *MBFI,
                         LiveIntervals *LIS = nullptr,
                         VirtRegMap *VRM = nullptr) const {
     return nullptr;
   }
 
   /// Target-dependent implementation for foldMemoryOperand.
   /// Target-independent code in foldMemoryOperand will
   /// take care of adding a MachineMemOperand to the newly created instruction.
   /// The instruction and any auxiliary instructions necessary will be inserted
   /// at InsertPt.
   virtual MachineInstr *foldMemoryOperandImpl(
       MachineFunction &MF, MachineInstr &MI, ArrayRef<unsigned> Ops,
       MachineBasicBlock::iterator InsertPt, MachineInstr &LoadMI,
+      ProfileSummaryInfo *PSI, const MachineBlockFrequencyInfo *MBFI,
       LiveIntervals *LIS = nullptr) const {
     return nullptr;
   }
 
   /// Target-dependent implementation of getRegSequenceInputs.
   ///
   /// \returns true if it is possible to build the equivalent
   /// REG_SEQUENCE inputs with the pair \p MI, \p DefIdx. False otherwise.
public:
   /// def and use are in the same BB. We only look at one load and see
   /// whether it can be folded into MI. FoldAsLoadDefReg is the virtual register
   /// defined by the load we are trying to fold. DefMI returns the machine
   /// instruction that defines FoldAsLoadDefReg, and the function returns
   /// the machine instruction generated due to folding.
   virtual MachineInstr *optimizeLoadInstr(MachineInstr &MI,
                                           const MachineRegisterInfo *MRI,
                                           unsigned &FoldAsLoadDefReg,
-                                          MachineInstr *&DefMI) const {
+                                          MachineInstr *&DefMI,
+                                          ProfileSummaryInfo *PSI,
+                                          const MachineBlockFrequencyInfo *MBFI) const {
     return nullptr;
   }
 
   /// 'Reg' is known to be defined by a move immediate instruction,
   /// try to fold the immediate into the use instruction.
   /// If MRI->hasOneNonDBGUse(Reg) is true, and this function returns true,
   /// then the caller may assume that DefMI has been erased from its parent
   /// block. The caller may assume that it will not be erased by this
public:
   getExecutionDomain(const MachineInstr &MI) const {
     return std::make_pair(0, 0);
   }
 
   /// Change the opcode of MI to execute in Domain.
   ///
   /// The bit (1 << Domain) must be set in the mask returned from
   /// getExecutionDomain(MI).
-  virtual void setExecutionDomain(MachineInstr &MI, unsigned Domain) const {}
+  virtual void setExecutionDomain(MachineInstr &MI, unsigned Domain,
+                                  ProfileSummaryInfo *PSI,
+                                  const MachineBlockFrequencyInfo *MBFI) const {}
 
   /// Returns the preferred minimum clearance
   /// before an instruction with an unwanted partial register update.
   ///
   /// Some instructions only write part of a register, and implicitly need to
   /// read the other parts of the register.  This may cause unwanted stalls
   /// preventing otherwise unrelated instructions from executing in parallel in
   /// an out-of-order CPU.

llvm/lib/CodeGen/ExecutionDomainFix.cpp

void ExecutionDomainFix::force(int rx, unsigned domain) {
   }
 }
 
 void ExecutionDomainFix::collapse(DomainValue *dv, unsigned domain) {
   assert(dv->hasDomain(domain) && "Cannot collapse");
 
   // Collapse all the instructions.
   while (!dv->Instrs.empty())
-    TII->setExecutionDomain(*dv->Instrs.pop_back_val(), domain);
+    TII->setExecutionDomain(*dv->Instrs.pop_back_val(), domain, nullptr,
+                            nullptr);
   dv->setSingleDomain(domain);
 
   // If there are multiple users, give them new, unique DomainValues.
   if (!LiveRegs.empty() && dv->Refs > 1)
     for (unsigned rx = 0; rx != NumRegs; ++rx)
       if (LiveRegs[rx] == dv)
         setLiveReg(rx, alloc(domain));
 }
for (unsigned i = mi->getDesc().getNumDefs(),
           // now.
           kill(rx);
       }
     }
 
   // If the collapsed operands force a single domain, propagate the collapse.
   if (isPowerOf2_32(available)) {
     unsigned domain = countTrailingZeros(available);
-    TII->setExecutionDomain(*mi, domain);
+    TII->setExecutionDomain(*mi, domain, nullptr, nullptr);
     visitHardInstr(mi, domain);
     return;
   }
 
   // Kill off any remaining uses that don't match available, and build a list of
   // incoming DomainValues that we want to merge.
   SmallVector<int, 4> Regs;
   for (int rx : used) {

llvm/lib/CodeGen/InlineSpiller.cpp

foldMemoryOperand(ArrayRef<std::pair<MachineInstr *, unsigned>> Ops,
   // If we only have implicit uses, we won't be able to fold that.
   // Moreover, TargetInstrInfo::foldMemoryOperand will assert if we try!
   if (FoldOps.empty())
     return false;
 
   MachineInstrSpan MIS(MI, MI->getParent());
 
   MachineInstr *FoldMI =
-      LoadMI ? TII.foldMemoryOperand(*MI, FoldOps, *LoadMI, &LIS)
-             : TII.foldMemoryOperand(*MI, FoldOps, StackSlot, &LIS, &VRM);
+      LoadMI ? TII.foldMemoryOperand(*MI, FoldOps, *LoadMI, nullptr, nullptr,
+                                     &LIS)
+             : TII.foldMemoryOperand(*MI, FoldOps, StackSlot, nullptr, nullptr,
+                                     &LIS, &VRM);
   if (!FoldMI)
     return false;
 
   // Remove LIS for any dead defs in the original MI not in FoldMI.
   for (MIBundleOperands MO(*MI); MO.isValid(); ++MO) {
     if (!MO->isReg())
       continue;
     Register Reg = MO->getReg();

llvm/lib/CodeGen/LiveRangeEdit.cpp

bool LiveRangeEdit::foldAsLoad(LiveInterval *LI,
 
   LLVM_DEBUG(dbgs() << "Try to fold single def: " << *DefMI
                     << "       into single use: " << *UseMI);
 
   SmallVector<unsigned, 8> Ops;
   if (UseMI->readsWritesVirtualRegister(LI->reg, &Ops).second)
     return false;
 
-  MachineInstr *FoldMI = TII.foldMemoryOperand(*UseMI, Ops, *DefMI, &LIS);
+  MachineInstr *FoldMI = TII.foldMemoryOperand(*UseMI, Ops, *DefMI,
+                                               nullptr, nullptr, &LIS);
   if (!FoldMI)
     return false;
   LLVM_DEBUG(dbgs() << "                folded: " << *FoldMI);
   LIS.ReplaceMachineInstrInMaps(*UseMI, *FoldMI);
   if (UseMI->isCall())
     UseMI->getMF()->moveCallSiteInfo(UseMI, FoldMI);
   UseMI->eraseFromParent();
   DefMI->addRegisterDead(LI->reg, nullptr);

llvm/lib/CodeGen/MachineCSE.cpp

if (!FoundCSE) {
         // Try again to see if CSE is possible.
         FoundCSE = VNT.count(MI);
       }
     }
 
     // Commute commutable instructions.
     bool Commuted = false;
     if (!FoundCSE && MI->isCommutable()) {
-      if (MachineInstr *NewMI = TII->commuteInstruction(*MI)) {
+      if (MachineInstr *NewMI = TII->commuteInstruction(*MI, nullptr,
+                                                        nullptr)) {
         Commuted = true;
         FoundCSE = VNT.count(NewMI);
         if (NewMI != MI) {
           // New instruction. It doesn't need to be kept.
           NewMI->eraseFromParent();
           Changed = true;
         } else if (!FoundCSE)
           // MI was changed but it didn't help, commute it back!
-          (void)TII->commuteInstruction(*MI);
+          (void)TII->commuteInstruction(*MI, nullptr, nullptr);
       }
     }
 
     // If the instruction defines physical registers and the values *may* be
     // used, then it's not safe to replace it with a common subexpression.
     // It's also not safe if the instruction uses physical registers.
     bool CrossMBBPhysDef = false;
     SmallSet<unsigned, 8> PhysRefs;

llvm/lib/CodeGen/PeepholeOptimizer.cpp

if (findTargetRecurrence(PHI.getOperand(0).getReg(), TargetRegs, RC)) {
     // Commutes operands of instructions in RC if necessary so that the copy to
     // be generated from PHI can be coalesced.
     LLVM_DEBUG(dbgs() << "Optimize recurrence chain from " << PHI);
     for (auto &RI : RC) {
       LLVM_DEBUG(dbgs() << "\tInst: " << *(RI.getMI()));
       auto CP = RI.getCommutePair();
       if (CP) {
         Changed = true;
-        TII->commuteInstruction(*(RI.getMI()), false, (*CP).first,
-                                (*CP).second);
+        TII->commuteInstruction(*(RI.getMI()), nullptr, nullptr, false,
+                                (*CP).first, (*CP).second);
         LLVM_DEBUG(dbgs() << "\t\tCommuted: " << *(RI.getMI()));
       }
     }
   }
 
   return Changed;
 }
 
for (MachineBasicBlock::iterator MII = MBB.begin(), MIE = MBB.end();
           if (FoldAsLoadDefCandidates.count(FoldAsLoadDefReg)) {
             // We need to fold load after optimizeCmpInstr, since
             // optimizeCmpInstr can enable folding by converting SUB to CMP.
             // Save FoldAsLoadDefReg because optimizeLoadInstr() resets it and
             // we need it for markUsesInDebugValueAsUndef().
             unsigned FoldedReg = FoldAsLoadDefReg;
             MachineInstr *DefMI = nullptr;
             if (MachineInstr *FoldMI =
-                    TII->optimizeLoadInstr(*MI, MRI, FoldAsLoadDefReg, DefMI)) {
+                    TII->optimizeLoadInstr(*MI, MRI, FoldAsLoadDefReg, DefMI,
+                                           nullptr, nullptr)) {
               // Update LocalMIs since we replaced MI with FoldMI and deleted
               // DefMI.
               LLVM_DEBUG(dbgs() << "Replacing: " << *MI);
               LLVM_DEBUG(dbgs() << "     With: " << *FoldMI);
               LocalMIs.erase(MI);
               LocalMIs.erase(DefMI);
               LocalMIs.insert(FoldMI);
               if (MI->isCall())

llvm/lib/CodeGen/RegisterCoalescer.cpp

RegisterCoalescer::removeCopyByCommutingDef(const CoalescerPair &CP,
 
   LLVM_DEBUG(dbgs() << "\tremoveCopyByCommutingDef: " << AValNo->def << '\t'
                     << *DefMI);
 
   // At this point we have decided that it is legal to do this
   // transformation.  Start by commuting the instruction.
   MachineBasicBlock *MBB = DefMI->getParent();
   MachineInstr *NewMI =
-      TII->commuteInstruction(*DefMI, false, UseOpIdx, NewDstIdx);
+      TII->commuteInstruction(*DefMI, nullptr, nullptr, false, UseOpIdx,
+                              NewDstIdx);
   if (!NewMI)
     return { false, false };
   if (Register::isVirtualRegister(IntA.reg) &&
       Register::isVirtualRegister(IntB.reg) &&
       !MRI->constrainRegClass(IntB.reg, MRI->getRegClass(IntA.reg)))
     return { false, false };
   if (NewMI != DefMI) {
     LIS->ReplaceMachineInstrInMaps(*DefMI, *NewMI);

llvm/lib/CodeGen/TargetInstrInfo.cpp

TargetInstrInfo::ReplaceTailWithBranchTo(MachineBasicBlock::iterator Tail,
 
   // If MBB isn't immediately before MBB, insert a branch to it.
   if (++MachineFunction::iterator(MBB) != MachineFunction::iterator(NewDest))
     insertBranch(*MBB, NewDest, nullptr, SmallVector<MachineOperand, 0>(), DL);
   MBB->addSuccessor(NewDest);
 }
 
 MachineInstr *TargetInstrInfo::commuteInstructionImpl(MachineInstr &MI,
+                                                      ProfileSummaryInfo *PSI,
+                                                      const MachineBlockFrequencyInfo *MBFI,
                                                       bool NewMI, unsigned Idx1,
                                                       unsigned Idx2) const {
   const MCInstrDesc &MCID = MI.getDesc();
   bool HasDef = MCID.getNumDefs();
   if (HasDef && !MI.getOperand(0).isReg())
     // No idea how to commute this instruction. Target should implement its own.
     return nullptr;
 
MachineInstr *TargetInstrInfo::commuteInstructionImpl(MachineInstr &MI,
   // renamable property is only queried/set for physical registers.
   if (Register::isPhysicalRegister(Reg1))
     CommutedMI->getOperand(Idx2).setIsRenamable(Reg1IsRenamable);
   if (Register::isPhysicalRegister(Reg2))
     CommutedMI->getOperand(Idx1).setIsRenamable(Reg2IsRenamable);
   return CommutedMI;
 }
 
-MachineInstr *TargetInstrInfo::commuteInstruction(MachineInstr &MI, bool NewMI,
+MachineInstr *TargetInstrInfo::commuteInstruction(MachineInstr &MI,
+                                                  ProfileSummaryInfo *PSI,
+                                                  const MachineBlockFrequencyInfo *MBFI,
+                                                  bool NewMI,
                                                   unsigned OpIdx1,
                                                   unsigned OpIdx2) const {
   // If OpIdx1 or OpIdx2 is not specified, then this method is free to choose
   // any commutable operand, which is done in findCommutedOpIndices() method
   // called below.
   if ((OpIdx1 == CommuteAnyOperandIndex || OpIdx2 == CommuteAnyOperandIndex) &&
       !findCommutedOpIndices(MI, OpIdx1, OpIdx2)) {
     assert(MI.isCommutable() &&
            "Precondition violation: MI must be commutable.");
     return nullptr;
   }
-  return commuteInstructionImpl(MI, NewMI, OpIdx1, OpIdx2);
+  return commuteInstructionImpl(MI, PSI, MBFI, NewMI, OpIdx1, OpIdx2);
 }
 
 bool TargetInstrInfo::fixCommutedOpIndices(unsigned &ResultIdx1,
                                            unsigned &ResultIdx2,
                                            unsigned CommutableOpIdx1,
                                            unsigned CommutableOpIdx2) {
   if (ResultIdx1 == CommuteAnyOperandIndex &&
       ResultIdx2 == CommuteAnyOperandIndex) {
for (unsigned i = StartIdx; i < MI.getNumOperands(); ++i) {
     else
       MIB.add(MO);
   }
   return NewMI;
 }
 
 MachineInstr *TargetInstrInfo::foldMemoryOperand(MachineInstr &MI,
                                                  ArrayRef<unsigned> Ops, int FI,
+                                                 ProfileSummaryInfo *PSI,
+                                                 const MachineBlockFrequencyInfo *MBFI,
                                                  LiveIntervals *LIS,
                                                  VirtRegMap *VRM) const {
   auto Flags = MachineMemOperand::MONone;
   for (unsigned OpIdx : Ops)
     Flags |= MI.getOperand(OpIdx).isDef() ? MachineMemOperand::MOStore
                                           : MachineMemOperand::MOLoad;
 
   MachineBasicBlock *MBB = MI.getParent();
if (MI.getOpcode() == TargetOpcode::STACKMAP ||
       MI.getOpcode() == TargetOpcode::PATCHPOINT ||
       MI.getOpcode() == TargetOpcode::STATEPOINT) {
     // Fold stackmap/patchpoint.
     NewMI = foldPatchpoint(MF, MI, Ops, FI, *this);
     if (NewMI)
       MBB->insert(MI, NewMI);
   } else {
     // Ask the target to do the actual folding.
-    NewMI = foldMemoryOperandImpl(MF, MI, Ops, MI, FI, LIS, VRM);
+    NewMI = foldMemoryOperandImpl(MF, MI, Ops, MI, FI, PSI, MBFI, LIS, VRM);
   }
 
   if (NewMI) {
     NewMI->setMemRefs(MF, MI.memoperands());
     // Add a memory operand, foldMemoryOperandImpl doesn't do that.
     assert((!(Flags & MachineMemOperand::MOStore) ||
             NewMI->mayStore()) &&
            "Folded a def to a non-store!");
MachineInstr *TargetInstrInfo::foldMemoryOperand(MachineInstr &MI,
   else
     loadRegFromStackSlot(*MBB, Pos, MO.getReg(), FI, RC, TRI);
   return &*--Pos;
 }
 
 MachineInstr *TargetInstrInfo::foldMemoryOperand(MachineInstr &MI,
                                                  ArrayRef<unsigned> Ops,
                                                  MachineInstr &LoadMI,
+                                                 ProfileSummaryInfo *PSI,
+                                                 const MachineBlockFrequencyInfo *MBFI,
                                                  LiveIntervals *LIS) const {
   assert(LoadMI.canFoldAsLoad() && "LoadMI isn't foldable!");
 #ifndef NDEBUG
   for (unsigned OpIdx : Ops)
     assert(MI.getOperand(OpIdx).isUse() && "Folding load into def!");
 #endif
 
   MachineBasicBlock &MBB = *MI.getParent();
   MachineFunction &MF = *MBB.getParent();
 
   // Ask the target to do the actual folding.
   MachineInstr *NewMI = nullptr;
   int FrameIndex = 0;
 
   if ((MI.getOpcode() == TargetOpcode::STACKMAP ||
        MI.getOpcode() == TargetOpcode::PATCHPOINT ||
        MI.getOpcode() == TargetOpcode::STATEPOINT) &&
       isLoadFromStackSlot(LoadMI, FrameIndex)) {
     // Fold stackmap/patchpoint.
     NewMI = foldPatchpoint(MF, MI, Ops, FrameIndex, *this);
     if (NewMI)
       NewMI = &*MBB.insert(MI, NewMI);
   } else {
     // Ask the target to do the actual folding.
-    NewMI = foldMemoryOperandImpl(MF, MI, Ops, MI, LoadMI, LIS);
+    NewMI = foldMemoryOperandImpl(MF, MI, Ops, MI, LoadMI, PSI, MBFI, LIS);
   }
 
   if (!NewMI)
     return nullptr;
 
   // Copy the memoperands from the load to the folded instruction.
   if (MI.memoperands_empty()) {
     NewMI->setMemRefs(MF, LoadMI.memoperands());

llvm/lib/CodeGen/TwoAddressInstructionPass.cpp

 /// and live variables if needed. Return true if it is successful.
 bool TwoAddressInstructionPass::commuteInstruction(MachineInstr *MI,
                                                    unsigned DstIdx,
                                                    unsigned RegBIdx,
                                                    unsigned RegCIdx,
                                                    unsigned Dist) {
   Register RegC = MI->getOperand(RegCIdx).getReg();
   LLVM_DEBUG(dbgs() << "2addr: COMMUTING  : " << *MI);
-  MachineInstr *NewMI = TII->commuteInstruction(*MI, false, RegBIdx, RegCIdx);
+  MachineInstr *NewMI = TII->commuteInstruction(*MI, nullptr, nullptr, false,
+                                                RegBIdx, RegCIdx);
 
   if (NewMI == nullptr) {
     LLVM_DEBUG(dbgs() << "2addr: COMMUTING FAILED!\n");
     return false;
   }
 
   LLVM_DEBUG(dbgs() << "2addr: COMMUTED TO: " << *NewMI);
   assert(NewMI == MI &&

llvm/lib/Target/AArch64/AArch64InstrInfo.h

public:
   // with subreg operands to foldMemoryOperandImpl.
   bool isSubregFoldable() const override { return true; }
 
   using TargetInstrInfo::foldMemoryOperandImpl;
   MachineInstr *
   foldMemoryOperandImpl(MachineFunction &MF, MachineInstr &MI,
                         ArrayRef<unsigned> Ops,
                         MachineBasicBlock::iterator InsertPt, int FrameIndex,
+                        ProfileSummaryInfo *PSI,
+                        const MachineBlockFrequencyInfo *MBFI,
                         LiveIntervals *LIS = nullptr,
                         VirtRegMap *VRM = nullptr) const override;
 
   /// \returns true if a branch from an instruction with opcode \p BranchOpc
   ///  bytes is capable of jumping to a position \p BrOffset bytes away.
   bool isBranchOffsetInRange(unsigned BranchOpc,
                              int64_t BrOffset) const override;
 

llvm/lib/Target/AArch64/AArch64InstrInfo.cpp

if (NumPredicateVectors) {
     emitFrameOffsetAdj(MBB, MBBI, DL, DestReg, SrcReg, NumPredicateVectors,
                        AArch64::ADDPL_XXI, TII, Flag, NeedsWinCFI, nullptr);
   }
 }
 
 MachineInstr *AArch64InstrInfo::foldMemoryOperandImpl(
     MachineFunction &MF, MachineInstr &MI, ArrayRef<unsigned> Ops,
     MachineBasicBlock::iterator InsertPt, int FrameIndex,
+    ProfileSummaryInfo *PSI, const MachineBlockFrequencyInfo *MBFI,
     LiveIntervals *LIS, VirtRegMap *VRM) const {
   // This is a bit of a hack. Consider this instruction:
   //
   //   %0 = COPY %sp; GPR64all:%0
   //
   // We explicitly chose GPR64all for the virtual register so such a copy might
   // be eliminated by RegisterCoalescer. However, that may not be possible, and
   // %0 may even spill. We can't spill %sp, and since it is in the GPR64all

llvm/lib/Target/AMDGPU/GCNDPPCombine.cpp

if (Use == TII->getNamedOperand(OrigMI, AMDGPU::OpName::src0)) {
         DPPMIs.push_back(DPPInst);
         Rollback = false;
       }
     } else if (OrigMI.isCommutable() &&
                Use == TII->getNamedOperand(OrigMI, AMDGPU::OpName::src1)) {
       auto *BB = OrigMI.getParent();
       auto *NewMI = BB->getParent()->CloneMachineInstr(&OrigMI);
       BB->insert(OrigMI, NewMI);
-      if (TII->commuteInstruction(*NewMI)) {
+      if (TII->commuteInstruction(*NewMI, nullptr, nullptr)) {
         LLVM_DEBUG(dbgs() << "  commuted:  " << *NewMI);
         if (auto *DPPInst = createDPPInst(*NewMI, MovMI, CombOldVGPR,
                                           OldOpndValue, CombBCZ)) {
           DPPMIs.push_back(DPPInst);
           Rollback = false;
         }
       } else
         LLVM_DEBUG(dbgs() << "  failed: cannot be commuted\n");

llvm/lib/Target/AMDGPU/SIFoldOperands.cpp

if ((Fold.isImm() || Fold.isFI() || Fold.isGlobal()) && Fold.needsShrink()) {
     // this. Should track set of foldable movs instead of looking for uses
     // when looking at a use.
     Dst0.setReg(NewReg0);
     for (unsigned I = MI->getNumOperands() - 1; I > 0; --I)
       MI->RemoveOperand(I);
     MI->setDesc(TII.get(AMDGPU::IMPLICIT_DEF));
 
     if (Fold.isCommuted())
-      TII.commuteInstruction(*Inst32, false);
+      TII.commuteInstruction(*Inst32, nullptr, nullptr, false);
     return true;
   }
 
   assert(!Fold.needsShrink() && "not handled");
 
   if (Fold.isImm()) {
     Old.ChangeToImmediate(Fold.ImmToFold);
     return true;
if (!TII->isOperandLegal(*MI, OpNo, OpToFold)) {
     // the call of commuteInstruction() below. Such situations are avoided
     // here explicitly as OpNo must be a register operand to be a candidate
     // for memory folding.
     if (CanCommute && (!MI->getOperand(CommuteIdx0).isReg() ||
                        !MI->getOperand(CommuteIdx1).isReg()))
       return false;
 
     if (!CanCommute ||
-        !TII->commuteInstruction(*MI, false, CommuteIdx0, CommuteIdx1))
+        !TII->commuteInstruction(*MI, nullptr, nullptr, false, CommuteIdx0,
+                                 CommuteIdx1))
       return false;
 
     if (!TII->isOperandLegal(*MI, CommuteOpNo, OpToFold)) {
       if ((Opc == AMDGPU::V_ADD_I32_e64 ||
            Opc == AMDGPU::V_SUB_I32_e64 ||
            Opc == AMDGPU::V_SUBREV_I32_e64) && // FIXME
           (OpToFold->isImm() || OpToFold->isFI() || OpToFold->isGlobal())) {
         MachineRegisterInfo &MRI = MI->getParent()->getParent()->getRegInfo();
if (!TII->isOperandLegal(*MI, CommuteOpNo, OpToFold)) {
         // Make sure to get the 32-bit version of the commuted opcode.
         unsigned MaybeCommutedOpc = MI->getOpcode();
         int Op32 = AMDGPU::getVOPe32(MaybeCommutedOpc);
 
         appendFoldCandidate(FoldList, MI, CommuteOpNo, OpToFold, true, Op32);
         return true;
       }
 
-      TII->commuteInstruction(*MI, false, CommuteIdx0, CommuteIdx1);
+      TII->commuteInstruction(*MI, nullptr, nullptr, false, CommuteIdx0,
+                              CommuteIdx1);
       return false;
     }
 
     appendFoldCandidate(FoldList, MI, CommuteOpNo, OpToFold, true);
     return true;
   }
 
   appendFoldCandidate(FoldList, MI, OpNo, OpToFold);
if (updateOperand(Fold, *TII, *TRI, *ST)) {
         MRI->clearKillFlags(Fold.OpToFold->getReg());
       }
       LLVM_DEBUG(dbgs() << "Folded source from " << MI << " into OpNo "
                         << static_cast<int>(Fold.UseOpNo) << " of "
                         << *Fold.UseMI << '\n');
       tryFoldInst(TII, Fold.UseMI);
     } else if (Fold.isCommuted()) {
       // Restoring instruction's original operand order if fold has failed.
-      TII->commuteInstruction(*Fold.UseMI, false);
+      TII->commuteInstruction(*Fold.UseMI, nullptr, nullptr, false);
     }
   }
 }
 
 // Clamp patterns are canonically selected to v_max_* instructions, so only
 // handle them.
 const MachineOperand *SIFoldOperands::isClamp(const MachineInstr &MI) const {
   unsigned Op = MI.getOpcode();

llvm/lib/Target/AMDGPU/SIInsertSkips.cpp

for (++A ; A != E ; ++A) {
     ReadsCond |= A->readsRegister(CondReg, TRI);
   }
   if (A == E)
     return false;
 
   MachineOperand &Op1 = A->getOperand(1);
   MachineOperand &Op2 = A->getOperand(2);
   if (Op1.getReg() != ExecReg && Op2.isReg() && Op2.getReg() == ExecReg) {
-    TII->commuteInstruction(*A);
+    TII->commuteInstruction(*A, nullptr, nullptr);
     Changed = true;
   }
   if (Op1.getReg() != ExecReg)
     return Changed;
   if (Op2.isImm() && Op2.getImm() != -1)
     return Changed;
 
   unsigned SReg = AMDGPU::NoRegister;

llvm/lib/Target/AMDGPU/SIInstrInfo.h

private:
 
   unsigned findUsedSGPR(const MachineInstr &MI, int OpIndices[3]) const;
 
 protected:
   bool swapSourceModifiers(MachineInstr &MI,
                            MachineOperand &Src0, unsigned Src0OpName,
                            MachineOperand &Src1, unsigned Src1OpName) const;
 
-  MachineInstr *commuteInstructionImpl(MachineInstr &MI, bool NewMI,
+  MachineInstr *commuteInstructionImpl(MachineInstr &MI,
+                                       ProfileSummaryInfo *PSI,
+                                       const MachineBlockFrequencyInfo *MBFI,
+                                       bool NewMI,
                                        unsigned OpIdx0,
                                        unsigned OpIdx1) const override;
 
 public:
   enum TargetOperandFlags {
     MO_MASK = 0xf,
 
     MO_NONE = 0,
public:
   }
 
   void fixImplicitOperands(MachineInstr &MI) const;
 
   MachineInstr *foldMemoryOperandImpl(MachineFunction &MF, MachineInstr &MI,
                                       ArrayRef<unsigned> Ops,
                                       MachineBasicBlock::iterator InsertPt,
                                       int FrameIndex,
+                                      ProfileSummaryInfo *PSI,
+                                      const MachineBlockFrequencyInfo *MBFI,
                                       LiveIntervals *LIS = nullptr,
                                       VirtRegMap *VRM = nullptr) const override;
 };
 
 /// \brief Returns true if a reg:subreg pair P has a TRC class
 inline bool isOfRegClass(const TargetInstrInfo::RegSubRegPair &P,
                          const TargetRegisterClass &TRC,
                          MachineRegisterInfo &MRI) {

llvm/lib/Target/AMDGPU/SIInstrInfo.cpp

else
     return nullptr;
 
   NonRegOp.ChangeToRegister(Reg, false, false, IsKill, IsDead, IsUndef, IsDebug);
   NonRegOp.setSubReg(SubReg);
 
   return &MI;
 }
 
-MachineInstr *SIInstrInfo::commuteInstructionImpl(MachineInstr &MI, bool NewMI,
+MachineInstr *SIInstrInfo::commuteInstructionImpl(MachineInstr &MI,
+                                                  ProfileSummaryInfo *PSI,
+                                                  const MachineBlockFrequencyInfo *MBFI,
+                                                  bool NewMI,
                                                   unsigned Src0Idx,
                                                   unsigned Src1Idx) const {
   assert(!NewMI && "this should never be used");
 
   unsigned Opc = MI.getOpcode();
   int CommutedOpcode = commuteOpcode(Opc);
   if (CommutedOpcode == -1)
     return nullptr;
 
   assert(AMDGPU::getNamedOperandIdx(Opc, AMDGPU::OpName::src0) ==
            static_cast<int>(Src0Idx) &&
          AMDGPU::getNamedOperandIdx(Opc, AMDGPU::OpName::src1) ==
            static_cast<int>(Src1Idx) &&
          "inconsistency with findCommutedOpIndices");
 
   MachineOperand &Src0 = MI.getOperand(Src0Idx);
   MachineOperand &Src1 = MI.getOperand(Src1Idx);
 
   MachineInstr *CommutedMI = nullptr;
   if (Src0.isReg() && Src1.isReg()) {
     if (isOperandLegal(MI, Src1Idx, &Src0)) {
       // Be sure to copy the source modifiers to the right place.
       CommutedMI
-        = TargetInstrInfo::commuteInstructionImpl(MI, NewMI, Src0Idx, Src1Idx);
+          = TargetInstrInfo::commuteInstructionImpl(MI, PSI, MBFI, NewMI,
+                                                    Src0Idx, Src1Idx);
     }
 
   } else if (Src0.isReg() && !Src1.isReg()) {
     // src0 should always be able to support any operand type, so no need to
     // check operand legality.
     CommutedMI = swapRegAndNonRegOperand(MI, Src0, Src1);
   } else if (!Src0.isReg() && Src1.isReg()) {
     if (isOperandLegal(MI, Src1Idx, &Src0))
if (Src2->isReg() && Src2->getReg() == Reg) {
       }
 
       if (Src1->isReg() && !Src0Inlined ) {
         // We have one slot for inlinable constant so far - try to fill it
         MachineInstr *Def = MRI->getUniqueVRegDef(Src1->getReg());
         if (Def && Def->isMoveImmediate() &&
             isInlineConstant(Def->getOperand(1)) &&
             MRI->hasOneUse(Src1->getReg()) &&
-            commuteInstruction(UseMI)) {
+            commuteInstruction(UseMI, nullptr, nullptr)) {
             Src0->ChangeToImmediate(Def->getOperand(1).getImm());
         } else if ((Register::isPhysicalRegister(Src1->getReg()) &&
                     RI.isSGPRClass(RI.getPhysRegClass(Src1->getReg()))) ||
                    (Register::isVirtualRegister(Src1->getReg()) &&
                     RI.isSGPRClass(MRI->getRegClass(Src1->getReg()))))
           return false;
           // VGPR is okay as Src1 - fallthrough
       }
MachineInstr *SIInstrInfo::createPHISourceCopy(
   return TargetInstrInfo::createPHISourceCopy(MBB, InsPt, DL, Src, SrcSubReg,
                                               Dst);
 }
 
 bool llvm::SIInstrInfo::isWave32() const { return ST.isWave32(); }
 
 MachineInstr *SIInstrInfo::foldMemoryOperandImpl(
     MachineFunction &MF, MachineInstr &MI, ArrayRef<unsigned> Ops,
-    MachineBasicBlock::iterator InsertPt, int FrameIndex, LiveIntervals *LIS,
-    VirtRegMap *VRM) const {
+    MachineBasicBlock::iterator InsertPt, int FrameIndex,
+    ProfileSummaryInfo *PSI, const MachineBlockFrequencyInfo *MBFI,
+    LiveIntervals *LIS, VirtRegMap *VRM) const {
   // This is a bit of a hack (copied from AArch64). Consider this instruction:
   //
   //   %0:sreg_32 = COPY $m0
   //
   // We explicitly chose SReg_32 for the virtual register so such a copy might
   // be eliminated by RegisterCoalescer. However, that may not be possible, and
   // %0 may even spill. We can't spill $m0 normally (it would require copying to
   // a numbered SGPR anyway), and since it is in the SReg_32 register class,

llvm/lib/Target/AMDGPU/SIShrinkInstructions.cpp

if (Register::isVirtualRegister(Reg) && MRI.hasOneUse(Reg)) {
           return true;
         }
       }
     }
   }
 
   // We have failed to fold src0, so commute the instruction and try again.
   if (TryToCommute && MI.isCommutable()) {
-    if (TII->commuteInstruction(MI)) {
+    if (TII->commuteInstruction(MI, nullptr, nullptr)) {
       if (foldImmediates(MI, TII, MRI, false))
         return true;
 
       // Commute back.
-      TII->commuteInstruction(MI);
+      TII->commuteInstruction(MI, nullptr, nullptr);
     }
   }
 
   return false;
 }
 
 static bool isKImmOperand(const SIInstrInfo *TII, const MachineOperand &Src) {
   return isInt<16>(Src.getImm()) &&
if ((MO.isReg() && MO.isImplicit()) || MO.isRegMask())
       NewMI.addOperand(MF, MO);
   }
 }
 
 static void shrinkScalarCompare(const SIInstrInfo *TII, MachineInstr &MI) {
   // cmpk instructions do scc = dst <cc op> imm16, so commute the instruction to
   // get constants on the RHS.
   if (!MI.getOperand(0).isReg())
-    TII->commuteInstruction(MI, false, 0, 1);
+    TII->commuteInstruction(MI, nullptr, nullptr, false, 0, 1);
 
   const MachineOperand &Src1 = MI.getOperand(1);
   if (!Src1.isImm())
     return;
 
   int SOPKOpc = AMDGPU::getSOPKOp(MI.getOpcode());
   if (SOPKOpc == -1)
     return;
if (Opc == AMDGPU::S_AND_B32) {
         Opc = AMDGPU::S_XNOR_B32;
       }
     } else {
       llvm_unreachable("unexpected opcode");
     }
 
     if ((Opc == AMDGPU::S_ANDN2_B32 || Opc == AMDGPU::S_ORN2_B32) &&
         SrcImm == Src0) {
-      if (!TII->commuteInstruction(MI, false, 1, 2))
+      if (!TII->commuteInstruction(MI, nullptr, nullptr, false, 1, 2))
         NewImm = 0;
     }
 
     if (NewImm != 0) {
       if (Register::isVirtualRegister(Dest->getReg()) && SrcReg->isReg()) {
         MRI.setRegAllocationHint(Dest->getReg(), 0, SrcReg->getReg());
         MRI.setRegAllocationHint(SrcReg->getReg(), 0, Dest->getReg());
         return true;
for (I = MBB.begin(); I != MBB.end(); I = Next) {
       // satisfied.
       if (MI.getOpcode() == AMDGPU::S_ADD_I32 ||
           MI.getOpcode() == AMDGPU::S_MUL_I32) {
         const MachineOperand *Dest = &MI.getOperand(0);
         MachineOperand *Src0 = &MI.getOperand(1);
         MachineOperand *Src1 = &MI.getOperand(2);
 
         if (!Src0->isReg() && Src1->isReg()) {
-          if (TII->commuteInstruction(MI, false, 1, 2))
+          if (TII->commuteInstruction(MI, nullptr, nullptr, false, 1, 2))
             std::swap(Src0, Src1);
         }
 
         // FIXME: This could work better if hints worked with subregisters. If
         // we have a vector add of a constant, we usually don't get the correct
         // allocation due to the subregister usage.
         if (Register::isVirtualRegister(Dest->getReg()) && Src0->isReg()) {
           MRI.setRegAllocationHint(Dest->getReg(), 0, Src0->getReg());
for (I = MBB.begin(); I != MBB.end(); I = Next) {
       }
 
       if (!TII->hasVALU32BitEncoding(MI.getOpcode()))
         continue;
 
       if (!TII->canShrink(MI, MRI)) {
         // Try commuting the instruction and see if that enables us to shrink
         // it.
-        if (!MI.isCommutable() || !TII->commuteInstruction(MI) ||
+        if (!MI.isCommutable() ||
+            !TII->commuteInstruction(MI, nullptr, nullptr) ||
             !TII->canShrink(MI, MRI))
           continue;
       }
 
       // getVOPe32 could be -1 here if we started with an instruction that had
       // a 32-bit encoding and then commuted it to an instruction that did not.
       if (!TII->hasVALU32BitEncoding(MI.getOpcode()))
         continue;

llvm/lib/Target/ARM/ARMBaseInstrInfo.h

protected:
 
   /// Commutes the operands in the given instruction.
   /// The commutable operands are specified by their indices OpIdx1 and OpIdx2.
   ///
   /// Do not call this method for a non-commutable instruction or for
   /// non-commutable pair of operand indices OpIdx1 and OpIdx2.
   /// Even though the instruction is commutable, the method may still
   /// fail to commute the operands, null pointer is returned in such cases.
-  MachineInstr *commuteInstructionImpl(MachineInstr &MI, bool NewMI,
+  MachineInstr *commuteInstructionImpl(MachineInstr &MI,
+                                       ProfileSummaryInfo *PSI,
+                                       const MachineBlockFrequencyInfo *MBFI,
+                                       bool NewMI,
                                        unsigned OpIdx1,
                                        unsigned OpIdx2) const override;
 
   /// If the specific machine instruction is a instruction that moves/copies
   /// value from one register to another register return true along with
   /// @Source machine operand and @Destination machine operand.
   bool isCopyInstrImpl(const MachineInstr &MI, const MachineOperand *&Source,
                        const MachineOperand *&Destination) const override;
int getOperandLatency(const InstrItineraryData *ItinData,
                         unsigned UseIdx) const override;
   int getOperandLatency(const InstrItineraryData *ItinData,
                         SDNode *DefNode, unsigned DefIdx,
                         SDNode *UseNode, unsigned UseIdx) const override;
 
   /// VFP/NEON execution domains.
   std::pair<uint16_t, uint16_t>
   getExecutionDomain(const MachineInstr &MI) const override;
-  void setExecutionDomain(MachineInstr &MI, unsigned Domain) const override;
+  void setExecutionDomain(MachineInstr &MI, unsigned Domain,
+                          ProfileSummaryInfo *PSI,
+                          const MachineBlockFrequencyInfo *MBFI) const override;
 
   unsigned
   getPartialRegUpdateClearance(const MachineInstr &, unsigned,
                                const TargetRegisterInfo *) const override;
   void breakPartialRegDependency(MachineInstr &, unsigned,
                                  const TargetRegisterInfo *TRI) const override;
 
   /// Get the number of addresses by LDM or VLDM or zero for unknown.

llvm/lib/Target/ARM/ARMBaseInstrInfo.cpp

if (Opc == ARM::tB)
     return ARM::tBcc;
   if (Opc == ARM::t2B)
     return ARM::t2Bcc;
 
   llvm_unreachable("Unknown unconditional branch opcode!");
 }
 
 MachineInstr *ARMBaseInstrInfo::commuteInstructionImpl(MachineInstr &MI,
+                                                       ProfileSummaryInfo *PSI,
+                                                       const MachineBlockFrequencyInfo *MBFI,
                                                        bool NewMI,
                                                        unsigned OpIdx1,
                                                        unsigned OpIdx2) const {
   switch (MI.getOpcode()) {
   case ARM::MOVCCr:
   case ARM::t2MOVCCr: {
     // MOVCC can be commuted by inverting the condition.
     unsigned PredReg = 0;
     ARMCC::CondCodes CC = getInstrPredicate(MI, PredReg);
     // MOVCC AL can't be inverted. Shouldn't happen.
     if (CC == ARMCC::AL || PredReg != ARM::CPSR)
       return nullptr;
     MachineInstr *CommutedMI =
-        TargetInstrInfo::commuteInstructionImpl(MI, NewMI, OpIdx1, OpIdx2);
+        TargetInstrInfo::commuteInstructionImpl(MI, PSI, MBFI, NewMI, OpIdx1,
+                                                OpIdx2);
     if (!CommutedMI)
       return nullptr;
     // After swapping the MOVCC operands, also invert the condition.
     CommutedMI->getOperand(CommutedMI->findFirstPredOperandIdx())
         .setImm(ARMCC::getOppositeCondition(CC));
     return CommutedMI;
   }
   }
-  return TargetInstrInfo::commuteInstructionImpl(MI, NewMI, OpIdx1, OpIdx2);
+  return TargetInstrInfo::commuteInstructionImpl(MI, PSI, MBFI, NewMI, OpIdx1,
+                                                 OpIdx2);
 }
 
 /// Identify instructions that can be folded into a MOVCC instruction, and
 /// return the defining instruction.
 MachineInstr *
 ARMBaseInstrInfo::canFoldIntoMOVCC(unsigned Reg, const MachineRegisterInfo &MRI,
                                    const TargetInstrInfo *TII) const {
   if (!Register::isVirtualRegister(Reg))
static bool getImplicitSPRUseForDPRUse(const TargetRegisterInfo *TRI,
 
   // If the register is known not to be live, there is no need to add an
   // implicit-use.
   ImplicitSReg = 0;
   return true;
 }
 
 void ARMBaseInstrInfo::setExecutionDomain(MachineInstr &MI,
-                                          unsigned Domain) const {
+                                          unsigned Domain,
+                                          ProfileSummaryInfo *PSI,
+                                          const MachineBlockFrequencyInfo *MBFI) const {
   unsigned DstReg, SrcReg, DReg;
   unsigned Lane;
   MachineInstrBuilder MIB(*MI.getParent()->getParent(), MI);
   const TargetRegisterInfo *TRI = &getRegisterInfo();
   switch (MI.getOpcode()) {
   default:
     llvm_unreachable("cannot handle opcode!");
     break;

llvm/lib/Target/ARM/Thumb2SizeReduction.cpp

if (!isARMLowRegister(Reg0) || !isARMLowRegister(Reg1)
         || !isARMLowRegister(Reg2))
       return false;
     if (Reg0 != Reg2) {
       // If the other operand also isn't the same as the destination, we
       // can't reduce.
       if (Reg1 != Reg0)
         return false;
       // Try to commute the operands to make it a 2-address instruction.
-      MachineInstr *CommutedMI = TII->commuteInstruction(*MI);
+      MachineInstr *CommutedMI = TII->commuteInstruction(*MI, nullptr, nullptr);
       if (!CommutedMI)
         return false;
     }
   } else if (Reg0 != Reg1) {
     // Try to commute the operands to make it a 2-address instruction.
     unsigned CommOpIdx1 = 1;
     unsigned CommOpIdx2 = TargetInstrInfo::CommuteAnyOperandIndex;
     if (!TII->findCommutedOpIndices(*MI, CommOpIdx1, CommOpIdx2) ||
         MI->getOperand(CommOpIdx2).getReg() != Reg0)
       return false;
     MachineInstr *CommutedMI =
-        TII->commuteInstruction(*MI, false, CommOpIdx1, CommOpIdx2);
+        TII->commuteInstruction(*MI, nullptr, nullptr, false,
+                                CommOpIdx1, CommOpIdx2);
     if (!CommutedMI)
       return false;
   }
   if (Entry.LowRegs2 && !isARMLowRegister(Reg0))
     return false;
   if (Entry.Imm2Limit) {
     unsigned Imm = MI->getOperand(2).getImm();
     unsigned Limit = (1 << Entry.Imm2Limit) - 1;

llvm/lib/Target/PowerPC/PPCInstrInfo.h

protected:
   ///
   /// Do not call this method for a non-commutable instruction or for
   /// non-commutable pair of operand indices OpIdx1 and OpIdx2.
   /// Even though the instruction is commutable, the method may still
   /// fail to commute the operands, null pointer is returned in such cases.
   ///
   /// For example, we can commute rlwimi instructions, but only if the
   /// rotate amt is zero.  We also have to munge the immediates a bit.
-  MachineInstr *commuteInstructionImpl(MachineInstr &MI, bool NewMI,
+  MachineInstr *commuteInstructionImpl(MachineInstr &MI,
+                                       ProfileSummaryInfo *PSI,
+                                       const MachineBlockFrequencyInfo *MBFI,
+                                       bool NewMI,
                                        unsigned OpIdx1,
                                        unsigned OpIdx2) const override;
 
 public:
   explicit PPCInstrInfo(PPCSubtarget &STI);
 
   /// getRegisterInfo - TargetInstrInfo is a superset of MRegister info.  As
   /// such, whenever a client has an instance of instruction info, it should

llvm/lib/Target/PowerPC/PPCInstrInfo.cpp

if (MI.getOperand(1).isImm() && !MI.getOperand(1).getImm() &&
         MI.getOperand(2).isFI()) {
       FrameIndex = MI.getOperand(2).getIndex();
       return MI.getOperand(0).getReg();
     }
   }
   return 0;
 }
 
-MachineInstr *PPCInstrInfo::commuteInstructionImpl(MachineInstr &MI, bool NewMI,
+MachineInstr *PPCInstrInfo::commuteInstructionImpl(MachineInstr &MI,
+                                                   ProfileSummaryInfo *PSI,
+                                                   const MachineBlockFrequencyInfo *MBFI,
+                                                   bool NewMI,
                                                    unsigned OpIdx1,
                                                    unsigned OpIdx2) const {
   MachineFunction &MF = *MI.getParent()->getParent();
 
   // Normal instructions can be commuted the obvious way.
   if (MI.getOpcode() != PPC::RLWIMI && MI.getOpcode() != PPC::RLWIMIo)
-    return TargetInstrInfo::commuteInstructionImpl(MI, NewMI, OpIdx1, OpIdx2);
+    return TargetInstrInfo::commuteInstructionImpl(MI, PSI, MBFI, NewMI, OpIdx1,
+                                                   OpIdx2);
   // Note that RLWIMI can be commuted as a 32-bit instruction, but not as a
   // 64-bit instruction (so we don't handle PPC::RLWIMI8 here), because
   // changing the relative order of the mask operands might change what happens
   // to the high-bits of the mask (and, thus, the result).
 
   // Cannot commute if it has a non-zero rotate count.
   if (MI.getOperand(3).getImm() != 0)
     return nullptr;

llvm/lib/Target/SystemZ/SystemZInstrInfo.h

protected:
   ///
   /// The arguments 'CommuteOpIdx1' and 'CommuteOpIdx2' specify the operands
   /// to be commuted.
   ///
   /// Do not call this method for a non-commutable instruction or
   /// non-commutable operands.
   /// Even though the instruction is commutable, the method may still
   /// fail to commute the operands, null pointer is returned in such cases.
-  MachineInstr *commuteInstructionImpl(MachineInstr &MI, bool NewMI,
+  MachineInstr *commuteInstructionImpl(MachineInstr &MI,
+                                       ProfileSummaryInfo *PSI,
+                                       const MachineBlockFrequencyInfo *MBFI,
+                                       bool NewMI,
                                        unsigned CommuteOpIdx1,
                                        unsigned CommuteOpIdx2) const override;
 
 public:
   explicit SystemZInstrInfo(SystemZSubtarget &STI);
 
   // Override TargetInstrInfo.
   unsigned isLoadFromStackSlot(const MachineInstr &MI,
void loadRegFromStackSlot(MachineBasicBlock &MBB,
                             const TargetRegisterInfo *TRI) const override;
   MachineInstr *convertToThreeAddress(MachineFunction::iterator &MFI,
                                       MachineInstr &MI,
                                       LiveVariables *LV) const override;
   MachineInstr *
   foldMemoryOperandImpl(MachineFunction &MF, MachineInstr &MI,
                         ArrayRef<unsigned> Ops,
                         MachineBasicBlock::iterator InsertPt, int FrameIndex,
+                        ProfileSummaryInfo *PSI,
+                        const MachineBlockFrequencyInfo *MBFI,
                         LiveIntervals *LIS = nullptr,
                         VirtRegMap *VRM = nullptr) const override;
   MachineInstr *foldMemoryOperandImpl(
       MachineFunction &MF, MachineInstr &MI, ArrayRef<unsigned> Ops,
       MachineBasicBlock::iterator InsertPt, MachineInstr &LoadMI,
+      ProfileSummaryInfo *PSI,
+      const MachineBlockFrequencyInfo *MBFI,
       LiveIntervals *LIS = nullptr) const override;
   bool expandPostRAPseudo(MachineInstr &MBBI) const override;
   bool reverseBranchCondition(SmallVectorImpl<MachineOperand> &Cond) const
     override;
 
   // Return the SystemZRegisterInfo, which this class owns.
   const SystemZRegisterInfo &getRegisterInfo() const { return RI; }
 

llvm/lib/Target/SystemZ/SystemZInstrInfo.cpp

SystemZInstrInfo::emitGRX32Move(MachineBasicBlock &MBB,
   unsigned Rotate = (DestIsHigh != SrcIsHigh ? 32 : 0);
   return BuildMI(MBB, MBBI, DL, get(Opcode), DestReg)
     .addReg(DestReg, RegState::Undef)
     .addReg(SrcReg, getKillRegState(KillSrc) | getUndefRegState(UndefSrc))
     .addImm(32 - Size).addImm(128 + 31).addImm(Rotate);
 }
 
 MachineInstr *SystemZInstrInfo::commuteInstructionImpl(MachineInstr &MI,
+                                                       ProfileSummaryInfo *PSI,
+                                                       const MachineBlockFrequencyInfo *MBFI,
                                                        bool NewMI,
                                                        unsigned OpIdx1,
                                                        unsigned OpIdx2) const {
   auto cloneIfNew = [NewMI](MachineInstr &MI) -> MachineInstr & {
     if (NewMI)
       return *MI.getParent()->getParent()->CloneMachineInstr(&MI);
     return MI;
   };
 
   switch (MI.getOpcode()) {
   case SystemZ::SELRMux:
   case SystemZ::SELFHR:
   case SystemZ::SELR:
   case SystemZ::SELGR:
   case SystemZ::LOCRMux:
   case SystemZ::LOCFHR:
   case SystemZ::LOCR:
   case SystemZ::LOCGR: {
     auto &WorkingMI = cloneIfNew(MI);
     // Invert condition.
     unsigned CCValid = WorkingMI.getOperand(3).getImm();
     unsigned CCMask = WorkingMI.getOperand(4).getImm();
     WorkingMI.getOperand(4).setImm(CCMask ^ CCValid);
-    return TargetInstrInfo::commuteInstructionImpl(WorkingMI, /*NewMI=*/false,
+    return TargetInstrInfo::commuteInstructionImpl(WorkingMI, PSI, MBFI,
+                                                   /*NewMI=*/false,
                                                    OpIdx1, OpIdx2);
   }
   default:
-    return TargetInstrInfo::commuteInstructionImpl(MI, NewMI, OpIdx1, OpIdx2);
+    return TargetInstrInfo::commuteInstructionImpl(MI, PSI, MBFI,
+                                                   NewMI, OpIdx1, OpIdx2);
   }
 }
 
 // If MI is a simple load or store for a frame object, return the register
 // it loads or stores and set FrameIndex to the index of the frame object.
 // Return 0 otherwise.
 //
 // Flag is SimpleBDXLoad for loads and SimpleBDXStore for stores.
case SystemZ::LOCGR:
     else
       return false;
     break;
   default:
     return false;
   }
 
   if (CommuteIdx != -1)
-    if (!commuteInstruction(UseMI, false, CommuteIdx, UseIdx))
+    if (!commuteInstruction(UseMI, nullptr, nullptr, false, CommuteIdx, UseIdx))
       return false;
 
   bool DeleteDef = MRI->hasOneNonDBGUse(Reg);
   UseMI.setDesc(get(NewUseOpc));
   if (TieOps)
     UseMI.tieOperands(0, 1);
   UseMI.getOperand(UseIdx).ChangeToImmediate(ImmVal);
   if (DeleteDef)
if (LogicOp And = interpretAndImmediate(MI.getOpcode())) {
     }
   }
   return nullptr;
 }
 
 MachineInstr *SystemZInstrInfo::foldMemoryOperandImpl(
     MachineFunction &MF, MachineInstr &MI, ArrayRef<unsigned> Ops,
     MachineBasicBlock::iterator InsertPt, int FrameIndex,
+    ProfileSummaryInfo *PSI, const MachineBlockFrequencyInfo *MBFI,
     LiveIntervals *LIS, VirtRegMap *VRM) const {
   const TargetRegisterInfo *TRI = MF.getSubtarget().getRegisterInfo();
   const MachineFrameInfo &MFI = MF.getFrameInfo();
   unsigned Size = MFI.getObjectSize(FrameIndex);
   unsigned Opcode = MI.getOpcode();
 
   if (Ops.size() == 2 && Ops[0] == 0 && Ops[1] == 1) {
     if (LIS != nullptr && (Opcode == SystemZ::LA || Opcode == SystemZ::LAY) &&
MachineInstr *SystemZInstrInfo::foldMemoryOperandImpl(
   }
 
   return nullptr;
 }
 
 MachineInstr *SystemZInstrInfo::foldMemoryOperandImpl(
     MachineFunction &MF, MachineInstr &MI, ArrayRef<unsigned> Ops,
     MachineBasicBlock::iterator InsertPt, MachineInstr &LoadMI,
+    ProfileSummaryInfo *PSI, const MachineBlockFrequencyInfo *MBFI,
     LiveIntervals *LIS) const {
   return nullptr;
 }
 
 bool SystemZInstrInfo::expandPostRAPseudo(MachineInstr &MI) const {
   switch (MI.getOpcode()) {
   case SystemZ::L128:
     splitMove(MI, SystemZ::LG);

llvm/lib/Target/SystemZ/SystemZPostRewrite.cpp

if (DestIsHigh != Src1IsHigh) {
       MBBI->getOperand(2).setReg(DestReg);
       Src2Reg = DestReg;
       Src2IsHigh = DestIsHigh;
     }
   }
 
   // If the destination (now) matches one source, prefer this to be first.
   if (DestReg != Src1Reg && DestReg == Src2Reg) {
-    TII->commuteInstruction(*MBBI, false, 1, 2);
+    TII->commuteInstruction(*MBBI, nullptr, nullptr, false, 1, 2);
     std::swap(Src1Reg, Src2Reg);
     std::swap(Src1IsHigh, Src2IsHigh);
   }
 
   if (!DestIsHigh && !Src1IsHigh && !Src2IsHigh)
     MBBI->setDesc(TII->get(LowOpcode));
   else if (DestIsHigh && Src1IsHigh && Src2IsHigh)
     MBBI->setDesc(TII->get(HighOpcode));
bool SystemZPostRewrite::runOnMachineFunction(MachineFunction &MF) {
   TII = static_cast<const SystemZInstrInfo *>(MF.getSubtarget().getInstrInfo());
 
   bool Modified = false;
   for (auto &MBB : MF)
     Modified |= selectMBB(MBB);
 
   return Modified;
 }
-

llvm/lib/Target/SystemZ/SystemZShortenInst.cpp

 // the destination, convert to the equivalent load-on-condition.
 bool SystemZShortenInst::shortenSelect(MachineInstr &MI, unsigned Opcode) {
   if (MI.getOperand(0).getReg() == MI.getOperand(1).getReg()) {
     MI.setDesc(TII->get(Opcode));
     MI.tieOperands(0, 1);
     return true;
   }
   if (MI.getOperand(0).getReg() == MI.getOperand(2).getReg()) {
-    TII->commuteInstruction(MI, false, 1, 2);
+    TII->commuteInstruction(MI, nullptr, nullptr, false, 1, 2);
     MI.setDesc(TII->get(Opcode));
     MI.tieOperands(0, 1);
     return true;
   }
   return false;
 }
 
 // Process all instructions in MBB.  Return true if something changed.
for (auto MBBI = MBB.rbegin(), MBBE = MBB.rend(); MBBI != MBBE; ++MBBI) {
     default: {
       int TwoOperandOpcode = SystemZ::getTwoOperandOpcode(MI.getOpcode());
       if (TwoOperandOpcode == -1)
         break;
 
       if ((MI.getOperand(0).getReg() != MI.getOperand(1).getReg()) &&
           (!MI.isCommutable() ||
            MI.getOperand(0).getReg() != MI.getOperand(2).getReg() ||
-           !TII->commuteInstruction(MI, false, 1, 2)))
+           !TII->commuteInstruction(MI, nullptr, nullptr, false, 1, 2)))
           break;
 
       MI.setDesc(TII->get(TwoOperandOpcode));
       MI.tieOperands(0, 1);
       if (TwoOperandOpcode == SystemZ::SLL ||
           TwoOperandOpcode == SystemZ::SLA ||
           TwoOperandOpcode == SystemZ::SRL ||
           TwoOperandOpcode == SystemZ::SRA) {

llvm/lib/Target/WebAssembly/WebAssemblyInstrInfo.h

public:
   const WebAssemblyRegisterInfo &getRegisterInfo() const { return RI; }
 
   bool isReallyTriviallyReMaterializable(const MachineInstr &MI,
                                          AAResults *AA) const override;
 
   void copyPhysReg(MachineBasicBlock &MBB, MachineBasicBlock::iterator MI,
                    const DebugLoc &DL, unsigned DestReg, unsigned SrcReg,
                    bool KillSrc) const override;
-  MachineInstr *commuteInstructionImpl(MachineInstr &MI, bool NewMI,
+  MachineInstr *commuteInstructionImpl(MachineInstr &MI,
+                                       ProfileSummaryInfo *PSI,
+                                       const MachineBlockFrequencyInfo *MBFI,
+                                       bool NewMI,
                                        unsigned OpIdx1,
                                        unsigned OpIdx2) const override;
 
   bool analyzeBranch(MachineBasicBlock &MBB, MachineBasicBlock *&TBB,
                      MachineBasicBlock *&FBB,
                      SmallVectorImpl<MachineOperand> &Cond,
                      bool AllowModify = false) const override;
   unsigned removeBranch(MachineBasicBlock &MBB,

llvm/lib/Target/WebAssembly/WebAssemblyInstrInfo.cpp

void WebAssemblyInstrInfo::copyPhysReg(MachineBasicBlock &MBB,
   else
     llvm_unreachable("Unexpected register class");
 
   BuildMI(MBB, I, DL, get(CopyOpcode), DestReg)
       .addReg(SrcReg, KillSrc ? RegState::Kill : 0);
 }
 
 MachineInstr *WebAssemblyInstrInfo::commuteInstructionImpl(
-    MachineInstr &MI, bool NewMI, unsigned OpIdx1, unsigned OpIdx2) const {
+    MachineInstr &MI,
+    ProfileSummaryInfo *PSI, const MachineBlockFrequencyInfo *MBFI,
+    bool NewMI, unsigned OpIdx1, unsigned OpIdx2) const {
   // If the operands are stackified, we can't reorder them.
   WebAssemblyFunctionInfo &MFI =
       *MI.getParent()->getParent()->getInfo<WebAssemblyFunctionInfo>();
   if (MFI.isVRegStackified(MI.getOperand(OpIdx1).getReg()) ||
       MFI.isVRegStackified(MI.getOperand(OpIdx2).getReg()))
     return nullptr;
 
   // Otherwise use the default implementation.
-  return TargetInstrInfo::commuteInstructionImpl(MI, NewMI, OpIdx1, OpIdx2);
+  return TargetInstrInfo::commuteInstructionImpl(MI, PSI, MBFI, NewMI,
+                                                 OpIdx1, OpIdx2);
 }
 
 // Branch analysis.
 bool WebAssemblyInstrInfo::analyzeBranch(MachineBasicBlock &MBB,
                                          MachineBasicBlock *&TBB,
                                          MachineBasicBlock *&FBB,
                                          SmallVectorImpl<MachineOperand> &Cond,
                                          bool /*AllowModify*/) const {

llvm/lib/Target/WebAssembly/WebAssemblyRegStackify.cpp

public:
   /// constraints. If possible, and if we haven't already tried it and declined
   /// it, commute Insert's operands and prepare to revisit it.
   void maybeCommute(MachineInstr *Insert, TreeWalkerState &TreeWalker,
                     const WebAssemblyInstrInfo *TII) {
     if (TentativelyCommuting) {
       assert(!Declined &&
              "Don't decline commuting until you've finished trying it");
       // Commuting didn't help. Revert it.
-      TII->commuteInstruction(*Insert, /*NewMI=*/false, Operand0, Operand1);
+      TII->commuteInstruction(*Insert, nullptr, nullptr, /*NewMI=*/false,
+                              Operand0, Operand1);
       TentativelyCommuting = false;
       Declined = true;
     } else if (!Declined && TreeWalker.hasRemainingOperands(Insert)) {
       Operand0 = TargetInstrInfo::CommuteAnyOperandIndex;
       Operand1 = TargetInstrInfo::CommuteAnyOperandIndex;
       if (TII->findCommutedOpIndices(*Insert, Operand0, Operand1)) {
         // Tentatively commute the operands and try again.
-        TII->commuteInstruction(*Insert, /*NewMI=*/false, Operand0, Operand1);
+        TII->commuteInstruction(*Insert, nullptr, nullptr, /*NewMI=*/false,
+                                Operand0, Operand1);
         TreeWalker.resetTopOperands(Insert);
         TentativelyCommuting = true;
         Declined = false;
       }
     }
   }
 
   /// Stackification for some operand was successful. Reset to the default

llvm/lib/Target/X86/X86FastISel.cpp

bool X86FastISel::tryToFoldLoadIntoMI(MachineInstr *MI, unsigned OpNo,
   if (Alignment == 0)  // Ensure that codegen never sees alignment 0
     Alignment = DL.getABITypeAlignment(LI->getType());
 
   SmallVector<MachineOperand, 8> AddrOps;
   AM.getFullAddress(AddrOps);
 
   MachineInstr *Result = XII.foldMemoryOperandImpl(
       *FuncInfo.MF, *MI, OpNo, AddrOps, FuncInfo.InsertPt, Size, Alignment,
-      /*AllowCommute=*/true);
+      /*AllowCommute=*/true, nullptr, nullptr);
   if (!Result)
     return false;
 
   // The index register could be in the wrong register class.  Unfortunately,
   // foldMemoryOperandImpl could have commuted the instruction so its not enough
   // to just look at OpNo + the offset to the index reg.  We actually need to
   // scan the instruction to find the index reg and see if its the correct reg
   // class.

llvm/lib/Target/X86/X86InstrInfo.h

 #include "llvm/CodeGen/ISDOpcodes.h"
 #include "llvm/CodeGen/TargetInstrInfo.h"
 #include <vector>
 
 #define GET_INSTRINFO_HEADER
 #include "X86GenInstrInfo.inc"
 
 namespace llvm {
+class MachineBlockFrequencyInfo;
 class MachineInstrBuilder;
+class ProfileSummaryInfo;
 class X86RegisterInfo;
 class X86Subtarget;
 
 namespace X86 {
 
 enum AsmComments {
   // For instr that was compressed from EVEX to VEX.
   AC_EVEX_2_VEX = MachineInstr::TAsmComments
public:
   /// specified operand(s).  If this is possible, the target should perform the
   /// folding and return true, otherwise it should return false.  If it folds
   /// the instruction, it is likely that the MachineInstruction the iterator
   /// references has been changed.
   MachineInstr *
   foldMemoryOperandImpl(MachineFunction &MF, MachineInstr &MI,
                         ArrayRef<unsigned> Ops,
                         MachineBasicBlock::iterator InsertPt, int FrameIndex,
+                        ProfileSummaryInfo *PSI,
+                        const MachineBlockFrequencyInfo *MBFI,
                         LiveIntervals *LIS = nullptr,
                         VirtRegMap *VRM = nullptr) const override;
 
   /// foldMemoryOperand - Same as the previous version except it allows folding
   /// of any load and store from / to any address, not just from a specific
   /// stack slot.
   MachineInstr *foldMemoryOperandImpl(
       MachineFunction &MF, MachineInstr &MI, ArrayRef<unsigned> Ops,
       MachineBasicBlock::iterator InsertPt, MachineInstr &LoadMI,
+      ProfileSummaryInfo *PSI, const MachineBlockFrequencyInfo *MBFI,
       LiveIntervals *LIS = nullptr) const override;
 
   /// unfoldMemoryOperand - Separate a single instruction which folded a load or
   /// a store or a load and a store into two or more instruction. If this is
   /// possible, returns true as well as the new instructions by reference.
   bool
   unfoldMemoryOperand(MachineFunction &MF, MachineInstr &MI, unsigned Reg,
                       bool UnfoldLoad, bool UnfoldStore,
public:
   ///
   unsigned getGlobalBaseReg(MachineFunction *MF) const;
 
   std::pair<uint16_t, uint16_t>
   getExecutionDomain(const MachineInstr &MI) const override;
 
   uint16_t getExecutionDomainCustom(const MachineInstr &MI) const;
 
-  void setExecutionDomain(MachineInstr &MI, unsigned Domain) const override;
-
-  bool setExecutionDomainCustom(MachineInstr &MI, unsigned Domain) const;
+  void setExecutionDomain(MachineInstr &MI, unsigned Domain,
+                          ProfileSummaryInfo *PSI,
+                          const MachineBlockFrequencyInfo *MBFI) const override;
+
+  bool setExecutionDomainCustom(MachineInstr &MI, unsigned Domain,
+                                ProfileSummaryInfo *PSI,
+                                const MachineBlockFrequencyInfo *MBFI) const;
 
   unsigned
   getPartialRegUpdateClearance(const MachineInstr &MI, unsigned OpNum,
                                const TargetRegisterInfo *TRI) const override;
   unsigned getUndefRegClearance(const MachineInstr &MI, unsigned &OpNum,
                                 const TargetRegisterInfo *TRI) const override;
   void breakPartialRegDependency(MachineInstr &MI, unsigned OpNum,
                                  const TargetRegisterInfo *TRI) const override;
 
   MachineInstr *foldMemoryOperandImpl(MachineFunction &MF, MachineInstr &MI,
                                       unsigned OpNum,
                                       ArrayRef<MachineOperand> MOs,
                                       MachineBasicBlock::iterator InsertPt,
                                       unsigned Size, unsigned Alignment,
-                                      bool AllowCommute) const;
+                                      bool AllowCommute,
+                                      ProfileSummaryInfo *PSI,
+                                      const MachineBlockFrequencyInfo *MBFI) const;
 
   bool isHighLatencyDef(int opc) const override;
 
   bool hasHighOperandLatency(const TargetSchedModel &SchedModel,
                              const MachineRegisterInfo *MRI,
                              const MachineInstr &DefMI, unsigned DefIdx,
                              const MachineInstr &UseMI,
                              unsigned UseIdx) const override;
public:
   /// def and use are in the same BB. We only look at one load and see
   /// whether it can be folded into MI. FoldAsLoadDefReg is the virtual register
   /// defined by the load we are trying to fold. DefMI returns the machine
   /// instruction that defines FoldAsLoadDefReg, and the function returns
   /// the machine instruction generated due to folding.
   MachineInstr *optimizeLoadInstr(MachineInstr &MI,
                                   const MachineRegisterInfo *MRI,
                                   unsigned &FoldAsLoadDefReg,
-                                  MachineInstr *&DefMI) const override;
+                                  MachineInstr *&DefMI,
+                                  ProfileSummaryInfo *PSI,
+                                  const MachineBlockFrequencyInfo *MBFI) const override;
 
   std::pair<unsigned, unsigned>
   decomposeMachineOperandsTargetFlags(unsigned TF) const override;
 
   ArrayRef<std::pair<unsigned, const char *>>
   getSerializableDirectMachineOperandTargetFlags() const override;
 
   virtual outliner::OutlinedFunction getOutliningCandidateInfo(
protected:
   ///
   /// The arguments 'CommuteOpIdx1' and 'CommuteOpIdx2' specify the operands
   /// to be commuted.
   ///
   /// Do not call this method for a non-commutable instruction or
   /// non-commutable operands.
   /// Even though the instruction is commutable, the method may still
   /// fail to commute the operands, null pointer is returned in such cases.
-  MachineInstr *commuteInstructionImpl(MachineInstr &MI, bool NewMI,
+  MachineInstr *commuteInstructionImpl(MachineInstr &MI,
+                                       ProfileSummaryInfo *PSI,
+                                       const MachineBlockFrequencyInfo *MBFI,
+                                       bool NewMI,
                                        unsigned CommuteOpIdx1,
                                        unsigned CommuteOpIdx2) const override;
 
   /// If the specific machine instruction is a instruction that moves/copies
   /// value from one register to another register return true along with
   /// @Source machine operand and @Destination machine operand.
   bool isCopyInstrImpl(const MachineInstr &MI, const MachineOperand *&Source,
                        const MachineOperand *&Destination) const override;

llvm/lib/Target/X86/X86InstrInfo.cpp

 #include "X86.h"
 #include "X86InstrBuilder.h"
 #include "X86InstrFoldTables.h"
 #include "X86MachineFunctionInfo.h"
 #include "X86Subtarget.h"
 #include "X86TargetMachine.h"
 #include "llvm/ADT/STLExtras.h"
 #include "llvm/ADT/Sequence.h"
+#include "llvm/Analysis/ProfileSummaryInfo.h"
 #include "llvm/CodeGen/LivePhysRegs.h"
 #include "llvm/CodeGen/LiveVariables.h"
+#include "llvm/CodeGen/MachineBlockFrequencyInfo.h"
 #include "llvm/CodeGen/MachineConstantPool.h"
 #include "llvm/CodeGen/MachineDominators.h"
 #include "llvm/CodeGen/MachineFrameInfo.h"
 #include "llvm/CodeGen/MachineInstrBuilder.h"
 #include "llvm/CodeGen/MachineModuleInfo.h"
 #include "llvm/CodeGen/MachineRegisterInfo.h"
+#include "llvm/CodeGen/MachineSizeOpts.h"
 #include "llvm/CodeGen/StackMaps.h"
 #include "llvm/IR/DerivedTypes.h"
 #include "llvm/IR/Function.h"
 #include "llvm/IR/DebugInfoMetadata.h"
 #include "llvm/MC/MCAsmInfo.h"
 #include "llvm/MC/MCExpr.h"
 #include "llvm/MC/MCInst.h"
 #include "llvm/Support/CommandLine.h"
#define VPERM_CASES_BROADCAST(Orig, New) \
   VPERM_CASES(VPERMT2W, VPERMI2W)
   }
 
   llvm_unreachable("Unreachable!");
 #undef VPERM_CASES_BROADCAST
 #undef VPERM_CASES
 }
 
-MachineInstr *X86InstrInfo::commuteInstructionImpl(MachineInstr &MI, bool NewMI,
+MachineInstr *X86InstrInfo::commuteInstructionImpl(MachineInstr &MI,
+                                                   ProfileSummaryInfo *PSI,
+                                                   const MachineBlockFrequencyInfo *MBFI,
+                                                   bool NewMI,
                                                    unsigned OpIdx1,
                                                    unsigned OpIdx2) const {
   auto cloneIfNew = [NewMI](MachineInstr &MI) -> MachineInstr & {
     if (NewMI)
       return *MI.getParent()->getParent()->CloneMachineInstr(&MI);
     return MI;
   };
 
case X86::SHLD64rri8:{// A = SHLD64rri8 B, C, I -> A = SHRD64rri8 C, B, (64-I)
     case X86::SHLD32rri8: Size = 32; Opc = X86::SHRD32rri8; break;
     case X86::SHRD64rri8: Size = 64; Opc = X86::SHLD64rri8; break;
     case X86::SHLD64rri8: Size = 64; Opc = X86::SHRD64rri8; break;
     }
     unsigned Amt = MI.getOperand(3).getImm();
     auto &WorkingMI = cloneIfNew(MI);
     WorkingMI.setDesc(get(Opc));
     WorkingMI.getOperand(3).setImm(Size - Amt);
-    return TargetInstrInfo::commuteInstructionImpl(WorkingMI, /*NewMI=*/false,
+    return TargetInstrInfo::commuteInstructionImpl(WorkingMI, PSI, MBFI,
+                                                   /*NewMI=*/false,
                                                    OpIdx1, OpIdx2);
   }
   case X86::PFSUBrr:
   case X86::PFSUBRrr: {
     // PFSUB  x, y: x = x - y
     // PFSUBR x, y: x = y - x
     unsigned Opc =
         (X86::PFSUBRrr == MI.getOpcode() ? X86::PFSUBrr : X86::PFSUBRrr);
     auto &WorkingMI = cloneIfNew(MI);
     WorkingMI.setDesc(get(Opc));
-    return TargetInstrInfo::commuteInstructionImpl(WorkingMI, /*NewMI=*/false,
+    return TargetInstrInfo::commuteInstructionImpl(WorkingMI, PSI, MBFI,
+                                                   /*NewMI=*/false,
                                                    OpIdx1, OpIdx2);
   }
   case X86::BLENDPDrri:
   case X86::BLENDPSrri:
   case X86::VBLENDPDrri:
-  case X86::VBLENDPSrri:
+  case X86::VBLENDPSrri: {
     // If we're optimizing for size, try to use MOVSD/MOVSS.
-    if (MI.getParent()->getParent()->getFunction().hasOptSize()) {
+    auto *MBB = MI.getParent();
+    auto MF = MBB->getParent();
+    bool OptForSize = MF->getFunction().hasOptSize() ||
+                      llvm::shouldOptimizeForSize(MBB, PSI, MBFI);
+    if (OptForSize) {
       unsigned Mask, Opc;
       switch (MI.getOpcode()) {
       default: llvm_unreachable("Unreachable!");
       case X86::BLENDPDrri:  Opc = X86::MOVSDrr;  Mask = 0x03; break;
       case X86::BLENDPSrri:  Opc = X86::MOVSSrr;  Mask = 0x0F; break;
       case X86::VBLENDPDrri: Opc = X86::VMOVSDrr; Mask = 0x03; break;
       case X86::VBLENDPSrri: Opc = X86::VMOVSSrr; Mask = 0x0F; break;
       }
       if ((MI.getOperand(3).getImm() ^ Mask) == 1) {
         auto &WorkingMI = cloneIfNew(MI);
         WorkingMI.setDesc(get(Opc));
         WorkingMI.RemoveOperand(3);
         return TargetInstrInfo::commuteInstructionImpl(WorkingMI,
+                                                       PSI, MBFI,
                                                        /*NewMI=*/false,
                                                        OpIdx1, OpIdx2);
       }
     }
     LLVM_FALLTHROUGH;
+  }
   case X86::PBLENDWrri:
   case X86::VBLENDPDYrri:
   case X86::VBLENDPSYrri:
   case X86::VPBLENDDrri:
   case X86::VPBLENDWrri:
   case X86::VPBLENDDYrri:
   case X86::VPBLENDWYrri:{
     int8_t Mask;
case X86::VPBLENDWYrri:{
     case X86::VPBLENDWYrri:  Mask = (int8_t)0xFF; break;
     }
     // Only the least significant bits of Imm are used.
     // Using int8_t to ensure it will be sign extended to the int64_t that
     // setImm takes in order to match isel behavior.
     int8_t Imm = MI.getOperand(3).getImm() & Mask;
     auto &WorkingMI = cloneIfNew(MI);
     WorkingMI.getOperand(3).setImm(Mask ^ Imm);
-    return TargetInstrInfo::commuteInstructionImpl(WorkingMI, /*NewMI=*/false,
+    return TargetInstrInfo::commuteInstructionImpl(WorkingMI, PSI, MBFI,
+                                                   /*NewMI=*/false,
                                                    OpIdx1, OpIdx2);
   }
   case X86::INSERTPSrr:
   case X86::VINSERTPSrr:
   case X86::VINSERTPSZrr: {
     unsigned Imm = MI.getOperand(MI.getNumOperands() - 1).getImm();
     unsigned ZMask = Imm & 15;
     unsigned DstIdx = (Imm >> 4) & 3;
     unsigned SrcIdx = (Imm >> 6) & 3;
 
     // We can commute insertps if we zero 2 of the elements, the insertion is
     // "inline" and we don't override the insertion with a zero.
     if (DstIdx == SrcIdx && (ZMask & (1 << DstIdx)) == 0 &&
         countPopulation(ZMask) == 2) {
       unsigned AltIdx = findFirstSet((ZMask | (1 << DstIdx)) ^ 15);
       assert(AltIdx < 4 && "Illegal insertion index");
       unsigned AltImm = (AltIdx << 6) | (AltIdx << 4) | ZMask;
       auto &WorkingMI = cloneIfNew(MI);
       WorkingMI.getOperand(MI.getNumOperands() - 1).setImm(AltImm);
-      return TargetInstrInfo::commuteInstructionImpl(WorkingMI, /*NewMI=*/false,
+      return TargetInstrInfo::commuteInstructionImpl(WorkingMI, PSI, MBFI,
+                                                     /*NewMI=*/false,
                                                      OpIdx1, OpIdx2);
     }
     return nullptr;
   }
   case X86::MOVSDrr:
   case X86::MOVSSrr:
   case X86::VMOVSDrr:
   case X86::VMOVSSrr:{
     // On SSE41 or later we can commute a MOVSS/MOVSD to a BLENDPS/BLENDPD.
     if (Subtarget.hasSSE41()) {
       unsigned Mask, Opc;
       switch (MI.getOpcode()) {
       default: llvm_unreachable("Unreachable!");
       case X86::MOVSDrr:  Opc = X86::BLENDPDrri;  Mask = 0x02; break;
       case X86::MOVSSrr:  Opc = X86::BLENDPSrri;  Mask = 0x0E; break;
       case X86::VMOVSDrr: Opc = X86::VBLENDPDrri; Mask = 0x02; break;
       case X86::VMOVSSrr: Opc = X86::VBLENDPSrri; Mask = 0x0E; break;
       }
 
       auto &WorkingMI = cloneIfNew(MI);
       WorkingMI.setDesc(get(Opc));
       WorkingMI.addOperand(MachineOperand::CreateImm(Mask));
-      return TargetInstrInfo::commuteInstructionImpl(WorkingMI, /*NewMI=*/false,
+      return TargetInstrInfo::commuteInstructionImpl(WorkingMI, PSI, MBFI,
+                                                     /*NewMI=*/false,
                                                      OpIdx1, OpIdx2);
     }
 
     // Convert to SHUFPD.
     assert(MI.getOpcode() == X86::MOVSDrr &&
            "Can only commute MOVSDrr without SSE4.1");
 
     auto &WorkingMI = cloneIfNew(MI);
     WorkingMI.setDesc(get(X86::SHUFPDrri));
     WorkingMI.addOperand(MachineOperand::CreateImm(0x02));
-    return TargetInstrInfo::commuteInstructionImpl(WorkingMI, /*NewMI=*/false,
+    return TargetInstrInfo::commuteInstructionImpl(WorkingMI, PSI, MBFI,
+                                                   /*NewMI=*/false,
                                                    OpIdx1, OpIdx2);
   }
   case X86::SHUFPDrri: {
     // Commute to MOVSD.
     assert(MI.getOperand(3).getImm() == 0x02 && "Unexpected immediate!");
     auto &WorkingMI = cloneIfNew(MI);
     WorkingMI.setDesc(get(X86::MOVSDrr));
     WorkingMI.RemoveOperand(3);
-    return TargetInstrInfo::commuteInstructionImpl(WorkingMI, /*NewMI=*/false,
+    return TargetInstrInfo::commuteInstructionImpl(WorkingMI, PSI, MBFI,
+                                                   /*NewMI=*/false,
                                                    OpIdx1, OpIdx2);
   }
   case X86::PCLMULQDQrr:
   case X86::VPCLMULQDQrr:
   case X86::VPCLMULQDQYrr:
   case X86::VPCLMULQDQZrr:
   case X86::VPCLMULQDQZ128rr:
   case X86::VPCLMULQDQZ256rr: {
     // SRC1 64bits = Imm[0] ? SRC1[127:64] : SRC1[63:0]
     // SRC2 64bits = Imm[4] ? SRC2[127:64] : SRC2[63:0]
     unsigned Imm = MI.getOperand(3).getImm();
     unsigned Src1Hi = Imm & 0x01;
     unsigned Src2Hi = Imm & 0x10;
     auto &WorkingMI = cloneIfNew(MI);
     WorkingMI.getOperand(3).setImm((Src1Hi << 4) | (Src2Hi >> 4));
-    return TargetInstrInfo::commuteInstructionImpl(WorkingMI, /*NewMI=*/false,
+    return TargetInstrInfo::commuteInstructionImpl(WorkingMI, PSI, MBFI,
+                                                   /*NewMI=*/false,
                                                    OpIdx1, OpIdx2);
   }
   case X86::VPCMPBZ128rri:  case X86::VPCMPUBZ128rri:
   case X86::VPCMPBZ256rri:  case X86::VPCMPUBZ256rri:
   case X86::VPCMPBZrri:     case X86::VPCMPUBZrri:
   case X86::VPCMPDZ128rri:  case X86::VPCMPUDZ128rri:
   case X86::VPCMPDZ256rri:  case X86::VPCMPUDZ256rri:
   case X86::VPCMPDZrri:     case X86::VPCMPUDZrri:
MachineInstr *X86InstrInfo::commuteInstructionImpl(MachineInstr &MI,
   case X86::VPCMPWZ128rrik: case X86::VPCMPUWZ128rrik:
   case X86::VPCMPWZ256rrik: case X86::VPCMPUWZ256rrik:
   case X86::VPCMPWZrrik:    case X86::VPCMPUWZrrik: {
     // Flip comparison mode immediate (if necessary).
     unsigned Imm = MI.getOperand(MI.getNumOperands() - 1).getImm() & 0x7;
     Imm = X86::getSwappedVPCMPImm(Imm);
     auto &WorkingMI = cloneIfNew(MI);
     WorkingMI.getOperand(MI.getNumOperands() - 1).setImm(Imm);
-    return TargetInstrInfo::commuteInstructionImpl(WorkingMI, /*NewMI=*/false,
+    return TargetInstrInfo::commuteInstructionImpl(WorkingMI, PSI, MBFI,
+                                                   /*NewMI=*/false,
                                                    OpIdx1, OpIdx2);
   }
   case X86::VPCOMBri: case X86::VPCOMUBri:
   case X86::VPCOMDri: case X86::VPCOMUDri:
   case X86::VPCOMQri: case X86::VPCOMUQri:
   case X86::VPCOMWri: case X86::VPCOMUWri: {
     // Flip comparison mode immediate (if necessary).
     unsigned Imm = MI.getOperand(3).getImm() & 0x7;
     Imm = X86::getSwappedVPCOMImm(Imm);
     auto &WorkingMI = cloneIfNew(MI);
     WorkingMI.getOperand(3).setImm(Imm);
-    return TargetInstrInfo::commuteInstructionImpl(WorkingMI, /*NewMI=*/false,
+    return TargetInstrInfo::commuteInstructionImpl(WorkingMI, PSI, MBFI,
+                                                   /*NewMI=*/false,
                                                    OpIdx1, OpIdx2);
   }
   case X86::VCMPSDZrr:
   case X86::VCMPSSZrr:
   case X86::VCMPPDZrri:
   case X86::VCMPPSZrri:
   case X86::VCMPPDZ128rri:
   case X86::VCMPPSZ128rri:
   case X86::VCMPPDZ256rri:
   case X86::VCMPPSZ256rri:
   case X86::VCMPPDZrrik:
   case X86::VCMPPSZrrik:
   case X86::VCMPPDZ128rrik:
   case X86::VCMPPSZ128rrik:
   case X86::VCMPPDZ256rrik:
   case X86::VCMPPSZ256rrik: {
     unsigned Imm = MI.getOperand(MI.getNumOperands() - 1).getImm() & 0x1f;
     Imm = X86::getSwappedVCMPImm(Imm);
     auto &WorkingMI = cloneIfNew(MI);
     WorkingMI.getOperand(MI.getNumOperands() - 1).setImm(Imm);
-    return TargetInstrInfo::commuteInstructionImpl(WorkingMI, /*NewMI=*/false,
+    return TargetInstrInfo::commuteInstructionImpl(WorkingMI, PSI, MBFI,
+                                                   /*NewMI=*/false,
                                                    OpIdx1, OpIdx2);
   }
   case X86::VPERM2F128rr:
   case X86::VPERM2I128rr: {
     // Flip permute source immediate.
     // Imm & 0x02: lo = if set, select Op1.lo/hi else Op0.lo/hi.
     // Imm & 0x20: hi = if set, select Op1.lo/hi else Op0.lo/hi.
     int8_t Imm = MI.getOperand(3).getImm() & 0xFF;
     auto &WorkingMI = cloneIfNew(MI);
     WorkingMI.getOperand(3).setImm(Imm ^ 0x22);
-    return TargetInstrInfo::commuteInstructionImpl(WorkingMI, /*NewMI=*/false,
+    return TargetInstrInfo::commuteInstructionImpl(WorkingMI, PSI, MBFI,
+                                                   /*NewMI=*/false,
                                                    OpIdx1, OpIdx2);
   }
   case X86::MOVHLPSrr:
   case X86::UNPCKHPDrr:
   case X86::VMOVHLPSrr:
   case X86::VUNPCKHPDrr:
   case X86::VMOVHLPSZrr:
   case X86::VUNPCKHPDZ128rr: {
     assert(Subtarget.hasSSE2() && "Commuting MOVHLP/UNPCKHPD requires SSE2!");
 
     unsigned Opc = MI.getOpcode();
     switch (Opc) {
     default: llvm_unreachable("Unreachable!");
     case X86::MOVHLPSrr:       Opc = X86::UNPCKHPDrr;      break;
     case X86::UNPCKHPDrr:      Opc = X86::MOVHLPSrr;       break;
     case X86::VMOVHLPSrr:      Opc = X86::VUNPCKHPDrr;     break;
     case X86::VUNPCKHPDrr:     Opc = X86::VMOVHLPSrr;      break;
     case X86::VMOVHLPSZrr:     Opc = X86::VUNPCKHPDZ128rr; break;
     case X86::VUNPCKHPDZ128rr: Opc = X86::VMOVHLPSZrr;     break;
     }
     auto &WorkingMI = cloneIfNew(MI);
     WorkingMI.setDesc(get(Opc));
-    return TargetInstrInfo::commuteInstructionImpl(WorkingMI, /*NewMI=*/false,
+    return TargetInstrInfo::commuteInstructionImpl(WorkingMI, PSI, MBFI,
+                                                   /*NewMI=*/false,
                                                    OpIdx1, OpIdx2);
   }
   case X86::CMOV16rr:  case X86::CMOV32rr:  case X86::CMOV64rr: {
     auto &WorkingMI = cloneIfNew(MI);
     unsigned OpNo = MI.getDesc().getNumOperands() - 1;
     X86::CondCode CC = static_cast<X86::CondCode>(MI.getOperand(OpNo).getImm());
     WorkingMI.getOperand(OpNo).setImm(X86::GetOppositeBranchCondition(CC));
-    return TargetInstrInfo::commuteInstructionImpl(WorkingMI, /*NewMI=*/false,
+    return TargetInstrInfo::commuteInstructionImpl(WorkingMI, PSI, MBFI,
+                                                   /*NewMI=*/false,
                                                    OpIdx1, OpIdx2);
   }
   case X86::VPTERNLOGDZrri:      case X86::VPTERNLOGDZrmi:
   case X86::VPTERNLOGDZ128rri:   case X86::VPTERNLOGDZ128rmi:
   case X86::VPTERNLOGDZ256rri:   case X86::VPTERNLOGDZ256rmi:
   case X86::VPTERNLOGQZrri:      case X86::VPTERNLOGQZrmi:
   case X86::VPTERNLOGQZ128rri:   case X86::VPTERNLOGQZ128rmi:
   case X86::VPTERNLOGQZ256rri:   case X86::VPTERNLOGQZ256rmi:
MachineInstr *X86InstrInfo::commuteInstructionImpl(MachineInstr &MI,
   case X86::VPTERNLOGDZ128rmbikz:
   case X86::VPTERNLOGDZ256rmbikz:
   case X86::VPTERNLOGDZrmbikz:
   case X86::VPTERNLOGQZ128rmbikz:
   case X86::VPTERNLOGQZ256rmbikz:
   case X86::VPTERNLOGQZrmbikz: {
     auto &WorkingMI = cloneIfNew(MI);
     commuteVPTERNLOG(WorkingMI, OpIdx1, OpIdx2);
-    return TargetInstrInfo::commuteInstructionImpl(WorkingMI, /*NewMI=*/false,
+    return TargetInstrInfo::commuteInstructionImpl(WorkingMI, PSI, MBFI,
+                                                   /*NewMI=*/false,
                                                    OpIdx1, OpIdx2);
   }
   default: {
     if (isCommutableVPERMV3Instruction(MI.getOpcode())) {
       unsigned Opc = getCommutedVPERMV3Opcode(MI.getOpcode());
       auto &WorkingMI = cloneIfNew(MI);
       WorkingMI.setDesc(get(Opc));
-      return TargetInstrInfo::commuteInstructionImpl(WorkingMI, /*NewMI=*/false,
+      return TargetInstrInfo::commuteInstructionImpl(WorkingMI, PSI, MBFI,
+                                                     /*NewMI=*/false,
                                                      OpIdx1, OpIdx2);
     }
 
     const X86InstrFMA3Group *FMA3Group = getFMA3Group(MI.getOpcode(),
                                                       MI.getDesc().TSFlags);
     if (FMA3Group) {
       unsigned Opc =
         getFMA3OpcodeToCommuteOperands(MI, OpIdx1, OpIdx2, *FMA3Group);
       auto &WorkingMI = cloneIfNew(MI);
       WorkingMI.setDesc(get(Opc));
-      return TargetInstrInfo::commuteInstructionImpl(WorkingMI, /*NewMI=*/false,
+      return TargetInstrInfo::commuteInstructionImpl(WorkingMI, PSI, MBFI,
+                                                     /*NewMI=*/false,
                                                      OpIdx1, OpIdx2);
     }
 
-    return TargetInstrInfo::commuteInstructionImpl(MI, NewMI, OpIdx1, OpIdx2);
+    return TargetInstrInfo::commuteInstructionImpl(MI, PSI, MBFI,
+                                                   NewMI, OpIdx1, OpIdx2);
   }
   }
 }
 
 bool
 X86InstrInfo::findThreeSrcCommutedOpIndices(const MachineInstr &MI,
                                             unsigned &SrcOpIdx1,
                                             unsigned &SrcOpIdx2,

 
 /// Try to remove the load by folding it to a register
 /// operand at the use. We fold the load instructions if load defines a virtual
 /// register, the virtual register is used once in the same BB, and the
 /// instructions in-between do not load or store, and have no side effects.
 MachineInstr *X86InstrInfo::optimizeLoadInstr(MachineInstr &MI,
                                               const MachineRegisterInfo *MRI,
                                               unsigned &FoldAsLoadDefReg,
-                                              MachineInstr *&DefMI) const {
+                                              MachineInstr *&DefMI,
+                                              ProfileSummaryInfo *PSI,
+                                              const MachineBlockFrequencyInfo *MBFI) const {
   // Check whether we can move DefMI here.
   DefMI = MRI->getVRegDef(FoldAsLoadDefReg);
   assert(DefMI);
   bool SawStore = false;
   if (!DefMI->isSafeToMove(nullptr, SawStore))
     return nullptr;
 
   // Collect information about virtual register operands of MI.
for (unsigned i = 0, e = MI.getNumOperands(); i != e; ++i) {
     if (MO.getSubReg() || MO.isDef())
       return nullptr;
     SrcOperandIds.push_back(i);
   }
   if (SrcOperandIds.empty())
     return nullptr;
 
   // Check whether we can fold the def into SrcOperandId.
-  if (MachineInstr *FoldMI = foldMemoryOperand(MI, SrcOperandIds, *DefMI)) {
+  if (MachineInstr *FoldMI = foldMemoryOperand(MI, SrcOperandIds, *DefMI,
+                                               PSI, MBFI)) {
     FoldAsLoadDefReg = 0;
     return FoldMI;
   }
 
   return nullptr;
 }
 
 /// Expand a single-def pseudo instruction to a two-addr
static bool shouldPreventUndefRegUpdateMemFold(MachineFunction &MF,
   MachineInstr *VRegDef = RegInfo.getUniqueVRegDef(MI.getOperand(1).getReg());
   return VRegDef && VRegDef->isImplicitDef();
 }
 
 
 MachineInstr *X86InstrInfo::foldMemoryOperandImpl(
     MachineFunction &MF, MachineInstr &MI, unsigned OpNum,
     ArrayRef<MachineOperand> MOs, MachineBasicBlock::iterator InsertPt,
-    unsigned Size, unsigned Align, bool AllowCommute) const {
+    unsigned Size, unsigned Align, bool AllowCommute,
+    ProfileSummaryInfo *PSI, const MachineBlockFrequencyInfo *MBFI) const {
   bool isSlowTwoMemOps = Subtarget.slowTwoMemOps();
   bool isTwoAddrFold = false;
 
   // For CPUs that favor the register form of a call or push,
   // do not fold loads into calls or pushes, unless optimizing for size
   // aggressively.
   if (isSlowTwoMemOps && !MF.getFunction().hasMinSize() &&
       (MI.getOpcode() == X86::CALL32r || MI.getOpcode() == X86::CALL64r ||
if (findCommutedOpIndices(MI, CommuteOpIdx1, CommuteOpIdx2)) {
 
       // If either of the commutable operands are tied to the destination
       // then we can not commute + fold.
       if ((HasDef && Reg0 == Reg1 && Tied1) ||
           (HasDef && Reg0 == Reg2 && Tied2))
         return nullptr;
 
       MachineInstr *CommutedMI =
-          commuteInstruction(MI, false, CommuteOpIdx1, CommuteOpIdx2);
+          commuteInstruction(MI, PSI, MBFI, false, CommuteOpIdx1, CommuteOpIdx2);
       if (!CommutedMI) {
         // Unable to commute.
         return nullptr;
       }
       if (CommutedMI != &MI) {
         // New instruction. We can't fold from this.
         CommutedMI->eraseFromParent();
         return nullptr;
       }
 
       // Attempt to fold with the commuted version of the instruction.
       NewMI = foldMemoryOperandImpl(MF, MI, CommuteOpIdx2, MOs, InsertPt,
-                                    Size, Align, /*AllowCommute=*/false);
+                                    Size, Align, /*AllowCommute=*/false,
+                                    PSI, MBFI);
       if (NewMI)
         return NewMI;
 
       // Folding failed again - undo the commute before returning.
       MachineInstr *UncommutedMI =
-          commuteInstruction(MI, false, CommuteOpIdx1, CommuteOpIdx2);
+          commuteInstruction(MI, PSI, MBFI, false, CommuteOpIdx1, CommuteOpIdx2);
       if (!UncommutedMI) {
         // Unable to commute.
         return nullptr;
       }
       if (UncommutedMI != &MI) {
         // New instruction. It doesn't need to be kept.
         UncommutedMI->eraseFromParent();
         return nullptr;
if (PrintFailedFusing && !MI.isCopy())
     dbgs() << "We failed to fuse operand " << OpNum << " in " << MI;
   return nullptr;
 }
 
 MachineInstr *
 X86InstrInfo::foldMemoryOperandImpl(MachineFunction &MF, MachineInstr &MI,
                                     ArrayRef<unsigned> Ops,
                                     MachineBasicBlock::iterator InsertPt,
-                                    int FrameIndex, LiveIntervals *LIS,
+                                    int FrameIndex,
+                                    ProfileSummaryInfo *PSI,
+                                    const MachineBlockFrequencyInfo *MBFI,
+                                    LiveIntervals *LIS,
                                     VirtRegMap *VRM) const {
   // Check switch flag
   if (NoFusing)
     return nullptr;
 
   // Avoid partial and undef register update stalls unless optimizing for size.
   if (!MF.getFunction().hasOptSize() &&
       (hasPartialRegUpdate(MI.getOpcode(), Subtarget, /*ForLoadFold*/true) ||
if (Ops.size() == 2 && Ops[0] == 0 && Ops[1] == 1) {
     // Change to CMPXXri r, 0 first.
     MI.setDesc(get(NewOpc));
     MI.getOperand(1).ChangeToImmediate(0);
   } else if (Ops.size() != 1)
     return nullptr;
 
   return foldMemoryOperandImpl(MF, MI, Ops[0],
                                MachineOperand::CreateFI(FrameIndex), InsertPt,
-                               Size, Alignment, /*AllowCommute=*/true);
+                               Size, Alignment, /*AllowCommute=*/true,
+                               PSI, MBFI);
 }
 
 /// Check if \p LoadMI is a partial register load that we can't fold into \p MI
 /// because the latter uses contents that wouldn't be defined in the folded
 /// version.  For instance, this transformation isn't legal:
 ///   movss (%rdi), %xmm0
 ///   addps %xmm0, %xmm0
 /// ->
static bool isNonFoldablePartialRegisterLoad(const MachineInstr &LoadMI,
   }
 
   return false;
 }
 
 MachineInstr *X86InstrInfo::foldMemoryOperandImpl(
     MachineFunction &MF, MachineInstr &MI, ArrayRef<unsigned> Ops,
     MachineBasicBlock::iterator InsertPt, MachineInstr &LoadMI,
+    ProfileSummaryInfo *PSI, const MachineBlockFrequencyInfo *MBFI,
     LiveIntervals *LIS) const {
 
   // TODO: Support the case where LoadMI loads a wide register, but MI
   // only uses a subreg.
   for (auto Op : Ops) {
     if (MI.getOperand(Op).getSubReg())
       return nullptr;
   }
 
   // If loading from a FrameIndex, fold directly from the FrameIndex.
   unsigned NumOps = LoadMI.getDesc().getNumOperands();
   int FrameIndex;
   if (isLoadFromStackSlot(LoadMI, FrameIndex)) {
     if (isNonFoldablePartialRegisterLoad(LoadMI, MI, MF))
       return nullptr;
-    return foldMemoryOperandImpl(MF, MI, Ops, InsertPt, FrameIndex, LIS);
+    return foldMemoryOperandImpl(MF, MI, Ops, InsertPt, FrameIndex, PSI, MBFI,
+                                 LIS);
   }
 
   // Check switch flag
   if (NoFusing) return nullptr;
 
   // Avoid partial and undef register update stalls unless optimizing for size.
   if (!MF.getFunction().hasOptSize() &&
       (hasPartialRegUpdate(MI.getOpcode(), Subtarget, /*ForLoadFold*/true) ||
default: {
 
     // Folding a normal load. Just copy the load's address operands.
     MOs.append(LoadMI.operands_begin() + NumOps - X86::AddrNumOperands,
                LoadMI.operands_begin() + NumOps);
     break;
   }
   }
   return foldMemoryOperandImpl(MF, MI, Ops[0], MOs, InsertPt,
-                               /*Size=*/0, Alignment, /*AllowCommute=*/true);
+                               /*Size=*/0, Alignment, /*AllowCommute=*/true,
+                               PSI, MBFI);
 }
 
 static SmallVector<MachineMemOperand *, 2>
 extractLoadMMOs(ArrayRef<MachineMemOperand *> MMOs, MachineFunction &MF) {
   SmallVector<MachineMemOperand *, 2> LoadMMOs;
 
   for (MachineMemOperand *MMO : MMOs) {
     if (!MMO->isLoad())
case X86::MOVHLPSrr:
     return 0;
   case X86::SHUFPDrri:
     return 0x6;
   }
   return 0;
 }
 
 bool X86InstrInfo::setExecutionDomainCustom(MachineInstr &MI,
-                                            unsigned Domain) const {
+                                            unsigned Domain,
+                                            ProfileSummaryInfo *PSI,
+                                            const MachineBlockFrequencyInfo *MBFI) const {
   assert(Domain > 0 && Domain < 4 && "Invalid execution domain");
   uint16_t dom = (MI.getDesc().TSFlags >> X86II::SSEDomainShift) & 3;
   assert(dom && "Not an SSE instruction");
 
   unsigned Opcode = MI.getOpcode();
   unsigned NumOperands = MI.getDesc().getNumOperands();
 
   auto SetBlendDomain = [&](unsigned ImmWidth, bool Is256) {
bool X86InstrInfo::setExecutionDomainCustom(MachineInstr &MI,
   case X86::UNPCKHPDrr:
   case X86::MOVHLPSrr:
     // We just need to commute the instruction which will switch the domains.
     if (Domain != dom && Domain != 3 &&
         MI.getOperand(1).getReg() == MI.getOperand(2).getReg() &&
         MI.getOperand(0).getSubReg() == 0 &&
         MI.getOperand(1).getSubReg() == 0 &&
         MI.getOperand(2).getSubReg() == 0) {
-      commuteInstruction(MI, false);
+      commuteInstruction(MI, PSI, MBFI, false);
       return true;
     }
     // We must always return true for MOVHLPSrr.
     if (Opcode == X86::MOVHLPSrr)
       return true;
     break;
   case X86::SHUFPDrri: {
     if (Domain == 1) {
if (lookup(opcode, domain, ReplaceableInstrs)) {
         else
           validDomains = 0xc;
       }
     }
   }
   return std::make_pair(domain, validDomains);
 }
 
-void X86InstrInfo::setExecutionDomain(MachineInstr &MI, unsigned Domain) const {
+void X86InstrInfo::setExecutionDomain(MachineInstr &MI, unsigned Domain,
+                                      ProfileSummaryInfo *PSI,
+                                      const MachineBlockFrequencyInfo *MBFI) const {
   assert(Domain>0 && Domain<4 && "Invalid execution domain");
   uint16_t dom = (MI.getDesc().TSFlags >> X86II::SSEDomainShift) & 3;
   assert(dom && "Not an SSE instruction");
 
   // Attempt to match for custom instructions.
-  if (setExecutionDomainCustom(MI, Domain))
+  if (setExecutionDomainCustom(MI, Domain, PSI, MBFI))
     return;
 
   const uint16_t *table = lookup(MI.getOpcode(), dom, ReplaceableInstrs);
   if (!table) { // try the other table
     assert((Subtarget.hasAVX2() || Domain < 3) &&
            "256-bit vector operations only available in AVX2");
     table = lookup(MI.getOpcode(), dom, ReplaceableInstrsAVX2);
   }