Add functions to MachineLICM to hoist invariant stores.

[syzaara]

Summary

This patch adds functions to allow MachineLICM to hoist invariant stores. Currently, MachineLICM does not hoist any store instructions, however when storing the same value to a constant spot on the stack, the store instruction should be considered invariant and be hoisted.
Such stores are common with indirect calls on PowerPC where we save the TOC pointer in R2 to a constant spot on the stack. R2 is constant throughout the function.

The function isInvariantStore iterates each operand of the store instruction and checks that each register operand satisfies isCallerPreservedPhysReg.

These store instructions are fed by a copy:
%vreg12<def> = COPY %X2; G8RC:%vreg12
STD %vreg12, 24, %X1; mem:ST8[Stack+24] G8RC:%vreg12

When hoisting the store, the copy should be hoisted as well. This is handled with isCopyFeedingInvariantStore.

For more details, please see RFC: http://lists.llvm.org/pipermail/llvm-dev/2017-November/119116.html

[nemanjai]

Hoisting these invariant stores is certainly something we want on PPC. It makes sense that MachineLICM is the place to do it. I've added a few people that I think are familiar with the pass to help with the review.

lib/CodeGen/MachineLICM.cpp

717

Looks like this is just

// If we have hoisted an instruction that may store, it can only be a
// constant store.
else if (MI->mayStore())
  NumStoreConst++;

lib/CodeGen/TargetInstrInfo.cpp

912

It should be safe to not have this condition (and perhaps just turn it into an assert).

913

Why not for (MachineInstr &UseMI : use_instructions(CopyDstReg))?

[qcolombet]

include/llvm/CodeGen/TargetInstrInfo.h

1141

Instead of patching every single optimization with this new hook, could you teach isSafeToMove this concept of invariant store?

1144

Why does this need to be a TII hook?

[MatzeB]

Didn't we already add TargetRegisterInfo::isCallerPreservedPhysReg() specifically for this PPC issue?

include/llvm/CodeGen/TargetInstrInfo.h

1141

This should also have more detailed explanations! Can you really move the store anywhere in the function? (As an example: Would in front of the prologue or before a return be still okay? I suspect not.)

[syzaara]

include/llvm/CodeGen/TargetInstrInfo.h

1141

There are other optimizations that use isSafeToMove where it does not make sense to teach it about invariant stores. For example, DeadMachineInstructionElim.cpp will end up deleting the store instruction if isSafeToMove was modified. We want to restrict what we do with these invariant stores to MachineLICM.

[syzaara]

In D40196#939508, @MatzeB wrote:

Didn't we already add TargetRegisterInfo::isCallerPreservedPhysReg() specifically for this PPC issue?

isCallerPreservedPhysReg was added to allow hoisting of load instructions whose operands were preserved physical regs.
In this case, we are trying to hoist a store instruction, which currently is not supported by MachineLICM. MachineLICM considers all stores unsafe to hoist.
This teaches MachineLICM to allow hoisting of invariant stores when all the operands of the store satisfy isCallerPreservedPhysReg.

[lei]

lib/CodeGen/MachineLICM.cpp

956

nit: since HoistConstStores is a boolean controlled by option, I'd put this as the first ck in this && cond.

1176

I don't think you need to ck for MI.isCopy() here since isCopyFeedingInvariantStore() checks for this and does an early exit if it's false.

lib/Target/PowerPC/PPCInstrInfo.cpp

306

Please add documentation for the following 2 functions.

334

why not use: for (const MachineOperand &MO : MI.operands())

lib/Target/PowerPC/PPCRegisterInfo.cpp

308–314

Need more explanation as to why X1 is caller preserved.

[hfinkel]

include/llvm/CodeGen/TargetInstrInfo.h

1143

Document here whether or not it is legal to call this on instructions that aren't stores.

lib/CodeGen/MachineLICM.cpp

956

If we decide to move the mayStore check into isInvariantStore, then remove it here.

1164

This really seems to represent the way that the PPC implementation of isInvariantStore works. What if other targets look through things that aren't pure copies? Your PPC implementation of isInvariantStore could be target independent. I'd just move it here to be next to this implementation. (This won't affect any other targets anyway, because only PPC currently implements isCallerPreservedPhysReg).

lib/Target/PowerPC/PPCInstrInfo.cpp

312

This is going to be added to PPCInstInfo in D31852, don't add it twice :-)

339

Add something here to validate that we've been called on an instruction that might store?

assert(MI->mayStore() && "isInvariantStore not called on a store instruction");

or put the check here:

if (!MI->mayStore())
  return false;

depending on the contract we want.

lib/Target/PowerPC/PPCRegisterInfo.cpp

69

I don't see any test cases for this. Can you add one?

296

This applies to ELFv1 too, please don't restrict this to ELFv2.

[syzaara]

lib/Target/PowerPC/PPCRegisterInfo.cpp

69

The testcase added test/CodeGen/PowerPC/loop-hoist-toc-save.ll tests for -ppc-stack-ptr-caller-preserved in combination with hoist-const-stores.

[hfinkel]

LGTM

[syzaara]

@echristo Hi Eric, do you have any comments regarding this patch?

[echristo]

A few comments inline.

include/llvm/CodeGen/TargetInstrInfo.h

303

This API is described a little awkwardly and you can see it from the comment - "... for a Machine instruction" and the API doesn't take an MI, but just a register and MRI.

Maybe instead it should be on TRI?

lib/CodeGen/MachineLICM.cpp

894–897

I understand the desire to make these functions static, but you should probably either pass in the information you need into them or make them class functions. There's no need to look up all of this information in a loop for sure when we have it cached in the class.

926–929

Should possibly just be a class function or take the variables it needs as arguments rather than grabbing them.

lib/CodeGen/TargetInstrInfo.cpp

301

const?

[syzaara]

@echristo Hi Eric, are you okay to approve this now?

[echristo]

In D40196#1005679, @syzaara wrote:

@echristo Hi Eric, are you okay to approve this now?

You didn't answer any of my questions?

[syzaara]

In D40196#1005685, @echristo wrote:

In D40196#1005679, @syzaara wrote:

@echristo Hi Eric, are you okay to approve this now?

You didn't answer any of my questions?

Sorry I missed the first point. I agree, I think it makes more sense to move lookThruCopyLike to TRI rather than TII.

[echristo]

Thanks!

-eric

include/llvm/CodeGen/TargetInstrInfo.h

virtual bool getStackSlotRange(const TargetRegisterClass *RC, unsigned SubIdx,
                                  const MachineFunction &MF) const;
 
   /// Returns the size in bytes of the specified MachineInstr, or ~0U
   /// when this function is not implemented by a target.
   virtual unsigned getInstSizeInBytes(const MachineInstr &MI) const {
     return ~0U;
   }
 
+  /// This is used to find the "true" source register for a Machine instruction.
+  /// Returns the original SrcReg unless it is the target of a copy-like
+  /// operation, in which case we chain backwards through all such operations
+  /// to the ultimate source register.  If a physical register is encountered,
+  /// we stop the search.
+  virtual unsigned lookThruCopyLike(unsigned SrcReg,
+                                    const MachineRegisterInfo *MRI) const;
+
   /// Return true if the instruction is as cheap as a move instruction.
   ///
   /// Targets for different archs need to override this, and different
   /// micro-architectures can also be finely tuned inside.
   virtual bool isAsCheapAsAMove(const MachineInstr &MI) const {
     return MI.isAsCheapAsAMove();
   }
 
public:
 
   /// Return the noop instruction to use for a noop.
   virtual void getNoop(MCInst &NopInst) const;
 
   /// Return true for post-incremented instructions.
   virtual bool isPostIncrement(const MachineInstr &MI) const { return false; }
 
   /// Returns true if the instruction is already predicated.
   virtual bool isPredicated(const MachineInstr &MI) const { return false; }
 
   /// Returns true if the instruction is a
   /// terminator instruction that has not been predicated.
   virtual bool isUnpredicatedTerminator(const MachineInstr &MI) const;
 
   /// Returns true if MI is an unconditional tail call.
   virtual bool isUnconditionalTailCall(const MachineInstr &MI) const {
     return false;
   }
 
   /// Returns true if the tail call can be made conditional on BranchCond.

lib/CodeGen/MachineLICM.cpp

HoistCheapInsts("hoist-cheap-insts",
                 cl::desc("MachineLICM should hoist even cheap instructions"),
                 cl::init(false), cl::Hidden);
 
 static cl::opt<bool>
 SinkInstsToAvoidSpills("sink-insts-to-avoid-spills",
                        cl::desc("MachineLICM should sink instructions into "
                                 "loops to avoid register spills"),
                        cl::init(false), cl::Hidden);
+static cl::opt<bool>
+HoistConstStores("hoist-const-stores",
+                 cl::desc("Hoist invariant stores"),
+                 cl::init(false), cl::Hidden);
 
 STATISTIC(NumHoisted,
           "Number of machine instructions hoisted out of loops");
 STATISTIC(NumLowRP,
           "Number of instructions hoisted in low reg pressure situation");
 STATISTIC(NumHighLatency,
           "Number of high latency instructions hoisted");
 STATISTIC(NumCSEed,
           "Number of hoisted machine instructions CSEed");
 STATISTIC(NumPostRAHoisted,
           "Number of machine instructions hoisted out of loops post regalloc");
+STATISTIC(NumStoreConst,
+          "Number of stores of const phys reg hoisted out of loops");
 
 namespace {
 
   class MachineLICM : public MachineFunctionPass {
     const TargetInstrInfo *TII;
     const TargetLoweringBase *TLI;
     const TargetRegisterInfo *TRI;
     const MachineFrameInfo *MFI;
for (MachineDomTreeNode *Node : Scopes) {
     // Process the block
     SpeculationState = SpeculateUnknown;
     for (MachineBasicBlock::iterator
          MII = MBB->begin(), E = MBB->end(); MII != E; ) {
       MachineBasicBlock::iterator NextMII = MII; ++NextMII;
       MachineInstr *MI = &*MII;
       if (!Hoist(MI, Preheader))
         UpdateRegPressure(MI);
+      // If we have hoisted an instruction that may store, it can only be a
+      // constant store.
+      else if (MI->mayStore())
+        NumStoreConst++;
       MII = NextMII;
     }
 
     // If it's a leaf node, it's done. Traverse upwards to pop ancestors.
     ExitScopeIfDone(Node, OpenChildren, ParentMap);
   }
 }
 
static bool mayLoadFromGOTOrConstantPool(MachineInstr &MI) {
   for (MachineMemOperand *MemOp : MI.memoperands())
     if (const PseudoSourceValue *PSV = MemOp->getPseudoValue())
       if (PSV->isGOT() || PSV->isConstantPool())
         return true;
 
   return false;
 }
 
+// This function iterates through all the operands of the input store MI and
+// checks that each register operand statisfies isCallerPreservedPhysReg.
+// This means, the value being stored and the address where it is being stored
+// is constant throughout the body of the function (not including prologue and
+// epilogue). When called with an MI that isn't a store, it returns false.
+static bool isInvariantStore(const MachineInstr &MI) {
+
+  if (!MI.mayStore())
+    return false;
+
+  const MachineFunction *MF = MI.getMF();
+  const TargetRegisterInfo *TRI = MF->getSubtarget().getRegisterInfo();
+  const MachineRegisterInfo *MRI = &MI.getParent()->getParent()->getRegInfo();
+  const TargetInstrInfo *TII = MF->getSubtarget().getInstrInfo();
+  // Check that all register operands are caller-preserved physical registers.
+  for (const MachineOperand &MO : MI.operands()) {
+    if (MO.isReg()) {
+      unsigned Reg = MO.getReg();
+      // If operand is a virtual register, check if it comes from a copy of a
+      // physical register.
+      if (TargetRegisterInfo::isVirtualRegister(Reg))
+        Reg = TII->lookThruCopyLike(MO.getReg(), MRI);
+      if (TargetRegisterInfo::isVirtualRegister(Reg))
+        return false;
+      if (!TRI->isCallerPreservedPhysReg(Reg, *MI.getMF()))
+        return false;
+    }
+  }
+  return true;
+}
+
+// Return true if the input MI is a copy instruction that feeds an invariant
+// store instruction. This means that the src of the copy has to satisfy
+// isCallerPreservedPhysReg and atleast one of it's users should satisfy
+// isInvariantStore.
+static bool isCopyFeedingInvariantStore(const MachineInstr &MI) {
+
+  // FIXME: If targets would like to look through instructions that aren't
+  // pure copies, this can be updated to a query.
+  if (!MI.isCopy())
+    return false;
+
+  const MachineFunction *MF = MI.getMF();
+  const MachineRegisterInfo *MRI = &MF->getRegInfo();
+  const TargetRegisterInfo *TRI = MF->getSubtarget().getRegisterInfo();
+  const TargetInstrInfo *TII = MF->getSubtarget().getInstrInfo();
+  // Check that we are copying a constant physical register.
+  unsigned CopySrcReg = MI.getOperand(1).getReg();
+  if (TargetRegisterInfo::isVirtualRegister(CopySrcReg))
+    return false;
+
+  if (!TRI->isCallerPreservedPhysReg(CopySrcReg, *MF))
+    return false;
+
+  unsigned CopyDstReg = MI.getOperand(0).getReg();
+  // Check if any of the uses of the copy are invariant stores.
+  assert (TargetRegisterInfo::isVirtualRegister(CopyDstReg) &&
+          "copy dst is not a virtual reg");
+
+  for (MachineInstr &UseMI : MRI->use_instructions(CopyDstReg)) {
+    if (UseMI.mayStore() && isInvariantStore(UseMI))
+      return true;
+  }
+  return false;
+}
+
 /// Returns true if the instruction may be a suitable candidate for LICM.
 /// e.g. If the instruction is a call, then it's obviously not safe to hoist it.
 bool MachineLICM::IsLICMCandidate(MachineInstr &I) {
   // Check if it's safe to move the instruction.
   bool DontMoveAcrossStore = true;
-  if (!I.isSafeToMove(AA, DontMoveAcrossStore))
+  if ((!I.isSafeToMove(AA, DontMoveAcrossStore)) &&
+      !(HoistConstStores && isInvariantStore(I))) {
     return false;
+  }
 
   // If it is load then check if it is guaranteed to execute by making sure that
   // it dominates all exiting blocks. If it doesn't, then there is a path out of
   // the loop which does not execute this load, so we can't hoist it. Loads
   // from constant memory are not safe to speculate all the time, for example
   // indexed load from a jump table.
   // Stores and side effects are already checked by isSafeToMove.
   if (I.mayLoad() && !mayLoadFromGOTOrConstantPool(I) &&

 }
 
 /// Return true if it is potentially profitable to hoist the given loop
 /// invariant.
 bool MachineLICM::IsProfitableToHoist(MachineInstr &MI) {
   if (MI.isImplicitDef())
     return true;
 
   // Besides removing computation from the loop, hoisting an instruction has
   // these effects:
   //
   // - The value defined by the instruction becomes live across the entire
   //   loop. This increases register pressure in the loop.
   //
   // - If the value is used by a PHI in the loop, a copy will be required for
   //   lowering the PHI after extending the live range.
   //
   // - When hoisting the last use of a value in the loop, that value no longer
   //   needs to be live in the loop. This lowers register pressure in the loop.
 
+  if (HoistConstStores &&  isCopyFeedingInvariantStore(MI))
+    return true;
+
   bool CheapInstr = IsCheapInstruction(MI);
   bool CreatesCopy = HasLoopPHIUse(&MI);
 
   // Don't hoist a cheap instruction if it would create a copy in the loop.
   if (CheapInstr && CreatesCopy) {
     DEBUG(dbgs() << "Won't hoist cheap instr with loop PHI use: " << MI);
     return false;
   }

lib/CodeGen/TargetInstrInfo.cpp

bool TargetInstrInfo::isUnpredicatedTerminator(const MachineInstr &MI) const {
   // Conditional branch is a special case.
   if (MI.isBranch() && !MI.isBarrier())
     return true;
   if (!MI.isPredicable())
     return true;
   return !isPredicated(MI);
 }
 
+unsigned TargetInstrInfo::lookThruCopyLike(unsigned SrcReg,
+                                           const MachineRegisterInfo *MRI) const {
+  while (true) {
+    MachineInstr *MI = MRI->getVRegDef(SrcReg);
+    if (!MI->isCopyLike())
+      return SrcReg;
+
+    unsigned CopySrcReg;
+    if (MI->isCopy())
+      CopySrcReg = MI->getOperand(1).getReg();
+    else {
+      assert(MI->isSubregToReg() && "Bad opcode for lookThruCopyLike");
+      CopySrcReg = MI->getOperand(2).getReg();
+    }
+
+    if (!TargetRegisterInfo::isVirtualRegister(CopySrcReg))
+      return CopySrcReg;
+
+    SrcReg = CopySrcReg;
+  }
+}
+
 bool TargetInstrInfo::PredicateInstruction(
     MachineInstr &MI, ArrayRef<MachineOperand> Pred) const {
   bool MadeChange = false;
 
   assert(!MI.isBundle() &&
          "TargetInstrInfo::PredicateInstruction() can't handle bundles");
 
   const MCInstrDesc &MCID = MI.getDesc();
bool TargetInstrInfo::isReallyTriviallyReMaterializableGeneric(
   // doesn't read the other parts of the register.  Otherwise it is really a
   // read-modify-write operation on the full virtual register which cannot be
   // moved safely.
   if (TargetRegisterInfo::isVirtualRegister(DefReg) &&
       MI.getOperand(0).getSubReg() && MI.readsVirtualRegister(DefReg))
     return false;
 
   // A load from a fixed stack slot can be rematerialized. This may be
   // redundant with subsequent checks, but it's target-independent,
   // simple, and a common case.
   int FrameIdx = 0;
   if (isLoadFromStackSlot(MI, FrameIdx) &&
       MF.getFrameInfo().isImmutableObjectIndex(FrameIdx))
     return true;
 
   // Avoid instructions obviously unsafe for remat.
   if (MI.isNotDuplicable() || MI.mayStore() || MI.hasUnmodeledSideEffects())
     return false;

lib/Target/PowerPC/PPCInstrInfo.h

public:
   bool isZeroExtended(const MachineInstr &MI, const unsigned depth = 0) const {
    return isSignOrZeroExtended(MI, false, depth);
   }
 
   bool convertToImmediateForm(MachineInstr &MI,
                               MachineInstr **KilledDef = nullptr) const;
   void replaceInstrWithLI(MachineInstr &MI, const LoadImmediateInfo &LII) const;
 
-  // This is used to find the "true" source register for n
-  // Machine instruction. Returns the original SrcReg unless it is the target
-  // of a copy-like operation, in which case we chain backwards through all
-  // such operations to the ultimate source register.  If a
-  // physical register is encountered, we stop the search.
-  static unsigned lookThruCopyLike(unsigned SrcReg,
-                                   const MachineRegisterInfo *MRI);
   bool instrHasImmForm(const MachineInstr &MI, ImmInstrInfo &III) const;
 };
 
 }
 
 #endif

lib/Target/PowerPC/PPCInstrInfo.cpp

if (MI.getOperand(1).isImm() && !MI.getOperand(1).getImm() &&
         MI.getOperand(2).isFI()) {
       FrameIndex = MI.getOperand(2).getIndex();
       return MI.getOperand(0).getReg();
     }
     break;
   }
   return 0;
 }
 
 // For opcodes with the ReMaterializable flag set, this function is called to
 // verify the instruction is really rematable.  
 bool PPCInstrInfo::isReallyTriviallyReMaterializable(const MachineInstr &MI,
                                                      AliasAnalysis *AA) const {
   switch (MI.getOpcode()) {
   default: 
     // This function should only be called for opcodes with the ReMaterializable
     // flag set.
     llvm_unreachable("Unknown rematerializable operation!");
     break;
   case PPC::LI:
   case PPC::LI8:
   case PPC::LIS:
   case PPC::LIS8:
   case PPC::QVGPCI:
   case PPC::ADDIStocHA:
   case PPC::ADDItocL:
   case PPC::LOAD_STACK_GUARD:
     return true;
   }
   return false;
 }
 
 unsigned PPCInstrInfo::isStoreToStackSlot(const MachineInstr &MI,
                                           int &FrameIndex) const {
   // Note: This list must be kept consistent with StoreRegToStackSlot.
   switch (MI.getOpcode()) {
   default: break;
   case PPC::STD:
   case PPC::STW:
   case PPC::STFS:
   case PPC::STFD:
   case PPC::SPILL_CR:
   case PPC::SPILL_CRBIT:
   case PPC::STVX:
   case PPC::STXVD2X:
   case PPC::STXV:
   case PPC::QVSTFDX:
   case PPC::QVSTFSXs:
   case PPC::QVSTFDXb:
   case PPC::SPILL_VRSAVE:
case PPC::CFENCE8: {
     MI.setDesc(get(PPC::ISYNC));
     MI.RemoveOperand(0);
     return true;
   }
   }
   return false;
 }
 
-unsigned PPCInstrInfo::lookThruCopyLike(unsigned SrcReg,
-                                        const MachineRegisterInfo *MRI) {
-  while (true) {
-    MachineInstr *MI = MRI->getVRegDef(SrcReg);
-    if (!MI->isCopyLike())
-      return SrcReg;
-
-    unsigned CopySrcReg;
-    if (MI->isCopy())
-      CopySrcReg = MI->getOperand(1).getReg();
-    else {
-      assert(MI->isSubregToReg() && "Bad opcode for lookThruCopyLike");
-      CopySrcReg = MI->getOperand(2).getReg();
-    }
-
-    if (!TargetRegisterInfo::isVirtualRegister(CopySrcReg))
-      return CopySrcReg;
-
-    SrcReg = CopySrcReg;
-  }
-}
-
 // Essentially a compile-time implementation of a compare->isel sequence.
 // It takes two constants to compare, along with the true/false registers
 // and the comparison type (as a subreg to a CR field) and returns one
 // of the true/false registers, depending on the comparison results.
 static unsigned selectReg(int64_t Imm1, int64_t Imm2, unsigned CompareOpc,
                           unsigned TrueReg, unsigned FalseReg,
                           unsigned CRSubReg) {
   // Signed comparisons. The immediates are assumed to be sign-extended.

lib/Target/PowerPC/PPCRegisterInfo.cpp

 static cl::opt<bool>
 AlwaysBasePointer("ppc-always-use-base-pointer", cl::Hidden, cl::init(false),
          cl::desc("Force the use of a base pointer in every function"));
 
 static cl::opt<bool>
 EnableGPRToVecSpills("ppc-enable-gpr-to-vsr-spills", cl::Hidden, cl::init(false),
          cl::desc("Enable spills from gpr to vsr rather than stack"));
 
+static cl::opt<bool>
+StackPtrConst("ppc-stack-ptr-caller-preserved",
+                cl::desc("Consider R1 caller preserved so stack saves of "
+                         "caller preserved registers can be LICM candidates"),
+                cl::init(false), cl::Hidden);
+
 PPCRegisterInfo::PPCRegisterInfo(const PPCTargetMachine &TM)
   : PPCGenRegisterInfo(TM.isPPC64() ? PPC::LR8 : PPC::LR,
                        TM.isPPC64() ? 0 : 1,
                        TM.isPPC64() ? 0 : 1),
     TM(TM) {
   ImmToIdxMap[PPC::LD]   = PPC::LDX;    ImmToIdxMap[PPC::STD]  = PPC::STDX;
   ImmToIdxMap[PPC::LBZ]  = PPC::LBZX;   ImmToIdxMap[PPC::STB]  = PPC::STBX;
   ImmToIdxMap[PPC::LHZ]  = PPC::LHZX;   ImmToIdxMap[PPC::LHA]  = PPC::LHAX;
BitVector PPCRegisterInfo::getReservedRegs(const MachineFunction &MF) const {
 
   assert(checkAllSuperRegsMarked(Reserved));
   return Reserved;
 }
 
 bool PPCRegisterInfo::isCallerPreservedPhysReg(unsigned PhysReg,
                                                const MachineFunction &MF) const {
   assert(TargetRegisterInfo::isPhysicalRegister(PhysReg));
-  if (TM.isELFv2ABI() && PhysReg == PPC::X2) {
+  const PPCSubtarget &Subtarget = MF.getSubtarget<PPCSubtarget>();
+  const MachineFrameInfo &MFI = MF.getFrameInfo();
+  if (!TM.isPPC64())
+    return false;
+
+  if (!Subtarget.isSVR4ABI())
+    return false;
+  if (PhysReg == PPC::X2)
     // X2 is guaranteed to be preserved within a function if it is reserved.
     // The reason it's reserved is that it's the TOC pointer (and the function
     // uses the TOC). In functions where it isn't reserved (i.e. leaf functions
     // with no TOC access), we can't claim that it is preserved.
     return (getReservedRegs(MF).test(PPC::X2));
-  } else {
-    return false;
-  }
-}
+  if (StackPtrConst && (PhysReg == PPC::X1) && !MFI.hasVarSizedObjects()
+      && !MFI.hasOpaqueSPAdjustment())
+    // The value of the stack pointer does not change within a function after
+    // the prologue and before the epilogue if there are no dynamic allocations
+    // and no inline asm which clobbers X1.
+    return true;
 
 unsigned PPCRegisterInfo::getRegPressureLimit(const TargetRegisterClass *RC,
                                               MachineFunction &MF) const {
   const PPCFrameLowering *TFI = getFrameLowering(MF);
   const unsigned DefaultSafety = 1;
 
   switch (RC->getID()) {
   default:

test/CodeGen/PowerPC/extra-toc-reg-deps.ll

 ; CHECK: addis [[REG1:[0-9]+]], 2, .LC0@toc@ha
 ; CHECK: std 2, 40(1)
 ; CHECK: ld {{[0-9]+}}, .LC0@toc@l([[REG1]])
 ; CHECK: {{mr|ld}} 2,
 ; CHECK: mtctr
 ; CHECK: bctrl
 ; CHECK: ld 2, 40(1)
 
-; CHECK: addis [[REG1:[0-9]+]], 2, .LC0@toc@ha
 ; CHECK: std 2, 40(1)
-; CHECK: ld {{[0-9]+}}, .LC0@toc@l([[REG1]])
 ; CHECK: {{mr|ld}} 2,
 ; CHECK: mtctr
 ; CHECK: bctrl
 ; CHECK: ld 2, 40(1)
 
 for.cond.cleanup:                                 ; preds = %entry
   br i1 undef, label %if.then.i, label %if.else.i
 

test/CodeGen/PowerPC/loop-hoist-toc-save.ll

+; RUN: llc -verify-machineinstrs -mtriple=powerpc64le-unknown-linux-gnu -hoist-const-stores -ppc-stack-ptr-caller-preserved < %s | FileCheck %s
+; RUN: llc -verify-machineinstrs -mtriple=powerpc64-unknown-linux-gnu -hoist-const-stores -ppc-stack-ptr-caller-preserved < %s | FileCheck %s -check-prefix=CHECKBE
+
+; Test hoist out of single loop
+define signext i32 @test1(i32 signext %lim, i32 (i32)* nocapture %Func) {
+entry:
+; CHECK-LABEL: test1
+; CHECK: for.body.preheader
+; CHECK: std 2, 24(1)
+; CHECK: for.body
+; CHECK-NOT: std 2, 24(1)
+; CHECKBE-LABEL: test1
+; CHECKBE: for.body.preheader
+; CHECKBE: std 2, 40(1)
+; CHECKBE: for.body
+; CHECKBE-NOT: std 2, 40(1)
+
+  %cmp6 = icmp sgt i32 %lim, 0
+  br i1 %cmp6, label %for.body.preheader, label %for.cond.cleanup
+
+for.body.preheader:                               ; preds = %entry
+  br label %for.body
+
+for.cond.cleanup:                                 ; preds = %for.body, %entry
+  %Sum.0.lcssa = phi i32 [ 0, %entry ], [ %add, %for.body ]
+  ret i32 %Sum.0.lcssa
+
+for.body:                                         ; preds = %for.body.preheader, %for.body
+  %i.08 = phi i32 [ %inc, %for.body ], [ 0, %for.body.preheader ]
+  %Sum.07 = phi i32 [ %add, %for.body ], [ 0, %for.body.preheader ]
+  %call = tail call signext i32 %Func(i32 signext %i.08)
+  %add = add nsw i32 %call, %Sum.07
+  %inc = add nuw nsw i32 %i.08, 1
+  %exitcond = icmp eq i32 %inc, %lim
+  br i1 %exitcond, label %for.cond.cleanup, label %for.body
+}
+
+; Test hoist of nested loop goes to outter loop preheader
+define signext i32 @test2(i32 signext %lim, i32 (i32)* nocapture %Func) {
+entry:
+; CHECK-LABEL: test2
+; CHECK: for.body4.lr.ph.preheader
+; CHECK: std 2, 24(1)
+; CHECK: for.body4.lr.ph
+; CHECK-NOT: std 2, 24(1)
+; CHECKBE-LABEL: test2
+; CHECKBE: for.body4.lr.ph.preheader
+; CHECKBE: std 2, 40(1)
+; CHECKBE: for.body4.lr.ph
+; CHECKBE-NOT: std 2, 40(1)
+
+  %cmp20 = icmp sgt i32 %lim, 0
+  br i1 %cmp20, label %for.body4.lr.ph.preheader, label %for.cond.cleanup
+
+for.body4.lr.ph.preheader:                        ; preds = %entry
+  br label %for.body4.lr.ph
+
+for.cond.cleanup:                                 ; preds = %for.cond.cleanup3, %entry
+  %Sum.0.lcssa = phi i32 [ 0, %entry ], [ %add, %for.cond.cleanup3 ]
+  ret i32 %Sum.0.lcssa
+
+for.body4.lr.ph:                                  ; preds = %for.body4.lr.ph.preheader, %for.cond.cleanup3
+  %j.022 = phi i32 [ %inc6, %for.cond.cleanup3 ], [ 0, %for.body4.lr.ph.preheader ]
+  %Sum.021 = phi i32 [ %add, %for.cond.cleanup3 ], [ 0, %for.body4.lr.ph.preheader ]
+  br label %for.body4
+
+for.cond.cleanup3:                                ; preds = %for.body4
+  %inc6 = add nuw nsw i32 %j.022, 1
+  %exitcond24 = icmp eq i32 %inc6, %lim
+  br i1 %exitcond24, label %for.cond.cleanup, label %for.body4.lr.ph
+
+for.body4:                                        ; preds = %for.body4, %for.body4.lr.ph
+  %i.019 = phi i32 [ %j.022, %for.body4.lr.ph ], [ %inc, %for.body4 ]
+  %Sum.118 = phi i32 [ %Sum.021, %for.body4.lr.ph ], [ %add, %for.body4 ]
+  %call = tail call signext i32 %Func(i32 signext %i.019)
+  %add = add nsw i32 %call, %Sum.118
+  %inc = add nuw nsw i32 %i.019, 1
+  %exitcond = icmp eq i32 %inc, %lim
+  br i1 %exitcond, label %for.cond.cleanup3, label %for.body4
+}
+
+; Test hoist out of if statement with low branch probability
+; FIXME: we shouldn't hoist in such cases as it could increase the number
+; of stores after hoisting.
+define signext i32 @test3(i32 signext %lim, i32 (i32)* nocapture %Func) {
+entry:
+; CHECK-LABEL: test3
+; CHECK: %for.body.lr.ph
+; CHECK: std 2, 24(1)
+; CHECK: %for.body
+; CHECK-NOT: std 2, 24(1)
+; CHECKBE-LABEL: test3
+; CHECKBE: %for.body.lr.ph
+; CHECKBE: std 2, 40(1)
+; CHECKBE: %for.body
+; CHECKBE-NOT: std 2, 40(1)
+
+  %cmp13 = icmp sgt i32 %lim, 0
+  br i1 %cmp13, label %for.body.lr.ph, label %for.cond.cleanup
+
+for.body.lr.ph:                                   ; preds = %entry
+  %sub = add nsw i32 %lim, -1
+  br label %for.body
+
+for.cond.cleanup:                                 ; preds = %if.end, %entry
+  %Sum.0.lcssa = phi i32 [ 0, %entry ], [ %add3, %if.end ]
+  ret i32 %Sum.0.lcssa
+
+for.body:                                         ; preds = %if.end, %for.body.lr.ph
+  %i.015 = phi i32 [ 0, %for.body.lr.ph ], [ %inc, %if.end ]
+  %Sum.014 = phi i32 [ 0, %for.body.lr.ph ], [ %add3, %if.end ]
+  %cmp1 = icmp eq i32 %i.015, %sub
+  br i1 %cmp1, label %if.then, label %if.end
+
+if.then:                                          ; preds = %for.body
+  %call = tail call signext i32 %Func(i32 signext %sub)
+  %add = add nsw i32 %call, %Sum.014
+  br label %if.end
+
+if.end:                                           ; preds = %if.then, %for.body
+  %Sum.1 = phi i32 [ %add, %if.then ], [ %Sum.014, %for.body ]
+  %call2 = tail call signext i32 @func(i32 signext %i.015)
+  %add3 = add nsw i32 %call2, %Sum.1
+  %inc = add nuw nsw i32 %i.015, 1
+  %exitcond = icmp eq i32 %inc, %lim
+  br i1 %exitcond, label %for.cond.cleanup, label %for.body
+}
+
+declare signext i32 @func(i32 signext)