[PowerPC] Add implementation for -msave-toc-indirect option - llvm portion

[syzaara]

Summary

This patch adds the implementation for -msave-toc-indirect. This allows the TOC register to be saved once in the prologue, if no dynamic stack allocations are in the function, rather than multiple times before each indirect call.

[hfinkel]

I want to take a step back and discuss what problem this is trying to solve and how to best solve it. I understand that GCC has a flag for this (added in 4.7), but it is not clear to me that this is the best approach. Two thoughts:

First, we have low-hanging fruit to take care of before I'd want to consider something like this. If you look at the code generated for foo2 in this program:

typedef void (*fp)(int);

void foo2(fp f) {
  f(5);
  f(10);
}

you'll see:

mflr 0
std 31, -8(1) 
std 0, 16(1) 
stdu 1, -48(1)
mr 31, 1 
std 2, 24(1) 
std 30, 32(31)                  # 8-byte Folded Spill 
mr 30, 3 
li 3, 5
mr 12, 30
mtctr 30
bctrl 
ld 2, 24(1) 
li 3, 10 
mr 12, 30
std 2, 24(1) 
mtctr 30
bctrl 
ld 2, 24(1) 
ld 30, 32(31)                   # 8-byte Folded Reload
addi 1, 1, 48 
ld 0, 16(1) 
ld 31, -8(1) 
mtlr 0
blr   
.long 0
.quad 0

note specifically the part:

...
bctrl
ld 2, 24(1)
li 3, 10
mr 12, 30
std 2, 24(1) ; <-- This can go away because of the load above (no need to store again).
mtctr 30
...

I'd like to think that we already have something that would clean this up, but apparently we don't. We should.

Second, is another option to always do this and then use shrink wrapping to account for cold-path-only calls?

[nemanjai]

I'd like to think that we already have something that would clean this up, but apparently we don't. We should.

I was under the impression that this patch was supposed to be the something that cleans it up. What I thought this would be is a patch that does 2 things:

1. Save the TOC in the prologue if the option is set and there's an indirect call in the function
2. Fix up the call lowering sequence to not do the TOC save prior to the call

If it accomplishes the stated goals, it would achieve what you're after, right @hfinkel? It certainly removes the two TOC saves from your test case. Of course, the prologue is just chosen because it is guaranteed to dominate all the indirect calls. Otherwise we may not always be able to do this without some kind of dataflow analysis + code motion + possibly creation of new basic blocks.
For example, where's the right place to save the TOC on something like this:

void foo2(void (*f)(int), int A) {
  if (A > 10)
    f(5);
  f(15);
}

Second, is another option to always do this and then use shrink wrapping to account for cold-path-only calls?

If we're saving the TOC in the prologue, won't shrink-wrapping handle this anyway? When it moves the prologue, it moves the whole prologue. Of course, as a result of this patch, we'll always save the TOC pointer in any function that has indirect calls. This is of course worse if the indirect call happens only on the cold path. It might be worthwhile to not perform this transformation on functions that have only 1 indirect call (as that call might be on a cold path). But of course, then we'd still miss the opportunities where multiple indirect calls are all on cold paths. But, can't win 'em all.

lib/Target/PowerPC/PPCFrameLowering.cpp

1718

Can you just fold these conditions (negated as required) into the original if statement so that we don't have the weird empty-body conditional?

test/CodeGen/PowerPC/save_toc_indirect2.ll

9

If this test case checks that we still do the TOC saves in a function that has dynamic allocations, that should be clearly noted in the test case through the name of the function and/or comments.

[hfinkel]

In D39344#910045, @nemanjai wrote:

I'd like to think that we already have something that would clean this up, but apparently we don't. We should.

I was under the impression that this patch was supposed to be the something that cleans it up.

Yes, but only in some special mode. I see no reason this needs to happen in some special mode.

What I thought this would be is a patch that does 2 things:

1. Save the TOC in the prologue if the option is set and there's an indirect call in the function
2. Fix up the call lowering sequence to not do the TOC save prior to the call

If it accomplishes the stated goals, it would achieve what you're after, right @hfinkel? It certainly removes the two TOC saves from your test case. Of course, the prologue is just chosen because it is guaranteed to dominate all the indirect calls. Otherwise we may not always be able to do this without some kind of dataflow analysis + code motion + possibly creation of new basic blocks.
For example, where's the right place to save the TOC on something like this:

void foo2(void (*f)(int), int A) {
  if (A > 10)
    f(5);
  f(15);
}

Second, is another option to always do this and then use shrink wrapping to account for cold-path-only calls?

If we're saving the TOC in the prologue, won't shrink-wrapping handle this anyway?

I'd hope so.

When it moves the prologue, it moves the whole prologue. Of course, as a result of this patch, we'll always save the TOC pointer in any function that has indirect calls. This is of course worse if the indirect call happens only on the cold path. It might be worthwhile to not perform this transformation on functions that have only 1 indirect call (as that call might be on a cold path). But of course, then we'd still miss the opportunities where multiple indirect calls are all on cold paths.

Right.

But, can't win 'em all.

This is why I recommend looking at cleaning up the existing generated code for saving/restoring r2, using some data-flow analysis to remove redundant saves, because that would be a pure win. Doing things with tradeoffs can come next.

include/llvm/CodeGen/MachineFrameInfo.h

class MachineFrameInfo {
   /// True if this is a varargs function that contains a musttail call.
   bool HasMustTailInVarArgFunc = false;
 
   /// True if this function contains a tail call. If so immutable objects like
   /// function arguments are no longer so. A tail call *can* override fixed
   /// stack objects like arguments so we can't treat them as immutable.
   bool HasTailCall = false;
 
+  /// True if this function contains an indirect call.
+  bool HasIndirectCall = false;
+
   /// Not null, if shrink-wrapping found a better place for the prologue.
   MachineBasicBlock *Save = nullptr;
   /// Not null, if shrink-wrapping found a better place for the epilogue.
   MachineBasicBlock *Restore = nullptr;
 
 public:
   explicit MachineFrameInfo(unsigned StackAlignment, bool StackRealignable,
                             bool ForcedRealign)
public:
   /// Returns true if the function is variadic and contains a musttail call.
   bool hasMustTailInVarArgFunc() const { return HasMustTailInVarArgFunc; }
   void setHasMustTailInVarArgFunc(bool B) { HasMustTailInVarArgFunc = B; }
 
   /// Returns true if the function contains a tail call.
   bool hasTailCall() const { return HasTailCall; }
   void setHasTailCall() { HasTailCall = true; }
 
+  /// Returns true if the function contains an indirect call.
+  bool hasIndirectCall() const { return HasIndirectCall; }
+  void setHasIndirectCall() { HasIndirectCall = true; }
+
   /// Computes the maximum size of a callframe and the AdjustsStack property.
   /// This only works for targets defining
   /// TargetInstrInfo::getCallFrameSetupOpcode(), getCallFrameDestroyOpcode(),
   /// and getFrameSize().
   /// This is usually computed by the prologue epilogue inserter but some
   /// targets may call this to compute it earlier.
   void computeMaxCallFrameSize(const MachineFunction &MF);
 

lib/Target/PowerPC/PPC.td

def FeatureP8Vector  : SubtargetFeature<"power8-vector", "HasP8Vector", "true",
                                         [FeatureVSX, FeatureP8Altivec]>;
 def FeatureDirectMove :
   SubtargetFeature<"direct-move", "HasDirectMove", "true",
                    "Enable Power8 direct move instructions",
                    [FeatureVSX]>;
 def FeaturePartwordAtomic : SubtargetFeature<"partword-atomics",
                                              "HasPartwordAtomics", "true",
                                              "Enable l[bh]arx and st[bh]cx.">;
+def FeatureSaveTOCIndirect :
+  SubtargetFeature<"save-toc-indirect",
+                   "HasSaveTOCIndirect", "true",
+                   "TOC reg is saved in prologue when there are indirect calls">;
+
 def FeatureInvariantFunctionDescriptors :
   SubtargetFeature<"invariant-function-descriptors",
                    "HasInvariantFunctionDescriptors", "true",
                    "Assume function descriptors are invariant">;
 def FeatureLongCall : SubtargetFeature<"longcall", "UseLongCalls", "true",
                                        "Always use indirect calls">;
 def FeatureHTM : SubtargetFeature<"htm", "HasHTM", "true",
                                   "Enable Hardware Transactional Memory instructions">;

lib/Target/PowerPC/PPCFrameLowering.cpp

static bool MustSaveLR(const MachineFunction &MF, unsigned LR) {
   // We need a save/restore of LR if there is any def of LR (which is
   // defined by calls, including the PIC setup sequence), or if there is
   // some use of the LR stack slot (e.g. for builtin_return_address).
   // (LR comes in 32 and 64 bit versions.)
   MachineRegisterInfo::def_iterator RI = MF.getRegInfo().def_begin(LR);
   return RI !=MF.getRegInfo().def_end() || MFI->isLRStoreRequired();
 }
 
+/// MustSaveTOC - Return true if this function requires that we save the
+/// TOC on the stack in the prologue rather than before each indirect call.
+static bool MustSaveTOC(const MachineFunction &MF) {
+  const MachineFrameInfo &MFI = MF.getFrameInfo();
+  return MF.getSubtarget<PPCSubtarget>().hasSaveTOCIndirect() &&
+         MFI.hasIndirectCall();
+}
 /// determineFrameLayout - Determine the size of the frame and maximum call
 /// frame size.
 unsigned PPCFrameLowering::determineFrameLayout(MachineFunction &MF,
                                                 bool UpdateMF,
                                                 bool UseEstimate) const {
   MachineFrameInfo &MFI = MF.getFrameInfo();
 
   // Get the number of bytes to allocate from the FrameInfo
if (!isInt<32>(NegFrameSize))
     llvm_unreachable("Unhandled stack size!");
 
   if (MFI.isFrameAddressTaken())
     replaceFPWithRealFP(MF);
 
   // Check if the link register (LR) must be saved.
   PPCFunctionInfo *FI = MF.getInfo<PPCFunctionInfo>();
   bool MustSaveLR = FI->mustSaveLR();
+  bool MustSaveTOC = FI->mustSaveTOC();
+
   const SmallVectorImpl<unsigned> &MustSaveCRs = FI->getMustSaveCRs();
   bool MustSaveCR = !MustSaveCRs.empty();
   // Do we have a frame pointer and/or base pointer for this function?
   bool HasFP = hasFP(MF);
   bool HasBP = RegInfo->hasBasePointer(MF);
   bool HasRedZone = isPPC64 || !isSVR4ABI;
 
   unsigned SPReg       = isPPC64 ? PPC::X1  : PPC::R1;
   unsigned BPReg       = RegInfo->getBaseRegister(MF);
   unsigned FPReg       = isPPC64 ? PPC::X31 : PPC::R31;
   unsigned LRReg       = isPPC64 ? PPC::LR8 : PPC::LR;
+  unsigned TOCReg      = isPPC64 ? PPC::X2 : PPC::R2;
   unsigned ScratchReg  = 0;
   unsigned TempReg     = isPPC64 ? PPC::X12 : PPC::R12; // another scratch reg
   //  ...(R12/X12 is volatile in both Darwin & SVR4, & can't be a function arg.)
   const MCInstrDesc& MFLRInst = TII.get(isPPC64 ? PPC::MFLR8
                                                 : PPC::MFLR );
   const MCInstrDesc& StoreInst = TII.get(isPPC64 ? PPC::STD
                                                  : PPC::STW );
   const MCInstrDesc& StoreUpdtInst = TII.get(isPPC64 ? PPC::STDU
BuildMI(MBB, MBBI, dl, OrImmInst, ScratchReg)
       .addImm(NegFrameSize & 0xFFFF);
     BuildMI(MBB, MBBI, dl, StoreUpdtIdxInst, SPReg)
       .addReg(SPReg, RegState::Kill)
       .addReg(SPReg)
       .addReg(ScratchReg);
     HasSTUX = true;
   }
 
+  if (MustSaveTOC) {
+    BuildMI(MBB, MBBI, dl, TII.get(PPC::STD))
+      .addReg(TOCReg, getKillRegState(true))
+      .addImm(24)
+      .addReg(SPReg);
+  }
+
   if (!HasRedZone) {
     assert(!isPPC64 && "A red zone is always available on PPC64");
     if (HasSTUX) {
       // The negated frame size is in ScratchReg, and the SPReg has been
       // decremented by the frame size: SPReg = old SPReg + ScratchReg.
       // Since FPOffset, PBPOffset, etc. are relative to the beginning of
       // the stack frame (i.e. the old SP), ideally, we would put the old
       // SP into a register and use it as the base for the stores. The
for (unsigned I = 0, E = CSI.size(); I != E; ++I) {
       unsigned Reg = CSI[I].getReg();
       if (Reg == PPC::LR || Reg == PPC::LR8 || Reg == PPC::RM) continue;
 
       // This is a bit of a hack: CR2LT, CR2GT, CR2EQ and CR2UN are just
       // subregisters of CR2. We just need to emit a move of CR2.
       if (PPC::CRBITRCRegClass.contains(Reg))
         continue;
 
+      if (Reg == PPC::X2 || Reg == PPC::R2)
+        continue;
       // For SVR4, don't emit a move for the CR spill slot if we haven't
       // spilled CRs.
       if (isSVR4ABI && (PPC::CR2 <= Reg && Reg <= PPC::CR4)
           && !MustSaveCR)
         continue;
 
       // For 64-bit SVR4 when we have spilled CRs, the spill location
       // is SP+8, not a frame-relative slot.
void PPCFrameLowering::determineCalleeSaves(MachineFunction &MF,
   const PPCRegisterInfo *RegInfo = Subtarget.getRegisterInfo();
 
   //  Save and clear the LR state.
   PPCFunctionInfo *FI = MF.getInfo<PPCFunctionInfo>();
   unsigned LR = RegInfo->getRARegister();
   FI->setMustSaveLR(MustSaveLR(MF, LR));
   SavedRegs.reset(LR);
 
+  MachineFrameInfo &MFI = MF.getFrameInfo();
+  FI->setMustSaveTOC(MustSaveTOC(MF) && !MFI.hasVarSizedObjects());
+
   //  Save R31 if necessary
   int FPSI = FI->getFramePointerSaveIndex();
   bool isPPC64 = Subtarget.isPPC64();
   bool isDarwinABI  = Subtarget.isDarwinABI();
-  MachineFrameInfo &MFI = MF.getFrameInfo();
 
   // If the frame pointer save index hasn't been defined yet.
   if (!FPSI && needsFP(MF)) {
     // Find out what the fix offset of the frame pointer save area.
     int FPOffset = getFramePointerSaveOffset();
     // Allocate the frame index for frame pointer save area.
     FPSI = MFI.CreateFixedObject(isPPC64? 8 : 4, FPOffset, true);
     // Save the result.
if (!Subtarget.isSVR4ABI()) {
     addScavengingSpillSlot(MF, RS);
     return;
   }
 
   // Get callee saved register information.
   MachineFrameInfo &MFI = MF.getFrameInfo();
   const std::vector<CalleeSavedInfo> &CSI = MFI.getCalleeSavedInfo();
 
+  PPCFunctionInfo *FI = MF.getInfo<PPCFunctionInfo>();
+  bool MustSaveTOC = FI->mustSaveTOC();
+
   // If the function is shrink-wrapped, and if the function has a tail call, the
   // tail call might not be in the new RestoreBlock, so real branch instruction
   // won't be generated by emitEpilogue(), because shrink-wrap has chosen new
   // RestoreBlock. So we handle this case here.
   if (MFI.getSavePoint() && MFI.hasTailCall()) {
     MachineBasicBlock *RestoreBlock = MFI.getRestorePoint();
     for (MachineBasicBlock &MBB : MF) {
       if (MBB.isReturnBlock() && (&MBB) != RestoreBlock)
void PPCFrameLowering::processFunctionBeforeFrameFinalized(MachineFunction &MF,
 
   SmallVector<CalleeSavedInfo, 18> GPRegs;
   SmallVector<CalleeSavedInfo, 18> G8Regs;
   SmallVector<CalleeSavedInfo, 18> FPRegs;
   SmallVector<CalleeSavedInfo, 18> VRegs;
 
   for (unsigned i = 0, e = CSI.size(); i != e; ++i) {
     unsigned Reg = CSI[i].getReg();
-    if (PPC::GPRCRegClass.contains(Reg)) {
+    if ((Reg == PPC::X2 || Reg == PPC::R2) && MustSaveTOC) {
+      ;
+    } else if (PPC::GPRCRegClass.contains(Reg)) {
       HasGPSaveArea = true;
 
       GPRegs.push_back(CSI[i]);
 
       if (Reg < MinGPR) {
         MinGPR = Reg;
       }
     } else if (PPC::G8RCRegClass.contains(Reg)) {
PPCFrameLowering::spillCalleeSavedRegisters(MachineBasicBlock &MBB,
                                      const TargetRegisterInfo *TRI) const {
 
   // Currently, this function only handles SVR4 32- and 64-bit ABIs.
   // Return false otherwise to maintain pre-existing behavior.
   if (!Subtarget.isSVR4ABI())
     return false;
 
   MachineFunction *MF = MBB.getParent();
+  PPCFunctionInfo *FI = MF->getInfo<PPCFunctionInfo>();
+  bool MustSaveTOC = FI->mustSaveTOC();
   const PPCInstrInfo &TII = *Subtarget.getInstrInfo();
   DebugLoc DL;
   bool CRSpilled = false;
   MachineInstrBuilder CRMIB;
 
   for (unsigned i = 0, e = CSI.size(); i != e; ++i) {
     unsigned Reg = CSI[i].getReg();
     // Only Darwin actually uses the VRSAVE register, but it can still appear
     // here if, for example, @llvm.eh.unwind.init() is used.  If we're not on
     // Darwin, ignore it.
     if (Reg == PPC::VRSAVE && !Subtarget.isDarwinABI())
       continue;
 
     // CR2 through CR4 are the nonvolatile CR fields.
     bool IsCRField = PPC::CR2 <= Reg && Reg <= PPC::CR4;
-
     // Add the callee-saved register as live-in; it's killed at the spill.
     MBB.addLiveIn(Reg);
 
     if (CRSpilled && IsCRField) {
       CRMIB.addReg(Reg, RegState::ImplicitKill);
       continue;
     }
 
+    // The actual spill will happen in the prologue.
+    if ((Reg == PPC::X2 || Reg == PPC::R2) && MustSaveTOC) {
+      continue;
+    }
+
+
     // Insert the spill to the stack frame.
     if (IsCRField) {
       PPCFunctionInfo *FuncInfo = MF->getInfo<PPCFunctionInfo>();
       if (Subtarget.isPPC64()) {
         // The actual spill will happen at the start of the prologue.
         FuncInfo->addMustSaveCR(Reg);
       } else {
         CRSpilled = true;
PPCFrameLowering::restoreCalleeSavedRegisters(MachineBasicBlock &MBB,
                                         const TargetRegisterInfo *TRI) const {
 
   // Currently, this function only handles SVR4 32- and 64-bit ABIs.
   // Return false otherwise to maintain pre-existing behavior.
   if (!Subtarget.isSVR4ABI())
     return false;
 
   MachineFunction *MF = MBB.getParent();
+  PPCFunctionInfo *FI = MF->getInfo<PPCFunctionInfo>();
+  bool MustSaveTOC = FI->mustSaveTOC();
+
   const PPCInstrInfo &TII = *Subtarget.getInstrInfo();
   bool CR2Spilled = false;
   bool CR3Spilled = false;
   bool CR4Spilled = false;
   unsigned CSIIndex = 0;
 
   // Initialize insertion-point logic; we will be restoring in reverse
   // order of spill.
   MachineBasicBlock::iterator I = MI, BeforeI = I;
   bool AtStart = I == MBB.begin();
 
   if (!AtStart)
     --BeforeI;
 
   for (unsigned i = 0, e = CSI.size(); i != e; ++i) {
     unsigned Reg = CSI[i].getReg();
 
     // Only Darwin actually uses the VRSAVE register, but it can still appear
     // here if, for example, @llvm.eh.unwind.init() is used.  If we're not on
     // Darwin, ignore it.
     if (Reg == PPC::VRSAVE && !Subtarget.isDarwinABI())
       continue;
 
+    if ((Reg == PPC::X2 || Reg == PPC::R2) && MustSaveTOC) {
+      continue;
+    }
+
     if (Reg == PPC::CR2) {
       CR2Spilled = true;
       // The spill slot is associated only with CR2, which is the
       // first nonvolatile spilled.  Save it here.
       CSIIndex = i;
       continue;
     } else if (Reg == PPC::CR3) {
       CR3Spilled = true;

lib/Target/PowerPC/PPCISelLowering.cpp

 cl::desc("disable setting the node scheduling preference to ILP on PPC"), cl::Hidden);
 
 static cl::opt<bool> DisablePPCUnaligned("disable-ppc-unaligned",
 cl::desc("disable unaligned load/store generation on PPC"), cl::Hidden);
 
 static cl::opt<bool> DisableSCO("disable-ppc-sco",
 cl::desc("disable sibling call optimization on ppc"), cl::Hidden);
 
+STATISTIC(NumSetHasIndirectCall, "Number of indirect calls");
 STATISTIC(NumTailCalls, "Number of tail calls");
 STATISTIC(NumSiblingCalls, "Number of sibling calls");
 
 // FIXME: Remove this once the bug has been fixed!
 extern cl::opt<bool> ANDIGlueBug;
 
 PPCTargetLowering::PPCTargetLowering(const PPCTargetMachine &TM,
                                      const PPCSubtarget &STI)
SDValue PPCTargetLowering::FinishCall(
   // which restores the TOC of the caller from the TOC save slot of the current
   // stack frame. If caller and callee belong to the same module (and have the
   // same TOC), the NOP will remain unchanged.
 
   MachineFunction &MF = DAG.getMachineFunction();
   if (!isTailCall && Subtarget.isSVR4ABI()&& Subtarget.isPPC64() &&
       !isPatchPoint) {
     if (CallOpc == PPCISD::BCTRL) {
+      DAG.getMachineFunction().getFrameInfo().setHasIndirectCall();
+      NumSetHasIndirectCall++;
       // This is a call through a function pointer.
       // Restore the caller TOC from the save area into R2.
       // See PrepareCall() for more information about calls through function
       // pointers in the 64-bit SVR4 ABI.
       // We are using a target-specific load with r2 hard coded, because the
       // result of a target-independent load would never go directly into r2,
       // since r2 is a reserved register (which prevents the register allocator
       // from allocating it), resulting in an additional register being
SDValue PPCTargetLowering::LowerCall_64SVR4(
   // Check if this is an indirect call (MTCTR/BCTRL).
   // See PrepareCall() for more information about calls through function
   // pointers in the 64-bit SVR4 ABI.
   if (!isTailCall && !isPatchPoint &&
       !isFunctionGlobalAddress(Callee) &&
       !isa<ExternalSymbolSDNode>(Callee)) {
     // Load r2 into a virtual register and store it to the TOC save area.
     setUsesTOCBasePtr(DAG);
+    // Don't emit a store of TOC before indirect call if using option
+    // -msave-toc-direct and the function doesn't have dynamic allocations
+    // on the stack.
+    const MachineFrameInfo &MFI = MF.getFrameInfo();
+    if (!(Subtarget.hasSaveTOCIndirect() && !MFI.hasVarSizedObjects())) {
       SDValue Val = DAG.getCopyFromReg(Chain, dl, PPC::X2, MVT::i64);
       // TOC save area offset.
       unsigned TOCSaveOffset = Subtarget.getFrameLowering()->getTOCSaveOffset();
       SDValue PtrOff = DAG.getIntPtrConstant(TOCSaveOffset, dl);
       SDValue AddPtr = DAG.getNode(ISD::ADD, dl, PtrVT, StackPtr, PtrOff);
       Chain = DAG.getStore(
           Val.getValue(1), dl, Val, AddPtr,
           MachinePointerInfo::getStack(DAG.getMachineFunction(), TOCSaveOffset));
+    }
     // In the ELFv2 ABI, R12 must contain the address of an indirect callee.
     // This does not mean the MTCTR instruction must use R12; it's easier
     // to model this as an extra parameter, so do that.
     if (isELFv2ABI && !isPatchPoint)
       RegsToPass.push_back(std::make_pair((unsigned)PPC::X12, Callee));
   }
 
   // Build a sequence of copy-to-reg nodes chained together with token chain

lib/Target/PowerPC/PPCMachineFunctionInfo.h

class PPCFunctionInfo : public MachineFunctionInfo {
   /// Frame index where the old PIC base pointer is stored.
   int PICBasePointerSaveIndex = 0;
 
   /// MustSaveLR - Indicates whether LR is defined (or clobbered) in the current
   /// function.  This is only valid after the initial scan of the function by
   /// PEI.
   bool MustSaveLR;
 
+  /// MustSaveTOC - Indicates whether TOC should be saved in prologue for option
+  /// -msave-toc-indirect. This is only valid after the initial scan of the
+  /// function by PEI.
+  bool MustSaveTOC;
+
   /// Does this function have any stack spills.
   bool HasSpills = false;
 
   /// Does this function spill using instructions with only r+r (not r+i)
   /// forms.
   bool HasNonRISpills = false;
 
   /// SpillsCR - Indicates whether CR is spilled in the current function.
public:
   /// MustSaveLR - This is set when the prolog/epilog inserter does its initial
   /// scan of the function. It is true if the LR/LR8 register is ever explicitly
   /// defined/clobbered in the machine function (e.g. by calls and movpctolr,
   /// which is used in PIC generation), or if the LR stack slot is explicitly
   /// referenced by builtin_return_address.
   void setMustSaveLR(bool U) { MustSaveLR = U; }
   bool mustSaveLR() const    { return MustSaveLR; }
 
+  /// MustSaveTOC - This is set when a function contains indirect calls and
+  /// no dynamic stack allocations. If using option -msave-toc-indirect,
+  /// the TOC will be saved once in the prologue rather than before each
+  /// indirect function call.
+  void setMustSaveTOC(bool U) { MustSaveTOC = U; }
+  bool mustSaveTOC() const    { return MustSaveTOC; }
+
   void setHasSpills()      { HasSpills = true; }
   bool hasSpills() const   { return HasSpills; }
 
   void setHasNonRISpills()    { HasNonRISpills = true; }
   bool hasNonRISpills() const { return HasNonRISpills; }
 
   void setSpillsCR()       { SpillsCR = true; }
   bool isCRSpilled() const { return SpillsCR; }

lib/Target/PowerPC/PPCSubtarget.h

protected:
   bool IsPPC4xx;
   bool IsPPC6xx;
   bool FeatureMFTB;
   bool DeprecatedDST;
   bool HasLazyResolverStubs;
   bool IsLittleEndian;
   bool HasICBT;
   bool HasInvariantFunctionDescriptors;
+  bool HasSaveTOCIndirect;
   bool HasPartwordAtomics;
   bool HasDirectMove;
   bool HasHTM;
   bool HasFusion;
   bool HasFloat128;
   bool IsISA3_0;
   bool UseLongCalls;
 
public:
   bool isPPC6xx() const { return IsPPC6xx; }
   bool isE500() const { return IsE500; }
   bool isFeatureMFTB() const { return FeatureMFTB; }
   bool isDeprecatedDST() const { return DeprecatedDST; }
   bool hasICBT() const { return HasICBT; }
   bool hasInvariantFunctionDescriptors() const {
     return HasInvariantFunctionDescriptors;
   }
+  bool hasSaveTOCIndirect() const {
+    return HasSaveTOCIndirect;
+  }
   bool hasPartwordAtomics() const { return HasPartwordAtomics; }
   bool hasDirectMove() const { return HasDirectMove; }
 
   bool isQPXStackUnaligned() const { return IsQPXStackUnaligned; }
   unsigned getPlatformStackAlignment() const {
     if ((hasQPX() || isBGQ()) && !isQPXStackUnaligned())
       return 32;
 

test/CodeGen/PowerPC/save_toc_indirect.ll

+; RUN: llc -verify-machineinstrs -mtriple=powerpc64le-unknown-linux-gnu -mattr=+save-toc-indirect < %s | FileCheck %s --check-prefix=TOC_INDIRECT
+; RUN: llc -verify-machineinstrs -mtriple=powerpc64le-unknown-linux-gnu < %s | FileCheck %s
+
+define signext i32 @test(i32 signext %i, i32 (i32)* nocapture %Func, i32 (i32)* nocapture %Func2) {
+; TOC_INDIRECT-LABEL: test:
+; TOC_INDIRECT:       # BB#0: # %entry
+; TOC_INDIRECT:    std 2, 24(1)
+; TOC_INDIRECT:    bctrl
+; TOC_INDIRECT:    ld 2, 24(1)
+; TOC_INDIRECT-NOT:    std 2, 24(1)
+; TOC_INDIRECT:    bctrl
+; TOC_INDIRECT:    ld 2, 24(1)
+
+; CHECK-LABEL: test:
+; CHECK:       # BB#0: # %entry
+; CHECK:    std 2, 24(1)
+; CHECK:    bctrl
+; CHECK:    ld 2, 24(1)
+; CHECK:    std 2, 24(1)
+; CHECK:    bctrl
+; CHECK:    ld 2, 24(1)
+
+entry:
+  %call = tail call signext i32 %Func(i32 signext %i)
+  %call1 = tail call signext i32 %Func2(i32 signext %i)
+  %add2 = add nsw i32 %call1, %call
+  ret i32 %add2
+}

test/CodeGen/PowerPC/save_toc_indirect2.ll

+; RUN: llc -verify-machineinstrs -mtriple=powerpc64le-unknown-linux-gnu -mattr=+save-toc-indirect < %s | FileCheck %s --check-prefix=TOC_INDIRECT
+
+define signext i32 @test(i32 signext %i, i32 (i32)* nocapture %Func, i32 (i32)* nocapture %Func2) {
+; TOC_INDIRECT-LABEL: test:
+; TOC_INDIRECT:       # BB#0: # %entry
+; TOC_INDIRECT:    std 2,  {{[0-9]+}}(1)
+; TOC_INDIRECT:    bctrl
+; TOC_INDIRECT:    ld 2, {{[0-9]+}}(1)
+; TOC_INDIRECT:    std 2, {{[0-9]+}}(1)
+; TOC_INDIRECT:    bctrl
+; TOC_INDIRECT:    ld 2, {{[0-9]+}}(1)
+; TOC_INDIRECT:    std 2, {{[0-9]+}}(1)
+; TOC_INDIRECT:    bctrl
+; TOC_INDIRECT:    ld 2, {{[0-9]+}}(1)
+entry:
+  %call = tail call signext i32 %Func(i32 signext %i)
+  %call1 = tail call signext i32 %Func2(i32 signext %i)
+  %add2 = add nsw i32 %call1, %call
+  %conv = sext i32 %add2 to i64
+  %0 = alloca i8, i64 %conv, align 16
+  %1 = bitcast i8* %0 to i32*
+  %cmp24 = icmp sgt i32 %add2, 0
+  br i1 %cmp24, label %for.body.lr.ph, label %for.cond.cleanup
+
+for.body.lr.ph:                                   ; preds = %entry
+  %wide.trip.count = zext i32 %add2 to i64
+  br label %for.body
+
+for.cond.cleanup:                                 ; preds = %for.body, %entry
+  %call7 = call signext i32 @UseAlloca(i32* nonnull %1)
+  %add8 = add nsw i32 %call7, %add2
+  ret i32 %add8
+
+for.body:                                         ; preds = %for.body, %for.body.lr.ph
+  %indvars.iv = phi i64 [ 0, %for.body.lr.ph ], [ %indvars.iv.next, %for.body ]
+  %2 = add nsw i64 %indvars.iv, %conv
+  %3 = trunc i64 %2 to i32
+  %call6 = tail call signext i32 %Func(i32 signext %3)
+  %arrayidx = getelementptr inbounds i32, i32* %1, i64 %indvars.iv
+  store i32 %call6, i32* %arrayidx, align 4
+  %indvars.iv.next = add nuw nsw i64 %indvars.iv, 1
+  %exitcond = icmp eq i64 %indvars.iv.next, %wide.trip.count
+  br i1 %exitcond, label %for.cond.cleanup, label %for.body
+}
+
+declare signext i32 @UseAlloca(i32*)