[Scheduling] Add a mutation to schedule GOT indirect instructions close to each other for linker optimization

[steven.zhang]

Summary

The linker optimization for pcrel load/store compiler side change has been added by D79864. However, we still failed to perform this optimization for some simple case:

pld 3, m@got@pcrel(0), 1
li 4, 4
stw 4, 0(3)
blr

And this is what linker want to do:

10010600:	02 00 10 06 	pstw    r4,132008
10010604:	04 00 80 38 	li      r4,4
10010608:	a8 03 80 90
1001060c:	00 00 00 60 	nop
10010610:	20 00 80 4e 	blr

The li 4, 4 here disable the optimization as the above transformation is wrong because we are define the r4 in betwwen pld and stw. So, we want to schedule such kind of instructions before the pld to avoid this happens.

[steven.zhang]

llvm/test/CodeGen/PowerPC/pcrel-tail-calls.ll

24

Is it a bug of our peephole that miss to create the relocation symbol ?

[steven.zhang]

Update tests.

[steven.zhang]

Gentle ping...

[nemanjai]

The fact that we catch all these additional opportunities is great. I don't fully understand all the code in the patch because I am not all that familiar with the scheduler so I'd like @stefanp to have a look at this as well.
I do have some nits inline and I'd like to see some tests that exercise the code paths where we can't reorder instructions to allow the optimization (i.e. cases where we return nullptr from getPLDpcAndLdStPair() as well as cases where we are unable to add the edges in schedHazardInsts()).

llvm/lib/Target/PowerPC/PPCMachineScheduler.cpp

165

It seems to me like this function could be documented a bit better to explain not only what we are doing but why. Of course, this may just be because I am not familiar with the scheduler so the code isn't self-documenting to me.

276

Sorry, I don't think this comment is clear enough. What is redefined? The concept of redefinition seems at odds with SSA.

292

This appears to make the assumption that operand 2 is the base address register. The comment should probably mention that.

[steven.zhang]

Simplify the logic and add tests. We still have cases that Peephole cannot handle due to missing the dependency information. But we don't want to handle it inside the scheduler as that is not right.

[steven.zhang]

llvm/test/CodeGen/PowerPC/pcrel-tail-calls.ll

26

std and pstd is not aliased. We can still do the linker opt for it but now, we can't as peephole didn't know they are not aliased. Further, we didn't emit the symbol because we have uses in-between pld and std which is too conservative also. If the other instr is store, we won't have problems if there is any uses in-between the pair.

[steven.zhang]

Gentle ping ...

[stefanp]

Overall I think that the patch looks good. I just had a couple of comments.
Thank you for finding the issues related to the peephole!

llvm/lib/Target/PowerPC/PPCMachineScheduler.cpp

192

Do we need to check that there isn't already a dependency between Pred and PLDpc?

If PLDpc already depends on Pred then there is nothing more to do and you probably don't need the artificial edge.
If Pred depends on PLDpc then we cannot add a new dependency because that would create a circular dependency.

198

I agree. We do need to merge the functionality of hasPCRelForm from here and hasPCRelativeForm into one place. (In a different patch though...)

272

nit:
Assume

llvm/test/CodeGen/PowerPC/pcrel-tail-calls.ll

26

Yes there is room for improvement in the peephole because we do not check aliases at all. This could potentially be a bug if we clobber memory without knowing it. Good catch!
The peephole also does not differentiate between loads and stores. I agree it is safe to have uses if the instruction is a store. However, it is not safe to have uses if the second instruction is a load.

[steven.zhang]

llvm/lib/Target/PowerPC/PPCMachineScheduler.cpp

192

Do we need to check that there isn't already a dependency between Pred and PLDpc?

Yes

If PLDpc already depends on Pred then there is nothing more to do and you probably don't need the artificial edge.

We need to check all the preds of PLDpc which will take up some time. And I think, there is not big problem to have an artificial edge here.

If Pred depends on PLDpc then we cannot add a new dependency because that would create a circular dependency.

It has been already checked by addEdge.

198

Agree

llvm/test/CodeGen/PowerPC/pcrel-tail-calls.ll

26

Yes, we need some follow up patch to fix this issue as such kind of bug is really hard to address.

[steven.zhang]

Address comments.

[steven.zhang]

Gentle ping...

[steven.zhang]

ping...

[steven.zhang]

ping

llvm/lib/Target/PowerPC/PPCMachineScheduler.h

protected:
   SUnit *pickNode(bool &IsTopNode) override;
   void enterMBB(MachineBasicBlock *MBB) override;
   void leaveMBB() override;
 
   void tryCandidate(SchedCandidate &Cand, SchedCandidate &TryCand) override;
   bool biasAddiCandidate(SchedCandidate &Cand, SchedCandidate &TryCand) const;
 };
 
+std::unique_ptr<ScheduleDAGMutation> createLinkerOptDAGMutation();
 } // end namespace llvm
 
 #endif // LLVM_LIB_TARGET_POWERPC_POWERPCMACHINESCHEDULER_H

llvm/lib/Target/PowerPC/PPCMachineScheduler.cpp

 //===- PPCMachineScheduler.cpp - MI Scheduler for PowerPC -------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "PPCMachineScheduler.h"
 #include "MCTargetDesc/PPCMCTargetDesc.h"
+#include "PPC.h"
+#include "llvm/CodeGen/TargetInstrInfo.h"
 
 using namespace llvm;
 
+#define DEBUG_TYPE "machine-scheduler"
+
 static cl::opt<bool> 
 DisableAddiLoadHeuristic("disable-ppc-sched-addi-load",
                          cl::desc("Disable scheduling addi instruction before" 
                                   "load for ppc"), cl::Hidden);
 static cl::opt<bool>
     EnableAddiHeuristic("ppc-postra-bias-addi",
                         cl::desc("Enable scheduling addi instruction as early"
                                  "as possible post ra"),
                         cl::Hidden, cl::init(true));
 
+static cl::opt<bool> EnableLinkerOptMutation(
+    "ppc-sched-linker-opt",
+    cl::desc("Enable the mutation for linker optimization"), cl::Hidden,
+    cl::init(true));
+
 static bool isADDIInstr(const GenericScheduler::SchedCandidate &Cand) {
   return Cand.SU->getInstr()->getOpcode() == PPC::ADDI ||
          Cand.SU->getInstr()->getOpcode() == PPC::ADDI8;
 }
 
 bool PPCPreRASchedStrategy::biasAddiLoadCandidate(SchedCandidate &Cand,
                                                   SchedCandidate &TryCand,
                                                   SchedBoundary &Zone) const {
void PPCPostRASchedStrategy::initialize(ScheduleDAGMI *Dag) {
   PostGenericScheduler::initialize(Dag);
 }
 
 SUnit *PPCPostRASchedStrategy::pickNode(bool &IsTopNode) {
   // Custom PPC PostRA specific scheduling here.
   return PostGenericScheduler::pickNode(IsTopNode);
 }
 
+//===----------------------------------------------------------------------===//
+// PPCLinkerOptMutation - DAG post-processing for linker opt.
+//===----------------------------------------------------------------------===//
+namespace {
+
+// x = PLDpc (GOT)
+// ...
+// stw y, off(x)
+//
+// We will create the symbol has MO_PCREL_OPT_FLAG flag for linker to do the
+// optimization if it meets the following conditions:
+//  1. The first instruction is PLDpc and it is loaded from GOT.
+//  2. The PLDpc has only single use.
+//  3. The single user of the PLDpc must have Pcrel form.
+//  4. No use/def of the user first operand is allowed in-between the PLDpc
+//     and the single user.
+// So, we need to move all the hazard instructions in-between the PLDpc and the
+// store.
+class PPCLinkerOptMutation : public ScheduleDAGMutation {
+public:
+private:
+  SUnit *getPLDpcAndLdStPair(SUnit &SU) const;
+  void schedHazardInsts(ScheduleDAGInstrs &DAG, SUnit &PLDpc, SUnit &LdSt);
+
+protected:
+  void apply(ScheduleDAGInstrs *DAG) override;
+};
+
+void PPCLinkerOptMutation::apply(ScheduleDAGInstrs *DAG) {
+  for (SUnit &PLDpc : DAG->SUnits)
+    if (SUnit *LdSt = getPLDpcAndLdStPair(PLDpc))
+      schedHazardInsts(*DAG, PLDpc, *LdSt);
+}
+
+// Create edges from PLDpc to all the pred hazard SU of the LdSt so that they
+// are scheduled before the PLDpc.
+void PPCLinkerOptMutation::schedHazardInsts(ScheduleDAGInstrs &DAG,
+                                            SUnit &PLDpc, SUnit &LdSt) {
+  LLVM_DEBUG(dbgs() << "Linker Opt: "; DAG.dumpNodeName(PLDpc); dbgs() << " - ";
+             DAG.dumpNodeName(LdSt); dbgs() << " /  ";
+             dbgs() << DAG.TII->getName(PLDpc.getInstr()->getOpcode()) << " - "
+                    << DAG.TII->getName(LdSt.getInstr()->getOpcode()) << '\n';);
+  assert(LdSt.getInstr()->mayLoadOrStore() && "Load/Store instr expected");
+  //                +-----------+ new pred edge
+  //                | x = PLDpc +-----------+
+  //                +-----^-----+           |
+  //                      |                 v
+  //                      |             +---+----+
+  //                      |             | inst Y |
+  //                      |             +---^----+
+  //                +-----+-----+           |
+  //                |  stw y, x +-----------+
+  //                +-----------+
+  // Create pred edges from PLDpc to the preds(inst Y) of the stw so that,
+  // these instructions are scheduled before PLDpc. Because, the linker
+  // will remove the stw and replace PLDpc with prefix store. It acts as
+  // scheduling the stw immediate after PLDpc.
+  for (SDep &Pred : LdSt.Preds) {
+    if (Pred.getSUnit() == &PLDpc)
+      continue;
+
+    LLVM_DEBUG(dbgs() << "  Copy Pred SU(" << Pred.getSUnit()->NodeNum
+                      << ")\n");
+    DAG.addEdge(&PLDpc, SDep(Pred.getSUnit(), SDep::Artificial));
+  }
+}
+
+static bool hasPCRelForm(const MachineInstr &MI) {
+  // TODO - Using InstrMapping to map the opcode for Pcrel form and non-Pcrel
+  // form.
+  switch (MI.getOpcode()) {
+  default:
+    return false;
+  case PPC::LBZ:
+  case PPC::LBZ8:
+  case PPC::LHA:
+  case PPC::LHA8:
+  case PPC::LHZ:
+  case PPC::LHZ8:
+  case PPC::LWZ:
+  case PPC::LWZ8:
+  case PPC::STB:
+  case PPC::STB8:
+  case PPC::STH:
+  case PPC::STH8:
+  case PPC::STW:
+  case PPC::STW8:
+  case PPC::LD:
+  case PPC::STD:
+  case PPC::LWA:
+  case PPC::LXSD:
+  case PPC::LXSSP:
+  case PPC::LXV:
+  case PPC::STXSD:
+  case PPC::STXSSP:
+  case PPC::STXV:
+  case PPC::LFD:
+  case PPC::LFS:
+  case PPC::STFD:
+  case PPC::STFS:
+  case PPC::DFLOADf32:
+  case PPC::DFLOADf64:
+  case PPC::DFSTOREf32:
+  case PPC::DFSTOREf64:
+    return true;
+  }
+}
+
+SUnit *PPCLinkerOptMutation::getPLDpcAndLdStPair(SUnit &SU) const {
+  // TODO - common the code with PPCPreEmitPeephole::isGOTPLDpc()
+  if (SU.getInstr()->getOpcode() != PPC::PLDpc)
+    return nullptr;
+
+  // The result must be a register.
+  const MachineOperand &DefMO = SU.getInstr()->getOperand(0);
+  if (!DefMO.isReg())
+    return nullptr;
+
+  // Make sure that this is a global symbol and it is loaded from GOT.
+  const MachineOperand &Global = SU.getInstr()->getOperand(1);
+  if (!Global.isGlobal() || !(Global.getTargetFlags() & PPCII::MO_GOT_FLAG))
+    return nullptr;
+
+  // Only when the PLDpc has single use so that, there is benefit to fold it
+  // into pcrel load/store and remove it.
+  SUnit *SingleUseSU = nullptr;
+  for (const SDep &Succ : SU.Succs) {
+    if (Succ.getKind() == SDep::Data) {
+      assert(Succ.getReg() == DefMO.getReg() && "Unexpected data dependency");
+      if (SingleUseSU)
+        return nullptr;
+      SingleUseSU = Succ.getSUnit();
+    }
+  }
+  if (!SingleUseSU)
+    return nullptr;
+
+  // The paired instruction must have pcrel form.
+  const MachineInstr *SingleUseMI = SingleUseSU->getInstr();
+  if (!hasPCRelForm(*SingleUseMI))
+    return nullptr;
+  assert(SingleUseMI->mayLoadOrStore() && "Unknown pcrel instruction");
+
+  // Assume that, the second operand is the base register of the load/store and
+  // it must be the result register of the PLDpc so that, it can be folded into
+  // pcrel load/store by linker.
+  if (!SingleUseMI->getOperand(2).isReg() ||
+      SingleUseMI->getOperand(2).getReg() != DefMO.getReg())
+    return nullptr;
+
+  return SingleUseSU;
+}
+
+} // end anonymous namespace
+
+namespace llvm {
+std::unique_ptr<ScheduleDAGMutation> createLinkerOptDAGMutation() {
+  return EnableLinkerOptMutation ? std::make_unique<PPCLinkerOptMutation>()
+                                 : nullptr;
+}
+} // end namespace llvm

llvm/lib/Target/PowerPC/PPCTargetMachine.cpp

ScheduleDAGMI *DAG =
     new ScheduleDAGMI(C, ST.usePPCPostRASchedStrategy() ?
                       std::make_unique<PPCPostRASchedStrategy>(C) :
                       std::make_unique<PostGenericScheduler>(C), true);
   // add DAG Mutations here.
   if (ST.hasStoreFusion())
     DAG->addMutation(createStoreClusterDAGMutation(DAG->TII, DAG->TRI));
   if (ST.hasFusion())
     DAG->addMutation(createPowerPCMacroFusionDAGMutation());
+  if (ST.isUsingPCRelativeCalls())
+    DAG->addMutation(createLinkerOptDAGMutation());
+
   return DAG;
 }
 
 // The FeatureString here is a little subtle. We are modifying the feature
 // string with what are (currently) non-function specific overrides as it goes
 // into the LLVMTargetMachine constructor and then using the stored value in the
 // Subtarget constructor below it.
 PPCTargetMachine::PPCTargetMachine(const Target &T, const Triple &TT,

llvm/test/CodeGen/PowerPC/pcrel-got-indirect.ll

entry:
   %0 = load i8, i8* @valChar, align 1
   %conv = zext i8 %0 to i32
   ret i32 %conv
 }
 
 define dso_local void @WriteGlobalVarChar() local_unnamed_addr  {
 ; LE-LABEL: WriteGlobalVarChar:
 ; LE:       # %bb.0: # %entry
-; LE-NEXT:    pld r3, valChar@got@pcrel(0), 1
 ; LE-NEXT:    li r4, 3
+; LE-NEXT:    pld r3, valChar@got@pcrel(0), 1
+; LE-NEXT:  .Lpcrel0:
+; LE-NEXT:    .reloc .Lpcrel0-8,R_PPC64_PCREL_OPT,.-(.Lpcrel0-8)
 ; LE-NEXT:    stb r4, 0(r3)
 ; LE-NEXT:    blr
 ;
 ; BE-LABEL: WriteGlobalVarChar:
 ; BE:       # %bb.0: # %entry
-; BE-NEXT:    pld r3, valChar@got@pcrel(0), 1
 ; BE-NEXT:    li r4, 3
+; BE-NEXT:    pld r3, valChar@got@pcrel(0), 1
+; BE-NEXT:  .Lpcrel0:
+; BE-NEXT:    .reloc .Lpcrel0-8,R_PPC64_PCREL_OPT,.-(.Lpcrel0-8)
 ; BE-NEXT:    stb r4, 0(r3)
 ; BE-NEXT:    blr
 entry:
   store i8 3, i8* @valChar, align 1
   ret void
 }
 
 define dso_local signext i32 @ReadGlobalVarShort() local_unnamed_addr  {
 ; LE-LABEL: ReadGlobalVarShort:
 ; LE:       # %bb.0: # %entry
 ; LE-NEXT:    pld r3, valShort@got@pcrel(0), 1
-; LE-NEXT:  .Lpcrel0:
-; LE-NEXT:    .reloc .Lpcrel0-8,R_PPC64_PCREL_OPT,.-(.Lpcrel0-8)
+; LE-NEXT:  .Lpcrel1:
+; LE-NEXT:    .reloc .Lpcrel1-8,R_PPC64_PCREL_OPT,.-(.Lpcrel1-8)
 ; LE-NEXT:    lha r3, 0(r3)
 ; LE-NEXT:    blr
 ;
 ; BE-LABEL: ReadGlobalVarShort:
 ; BE:       # %bb.0: # %entry
 ; BE-NEXT:    pld r3, valShort@got@pcrel(0), 1
-; BE-NEXT:  .Lpcrel0:
-; BE-NEXT:    .reloc .Lpcrel0-8,R_PPC64_PCREL_OPT,.-(.Lpcrel0-8)
+; BE-NEXT:  .Lpcrel1:
+; BE-NEXT:    .reloc .Lpcrel1-8,R_PPC64_PCREL_OPT,.-(.Lpcrel1-8)
 ; BE-NEXT:    lha r3, 0(r3)
 ; BE-NEXT:    blr
 entry:
   %0 = load i16, i16* @valShort, align 2
   %conv = sext i16 %0 to i32
   ret i32 %conv
 }
 
 define dso_local void @WriteGlobalVarShort() local_unnamed_addr  {
 ; LE-LABEL: WriteGlobalVarShort:
 ; LE:       # %bb.0: # %entry
-; LE-NEXT:    pld r3, valShort@got@pcrel(0), 1
 ; LE-NEXT:    li r4, 3
+; LE-NEXT:    pld r3, valShort@got@pcrel(0), 1
+; LE-NEXT:  .Lpcrel2:
+; LE-NEXT:    .reloc .Lpcrel2-8,R_PPC64_PCREL_OPT,.-(.Lpcrel2-8)
 ; LE-NEXT:    sth r4, 0(r3)
 ; LE-NEXT:    blr
 ;
 ; BE-LABEL: WriteGlobalVarShort:
 ; BE:       # %bb.0: # %entry
-; BE-NEXT:    pld r3, valShort@got@pcrel(0), 1
 ; BE-NEXT:    li r4, 3
+; BE-NEXT:    pld r3, valShort@got@pcrel(0), 1
+; BE-NEXT:  .Lpcrel2:
+; BE-NEXT:    .reloc .Lpcrel2-8,R_PPC64_PCREL_OPT,.-(.Lpcrel2-8)
 ; BE-NEXT:    sth r4, 0(r3)
 ; BE-NEXT:    blr
 entry:
   store i16 3, i16* @valShort, align 2
   ret void
 }
 
 define dso_local signext i32 @ReadGlobalVarInt() local_unnamed_addr  {
 ; LE-LABEL: ReadGlobalVarInt:
 ; LE:       # %bb.0: # %entry
 ; LE-NEXT:    pld r3, valInt@got@pcrel(0), 1
-; LE-NEXT:  .Lpcrel1:
-; LE-NEXT:    .reloc .Lpcrel1-8,R_PPC64_PCREL_OPT,.-(.Lpcrel1-8)
+; LE-NEXT:  .Lpcrel3:
+; LE-NEXT:    .reloc .Lpcrel3-8,R_PPC64_PCREL_OPT,.-(.Lpcrel3-8)
 ; LE-NEXT:    lwa r3, 0(r3)
 ; LE-NEXT:    blr
 ;
 ; BE-LABEL: ReadGlobalVarInt:
 ; BE:       # %bb.0: # %entry
 ; BE-NEXT:    pld r3, valInt@got@pcrel(0), 1
-; BE-NEXT:  .Lpcrel1:
-; BE-NEXT:    .reloc .Lpcrel1-8,R_PPC64_PCREL_OPT,.-(.Lpcrel1-8)
+; BE-NEXT:  .Lpcrel3:
+; BE-NEXT:    .reloc .Lpcrel3-8,R_PPC64_PCREL_OPT,.-(.Lpcrel3-8)
 ; BE-NEXT:    lwa r3, 0(r3)
 ; BE-NEXT:    blr
 entry:
   %0 = load i32, i32* @valInt, align 4
   ret i32 %0
 }
 
 define dso_local void @WriteGlobalVarInt() local_unnamed_addr  {
 ; LE-LABEL: WriteGlobalVarInt:
 ; LE:       # %bb.0: # %entry
-; LE-NEXT:    pld r3, valInt@got@pcrel(0), 1
 ; LE-NEXT:    li r4, 33
+; LE-NEXT:    pld r3, valInt@got@pcrel(0), 1
+; LE-NEXT:  .Lpcrel4:
+; LE-NEXT:    .reloc .Lpcrel4-8,R_PPC64_PCREL_OPT,.-(.Lpcrel4-8)
 ; LE-NEXT:    stw r4, 0(r3)
 ; LE-NEXT:    blr
 ;
 ; BE-LABEL: WriteGlobalVarInt:
 ; BE:       # %bb.0: # %entry
-; BE-NEXT:    pld r3, valInt@got@pcrel(0), 1
 ; BE-NEXT:    li r4, 33
+; BE-NEXT:    pld r3, valInt@got@pcrel(0), 1
+; BE-NEXT:  .Lpcrel4:
+; BE-NEXT:    .reloc .Lpcrel4-8,R_PPC64_PCREL_OPT,.-(.Lpcrel4-8)
 ; BE-NEXT:    stw r4, 0(r3)
 ; BE-NEXT:    blr
 entry:
   store i32 33, i32* @valInt, align 4
   ret void
 }
 
 define dso_local signext i32 @ReadGlobalVarUnsigned() local_unnamed_addr  {
 ; LE-LABEL: ReadGlobalVarUnsigned:
 ; LE:       # %bb.0: # %entry
 ; LE-NEXT:    pld r3, valUnsigned@got@pcrel(0), 1
-; LE-NEXT:  .Lpcrel2:
-; LE-NEXT:    .reloc .Lpcrel2-8,R_PPC64_PCREL_OPT,.-(.Lpcrel2-8)
+; LE-NEXT:  .Lpcrel5:
+; LE-NEXT:    .reloc .Lpcrel5-8,R_PPC64_PCREL_OPT,.-(.Lpcrel5-8)
 ; LE-NEXT:    lwa r3, 0(r3)
 ; LE-NEXT:    blr
 ;
 ; BE-LABEL: ReadGlobalVarUnsigned:
 ; BE:       # %bb.0: # %entry
 ; BE-NEXT:    pld r3, valUnsigned@got@pcrel(0), 1
-; BE-NEXT:  .Lpcrel2:
-; BE-NEXT:    .reloc .Lpcrel2-8,R_PPC64_PCREL_OPT,.-(.Lpcrel2-8)
+; BE-NEXT:  .Lpcrel5:
+; BE-NEXT:    .reloc .Lpcrel5-8,R_PPC64_PCREL_OPT,.-(.Lpcrel5-8)
 ; BE-NEXT:    lwa r3, 0(r3)
 ; BE-NEXT:    blr
 entry:
   %0 = load i32, i32* @valUnsigned, align 4
   ret i32 %0
 }
 
 define dso_local void @WriteGlobalVarUnsigned() local_unnamed_addr  {
 ; LE-LABEL: WriteGlobalVarUnsigned:
 ; LE:       # %bb.0: # %entry
-; LE-NEXT:    pld r3, valUnsigned@got@pcrel(0), 1
 ; LE-NEXT:    li r4, 33
+; LE-NEXT:    pld r3, valUnsigned@got@pcrel(0), 1
+; LE-NEXT:  .Lpcrel6:
+; LE-NEXT:    .reloc .Lpcrel6-8,R_PPC64_PCREL_OPT,.-(.Lpcrel6-8)
 ; LE-NEXT:    stw r4, 0(r3)
 ; LE-NEXT:    blr
 ;
 ; BE-LABEL: WriteGlobalVarUnsigned:
 ; BE:       # %bb.0: # %entry
-; BE-NEXT:    pld r3, valUnsigned@got@pcrel(0), 1
 ; BE-NEXT:    li r4, 33
+; BE-NEXT:    pld r3, valUnsigned@got@pcrel(0), 1
+; BE-NEXT:  .Lpcrel6:
+; BE-NEXT:    .reloc .Lpcrel6-8,R_PPC64_PCREL_OPT,.-(.Lpcrel6-8)
 ; BE-NEXT:    stw r4, 0(r3)
 ; BE-NEXT:    blr
 entry:
   store i32 33, i32* @valUnsigned, align 4
   ret void
 }
 
 define dso_local signext i32 @ReadGlobalVarLong() local_unnamed_addr  {
 ; LE-LABEL: ReadGlobalVarLong:
 ; LE:       # %bb.0: # %entry
 ; LE-NEXT:    pld r3, valLong@got@pcrel(0), 1
-; LE-NEXT:  .Lpcrel3:
-; LE-NEXT:    .reloc .Lpcrel3-8,R_PPC64_PCREL_OPT,.-(.Lpcrel3-8)
+; LE-NEXT:  .Lpcrel7:
+; LE-NEXT:    .reloc .Lpcrel7-8,R_PPC64_PCREL_OPT,.-(.Lpcrel7-8)
 ; LE-NEXT:    lwa r3, 0(r3)
 ; LE-NEXT:    blr
 ;
 ; BE-LABEL: ReadGlobalVarLong:
 ; BE:       # %bb.0: # %entry
 ; BE-NEXT:    pld r3, valLong@got@pcrel(0), 1
-; BE-NEXT:  .Lpcrel3:
-; BE-NEXT:    .reloc .Lpcrel3-8,R_PPC64_PCREL_OPT,.-(.Lpcrel3-8)
+; BE-NEXT:  .Lpcrel7:
+; BE-NEXT:    .reloc .Lpcrel7-8,R_PPC64_PCREL_OPT,.-(.Lpcrel7-8)
 ; BE-NEXT:    lwa r3, 4(r3)
 ; BE-NEXT:    blr
 entry:
   %0 = load i64, i64* @valLong, align 8
   %conv = trunc i64 %0 to i32
   ret i32 %conv
 }
 
 define dso_local void @WriteGlobalVarLong() local_unnamed_addr  {
 ; LE-LABEL: WriteGlobalVarLong:
 ; LE:       # %bb.0: # %entry
-; LE-NEXT:    pld r3, valLong@got@pcrel(0), 1
 ; LE-NEXT:    li r4, 3333
+; LE-NEXT:    pld r3, valLong@got@pcrel(0), 1
+; LE-NEXT:  .Lpcrel8:
+; LE-NEXT:    .reloc .Lpcrel8-8,R_PPC64_PCREL_OPT,.-(.Lpcrel8-8)
 ; LE-NEXT:    std r4, 0(r3)
 ; LE-NEXT:    blr
 ;
 ; BE-LABEL: WriteGlobalVarLong:
 ; BE:       # %bb.0: # %entry
-; BE-NEXT:    pld r3, valLong@got@pcrel(0), 1
 ; BE-NEXT:    li r4, 3333
+; BE-NEXT:    pld r3, valLong@got@pcrel(0), 1
+; BE-NEXT:  .Lpcrel8:
+; BE-NEXT:    .reloc .Lpcrel8-8,R_PPC64_PCREL_OPT,.-(.Lpcrel8-8)
 ; BE-NEXT:    std r4, 0(r3)
 ; BE-NEXT:    blr
 entry:
   store i64 3333, i64* @valLong, align 8
   ret void
 }
 
 define dso_local i32* @ReadGlobalPtr() local_unnamed_addr  {
 ; LE-LABEL: ReadGlobalPtr:
 ; LE:       # %bb.0: # %entry
 ; LE-NEXT:    pld r3, ptr@got@pcrel(0), 1
-; LE-NEXT:  .Lpcrel4:
-; LE-NEXT:    .reloc .Lpcrel4-8,R_PPC64_PCREL_OPT,.-(.Lpcrel4-8)
+; LE-NEXT:  .Lpcrel9:
+; LE-NEXT:    .reloc .Lpcrel9-8,R_PPC64_PCREL_OPT,.-(.Lpcrel9-8)
 ; LE-NEXT:    ld r3, 0(r3)
 ; LE-NEXT:    blr
 ;
 ; BE-LABEL: ReadGlobalPtr:
 ; BE:       # %bb.0: # %entry
 ; BE-NEXT:    pld r3, ptr@got@pcrel(0), 1
-; BE-NEXT:  .Lpcrel4:
-; BE-NEXT:    .reloc .Lpcrel4-8,R_PPC64_PCREL_OPT,.-(.Lpcrel4-8)
+; BE-NEXT:  .Lpcrel9:
+; BE-NEXT:    .reloc .Lpcrel9-8,R_PPC64_PCREL_OPT,.-(.Lpcrel9-8)
 ; BE-NEXT:    ld r3, 0(r3)
 ; BE-NEXT:    blr
 entry:
   %0 = load i32*, i32** @ptr, align 8
   ret i32* %0
 }
 
 define dso_local void @WriteGlobalPtr() local_unnamed_addr  {
 ; LE-LABEL: WriteGlobalPtr:
 ; LE:       # %bb.0: # %entry
 ; LE-NEXT:    pld r3, ptr@got@pcrel(0), 1
-; LE-NEXT:  .Lpcrel5:
+; LE-NEXT:  .Lpcrel10:
 ; LE-NEXT:    li r4, 3
-; LE-NEXT:    .reloc .Lpcrel5-8,R_PPC64_PCREL_OPT,.-(.Lpcrel5-8)
+; LE-NEXT:    .reloc .Lpcrel10-8,R_PPC64_PCREL_OPT,.-(.Lpcrel10-8)
 ; LE-NEXT:    ld r3, 0(r3)
 ; LE-NEXT:    stw r4, 0(r3)
 ; LE-NEXT:    blr
 ;
 ; BE-LABEL: WriteGlobalPtr:
 ; BE:       # %bb.0: # %entry
 ; BE-NEXT:    pld r3, ptr@got@pcrel(0), 1
-; BE-NEXT:  .Lpcrel5:
+; BE-NEXT:  .Lpcrel10:
 ; BE-NEXT:    li r4, 3
-; BE-NEXT:    .reloc .Lpcrel5-8,R_PPC64_PCREL_OPT,.-(.Lpcrel5-8)
+; BE-NEXT:    .reloc .Lpcrel10-8,R_PPC64_PCREL_OPT,.-(.Lpcrel10-8)
 ; BE-NEXT:    ld r3, 0(r3)
 ; BE-NEXT:    stw r4, 0(r3)
 ; BE-NEXT:    blr
 entry:
   %0 = load i32*, i32** @ptr, align 8
   store i32 3, i32* %0, align 4
   ret void
 }

 entry:
   ret i32* @valInt
 }
 
 define dso_local signext i32 @ReadGlobalArray() local_unnamed_addr  {
 ; LE-LABEL: ReadGlobalArray:
 ; LE:       # %bb.0: # %entry
 ; LE-NEXT:    pld r3, array@got@pcrel(0), 1
-; LE-NEXT:  .Lpcrel6:
-; LE-NEXT:    .reloc .Lpcrel6-8,R_PPC64_PCREL_OPT,.-(.Lpcrel6-8)
+; LE-NEXT:  .Lpcrel11:
+; LE-NEXT:    .reloc .Lpcrel11-8,R_PPC64_PCREL_OPT,.-(.Lpcrel11-8)
 ; LE-NEXT:    lwa r3, 12(r3)
 ; LE-NEXT:    blr
 ;
 ; BE-LABEL: ReadGlobalArray:
 ; BE:       # %bb.0: # %entry
 ; BE-NEXT:    pld r3, array@got@pcrel(0), 1
-; BE-NEXT:  .Lpcrel6:
-; BE-NEXT:    .reloc .Lpcrel6-8,R_PPC64_PCREL_OPT,.-(.Lpcrel6-8)
+; BE-NEXT:  .Lpcrel11:
+; BE-NEXT:    .reloc .Lpcrel11-8,R_PPC64_PCREL_OPT,.-(.Lpcrel11-8)
 ; BE-NEXT:    lwa r3, 12(r3)
 ; BE-NEXT:    blr
 entry:
   %0 = load i32, i32* getelementptr inbounds ([10 x i32], [10 x i32]* @array, i64 0, i64 3), align 4
   ret i32 %0
 }
 
 define dso_local void @WriteGlobalArray() local_unnamed_addr  {
 ; LE-LABEL: WriteGlobalArray:
 ; LE:       # %bb.0: # %entry
-; LE-NEXT:    pld r3, array@got@pcrel(0), 1
 ; LE-NEXT:    li r4, 5
+; LE-NEXT:    pld r3, array@got@pcrel(0), 1
+; LE-NEXT:  .Lpcrel12:
+; LE-NEXT:    .reloc .Lpcrel12-8,R_PPC64_PCREL_OPT,.-(.Lpcrel12-8)
 ; LE-NEXT:    stw r4, 12(r3)
 ; LE-NEXT:    blr
 ;
 ; BE-LABEL: WriteGlobalArray:
 ; BE:       # %bb.0: # %entry
-; BE-NEXT:    pld r3, array@got@pcrel(0), 1
 ; BE-NEXT:    li r4, 5
+; BE-NEXT:    pld r3, array@got@pcrel(0), 1
+; BE-NEXT:  .Lpcrel12:
+; BE-NEXT:    .reloc .Lpcrel12-8,R_PPC64_PCREL_OPT,.-(.Lpcrel12-8)
 ; BE-NEXT:    stw r4, 12(r3)
 ; BE-NEXT:    blr
 entry:
   store i32 5, i32* getelementptr inbounds ([10 x i32], [10 x i32]* @array, i64 0, i64 3), align 4
   ret void
 }
 
 define dso_local signext i32 @ReadGlobalStruct() local_unnamed_addr  {
 ; LE-LABEL: ReadGlobalStruct:
 ; LE:       # %bb.0: # %entry
 ; LE-NEXT:    pld r3, structure@got@pcrel(0), 1
-; LE-NEXT:  .Lpcrel7:
-; LE-NEXT:    .reloc .Lpcrel7-8,R_PPC64_PCREL_OPT,.-(.Lpcrel7-8)
+; LE-NEXT:  .Lpcrel13:
+; LE-NEXT:    .reloc .Lpcrel13-8,R_PPC64_PCREL_OPT,.-(.Lpcrel13-8)
 ; LE-NEXT:    lwa r3, 4(r3)
 ; LE-NEXT:    blr
 ;
 ; BE-LABEL: ReadGlobalStruct:
 ; BE:       # %bb.0: # %entry
 ; BE-NEXT:    pld r3, structure@got@pcrel(0), 1
-; BE-NEXT:  .Lpcrel7:
-; BE-NEXT:    .reloc .Lpcrel7-8,R_PPC64_PCREL_OPT,.-(.Lpcrel7-8)
+; BE-NEXT:  .Lpcrel13:
+; BE-NEXT:    .reloc .Lpcrel13-8,R_PPC64_PCREL_OPT,.-(.Lpcrel13-8)
 ; BE-NEXT:    lwa r3, 4(r3)
 ; BE-NEXT:    blr
 entry:
   %0 = load i32, i32* getelementptr inbounds (%struct.Struct, %struct.Struct* @structure, i64 0, i32 2), align 4
   ret i32 %0
 }
 
 define dso_local void @WriteGlobalStruct() local_unnamed_addr  {
 ; LE-LABEL: WriteGlobalStruct:
 ; LE:       # %bb.0: # %entry
-; LE-NEXT:    pld r3, structure@got@pcrel(0), 1
 ; LE-NEXT:    li r4, 3
+; LE-NEXT:    pld r3, structure@got@pcrel(0), 1
+; LE-NEXT:  .Lpcrel14:
+; LE-NEXT:    .reloc .Lpcrel14-8,R_PPC64_PCREL_OPT,.-(.Lpcrel14-8)
 ; LE-NEXT:    stw r4, 4(r3)
 ; LE-NEXT:    blr
 ;
 ; BE-LABEL: WriteGlobalStruct:
 ; BE:       # %bb.0: # %entry
-; BE-NEXT:    pld r3, structure@got@pcrel(0), 1
 ; BE-NEXT:    li r4, 3
+; BE-NEXT:    pld r3, structure@got@pcrel(0), 1
+; BE-NEXT:  .Lpcrel14:
+; BE-NEXT:    .reloc .Lpcrel14-8,R_PPC64_PCREL_OPT,.-(.Lpcrel14-8)
 ; BE-NEXT:    stw r4, 4(r3)
 ; BE-NEXT:    blr
 entry:
   store i32 3, i32* getelementptr inbounds (%struct.Struct, %struct.Struct* @structure, i64 0, i32 2), align 4
   ret void
 }
 
 define dso_local void @ReadFuncPtr() local_unnamed_addr  {
 ; LE-LABEL: ReadFuncPtr:
 ; LE:         .localentry ReadFuncPtr, 1
 ; LE-NEXT:  # %bb.0: # %entry
 ; LE-NEXT:    pld r3, ptrfunc@got@pcrel(0), 1
-; LE-NEXT:  .Lpcrel8:
-; LE-NEXT:    .reloc .Lpcrel8-8,R_PPC64_PCREL_OPT,.-(.Lpcrel8-8)
+; LE-NEXT:  .Lpcrel15:
+; LE-NEXT:    .reloc .Lpcrel15-8,R_PPC64_PCREL_OPT,.-(.Lpcrel15-8)
 ; LE-NEXT:    ld r12, 0(r3)
 ; LE-NEXT:    mtctr r12
 ; LE-NEXT:    bctr
 ; LE-NEXT:    #TC_RETURNr8 ctr 0
 ;
 ; BE-LABEL: ReadFuncPtr:
 ; BE:         .localentry ReadFuncPtr, 1
 ; BE-NEXT:  # %bb.0: # %entry
 ; BE-NEXT:    pld r3, ptrfunc@got@pcrel(0), 1
-; BE-NEXT:  .Lpcrel8:
-; BE-NEXT:    .reloc .Lpcrel8-8,R_PPC64_PCREL_OPT,.-(.Lpcrel8-8)
+; BE-NEXT:  .Lpcrel15:
+; BE-NEXT:    .reloc .Lpcrel15-8,R_PPC64_PCREL_OPT,.-(.Lpcrel15-8)
 ; BE-NEXT:    ld r12, 0(r3)
 ; BE-NEXT:    mtctr r12
 ; BE-NEXT:    bctr
 ; BE-NEXT:    #TC_RETURNr8 ctr 0
 entry:
   %0 = load void ()*, void ()** bitcast (void (...)** @ptrfunc to void ()**), align 8
   tail call void %0()
   ret void
 }
 
+; FIXME - The linker opt relocation symbol is missing for pld - std pair.
 define dso_local void @WriteFuncPtr() local_unnamed_addr  {
 ; LE-LABEL: WriteFuncPtr:
 ; LE:       # %bb.0: # %entry
-; LE-NEXT:    pld r3, ptrfunc@got@pcrel(0), 1
 ; LE-NEXT:    pld r4, function@got@pcrel(0), 1
+; LE-NEXT:    pld r3, ptrfunc@got@pcrel(0), 1
 ; LE-NEXT:    std r4, 0(r3)
 ; LE-NEXT:    blr
 ;
 ; BE-LABEL: WriteFuncPtr:
 ; BE:       # %bb.0: # %entry
-; BE-NEXT:    pld r3, ptrfunc@got@pcrel(0), 1
 ; BE-NEXT:    pld r4, function@got@pcrel(0), 1
+; BE-NEXT:    pld r3, ptrfunc@got@pcrel(0), 1
 ; BE-NEXT:    std r4, 0(r3)
 ; BE-NEXT:    blr
 entry:
   store void (...)* @function, void (...)** @ptrfunc, align 8
   ret void
 }
 
 declare void @function(...)
 

llvm/test/CodeGen/PowerPC/pcrel-linkeropt.ll

 @FuncPtrIn = external local_unnamed_addr global void (...)*, align 8
 @FuncPtrOut = external local_unnamed_addr global void (...)*, align 8
 
 define dso_local void @ReadWrite8() local_unnamed_addr #0 {
 ; CHECK-LABEL: ReadWrite8:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    pld r3, input8@got@pcrel(0), 1
 ; CHECK-NEXT:  .Lpcrel:
-; CHECK-NEXT:    pld r4, output8@got@pcrel(0), 1
 ; CHECK-NEXT:    .reloc .Lpcrel-8,R_PPC64_PCREL_OPT,.-(.Lpcrel-8)
 ; CHECK-NEXT:    lbz r3, 0(r3)
-; In this test the stb r3, 0(r4) cannot be optimized because it
-; uses the register r3 and that register is defined by lbz r3, 0(r3)
-; which is defined between the pld and the stb.
+; CHECK-NEXT:    pld r4, output8@got@pcrel(0), 1
+; CHECK-NEXT:  .Lpcrel0:
+; CHECK-NEXT:    .reloc .Lpcrel0-8,R_PPC64_PCREL_OPT,.-(.Lpcrel0-8)
 ; CHECK-NEXT:    stb r3, 0(r4)
 ; CHECK-NEXT:    blr
 entry:
   %0 = load i8, i8* @input8, align 1
   store i8 %0, i8* @output8, align 1
   ret void
 }
 
 define dso_local void @ReadWrite16() local_unnamed_addr #0 {
 ; CHECK-LABEL: ReadWrite16:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    pld r3, input16@got@pcrel(0), 1
-; CHECK-NEXT:  .Lpcrel0:
-; CHECK-NEXT:    pld r4, output16@got@pcrel(0), 1
-; CHECK-NEXT:    .reloc .Lpcrel0-8,R_PPC64_PCREL_OPT,.-(.Lpcrel0-8)
+; CHECK-NEXT:  .Lpcrel1:
+; CHECK-NEXT:    .reloc .Lpcrel1-8,R_PPC64_PCREL_OPT,.-(.Lpcrel1-8)
 ; CHECK-NEXT:    lhz r3, 0(r3)
-; In this test the sth r3, 0(r4) cannot be optimized because it
-; uses the register r3 and that register is defined by lhz r3, 0(r3)
-; which is defined between the pld and the sth.
+; CHECK-NEXT:    pld r4, output16@got@pcrel(0), 1
+; CHECK-NEXT:  .Lpcrel2:
+; CHECK-NEXT:    .reloc .Lpcrel2-8,R_PPC64_PCREL_OPT,.-(.Lpcrel2-8)
 ; CHECK-NEXT:    sth r3, 0(r4)
 ; CHECK-NEXT:    blr
 entry:
   %0 = load i16, i16* @input16, align 2
   store i16 %0, i16* @output16, align 2
   ret void
 }
 
 define dso_local void @ReadWrite32() local_unnamed_addr #0 {
 ; CHECK-LABEL: ReadWrite32:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    pld r3, input32@got@pcrel(0), 1
-; CHECK-NEXT:  .Lpcrel1:
-; CHECK-NEXT:    pld r4, output32@got@pcrel(0), 1
-; CHECK-NEXT:    .reloc .Lpcrel1-8,R_PPC64_PCREL_OPT,.-(.Lpcrel1-8)
+; CHECK-NEXT:  .Lpcrel3:
+; CHECK-NEXT:    .reloc .Lpcrel3-8,R_PPC64_PCREL_OPT,.-(.Lpcrel3-8)
 ; CHECK-NEXT:    lwz r3, 0(r3)
+; CHECK-NEXT:    pld r4, output32@got@pcrel(0), 1
+; CHECK-NEXT:  .Lpcrel4:
+; CHECK-NEXT:    .reloc .Lpcrel4-8,R_PPC64_PCREL_OPT,.-(.Lpcrel4-8)
 ; CHECK-NEXT:    stw r3, 0(r4)
 ; CHECK-NEXT:    blr
 entry:
   %0 = load i32, i32* @input32, align 4
   store i32 %0, i32* @output32, align 4
   ret void
 }
 
 define dso_local void @ReadWrite64() local_unnamed_addr #0 {
 ; CHECK-LABEL: ReadWrite64:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    pld r3, input64@got@pcrel(0), 1
-; CHECK-NEXT:  .Lpcrel2:
-; CHECK-NEXT:    pld r4, output64@got@pcrel(0), 1
-; CHECK-NEXT:    .reloc .Lpcrel2-8,R_PPC64_PCREL_OPT,.-(.Lpcrel2-8)
+; CHECK-NEXT:  .Lpcrel5:
+; CHECK-NEXT:    .reloc .Lpcrel5-8,R_PPC64_PCREL_OPT,.-(.Lpcrel5-8)
 ; CHECK-NEXT:    ld r3, 0(r3)
+; CHECK-NEXT:    pld r4, output64@got@pcrel(0), 1
+; CHECK-NEXT:  .Lpcrel6:
+; CHECK-NEXT:    .reloc .Lpcrel6-8,R_PPC64_PCREL_OPT,.-(.Lpcrel6-8)
 ; CHECK-NEXT:    std r3, 0(r4)
 ; CHECK-NEXT:    blr
 entry:
   %0 = load i64, i64* @input64, align 8
   store i64 %0, i64* @output64, align 8
   ret void
 }
 
entry:
   store i128 %0, i128* @output128, align 16
   ret void
 }
 
 define dso_local void @ReadWritef32() local_unnamed_addr #0 {
 ; CHECK-LABEL: ReadWritef32:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    pld r3, inputf32@got@pcrel(0), 1
-; CHECK-NEXT:  .Lpcrel3:
+; CHECK-NEXT:  .Lpcrel7:
 ; CHECK-NEXT:    xxspltidp vs1, 1078103900
-; CHECK-NEXT:    .reloc .Lpcrel3-8,R_PPC64_PCREL_OPT,.-(.Lpcrel3-8)
+; CHECK-NEXT:    .reloc .Lpcrel7-8,R_PPC64_PCREL_OPT,.-(.Lpcrel7-8)
 ; CHECK-NEXT:    lfs f0, 0(r3)
-; CHECK-NEXT:    pld r3, outputf32@got@pcrel(0), 1
 ; CHECK-NEXT:    xsaddsp f0, f0, f1
+; CHECK-NEXT:    pld r3, outputf32@got@pcrel(0), 1
+; CHECK-NEXT:  .Lpcrel8:
+; CHECK-NEXT:    .reloc .Lpcrel8-8,R_PPC64_PCREL_OPT,.-(.Lpcrel8-8)
 ; CHECK-NEXT:    stfs f0, 0(r3)
 ; CHECK-NEXT:    blr
 entry:
   %0 = load float, float* @inputf32, align 4
   %add = fadd float %0, 0x400851EB80000000
   store float %add, float* @outputf32, align 4
   ret void
 }
 
 define dso_local void @ReadWritef64() local_unnamed_addr #0 {
 ; CHECK-LABEL: ReadWritef64:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    pld r3, inputf64@got@pcrel(0), 1
-; CHECK-NEXT:  .Lpcrel4:
+; CHECK-NEXT:  .Lpcrel9:
 ; CHECK-NEXT:    plfd f1, .LCPI6_0@PCREL(0), 1
-; CHECK-NEXT:    .reloc .Lpcrel4-8,R_PPC64_PCREL_OPT,.-(.Lpcrel4-8)
+; CHECK-NEXT:    .reloc .Lpcrel9-8,R_PPC64_PCREL_OPT,.-(.Lpcrel9-8)
 ; CHECK-NEXT:    lfd f0, 0(r3)
-; CHECK-NEXT:    pld r3, outputf64@got@pcrel(0), 1
 ; CHECK-NEXT:    xsadddp f0, f0, f1
+; CHECK-NEXT:    pld r3, outputf64@got@pcrel(0), 1
+; CHECK-NEXT:  .Lpcrel10:
+; CHECK-NEXT:    .reloc .Lpcrel10-8,R_PPC64_PCREL_OPT,.-(.Lpcrel10-8)
 ; CHECK-NEXT:    stfd f0, 0(r3)
 ; CHECK-NEXT:    blr
 entry:
   %0 = load double, double* @inputf64, align 8
   %add = fadd double %0, 6.800000e+00
   store double %add, double* @outputf64, align 8
   ret void
 }
entry:
   store <2 x i64> %0, <2 x i64>* @outputVi64, align 16
   ret void
 }
 
 define dso_local void @ReadWriteArray() local_unnamed_addr #0 {
 ; CHECK-LABEL: ReadWriteArray:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    pld r3, ArrayIn@got@pcrel(0), 1
-; CHECK-NEXT:  .Lpcrel5:
-; CHECK-NEXT:    pld r4, ArrayOut@got@pcrel(0), 1
-; CHECK-NEXT:    .reloc .Lpcrel5-8,R_PPC64_PCREL_OPT,.-(.Lpcrel5-8)
+; CHECK-NEXT:  .Lpcrel11:
+; CHECK-NEXT:    .reloc .Lpcrel11-8,R_PPC64_PCREL_OPT,.-(.Lpcrel11-8)
 ; CHECK-NEXT:    lwz r3, 28(r3)
 ; CHECK-NEXT:    addi r3, r3, 42
+; CHECK-NEXT:    pld r4, ArrayOut@got@pcrel(0), 1
+; CHECK-NEXT:  .Lpcrel12:
+; CHECK-NEXT:    .reloc .Lpcrel12-8,R_PPC64_PCREL_OPT,.-(.Lpcrel12-8)
 ; CHECK-NEXT:    stw r3, 8(r4)
 ; CHECK-NEXT:    blr
 entry:
   %0 = load i32, i32* getelementptr inbounds ([10 x i32], [10 x i32]* @ArrayIn, i64 0, i64 7), align 4
   %add = add nsw i32 %0, 42
   store i32 %add, i32* getelementptr inbounds ([10 x i32], [10 x i32]* @ArrayOut, i64 0, i64 2), align 4
   ret void
 }
entry:
   store i32 %add, i32* getelementptr inbounds ([10 x i32], [10 x i32]* @ArrayIn, i64 0, i64 6), align 4
   ret void
 }
 
 define dso_local void @ReadWriteIntPtr() local_unnamed_addr #0 {
 ; CHECK-LABEL: ReadWriteIntPtr:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    pld r3, IntPtrIn@got@pcrel(0), 1
-; CHECK-NEXT:  .Lpcrel6:
+; CHECK-NEXT:  .Lpcrel13:
 ; CHECK-NEXT:    pld r4, IntPtrOut@got@pcrel(0), 1
-; CHECK-NEXT:  .Lpcrel7:
-; CHECK-NEXT:    .reloc .Lpcrel6-8,R_PPC64_PCREL_OPT,.-(.Lpcrel6-8)
+; CHECK-NEXT:  .Lpcrel14:
+; CHECK-NEXT:    .reloc .Lpcrel13-8,R_PPC64_PCREL_OPT,.-(.Lpcrel13-8)
 ; CHECK-NEXT:    ld r3, 0(r3)
-; CHECK-NEXT:    .reloc .Lpcrel7-8,R_PPC64_PCREL_OPT,.-(.Lpcrel7-8)
+; CHECK-NEXT:    .reloc .Lpcrel14-8,R_PPC64_PCREL_OPT,.-(.Lpcrel14-8)
 ; CHECK-NEXT:    ld r4, 0(r4)
 ; CHECK-NEXT:    lwz r5, 216(r3)
 ; CHECK-NEXT:    lwz r3, 48(r3)
 ; CHECK-NEXT:    add r3, r3, r5
 ; CHECK-NEXT:    stw r3, 136(r4)
 ; CHECK-NEXT:    blr
 entry:
   %0 = load i32*, i32** @IntPtrIn, align 8
   %arrayidx = getelementptr inbounds i32, i32* %0, i64 54
   %1 = load i32, i32* %arrayidx, align 4
   %arrayidx1 = getelementptr inbounds i32, i32* %0, i64 12
   %2 = load i32, i32* %arrayidx1, align 4
   %add = add nsw i32 %2, %1
   %3 = load i32*, i32** @IntPtrOut, align 8
   %arrayidx2 = getelementptr inbounds i32, i32* %3, i64 34
   store i32 %add, i32* %arrayidx2, align 4
   ret void
 }
 
 define dso_local void @ReadWriteFuncPtr() local_unnamed_addr #0 {
 ; CHECK-LABEL: ReadWriteFuncPtr:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    pld r3, FuncPtrIn@got@pcrel(0), 1
-; CHECK-NEXT:  .Lpcrel8:
-; CHECK-NEXT:    pld r4, FuncPtrOut@got@pcrel(0), 1
-; CHECK-NEXT:    .reloc .Lpcrel8-8,R_PPC64_PCREL_OPT,.-(.Lpcrel8-8)
+; CHECK-NEXT:  .Lpcrel15:
+; CHECK-NEXT:    .reloc .Lpcrel15-8,R_PPC64_PCREL_OPT,.-(.Lpcrel15-8)
 ; CHECK-NEXT:    ld r3, 0(r3)
+; CHECK-NEXT:    pld r4, FuncPtrOut@got@pcrel(0), 1
+; CHECK-NEXT:  .Lpcrel16:
+; CHECK-NEXT:    .reloc .Lpcrel16-8,R_PPC64_PCREL_OPT,.-(.Lpcrel16-8)
 ; CHECK-NEXT:    std r3, 0(r4)
 ; CHECK-NEXT:    blr
 entry:
   %0 = load i64, i64* bitcast (void (...)** @FuncPtrIn to i64*), align 8
   store i64 %0, i64* bitcast (void (...)** @FuncPtrOut to i64*), align 8
   ret void
 }
 
 define dso_local void @FuncPtrCopy() local_unnamed_addr #0 {
 ; CHECK-LABEL: FuncPtrCopy:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    pld r3, FuncPtrOut@got@pcrel(0), 1
 ; CHECK-NEXT:    pld r4, Callee@got@pcrel(0), 1
+; CHECK-NEXT:    pld r3, FuncPtrOut@got@pcrel(0), 1
 ; CHECK-NEXT:    std r4, 0(r3)
 ; CHECK-NEXT:    blr
 entry:
   store void (...)* @Callee, void (...)** @FuncPtrOut, align 8
   ret void
 }
 
 declare void @Callee(...)
 
 define dso_local void @FuncPtrCall() local_unnamed_addr #0 {
 ; CHECK-LABEL: FuncPtrCall:
 ; CHECK:         .localentry FuncPtrCall, 1
 ; CHECK-NEXT:  # %bb.0: # %entry
 ; CHECK-NEXT:    pld r3, FuncPtrIn@got@pcrel(0), 1
-; CHECK-NEXT:  .Lpcrel9:
-; CHECK-NEXT:    .reloc .Lpcrel9-8,R_PPC64_PCREL_OPT,.-(.Lpcrel9-8)
+; CHECK-NEXT:  .Lpcrel17:
+; CHECK-NEXT:    .reloc .Lpcrel17-8,R_PPC64_PCREL_OPT,.-(.Lpcrel17-8)
 ; CHECK-NEXT:    ld r12, 0(r3)
 ; CHECK-NEXT:    mtctr r12
 ; CHECK-NEXT:    bctr
 ; CHECK-NEXT:    #TC_RETURNr8 ctr 0
 entry:
   %0 = load void ()*, void ()** bitcast (void (...)** @FuncPtrIn to void ()**), align 8
   tail call void %0()
   ret void
 }
 
 define dso_local signext i32 @ReadVecElement() local_unnamed_addr #0 {
 ; CHECK-LABEL: ReadVecElement:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    pld r3, inputVi32@got@pcrel(0), 1
-; CHECK-NEXT:  .Lpcrel10:
-; CHECK-NEXT:    .reloc .Lpcrel10-8,R_PPC64_PCREL_OPT,.-(.Lpcrel10-8)
+; CHECK-NEXT:  .Lpcrel18:
+; CHECK-NEXT:    .reloc .Lpcrel18-8,R_PPC64_PCREL_OPT,.-(.Lpcrel18-8)
 ; CHECK-NEXT:    lwa r3, 4(r3)
 ; CHECK-NEXT:    blr
 entry:
   %0 = load <4 x i32>, <4 x i32>* @inputVi32, align 16
   %vecext = extractelement <4 x i32> %0, i32 1
   ret i32 %vecext
 }
 

 ; CHECK-LABEL: AddrTaken32:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    pld r3, input32@got@pcrel(0), 1
 ; CHECK-NEXT:    blr
 entry:
   ret i32* @input32
 }
 
+; Verify that mulli can still be scheduled in-between pld and stw.
+define dso_local i32 @interleave(i32* %t) local_unnamed_addr #0 {
+; CHECK-LABEL: interleave:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    lwz r3, 0(r3)
+; CHECK-NEXT:    li r5, 4
+; CHECK-NEXT:    pld r4, input32@got@pcrel(0), 1
+; CHECK-NEXT:  .Lpcrel19:
+; CHECK-NEXT:    mulli r3, r3, 999
+; CHECK-NEXT:    .reloc .Lpcrel19-8,R_PPC64_PCREL_OPT,.-(.Lpcrel19-8)
+; CHECK-NEXT:    stw r5, 0(r4)
+; CHECK-NEXT:    blr
+entry:
+  %0 = load i32, i32* %t, align 4
+  store i32 4, i32* @input32, align 4
+  %mul = mul nsw i32 %0, 999
+  ret i32 %mul
+}
+
+define dso_local void @multi_uses() local_unnamed_addr #0 {
+; CHECK-LABEL: multi_uses:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    pld r3, ArrayIn@got@pcrel(0), 1
+; CHECK-NEXT:    li r4, 3
+; CHECK-NEXT:    stw r4, 4(r3)
+; CHECK-NEXT:    li r4, 4
+; CHECK-NEXT:    stw r4, 12(r3)
+; CHECK-NEXT:    blr
+entry:
+  store i32 3, i32* getelementptr inbounds ([10 x i32], [10 x i32]* @ArrayIn, i64 0, i64 1), align 4
+  store i32 4, i32* getelementptr inbounds ([10 x i32], [10 x i32]* @ArrayIn, i64 0, i64 3), align 4
+  ret void
+}
+
 attributes #0 = { nounwind }

llvm/test/CodeGen/PowerPC/pcrel-tail-calls.ll

 ; PC Relative we are able to do more tail calling than we have done in
 ; the past as we no longer need to restore the TOC pointer into R2 after
 ; most calls.
 
 @Func = external local_unnamed_addr global i32 (...)*, align 8
 @FuncLocal = common dso_local local_unnamed_addr global i32 (...)* null, align 8
 
 ; No calls in this function but we assign the function pointers.
+; FIXME - the pcrel opt relocation is missing for pld r3 - std r4 pair.
 define dso_local void @AssignFuncPtr() local_unnamed_addr {
 ; CHECK-LABEL: AssignFuncPtr:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    pld r3, Func@got@pcrel(0), 1
 ; CHECK-NEXT:    pld r4, Function@got@pcrel(0), 1
-; CHECK-NEXT:    std r4, 0(r3)
+; CHECK-NEXT:    pld r3, Func@got@pcrel(0), 1
 ; CHECK-NEXT:    pstd r4, FuncLocal@PCREL(0), 1
+; CHECK-NEXT:    std r4, 0(r3)
 ; CHECK-NEXT:    blr
 entry:
   store i32 (...)* @Function, i32 (...)** @Func, align 8
   store i32 (...)* @Function, i32 (...)** @FuncLocal, align 8
   ret void
 }
 
 declare signext i32 @Function(...)