[ELF2] Allow PPC64 to add the TOC-restore after .plt-based relocations

[hfinkel]

Summary

Under the PPC64 ELF ABI, functions that might call into other modules (and, thus, need to load a different TOC base value into %r2), need to restore the old value after the call. The old value is saved by the .plt code, and the caller only need to include a nop instruction after the call, which the linker will transform into a TOC restore if necessary.

In order to do this the relocation handler needs two things:

1. It needs to know whether the call instruction it is modifying is targeting a .plt stub that will load a new TOC base value (necessitating a restore after the call).
2. It needs to know where the buffer ends, so that it does not accidentally run off the end of the buffer when looking for the 'nop' instruction after the call.

Given these two pieces of information, we can insert the restore instruction in place of the following nop when necessary.

Test cases are needed; feedback is otherwise appreciated!

[ruiu]

What if the linker has to write an instruction to restore the register but there's no space at end of function? This patch simply don't do anything for such functions, but if that works without the instruction, then why do we need that in the first place?

[hfinkel]

In D13565#263097, @ruiu wrote:

What if the linker has to write an instruction to restore the register but there's no space at end of function? This patch simply don't do anything for such functions, but if that works without the instruction, then why do we need that in the first place?

Generically you're correct, however, some code that calls functions that don't return (like _exit) will omit the trailing nop, and furthermore, will appear at the end of a function. Since they don't return, they legitimately don't need the TOC restore.

[ruiu]

Does writing the instruction only when there's nop there work? I don't like to pass BufEnd for all these functions.

[hfinkel]

In D13565#263112, @ruiu wrote:

Does writing the instruction only when there's nop there work? I don't like to pass BufEnd for all these functions.

Sure. But how do I avoid running off the end of the buffer checking if the nop is there in the first place?

More generally, however, since we don't otherwise validate the offset member of the relocation field (and probably can't in a completely-target-independent way), it seems like we should be checking that against the end of the buffer anyway for all relocations everywhere.

Nevertheless, I agree this looks somewhat ugly. Would it be better to pass a pair of some kind?

[ruiu]

For ForPltEntry, you can do

uint64_t Start = Out<ELF64BE>::Plt->getVA();
uint64_t End = Start + Out<ELF64BE>::Plt->getSize();
// Address A is pointing an PLT entry if Start <= A && A < End

By using this you can eliminate one parameter. I removed other parameter from the function, which is GotVA.

I don't think there's an obvious way to eliminate BufEnd, so I can live with that. But the parameter should be used only for that PPC relocation type. We don't want to do boundary check in general because Buf is guaranteed to not overrun.

[hfinkel]

Updated to reflect reviewer comments (and added a test case).

[ruiu]

ELF/Target.cpp

398–401

Move this just above if (ForPltEntry &&, so that it's apparent that the following four lines don't use these variables.

400

I'd write this like the number line.

bool InPlt = PltStart <= S + A && S + A  < PltEnd;

test/elf2/Inputs/shared-ppc64.s

12

Remove this trailing line.

test/elf2/ppc64-toc-restore.s

47

Ditto

[hfinkel]

Updated to reflect review comments.

[ruiu]

LGTM with a nit.

ELF/Target.cpp

405

Can you rename InPlt?

[hfinkel]

r250110, thanks!

ELF/InputSection.h

public:
   void setOutputSection(OutputSection<ELFT> *O) { OutSec = O; }
   OutputSection<ELFT> *getOutputSection() const { return OutSec; }
 
   // Relocation sections that refer to this one.
   SmallVector<const Elf_Shdr *, 1> RelocSections;
 
 private:
   template <bool isRela>
-  void relocate(uint8_t *Buf,
+  void relocate(uint8_t *Buf, uint8_t *BufEnd,
                 llvm::iterator_range<
                     const llvm::object::Elf_Rel_Impl<ELFT, isRela> *> Rels,
                 const ObjectFile<ELFT> &File, uintX_t BaseAddr);
 
   // The offset from beginning of the output sections this section was assigned
   // to. The writer sets a value.
   uint64_t OutputSectionOff = 0;
 

ELF/InputSection.cpp

 
 template <class ELFT>
 InputSection<ELFT>::InputSection(ObjectFile<ELFT> *F, const Elf_Shdr *Header)
     : File(F), Header(Header) {}
 
 template <class ELFT>
 template <bool isRela>
 void InputSection<ELFT>::relocate(
-    uint8_t *Buf, iterator_range<const Elf_Rel_Impl<ELFT, isRela> *> Rels,
+    uint8_t *Buf, uint8_t *BufEnd,
+    iterator_range<const Elf_Rel_Impl<ELFT, isRela> *> Rels,
     const ObjectFile<ELFT> &File, uintX_t BaseAddr) {
   typedef Elf_Rel_Impl<ELFT, isRela> RelType;
   bool IsMips64EL = File.getObj().isMips64EL();
   for (const RelType &RI : Rels) {
     uint32_t SymIndex = RI.getSymbol(IsMips64EL);
     uint32_t Type = RI.getType(IsMips64EL);
     uintX_t SymVA = 0;
 
if (SymIndex < SymTab->sh_info) {
       } else if (Target->relocPointsToGot(Type)) {
         SymVA = Out<ELFT>::Got->getVA();
         Type = Target->getPCRelReloc();
       } else if (isa<SharedSymbol<ELFT>>(Body)) {
         continue;
       }
     }
 
-    Target->relocateOne(Buf, reinterpret_cast<const void *>(&RI), Type,
+    Target->relocateOne(Buf, BufEnd, reinterpret_cast<const void *>(&RI), Type,
                         BaseAddr, SymVA);
   }
 }
 
 template <class ELFT> void InputSection<ELFT>::writeTo(uint8_t *Buf) {
   if (Header->sh_type == SHT_NOBITS)
     return;
   // Copy section contents from source object file to output file.
   ArrayRef<uint8_t> Data = *File->getObj().getSectionContents(Header);
   memcpy(Buf + OutputSectionOff, Data.data(), Data.size());
 
   ObjectFile<ELFT> *File = getFile();
   ELFFile<ELFT> &EObj = File->getObj();
   uint8_t *Base = Buf + getOutputSectionOff();
   uintX_t BaseAddr = OutSec->getVA() + getOutputSectionOff();
   // Iterate over all relocation sections that apply to this section.
   for (const Elf_Shdr *RelSec : RelocSections) {
     if (RelSec->sh_type == SHT_RELA)
-      relocate(Base, EObj.relas(RelSec), *File, BaseAddr);
+      relocate(Base, Base + Data.size(), EObj.relas(RelSec), *File, BaseAddr);
     else
-      relocate(Base, EObj.rels(RelSec), *File, BaseAddr);
+      relocate(Base, Base + Data.size(), EObj.rels(RelSec), *File, BaseAddr);
   }
 }
 
 template <class ELFT> StringRef InputSection<ELFT>::getSectionName() const {
   ErrorOr<StringRef> Name = File->getObj().getSectionName(Header);
   error(Name);
   return *Name;
 }

ELF/Target.h

public:
   unsigned getRelativeReloc() const { return RelativeReloc; }
   unsigned getPltEntrySize() const { return PltEntrySize; }
   virtual void writePltEntry(uint8_t *Buf, uint64_t GotEntryAddr,
                              uint64_t PltEntryAddr) const = 0;
   virtual bool isRelRelative(uint32_t Type) const;
   virtual bool relocNeedsGot(uint32_t Type, const SymbolBody &S) const = 0;
   virtual bool relocPointsToGot(uint32_t Type) const;
   virtual bool relocNeedsPlt(uint32_t Type, const SymbolBody &S) const = 0;
-  virtual void relocateOne(uint8_t *Buf, const void *RelP, uint32_t Type,
-                           uint64_t BaseAddr, uint64_t SymVA) const = 0;
+  virtual void relocateOne(uint8_t *Buf, uint8_t *BufEnd, const void *RelP,
+                           uint32_t Type, uint64_t BaseAddr,
+                           uint64_t SymVA) const = 0;
 
   virtual ~TargetInfo();
 
 protected:
   unsigned PageSize = 4096;
   uint64_t VAStart;
   unsigned PCRelReloc;
   unsigned GotRefReloc;
   unsigned GotReloc;
   unsigned RelativeReloc;
   unsigned PltEntrySize = 8;
   llvm::StringRef DefaultEntry = "_start";
 };
 
 class X86TargetInfo final : public TargetInfo {
 public:
   X86TargetInfo();
   void writePltEntry(uint8_t *Buf, uint64_t GotEntryAddr,
                      uint64_t PltEntryAddr) const override;
   bool relocNeedsGot(uint32_t Type, const SymbolBody &S) const override;
   bool relocPointsToGot(uint32_t Type) const override;
   bool relocNeedsPlt(uint32_t Type, const SymbolBody &S) const override;
-  void relocateOne(uint8_t *Buf, const void *RelP, uint32_t Type,
-                   uint64_t BaseAddr, uint64_t SymVA) const override;
+  void relocateOne(uint8_t *Buf, uint8_t *BufEnd, const void *RelP,
+                   uint32_t Type, uint64_t BaseAddr,
+                   uint64_t SymVA) const override;
 };
 
 class X86_64TargetInfo final : public TargetInfo {
 public:
   X86_64TargetInfo();
   void writePltEntry(uint8_t *Buf, uint64_t GotEntryAddr,
                      uint64_t PltEntryAddr) const override;
   bool relocNeedsGot(uint32_t Type, const SymbolBody &S) const override;
   bool relocNeedsPlt(uint32_t Type, const SymbolBody &S) const override;
-  void relocateOne(uint8_t *Buf, const void *RelP, uint32_t Type,
-                   uint64_t BaseAddr, uint64_t SymVA) const override;
+  void relocateOne(uint8_t *Buf, uint8_t *BufEnd, const void *RelP,
+                   uint32_t Type, uint64_t BaseAddr,
+                   uint64_t SymVA) const override;
   bool isRelRelative(uint32_t Type) const override;
 };
 
 class PPC64TargetInfo final : public TargetInfo {
 public:
   PPC64TargetInfo();
   void writePltEntry(uint8_t *Buf, uint64_t GotEntryAddr,
                      uint64_t PltEntryAddr) const override;
   bool relocNeedsGot(uint32_t Type, const SymbolBody &S) const override;
   bool relocNeedsPlt(uint32_t Type, const SymbolBody &S) const override;
-  void relocateOne(uint8_t *Buf, const void *RelP, uint32_t Type,
-                   uint64_t BaseAddr, uint64_t SymVA) const override;
+  void relocateOne(uint8_t *Buf, uint8_t *BufEnd, const void *RelP,
+                   uint32_t Type, uint64_t BaseAddr,
+                   uint64_t SymVA) const override;
   bool isRelRelative(uint32_t Type) const override;
 };
 
 class PPCTargetInfo final : public TargetInfo {
 public:
   PPCTargetInfo();
   void writePltEntry(uint8_t *Buf, uint64_t GotEntryAddr,
                      uint64_t PltEntryAddr) const override;
   bool relocNeedsGot(uint32_t Type, const SymbolBody &S) const override;
   bool relocNeedsPlt(uint32_t Type, const SymbolBody &S) const override;
-  void relocateOne(uint8_t *Buf, const void *RelP, uint32_t Type,
-                   uint64_t BaseAddr, uint64_t SymVA) const override;
+  void relocateOne(uint8_t *Buf, uint8_t *BufEnd, const void *RelP,
+                   uint32_t Type, uint64_t BaseAddr,
+                   uint64_t SymVA) const override;
 };
 
 class ARMTargetInfo final : public TargetInfo {
 public:
   ARMTargetInfo();
   void writePltEntry(uint8_t *Buf, uint64_t GotEntryAddr,
                      uint64_t PltEntryAddr) const override;
   bool relocNeedsGot(uint32_t Type, const SymbolBody &S) const override;
   bool relocNeedsPlt(uint32_t Type, const SymbolBody &S) const override;
-  void relocateOne(uint8_t *Buf, const void *RelP, uint32_t Type,
-                   uint64_t BaseAddr, uint64_t SymVA) const override;
+  void relocateOne(uint8_t *Buf, uint8_t *BufEnd, const void *RelP,
+                   uint32_t Type, uint64_t BaseAddr,
+                   uint64_t SymVA) const override;
 };
 
 class AArch64TargetInfo final : public TargetInfo {
 public:
   AArch64TargetInfo();
   void writePltEntry(uint8_t *Buf, uint64_t GotEntryAddr,
                      uint64_t PltEntryAddr) const override;
   bool relocNeedsGot(uint32_t Type, const SymbolBody &S) const override;
   bool relocNeedsPlt(uint32_t Type, const SymbolBody &S) const override;
-  void relocateOne(uint8_t *Buf, const void *RelP, uint32_t Type,
-                   uint64_t BaseAddr, uint64_t SymVA) const override;
+  void relocateOne(uint8_t *Buf, uint8_t *BufEnd, const void *RelP,
+                   uint32_t Type, uint64_t BaseAddr,
+                   uint64_t SymVA) const override;
 };
 
 class MipsTargetInfo final : public TargetInfo {
 public:
   MipsTargetInfo();
   void writePltEntry(uint8_t *Buf, uint64_t GotEntryAddr,
                      uint64_t PltEntryAddr) const override;
   bool relocNeedsGot(uint32_t Type, const SymbolBody &S) const override;
   bool relocNeedsPlt(uint32_t Type, const SymbolBody &S) const override;
-  void relocateOne(uint8_t *Buf, const void *RelP, uint32_t Type,
-                   uint64_t BaseAddr, uint64_t SymVA) const override;
+  void relocateOne(uint8_t *Buf, uint8_t *BufEnd, const void *RelP,
+                   uint32_t Type, uint64_t BaseAddr,
+                   uint64_t SymVA) const override;
 };
 
 extern std::unique_ptr<TargetInfo> Target;
 }
 }
 
 #endif

ELF/Target.cpp

 
 bool X86TargetInfo::relocNeedsPlt(uint32_t Type, const SymbolBody &S) const {
   return Type == R_386_PLT32 || (Type == R_386_PC32 && S.isShared());
 }
 
 static void add32le(uint8_t *L, int32_t V) { write32le(L, read32le(L) + V); }
 static void or32le(uint8_t *L, int32_t V) { write32le(L, read32le(L) | V); }
 
-void X86TargetInfo::relocateOne(uint8_t *Buf, const void *RelP, uint32_t Type,
-                                uint64_t BaseAddr, uint64_t SymVA) const {
+void X86TargetInfo::relocateOne(uint8_t *Buf, uint8_t *BufEnd, const void *RelP,
+                                uint32_t Type, uint64_t BaseAddr,
+                                uint64_t SymVA) const {
   typedef ELFFile<ELF32LE>::Elf_Rel Elf_Rel;
   auto &Rel = *reinterpret_cast<const Elf_Rel *>(RelP);
 
   uint32_t Offset = Rel.r_offset;
   uint8_t *Loc = Buf + Offset;
   switch (Type) {
   case R_386_GOT32:
     add32le(Loc, SymVA - Out<ELF32LE>::Got->getVA());
bool X86_64TargetInfo::isRelRelative(uint32_t Type) const {
   case R_X86_64_PC64:
   case R_X86_64_PC32:
   case R_X86_64_PC16:
   case R_X86_64_PC8:
     return true;
   }
 }
 
-void X86_64TargetInfo::relocateOne(uint8_t *Buf, const void *RelP,
-                                   uint32_t Type, uint64_t BaseAddr,
-                                   uint64_t SymVA) const {
+void X86_64TargetInfo::relocateOne(uint8_t *Buf, uint8_t *BufEnd,
+                                   const void *RelP, uint32_t Type,
+                                   uint64_t BaseAddr, uint64_t SymVA) const {
   typedef ELFFile<ELF64LE>::Elf_Rela Elf_Rela;
   auto &Rel = *reinterpret_cast<const Elf_Rela *>(RelP);
 
   uint64_t Offset = Rel.r_offset;
   uint8_t *Loc = Buf + Offset;
   switch (Type) {
   case R_X86_64_PC32:
   case R_X86_64_GOTPCREL:
bool PPC64TargetInfo::isRelRelative(uint32_t Type) const {
   case R_PPC64_REL14_BRTAKEN:
   case R_PPC64_REL14_BRNTAKEN:
   case R_PPC64_REL32:
   case R_PPC64_REL64:
     return true;
   }
 }
 
-void PPC64TargetInfo::relocateOne(uint8_t *Buf, const void *RelP, uint32_t Type,
+void PPC64TargetInfo::relocateOne(uint8_t *Buf, uint8_t *BufEnd,
+                                  const void *RelP, uint32_t Type,
                                   uint64_t BaseAddr, uint64_t SymVA) const {
   typedef ELFFile<ELF64BE>::Elf_Rela Elf_Rela;
   auto &Rel = *reinterpret_cast<const Elf_Rela *>(RelP);
 
   uint8_t *L = Buf + Rel.r_offset;
   uint64_t S = SymVA;
   int64_t A = Rel.r_addend;
   uint64_t P = BaseAddr + Rel.r_offset;
case R_PPC64_REL16_HA:
     write16be(L, applyPPCHa(R - P));
     break;
   case R_PPC64_ADDR32:
     if (!isInt<32>(R))
       error("Relocation R_PPC64_ADDR32 overflow");
     write32be(L, R);
     break;
   case R_PPC64_REL24: {
     uint32_t Mask = 0x03FFFFFC;
     if (!isInt<24>(R - P))
       error("Relocation R_PPC64_REL24 overflow");
     write32be(L, (read32be(L) & ~Mask) | ((R - P) & Mask));
+
+    uint64_t PltStart = Out<ELF64BE>::Plt->getVA();
+    uint64_t PltEnd = PltStart + Out<ELF64BE>::Plt->getSize();
+    bool ForPltEntry = PltStart <= S + A && S + A < PltEnd;
+
+    if (ForPltEntry && L + 8 < BufEnd &&
+        read32be(L + 4) == 0x60000000 /* nop */)
+      write32be(L + 4, 0xe8410028); // ld %r2, 40(%r1)
     break;
   }
   case R_PPC64_REL32:
     if (!isInt<32>(R - P))
       error("Relocation R_PPC64_REL32 overflow");
     write32be(L, R - P);
     break;
   case R_PPC64_REL64:

 void PPCTargetInfo::writePltEntry(uint8_t *Buf, uint64_t GotEntryAddr,
                                   uint64_t PltEntryAddr) const {}
 bool PPCTargetInfo::relocNeedsGot(uint32_t Type, const SymbolBody &S) const {
   return false;
 }
 bool PPCTargetInfo::relocNeedsPlt(uint32_t Type, const SymbolBody &S) const {
   return false;
 }
-void PPCTargetInfo::relocateOne(uint8_t *Buf, const void *RelP, uint32_t Type,
+void PPCTargetInfo::relocateOne(uint8_t *Buf, uint8_t *BufEnd,
+                                const void *RelP, uint32_t Type,
                                 uint64_t BaseAddr, uint64_t SymVA) const {}
 
 ARMTargetInfo::ARMTargetInfo() {
   // PCRelReloc = FIXME
   // GotReloc = FIXME
   VAStart = 0x8000;
 }
 void ARMTargetInfo::writePltEntry(uint8_t *Buf, uint64_t GotEntryAddr,
                                   uint64_t PltEntryAddr) const {}
 bool ARMTargetInfo::relocNeedsGot(uint32_t Type, const SymbolBody &S) const {
   return false;
 }
 bool ARMTargetInfo::relocNeedsPlt(uint32_t Type, const SymbolBody &S) const {
   return false;
 }
-void ARMTargetInfo::relocateOne(uint8_t *Buf, const void *RelP, uint32_t Type,
+void ARMTargetInfo::relocateOne(uint8_t *Buf, uint8_t *BufEnd,
+                                const void *RelP, uint32_t Type,
                                 uint64_t BaseAddr, uint64_t SymVA) const {}
 
 AArch64TargetInfo::AArch64TargetInfo() {
   // PCRelReloc = FIXME
   // GotReloc = FIXME
   VAStart = 0x400000;
 }
 void AArch64TargetInfo::writePltEntry(uint8_t *Buf, uint64_t GotEntryAddr,

 
 // Page(Expr) is the page address of the expression Expr, defined
 // as (Expr & ~0xFFF). (This applies even if the machine page size
 // supported by the platform has a different value.)
 static uint64_t getAArch64Page(uint64_t Expr) {
   return Expr & (~static_cast<uint64_t>(0xFFF));
 }
 
-void AArch64TargetInfo::relocateOne(uint8_t *Buf, const void *RelP,
-                                    uint32_t Type, uint64_t BaseAddr,
-                                    uint64_t SymVA) const {
+void AArch64TargetInfo::relocateOne(uint8_t *Buf, uint8_t *BufEnd,
+                                    const void *RelP, uint32_t Type,
+                                    uint64_t BaseAddr, uint64_t SymVA) const {
   typedef ELFFile<ELF64LE>::Elf_Rela Elf_Rela;
   auto &Rel = *reinterpret_cast<const Elf_Rela *>(RelP);
 
   uint8_t *L = Buf + Rel.r_offset;
   uint64_t S = SymVA;
   int64_t A = Rel.r_addend;
   uint64_t P = BaseAddr + Rel.r_offset;
   switch (Type) {

 bool MipsTargetInfo::relocNeedsGot(uint32_t Type, const SymbolBody &S) const {
   return false;
 }
 
 bool MipsTargetInfo::relocNeedsPlt(uint32_t Type, const SymbolBody &S) const {
   return false;
 }
 
-void MipsTargetInfo::relocateOne(uint8_t *Buf, const void *RelP, uint32_t Type,
+void MipsTargetInfo::relocateOne(uint8_t *Buf, uint8_t *BufEnd,
+                                 const void *RelP, uint32_t Type,
                                  uint64_t BaseAddr, uint64_t SymVA) const {}
 }
 }

test/elf2/Inputs/shared-ppc64.s

+.section        ".opd","aw"
+.global bar
+bar:
+# FIXME: This should be: .quad   .Lbar,.TOC.@tocbase,0
+# But we can't handle .TOC.@tocbase for shared libraries yet
+.quad   .Lbar,0,0
+
+.text
+.Lbar:
+        blr

test/elf2/ppc64-toc-restore.s

+// RUN: llvm-mc -filetype=obj -triple=powerpc64-unknown-linux %s -o %t.o
+// RUN: llvm-mc -filetype=obj -triple=powerpc64-unknown-linux %p/Inputs/shared-ppc64.s -o %t2.o
+// RUN: ld.lld2 -shared %t2.o -o %t2.so
+// RUN: ld.lld2 -dynamic-linker /lib64/ld64.so.1 -rpath foo -rpath bar --export-dynamic %t.o %t2.so -o %t
+// RUN: llvm-objdump -d %t | FileCheck %s
+// REQUIRES: ppc
+
+// CHECK: Disassembly of section .text:
+
+.global _start
+_start:
+  bl bar
+  nop
+
+// CHECK: _start:
+// CHECK: 10020000:       48 00 00 21     bl .+32
+// CHECK-NOT: 10020004:       60 00 00 00     nop
+// CHECK: 10020004:       e8 41 00 28     ld 2, 40(1)
+
+.global noret
+noret:
+  bl bar
+  li 5, 7
+
+// CHECK: noret:
+// CHECK: 10020008:       48 00 00 19     bl .+24
+// CHECK: 1002000c:       38 a0 00 07     li 5, 7
+
+.global noretend
+noretend:
+  bl bar
+
+// CHECK: noretend:
+// CHECK: 10020010:       48 00 00 11     bl .+16
+
+.global noretb
+noretb:
+  b bar
+
+// CHECK: noretb:
+// CHECK: 10020014:       48 00 00 0c     b .+12
+
+// CHECK: Disassembly of section .plt:
+// CHECK: .plt:
+// CHECK: 10020020: