[PPC32] Improve the 32-bit PowerPC port

[MaskRay]

Summary

Many -static/-no-pie/-shared/-pie applications linked against glibc or musl
should work with this patch. This also helps FreeBSD PowerPC64 to migrate
their lib32 (PR40888).

• Fix default image base and max page size.
• Support new-style Secure PLT (see below). Old-style BSS PLT is not implemented, so it is not suitable for FreeBSD rtld now because it doesn't support Secure PLT yet.
• Support more initial relocation types: R_PPC_ADDR32, R_PPC_REL16*, R_PPC_LOCAL24PC, R_PPC_PLTREL24, and R_PPC_GOT16. The addend of R_PPC_PLTREL24 is special: it decides the call stub PLT type but it should be ignored for the computation of target symbol VA.
• Support GNU ifunc
• Support .glink used for lazy PLT resolution in glibc
• Add a new thunk type: PPC32PltCallStub that is similar to PPC64PltCallStub. It is used by R_PPC_REL24 and R_PPC_PLTREL24.

A PLT stub used in -fPIE/-fPIC usually loads an address relative to
.got2+0x8000 (-fpie/-fpic code uses _GLOBAL_OFFSET_TABLE_ relative
addresses).
Two .got2 sections in two object files have different addresses, thus a PLT stub
can't be shared by two object files. To handle this incompatibility,
change the parameters of Thunk::isCompatibleWith to
const InputSection &, const Relocation &.

PowerPC psABI specified an old-style .plt (BSS PLT) that is both
writable and executable. Linkers don't make separate RW- and RWE segments,
which causes all initially writable memory (think .data) executable.
This is a big security concern so a new PLT scheme (secure PLT) was developed to
address the security issue.

TLS will be implemented in D62940.

glibc older than ~2012 requires .rela.dyn to include .rela.plt, it can
not handle the DT_RELA+DT_RELASZ == DT_JMPREL case correctly. A hack
(not included in this patch) in LinkerScript.cpp addOrphanSections() is
required:

if (Config->EMachine == EM_PPC) {
  // Older glibc assumes .rela.dyn includes .rela.plt
  Add(In.RelaDyn);
  if (In.RelaPlt->isLive() && !In.RelaPlt->Parent)
    In.RelaDyn->getParent()->addSection(In.RelaPlt);
}

[MaskRay]

Fix a typo

[ruiu]

I think that the commit message is a bit too terse. Could you explain the current status of ppc32 if you can? How much is it usable and how much does this patch improve?

[MaskRay]

.

[MaskRay]

Reword

[ruiu]

Generally looking great.

ELF/Options.td

30 ↗

(On Diff #202604)

nit: powerpc32 → PowerPC32

311 ↗

(On Diff #202604)

Ditto

ELF/SyntheticSections.cpp

1042

Can you add return here? Or maybe you want to make this a switch whose each "case" ends with a return.

1043

This magic number needs a comment.

2260

nit: add parentheses before ?.

3217

Can you add a comment?

[MaskRay]

Add some support for BSS-PLT ABI

Will add some more tests.

[MaskRay]

ELF/SyntheticSections.cpp

2260

I think PowerPC32 doesn't need .glink. I'll just delete this change.

[MaskRay]

Address review comments

[ruiu]

What is a motivation to support the old-style PLT? If a program really expects that the PLT is in a writable segment, something is wrong with the program...

[MaskRay]

In D62464#1529079, @ruiu wrote:

What is a motivation to support the old-style PLT? If a program really expects that the PLT is in a writable segment, something is wrong with the program...

I didn't want to support BSS PLT (old-style PLT).

For Secure PLT, I can get musl -no-pie/-pie/-shared and glibc -no-pie work with this patch.

For BSS PLT, because musl doesn't support it, I have to debug glibc in qemu-system-ppc -hda debian_wheezy_powerpc_standard.qcow2, which is a pain... (glibc cannot be compiled without -O2 => the quality of debug info is at gcc's mercy).

On FreeBSD's side, I believe its rtld doesn't support Secure PLT.

@adalava @emaste It would be great if you can migrate from BSS PLT to Secure PLT (-msecure-plt; default in gcc if it is configured with --enable-secure-plt; many Linux gcc packages are configured this way).

[MaskRay]

Ready for review.

Many -no-pie/-pie/-shared linked against musl or glibc should work now.

[ruiu]

ELF/Arch/PPC.cpp

105

Is this a nop? Can you add a comment?

ELF/SyntheticSections.cpp

2275

Hmm, so we don't call writePlt() if a target is PPC32?

ELF/Thunks.cpp

243 ↗

(On Diff #203113)

nit: usually we write public members before private members.

709 ↗

(On Diff #203113)

nit: flip the condition and return early so that the indentation is one level shallower?

717–718 ↗

(On Diff #203113)

format.

[MaskRay]

ELF/SyntheticSections.cpp

2275

On PPC32, how PLT works is different from targets (it is still similar to PPC64). I've added more comments in PPC::writePltHeader.

[MaskRay]

Address review comments

[ruiu]

ELF/SyntheticSections.cpp

2275

So, this is not really a PLT header but a PPC32-specific mechanism. Is writePltHeader a correct name? Maybe we should rename this writePPC32PltResolver or something like that?

[MaskRay]

Also add R_PPC_ADDR16

[MaskRay]

ELF/SyntheticSections.cpp

2275

.glink is indeed a PPC-specific mechanism (PPC64 moves the "footer" (PLTresolve) to the "header" so it looks more conventional).

The PPC class is in an unnamed namespace so I cannot access it from SyntheticSections.cpp. If you still think writePPC32PltResolver is the way to go, I can extract it from the class and place it into the lld::elf:: namespace. I still feel it may not worth the trouble.

namespace {
class PPC final : public TargetInfo {
public:
...
  void writePltHeader(uint8_t *Buf) const override;

[ruiu]

ELF/SyntheticSections.cpp

2275

I feel like a PPC32-specific knowledge such as the actual machine instruction code leaks here, which is I think not ideal. How about moving the code to write b PLTresolve to writePltHeader and rename it writePPC32PltSection? Currently, a concrete implementation to write the first half of PPC32's PLT section is in this function while the other is in writePltHeader, which is not actually a header but a trailer. Nothing is really complicated, but there are perhaps too many things that are slightly confusing.

[MaskRay]

Add Target::writePPC32PltSection and let PPC::writePPC32PltSection override it.
This is to isolate the PPC32 knowledge.

[ruiu]

ELF/Arch/PPC.cpp

31

To make it clear that we write a PLT section in a different way, I'd define writePlt and writePltHeader which just call llvm_unreachable().

ELF/Target.h

45 ↗

(On Diff #203295)

This function doesn't have to be in this class because no dynamic dispatching is necessary. Only PPC32 implements this as its name implies. I'd make this a non-member function.

[MaskRay]

Forgot to add Buf += 4 * NumEntries;

[MaskRay]

Remove Target::writePPC32GlinkSection()
Add elf::writePPC32GlinkSection()

[ruiu]

LGTM

[MaskRay]

I cross compiled libc-test with powerpc-linux-gnu-gcc + lld.

% cd ~/libc-test # config.mak is below
% make > /tmp/libc-test.lld
# /tmp/libc-test.bfd was created in a similar way
% grep -c FAIL /tmp/libc-test.bfd /tmp/libc-test.lld
/tmp/libc-test.bfd:112
/tmp/libc-test.lld:112

So lld linked libc-test doesn't have more failures than bfd linked libc-test :)
(Tests ran under qemu-static-ppc. The situation may get better on real hardware.)

CROSS_COMPILE=powerpc-linux-gnu-
CFLAGS += -mlong-double-64 -msecure-plt
CC = powerpc-linux-gnu-gcc

CFLAGS += -specs ~/musl/powerpc/lib/musl-gcc.specs
LDFLAGS += -specs ~/musl/powerpc/lib/musl-gcc.specs
...
LDFLAGS += -fuse-ld=gold -g  # my /usr/powerpc-linux-gnu/bin/ld.gold points to lld... gcc 9 starts to understand -fuse-ld=lld

[MaskRay]

Implement R_PPC_GLOB_DAT and improve a R_PPC_GOT16 test while testing libc-test with glibc

[adalava]

Great job, @MaskRay !!

By initial tests, it seems that the PPC 32 bit FreeBSD "loader/bootloader" is now working fine when compiled with clang-8 and linked with LLD-9 with this patch. It is a static-stand-alone binary that doesn't depend on *crt*stuff.

For the 32 compatibility packages I now receive the error: "<unknown>:0: error: Undefined temporary symbol" when compiling everything with -msecure-plt. I'll retest with D62940 patch applyed

[MaskRay]

Add more tests

ELF/Arch/PPC.cpp

 //===- PPC.cpp ------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "Symbols.h"
+#include "SyntheticSections.h"
 #include "Target.h"
 #include "lld/Common/ErrorHandler.h"
 #include "llvm/Support/Endian.h"
 
 using namespace llvm;
 using namespace llvm::support::endian;
 using namespace llvm::ELF;
 using namespace lld;
 using namespace lld::elf;
 
 namespace {
 class PPC final : public TargetInfo {
 public:
   PPC();
+  void writeGotHeader(uint8_t *Buf) const override;
   void relocateOne(uint8_t *Loc, RelType Type, uint64_t Val) const override;
   RelExpr getRelExpr(RelType Type, const Symbol &S,
                      const uint8_t *Loc) const override;
 };
 } // namespace
 
 PPC::PPC() {
+  GotRel = R_PPC_GLOB_DAT;
   NoneRel = R_PPC_NONE;
+  PltRel = R_PPC_JMP_SLOT;
+  RelativeRel = R_PPC_RELATIVE;
   GotBaseSymInGotPlt = false;
+  RelativeRel = R_PPC_RELATIVE;
+  GotEntrySize = 4;
+  PltEntrySize = 12;
+  GotHeaderEntriesNum = 4;
+  PltHeaderSize = 72;
+}
+
+void PPC::writeGotHeader(uint8_t *Buf) const {
+  // _GLOBAL_OFFSET_TABLE[-1] = blrl
+  write32(Buf, 0x4e800021);
+  // _GLOBAL_OFFSET_TABLE_[0] = _DYNAMIC
+  write32(Buf + 4, In.Dynamic->getVA());
+  // _GLOBAL_OFFSET_TABLE_[1,2] are reserved.
 }
 
 RelExpr PPC::getRelExpr(RelType Type, const Symbol &S,
                         const uint8_t *Loc) const {
   switch (Type) {
   case R_PPC_REL14:
   case R_PPC_REL24:
   case R_PPC_REL32:
+  case R_PPC_LOCAL24PC:
+  case R_PPC_REL16_LO:
+  case R_PPC_REL16_HI:
+  case R_PPC_REL16_HA:
     return R_PC;
+  case R_PPC64_GOT16:
+    return R_GOT_OFF;
   case R_PPC_PLTREL24:
     return R_PLT_PC;
   default:
     return R_ABS;
   }
 }
 
 void PPC::relocateOne(uint8_t *Loc, RelType Type, uint64_t Val) const {
   switch (Type) {
   case R_PPC_ADDR16_HA:
-    write16be(Loc, (Val + 0x8000) >> 16);
+  case R_PPC_REL16_HA:
+    write16(Loc, (Val + 0x8000) >> 16);
     break;
   case R_PPC_ADDR16_HI:
-    write16be(Loc, Val >> 16);
+  case R_PPC_REL16_HI:
+    write16(Loc, Val >> 16);
     break;
   case R_PPC_ADDR16_LO:
-    write16be(Loc, Val);
+  case R_PPC_REL16_LO:
+    write16(Loc, Val);
     break;
   case R_PPC_ADDR32:
   case R_PPC_REL32:
-    write32be(Loc, Val);
+    checkInt(Loc, Val, 32, Type);
+    write32(Loc, Val);
     break;
-  case R_PPC_REL14:
-    write32be(Loc, read32be(Loc) | (Val & 0xFFFC));
+  case R_PPC_REL14: {
+    uint32_t Mask = 0x0000FFFC;
+    checkInt(Loc, Val, 16, Type);
+    checkAlignment(Loc, Val, 4, Type);
+    write32(Loc, (read32(Loc) & ~Mask) | (Val & Mask));
+    break;
+  }
+  case R_PPC_GOT16:
+    checkInt(Loc, Val, 16, Type);
+    write16(Loc, Val);
     break;
   case R_PPC_PLTREL24:
   case R_PPC_REL24:
-    write32be(Loc, read32be(Loc) | (Val & 0x3FFFFFC));
+  case R_PPC_LOCAL24PC: {
+    uint32_t Mask = 0x03FFFFFC;
+    checkInt(Loc, Val, 26, Type);
+    checkAlignment(Loc, Val, 4, Type);
+    write32(Loc, (read32(Loc) & ~Mask) | (Val & Mask));
     break;
+  }
   default:
     error(getErrorLocation(Loc) + "unrecognized relocation " + toString(Type));
   }
 }
 
 TargetInfo *elf::getPPCTargetInfo() {
   static PPC Target;
   return &Target;
 }

ELF/Relocations.cpp

static void addRelativeReloc(InputSectionBase *IS, uint64_t OffsetInSec,
   In.RelaDyn->addReloc(Target->RelativeRel, IS, OffsetInSec, Sym, Addend, Expr,
                        Type);
 }
 
 template <class ELFT, class GotPltSection>
 static void addPltEntry(PltSection *Plt, GotPltSection *GotPlt,
                         RelocationBaseSection *Rel, RelType Type, Symbol &Sym) {
   Plt->addEntry<ELFT>(Sym);
+  if (Config->EMachine == EM_PPC) {
+    Rel->addReloc(
+        {Type, Plt, 72 + Sym.getGotPltOffset(), !Sym.IsPreemptible, &Sym, 0});
+  } else {
     GotPlt->addEntry(Sym);
     Rel->addReloc(
         {Type, GotPlt, Sym.getGotPltOffset(), !Sym.IsPreemptible, &Sym, 0});
   }
+}
 
 static void addGotEntry(Symbol &Sym) {
   In.Got->addEntry(Sym);
 
   RelExpr Expr = Sym.isTls() ? R_TLS : R_ABS;
   uint64_t Off = Sym.getGotOffset();
 
   // If a GOT slot value can be calculated at link-time, which is now,

ELF/SyntheticSections.cpp

: SyntheticSection(SHF_ALLOC | SHF_WRITE,
                        Config->EMachine == EM_PPC64 ? ".plt" : ".got.plt") {}
 
 void GotPltSection::addEntry(Symbol &Sym) {
   assert(Sym.PltIndex == Entries.size());
   Entries.push_back(&Sym);
 }
 
 size_t GotPltSection::getSize() const {
   return (Target->GotPltHeaderEntriesNum + Entries.size()) *
          Target->GotPltEntrySize;
 }
 
 void GotPltSection::writeTo(uint8_t *Buf) {
   Target->writeGotPltHeader(Buf);
   Buf += Target->GotPltHeaderEntriesNum * Target->GotPltEntrySize;
   for (const Symbol *B : Entries) {
     Target->writeGotPlt(Buf, *B);
     Buf += Config->Wordsize;
for (const SymbolTableEntry &S : In.DynSymTab->getSymbols()) {
     Chains[I] = Buckets[Hash];
     write32(Buckets + Hash, I);
   }
 }
 
 // On PowerPC64 the lazy symbol resolvers go into the `global linkage table`
 // in the .glink section, rather then the typical .plt section.
 PltSection::PltSection(bool IsIplt)
-    : SyntheticSection(SHF_ALLOC | SHF_EXECINSTR, SHT_PROGBITS, 16,
+    : SyntheticSection(SHF_ALLOC | SHF_EXECINSTR |
+                           (Config->EMachine == EM_PPC ? SHF_WRITE : 0),
+                       Config->EMachine == EM_PPC ? SHT_NOBITS : SHT_PROGBITS,
+                       Config->EMachine == EM_PPC ? 4 : 16,
                        Config->EMachine == EM_PPC64 ? ".glink" : ".plt"),
       HeaderSize(!IsIplt || Config->ZRetpolineplt ? Target->PltHeaderSize : 0),
       IsIplt(IsIplt) {
   // The PLT needs to be writable on SPARC as the dynamic linker will
   // modify the instructions in the PLT entries.
   if (Config->EMachine == EM_SPARCV9)
     this->Flags |= SHF_WRITE;
 }
 
 void PltSection::writeTo(uint8_t *Buf) {
   // At beginning of PLT or retpoline IPLT, we have code to call the dynamic
   // linker to resolve dynsyms at runtime. Write such code.
   if (HeaderSize)
     Target->writePltHeader(Buf);
   size_t Off = HeaderSize;
 
   RelocationBaseSection *RelSec = IsIplt ? In.RelaIplt : In.RelaPlt;
 
   // The IPlt is immediately after the Plt, account for this in RelOff
   size_t PltOff = IsIplt ? In.Plt->getSize() : 0;
 
   for (size_t I = 0, E = Entries.size(); I != E; ++I) {
     const Symbol *B = Entries[I];
     unsigned RelOff = RelSec->Entsize * I + PltOff;
     uint64_t Got = B->getGotPltVA();
     uint64_t Plt = this->getVA() + Off;
bool ThunkSection::assignOffsets() {
   bool Changed = Off != Size;
   Size = Off;
   return Changed;
 }
 
 // If linking position-dependent code then the table will store the addresses
 // directly in the binary so the section has type SHT_PROGBITS. If linking
 // position-independent code the section has type SHT_NOBITS since it will be
 // allocated and filled in by the dynamic linker.
 PPC64LongBranchTargetSection::PPC64LongBranchTargetSection()
     : SyntheticSection(SHF_ALLOC | SHF_WRITE,
                        Config->Pic ? SHT_NOBITS : SHT_PROGBITS, 8,
                        ".branch_lt") {}
 
 void PPC64LongBranchTargetSection::addEntry(Symbol &Sym) {
   assert(Sym.PPC64BranchltIndex == 0xffff);
   Sym.PPC64BranchltIndex = Entries.size();

ELF/Writer.cpp

void elf::addReservedSymbols() {
   if (Symbol *S = Symtab->find(GotSymName)) {
     if (S->isDefined()) {
       error(toString(S->File) + " cannot redefine linker defined symbol '" +
             GotSymName + "'");
       return;
     }
 
     uint64_t GotOff = 0;
-    if (Config->EMachine == EM_PPC || Config->EMachine == EM_PPC64)
+    if (Config->EMachine == EM_PPC)
+      GotOff = 4;
+    else if (Config->EMachine == EM_PPC64)
       GotOff = 0x8000;
 
     Symtab->addSymbol(Defined{/*File=*/nullptr, GotSymName, STB_GLOBAL,
                               STV_HIDDEN, STT_NOTYPE, GotOff, /*Size=*/0,
                               Out::ElfHeader});
     ElfSym::GlobalOffsetTable = cast<Defined>(S);
   }
 
+  if (Config->EMachine == EM_PPC)
+    if (Symbol *S = Symtab->find("_SDA_BASE_"))
+      if (!S->isDefined())
+        S->resolve(Defined{/*File=*/nullptr, "_SDA_BASE_", STB_GLOBAL,
+                           STV_HIDDEN, STT_NOTYPE, 4, /*Size=*/0, In.Got});
+
   // __ehdr_start is the location of ELF file headers. Note that we define
   // this symbol unconditionally even when using a linker script, which
   // differs from the behavior implemented by GNU linker which only define
   // this symbol if ELF headers are in the memory mapped segment.
   addOptionalRegular("__ehdr_start", Out::ElfHeader, 0, STV_HIDDEN);
 
   // __executable_start is not documented, but the expectation of at
   // least the Android libc is that it points to the ELF header.