[ELF][PPC64] Synthesize _savegpr[01]_{14..31} and _restgpr[01]_{14..31}

[MaskRay]

Summary

In the 64-bit ELF V2 API Specification: Power Architecture, 2.3.3.1. GPR
Save and Restore Functions defines some special functions which may be
referenced by GCC produced assembly (LLVM does not reference them).

With GCC -Os, when the number of call-saved registers exceeds a certain
threshold, GCC generates _savegpr0_* _restgpr0_* calls and expects the
linker to define them. See
https://sourceware.org/pipermail/binutils/2002-February/017444.html and
https://sourceware.org/pipermail/binutils/2004-August/036765.html . This
is weird because libgcc.a would be the natural place. However, the linker
generation approach has the advantage that the linker can generate
multiple copies to avoid long branch thunks. We don't consider the
advantage significant enough to complicate our trunk implementation, so
we take a simple approach.

• Check whether _savegpr[01]_{14..31} and _restgpr[01]_{14..31} are used
• If yes, assemble the predefined .s and add the object file as LazyObjFile.

[MaskRay]

lld/ELF/Arch/PPC64SaveRest.cpp

61 ↗

(On Diff #264134)

I should add .hidden as well.

lld/test/ELF/ppc64-savegpr1.s

15

I should change one test to check -shared

[MaskRay]

adjust

[MaskRay]

Better approach.
Synthesize InputSections

[Bdragon28]

lld/ELF/Arch/PPC64.cpp

165

The example in the ABI doc was illustrating that you can split it into multiple chunks if you make sure both sides finish off saving the rest of the registers. It's actually cleaner to just have it continue dropping through until the end.

I believe the reason for the ABI doc having the example like it is is probably something to do with r30,r31 save/restore being much more common than full save/restore so binutils ld has a small and a large version.

Since we're not bothering with sticking multiple copies in anyway, it doesn't matter and we can go with the simpler implementation here of not treating r29 differently.

179

These are at fixed offsets to catch counting errors in the code above, correct?

[MaskRay]

Simplify _restgpr0_*

lld/ELF/Arch/PPC64.cpp

165

Thanks for the comment. Let me simplify _restgpr0_*

This is probably some microarchitectural optimization we don't care much.

179

Yes.

[sfertile]

Thanks for implementing this MaskRay. I've left a couple comments, but still need to go through the testing.

lld/ELF/Arch/PPC64.cpp

133

Since we are always writing the entire routine into the buffer anyway why not just hard code the array contents so we don't have to compute it at runtime?

143

real minor nit: I think the code would be a bit clearer if this check cake outside the lamda: eg

if (!defined.empty())
  addSection(...)

152

Does it clutter the code too much to keep track of the correct value in the loop that calls `addOptional' and not have to adjust this later?

[MaskRay]

lld/ELF/Arch/PPC64.cpp

133

We need to define symbols _savegpr* and _restgpr*, so a loop is needed. There is little saving if we move instructions alone to a constant array.

143

Mostly there are 4 callers. Placing it here saves some code.

152

Yes, the alternative approach will clutter the code. It will require 2 loops, with one finding the start point and the second placing instructions and definitions. This post adjustment approach is the simplest I can find.

[sfertile]

lld/ELF/Arch/PPC64.cpp

133

The savings mostly come from making the 4 loops we have now (and the 2 that need to be added later for the fpr save/restore routines) nearly identical. We can then write a helper function or lambda and call that 6 time instead of having 6 loops once we add support for _savefpr* and _restfpr* as well. With a minor tweak I believe the helper would also work for the vector save restore routines as well.

Something like:

Helper(uint32_t *Buffer, uint32_t BufferSize, StringRef Prefix) {
  int r = findFirstOf(prefix);  // returns the register number corresponding to the first symbol we need to define.
  // Any referenced symbols are defined, nothing to do.
  if (r == 32)
  return;

  int Value = 0;
  const int Offset = 4 * (r - 14);
  for (r < 32; ++r) {
   Name = FormatName(prefix, r)
    addOptional(Name, Value, defined);
    Value += 4;
  }
  // No need to check if defined is empty becuase we would have returned already if no symbols need to be defiend.
  addSection(Buffer + Offset, Buffer + Size, defined);
}

Its longer and more verbose then one of the for loops, but I think its much easier to follow the logic, and then it only takes 6 lines like:

Helper(savegpr0, ArraySize(savegpr0), "__savegpr0_");

Its more lines of code total, since defining the arrays adds more lines then we save with the replacing the for loops with the helper but I think its much more approachable for a reader.

152

I think this approach is simpler in terms of less lines of code, but not necessarily simpler to read or understand.

[MaskRay]

Simplify

lld/ELF/Arch/PPC64.cpp

133

Thanks for the suggestion. I have found a way to simplify the code.

[MaskRay]

Simplify with MutableArrayRef<uint32_t>

[MaskRay]

@sfertile Looks good now?

[sfertile]

One comment but otherwise LGTM.

lld/ELF/Arch/PPC64.cpp

125

nit: from is always 14 so we don't need it as an argument, we can just always start the loop at 14.

[MaskRay]

Thanks for the review! Will commit soon...

lld/ELF/Arch/PPC64.cpp

125

This is reserved for _savevr_20 and _restvr_20 which start from 20 instead of 14..

lld/ELF/Arch/PPC64.cpp

 //===- PPC64.cpp ----------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
+#include "SymbolTable.h"
 #include "Symbols.h"
 #include "SyntheticSections.h"
 #include "Target.h"
 #include "Thunks.h"
 #include "lld/Common/ErrorHandler.h"
+#include "lld/Common/Memory.h"
 #include "llvm/Support/Endian.h"
 
 using namespace llvm;
 using namespace llvm::object;
 using namespace llvm::support::endian;
 using namespace llvm::ELF;
 using namespace lld;
 using namespace lld::elf;
unsigned elf::getPPC64GlobalEntryToLocalEntryOffset(uint8_t stOther) {
   return 0;
 }
 
 bool elf::isPPC64SmallCodeModelTocReloc(RelType type) {
   // The only small code model relocations that access the .toc section.
   return type == R_PPC64_TOC16 || type == R_PPC64_TOC16_DS;
 }
 
+static bool addOptional(StringRef name, uint64_t value,
+                        std::vector<Defined *> &defined) {
+  Symbol *sym = symtab->find(name);
+  if (!sym || sym->isDefined())
+    return false;
+  sym->resolve(Defined{/*file=*/nullptr, saver.save(name), STB_GLOBAL,
+                       STV_HIDDEN, STT_FUNC, value,
+                       /*size=*/0, /*section=*/nullptr});
+  defined.push_back(cast<Defined>(sym));
+  return true;
+}
+
+// If from is 14, write ${prefix}14: firstInsn; ${prefix}15:
+// firstInsn+0x200008; ...; ${prefix}31: firstInsn+(31-14)*0x200008; $tail
+// The labels are defined only if they exist in the symbol table.
+static void writeSequence(MutableArrayRef<uint32_t> buf, const char *prefix,
+                          int from, uint32_t firstInsn,
+                          ArrayRef<uint32_t> tail) {
+  std::vector<Defined *> defined;
+  char name[16];
+  int first;
+  uint32_t *ptr = buf.data();
+  for (int r = from; r < 32; ++r) {
+    format("%s%d", prefix, r).snprint(name, sizeof(name));
+    if (addOptional(name, 4 * (r - from), defined) && defined.size() == 1)
+      first = r - from;
+    write32(ptr++, firstInsn + 0x200008 * (r - from));
+  }
+  for (uint32_t insn : tail)
+    write32(ptr++, insn);
+  assert(ptr == &*buf.end());
+
+  if (defined.empty())
+    return;
+  // The full section content has the extent of [begin, end). We drop unused
+  // instructions and write [first,end).
+  auto *sec = make<InputSection>(
+      nullptr, SHF_ALLOC, SHT_PROGBITS, 4,
+      makeArrayRef(reinterpret_cast<uint8_t *>(buf.data() + first),
+                   4 * (buf.size() - first)),
+      ".text");
+  inputSections.push_back(sec);
+  for (Defined *sym : defined) {
+    sym->section = sec;
+    sym->value -= 4 * first;
+  }
+}
+
+// Implements some save and restore functions as described by ELF V2 ABI to be
+// compatible with GCC. With GCC -Os, when the number of call-saved registers
+// exceeds a certain threshold, GCC generates _savegpr0_* _restgpr0_* calls and
+// expects the linker to define them. See
+// https://sourceware.org/pipermail/binutils/2002-February/017444.html and
+// https://sourceware.org/pipermail/binutils/2004-August/036765.html . This is
+// weird because libgcc.a would be the natural place. The linker generation
+// approach has the advantage that the linker can generate multiple copies to
+// avoid long branch thunks. However, we don't consider the advantage
+// significant enough to complicate our trunk implementation, so we take the
+// simple approach and synthesize .text sections providing the implementation.
+void elf::addPPC64SaveRestore() {
+  static uint32_t savegpr0[20], restgpr0[21], savegpr1[19], restgpr1[19];
+  constexpr uint32_t blr = 0x4e800020, mtlr_0 = 0x7c0803a6;
+
+  // _restgpr0_14: ld 14, -144(1); _restgpr0_15: ld 15, -136(1); ...
+  // Tail: ld 0, 16(1); mtlr 0; blr
+  writeSequence(restgpr0, "_restgpr0_", 14, 0xe9c1ff70,
+                {0xe8010010, mtlr_0, blr});
+  // _restgpr1_14: ld 14, -144(12); _restgpr1_15: ld 15, -136(12); ...
+  // Tail: blr
+  writeSequence(restgpr1, "_restgpr1_", 14, 0xe9ccff70, {blr});
+  // _savegpr0_14: std 14, -144(1); _savegpr0_15: std 15, -136(1); ...
+  // Tail: std 0, 16(1); blr
+  writeSequence(savegpr0, "_savegpr0_", 14, 0xf9c1ff70, {0xf8010010, blr});
+  // _savegpr1_14: std 14, -144(12); _savegpr1_15: std 15, -136(12); ...
+  // Tail: blr
+  writeSequence(savegpr1, "_savegpr1_", 14, 0xf9ccff70, {blr});
+}
+
 // Find the R_PPC64_ADDR64 in .rela.toc with matching offset.
 template <typename ELFT>
 static std::pair<Defined *, int64_t>
 getRelaTocSymAndAddend(InputSectionBase *tocSec, uint64_t offset) {
   if (tocSec->numRelocations == 0)
     return {};
 
   // .rela.toc contains exclusively R_PPC64_ADDR64 relocations sorted by

lld/ELF/Target.h

 // This function will return the offset (in bytes) from the global entry-point
 // to the local entry-point.
 unsigned getPPC64GlobalEntryToLocalEntryOffset(uint8_t stOther);
 
 // Returns true if a relocation is a small code model relocation that accesses
 // the .toc section.
 bool isPPC64SmallCodeModelTocReloc(RelType type);
 
+void addPPC64SaveRestore();
 uint64_t getPPC64TocBase();
 uint64_t getAArch64Page(uint64_t expr);
 
 extern const TargetInfo *target;
 TargetInfo *getTarget();
 
 template <class ELFT> bool isMipsPIC(const Defined *sym);
 

lld/ELF/Writer.cpp

if (config->emachine == EM_MIPS) {
     // in case of using -mno-shared option.
     // https://sourceware.org/ml/binutils/2004-12/msg00094.html
     if (symtab->find("__gnu_local_gp"))
       ElfSym::mipsLocalGp = addAbsolute("__gnu_local_gp");
   } else if (config->emachine == EM_PPC) {
     // glibc *crt1.o has a undefined reference to _SDA_BASE_. Since we don't
     // support Small Data Area, define it arbitrarily as 0.
     addOptionalRegular("_SDA_BASE_", nullptr, 0, STV_HIDDEN);
+  } else if (config->emachine == EM_PPC64) {
+    addPPC64SaveRestore();
   }
 
   // The Power Architecture 64-bit v2 ABI defines a TableOfContents (TOC) which
   // combines the typical ELF GOT with the small data sections. It commonly
   // includes .got .toc .sdata .sbss. The .TOC. symbol replaces both
   // _GLOBAL_OFFSET_TABLE_ and _SDA_BASE_ from the 32-bit ABI. It is used to
   // represent the TOC base which is offset by 0x8000 bytes from the start of
   // the .got section.

lld/test/ELF/ppc64-restgpr0.s

+# REQUIRES: ppc
+## Test code sequences of synthesized _restgpr0_{14..31}
+
+# RUN: llvm-mc -filetype=obj -triple=ppc64le %s -o %t14.o
+# RUN: ld.lld %t14.o -o %t14
+# RUN: llvm-objdump -d %t14 | FileCheck --check-prefix=R14 %s
+
+# R14-LABEL: <_restgpr0_14>:
+# R14-NEXT:    ld 14, -144(1)
+# R14-NEXT:    ld 15, -136(1)
+# R14-EMPTY:
+# R14-NEXT:  <_restgpr0_16>:
+# R14-NEXT:    ld 16, -128(1)
+# R14:         ld 31, -8(1)
+# R14-NEXT:    ld 0, 16(1)
+# R14-NEXT:    mtlr 0
+# R14-NEXT:    blr
+
+## Don't synthesize _restgpr0_{14..30} because they are unused.
+# RUN: echo 'bl _restgpr0_31' | llvm-mc -filetype=obj -triple=ppc64 - -o %t31.o
+# RUN: ld.lld %t31.o -o %t31
+# RUN: llvm-objdump -d %t31 | FileCheck --check-prefix=R31 %s
+
+# R31-LABEL: Disassembly of section .text:
+# R31-EMPTY:
+# R31-NEXT:  <_restgpr0_31>:
+# R31-NEXT:    ld 31, -8(1)
+# R31-NEXT:    ld 0, 16(1)
+# R31-NEXT:    mtlr 0
+# R31-NEXT:    blr
+
+# RUN: echo 'bl _restgpr0_32' | llvm-mc -filetype=obj -triple=ppc64 - -o %t32.o
+# RUN: not ld.lld %t32.o -o /dev/null
+
+.globl _start
+_start:
+  bl _restgpr0_14
+  bl _restgpr0_16

lld/test/ELF/ppc64-restgpr1.s

+# REQUIRES: ppc
+## Test code sequences of synthesized _restgpr1_{14..31}
+
+# RUN: llvm-mc -filetype=obj -triple=ppc64le %s -o %t14.o
+# RUN: ld.lld %t14.o -o %t14
+# RUN: llvm-objdump -d %t14 | FileCheck --check-prefix=R14 %s
+
+# R14:       <_restgpr1_14>:
+# R14-NEXT:    ld 14, -144(12)
+# R14-NEXT:    ld 15, -136(12)
+# R14-EMPTY:
+# R14-NEXT:  <_restgpr1_16>:
+# R14-NEXT:    ld 16, -128(12)
+# R14:         ld 31, -8(12)
+# R14-NEXT:    blr
+
+## Don't synthesize _restgpr1_{14..30} because they are unused.
+# RUN: echo 'bl _restgpr1_31' | llvm-mc -filetype=obj -triple=ppc64 - -o %t31.o
+# RUN: ld.lld %t31.o -o %t31
+# RUN: llvm-objdump -d %t31 | FileCheck --check-prefix=R31 %s
+
+# R31-LABEL: Disassembly of section .text:
+# R31-EMPTY:
+# R31-NEXT:  <_restgpr1_31>:
+# R31-NEXT:    ld 31, -8(12)
+# R31-NEXT:    blr
+
+# RUN: echo 'bl _restgpr1_32' | llvm-mc -filetype=obj -triple=ppc64le - -o %t32.o
+# RUN: not ld.lld %t32.o -o /dev/null
+
+.globl _start
+_start:
+  bl _restgpr1_14
+  bl _restgpr1_16

lld/test/ELF/ppc64-savegpr0.s

+# REQUIRES: ppc
+## Test code sequences of synthesized _savegpr0_{14..31}
+
+# RUN: llvm-mc -filetype=obj -triple=ppc64le %s -o %t14.o
+# RUN: ld.lld %t14.o -o %t14
+# RUN: llvm-objdump -d %t14 | FileCheck --check-prefix=R14 %s
+
+# R14-LABEL: <_savegpr0_14>:
+# R14-NEXT:    std 14, -144(1)
+# R14-NEXT:    std 15, -136(1)
+# R14-EMPTY:
+# R14-NEXT:  <_savegpr0_16>:
+# R14-NEXT:    std 16, -128(1)
+# R14:         std 31, -8(1)
+# R14-NEXT:    std 0, 16(1)
+# R14-NEXT:    blr
+
+## Don't synthesize _savegpr0_{14..30} because they are unused.
+# RUN: echo 'bl _savegpr0_31' | llvm-mc -filetype=obj -triple=ppc64 - -o %t31.o
+# RUN: ld.lld %t31.o -o %t31
+# RUN: llvm-objdump -d %t31 | FileCheck --check-prefix=R31 %s
+
+# R31-LABEL: Disassembly of section .text:
+# R31-EMPTY:
+# R31-NEXT:  <_savegpr0_31>:
+# R31-NEXT:    std 31, -8(1)
+# R31-NEXT:    std 0, 16(1)
+# R31-NEXT:    blr
+
+# RUN: echo 'bl _savegpr0_32' | llvm-mc -filetype=obj -triple=ppc64 - -o %t32.o
+# RUN: not ld.lld %t32.o -o /dev/null
+
+.globl _start
+_start:
+  bl _savegpr0_14
+  bl _savegpr0_16

lld/test/ELF/ppc64-savegpr1.s

+# REQUIRES: ppc
+## Test code sequences of synthesized _savegpr1_{14..31}
+
+# RUN: llvm-mc -filetype=obj -triple=ppc64le %s -o %t14.o
+# RUN: ld.lld %t14.o -o %t14
+# RUN: llvm-objdump -d %t14 | FileCheck --check-prefix=R14 %s
+
+# R14-LABEL: <_savegpr1_14>:
+# R14-NEXT:    std 14, -144(12)
+# R14-NEXT:    std 15, -136(12)
+# R14-EMPTY:
+# R14-NEXT:  <_savegpr1_16>:
+# R14-NEXT:    std 16, -128(12)
+# R14:         std 31, -8(12)
+# R14-NEXT:    blr
+
+## Don't synthesize _savegpr1_{14..30} because they are unused.
+# RUN: echo 'bl _savegpr1_31' | llvm-mc -filetype=obj -triple=ppc64 - -o %t31.o
+# RUN: ld.lld %t31.o -o %t31
+# RUN: llvm-objdump -d %t31 | FileCheck --check-prefix=R31 %s
+
+# R31-LABEL: Disassembly of section .text:
+# R31-EMPTY:
+# R31-NEXT:  <_savegpr1_31>:
+# R31-NEXT:    std 31, -8(12)
+# R31-NEXT:    blr
+
+# RUN: echo 'bl _savegpr1_32' | llvm-mc -filetype=obj -triple=ppc64le - -o %t32.o
+# RUN: not ld.lld %t32.o -o /dev/null
+
+.globl _start
+_start:
+  bl _savegpr1_14
+  bl _savegpr1_16

lld/test/ELF/ppc64-saveres.s

+# REQUIRES: ppc
+## Test that some save and restore functions can be synthesized.
+## The code sequences are tested by ppc64-restgpr*.s and ppc64-savegpr*.s
+
+# RUN: llvm-mc -filetype=obj -triple=ppc64le %s -o %t.o
+# RUN: ld.lld -shared %t.o -o %t.so
+# RUN: llvm-readelf -s %t.so | FileCheck --check-prefix=NM %s
+# RUN: llvm-objdump -d %t.so | FileCheck %s
+
+## The synthesized symbols are not exported.
+# NM:      FUNC LOCAL HIDDEN {{.*}} _restgpr0_30
+# NM-NEXT: FUNC LOCAL HIDDEN {{.*}} _restgpr1_30
+# NM-NEXT: FUNC LOCAL HIDDEN {{.*}} _savegpr0_30
+# NM-NEXT: FUNC LOCAL HIDDEN {{.*}} _savegpr1_30
+
+# CHECK: 00000000000[[#%x,RESTGPR0:]] <_restgpr0_30>:
+# CHECK: 00000000000[[#%x,RESTGPR1:]] <_restgpr1_30>:
+# CHECK: 00000000000[[#%x,SAVEGPR0:]] <_savegpr0_30>:
+# CHECK: 00000000000[[#%x,SAVEGPR1:]] <_savegpr1_30>:
+# CHECK-LABEL: <_start>:
+# CHECK-NEXT:    bl 0x[[#RESTGPR0]]
+# CHECK-NEXT:    bl 0x[[#RESTGPR1]]
+# CHECK-NEXT:    bl 0x[[#SAVEGPR0]]
+# CHECK-NEXT:    bl 0x[[#SAVEGPR1]]
+
+.globl _start
+_start:
+  bl _restgpr0_30
+  bl _restgpr1_30
+  bl _savegpr0_30
+  bl _savegpr1_30