[PPC64] toc-indirect to toc-relative relaxation

[MaskRay]

Summary

This is based on D54720 by Sean Fertile.

When accessing a global symbol which is not defined in the translation unit,
compilers will generate instructions that load the address from the toc entry.

If the symbol is defined, non-preemptable, and addressable with a 32-bit
signed offset from the toc pointer, the address can be computed
directly. e.g.

addis 3, 2, .LC0@toc@ha  # R_PPC64_TOC16_HA
ld    3, .LC0@toc@l(3)   # R_PPC64_TOC16_LO_DS, load the address from a .toc entry
ld/lwa 3, 0(3)           # load the value from the address

.section .toc,"aw",@progbits
.LC0: .tc var[TC],var

can be relaxed to

addis 3,2,var@toc@ha     # this may be relaxed to a nop,
addi  3,3,var@toc@l      # then this becomes addi 3,2,var@toc
ld/lwa 3, 0(3)           # load the value from the address

We can delete the test ppc64-got-indirect.s as its purpose is covered by
newly added ppc64-toc-relax.s and ppc64-toc-relax-constants.s

[MaskRay]

Revamp tests

[MaskRay]

Delete ppc64-got-indirect.s

[MaskRay]

Delete unused Inputs/ppc64-toc-relax-constants.s

[ruiu]

ELF/Arch/PPC64.cpp

114–115

How fast is this statement? Don't you have to do binary search?

122–125

I'm not familiar with the PPC64 instruction set, and I'm wondering why the relaxed instructions are faster than the original form. Can you add that to the comment?

ELF/Relocations.cpp

1248

nit: please add }.

[MaskRay]

ELF/Arch/PPC64.cpp

114–115

When resolving R_PPC16_TOC_* in a .rela.text section, TocRelIdx is monotonically increasing.
The loop is executed at most num(R_PPC16_TOC_*)+size(.rela.text) times.

The pessimistic case happens when you have many .rela.text* sections (-ffunction-sections or comdat .text*) that contain toc relocations.

When linking clang, the 4 object files with the most numbers of .rela.toc entries:
CheckerRegistry.o (.rela.toc has 951 entries) (8 .rela.text have toc relocations) (1172 toc relocations)
X86ISelLowering.o (.rela.toc has 723 entries) (208 .rela.text have toc relocations) (1108 toc relocations)
ScalarEvolution.o (.rela.toc has 492 entries) (153 .rela.text have toc relocations) (754 toc relocations)
X86FastIsel.o (.rela.toc has 462 entries) (242 .rela.text have toc relocations) (3901 toc relocations)
The 50th has 123 entries in its .rela.toc.

For CheckerRegistry.o, the TocRelIdx optimization decreases the execution times of the loop from at most (951+1172)*951 to at most (951+8)*951. Using linear probing here should be ok.

[MaskRay]

Add {} around the EM_RISCV branch

[MaskRay]

Rebase after the llvm::stable_sort change

[sfertile]

Hi MaskRay.

Thanks for doing this, when you originally described this too me I didn't realize you intended to partition and sort the relocations in the sections other then .rela.toc. This clears up my question regarding the implementation. I'm still a little hesitant on this approach though. Did you profile link times between the 2 approaches? I understand the speed up in number of access (you have outlined that very well in another comment), however that fails to consider both how small the number of relocation in rela.toc is compared to all the relocations in the text section, and how infrequently we have an object file that actually enters the loop.

The main reason I wasn't so concerned over the n^2 look up in the original patch is because we would hit that loop so infrequently, and even though it is technically n^2, in practice the typical object files gcc produces when it does put a constant in the TOC would typically lead to 1 or 2 extra array access rather then n extra array accesses. I've compiled a couple of projects to show what I mean.

Protobuf: 628 objects, 27 where there is a constant in the TOC. (4.3%)
 Missing reloc count               Frequency
       1                             19
       2                              8

Postgres: 704 objects 12 with a constant in the TOC (1.7%)
Missing reloc count                Frequency
        1                            10
        2                             1
        9                             1

FMPEG: 681 objects,  28 with constants in the TOC (4.1%)
Missing reloc count                Frequency
      1                               22
      2                                4
      3                                1
      96                               1

LLVM: 3294 Objects, 355 with constant in the TOC.  (10.8%)
Missing reloc count                Frequency
     1                               180
     2                                61
     3                                48
     4                                22
     5                                13
     6                                 4
     7                                 5
     8                                 3
     9                                 2
    10                                 2
    11                                 3
    12                                 3
    13                                 2
    19                                 1
    22                                 1
    28                                 2
    32                                 1
    40                                 1
    80                                 1

Clearly there are a few objects where the number of missing relocation's does start to get worryingly large (30/40/80/96), but 90%-95% of the files never hit the loop, and 70% of those that do need the loop will have at most 1 or 2 extra array accesses per lookup. Note that this is all compiled with gcc, when compiling with clang as the build compiler we end up with *no* objects falling though to the loop.

FWIW, I think this implementation is clean and understandable enough that we can switch to it, but I would like to know how this affects the link time of say llvm when clang is the build compiler and when gcc is the build compiler before deciding this is the best approach.

[MaskRay]

Switch to Sean Fertile's original approach and update tests.

[MaskRay]

In D60958#1475755, @sfertile wrote:

Hi MaskRay.

Thanks for doing this, when you originally described this too me I didn't realize you intended to partition and sort the relocations in the sections other then .rela.toc. This clears up my question regarding the implementation. I'm still a little hesitant on this approach though. Did you profile link times between the 2 approaches? I understand the speed up in number of access (you have outlined that very well in another comment), however that fails to consider both how small the number of relocation in rela.toc is compared to all the relocations in the text section, and how infrequently we have an object file that actually enters the loop.

The main reason I wasn't so concerned over the n^2 look up in the original patch is because we would hit that loop so infrequently, and even though it is technically n^2, in practice the typical object files gcc produces when it does put a constant in the TOC would typically lead to 1 or 2 extra array access rather then n extra array accesses. I've compiled a couple of projects to show what I mean.

Protobuf: 628 objects, 27 where there is a constant in the TOC. (4.3%)
 Missing reloc count               Frequency
       1                             19
       2                              8

Postgres: 704 objects 12 with a constant in the TOC (1.7%)
Missing reloc count                Frequency
        1                            10
        2                             1
        9                             1

FMPEG: 681 objects,  28 with constants in the TOC (4.1%)
Missing reloc count                Frequency
      1                               22
      2                                4
      3                                1
      96                               1

LLVM: 3294 Objects, 355 with constant in the TOC.  (10.8%)
Missing reloc count                Frequency
     1                               180
     2                                61
     3                                48
     4                                22
     5                                13
     6                                 4
     7                                 5
     8                                 3
     9                                 2
    10                                 2
    11                                 3
    12                                 3
    13                                 2
    19                                 1
    22                                 1
    28                                 2
    32                                 1
    40                                 1
    80                                 1

Clearly there are a few objects where the number of missing relocation's does start to get worryingly large (30/40/80/96), but 90%-95% of the files never hit the loop, and 70% of those that do need the loop will have at most 1 or 2 extra array accesses per lookup. Note that this is all compiled with gcc, when compiling with clang as the build compiler we end up with *no* objects falling though to the loop.

FWIW, I think this implementation is clean and understandable enough that we can switch to it, but I would like to know how this affects the link time of say llvm when clang is the build compiler and when gcc is the build compiler before deciding this is the best approach.

Hi Sean,

Thank you for the statistics and explanation!

As I updated some PPC64 tests yesterday, now I think I have a better understanding of what these relaxations are... I didn't really understand the R_PPC64_ADDR64 relocations in .toc well until now. I think your approach in D54720 is superior. (Though I tested an earlier version of this revision on numerous internal targets and didn't see issues caused by it (there are many unrelated long-standing issues))

I now understand that .rela.toc consists of exclusively R_PPC64_ADDR64 relocations. What you mentioned before is that clang/llvm never emits constants into .toc so the linear loop can't be a problem. For gcc, as your statistics say, it isn't a problem either given the very small number of such cases.

I switched to your approach but tried inlining some functions (fixed an 32-bit compile error and an assertion error when Relas is empty, etc) and adjusted some code to make it (hopefully) easier to understand.

[MaskRay]

Switch to sfertile's original approach.

[MaskRay]

Enhance ppc64-toc-relax.s : add another symbol hidden2

[MaskRay]

Elaborate what this relaxation does

[MaskRay]

ELF/Arch/PPC64.cpp

122–125

Elaborated the comments.

[MaskRay]

Improve comments

[MaskRay]

Better comments

[sfertile]

Patch looks really good. I think you managed to explain the toc-indirection without getting to verbose which is something I've struggled with. I have a couple minor suggestions and I need to go over the test a bit more indepth but overall this looks good to me.

ELF/Arch/PPC64.cpp

122

I think if Relas.size() is not equal to zero, then we want to set Index to index the last element of the array, and perform the linear probing. For example trying to relax an access to last element of a .toc section that contains a constant as the first entry and a symbol as the second: Index will be 1, but Relas[0] is the relocation we want to find to perform the relaxation.

139

I think we need to make a small change to this comment. The compiler generates the indirection by adding a private label at the address of the toc entry for the symbol, and the @toc symbol modifier indicates to the assembler to create a toc-relative relocation for that private label. If we relax then we rewrite the instructions to access the symbol toc-relative:

// .toc
// ...
// .LC0:
//  .tc foo               # will allocate 8 bytes storage and a R_PPC64_ADDR64 relocation for 'foo'


//  addis 3, 2, LC0@toc@ha    # R_PPC64_TOC16_HA
//  ld 3, LC0@toc@l(3)        # R_PPC64_TOC16_LO_DS, load the address of 'foo' from the .toc entry
//  ld/lwa  3, 0(3)           # load the value of foo


// If foo is defined, non-preemptable and addressable with a 32-bit signed
// offset from the toc base, the address can be computed by adding an offset to
// the toc base, saving a load.

//  addis 3, 2, foo@toc@ha ...
`

359

Need to make similar adjustments to this comment and the next.

ELF/InputSection.cpp

73

Is this related or is it from a different patch?

[MaskRay]

ELF/Arch/PPC64.cpp

122

My bad. The test ppc64-toc-relax-constants.s had demonstrated the failure. Fixed.

139

Reworded.

ELF/InputSection.cpp

73

It is related to an assertion failure when relas() is called with zero relocations.

template <class ELFT> ArrayRef<typename ELFT::Rela> relas() const {
  assert(AreRelocsRela); // If relas is empty(), this is false
  return llvm::makeArrayRef(
      static_cast<const typename ELFT::Rela *>(FirstRelocation),
      NumRelocations);
}

I have switched to early return if TocReloc->NumRelocations == 0. The assertion will not fire.

[MaskRay]

Update description based on sfertile's review

[sfertile]

LGTM.

test/ELF/ppc64-toc-relax-constants.s

48

remove the : 8*2 - 0x8000 = -32752 part of the comment.

test/ELF/ppc64-toc-relax.s

45

minor nit: why not check for the relocation for shared 0x0 as well.

70

Maybe, "The definition of 'shared' used is determined at loadtime (by the dynamic linker) so the extra indirection cannot be relaxed."

[MaskRay]

Address sfertile's comments

[MaskRay]

Fix a trailing comma in a comment

[MaskRay]

@ruiu Do you have more suggestions? :)

[ruiu]

LGTM

ELF/Arch/PPC64.cpp

113

Nice comment. Thank you for writing this.

ELF/Arch/PPC64.cpp

unsigned elf::getPPC64GlobalEntryToLocalEntryOffset(uint8_t StOther) {
   return 0;
 }
 
 bool elf::isPPC64SmallCodeModelTocReloc(RelType Type) {
   // The only small code model relocations that access the .toc section.
   return Type == R_PPC64_TOC16 || Type == R_PPC64_TOC16_DS;
 }
 
+// Find the R_PPC64_ADDR64 in .rela.toc with matching offset.
+template <typename ELFT>
+static std::pair<Defined *, int64_t>
+getRelaTocSymAndAddend(InputSectionBase *TocSec, uint64_t Offset) {
+  if (TocSec->NumRelocations == 0)
+    return {};
+
+  // .rela.toc contains exclusively R_PPC64_ADDR64 relocations sorted by
+  // r_offset: 0, 8, 16, etc. For a given Offset, Offset / 8 gives us the
+  // relocation index in most cases.
+  //
+  // In rare cases a TOC entry may store a constant that doesn't need an
+  // R_PPC64_ADDR64, the corresponding r_offset is therefore missing. Offset / 8
+  // points to a relocation with larger r_offset. Do a linear probe then.
+  // Constants are extremely uncommon in .toc and the extra number of array
+  // accesses can be seen as a small constant.
+  ArrayRef<typename ELFT::Rela> Relas = TocSec->template relas<ELFT>();
+  uint64_t Index = std::min<uint64_t>(Offset / 8, Relas.size() - 1);
+  for (;;) {
+    if (Relas[Index].r_offset == Offset) {
+      Symbol &Sym = TocSec->getFile<ELFT>()->getRelocTargetSym(Relas[Index]);
+      return {dyn_cast<Defined>(&Sym), getAddend<ELFT>(Relas[Index])};
+    }
+    if (Relas[Index].r_offset < Offset || Index == 0)
+      break;
+    --Index;
+  }
+  return {};
+}
+
+// When accessing a symbol defined in another translation unit, compilers
+// reserve a .toc entry, allocate a local label and generate toc-indirect
+// instuctions:
+//
+//   addis 3, 2, .LC0@toc@ha  # R_PPC64_TOC16_HA
+//   ld    3, .LC0@toc@l(3)   # R_PPC64_TOC16_LO_DS, load the address from a .toc entry
+//   ld/lwa 3, 0(3)           # load the value from the address
+//
+//   .section .toc,"aw",@progbits
+//   .LC0: .tc var[TC],var
+//
+// If var is defined, non-preemptable and addressable with a 32-bit signed
+// offset from the toc base, the address of var can be computed by adding an
+// offset to the toc base, saving a load.
+//
+//   addis 3,2,var@toc@ha     # this may be relaxed to a nop,
+//   addi  3,3,var@toc@l      # then this becomes addi 3,2,var@toc
+//   ld/lwa 3, 0(3)           # load the value from the address
+//
+// Returns true if the relaxation is performed.
+bool elf::tryRelaxPPC64TocIndirection(RelType Type, const Relocation &Rel,
+                                      uint8_t *BufLoc) {
+  assert(Config->TocOptimize);
+  if (Rel.Addend < 0)
+    return false;
+
+  // If the symbol is not the .toc section, this isn't a toc-indirection.
+  Defined *DefSym = dyn_cast<Defined>(Rel.Sym);
+  if (!DefSym || !DefSym->isSection() || DefSym->Section->Name != ".toc")
+    return false;
+
+  Defined *D;
+  int64_t Addend;
+  auto *TocISB = cast<InputSectionBase>(DefSym->Section);
+  std::tie(D, Addend) =
+      Config->IsLE ? getRelaTocSymAndAddend<ELF64LE>(TocISB, Rel.Addend)
+                   : getRelaTocSymAndAddend<ELF64BE>(TocISB, Rel.Addend);
+
+  // Only non-preemptable defined symbols can be relaxed.
+  if (!D || D->IsPreemptible)
+    return false;
+
+  // Two instructions can materialize a 32-bit signed offset from the toc base.
+  uint64_t TocRelative = D->getVA(Addend) - getPPC64TocBase();
+  if (!isInt<32>(TocRelative))
+    return false;
+
+  // Add PPC64TocOffset that will be subtracted by relocateOne().
+  Target->relaxGot(BufLoc, Type, TocRelative + PPC64TocOffset);
+  return true;
+}
+
 namespace {
 class PPC64 final : public TargetInfo {
 public:
   PPC64();
   int getTlsGdRelaxSkip(RelType Type) const override;
   uint32_t calcEFlags() const override;
   RelExpr getRelExpr(RelType Type, const Symbol &S,
                      const uint8_t *Loc) const override;
   void writePltHeader(uint8_t *Buf) const override;
   void writePlt(uint8_t *Buf, uint64_t GotPltEntryAddr, uint64_t PltEntryAddr,
                 int32_t Index, unsigned RelOff) const override;
   void relocateOne(uint8_t *Loc, RelType Type, uint64_t Val) const override;
   void writeGotHeader(uint8_t *Buf) const override;
   bool needsThunk(RelExpr Expr, RelType Type, const InputFile *File,
                   uint64_t BranchAddr, const Symbol &S) const override;
   bool inBranchRange(RelType Type, uint64_t Src, uint64_t Dst) const override;
   RelExpr adjustRelaxExpr(RelType Type, const uint8_t *Data,
                           RelExpr Expr) const override;
+  void relaxGot(uint8_t *Loc, RelType Type, uint64_t Val) const override;
   void relaxTlsGdToIe(uint8_t *Loc, RelType Type, uint64_t Val) const override;
   void relaxTlsGdToLe(uint8_t *Loc, RelType Type, uint64_t Val) const override;
   void relaxTlsLdToLe(uint8_t *Loc, RelType Type, uint64_t Val) const override;
   void relaxTlsIeToLe(uint8_t *Loc, RelType Type, uint64_t Val) const override;
 
   bool adjustPrologueForCrossSplitStack(uint8_t *Loc, uint8_t *End,
                                         uint8_t StOther) const override;
 };
for (InputFile *F : ObjectFiles) {
     if (Flag == 1)
       error(toString(F) + ": ABI version 1 is not supported");
     else if (Flag > 2)
       error(toString(F) + ": unrecognized e_flags: " + Twine(Flag));
   }
   return 2;
 }
 
+void PPC64::relaxGot(uint8_t *Loc, RelType Type, uint64_t Val) const {
+  switch (Type) {
+  case R_PPC64_TOC16_HA:
+    // Convert "addis reg, 2, .LC0@toc@h" to "addis reg, 2, var@toc@h" or "nop".
+    relocateOne(Loc, Type, Val);
+    break;
+  case R_PPC64_TOC16_LO_DS: {
+    // Convert "ld reg, .LC0@toc@l(reg)" to "addi reg, reg, var@toc@l" or
+    // "addi reg, 2, var@toc".
+    uint32_t Instr = readInstrFromHalf16(Loc);
+    if (getPrimaryOpCode(Instr) != LD)
+      error("expected a 'ld' for got-indirect to toc-relative relaxing");
+    writeInstrFromHalf16(Loc, (Instr & 0x03FFFFFF) | 0x38000000);
+    relocateOne(Loc, R_PPC64_TOC16_LO, Val);
+    break;
+  }
+  default:
+    llvm_unreachable("unexpected relocation type");
+  }
+}
+
 void PPC64::relaxTlsGdToLe(uint8_t *Loc, RelType Type, uint64_t Val) const {
   // Reference: 3.7.4.2 of the 64-bit ELF V2 abi supplement.
   // The general dynamic code sequence for a global `x` will look like:
   // Instruction                    Relocation                Symbol
   // addis r3, r2, x@got@tlsgd@ha   R_PPC64_GOT_TLSGD16_HA      x
   // addi  r3, r3, x@got@tlsgd@l    R_PPC64_GOT_TLSGD16_LO      x
   // bl __tls_get_addr(x@tlsgd)     R_PPC64_TLSGD               x
   //                                R_PPC64_REL24               __tls_get_addr
RelExpr PPC64::getRelExpr(RelType Type, const Symbol &S,
   case R_PPC64_GOT16_DS:
   case R_PPC64_GOT16_HA:
   case R_PPC64_GOT16_HI:
   case R_PPC64_GOT16_LO:
   case R_PPC64_GOT16_LO_DS:
     return R_GOT_OFF;
   case R_PPC64_TOC16:
   case R_PPC64_TOC16_DS:
-  case R_PPC64_TOC16_HA:
   case R_PPC64_TOC16_HI:
   case R_PPC64_TOC16_LO:
-  case R_PPC64_TOC16_LO_DS:
     return R_GOTREL;
+  case R_PPC64_TOC16_HA:
+  case R_PPC64_TOC16_LO_DS:
+    return Config->TocOptimize ? R_PPC64_RELAX_TOC : R_GOTREL;
   case R_PPC64_TOC:
     return R_PPC_TOC;
   case R_PPC64_REL14:
   case R_PPC64_REL24:
     return R_PPC_CALL_PLT;
   case R_PPC64_REL16_LO:
   case R_PPC64_REL16_HA:
   case R_PPC64_REL32:

ELF/Arch/X86_64.cpp

public:
   void writeGotPlt(uint8_t *Buf, const Symbol &S) const override;
   void writePltHeader(uint8_t *Buf) const override;
   void writePlt(uint8_t *Buf, uint64_t GotPltEntryAddr, uint64_t PltEntryAddr,
                 int32_t Index, unsigned RelOff) const override;
   void relocateOne(uint8_t *Loc, RelType Type, uint64_t Val) const override;
 
   RelExpr adjustRelaxExpr(RelType Type, const uint8_t *Data,
                           RelExpr Expr) const override;
-  void relaxGot(uint8_t *Loc, uint64_t Val) const override;
+  void relaxGot(uint8_t *Loc, RelType Type, uint64_t Val) const override;
   void relaxTlsGdToIe(uint8_t *Loc, RelType Type, uint64_t Val) const override;
   void relaxTlsGdToLe(uint8_t *Loc, RelType Type, uint64_t Val) const override;
   void relaxTlsIeToLe(uint8_t *Loc, RelType Type, uint64_t Val) const override;
   void relaxTlsLdToLe(uint8_t *Loc, RelType Type, uint64_t Val) const override;
   bool adjustPrologueForCrossSplitStack(uint8_t *Loc, uint8_t *End,
                                         uint8_t StOther) const override;
 };
 } // namespace
static void relaxGotNoPic(uint8_t *Loc, uint64_t Val, uint8_t Op,
   // See "3.2 INSTRUCTIONS (A-M)" (Vol. 2A 3-15),
   // "INSTRUCTION SET REFERENCE, N-Z" (Vol. 2B 4-1) for
   // descriptions about each operation.
   Loc[-2] = 0x81;
   Loc[-3] = (Rex & ~0x4) | (Rex & 0x4) >> 2;
   write32le(Loc, Val);
 }
 
-void X86_64::relaxGot(uint8_t *Loc, uint64_t Val) const {
+void X86_64::relaxGot(uint8_t *Loc, RelType Type, uint64_t Val) const {
   const uint8_t Op = Loc[-2];
   const uint8_t ModRm = Loc[-1];
 
   // Convert "mov foo@GOTPCREL(%rip),%reg" to "lea foo(%rip),%reg".
   if (Op == 0x8b) {
     Loc[-2] = 0x8d;
     write32le(Loc, Val);
     return;

ELF/InputSection.cpp

: SectionBase(SectionKind, Name, Flags, Entsize, Alignment, Type, Info,
       File(File), RawData(Data) {
   // In order to reduce memory allocation, we assume that mergeable
   // sections are smaller than 4 GiB, which is not an unreasonable
   // assumption as of 2017.
   if (SectionKind == SectionBase::Merge && RawData.size() > UINT32_MAX)
     error(toString(this) + ": section too large");
 
   NumRelocations = 0;
   AreRelocsRela = false;
   Debug = Name.startswith(".debug") || Name.startswith(".zdebug");
 
   // The ELF spec states that a value of 0 means the section has
   // no alignment constraits.
   uint32_t V = std::max<uint32_t>(Alignment, 1);
   if (!isPowerOf2_64(V))
     fatal(toString(this) + ": sh_addralign is not a power of 2");
   this->Alignment = V;
static uint64_t getRelocTargetVA(const InputFile *File, RelType Type, int64_t A,
   case R_GOT:
   case R_RELAX_TLS_GD_TO_IE_ABS:
     return Sym.getGotVA() + A;
   case R_GOTONLY_PC:
     return In.Got->getVA() + A - P;
   case R_GOTPLTONLY_PC:
     return In.GotPlt->getVA() + A - P;
   case R_GOTREL:
+  case R_PPC64_RELAX_TOC:
     return Sym.getVA(A) - In.Got->getVA();
   case R_GOTPLTREL:
     return Sym.getVA(A) - In.GotPlt->getVA();
   case R_GOTPLT:
   case R_RELAX_TLS_GD_TO_IE_GOTPLT:
     return Sym.getGotVA() + A - In.GotPlt->getVA();
   case R_TLSLD_GOT_OFF:
   case R_GOT_OFF:
for (const Relocation &Rel : Relocations) {
     RelExpr Expr = Rel.Expr;
     uint64_t TargetVA = SignExtend64(
         getRelocTargetVA(File, Type, Rel.Addend, AddrLoc, *Rel.Sym, Expr),
         Bits);
 
     switch (Expr) {
     case R_RELAX_GOT_PC:
     case R_RELAX_GOT_PC_NOPIC:
-      Target->relaxGot(BufLoc, TargetVA);
+      Target->relaxGot(BufLoc, Type, TargetVA);
+      break;
+    case R_PPC64_RELAX_TOC:
+      if (!tryRelaxPPC64TocIndirection(Type, Rel, BufLoc))
+        Target->relocateOne(BufLoc, Type, TargetVA);
       break;
     case R_RELAX_TLS_IE_TO_LE:
       Target->relaxTlsIeToLe(BufLoc, Type, TargetVA);
       break;
     case R_RELAX_TLS_LD_TO_LE:
     case R_RELAX_TLS_LD_TO_LE_ABS:
       Target->relaxTlsLdToLe(BufLoc, Type, TargetVA);
       break;

ELF/Relocations.h

enum RelExpr {
   R_MIPS_GOT_LOCAL_PAGE,
   R_MIPS_GOT_OFF,
   R_MIPS_GOT_OFF32,
   R_MIPS_TLSGD,
   R_MIPS_TLSLD,
   R_PPC_CALL,
   R_PPC_CALL_PLT,
   R_PPC_TOC,
+  R_PPC64_RELAX_TOC,
   R_RISCV_PC_INDIRECT,
 };
 
 // Architecture-neutral representation of relocation.
 struct Relocation {
   RelExpr Expr;
   RelType Type;
   uint64_t Offset;

ELF/Relocations.cpp

return oneof<R_GOT, R_GOT_OFF, R_HEXAGON_GOT, R_MIPS_GOT_LOCAL_PAGE,
                R_MIPS_GOT_OFF, R_MIPS_GOT_OFF32, R_AARCH64_GOT_PAGE_PC,
                R_GOT_PC, R_GOTPLT>(Expr);
 }
 
 // True if this expression is of the form Sym - X, where X is a position in the
 // file (PC, or GOT for example).
 static bool isRelExpr(RelExpr Expr) {
   return oneof<R_PC, R_GOTREL, R_GOTPLTREL, R_MIPS_GOTREL, R_PPC_CALL,
-               R_PPC_CALL_PLT, R_AARCH64_PAGE_PC, R_RELAX_GOT_PC>(Expr);
+               R_PPC64_RELAX_TOC, R_PPC_CALL_PLT, R_AARCH64_PAGE_PC, R_RELAX_GOT_PC>(Expr);
 }
 
 // Returns true if a given relocation can be computed at link-time.
 //
 // For instance, we know the offset from a relocation to its target at
 // link-time if the relocation is PC-relative and refers a
 // non-interposable function in the same executable. This function
 // will return true for such relocation.
 //
 // If this function returns false, that means we need to emit a
 // dynamic relocation so that the relocation will be fixed at load-time.
 static bool isStaticLinkTimeConstant(RelExpr E, RelType Type, const Symbol &Sym,
                                      InputSectionBase &S, uint64_t RelOff) {
   // These expressions always compute a constant
   if (oneof<R_DTPREL, R_GOTPLT, R_GOT_OFF, R_HEXAGON_GOT, R_TLSLD_GOT_OFF,
             R_MIPS_GOT_LOCAL_PAGE, R_MIPS_GOTREL, R_MIPS_GOT_OFF,
             R_MIPS_GOT_OFF32, R_MIPS_GOT_GP_PC, R_MIPS_TLSGD,
             R_AARCH64_GOT_PAGE_PC, R_GOT_PC, R_GOTONLY_PC, R_GOTPLTONLY_PC,
             R_PLT_PC, R_TLSGD_GOT, R_TLSGD_GOTPLT, R_TLSGD_PC, R_PPC_CALL_PLT,
-            R_TLSDESC_CALL, R_AARCH64_TLSDESC_PAGE, R_HINT, R_TLSLD_HINT,
+            R_PPC64_RELAX_TOC, R_TLSDESC_CALL, R_AARCH64_TLSDESC_PAGE, R_HINT, R_TLSLD_HINT,
             R_TLSIE_HINT>(E))
     return true;
 
   // These never do, except if the entire file is position dependent or if
   // only the low bits are used.
   if (E == R_GOT || E == R_PLT || E == R_TLSDESC)
     return Target->usesOnlyLowPageBits(Type) || !Config->Pic;
 
static void scanReloc(InputSectionBase &Sec, OffsetGetter &GetOffset, RelTy *&I,
   }
 
   // If the relocation does not emit a GOT or GOTPLT entry but its computation
   // uses their addresses, we need GOT or GOTPLT to be created.
   //
   // The 4 types that relative GOTPLT are all x86 and x86-64 specific.
   if (oneof<R_GOTPLTONLY_PC, R_GOTPLTREL, R_GOTPLT, R_TLSGD_GOTPLT>(Expr)) {
     In.GotPlt->HasGotPltOffRel = true;
-  } else if (oneof<R_GOTONLY_PC, R_GOTREL, R_PPC_TOC>(Expr)) {
+  } else if (oneof<R_GOTONLY_PC, R_GOTREL, R_PPC_TOC, R_PPC64_RELAX_TOC>(Expr)) {
     In.Got->HasGotOffRel = true;
   }
 
   // Read an addend.
   int64_t Addend = computeAddend<ELFT>(Rel, End, Sec, Expr, Sym.isLocal());
 
   // Process some TLS relocations, including relaxing TLS relocations.
   // Note that this function does not handle all TLS relocations.
static void scanRelocs(InputSectionBase &Sec, ArrayRef<RelTy> Rels) {
   OffsetGetter GetOffset(Sec);
 
   // Not all relocations end up in Sec.Relocations, but a lot do.
   Sec.Relocations.reserve(Rels.size());
 
   for (auto I = Rels.begin(), End = Rels.end(); I != End;)
     scanReloc<ELFT>(Sec, GetOffset, I, End);
 
-  // Sort relocations by offset to binary search for R_RISCV_PCREL_HI20
-  if (Config->EMachine == EM_RISCV)
+  // Sort relocations by offset for more efficient searching for
+  // R_RISCV_PCREL_HI20 and R_PPC64_ADDR64.
+  if (Config->EMachine == EM_RISCV ||
+      (Config->EMachine == EM_PPC64 && Sec.Name == ".toc"))
     llvm::stable_sort(Sec.Relocations,
                       [](const Relocation &LHS, const Relocation &RHS) {
                         return LHS.Offset < RHS.Offset;
                       });
 }
 
 template <class ELFT> void elf::scanRelocations(InputSectionBase &S) {
   if (S.AreRelocsRela)
     scanRelocs<ELFT>(S, S.relas<ELFT>());
   else

ELF/Target.h

public:
 
   // If a target needs to rewrite calls to __morestack to instead call
   // __morestack_non_split when a split-stack enabled caller calls a
   // non-split-stack callee this will return true. Otherwise returns false.
   bool NeedsMoreStackNonSplit = true;
 
   virtual RelExpr adjustRelaxExpr(RelType Type, const uint8_t *Data,
                                   RelExpr Expr) const;
-  virtual void relaxGot(uint8_t *Loc, uint64_t Val) const;
+  virtual void relaxGot(uint8_t *Loc, RelType Type, uint64_t Val) const;
   virtual void relaxTlsGdToIe(uint8_t *Loc, RelType Type, uint64_t Val) const;
   virtual void relaxTlsGdToLe(uint8_t *Loc, RelType Type, uint64_t Val) const;
   virtual void relaxTlsIeToLe(uint8_t *Loc, RelType Type, uint64_t Val) const;
   virtual void relaxTlsLdToLe(uint8_t *Loc, RelType Type, uint64_t Val) const;
 
 protected:
   // On FreeBSD x86_64 the first page cannot be mmaped.
   // On Linux that is controled by vm.mmap_min_addr. At least on some x86_64

 
 // Returns input section and corresponding source string for the given location.
 ErrorPlace getErrorPlace(const uint8_t *Loc);
 
 static inline std::string getErrorLocation(const uint8_t *Loc) {
   return getErrorPlace(Loc).Loc;
 }
 
-// In the PowerPC64 Elf V2 abi a function can have 2 entry points.  The first is
-// a global entry point (GEP) which typically is used to intiailzie the TOC
+bool tryRelaxPPC64TocIndirection(RelType Type, const Relocation &Rel,
+                                 uint8_t *BufLoc);
+
+// In the PowerPC64 Elf V2 abi a function can have 2 entry points.  The first
+// is a global entry point (GEP) which typically is used to initialize the TOC
 // pointer in general purpose register 2.  The second is a local entry
 // point (LEP) which bypasses the TOC pointer initialization code. The
 // offset between GEP and LEP is encoded in a function's st_other flags.
 // This function will return the offset (in bytes) from the global entry-point
 // to the local entry-point.
 unsigned getPPC64GlobalEntryToLocalEntryOffset(uint8_t StOther);
 
 // Returns true if a relocation is a small code model relocation that accesses

ELF/Target.cpp

void TargetInfo::writeIgotPlt(uint8_t *Buf, const Symbol &S) const {
   writeGotPlt(Buf, S);
 }
 
 RelExpr TargetInfo::adjustRelaxExpr(RelType Type, const uint8_t *Data,
                                     RelExpr Expr) const {
   return Expr;
 }
 
-void TargetInfo::relaxGot(uint8_t *Loc, uint64_t Val) const {
+void TargetInfo::relaxGot(uint8_t *Loc, RelType Type, uint64_t Val) const {
   llvm_unreachable("Should not have claimed to be relaxable");
 }
 
 void TargetInfo::relaxTlsGdToLe(uint8_t *Loc, RelType Type,
                                 uint64_t Val) const {
   llvm_unreachable("Should not have claimed to be relaxable");
 }
 

test/ELF/Inputs/ppc64-toc-relax-shared.s

+.data
+
+.type shared,@object
+.globl shared
+shared:
+  .long 8
+  .size shared, 4

test/ELF/Inputs/ppc64-toc-relax.s

+.data
+
+.globl default, hidden
+.hidden hidden
+
+default:
+hidden:
+  .long 0
+
+.space 65532
+
+.globl hidden2
+.hidden hidden2
+hidden2:
+  .long 0

test/ELF/ppc64-func-entry-points.s

 # point and we branch to start of foo_external_same.
 
 // CHECK: _start:
 // CHECK: 10010020:       {{.*}}     bl .+144
 // CHECK: 10010034:       {{.*}}     bl .+84
 // CHECK: foo_external_diff:
 // CHECK-NEXT: 10010080:       {{.*}}     addis 2, 12, 1
 // CHECK-NEXT: 10010084:       {{.*}}     addi 2, 2, 32640
-// CHECK-NEXT: 10010088:       {{.*}}     nop
+// CHECK-NEXT: 10010088:       {{.*}}     addis 5, 2, 1
 // CHECK: foo_external_same:
 // CHECK-NEXT: 100100b0:       {{.*}}     add 3, 4, 3

test/ELF/ppc64-got-indirect.s

-# REQUIRES: ppc
-
-# RUN: llvm-mc -filetype=obj -triple=powerpc64le-unknown-linux %s -o %t.o
-# RUN: llvm-readobj -r %t.o | FileCheck -check-prefix=RELOCS-LE %s
-# RUN: ld.lld %t.o -o %t2
-# RUN: llvm-objdump -D %t2 | FileCheck %s --check-prefix=CHECK-LE
-# RUN: llvm-objdump -D %t2 | FileCheck %s
-
-# RUN: llvm-mc -filetype=obj -triple=powerpc64-unknown-linux %s -o %t.o
-# RUN: llvm-readobj -r %t.o | FileCheck -check-prefix=RELOCS-BE %s
-# RUN: ld.lld %t.o -o %t2
-# RUN: llvm-objdump -D %t2 | FileCheck %s --check-prefix=CHECK-BE
-# RUN: llvm-objdump -D %t2 | FileCheck %s
-
-# Make sure we calculate the offset correctly for a got-indirect access to a
-# global variable as described by the PPC64 ELF V2 abi.
-  .text
-  .abiversion 2
-  .globl  _start                    # -- Begin function _start
-  .p2align  4
-  .type  _start,@function
-_start:                                   # @_start
-.Lfunc_begin0:
-.Lfunc_gep0:
-  addis 2, 12, .TOC.-.Lfunc_gep0@ha
-  addi 2, 2, .TOC.-.Lfunc_gep0@l
-.Lfunc_lep0:
-  .localentry  _start, .Lfunc_lep0-.Lfunc_gep0
-# %bb.0:                                # %entry
-  addis 3, 2, .LC0@toc@ha
-  ld 3, .LC0@toc@l(3)
-  li 4, 0
-  stw 4, -12(1)
-  li 0,1
-        lwa 3, 0(3)
-  sc
-  .long  0
-  .quad  0
-.Lfunc_end0:
-  .size  _start, .Lfunc_end0-.Lfunc_begin0
-                                        # -- End function
-  .section  .toc,"aw",@progbits
-.LC0:
-  .tc glob[TC],glob
-  .type  glob,@object            # @glob
-  .data
-  .globl  glob
-  .p2align  2
-glob:
-  .long  55                      # 0x37
-  .size  glob, 4
-
-# Verify the relocations emitted for glob are through the .toc
-
-# RELOCS-LE: Relocations [
-# RELOCS-LE:  .rela.text {
-# RELOCS-LE:     0x0 R_PPC64_REL16_HA .TOC. 0x0
-# RELOCS-LE:     0x4 R_PPC64_REL16_LO .TOC. 0x4
-# RELOCS-LE:     0x8 R_PPC64_TOC16_HA .toc 0x0
-# RELOCS-LE:     0xC R_PPC64_TOC16_LO_DS .toc 0x0
-# RELOCS-LE:   }
-# RELOCS-LE:   .rela.toc {
-# RELOCS-LE:     0x0 R_PPC64_ADDR64 glob 0x0
-# RELOCS-LE:   }
-
-# RELOCS-BE: Relocations [
-# RELOCS-BE:  .rela.text {
-# RELOCS-BE:    0x2 R_PPC64_REL16_HA .TOC. 0x2
-# RELOCS-BE:    0x6 R_PPC64_REL16_LO .TOC. 0x6
-# RELOCS-BE:    0xA R_PPC64_TOC16_HA .toc 0x0
-# RELOCS-BE:    0xE R_PPC64_TOC16_LO_DS .toc 0x0
-# RELOCS-BE:  }
-# RELOCS-BE:  .rela.toc {
-# RELOCS-BE:    0x0 R_PPC64_ADDR64 glob 0x0
-# RELOCS-BE:  }
-# RELOCS-BE:]
-
-# Verify that the global variable access is done through the correct
-# toc entry:
-# r2 = .TOC. = 0x10038000.
-# r3 = r2 - 32760 = 0x10030008 -> .toc entry for glob.
-
-# CHECK: _start:
-# CHECK-NEXT: 10010000:  {{.*}}   addis 2, 12, 2
-# CHECK-NEXT: 10010004:  {{.*}}   addi 2, 2, -32768
-# CHECK-NEXT: 10010008:  {{.*}}   nop
-# CHECK-NEXT: 1001000c:  {{.*}}   ld 3, -32760(2)
-# CHECK: 1001001c:  {{.*}}   lwa 3, 0(3)
-
-# CHECK-LE: Disassembly of section .got:
-# CHECK-LE-EMPTY:
-# CHECK-LE-NEXT: .got:
-# CHECK-LE-NEXT: 10020000:       00 80 02 10
-# CHECK-LE-NEXT: 10020004:       00 00 00 00
-
-# Verify that .toc comes right after .got
-# CHECK-LE: Disassembly of section .toc:
-# CHECK-LE-EMPTY:
-# CHECK-LE: 10020008:       00 00 03 10
-
-# CHECK-LE: Disassembly of section .data:
-# CHECK-LE-EMPTY:
-# CHECK-LE-NEXT: glob:
-# CHECK-LE-NEXT: 10030000:       37 00 00 00
-
-# CHECK-BE: Disassembly of section .got:
-# CHECK-BE-EMPTY:
-# CHECK-BE-NEXT: .got:
-# CHECK-BE-NEXT: 10020000:       00 00 00 00
-# CHECK-BE-NEXT: 10020004:       10 02 80 00
-
-# Verify that .toc comes right after .got
-# CHECK-BE: Disassembly of section .toc:
-# CHECK-BE-EMPTY:
-# CHECK-BE: 10020008:       00 00 00 00
-# CHECK-BE: 1002000c:       10 03 00 00
-
-# CHECK-BE: Disassembly of section .data:
-# CHECK-BE-EMPTY:
-# CHECK-BE-NEXT: glob:
-# CHECK-BE-NEXT: 10030000:       00 00 00 37

test/ELF/ppc64-relocs.s

 # REQUIRES: ppc
 
-# RUN: llvm-mc -filetype=obj -triple=powerpc64le-unknown-linux %s -o %t
-# RUN: ld.lld %t -o %t2
-# RUN: llvm-readelf -x .rodata -x .eh_frame %t2 | FileCheck %s --check-prefix=DATALE
-# RUN: llvm-objdump -d --no-show-raw-insn %t2 | FileCheck %s
-
-# RUN: llvm-mc -filetype=obj -triple=powerpc64-unknown-linux %s -o %t
-# RUN: ld.lld %t -o %t2
-# RUN: llvm-readelf -x .rodata -x .eh_frame %t2 | FileCheck %s --check-prefix=DATABE
-# RUN: llvm-objdump -d --no-show-raw-insn %t2 | FileCheck %s
+# RUN: llvm-mc -filetype=obj -triple=powerpc64le-unknown-linux %s -o %t.o
+# RUN: ld.lld --no-toc-optimize %t.o -o %t
+# RUN: llvm-readelf -x .rodata -x .eh_frame %t | FileCheck %s --check-prefix=DATALE
+# RUN: llvm-objdump -d --no-show-raw-insn %t | FileCheck %s
+
+# RUN: llvm-mc -filetype=obj -triple=powerpc64-unknown-linux %s -o %t.o
+# RUN: ld.lld --no-toc-optimize %t.o -o %t
+# RUN: llvm-readelf -x .rodata -x .eh_frame %t | FileCheck %s --check-prefix=DATABE
+# RUN: llvm-objdump -d --no-show-raw-insn %t | FileCheck %s
 
 .text
 .global _start
 _start:
 .Lfoo:
   li      0,1
   li      3,42
   sc

 .section .R_PPC64_TOC16_HA,"ax",@progbits
 .globl .FR_PPC64_TOC16_HA
 .FR_PPC64_TOC16_HA:
   addis 1, 2, .L1@toc@ha
 
 # CHECK: Disassembly of section .R_PPC64_TOC16_HA:
 # CHECK-EMPTY:
 # CHECK: .FR_PPC64_TOC16_HA:
-# CHECK: 10010018:       nop
+# CHECK: 10010018:       addis 1, 2, 0
 
 .section .R_PPC64_REL24,"ax",@progbits
 .globl .FR_PPC64_REL24
 .FR_PPC64_REL24:
   b .Lfoox
 .section .R_PPC64_REL24_2,"ax",@progbits
 .Lfoox:
 

 # DATABE: 0x100001c8 0000fe80
 
 # Address of rodata + value stored at rodata entry
 # should equal address of LBB0_2.
 # 0x10000190 + 0xfeb4 = 0x10010044
 # CHECK: Disassembly of section .R_PPC64_REL32:
 # CHECK-EMPTY:
 # CHECK: .FR_PPC64_REL32:
-# CHECK: 10010040:       nop
-# CHECK: 10010044:       ld 5, -32736(2)
+# CHECK: 10010040:       addis 5, 2, 0
+# CHECK: 10010044:       ld 5, -32736(5)
 # CHECK: 10010048:       add 3, 3, 4
 
 .section .R_PPC64_REL64, "ax",@progbits
 .globl  .FR_PPC64_REL64
 .FR_PPC64_REL64:
         .cfi_startproc
         .cfi_personality 148, __foo
         li 0, 1

test/ELF/ppc64-toc-relax-constants.s

+# REQUIRES: ppc
+
+# RUN: llvm-mc -filetype=obj -triple=powerpc64le-unkown-linux %p/Inputs/ppc64-toc-relax-shared.s -o %t.o
+# RUN: ld.lld -shared %t.o -o %t.so
+# RUN: llvm-mc -filetype=obj -triple=powerpc64le-unknown-linux %s -o %t1.o
+# RUN: llvm-mc -filetype=obj -triple=powerpc64le-unknown-linux %p/Inputs/ppc64-toc-relax.s -o %t2.o
+# RUN: llvm-readobj -r %t1.o | FileCheck --check-prefix=RELOCS %s
+# RUN: ld.lld %t1.o %t2.o %t.so -o %t
+# RUN: llvm-readelf -S %t | FileCheck --check-prefix=SECTIONS %s
+# RUN: llvm-nm %t | FileCheck --check-prefix=NM %s
+# RUN: llvm-objdump -D %t | FileCheck %s
+
+# In most cases, .toc contains exclusively addresses relocated by R_PPC64_ADDR16.
+# Rarely .toc contain constants or variables.
+# Test we can still perform toc-indirect to toc-relative relaxation.
+
+# RELOCS:      .rela.text {
+# RELOCS-NEXT:   0x0 R_PPC64_TOC16_HA .toc 0x0
+# RELOCS-NEXT:   0x4 R_PPC64_TOC16_LO_DS .toc 0x0
+# RELOCS-NEXT:   0x8 R_PPC64_TOC16_HA .toc 0x8
+# RELOCS-NEXT:   0xC R_PPC64_TOC16_LO_DS .toc 0x8
+# RELOCS-NEXT:   0x10 R_PPC64_TOC16_HA .toc 0x10
+# RELOCS-NEXT:   0x14 R_PPC64_TOC16_LO_DS .toc 0x10
+# RELOCS-NEXT: }
+
+# SECTIONS: .got              PROGBITS        0000000010020090
+# SECTIONS: .toc              PROGBITS        0000000010020090
+
+# NM: 0000000010030000 D default
+
+# .LCONST1 is .toc[0].
+# .LCONST1 - (.got+0x8000) = 0x10020090 - (0x10020090+0x8000) = -32768
+# CHECK: nop
+# CHECK: lwa 3, -32768(2)
+  addis 3, 2, .LCONST1@toc@ha
+  lwa 3, .LCONST1@toc@l(3)
+
+# .LCONST2 is .toc[1]
+# .LCONST2 - (.got+0x8000) = 0x10020098 - (0x10020090+0x8000) = -32760
+# CHECK: nop
+# CHECK: ld 4, -32760(2)
+  addis 4, 2, .LCONST2@toc@ha
+  ld 4, .LCONST2@toc@l(4)
+
+# .Ldefault is .toc[2]. `default` is not preemptable when producing an executable.
+# After toc-indirection to toc-relative relaxation, it is loaded using an
+# offset relative to r2.
+# CHECK: nop
+# CHECK: addi 5, 2, 32624
+# CHECK: lwa 5, 0(5)
+  addis 5, 2, .Ldefault@toc@ha
+  ld    5, .Ldefault@toc@l(5)
+  lwa   5, 0(5)
+
+.section .toc,"aw",@progbits
+.LCONST1:
+  .quad 11
+.LCONST2:
+  .quad 22
+.Ldefault:
+  .tc default[TC],default

test/ELF/ppc64-toc-relax-jumptable.s

+# REQUIRES: ppc
+# RUN: llvm-mc -filetype=obj -triple=powerpc64le-unknown-linux %s -o %t.o
+# RUN: ld.lld %t.o -o %t
+# RUN: llvm-readelf -S %t | FileCheck --check-prefixes=SECTIONS %s
+# RUN: llvm-readelf -x .toc %t | FileCheck --check-prefixes=HEX-LE %s
+# RUN: llvm-objdump -d %t | FileCheck --check-prefixes=CHECK %s
+
+# RUN: llvm-mc -filetype=obj -triple=powerpc64-unknown-linux %s -o %t.o
+# RUN: ld.lld %t.o -o %t
+# RUN: llvm-readelf -S %t | FileCheck --check-prefixes=SECTIONS %s
+# RUN: llvm-readelf -x .toc %t | FileCheck --check-prefixes=HEX-BE %s
+# RUN: llvm-objdump -d %t | FileCheck --check-prefixes=CHECK %s
+
+# .LJT is a local symbol (non-preemptable).
+# Test we can perform the toc-indirect to toc-relative relaxation.
+
+# SECTIONS: .rodata PROGBITS 00000000100001c8
+
+# HEX-LE:      section '.toc':
+# HEX-LE-NEXT: 10020008 c8010010 00000000
+
+# HEX-BE:      section '.toc':
+# HEX-BE-NEXT: 10020008 00000000 100001c8
+
+# CHECK-LABEL: _start
+# CHECK:       clrldi  3, 3, 62
+# CHECK-NEXT:  addis 4, 2, -2
+# CHECK-NEXT:  addi  4, 4, -32312
+# CHECK-NEXT:  sldi  3, 3, 2
+
+    .text
+    .global _start
+    .type _start, @function
+_start:
+.Lstart_gep:
+    addis 2, 12, .TOC.-.Lstart_gep@ha
+    addi  2,  2, .TOC.-.Lstart_gep@l
+.Lstart_lep:
+    .localentry _start, .Lstart_lep-.Lstart_gep
+    rldicl 3, 3, 0, 62
+    addis 4, 2, .LJTI_TE@toc@ha
+    ld    4, .LJTI_TE@toc@l(4)
+    sldi  3, 3, 2
+    lwax  3, 3, 4
+    add   3, 3, 4
+    mtctr 3
+    bctr
+
+.LBB1:
+    li 3, 0
+    blr
+.LBB2:
+    li 3, 10
+    blr
+.LBB3:
+    li 3, 55
+    blr
+.LBB4:
+    li 3, 255
+    blr
+
+    .section        .rodata,"a",@progbits
+    .p2align        2
+.LJT:
+    .long   .LBB1-.LJT
+    .long   .LBB2-.LJT
+    .long   .LBB3-.LJT
+    .long   .LBB4-.LJT
+
+.section        .toc,"aw",@progbits
+# TOC entry for the jumptable address.
+.LJTI_TE:
+    .tc .LJT[TC],.LJT

test/ELF/ppc64-toc-relax.s

+# REQUIRES: ppc
+# RUN: llvm-mc -filetype=obj -triple=powerpc64le-unknown-linux %p/Inputs/ppc64-toc-relax-shared.s -o %t.o
+# RUN: ld.lld -shared %t.o -o %t.so
+# RUN: llvm-mc -filetype=obj -triple=powerpc64le-unknown-linux %s -o %t1.o
+# RUN: llvm-mc -filetype=obj -triple=powerpc64le-unknown-linux %p/Inputs/ppc64-toc-relax.s -o %t2.o
+# RUN: llvm-readobj -r %t1.o | FileCheck --check-prefixes=RELOCS-LE,RELOCS %s
+# RUN: ld.lld %t1.o %t2.o %t.so -o %t
+# RUN: llvm-objdump -d --no-show-raw-insn %t | FileCheck --check-prefixes=COMMON,EXE %s
+
+# RUN: ld.lld -shared %t1.o %t2.o %t.so -o %t2.so
+# RUN: llvm-objdump -d --no-show-raw-insn %t2.so | FileCheck --check-prefixes=COMMON,SHARED %s
+
+# RUN: llvm-mc -filetype=obj -triple=powerpc64-unknown-linux %p/Inputs/ppc64-toc-relax-shared.s -o %t.o
+# RUN: ld.lld -shared %t.o -o %t.so
+# RUN: llvm-mc -filetype=obj -triple=powerpc64-unknown-linux %s -o %t1.o
+# RUN: llvm-mc -filetype=obj -triple=powerpc64-unknown-linux %p/Inputs/ppc64-toc-relax.s -o %t2.o
+# RUN: llvm-readobj -r %t1.o | FileCheck --check-prefixes=RELOCS-BE,RELOCS %s
+# RUN: ld.lld %t1.o %t2.o %t.so -o %t
+# RUN: llvm-objdump -d --no-show-raw-insn %t | FileCheck --check-prefixes=COMMON,EXE %s
+
+# RUN: ld.lld -shared %t1.o %t2.o %t.so -o %t2.so
+# RUN: llvm-objdump -d --no-show-raw-insn %t2.so | FileCheck --check-prefixes=COMMON,SHARED %s
+
+# RELOCS-LE:      .rela.text {
+# RELOCS-LE-NEXT:   0x0 R_PPC64_TOC16_HA .toc 0x0
+# RELOCS-LE-NEXT:   0x4 R_PPC64_TOC16_LO_DS .toc 0x0
+# RELOCS-LE-NEXT:   0xC R_PPC64_TOC16_HA .toc 0x8
+# RELOCS-LE-NEXT:   0x10 R_PPC64_TOC16_LO_DS .toc 0x8
+# RELOCS-LE-NEXT:   0x18 R_PPC64_TOC16_HA .toc 0x10
+# RELOCS-LE-NEXT:   0x1C R_PPC64_TOC16_LO_DS .toc 0x10
+# RELOCS-LE-NEXT:   0x24 R_PPC64_TOC16_HA .toc 0x18
+# RELOCS-LE-NEXT:   0x28 R_PPC64_TOC16_LO_DS .toc 0x18
+# RELOCS-LE-NEXT: }
+
+# RELOCS-BE:      .rela.text {
+# RELOCS-BE-NEXT:   0x2 R_PPC64_TOC16_HA .toc 0x0
+# RELOCS-BE-NEXT:   0x6 R_PPC64_TOC16_LO_DS .toc 0x0
+# RELOCS-BE-NEXT:   0xE R_PPC64_TOC16_HA .toc 0x8
+# RELOCS-BE-NEXT:   0x12 R_PPC64_TOC16_LO_DS .toc 0x8
+# RELOCS-BE-NEXT:   0x1A R_PPC64_TOC16_HA .toc 0x10
+# RELOCS-BE-NEXT:   0x1E R_PPC64_TOC16_LO_DS .toc 0x10
+# RELOCS-BE-NEXT:   0x26 R_PPC64_TOC16_HA .toc 0x18
+# RELOCS-BE-NEXT:   0x2A R_PPC64_TOC16_LO_DS .toc 0x18
+# RELOCS-BE-NEXT: }
+
+# RELOCS:         .rela.toc {
+# RELOCS-NEXT:      0x0 R_PPC64_ADDR64 hidden 0x0
+# RELOCS-NEXT:      0x8 R_PPC64_ADDR64 hidden2 0x0
+# RELOCS-NEXT:      0x10 R_PPC64_ADDR64 shared 0x0
+# RELOCS-NEXT:      0x18 R_PPC64_ADDR64 default 0x0
+# RELOCS-NEXT:    }
+
+# NM-DAG: 0000000010030000 D default
+# NM-DAG: 0000000010030000 d hidden
+# NM-DAG: 0000000010040000 d hidden2
+
+# 'hidden' is non-preemptable. It is relaxed.
+# address(hidden) - (.got+0x8000) = 0x10030000 - (0x100200c0+0x8000) = 32576
+# COMMON: nop
+# COMMON: addi 3, 2, 32576
+# COMMON: lwa 3, 0(3)
+  addis 3, 2, .Lhidden@toc@ha
+  ld    3, .Lhidden@toc@l(3)
+  lwa   3, 0(3)
+
+# address(hidden2) - (.got+0x8000) = 0x10040000 - (0x100200c0+0x8000) = (1<<16)+32576
+# COMMON: addis 3, 2, 1
+# COMMON: addi 3, 3, 32576
+# COMMON: lwa 3, 0(3)
+  addis 3, 2, .Lhidden2@toc@ha
+  ld    3, .Lhidden2@toc@l(3)
+  lwa   3, 0(3)
+
+# 'shared' is not defined in an object file. Its definition is determined at
+# runtime by the dynamic linker, so the extra indirection cannot be relaxed.
+# The first addis can still be relaxed to nop, though.
+# COMMON: nop
+# COMMON: ld 4, -32752(2)
+# COMMON: lwa 4, 0(4)
+  addis 4, 2, .Lshared@toc@ha
+  ld    4, .Lshared@toc@l(4)
+  lwa   4, 0(4)
+
+# 'default' has default visibility. It is non-preemptable when producing an executable.
+# address(default) - (.got+0x8000) = 0x10030000 - (0x100200c0+0x8000) = 32576
+# EXE: nop
+# EXE: addi 5, 2, 32576
+# EXE: lwa 5, 0(5)
+
+# SHARED: nop
+# SHARED: ld 5, -32744(2)
+# SHARED: lwa 5, 0(5)
+  addis 5, 2, .Ldefault@toc@ha
+  ld    5, .Ldefault@toc@l(5)
+  lwa   5, 0(5)
+
+.section .toc,"aw",@progbits
+.Lhidden:
+  .tc hidden[TC], hidden
+.Lhidden2:
+  .tc hidden2[TC], hidden2
+.Lshared:
+  .tc shared[TC], shared
+.Ldefault:
+  .tc default[TC], default