[PPC64] toc-indirect to toc-relative relaxation.

[sfertile]

Summary

When accessing a global variable which is not defined in the translation-unit being compiled the compilers on ppc64 will generate a toc-entry for the global and a got-indirect access using that toc-entry (as if it were a .got entry)

For example the following C code

extern int aGlobal;

int foo(void) {
  return aGlobal;
}

would get translated to:

        addis 3, 2, .LC0@toc@ha 
        ld 3, .LC0@toc@l(3)
        lwa 3, 0(3)

...

        .section        .toc,"aw",@progbits
.LC0:
        .tc aGlobal[TC],aGlobal

Where .LC0@toc is the offset from the TOC base-pointer to the label .LC0, and .tc aGlobal[TC],aGlobal creates an entry in the .toc section to store the address of aGlobal.
The first 2 instructions build the address of the toc-entry and load the address of aGlobal into r3, while the following lwa instruction loads the value of the global into r3.

If the global being accessed is a non-preemptable definition and the offset from the TOC pointer to the definition can be materialized with 2 instructions then we can relax the first 2 instructions to add that offset to the TOC base pointer rather then loading the address out of the .toc.

[sfertile]

Zaara pointed out I am missing the full context, I'll re post the patch with the context fixed shortly.

[sfertile]

Added full context

[syzaara]

ELF/Arch/PPC64.cpp

369

Instead of 58, we can use the enum DFormOpcd which has LD defined to 58.

371

Expeceted -> Expected

ELF/InputSection.cpp

879

Maybe use Target->GotEntrySize instead of 8.

886

entires -> entries

894

relxation -> relaxation

[MaskRay]

ELF/Arch/PPC64.cpp

372

Instr &= 0x03FFFFFF;

ELF/InputSection.cpp

889

Offset / 8 /* Toc Entry size */ -> Index

902

Maybe just return None;

[MaskRay]

ELF/InputSection.cpp

896

They are put here probably because of ARM/AArch64 precedents: getAArch64UndefinedRelativeWeakVA getARMStaticBase etc.

If TOC is PPC/PPC64 specific, the static function tryRelaxToc and many of its callees may be renamed to include the substring PPC.

ELF/Relocations.h

77

If the TOC (Table of Contents) section is PPC/PPC64 specific this should probably be named R_PPC_RELAX_TOC. R_AARCH64_RELAX_TLS_GD_TO_IE_PAGE_PC is a precedent.

[sfertile]

Addressed initial review comments.

[sfertile]

ELF/InputSection.cpp

902

Thanks, I wasn't aware of 'None'.

[ruiu]

ELF/Arch/PPC64.cpp

371

Error messages should start with a lowercase letter.

I think you should use error() instead of llvm_unreachable() because this condition can be triggered by a user. llvm_unreachable should be used only to report an internal error or a bug.

372–374

I think Instr = (Instr & 0x3FFFFFF) | 0x38000000; is a bit more straightforward.

Or,

writeInstrFromHalf16(Loc, (Instr & 0x3FFFFFF) | 0x38000000);

is perhaps better.

376

This is an unusual formatting style in lld. Please put break before }.

379

U -> u

ELF/InputSection.cpp

874

ElfTy -> ELFT

875

Can you return std::pair<Defined *, int64_t> instead of Optional<std::pair<Defined *, int64_t>>? It looks like you can simply return {nullptr, 0} to indicate a null value.

877

Replace auto with the actual type.

889

Ditto

930

Please do not use auto in lld unless its actual type is obvious from right-hand side.

[grimar]

ELF/InputSection.cpp

907

Seems the intention was to name it 'relaxTocPPC64'?

918

dyn_cast -> cast as null return value is not expected here.

937

Defined* -> Defined

[sfertile]

Addressed further review comments.

[sfertile]

ELF/InputSection.cpp

907

I named it relaxToPPC64 because the function is looking up the symbol we will potentially relax to. I failed to realize that its unfortunately close to relaxToc which would also make sense as a name :)

[grimar]

I did not have a chance to debug it, but my comment is below.

ELF/InputSection.cpp

897

I think we usually reach here (relocateAlloc) when we already know will we do the relaxation or not.
I.e. for example on x86_64 for R_RELAX_GOT_PC we just call Target->relaxGot because check in the adjustRelaxExpr the conditions for relaxing. And here then we either have R_RELAX_GOT_PC which name contains RELAX and means we *will do* the relaxation or we would have another R_* expression if we are not going to relax.

So I think you should not have a R_PPC64_RELAX_TOC expression here if you are not going to relax. I think you should check the conditions somewhere earlier. Also, the platform-specific code can live into own file (like ELF\Arch\PPC64.cpp) then. I think you can put all the functions you added there.

[sfertile]

Related to @grimar comment, I am also playing around with doing some of the checking early and mapping toc based relocations that are not-relaxable to got based relocations during scanRelocs. Ideally I'd like to be able to add a separate step for PPC64 once all relocations have been scanned that will iterate over the .toc sections to handle large-code model where any symbols that can't be safely relaxed get added to the .got and all remaining toc based relocations get relaxed. This should hopefully allow me to get rid of the .toc sections in the final binaries, but I'm not familiar enough with large-code model, or gcc and xlc codegen to know if this is feasible yet.

ELF/InputSection.cpp

897

The got-indirect to toc-relative relaxation is unique in this regard because we can't decide if we can relax until the data segment has been fully built. During scanRelocs we could see if the indirection is to a defined symbol, but that is only enough to know that the relocation is potentially relax-able. We still need the offset from the TOC base pointer to the symbols definition to fit in a signed 32-bit offset (which isn't guaranteed in the large code model), and scanRelocs is too early to determine that. I could add the relax checking earlier than here but that would involve doing the same scan over the .toc sections relocations and replacing those that are indeed relaxable. I decided not to do that in the spirit of keeping it simple.

I like the idea of moving the platform specific code into PPC64.cpp, and will update the patch with that change, but I think here is the right place to perform the relaxation.

[sfertile]

Moved the toc relaxation logic into the PPC64 target.

[grimar]

Few comments/suggestions.

ELF/Arch/PPC64.cpp

109

Warnings and error messages should start from the lowercase.

119

The same.

126

You can write shorter:

return {dyn_cast_or_null<Defined>(&IndirectSym), getAddend<ELFT>(Rela)};

136

Why not DefSym->Section->Name != ".toc" ?

137

return {};

140

return {};

154

So maybe you should never reach here if !Config->TocOptimize?
Seems you can do something like the following in getRelExpr:

case R_PPC64_TOC16_HA:
case R_PPC64_TOC16_LO_DS:
  return Config->TocOptimize ? R_PPC64_RELAX_TOC : R_XXXX;

156

RelaxTo.first/RelaxTo.second are a bit hard to follow and read. Lets do:

Defined *D;
int64_t Addend;
std::tie(D, Addend) = relaxTo(Rel);

ELF/InputSection.cpp

897

OK, thanks for the explanation.

ELF/Target.h

172

I think Returns false otherwise part is an obvious continuation and hence excessive.

test/ELF/ppc64-got-indirect.s

4–5

You can use

--check-prefixes=CHECK,CHECK-LE

test/ELF/ppc64-toc-relax-jumptable.s

37

Can you remove the excessive indentations please?

test/ELF/ppc64-toc-relax.s

63

# CHECK-LABEL: ...

[sfertile]

Addressed review comments

[sfertile]

Forgot to update formatting in one of the tests.

[ruiu]

ELF/Arch/PPC64.cpp

109

Isn't this an error?

119

Ditto

[sfertile]

ELF/Arch/PPC64.cpp

109

Ideally it would be, however I found a gcc bug where it emits a toc entry without a corresponding relocation. This causes the look ups for any relocations after the skipped one to fail. I am still trying to cut the problem down to something smaller so I can understand it better (and open a bug report to get it fixed), but it seems that the rest of the codegen is valid (ie nothing tries to read or write to the toc entry missing the relocation). I don't think the problem is prevalent enough to warrant changing to a slower scan of the relocations in the case we do find a mismatch, and because the rest of the codegen is 'good' I don't think we should emit an error either.

[sfertile]

ELF/Arch/PPC64.cpp

109

I've got some clarification from the ppc64 gcc/binutils developers and although the case I found was a bug , both gcc and xlc will emit values that don't need relocations into the .toc section in certian circumstances. I'll have to fix the lookup logic to deal with the fact I can't assume every .toc entry has a relocation.

[sfertile]

Fixed look up to handle .toc sections that don't have a relocation for every entry and added a comprehensive test to verify we perform the correct optimization on all access types in this case.

[sfertile]

rebased

[sfertile]

ping

[MaskRay]

ELF/Arch/PPC64.cpp

174

The title of this revision says "toc-indirect" while here and some other places say "got-indirect". Which term is preferred? (Or maybe they are two different things and I am confused..)

[MaskRay]

ELF/Arch/PPC64.cpp

115

if (Relas.empty())

[ruiu]

ELF/Arch/PPC64.cpp

110

It feels this comment can be improved. IIUC, this function scans relocations for a given section to find one whose offset is the same as a given offset. Is this correct? If so, can you add something like that as a comment?

114

I'd define an alias at the beginning of this function like this:

typedef typename ELFT::Rela RelTy;

120

When we define a local helper lambda function inside a function, we capture variables just by [&] instead of explicitly enumerating variables.

121

nit: use shorter variable name for a variable with just two lines of scope.

171

Can you add a high-level comment as to what optimization this function is supposed to do?

[sfertile]

Addressed several review comments, and fixed bug where I was erroneously relaxing preepmtible symbols.

[sfertile]

ELF/Arch/PPC64.cpp

120

I've updated to follow suit, but can I ask the rational behind this? For a lambda as small as this one its easy to see that only one variable is used and its not modified, but in general I would prefer to be explicit and not capture anymore then I used.

174

got-indirect is the term used in the ABI and so I have tried to stick mostly to that, especially for anything a user would see like a diagnostic. toc-indirect is a term I made up to describe the way gcc/clang/xlc actually produce a got-indirecton as its a bit different then what I think most would expect. The 2 ways have slightly different input, but the output from the linker is the same.

got-indirect -> its up to the linker to allocate a got entry for sym. The got entry is accessed relative to the toc-pointer.

 addis r3, r2, sym@got@ha   
 ld        r3, sym@got@l(r3) 
lwa       r3, r3

vs toc-indirect --> the compiler allocates an entry in the .toc section and puts a relocation for the symbols address into that .toc entry. The toc entry is accessed relative to the toc-pointer.

    addis r3, r2, .LC0@toc@ha   
    ld        r3, .LC0@toc@l(r3) 
   lwa       r3, r3
...

   .toc
.LC0:
   .tc sym[TC], sym # emits a relocation for sym. Roughly equivalent to .quad sym?

All this extra work in looking up the relocation to find the symbol thats being accessed is because the compiler emits the second form rather then the first. In the final binary the .toc sections are either merged into the .got section (bfd/gold) or allocated right after the .got section (lld) and you end up with a got-indirect access.

[ruiu]

Generally looking good, but I have one question.

ELF/Arch/PPC64.cpp

137

I think I don't understand why you can't relax if the number of relocations for the .toc is smaller than the number of entires in .toc. I thought that if you can find a relocation that has a specific offset, that's the relocation that you are looking for, and you can read a symbol and an offset from that relocation. I think there's something more about PPC64 ABI that I don't know of. Could you explain?

[sfertile]

ELF/Arch/PPC64.cpp

137

I think the missing piece here is that I am assuming each toc entry has at most 1 relocation. The relocations are R_PPC64_ADDR64 and relocates one full entry each, or we have a constant/value with no relocation. That way a look up should always find an r_offset greater then or equal to Offset on the first look-up, except for when we truncated the index because the one we calculated was outside the array bounds. If we truncated the index and find a r_offset < Offset we can return because the array is sorted by offset.

[MaskRay]

The patch needs a rebase.

ELF/Arch/PPC64.cpp

123

Sym as a reference can't be null, so you can use dyn_cast instead of dyn_cast_or_null.

126

The relocations are sorted by offset

IIRC this is not guaranteed but I think it is fine to lose toc optimization if the input does not have this property.

The linear search below for (auto R : llvm::reverse(Relas.slice(0, Index))) { concerns me.

Is it possible to add a LastRelaIndex variable in InputSectionBase::relocateAlloc:

if (!tryRelaxTocPPC64(Type, Rel, Expr, BufLoc, LastRelaIndex))

and keep it monotonically increasing in getSymAndAddend? If the input doesn't guarantee increasing r_offset, it may lose TOC optimizations for some relocations, but as I said it should be fine.

[sfertile]

ELF/Arch/PPC64.cpp

126

The relocations are sorted by offset

IIRC this is not guaranteed

I believe its guaranteed for binutils tools. We also sort the relocations for the .toc section in scanRelocs in case there are other tools which don't produce the relocations in sorted order.

The linear search below for (auto R : llvm::reverse(Relas.slice(0, Index))) { concerns me

I believe missing toc relocations to be an uncommon thing. As far as I am aware llvm will not place constants into the .toc section. gcc will for a few very specific cases but it is still not very common. xlc is the outlier here as I heard it will do this some what commonly despite there being no need to with the V2 abi.

For the objects I found that were missing relocations when compiled with gcc , we either found the relocation we were looking for, or broke out of the search in a few comparisons. If a linear search does become a problem for some pathological inputs we could switch to a binary search but I think doing that initially we would end up pessimaling the common case for something that might not ever naturally occur.

[MaskRay]

ELF/Arch/PPC64.cpp

126

I was not suggesting a binary search.

I suggested to change:

for (const Relocation &Rel : Relocations) { // the loop is in relocateAlloc
  // Offset is increasing
  for (Index = computed(Rel); J >= 0; J--) // the loop is in PPC64.cpp getSymAndAddend
    if (Offset == Relas[Index].r_offset) {
      /* found */
      break;
    }
}

to

Index = 0
for (const Relocation &Rel : Relocations) { // the loop is in relocateAlloc
  // Offset is increasing
  for (; Index < Relas.size(); ++Index)
    // Index is increasing
    if (Offset == Relas[Index].r_offset) {
      /* found */
      break;
    }
}

[MaskRay]

ELF/Arch/PPC64.cpp

126

Sorry, see below:

for (const Relocation &Rel : Relocations) { // the loop is in relocateAlloc
  // Offset is increasing
  for (Index = computed(Rel); Index >= 0; Index--) // the loop is in PPC64.cpp getSymAndAddend
    if (Offset == Relas[Index].r_offset) {
      /* found */
      break;
    }
}

to

Index = 0
for (const Relocation &Rel : Relocations) { // the loop is in relocateAlloc
  // Offset is increasing
  for (; Index < Relas.size(); ++Index)
    // Index is increasing
    if (Offset == Relas[Index].r_offset) {
      /* found */
      break;
    } else if (Offset < Relas[Index].r_offset)
      break;
}

[sfertile]

ELF/Arch/PPC64.cpp

126

I think I may have misunderstood which relocations you were referring too with the first comment about sort order. Only the relocations that operate on the .toc section are sorted. (Otherwise I'm misunderstanding this proposed change, perhaps we could talk on IRC to clairfy?)

Here is my understanding of the proposed look-up:

Index = 0
for (const Relocation &Rel : Relocations) { // the loop is in relocateAlloc
  // Offset is not (guranteed) increasing, Addend is not (typically) increasing.
  Rel.Sym    // This must be .toc if its an indirect.
  Rel.Addend // The addends will not be in any particular order. 
                         // When looking up the symbol we indirect too, we want to find the
                        //  relocation that relocates the toc section at offset `Rel.Addend`.
  Rel.Offset // The offset in the section being relocated. Unrelated to the lookup performed in the .toc section.

  // Inside the PPC64 target.
  // Loop over the relocations for the .toc section. These **are** sorted based on offset.

  for (; Index < Relas.size(); ++Index)
    // Index is increasing
    if (Offset == Relas[Index].r_offset) {
      /* found */
      break;
    } else if (Offset < Relas[Index].r_offset)
      break;
}

1. Any indirect that refers to an offset smaller then the first missing relocation would still be optimized as its array mapping is still correct.
2. Any offset strictly between the first missing relocation and the largest toc offset seen so far will not be optimized as Relas[Index].r_offset > Offset
3. Any toc offset greater then or equal too the largest toc offset seen so far will be optimized (as long as it has a relocation). If the new offset is greater then the previous maximum offset, it increases the size of the window described in #2.

[MaskRay]

@sfertile Didn't catch you these days on IRC...

I've created D60958 for my idea.

[sfertile]

In D54720#1473968, @MaskRay wrote:

@sfertile Didn't catch you these days on IRC...

I've created D60958 for my idea.

Sorry about that. I had a workshop the week following the last time we talked and I've been focused on getting parts of our AIX work moving since then. I'm going over D60958 right now. Thanks for doing this.

[sfertile]

Abandoning in favour of D60958.

ELF/Arch/PPC64.cpp

 bool elf::isPPC64SmallCodeModelReloc(RelType Type) {
   // List is not yet complete, at the very least the got based tls related
   // relocations need to be added, and we need to determine how the section
   // sorting interacts with the thread pointer and dynamic thread pointer
   // relative tls relocations.
   return Type == R_PPC64_GOT16 || Type == R_PPC64_TOC16 ||
          Type == R_PPC64_TOC16_DS;
 }
 
+// Searches through the relocations in TocSec looking for the one with
+// matching Offset.
+template <typename ELFT>
+static std::pair<Defined *, int64_t> getSymAndAddend(InputSectionBase *TocSec,
+                                                     uint64_t Offset) {
+  typedef typename ELFT::Rela RelaTy;
+  ArrayRef<RelaTy> Relas = TocSec->template relas<ELFT>();
+  if (Relas.empty())
+    return {};
+
+  // Extracts the Symbol and addend from a relocation.
+  auto SymAndAddend = [&](RelaTy Rela) -> std::pair<Defined *, int64_t> {
+    Symbol &Sym = TocSec->getFile<ELFT>()->getRelocTargetSym(Rela);
+    return {dyn_cast_or_null<Defined>(&Sym), getAddend<ELFT>(Rela)};
+  };
+
+  // The relocations are sorted by offset, and while the most common case is
+  // that every .toc entry will have a relocation, it is not guaranteed. For
+  // example objects compiled with small code model are likely to have
+  // constants placed in the .toc section. Start looking by mapping the offset
+  // directly into its corresponding toc entry.
+  size_t Index = std::min(Offset / Target->GotEntrySize, Relas.size() - 1);
+  if (Offset == Relas[Index].r_offset)
+    return SymAndAddend(Relas[Index]);
+
+  // If Offset is greater than r_offset on the relocation we looked up we must
+  // have less relocations then toc enties and Relas[Index] is the last reloc
+  // in the array. In this case we have nothing to relax to.
+  if (Offset > Relas[Index].r_offset)
+    return {};
+
+  // Walk back looking for a relocation that relocates .toc + Offset.
+  for (auto R : llvm::reverse(Relas.slice(0, Index))) {
+    if (R.r_offset == Offset)
+      return SymAndAddend(R);
+    if (R.r_offset < Offset)
+      return {};
+  }
+
+  // No relocation for the toc entry .toc + Offset.
+  return {};
+}
+
+// Determines the target of a toc-indirection to relax to. Returns the symbol +
+// addend pair if there is a symbol to relax to.
+static std::pair<Defined *, int64_t> relaxTo(Relocation Rel) {
+  // If the symbol the relocation refers to is not the .toc section
+  // this can't be a toc-indirection.
+  Defined *DefSym = dyn_cast<Defined>(Rel.Sym);
+  if (!DefSym || !DefSym->isSection() || DefSym->Section->Name != ".toc")
+    return {};
+
+  if (Rel.Addend < 0)
+    return {};
+
+  auto *TocISB = cast<InputSectionBase>(DefSym->Section);
+  // Get the symbol + addend that the toc-indirection refers to.
+  return Config->IsLE ? getSymAndAddend<ELF64LE>(TocISB, Rel.Addend)
+                      : getSymAndAddend<ELF64BE>(TocISB, Rel.Addend);
+}
+
+// Attempts to relax loading a symbols address from the .toc section into
+// calculating the symbols address using an offset relative to the toc pointer.
+// To be able to relax the target symbol must be defined, not preemptible and
+// addressable with a 32-bit signed offset from the toc pointer.
+bool elf::tryRelaxTocPPC64(RelType Type, const Relocation &Rel, RelExpr Expr,
+                           uint8_t *BufLoc) {
+  assert(Expr == R_PPC64_RELAX_TOC &&
+         "unexpected RelExpr for got-indirect to toc-relative relaxation");
+  assert(Config->TocOptimize &&
+      "tryRelaxTocPPC64 should only be called when --toc-optimize is enabled");
+
+  Defined *D;
+  int64_t Addend;
+  std::tie(D, Addend) = relaxTo(Rel);
+
+  // If there is no defined symbol or if the defined symbol is preemptible then
+  // the relocation cannot be relaxed.
+  if (!D || D->IsPreemptible)
+    return false;
+
+  uint64_t SymVA = D->getVA(Addend);
+  // Because we only have the 2 instructions of the got-indirect access sequence
+  // to rewrite, we can only relax if the symbol definition is within 32 bits of
+  // the TOC base-pointer. For medium code model this is guaranteed but in the
+  // large code model parts of the data segment will be too far away to relax.
+  if (!isInt<32>(SymVA - getPPC64TocBase()))
+    return false;
+
+  // Safe to relax, let the target callback perform the actual relaxation.
+  // Note that we use the offset between the symbols VA and the .got VA as the
+  // relocation value since relocateOne will perform the adjustment by the TOC
+  // bias.
+  Target->relaxGot(BufLoc, Type, SymVA - In.Got->getVA());
+  return true;
+}
+
 namespace {
 class PPC64 final : public TargetInfo {
 public:
   PPC64();
   uint32_t calcEFlags() const override;
   RelExpr getRelExpr(RelType Type, const Symbol &S,
                      const uint8_t *Loc) const override;
   void writePltHeader(uint8_t *Buf) const override;
   void writePlt(uint8_t *Buf, uint64_t GotPltEntryAddr, uint64_t PltEntryAddr,
                 int32_t Index, unsigned RelOff) const override;
   void relocateOne(uint8_t *Loc, RelType Type, uint64_t Val) const override;
   void writeGotHeader(uint8_t *Buf) const override;
   bool needsThunk(RelExpr Expr, RelType Type, const InputFile *File,
                   uint64_t BranchAddr, const Symbol &S) const override;
   bool inBranchRange(RelType Type, uint64_t Src, uint64_t Dst) const override;
   RelExpr adjustRelaxExpr(RelType Type, const uint8_t *Data,
                           RelExpr Expr) const override;
+  void relaxGot(uint8_t *Loc, RelType Type, uint64_t Val) const override;
   void relaxTlsGdToIe(uint8_t *Loc, RelType Type, uint64_t Val) const override;
   void relaxTlsGdToLe(uint8_t *Loc, RelType Type, uint64_t Val) const override;
   void relaxTlsLdToLe(uint8_t *Loc, RelType Type, uint64_t Val) const override;
   void relaxTlsIeToLe(uint8_t *Loc, RelType Type, uint64_t Val) const override;
 
   bool adjustPrologueForCrossSplitStack(uint8_t *Loc, uint8_t *End,
                                         uint8_t StOther) const override;
 };
for (InputFile *F : ObjectFiles) {
     if (Flag == 1)
       error(toString(F) + ": ABI version 1 is not supported");
     else if (Flag > 2)
       error(toString(F) + ": unrecognized e_flags: " + Twine(Flag));
   }
   return 2;
 }
 
+void PPC64::relaxGot(uint8_t *Loc, RelType Type, uint64_t Val) const {
+  switch (Type) {
+  case R_PPC64_TOC16_HA:
+    relocateOne(Loc, Type, Val);
+    break;
+  case R_PPC64_TOC16_LO_DS: {
+    // rewrite the instuction from a load to an addi.
+    uint32_t Instr = readInstrFromHalf16(Loc);
+    if (getPrimaryOpCode(Instr) != LD)
+      error("expected a 'ld' for got-indirect to toc-relative relaxing");
+    writeInstrFromHalf16(Loc, (Instr & 0x03FFFFFF) | 0x38000000);
+    relocateOne(Loc, R_PPC64_TOC16_LO, Val);
+    break;
+  }
+  default:
+    error(
+        "unexpected relocation type for got-indirect to toc-relative relaxing");
+  }
+}
+
 void PPC64::relaxTlsGdToLe(uint8_t *Loc, RelType Type, uint64_t Val) const {
   // Reference: 3.7.4.2 of the 64-bit ELF V2 abi supplement.
   // The general dynamic code sequence for a global `x` will look like:
   // Instruction                    Relocation                Symbol
   // addis r3, r2, x@got@tlsgd@ha   R_PPC64_GOT_TLSGD16_HA      x
   // addi  r3, r3, x@got@tlsgd@l    R_PPC64_GOT_TLSGD16_LO      x
   // bl __tls_get_addr(x@tlsgd)     R_PPC64_TLSGD               x
   //                                R_PPC64_REL24               __tls_get_addr
RelExpr PPC64::getRelExpr(RelType Type, const Symbol &S,
   case R_PPC64_GOT16_DS:
   case R_PPC64_GOT16_HA:
   case R_PPC64_GOT16_HI:
   case R_PPC64_GOT16_LO:
   case R_PPC64_GOT16_LO_DS:
     return R_GOT_OFF;
   case R_PPC64_TOC16:
   case R_PPC64_TOC16_DS:
-  case R_PPC64_TOC16_HA:
   case R_PPC64_TOC16_HI:
   case R_PPC64_TOC16_LO:
-  case R_PPC64_TOC16_LO_DS:
     return R_GOTREL;
+  case R_PPC64_TOC16_HA:
+  case R_PPC64_TOC16_LO_DS:
+    return Config->TocOptimize ? R_PPC64_RELAX_TOC : R_GOTREL;
   case R_PPC64_TOC:
     return R_PPC_TOC;
   case R_PPC64_REL14:
   case R_PPC64_REL24:
     return R_PPC_CALL_PLT;
   case R_PPC64_REL16_LO:
   case R_PPC64_REL16_HA:
   case R_PPC64_REL32:

ELF/Arch/X86_64.cpp

public:
   void writeGotPlt(uint8_t *Buf, const Symbol &S) const override;
   void writePltHeader(uint8_t *Buf) const override;
   void writePlt(uint8_t *Buf, uint64_t GotPltEntryAddr, uint64_t PltEntryAddr,
                 int32_t Index, unsigned RelOff) const override;
   void relocateOne(uint8_t *Loc, RelType Type, uint64_t Val) const override;
 
   RelExpr adjustRelaxExpr(RelType Type, const uint8_t *Data,
                           RelExpr Expr) const override;
-  void relaxGot(uint8_t *Loc, uint64_t Val) const override;
+  void relaxGot(uint8_t *Loc, RelType Type, uint64_t Val) const override;
   void relaxTlsGdToIe(uint8_t *Loc, RelType Type, uint64_t Val) const override;
   void relaxTlsGdToLe(uint8_t *Loc, RelType Type, uint64_t Val) const override;
   void relaxTlsIeToLe(uint8_t *Loc, RelType Type, uint64_t Val) const override;
   void relaxTlsLdToLe(uint8_t *Loc, RelType Type, uint64_t Val) const override;
   bool adjustPrologueForCrossSplitStack(uint8_t *Loc, uint8_t *End,
                                         uint8_t StOther) const override;
 
 private:
void X86_64<ELFT>::relaxGotNoPic(uint8_t *Loc, uint64_t Val, uint8_t Op,
   // "INSTRUCTION SET REFERENCE, N-Z" (Vol. 2B 4-1) for
   // descriptions about each operation.
   Loc[-2] = 0x81;
   Loc[-3] = (Rex & ~0x4) | (Rex & 0x4) >> 2;
   write32le(Loc, Val);
 }
 
 template <class ELFT>
-void X86_64<ELFT>::relaxGot(uint8_t *Loc, uint64_t Val) const {
+void X86_64<ELFT>::relaxGot(uint8_t *Loc, RelType Type, uint64_t Val) const {
   const uint8_t Op = Loc[-2];
   const uint8_t ModRm = Loc[-1];
 
   // Convert "mov foo@GOTPCREL(%rip),%reg" to "lea foo(%rip),%reg".
   if (Op == 0x8b) {
     Loc[-2] = 0x8d;
     write32le(Loc, Val);
     return;

ELF/InputSection.cpp

static uint64_t getRelocTargetVA(const InputFile *File, RelType Type, int64_t A,
   case R_GOT_PLT:
   case R_RELAX_TLS_GD_TO_IE_ABS:
     return Sym.getGotVA() + A;
   case R_GOTONLY_PC:
     return In.Got->getVA() + A - P;
   case R_GOTONLY_PC_FROM_END:
     return In.Got->getVA() + A - P + In.Got->getSize();
   case R_GOTREL:
+  case R_PPC64_RELAX_TOC:
     return Sym.getVA(A) - In.Got->getVA();
   case R_GOTREL_FROM_END:
     return Sym.getVA(A) - In.Got->getVA() - In.Got->getSize();
   case R_GOT_FROM_END:
   case R_RELAX_TLS_GD_TO_IE_END:
     return Sym.getGotOffset() + A - In.Got->getSize();
   case R_TLSLD_GOT_OFF:
   case R_GOT_OFF:
else if (Sec->AreRelocsRela)
     Sec->relocateNonAlloc<ELFT>(Buf, Sec->template relas<ELFT>());
   else
     Sec->relocateNonAlloc<ELFT>(Buf, Sec->template rels<ELFT>());
 }
 
 void InputSectionBase::relocateAlloc(uint8_t *Buf, uint8_t *BufEnd) {
   assert(Flags & SHF_ALLOC);
   const unsigned Bits = Config->Wordsize * 8;
 
   for (const Relocation &Rel : Relocations) {
     uint64_t Offset = Rel.Offset;
     if (auto *Sec = dyn_cast<InputSection>(this))
       Offset += Sec->OutSecOff;
     uint8_t *BufLoc = Buf + Offset;
     RelType Type = Rel.Type;
 
     uint64_t AddrLoc = getOutputSection()->Addr + Offset;
     RelExpr Expr = Rel.Expr;
     uint64_t TargetVA = SignExtend64(
         getRelocTargetVA(File, Type, Rel.Addend, AddrLoc, *Rel.Sym, Expr),
         Bits);
 
     switch (Expr) {
     case R_RELAX_GOT_PC:
     case R_RELAX_GOT_PC_NOPIC:
-      Target->relaxGot(BufLoc, TargetVA);
+      Target->relaxGot(BufLoc, Type, TargetVA);
+      break;
+    case R_PPC64_RELAX_TOC:
+      // If the relocation is relaxable tryTocRelax will perform the relaxation
+      // and return true. Otherwise relocate to the original .toc entry.
+      if (!tryRelaxTocPPC64(Type, Rel, Expr, BufLoc))
+        Target->relocateOne(BufLoc, Type, TargetVA);
       break;
     case R_RELAX_TLS_IE_TO_LE:
       Target->relaxTlsIeToLe(BufLoc, Type, TargetVA);
       break;
     case R_RELAX_TLS_LD_TO_LE:
     case R_RELAX_TLS_LD_TO_LE_ABS:
       Target->relaxTlsLdToLe(BufLoc, Type, TargetVA);
       break;
     case R_RELAX_TLS_GD_TO_LE:
     case R_RELAX_TLS_GD_TO_LE_NEG:
       Target->relaxTlsGdToLe(BufLoc, Type, TargetVA);
       break;
     case R_AARCH64_RELAX_TLS_GD_TO_IE_PAGE_PC:
     case R_RELAX_TLS_GD_TO_IE:
     case R_RELAX_TLS_GD_TO_IE_ABS:
     case R_RELAX_TLS_GD_TO_IE_GOT_OFF:
     case R_RELAX_TLS_GD_TO_IE_END:
       Target->relaxTlsGdToIe(BufLoc, Type, TargetVA);
       break;
     case R_PPC_CALL:
       // If this is a call to __tls_get_addr, it may be part of a TLS
       // sequence that has been relaxed and turned into a nop. In this
       // case, we don't want to handle it as a call.
       if (read32(BufLoc) == 0x60000000) // nop
         break;
 
       // Patch a nop (0x60000000) to a ld.
       if (Rel.Sym->NeedsTocRestore) {
         if (BufLoc + 8 > BufEnd || read32(BufLoc + 4) != 0x60000000) {
           error(getErrorLocation(BufLoc) + "call lacks nop, can't restore toc");
           break;
         }
         write32(BufLoc + 4, 0xe8410018); // ld %r2, 24(%r1)
       }
       Target->relocateOne(BufLoc, Type, TargetVA);
       break;
     default:
       Target->relocateOne(BufLoc, Type, TargetVA);
       break;
     }
   }
 }
 
 // For each function-defining prologue, find any calls to __morestack,
 // and replace them with calls to __morestack_non_split.
 static void switchMorestackCallsToMorestackNonSplit(
     DenseSet<Defined *> &Prologues, std::vector<Relocation *> &MorestackCalls) {
 

ELF/Relocations.h

enum RelExpr {
   R_NEG_TLS,
   R_NONE,
   R_PC,
   R_PLT,
   R_PLT_PC,
   R_PPC_CALL,
   R_PPC_CALL_PLT,
   R_PPC_TOC,
+  R_PPC64_RELAX_TOC,
   R_RELAX_GOT_PC,
   R_RELAX_GOT_PC_NOPIC,
   R_RELAX_TLS_GD_TO_IE,
   R_RELAX_TLS_GD_TO_IE_ABS,
   R_RELAX_TLS_GD_TO_IE_END,
   R_RELAX_TLS_GD_TO_IE_GOT_OFF,
   R_RELAX_TLS_GD_TO_LE,
   R_RELAX_TLS_GD_TO_LE_NEG,
   R_RELAX_TLS_IE_TO_LE,
   R_RELAX_TLS_LD_TO_LE,
   R_RELAX_TLS_LD_TO_LE_ABS,

ELF/Relocations.cpp

return isRelExprOneOf<R_GOT, R_GOT_OFF, R_HEXAGON_GOT, R_MIPS_GOT_LOCAL_PAGE,
                         R_AARCH64_GOT_PAGE_PC_PLT, R_GOT_PC, R_GOT_FROM_END,
                         R_GOT_PLT>(Expr);
 }
 
 // True if this expression is of the form Sym - X, where X is a position in the
 // file (PC, or GOT for example).
 static bool isRelExpr(RelExpr Expr) {
   return isRelExprOneOf<R_PC, R_GOTREL, R_GOTREL_FROM_END, R_MIPS_GOTREL,
-                        R_PPC_CALL, R_PPC_CALL_PLT, R_AARCH64_PAGE_PC,
-                        R_AARCH64_PLT_PAGE_PC, R_RELAX_GOT_PC>(Expr);
+                        R_PPC_CALL, R_PPC_CALL_PLT, R_PPC64_RELAX_TOC,
+                        R_AARCH64_PAGE_PC, R_AARCH64_PLT_PAGE_PC,
+                        R_RELAX_GOT_PC>(Expr);
 }
 
 // Returns true if a given relocation can be computed at link-time.
 //
 // For instance, we know the offset from a relocation to its target at
 // link-time if the relocation is PC-relative and refers a
 // non-interposable function in the same executable. This function
 // will return true for such relocation.
 //
 // If this function returns false, that means we need to emit a
 // dynamic relocation so that the relocation will be fixed at load-time.
 static bool isStaticLinkTimeConstant(RelExpr E, RelType Type, const Symbol &Sym,
                                      InputSectionBase &S, uint64_t RelOff) {
   // These expressions always compute a constant
   if (isRelExprOneOf<R_GOT_FROM_END, R_GOT_OFF, R_HEXAGON_GOT, R_TLSLD_GOT_OFF,
                      R_MIPS_GOT_LOCAL_PAGE, R_MIPS_GOTREL, R_MIPS_GOT_OFF,
                      R_MIPS_GOT_OFF32, R_MIPS_GOT_GP_PC, R_MIPS_TLSGD,
                      R_AARCH64_GOT_PAGE_PC, R_AARCH64_GOT_PAGE_PC_PLT, R_GOT_PC,
                      R_GOTONLY_PC, R_GOTONLY_PC_FROM_END, R_PLT_PC, R_TLSGD_GOT,
                      R_TLSGD_GOT_FROM_END, R_TLSGD_PC, R_PPC_CALL_PLT,
-                     R_TLSDESC_CALL, R_AARCH64_TLSDESC_PAGE, R_HINT,
-                     R_TLSLD_HINT, R_TLSIE_HINT>(E))
+                     R_PPC64_RELAX_TOC, R_TLSDESC_CALL, R_AARCH64_TLSDESC_PAGE,
+                     R_HINT, R_TLSLD_HINT, R_TLSIE_HINT>(E))
     return true;
 
   // These never do, except if the entire file is position dependent or if
   // only the low bits are used.
   if (E == R_GOT || E == R_GOT_PLT || E == R_PLT || E == R_TLSDESC)
     return Target->usesOnlyLowPageBits(Type) || !Config->Pic;
 
   if (Sym.IsPreemptible)
if (Sym.isGnuIFunc()) {
     Expr = Target->adjustRelaxExpr(Type, RelocatedAddr, Expr);
   } else if (!Sym.IsPreemptible) {
     Expr = fromPlt(Expr);
   }
 
   // This relocation does not require got entry, but it is relative to got and
   // needs it to be created. Here we request for that.
   if (isRelExprOneOf<R_GOTONLY_PC, R_GOTONLY_PC_FROM_END, R_GOTREL,
-                     R_GOTREL_FROM_END, R_PPC_TOC>(Expr))
+                     R_GOTREL_FROM_END, R_PPC_TOC, R_PPC64_RELAX_TOC>(Expr))
     In.Got->HasGotOffRel = true;
 
   // Read an addend.
   int64_t Addend = computeAddend<ELFT>(Rel, End, Sec, Expr, Sym.isLocal());
 
   // Process some TLS relocations, including relaxing TLS relocations.
   // Note that this function does not handle all TLS relocations.
   if (unsigned Processed =
static void scanRelocs(InputSectionBase &Sec, ArrayRef<RelTy> Rels) {
   OffsetGetter GetOffset(Sec);
 
   // Not all relocations end up in Sec.Relocations, but a lot do.
   Sec.Relocations.reserve(Rels.size());
 
   for (auto I = Rels.begin(), End = Rels.end(); I != End;)
     scanReloc<ELFT>(Sec, GetOffset, I, End);
 
-  // Sort relocations by offset to binary search for R_RISCV_PCREL_HI20
-  if (Config->EMachine == EM_RISCV)
+  // Sort relocations by offset for more efficient searching for
+  // R_RISCV_PCREL_HI20 and targets of toc-indirections on PPC64.
+  if (Config->EMachine == EM_RISCV ||
+      (Config->EMachine == EM_PPC64 && Sec.Name == ".toc"))
     std::stable_sort(Sec.Relocations.begin(), Sec.Relocations.end(),
                      RelocationOffsetComparator{});
 }
 
 template <class ELFT> void elf::scanRelocations(InputSectionBase &S) {
   if (S.AreRelocsRela)
     scanRelocs<ELFT>(S, S.relas<ELFT>());
   else

ELF/Target.h

public:
 
   // If a target needs to rewrite calls to __morestack to instead call
   // __morestack_non_split when a split-stack enabled caller calls a
   // non-split-stack callee this will return true. Otherwise returns false.
   bool NeedsMoreStackNonSplit = true;
 
   virtual RelExpr adjustRelaxExpr(RelType Type, const uint8_t *Data,
                                   RelExpr Expr) const;
-  virtual void relaxGot(uint8_t *Loc, uint64_t Val) const;
+  virtual void relaxGot(uint8_t *Loc, RelType Type, uint64_t Val) const;
   virtual void relaxTlsGdToIe(uint8_t *Loc, RelType Type, uint64_t Val) const;
   virtual void relaxTlsGdToLe(uint8_t *Loc, RelType Type, uint64_t Val) const;
   virtual void relaxTlsIeToLe(uint8_t *Loc, RelType Type, uint64_t Val) const;
   virtual void relaxTlsLdToLe(uint8_t *Loc, RelType Type, uint64_t Val) const;
 
 protected:
   // On FreeBSD x86_64 the first page cannot be mmaped.
   // On Linux that is controled by vm.mmap_min_addr. At least on some x86_64

 
 // Returns input section and corresponding source string for the given location.
 ErrorPlace getErrorPlace(const uint8_t *Loc);
 
 static inline std::string getErrorLocation(const uint8_t *Loc) {
   return getErrorPlace(Loc).Loc;
 }
 
-// In the PowerPC64 Elf V2 abi a function can have 2 entry points.  The first is
-// a global entry point (GEP) which typically is used to intiailzie the TOC
+// Tries to relax a toc-indirection. If an indirection can be safely relaxed
+// this function performs the relaxation and returns true.
+bool tryRelaxTocPPC64(RelType Type, const Relocation &Rel,
+                      RelExpr Expr, uint8_t *BufLoc);
+
+// In the PowerPC64 Elf V2 abi a function can have 2 entry points.  The first
+// is a global entry point (GEP) which typically is used to intiailzie the TOC
 // pointer in general purpose register 2.  The second is a local entry
 // point (LEP) which bypasses the TOC pointer initialization code. The
 // offset between GEP and LEP is encoded in a function's st_other flags.
 // This function will return the offset (in bytes) from the global entry-point
 // to the local entry-point.
 unsigned getPPC64GlobalEntryToLocalEntryOffset(uint8_t StOther);
 
 bool isPPC64SmallCodeModelReloc(RelType Type);

ELF/Target.cpp

void TargetInfo::writeIgotPlt(uint8_t *Buf, const Symbol &S) const {
   writeGotPlt(Buf, S);
 }
 
 RelExpr TargetInfo::adjustRelaxExpr(RelType Type, const uint8_t *Data,
                                     RelExpr Expr) const {
   return Expr;
 }
 
-void TargetInfo::relaxGot(uint8_t *Loc, uint64_t Val) const {
+void TargetInfo::relaxGot(uint8_t *Loc, RelType Type, uint64_t Val) const {
   llvm_unreachable("Should not have claimed to be relaxable");
 }
 
 void TargetInfo::relaxTlsGdToLe(uint8_t *Loc, RelType Type,
                                 uint64_t Val) const {
   llvm_unreachable("Should not have claimed to be relaxable");
 }
 

test/ELF/Inputs/ppc64-global.s

+    .type  glob,@object
+    .data
+    .global  glob
+    .p2align  2
+glob:
+    .long  55
+    .size  glob, 4
+

test/ELF/Inputs/ppc64-local.s

+    .data
+
+    .type  local_sym,@object
+    .hidden local_sym
+    .global local_sym
+    .p2align  2
+local_sym:
+    .long  55
+    .size  local_sym, 4
+
+    .type defualt_sym,@object
+    .global default_sym
+default_sym:
+    .long 0
+    .size default_sym, 4

test/ELF/Inputs/ppc64-toc-opt-defs.s

+    .data
+
+    .type   a,@object
+    .globl  a
+    .p2align        2
+a:
+    .long   1
+    .size   a, 4
+
+    .type   b,@object
+    .globl  b
+b:
+    .long   2
+    .size   b, 4
+
+    .type   c,@object
+    .globl  c
+c:
+    .long   3
+    .size   c, 4
+
+    .type   d,@object
+    .globl  d
+d:
+    .long   4
+    .size   d, 4
+
+    .type   e,@object
+    .globl  e
+e:
+    .long   5
+    .size   e, 4
+
+    .type   f,@object
+    .globl  f
+f:
+    .long   6
+    .size   f, 4
+
+    .type   g,@object
+    .globl  g
+    .p2align        2
+g:
+    .long   7
+    .size   g, 4

test/ELF/Inputs/ppc64-toc-opt-shared.s

+    .data
+
+    .type ext_1,@object
+    .global ext_1
+    .p2align 2
+ext_1:
+    .long 8
+    .size ext_1, 4
+
+    .type ext_2,@object
+    .global ext_2
+    .p2align 2
+ext_2:
+    .long 8
+    .size ext_2, 4
+

test/ELF/ppc64-func-entry-points.s

 # point and we branch to start of foo_external_same.
 
 // CHECK: _start:
 // CHECK: 10010020:       {{.*}}     bl .+144
 // CHECK: 10010034:       {{.*}}     bl .+84
 // CHECK: foo_external_diff:
 // CHECK-NEXT: 10010080:       {{.*}}     addis 2, 12, 2
 // CHECK-NEXT: 10010084:       {{.*}}     addi 2, 2, 32640
-// CHECK-NEXT: 10010088:       {{.*}}     nop
+// CHECK-NEXT: 10010088:       {{.*}}     addis 5, 2, -1
 // CHECK: foo_external_same:
 // CHECK-NEXT: 100100b0:       {{.*}}     add 3, 4, 3

test/ELF/ppc64-got-indirect.s

 # REQUIRES: ppc
 
 # RUN: llvm-mc -filetype=obj -triple=powerpc64le-unknown-linux %s -o %t.o
 # RUN: llvm-readobj -relocations %t.o | FileCheck -check-prefix=RELOCS-LE %s
-# RUN: ld.lld %t.o -o %t2
-# RUN: llvm-objdump -D %t2 | FileCheck %s --check-prefix=CHECK-LE
-# RUN: llvm-objdump -D %t2 | FileCheck %s
+# RUN: llvm-mc -filetype=obj -triple=powerpc64le-unknown-linux %p/Inputs/ppc64-global.s -o %t1.o
+# RUN: ld.lld -shared -o %t1.so %t1.o
+# RUN: ld.lld %t.o %t1.so -o %t2
+# RUN: llvm-objdump -D %t2 | FileCheck %s --check-prefixes=CHECK,CHECK-LE
 
 # RUN: llvm-mc -filetype=obj -triple=powerpc64-unknown-linux %s -o %t.o
 # RUN: llvm-readobj -relocations %t.o | FileCheck -check-prefix=RELOCS-BE %s
-# RUN: ld.lld %t.o -o %t2
-# RUN: llvm-objdump -D %t2 | FileCheck %s --check-prefix=CHECK-BE
-# RUN: llvm-objdump -D %t2 | FileCheck %s
+# RUN: llvm-mc -filetype=obj -triple=powerpc64-unknown-linux %p/Inputs/ppc64-global.s -o %t1.o
+# RUN: ld.lld -shared -o %t1.so %t1.o
+# RUN: ld.lld  %t.o %t1.so -o %t2
+# RUN: llvm-objdump -D %t2 | FileCheck %s --check-prefixes=CHECK,CHECK-BE
 
 # Make sure we calculate the offset correctly for a got-indirect access to a
 # global variable as described by the PPC64 ELF V2 abi.
   .text
   .abiversion 2
   .globl  _start                    # -- Begin function _start
   .p2align  4
   .type  _start,@function
# %bb.0:                                # %entry
   .long  0
   .quad  0
 .Lfunc_end0:
   .size  _start, .Lfunc_end0-.Lfunc_begin0
                                         # -- End function
   .section  .toc,"aw",@progbits
 .LC0:
   .tc glob[TC],glob
-  .type  glob,@object            # @glob
-  .data
-  .globl  glob
-  .p2align  2
-glob:
-  .long  55                      # 0x37
-  .size  glob, 4
 
 # Verify the relocations emitted for glob are through the .toc
 
 # RELOCS-LE: Relocations [
 # RELOCS-LE:  .rela.text {
 # RELOCS-LE:     0x0 R_PPC64_REL16_HA .TOC. 0x0
 # RELOCS-LE:     0x4 R_PPC64_REL16_LO .TOC. 0x4
 # RELOCS-LE:     0x8 R_PPC64_TOC16_HA .toc 0x0

 # RELOCS-BE:  }
 # RELOCS-BE:  .rela.toc {
 # RELOCS-BE:    0x0 R_PPC64_ADDR64 glob 0x0
 # RELOCS-BE:  }
 # RELOCS-BE:]
 
 # Verify that the global variable access is done through the correct
 # toc entry:
-# r2 = .TOC. = 0x10038000.
-# r3 = r2 - 32760 = 0x10030008 -> .toc entry for glob.
+# r2 = .TOC. = 0x100280c0
+# r3 = *(r2 - 32760) -->  (r2 - 32760) = 0x100280c8 -> .toc entry for glob.
 
 # CHECK: _start:
-# CHECK-NEXT: 10010000:  {{.*}}   addis 2, 12, 3
-# CHECK-NEXT: 10010004:  {{.*}}   addi 2, 2, -32768
+# CHECK-NEXT: 10010000:  {{.*}}   addis 2, 12, 2
+# CHECK-NEXT: 10010004:  {{.*}}   addi 2, 2, -32576
 # CHECK-NEXT: 10010008:  {{.*}}   nop
 # CHECK-NEXT: 1001000c:  {{.*}}   ld 3, -32760(2)
 # CHECK: 1001001c:  {{.*}}   lwa 3, 0(3)
 
-# CHECK-LE: Disassembly of section .data:
-# CHECK-LE-NEXT: glob:
-# CHECK-LE-NEXT: 10020000:       37 00 00 00
-
 # CHECK-LE: Disassembly of section .got:
 # CHECK-LE-NEXT: .got:
-# CHECK-LE-NEXT: 10030000:       00 80 03 10
-# CHECK-LE-NEXT: 10030004:       00 00 00 00
+# CHECK-LE-NEXT: 100200c0:       c0 80 02 10
+# CHECK-LE-NEXT: 100200c4:       00 00 00 00
 
 # Verify that .toc comes right after .got
 # CHECK-LE: Disassembly of section .toc:
-# CHECK-LE: 10030008:       00 00 02 10
-
-# CHECK-BE: Disassembly of section .data:
-# CHECK-BE-NEXT: glob:
-# CHECK-BE-NEXT: 10020000:       00 00 00 37
+# CHECK-LE: 100200c8 .toc
 
 # CHECK-BE: Disassembly of section .got:
 # CHECK-BE-NEXT: .got:
-# CHECK-BE-NEXT: 10030000:       00 00 00 00
-# CHECK-BE-NEXT: 10030004:       10 03 80 00
+# CHECK-BE-NEXT: 100200c0:       00 00 00 00
+# CHECK-BE-NEXT: 100200c4:       10 02 80 c0
 
 # Verify that .toc comes right after .got
 # CHECK-BE: Disassembly of section .toc:
-# CHECK-BE: 10030008:       00 00 00 00
-# CHECK-BE: 1003000c:       10 02 00 00
+# CHECK-BE: 100200c8 .toc

test/ELF/ppc64-relocs.s

 # REQUIRES: ppc
 
 # RUN: llvm-mc -filetype=obj -triple=powerpc64le-unknown-linux %s -o %t
-# RUN: ld.lld %t -o %t2
+# RUN: ld.lld --no-toc-optimize %t -o %t2
 # RUN: llvm-objdump -D %t2 | FileCheck %s --check-prefix=DATALE
 # RUN: llvm-objdump -D %t2 | FileCheck %s
 
 # RUN: llvm-mc -filetype=obj -triple=powerpc64-unknown-linux %s -o %t
-# RUN: ld.lld %t -o %t2
+# RUN: ld.lld --no-toc-optimize %t -o %t2
 # RUN: llvm-objdump -D %t2 | FileCheck %s --check-prefix=DATABE
 # RUN: llvm-objdump -D %t2 | FileCheck %s
 
 .text
 .global _start
 _start:
 .Lfoo:
   li      0,1

 
 .section .R_PPC64_TOC16_HA,"ax",@progbits
 .globl .FR_PPC64_TOC16_HA
 .FR_PPC64_TOC16_HA:
   addis 1, 2, .L1@toc@ha
 
 # CHECK: Disassembly of section .R_PPC64_TOC16_HA:
 # CHECK: .FR_PPC64_TOC16_HA:
-# CHECK: 10010018: {{.*}} nop
+# CHECK: 10010018: {{.*}} addis 1, 2, 0
 
 .section .R_PPC64_REL24,"ax",@progbits
 .globl .FR_PPC64_REL24
 .FR_PPC64_REL24:
   b .Lfoox
 .section .R_PPC64_REL24_2,"ax",@progbits
 .Lfoox:
 

 # DATABE: .rodata:
 # DATABE: 10000190: 00 00 fe b8
 
 # Address of rodata + value stored at rodata entry
 # should equal address of LBB0_2.
 # 0x10000190 + 0xfeb4 = 0x10010044
 # CHECK: Disassembly of section .R_PPC64_REL32:
 # CHECK: .FR_PPC64_REL32:
-# CHECK: 10010040: {{.*}} nop
-# CHECK: 10010044: {{.*}} ld 5, -32736(2)
+# CHECK: 10010040: {{.*}} addis 5, 2, 0
+# CHECK: 10010044: {{.*}} ld 5, -32736(5)
 # CHECK: 10010048: {{.*}} add 3, 3, 4
 
 .section .R_PPC64_REL64, "ax",@progbits
 .globl  .FR_PPC64_REL64
 .FR_PPC64_REL64:
         .cfi_startproc
         .cfi_personality 148, __foo
         li 0, 1

test/ELF/ppc64-toc-relax-jumptable.s

+# RUN: llvm-mc -filetype=obj -triple=powerpc64le-unknown-linux %s -o %t.o
+# RUN: ld.lld %t.o -o %t
+# RUN: llvm-objdump -D %t | FileCheck --check-prefixes=CHECK,LE %s
+
+# RUN: llvm-mc -filetype=obj -triple=powerpc64-unknown-linux %s -o %t.o
+# RUN: ld.lld %t.o -o %t
+# RUN: llvm-objdump -D %t | FileCheck --check-prefixes=CHECK,BE %s
+
+
+# Verify that the load from the .toc section was relaxed to an
+# add of an offset to the TOC base-pointer (calculating the address
+# of the jump table rather then loading the address from the .toc).
+
+# CHECK: Disassembly of section .rodata:
+# CHECK-NEXT: .rodata:
+# CHECK-NEXT: 10000190
+
+# CHECK-LABEL: _start
+# CHECK:       clrldi  3, 3, 62
+# CHECK-NEXT:  addis 4, 2, -2
+# CHECK-NEXT:  addi  4, 4, -32368
+# CHECK-NEXT:  sldi  3, 3, 2
+
+# LE: Disassembly of section .toc:
+# LE-NEXT: .toc:
+# LE-NEXT: 10020008:       90 01 00 10
+# LE-NEXT: 1002000c:       00 00 00 00
+
+# BE: Disassembly of section .toc:
+# BE-NEXT: .toc:
+# BE-NEXT: 10020008:       00 00 00 00
+# BE-NEXT: 1002000c:       10 00 01 90
+
+    .text
+    .global _start
+    .type _start, @function
+_start:
+.Lstart_gep:
+    addis 2, 12, .TOC.-.Lstart_gep@ha
+    addi  2,  2, .TOC.-.Lstart_gep@l
+.Lstart_lep:
+    .localentry _start, .Lstart_lep-.Lstart_gep
+    rldicl 3, 3, 0, 62
+    addis 4, 2, .LJTI_TE@toc@ha
+    ld    4, .LJTI_TE@toc@l(4)
+    sldi  3, 3, 2
+    lwax  3, 3, 4
+    add   3, 3, 4
+    mtctr 3
+    bctr
+
+.LBB1:
+    li 3, 0
+    blr
+.LBB2:
+    li 3, 10
+    blr
+.LBB3:
+    li 3, 55
+    blr
+.LBB4:
+    li 3, 255
+    blr
+
+    .section        .rodata,"a",@progbits
+    .p2align        2
+.LJT:
+    .long   .LBB1-.LJT
+    .long   .LBB2-.LJT
+    .long   .LBB3-.LJT
+    .long   .LBB4-.LJT
+
+.section        .toc,"aw",@progbits
+# TOC entry for the jumptable address.
+.LJTI_TE:
+    .tc .LJT[TC],.LJT
+

test/ELF/ppc64-toc-relax.s

+# RUN: llvm-mc -filetype=obj -triple=powerpc64le-unknown-linux %s -o %t.o
+# RUN: llvm-mc -filetype=obj -triple=powerpc64le-unknown-linux %p/Inputs/ppc64-global.s -o %t1.shared.o
+# RUN: llvm-mc -filetype=obj -triple=powerpc64le-unknown-linux %p/Inputs/ppc64-local.s -o %t2.o
+# RUN: ld.lld -shared %t1.shared.o -o %t.so
+# RUN: ld.lld %t.o %t2.o %t.so -o %t
+# RUN: llvm-objdump -d %t | FileCheck %s
+
+# RUN: ld.lld -shared %t.o %t2.o %t.so -o %t2.so
+# RUN: llvm-objdump -d %t2.so | FileCheck --check-prefix=SHARED %s
+
+# RUN: llvm-mc -filetype=obj -triple=powerpc64-unknown-linux %s -o %t.o
+# RUN: llvm-mc -filetype=obj -triple=powerpc64-unknown-linux %p/Inputs/ppc64-global.s -o %t1.shared.o
+# RUN: llvm-mc -filetype=obj -triple=powerpc64-unknown-linux %p/Inputs/ppc64-local.s -o %t2.o
+# RUN: ld.lld -shared %t1.shared.o -o %t.so
+# RUN: ld.lld %t.o %t2.o %t.so -o %t
+# RUN: llvm-objdump -d %t | FileCheck %s
+
+# RUN: ld.lld -shared %t.o %t2.o %t.so -o %t2.so
+# RUN: llvm-objdump -d %t2.so | FileCheck --check-prefix=SHARED %s
+
+    .abiversion 2
+    .text
+
+    .global should_relax
+    .type should_relax,@function
+
+    .global can_not_relax
+    .type can_not_relax,@function
+
+    .global maybe_relax
+    .type  maybe_relax,@function
+
+# Compiler emits a .toc entry for the symbol becuase it is not defined in this
+# compilation unit. A definition will get linked in from another object file.
+# The symbol is hidden making it locally defined and so relaxable from a
+# got-indirect access to a toc-relative access in both shared objects and
+# executables.
+should_relax:
+.Lshould_relax_gep:
+    addis 2, 12, .TOC.-.Lshould_relax_gep@ha
+    addi  2,  2, .TOC.-.Lshould_relax_gep@l
+.Lshould_relax_lep:
+    .localentry     should_relax, .Lshould_relax_lep-.Lshould_relax_gep
+    addis 3, 2, .LTE1@toc@ha
+    ld    3, .LTE1@toc@l(3)
+    lwa   3, 0(3)
+    blr
+
+# CHECK-LABEL: should_relax
+# CHECK-NEXT:  10010000:
+# CHECK:       10010008: {{.*}} addis 3, 2, -2
+# CHECK-NEXT:  1001000c: {{.*}} addi 3, 3, 32576
+# CHECK-NEXT:  10010010: {{.*}} lwa 3, 0(3)
+
+# SHARED-LABEL: should_relax:
+# SHARED-NEXT: 10000:
+# SHARED:      10008: {{.*}} addis 3, 2, -2
+# SHARED-NEXT: 1000c: {{.*}} addi 3, 3, 32576
+# SHARED-NEXT: 10010: {{.*}} lwa 3, 0(3)
+
+# Compiler emits a .toc entry for the symbol becuase it is not defined in this
+# compilation unit. A definition will get linked in from a shared-object making
+# it unrelaxable.
+can_not_relax:
+.Lcan_not_relax_gep:
+    addis  2, 12, .TOC.-.Lcan_not_relax_gep@ha
+    addi   2,  2, .TOC.-.Lcan_not_relax_gep@l
+.Lcan_not_relax_lep:
+    .localentry  can_not_relax, .Lcan_not_relax_lep-.Lcan_not_relax_gep
+    addis 3, 2, .LTE2@toc@ha
+    ld    3, .LTE2@toc@l(3)
+    lwa   3, 0(3)
+    blr
+
+# CHECK-LABEL: can_not_relax:
+# CHECK-NEXT: 10010018:
+# CHECK:      10010020: {{.*}} nop
+# CHECK-NEXT: 10010024: {{.*}} ld 3, -32752(2)
+# CHECK-NEXT: 10010028: {{.*}} lwa 3, 0(3)
+
+# SHARED-LABEL: can_not_relax:
+# SHARED-NEXT: 10018:
+# SHARED:      10020: {{.*}} nop
+# SHARED-NEXT: 10024: {{.*}} ld 3, -32752(2)
+# SHARED-NEXT: 10028: {{.*}} lwa 3, 0(3)
+
+# Compiler emits a .toc entry for the symbol becuase it is not defined in this
+# compilation unit. A definition with default visibility gets linked in locally.
+# It should be relaxed when linking an executable, and not relaxed if linking a
+# shared object.
+maybe_relax:
+.Lmaybe_relax_gep:
+    addis 2, 12, .TOC.-.Lmaybe_relax_gep@ha
+    addi  2,  2, .TOC.-.Lmaybe_relax_gep@l
+.Lmaybe_relax_lep:
+    .localentry  can_not_relax, .Lcan_not_relax_lep-.Lcan_not_relax_gep
+    addis 3, 2, .LTE3@toc@ha
+    ld    3, .LTE3@toc@l(3)
+    lwa   3, 0(3)
+    blr
+
+# CHECK-LABEL: maybe_relax:
+# CHECK-NEXT:   10010030:
+# CHECK:        10010038: {{.*}} addis 3, 2, -2
+# CHECK-NEXT:   1001003c: {{.*}} addi 3, 3, 32580
+# CHECK-NEXT:   10010040: {{.*}} lwa 3, 0(3)
+
+# SHARED-LABEL: maybe_relax:
+# SHARED-NEXT:   10030:
+# SHARED:        10038: {{.*}} nop
+# SHARED-NEXT:   1003c: {{.*}} ld 3, -32744(2)
+# SHARED-NEXT:   10040: {{.*}} lwa 3, 0(3)
+
+    .section        .toc,"aw",@progbits
+.LTE1:
+    .tc local_sym[TC], local_sym
+.LTE2:
+    .tc glob[TC], glob
+.LTE3:
+    .tc default_sym[TC], default_sym

test/ELF/ppc64-tocopt-constants-in-toc.s

+# REQUIRES: ppc
+
+# RUN: llvm-mc -filetype=obj -triple=powerpc64le-unknown-linux %s -o %t1.o
+# RUN: llvm-mc -filetype=obj -triple=powerpc64le-unknown-linux %p/Inputs/ppc64-toc-opt-defs.s -o %t2.o
+# RUN: llvm-mc -filetype=obj -triple=powerpc64le-unkown-linux %p/Inputs/ppc64-toc-opt-shared.s -o %t3.o
+# RUN: ld.lld -shared %t3.o -o %t.so
+# RUN: ld.lld %t1.o %t2.o %t.so -o %t
+# RUN: llvm-objdump -D %t | FileCheck %s
+
+# Test to ensure that we can still perform toc-optimizations on .toc sections
+# that contain more then just symbol addresses.
+
+    .text
+    .global test
+    .p2align        4
+    .type   test,@function
+test:
+.Lgep:
+    addis 2, 12, .TOC.-.Lgep@ha
+    addi 2, 2, .TOC.-.Lgep@l
+.Llep:
+    .localentry     test, .Llep-.Lgep
+
+    addis 3, 2, .LA@toc@ha
+    ld 3, .LA@toc@l(3)
+    lwa 0, 0(3)
+
+    addis 4, 2, .LB@toc@ha
+    ld 4, .LB@toc@l(4)
+    lwa 4, 0(4)
+
+    addis 5, 2, .LC@toc@ha
+    ld 5, .LC@toc@l(5)
+    lwa 5, 0(5)
+
+    addis 6, 2, .LCONST1@toc@ha
+    lwa 6, .LCONST1@toc@l(6)
+
+    addis 7, 2, .LD@toc@ha
+    ld 7, .LD@toc@l(7)
+    lwa 7, 0(7)
+
+    addis 8, 2, .LE@toc@ha
+    ld 8, .LE@toc@l(8)
+    lwa 8, 0(8)
+
+    addis 9, 2, .LCONST2@toc@ha
+    ld 9, .LCONST2@toc@l(9)
+
+    addis 10, 2, .LEXT1@toc@ha
+    ld 10, .LEXT1@toc@l(10)
+    lwa 10, 0(10)
+
+    addis 11, 2, .LF@toc@ha
+    ld 11, .LF@toc@l(11)
+    lwa 11, 0(11)
+
+    addis 12, 2, .LEXT2@toc@ha
+    ld 12, .LEXT2@toc@l(12)
+    lwa 12, 0(12)
+
+    addis 3, 2, .LG@toc@ha
+    ld 3,  .LG@toc@l(3)
+    lwa 3, 0(3)
+
+    blr
+
+    .section        .toc,"aw",@progbits
+.LA:
+    .tc a[TC],a
+.LB:
+    .tc b[TC],b
+.LC:
+    .tc c[TC],c
+.LCONST1:
+    .quad 1
+.LD:
+    .tc d[TC],d
+.LE:
+    .tc e[TC],e
+.LCONST2:
+    .quad 22
+.LEXT1:
+    .tc ext_1[TC],ext_1
+.LF:
+    .tc f[TC],f
+.LEXT2:
+    .tc ext_2[TC],ext_2
+.LG:
+    .tc g[TC],g
+
+# a is at address 10020000 in the .data section, TOC-pointer points to 100380c0
+# offset from the TOC-pointer to a is −98496  ==> (-2 << 16) + 32576
+# CHECK: addis 3, 2, -2
+# CHECK: addi 3, 3, 32576
+# CHECK: lwa 0, 0(3)
+# CHECK: addis 4, 2, -2
+# CHECK: addi 4, 4, 32580
+# CHECK: lwa 4, 0(4)
+# CHECK: addis 5, 2, -2
+# CHECK: addi 5, 5, 32584
+# CHECK: lwa 5, 0(5)
+
+# The first constant stored in the toc is at address 100300e0 (.toc[3]).
+# The offset from the TOC-pointer to .toc[3] is -32736 which fits in 16 bits of
+# displacement. The high-adjusted part of the access is nop-ed out and the low
+# part of the access is rewritten to depend on r2.
+# CHECK: nop
+# CHECK: lwa 6, -32736(2)
+
+# CHECK: addis 7, 2, -2
+# CHECK: addi 7, 7, 32588
+# CHECK: lwa 7, 0(7)
+# CHECK: addis 8, 2, -2
+# CHECK: addi 8, 8, 32592
+# CHECK: lwa 8, 0(8)
+# CHECK: nop
+# CHECK: ld 9, -32712(2)
+
+# Access of an externaly defined symbol, so it can't be relaxed to a
+# toc-relative access.
+# CHECK: nop
+# CHECK: ld 10, -32704(2)
+# CHECK: lwa 10, 0(10)
+
+# CHECK: addis 11, 2, -2
+# CHECK: addi  11, 11, 32596
+# CHECK: lwa   11, 0(11)
+# CHECK: nop
+# CHECK: 12, -32688(2)
+# CHECK: lwa 12, 0(12)
+# CHECK: addis 3, 2, -2
+# CHECK: addi 3, 3, 32600
+# CHECK: lwa 3, 0(3)
+
+
+# CHECK: Disassembly of section .data:
+# CHECK:    0000000010020000 a:
+# CHECK:    0000000010020004 b:
+# CHECK:    0000000010020008 c:
+# CHECK:    000000001002000c d:
+# CHECK:    0000000010020010 e:
+# CHECK:    0000000010020014 f:
+# CHECK:    0000000010020018 g:
+
+# CHECK: 00000000100300c0 .got:
+# CHECK: 100300c0:       c0 80 03 10
+# CHECK: 100300c4:       00 00 00 00
+# CHECK: 00000000100300c8 .toc: