[RFC][ELF] Refactor relocation processing

[MaskRay]

Summary

Current relocation processing does:

if (the relocation expr belongs to category A)
  do something;
if (the relocation expr belongs to category B)
  do something;
if (the relocation expr belongs to category C)
  do something;

This refactor flattens it into

switch (expr) {
case R_PC:
case R_...:
  do something;
case R_...:
  do something;
...
}

which may potentially improve performance.
In addition, if some architectures have strange logic, we can add a new RelExpr member and make it interrupt less to the generic code.

[peter.smith]

Will take a look later this week, hopefully tomorrow. Just got back into the office and have a bit of a backlog of stuff to go through.

[peter.smith]

On a first look I like the way this is going. There is more information in a single place. I've made a few suggestions but none are particularly important for an RFC. If I get time I'll keep thinking to see if I can think of any other suggestions.

lld/ELF/Relocations.cpp

224

Is it worth moving closer to the only caller processRelocAux. Perhaps worth emphasisng that this only handles relocations that are expected in processRelocaAux. Could do with a comment or perhaps renaming to something like customRelocIsStaticLinkTimeConstant

If there were a way to know if a particular expr code could be CUSTOM outside of scanReloc then we could assert that which would make it clear, but I'm not sure if there is an easily maintainable way to do that.

227

I'm guessing that some of the entries have been taken out as the relocations were not CUSTOM so they would never expect to be called?

Perhaps worth changing the comment
// The following expressions always compute a constant. The list does not contain expressions marked RELOCATE in scanReloc.

952

suggest cantUseRelocationAgainstLocal or invalidRelocToLocal

1161

Perhaps worth renaming to handleTlsNonIE

1371

This is also in handleTlsRelocation, worth making an inline function? Something like toExecRelax()

1444

Would be a shame to lose the information in the comment, even if it had to be moved/split up.

[MaskRay]

Thanks for the comments. Applies changes.

I put up this patch because I think our relocation processing still has some problems
wand want explore how we should organize relocations.

lld/ELF/Relocations.cpp

224

Thanks for the suggestion. I just pushed the isStaticLinkTimeConstant move and comment adding separately.

952

Used invalidRelocAgainstSym.

I was thinking of extracting this out as a separate function to be shared, but currently it is only used once. I may inline it again.

[stefanp]

I feel like there is a lot going on in this patch. It may be easier if it was just done little by little.
For example: toExecRelax is now a static function. This can be separated out into a small NFC patch where the only thing that is done is the refactoring of that one static function.
Also, handleTlsRelocation returns a slightly different value (what should be skipped vs what was processed). This can also be done as a separate mini patch.

Anyway, those are my 2 cents.

lld/ELF/Relocations.cpp

1267–1269

Do we need this } and the return 0; ?
The if has been changed to an assert so now we just return return target->getTlsGdRelaxSkip(type) - 1;.

You can also remove the starting }.

[MaskRay]

In D113228#3122697, @stefanp wrote:

I feel like there is a lot going on in this patch. It may be easier if it was just done little by little.
For example: toExecRelax is now a static function. This can be separated out into a small NFC patch where the only thing that is done is the refactoring of that one static function.
Also, handleTlsRelocation returns a slightly different value (what should be skipped vs what was processed). This can also be done as a separate mini patch.

Anyway, those are my 2 cents.

Structurally it's difficult to do it piecemeal. This patch explores an alternative way handling relocations.
If we consider the current state and the exploration local optima, some changes (like moving toExecRelax outside) are actually strictly inferior (if we don't split handleTlsRelocations).

For minor details, some changes can be split and I've pushed such commits separately (e.g. moving isStaticLinkTimeConstant).

I have spent much time on this patch (some architectures like Mips and PPC64 add significant complexity) but not entirely decided this is the way to go.

[pkasting]

It's about a year later; I'm curious if you've gained any insight into whether this is a good direction or not.

[MaskRay]

In D113228#3869752, @pkasting wrote:

It's about a year later; I'm curious if you've gained any insight into whether this is a good direction or not.

I think there is value in optimizing relocation scanning. For some workload lld's pass takes 2.x time as mold's.
InputSectionBase::relocations takes some time but other checks contribute to the other slowdown.

We probably need to be more aggressive and possibly moving more relocation scanning into arch-specific ELF/Arch/*.cpp.

lld/ELF/Arch/AArch64.cpp

case R_AARCH64_ADR_PREL_PG_HI21_NC:
     return R_AARCH64_PAGE_PC;
   case R_AARCH64_LD64_GOT_LO12_NC:
     return R_GOT;
   case R_AARCH64_TLSIE_LD64_GOTTPREL_LO12_NC:
     return R_TLS_GOT;
   case R_AARCH64_LD64_GOTPAGE_LO15:
     return R_AARCH64_GOT_PAGE;
   case R_AARCH64_ADR_GOT_PAGE:
-  case R_AARCH64_TLSIE_ADR_GOTTPREL_PAGE21:
     return R_AARCH64_GOT_PAGE_PC;
+  case R_AARCH64_TLSIE_ADR_GOTTPREL_PAGE21:
+    return R_AARCH64_TLS_GOT_PAGE_PC;
   case R_AARCH64_NONE:
     return R_NONE;
   default:
     error(getErrorLocation(loc) + "unknown relocation (" + Twine(type) +
           ") against symbol " + toString(s));
     return R_NONE;
   }
 }

lld/ELF/Arch/PPC64.cpp

unsigned elf::getPPC64GlobalEntryToLocalEntryOffset(uint8_t stOther) {
   // log-base-2(offset).
   if (gepToLep < 7)
     return 1 << gepToLep;
 
   error("reserved value of 7 in the 3 most-significant-bits of st_other");
   return 0;
 }
 
-bool elf::isPPC64SmallCodeModelTocReloc(RelType type) {
-  // The only small code model relocations that access the .toc section.
-  return type == R_PPC64_TOC16 || type == R_PPC64_TOC16_DS;
-}
-
 void elf::writePrefixedInstruction(uint8_t *loc, uint64_t insn) {
   insn = config->isLE ? insn << 32 | insn >> 32 : insn;
   write64(loc, insn);
 }
 
 static bool addOptional(StringRef name, uint64_t value,
                         std::vector<Defined *> &defined) {
   Symbol *sym = symtab->find(name);

lld/ELF/InputSection.cpp

uint64_t InputSectionBase::getRelocTargetVA(const InputFile *file, RelType type,
   case R_RELAX_TLS_GD_TO_IE_GOTPLT:
     return sym.getGotVA() + a - in.gotPlt->getVA();
   case R_TLSLD_GOT_OFF:
   case R_GOT_OFF:
   case R_TLS_GOTOFF:
   case R_RELAX_TLS_GD_TO_IE_GOT_OFF:
     return sym.getGotOffset() + a;
   case R_AARCH64_GOT_PAGE_PC:
+  case R_AARCH64_TLS_GOT_PAGE_PC:
   case R_AARCH64_RELAX_TLS_GD_TO_IE_PAGE_PC:
     return getAArch64Page(sym.getGotVA() + a) - getAArch64Page(p);
   case R_AARCH64_GOT_PAGE:
     return sym.getGotVA() + a - getAArch64Page(in.got->getVA());
   case R_GOT_PC:
   case R_TLS_GOT_PC:
   case R_RELAX_TLS_GD_TO_IE:
     return sym.getGotVA() + a - p;

lld/ELF/Relocations.h

enum RelExpr {
   R_TLS_GOTOFF,
 
   // The following is abstract relocation types used for only one target.
   //
   // Even though RelExpr is intended to be a target-neutral representation
   // of a relocation type, there are some relocations whose semantics are
   // unique to a target. Such relocation are marked with R_<TARGET_NAME>.
   R_AARCH64_GOT_PAGE_PC,
+  R_AARCH64_TLS_GOT_PAGE_PC,
   R_AARCH64_GOT_PAGE,
   R_AARCH64_PAGE_PC,
   R_AARCH64_RELAX_TLS_GD_TO_IE_PAGE_PC,
   R_AARCH64_TLSDESC_PAGE,
   R_ARM_PCA,
   R_ARM_SBREL,
   R_MIPS_GOTREL,
   R_MIPS_GOT_GP,

lld/ELF/Relocations.cpp

 //
 // For instance, we know the offset from a relocation to its target at
 // link-time if the relocation is PC-relative and refers a
 // non-interposable function in the same executable. This function
 // will return true for such relocation.
 //
 // If this function returns false, that means we need to emit a
 // dynamic relocation so that the relocation will be fixed at load-time.
 static bool isStaticLinkTimeConstant(RelExpr e, RelType type, const Symbol &sym,
                                      InputSectionBase &s, uint64_t relOff) {
   // These expressions always compute a constant
-  if (oneof<R_GOTPLT, R_GOT_OFF, R_MIPS_GOT_LOCAL_PAGE, R_MIPS_GOTREL,
-            R_MIPS_GOT_OFF, R_MIPS_GOT_OFF32, R_MIPS_GOT_GP_PC,
-            R_AARCH64_GOT_PAGE, R_AARCH64_GOT_PAGE_PC, R_GOT_PC, R_GOTONLY_PC,
+  if (oneof<R_TLSLD_GOT_OFF, R_AARCH64_GOT_PAGE_PC, R_GOT_PC, R_GOTONLY_PC,
             R_GOTPLTONLY_PC, R_PLT_PC, R_PLT_GOTPLT, R_PPC32_PLTREL,
-            R_PPC64_CALL_PLT, R_PPC64_RELAX_TOC, R_RISCV_ADD>(e))
+            R_PPC64_RELAX_TOC>(e))
     return true;
 
   // These never do, except if the entire file is position dependent or if
   // only the low bits are used.
-  if (e == R_GOT || e == R_PLT || e == R_TLS_GOT)
+  if (e == R_GOT || e == R_PLT)
     return target->usesOnlyLowPageBits(type) || !config->isPic;
 
   if (sym.isPreemptible)
     return false;
   if (!config->isPic)
     return true;
 
   // The size of a non preemptible symbol is a constant.
static bool canDefineSymbolInExecutable(Symbol &sym) {
   // If we are allowed to break address equality of functions, defining
   // a plt entry will allow the program to call the function in the
   // .so, but the .so and the executable will no agree on the address
   // of the function. Similar logic for objects.
   return ((sym.isFunc() && config->ignoreFunctionAddressEquality) ||
           (sym.isObject() && config->ignoreDataAddressEquality));
 }
 
+static void cantUseRelocation(InputSectionBase &sec, uint64_t offset,
+                              RelType type, Symbol &sym) {
+  errorOrWarn("relocation " + toString(type) + " cannot be used against " +
+              (sym.getName().empty() ? "local symbol"
+                                     : "symbol '" + toString(sym) + "'") +
+              "; recompile with -fPIC" + getLocation(sec, sym, offset));
+}
+
 // The reason we have to do this early scan is as follows
 // * To mmap the output file, we need to know the size
 // * For that, we need to know how many dynamic relocs we will have.
 // It might be possible to avoid this by outputting the file with write:
 // * Write the allocated output sections, computing addresses.
 // * Apply relocations, recording which ones require a dynamic reloc.
 // * Write the dynamic relocations.
 // * Write the rest of the file.
if (sym.isFunc()) {
         }
       }
       sym.needsPltAddr = true;
       sec.relocations.push_back({expr, type, offset, addend, &sym});
       return;
     }
   }
 
-  errorOrWarn("relocation " + toString(type) + " cannot be used against " +
-              (sym.getName().empty() ? "local symbol"
-                                     : "symbol '" + toString(sym) + "'") +
-              "; recompile with -fPIC" + getLocation(sec, sym, offset));
+  cantUseRelocation(sec, offset, type, sym);
 }
 
 // This function is similar to the `handleTlsRelocation`. MIPS does not
 // support any relaxations for TLS relocations so by factoring out MIPS
 // handling in to the separate function we can simplify the code and do not
 // pollute other `handleTlsRelocation` by MIPS `ifs` statements.
 // Mips has a custom MipsGotSection that handles the writing of GOT entries
 // without dynamic relocations.
 static unsigned handleMipsTlsRelocation(RelType type, Symbol &sym,
                                         InputSectionBase &c, uint64_t offset,
                                         int64_t addend, RelExpr expr) {
   if (expr == R_MIPS_TLSLD) {
     in.mipsGot->addTlsIndex(*c.file);
     c.relocations.push_back({expr, type, offset, addend, &sym});
     return 1;
   }
   if (expr == R_MIPS_TLSGD) {
     in.mipsGot->addDynTlsEntry(*c.file, sym);
     c.relocations.push_back({expr, type, offset, addend, &sym});
     return 1;
   }
+  if (expr == R_DTPREL) {
+    c.relocations.push_back({expr, type, offset, addend, &sym});
+    return 1;
+  }
   return 0;
 }
 
+template <class ELFT>
+static void handleTlsIe(RelType type, Symbol &sym, InputSectionBase &c,
+                        typename ELFT::uint offset, int64_t addend,
+                        RelExpr expr, bool relax) {
+  // Initial-Exec relocs can be relaxed to Local-Exec if the symbol is locally
+  // defined.
+  if (relax) {
+    c.relocations.push_back({R_RELAX_TLS_IE_TO_LE, type, offset, addend, &sym});
+  } else if (expr != R_TLSIE_HINT) {
+    if (!sym.isInGot())
+      addTpOffsetGotEntry(sym);
+    // R_TLS_GOT needs a relative relocation for PIC on i386 and Hexagon.
+    if (expr == R_TLS_GOT && config->isPic &&
+        !target->usesOnlyLowPageBits(type))
+      addRelativeReloc(&c, offset, sym, addend, expr, type);
+    else
+      c.relocations.push_back({expr, type, offset, addend, &sym});
+  }
+}
+
 // Notes about General Dynamic and Local Dynamic TLS models below. They may
 // require the generation of a pair of GOT entries that have associated dynamic
 // relocations. The pair of GOT entries created are of the form GOT[e0] Module
 // Index (Used to find pointer to TLS block at run-time) GOT[e1] Offset of
 // symbol in TLS block.
 //
-// Returns the number of relocations processed.
+// Returns the number of subsequent relocations which should be skipped.
 template <class ELFT>
 static unsigned
 handleTlsRelocation(RelType type, Symbol &sym, InputSectionBase &c,
                     typename ELFT::uint offset, int64_t addend, RelExpr expr) {
-  if (!sym.isTls())
-    return 0;
-
-  if (config->emachine == EM_MIPS)
-    return handleMipsTlsRelocation(type, sym, c, offset, addend, expr);
-
   if (oneof<R_AARCH64_TLSDESC_PAGE, R_TLSDESC, R_TLSDESC_CALL, R_TLSDESC_PC,
             R_TLSDESC_GOTPLT>(expr) &&
       config->shared) {
     if (in.got->addDynTlsEntry(sym)) {
       uint64_t off = in.got->getGlobalDynOffset(sym);
       mainPart->relaDyn->addAddendOnlyRelocIfNonPreemptible(
           target->tlsDescRel, in.got, off, sym, target->tlsDescRel);
     }
     if (expr != R_TLSDESC_CALL)
       c.relocations.push_back({expr, type, offset, addend, &sym});
-    return 1;
+    return 0;
   }
 
   // ARM, Hexagon and RISC-V do not support GD/LD to IE/LE relaxation.  For
   // PPC64, if the file has missing R_PPC64_TLSGD/R_PPC64_TLSLD, disable
   // relaxation as well.
   bool toExecRelax = !config->shared && config->emachine != EM_ARM &&
                      config->emachine != EM_HEXAGON &&
                      config->emachine != EM_RISCV &&
handleTlsRelocation(RelType type, Symbol &sym, InputSectionBase &c,
   // unused. There only needs to be one module index entry.
   if (oneof<R_TLSLD_GOT, R_TLSLD_GOTPLT, R_TLSLD_PC, R_TLSLD_HINT>(
           expr)) {
     // Local-Dynamic relocs can be relaxed to Local-Exec.
     if (toExecRelax) {
       c.relocations.push_back(
           {target->adjustTlsExpr(type, R_RELAX_TLS_LD_TO_LE), type, offset,
            addend, &sym});
-      return target->getTlsGdRelaxSkip(type);
+      return target->getTlsGdRelaxSkip(type) - 1;
     }
     if (expr == R_TLSLD_HINT)
-      return 1;
+      return 0;
     if (in.got->addTlsIndex()) {
       if (isLocalInExecutable)
         in.got->relocations.push_back(
             {R_ADDEND, target->symbolicRel, in.got->getTlsIndexOff(), 1, &sym});
       else
         mainPart->relaDyn->addReloc(
             {target->tlsModuleIndexRel, in.got, in.got->getTlsIndexOff()});
     }
     c.relocations.push_back({expr, type, offset, addend, &sym});
-    return 1;
-  }
-
-  // Local-Dynamic relocs can be relaxed to Local-Exec.
-  if (expr == R_DTPREL) {
-    if (toExecRelax)
-      expr = target->adjustTlsExpr(type, R_RELAX_TLS_LD_TO_LE);
-    c.relocations.push_back({expr, type, offset, addend, &sym});
-    return 1;
-  }
-
-  // Local-Dynamic sequence where offset of tls variable relative to dynamic
-  // thread pointer is stored in the got. This cannot be relaxed to Local-Exec.
-  if (expr == R_TLSLD_GOT_OFF) {
-    if (!sym.isInGot()) {
-      in.got->addEntry(sym);
-      uint64_t off = sym.getGotOffset();
-      in.got->relocations.push_back(
-          {R_ABS, target->tlsOffsetRel, off, 0, &sym});
-    }
-    c.relocations.push_back({expr, type, offset, addend, &sym});
-    return 1;
+    return 0;
   }
 
-  if (oneof<R_AARCH64_TLSDESC_PAGE, R_TLSDESC, R_TLSDESC_CALL, R_TLSDESC_PC,
-            R_TLSDESC_GOTPLT, R_TLSGD_GOT, R_TLSGD_GOTPLT, R_TLSGD_PC>(expr)) {
+  assert(
+      (oneof<R_AARCH64_TLSDESC_PAGE, R_TLSDESC, R_TLSDESC_CALL, R_TLSDESC_PC,
+             R_TLSDESC_GOTPLT, R_TLSGD_GOT, R_TLSGD_GOTPLT, R_TLSGD_PC>(expr)));
+  {
     if (!toExecRelax) {
       if (in.got->addDynTlsEntry(sym)) {
         uint64_t off = in.got->getGlobalDynOffset(sym);
 
         if (isLocalInExecutable)
           // Write one to the GOT slot.
           in.got->relocations.push_back(
               {R_ADDEND, target->symbolicRel, off, 1, &sym});
         else
           mainPart->relaDyn->addSymbolReloc(target->tlsModuleIndexRel, in.got,
                                             off, sym);
 
         // If the symbol is preemptible we need the dynamic linker to write
         // the offset too.
         uint64_t offsetOff = off + config->wordsize;
         if (sym.isPreemptible)
           mainPart->relaDyn->addSymbolReloc(target->tlsOffsetRel, in.got,
                                             offsetOff, sym);
         else
           in.got->relocations.push_back(
               {R_ABS, target->tlsOffsetRel, offsetOff, 0, &sym});
       }
       c.relocations.push_back({expr, type, offset, addend, &sym});
-      return 1;
+      return 0;
     }
 
     // Global-Dynamic relocs can be relaxed to Initial-Exec or Local-Exec
     // depending on the symbol being locally defined or not.
     if (sym.isPreemptible) {
       c.relocations.push_back(
           {target->adjustTlsExpr(type, R_RELAX_TLS_GD_TO_IE), type, offset,
            addend, &sym});
       if (!sym.isInGot()) {
         in.got->addEntry(sym);
         mainPart->relaDyn->addSymbolReloc(target->tlsGotRel, in.got,
                                           sym.getGotOffset(), sym);
       }
     } else {
       c.relocations.push_back(
           {target->adjustTlsExpr(type, R_RELAX_TLS_GD_TO_LE), type, offset,
            addend, &sym});
     }
-    return target->getTlsGdRelaxSkip(type);
-  }
-
-  if (oneof<R_TLS_GOT, R_TLS_GOT_GOTPLT, R_TLS_GOT_PC, R_AARCH64_GOT_PAGE_PC,
-            R_TLS_GOTOFF, R_TLSIE_HINT>(expr)) {
-    // Initial-Exec relocs can be relaxed to Local-Exec if the symbol is locally
-    // defined.
-    if (toExecRelax && isLocalInExecutable) {
-      c.relocations.push_back(
-          {R_RELAX_TLS_IE_TO_LE, type, offset, addend, &sym});
-    } else if (expr != R_TLSIE_HINT) {
-      if (!sym.isInGot())
-        addTpOffsetGotEntry(sym);
-      // R_TLS_GOT needs a relative relocation for PIC on i386 and Hexagon.
-      if (expr == R_TLS_GOT && config->isPic &&
-          !target->usesOnlyLowPageBits(type))
-        addRelativeReloc(&c, offset, sym, addend, expr, type);
-      else
-        c.relocations.push_back({expr, type, offset, addend, &sym});
-    }
-    return 1;
+    return target->getTlsGdRelaxSkip(type) - 1;
   }
 
   return 0;
 }
 
 template <class ELFT, class RelTy>
 static void scanReloc(InputSectionBase &sec, OffsetGetter &getOffset, RelTy *&i,
                       RelTy *start, RelTy *end) {
   const RelTy &rel = *i;
   uint32_t symIndex = rel.getSymbol(config->isMips64EL);
   Symbol &sym = sec.getFile<ELFT>()->getSymbol(symIndex);
static void scanReloc(InputSectionBase &sec, OffsetGetter &getOffset, RelTy *&i,
   // Error if the target symbol is undefined. Symbol index 0 may be used by
   // marker relocations, e.g. R_*_NONE and R_ARM_V4BX. Don't error on them.
   if (symIndex != 0 && maybeReportUndefined(sym, sec, rel.r_offset))
     return;
 
   const uint8_t *relocatedAddr = sec.data().begin() + rel.r_offset;
   RelExpr expr = target->getRelExpr(type, sym, relocatedAddr);
 
-  // Ignore R_*_NONE and other marker relocations.
-  if (expr == R_NONE)
-    return;
-
   // Read an addend.
   int64_t addend = computeAddend<ELFT>(rel, end, sec, expr, sym.isLocal());
 
-  if (config->emachine == EM_PPC64) {
-    // We can separate the small code model relocations into 2 categories:
-    // 1) Those that access the compiler generated .toc sections.
-    // 2) Those that access the linker allocated got entries.
-    // lld allocates got entries to symbols on demand. Since we don't try to
-    // sort the got entries in any way, we don't have to track which objects
-    // have got-based small code model relocs. The .toc sections get placed
-    // after the end of the linker allocated .got section and we do sort those
-    // so sections addressed with small code model relocations come first.
-    if (isPPC64SmallCodeModelTocReloc(type))
-      sec.file->ppc64SmallCodeModelTocRelocs = true;
-
-    // Record the TOC entry (.toc + addend) as not relaxable. See the comment in
-    // InputSectionBase::relocateAlloc().
-    if (type == R_PPC64_TOC16_LO && sym.isSection() && isa<Defined>(sym) &&
-        cast<Defined>(sym).section->name == ".toc")
-      ppc64noTocRelax.insert({&sym, addend});
-
-    if ((type == R_PPC64_TLSGD && expr == R_TLSDESC_CALL) ||
-        (type == R_PPC64_TLSLD && expr == R_TLSLD_HINT)) {
-      if (i == end) {
-        errorOrWarn("R_PPC64_TLSGD/R_PPC64_TLSLD may not be the last "
-                    "relocation" +
-                    getLocation(sec, sym, offset));
-        return;
-      }
-
-      // Offset the 4-byte aligned R_PPC64_TLSGD by one byte in the NOTOC case,
-      // so we can discern it later from the toc-case.
-      if (i->getType(/*isMips64EL=*/false) == R_PPC64_REL24_NOTOC)
-        ++offset;
-    }
-  }
-
-  // Relax relocations.
-  //
-  // If we know that a PLT entry will be resolved within the same ELF module, we
-  // can skip PLT access and directly jump to the destination function. For
-  // example, if we are linking a main executable, all dynamic symbols that can
-  // be resolved within the executable will actually be resolved that way at
-  // runtime, because the main executable is always at the beginning of a search
-  // list. We can leverage that fact.
-  if (!sym.isPreemptible && (!sym.isGnuIFunc() || config->zIfuncNoplt)) {
-    if (expr != R_GOT_PC) {
-      // The 0x8000 bit of r_addend of R_PPC_PLTREL24 is used to choose call
-      // stub type. It should be ignored if optimized to R_PC.
-      if (config->emachine == EM_PPC && expr == R_PPC32_PLTREL)
+  if (sym.isGnuIFunc()) {
+    if (config->zIfuncNoplt) {
+      if (!sym.isPreemptible) {
+        if (expr == R_PPC32_PLTREL)
           addend &= ~0x8000;
-      // R_HEX_GD_PLT_B22_PCREL (call a@GDPLT) is transformed into
-      // call __tls_get_addr even if the symbol is non-preemptible.
-      if (!(config->emachine == EM_HEXAGON &&
-           (type == R_HEX_GD_PLT_B22_PCREL ||
-            type == R_HEX_GD_PLT_B22_PCREL_X ||
-            type == R_HEX_GD_PLT_B32_PCREL_X)))
         expr = fromPlt(expr);
-    } else if (!isAbsoluteValue(sym)) {
-      expr = target->adjustGotPcExpr(type, addend, relocatedAddr);
-    }
       }
-
-  // If the relocation does not emit a GOT or GOTPLT entry but its computation
-  // uses their addresses, we need GOT or GOTPLT to be created.
-  //
-  // The 5 types that relative GOTPLT are all x86 and x86-64 specific.
-  if (oneof<R_GOTPLTONLY_PC, R_GOTPLTREL, R_GOTPLT, R_PLT_GOTPLT,
-            R_TLSDESC_GOTPLT, R_TLSGD_GOTPLT, R_TLS_GOT_GOTPLT>(expr)) {
-    in.gotPlt->hasGotPltOffRel = true;
-  } else if (oneof<R_GOTONLY_PC, R_GOTREL, R_PPC64_TOCBASE, R_PPC64_RELAX_TOC>(
-                 expr)) {
-    in.got->hasGotOffRel = true;
-  }
-
-  // Process TLS relocations, including relaxing TLS relocations. Note that
-  // R_TPREL and R_TPREL_NEG relocations are resolved in processRelocAux.
-  if (expr == R_TPREL || expr == R_TPREL_NEG) {
-    if (config->shared) {
-      errorOrWarn("relocation " + toString(type) + " against " + toString(sym) +
-                  " cannot be used with -shared" +
-                  getLocation(sec, sym, offset));
-      return;
-    }
-  } else if (unsigned processed = handleTlsRelocation<ELFT>(
-                 type, sym, sec, offset, addend, expr)) {
-    i += (processed - 1);
-    return;
-  }
-
-  // We were asked not to generate PLT entries for ifuncs. Instead, pass the
-  // direct relocation on through.
-  if (sym.isGnuIFunc() && config->zIfuncNoplt) {
+      // TODO adjustGotPcExpr
       sym.exportDynamic = true;
       mainPart->relaDyn->addSymbolReloc(type, &sec, offset, sym, addend, type);
       return;
     }
 
-  // Non-preemptible ifuncs require special handling. First, handle the usual
-  // case where the symbol isn't one of these.
-  if (!sym.isGnuIFunc() || sym.isPreemptible) {
-    // If a relocation needs PLT, we create PLT and GOTPLT slots for the symbol.
-    if (needsPlt(expr) && !sym.isInPlt())
-      addPltEntry(in.plt, in.gotPlt, in.relaPlt, target->pltRel, sym);
-
-    // Create a GOT slot if a relocation needs GOT.
-    if (needsGot(expr)) {
-      if (config->emachine == EM_MIPS) {
-        // MIPS ABI has special rules to process GOT entries and doesn't
-        // require relocation entries for them. A special case is TLS
-        // relocations. In that case dynamic loader applies dynamic
-        // relocations to initialize TLS GOT entries.
-        // See "Global Offset Table" in Chapter 5 in the following document
-        // for detailed description:
-        // ftp://www.linux-mips.org/pub/linux/mips/doc/ABI/mipsabi.pdf
-        in.mipsGot->addEntry(*sec.file, sym, addend, expr);
-      } else if (!sym.isInGot()) {
-        addGotEntry(sym);
-      }
-    }
-  } else {
-    // Handle a reference to a non-preemptible ifunc. These are special in a
-    // few ways:
-    //
-    // - Unlike most non-preemptible symbols, non-preemptible ifuncs do not have
-    //   a fixed value. But assuming that all references to the ifunc are
-    //   GOT-generating or PLT-generating, the handling of an ifunc is
-    //   relatively straightforward. We create a PLT entry in Iplt, which is
-    //   usually at the end of .plt, which makes an indirect call using a
-    //   matching GOT entry in igotPlt, which is usually at the end of .got.plt.
-    //   The GOT entry is relocated using an IRELATIVE relocation in relaIplt,
-    //   which is usually at the end of .rela.plt. Unlike most relocations in
-    //   .rela.plt, which may be evaluated lazily without -z now, dynamic
-    //   loaders evaluate IRELATIVE relocs eagerly, which means that for
-    //   IRELATIVE relocs only, GOT-generating relocations can point directly to
-    //   .got.plt without requiring a separate GOT entry.
-    //
-    // - Despite the fact that an ifunc does not have a fixed value, compilers
-    //   that are not passed -fPIC will assume that they do, and will emit
-    //   direct (non-GOT-generating, non-PLT-generating) relocations to the
-    //   symbol. This means that if a direct relocation to the symbol is
-    //   seen, the linker must set a value for the symbol, and this value must
-    //   be consistent no matter what type of reference is made to the symbol.
-    //   This can be done by creating a PLT entry for the symbol in the way
-    //   described above and making it canonical, that is, making all references
-    //   point to the PLT entry instead of the resolver. In lld we also store
-    //   the address of the PLT entry in the dynamic symbol table, which means
-    //   that the symbol will also have the same value in other modules.
-    //   Because the value loaded from the GOT needs to be consistent with
-    //   the value computed using a direct relocation, a non-preemptible ifunc
-    //   may end up with two GOT entries, one in .got.plt that points to the
-    //   address returned by the resolver and is used only by the PLT entry,
-    //   and another in .got that points to the PLT entry and is used by
-    //   GOT-generating relocations.
-    //
-    // - The fact that these symbols do not have a fixed value makes them an
-    //   exception to the general rule that a statically linked executable does
-    //   not require any form of dynamic relocation. To handle these relocations
-    //   correctly, the IRELATIVE relocations are stored in an array which a
-    //   statically linked executable's startup code must enumerate using the
-    //   linker-defined symbols __rela?_iplt_{start,end}.
+    if (!sym.isPreemptible) {
       if (!sym.isInPlt()) {
         // Create PLT and GOTPLT slots for the symbol.
         sym.isInIplt = true;
 
         // Create a copy of the symbol to use as the target of the IRELATIVE
         // relocation in the igotPlt. This is in case we make the PLT canonical
         // later, which would overwrite the original symbol.
         //
         // FIXME: Creating a copy of the symbol here is a bit of a hack. All
-      // that's really needed to create the IRELATIVE is the section and value,
-      // so ideally we should just need to copy those.
+        // that's really needed to create the IRELATIVE is the section and
+        // value, so ideally we should just need to copy those.
         auto *directSym = make<Defined>(cast<Defined>(sym));
         addPltEntry(in.iplt, in.igotPlt, in.relaIplt, target->iRelativeRel,
                     *directSym);
         sym.pltIndex = directSym->pltIndex;
       }
       if (needsGot(expr)) {
         // Redirect GOT accesses to point to the Igot.
         //
         // This field is also used to keep track of whether we ever needed a GOT
         // entry. If we did and we make the PLT canonical later, we'll need to
         // create a GOT entry pointing to the PLT entry for Sym.
         sym.gotInIgot = true;
       } else if (!needsPlt(expr)) {
         // Make the ifunc's PLT entry canonical by changing the value of its
         // symbol to redirect all references to point to it.
         auto &d = cast<Defined>(sym);
         d.section = in.iplt;
         d.value = sym.pltIndex * target->ipltEntrySize;
         d.size = 0;
         // It's important to set the symbol type here so that dynamic loaders
         // don't try to call the PLT as if it were an ifunc resolver.
         d.type = STT_FUNC;
 
         if (sym.gotInIgot) {
           // We previously encountered a GOT generating reference that we
           // redirected to the Igot. Now that the PLT entry is canonical we must
           // clear the redirection to the Igot and add a GOT entry. As we've
-        // changed the symbol type to STT_FUNC future GOT generating references
-        // will naturally use this GOT entry.
+          // changed the symbol type to STT_FUNC future GOT generating
+          // references will naturally use this GOT entry.
           //
-        // We don't need to worry about creating a MIPS GOT here because ifuncs
-        // aren't a thing on MIPS.
+          // We don't need to worry about creating a MIPS GOT here because
+          // ifuncs aren't a thing on MIPS.
           sym.gotInIgot = false;
           addGotEntry(sym);
         }
       }
+      processRelocAux<ELFT>(sec, expr, type, offset, sym, addend);
+      return;
+    }
+  }
+
+#define OPTIMIZE_PLT (!sym.isPreemptible && !sym.isGnuIFunc())
+#define toExecRelax                                                            \
+  (!config->shared && config->emachine != EM_ARM &&                            \
+   config->emachine != EM_MIPS && config->emachine != EM_HEXAGON &&            \
+   config->emachine != EM_RISCV && !sec.file->ppc64DisableTLSRelax)
+
+  enum Action { RELOCATE, IGNORE, CUSTOM };
+  Action action = RELOCATE;
+  switch (expr) {
+  case R_NONE:
+    action = IGNORE;
+    break;
+
+  case R_ABS:
+  case R_SIZE:
+  case R_PC:
+  case R_AARCH64_PAGE_PC:
+    action = CUSTOM;
+    break;
+
+    //*** GOT
+  case R_GOT_PC:
+    if (!sym.isPreemptible && !isAbsoluteValue(sym))
+      expr = target->adjustGotPcExpr(type, addend, relocatedAddr);
+    if (expr != R_GOT_PC)
+      action = CUSTOM;
+    else if (!sym.isInGot())
+      addGotEntry(sym);
+    break;
+
+  case R_GOT:
+    if (!sym.isInGot())
+      addGotEntry(sym);
+    action = CUSTOM;
+    break;
+  case R_GOT_OFF:
+  case R_GOTPLT:
+  case R_AARCH64_GOT_PAGE_PC:
+  case R_AARCH64_GOT_PAGE:
+    if (expr == R_GOTPLT)
+      in.gotPlt->hasGotPltOffRel = true;
+    if (!sym.isInGot())
+      addGotEntry(sym);
+    break;
+
+  case R_GOTONLY_PC:
+    in.got->hasGotOffRel = true;
+    break;
+  case R_PPC64_TOCBASE:
+    in.got->hasGotOffRel = true;
+    action = CUSTOM;
+    break;
+  case R_GOTREL:
+    if (config->emachine == EM_PPC64) {
+      // We can separate the small code model relocations into 2 categories:
+      // 1) Those that access the compiler generated .toc sections.
+      // 2) Those that access the linker allocated got entries.
+      // lld allocates got entries to symbols on demand. Since we don't try to
+      // sort the got entries in any way, we don't have to track which objects
+      // have got-based small code model relocs. The .toc sections get placed
+      // after the end of the linker allocated .got section and we do sort those
+      // so sections addressed with small code model relocations come first.
+      if (type == R_PPC64_TOC16 || type == R_PPC64_TOC16_DS)
+        sec.file->ppc64SmallCodeModelTocRelocs = true;
+      // Record the TOC entry (.toc + addend) as not relaxable. See the comment
+      // in InputSectionBase::relocateAlloc().
+      if (type == R_PPC64_TOC16_LO && sym.isSection() && isa<Defined>(sym) &&
+          cast<Defined>(sym).section->name == ".toc")
+        ppc64noTocRelax.insert({&sym, addend});
     }
+    in.got->hasGotOffRel = true;
+    break;
+  case R_PPC64_RELAX_TOC:
+    in.got->hasGotOffRel = true;
+    break;
 
+  case R_MIPS_GOT_LOCAL_PAGE:
+  case R_MIPS_GOT_OFF:
+  case R_MIPS_GOT_OFF32:
+    // MIPS ABI has special rules to process GOT entries and doesn't
+    // require relocation entries for them. A special case is TLS
+    // relocations. In that case dynamic loader applies dynamic
+    // relocations to initialize TLS GOT entries.
+    // See "Global Offset Table" in Chapter 5 in the following document
+    // for detailed description:
+    // ftp://www.linux-mips.org/pub/linux/mips/doc/ABI/mipsabi.pdf
+    in.mipsGot->addEntry(*sec.file, sym, addend, expr);
+    break;
+
+    //*** .got.plt
+  case R_GOTPLTONLY_PC:
+  case R_GOTPLTREL:
+    in.gotPlt->hasGotPltOffRel = true;
+    break;
+
+    //*** PLT
+  case R_PLT:
+  case R_PLT_PC:
+  case R_PLT_GOTPLT:
+  case R_PPC64_CALL_PLT:
+    if (expr == R_PLT_GOTPLT)
+      in.gotPlt->hasGotPltOffRel = true;
+    if (OPTIMIZE_PLT) {
+      if (config->emachine == EM_HEXAGON &&
+          (type == R_HEX_GD_PLT_B22_PCREL || type == R_HEX_GD_PLT_B22_PCREL_X ||
+           type == R_HEX_GD_PLT_B32_PCREL_X)) {
+        if (!sym.isInPlt())
+          addPltEntry(in.plt, in.gotPlt, in.relaPlt, target->pltRel, sym);
+      } else {
+        expr = fromPlt(expr);
+        action = CUSTOM; // TODO R_PC vs SHN_ABS
+      }
+    } else {
+      if (!sym.isInPlt())
+        addPltEntry(in.plt, in.gotPlt, in.relaPlt, target->pltRel, sym);
+    }
+    break;
+  case R_PPC32_PLTREL:
+    if (OPTIMIZE_PLT) {
+      expr = R_PC;
+      addend &= ~0x8000;
+    } else if (!sym.isInPlt()) {
+      addPltEntry(in.plt, in.gotPlt, in.relaPlt, target->pltRel, sym);
+    }
+    break;
+
+    //*** TPREL
+  case R_TPREL:
+  case R_TPREL_NEG:
+    if (config->shared) {
+      errorOrWarn("relocation " + toString(type) + " against " + toString(sym) +
+                  " cannot be used with -shared" +
+                  getLocation(sec, sym, offset));
+      return;
+    }
+    if (!sym.isDefined())
+      action = CUSTOM;
+    break;
+
+    //*** TLS (non-TPREL)
+  case R_TLSDESC:
+  case R_TLSDESC_PC:
+  case R_AARCH64_TLSDESC_PAGE:
+  case R_TLSGD_GOT:
+  case R_TLSGD_PC:
+  case R_TLSLD_GOT:
+  case R_TLSLD_GOTPLT:
+  case R_TLSLD_PC:
+  case R_MIPS_TLSGD:
+  case R_MIPS_TLSLD:
+    if (config->emachine == EM_MIPS)
+      handleMipsTlsRelocation(type, sym, sec, offset, addend, expr);
+    else
+      i += handleTlsRelocation<ELFT>(type, sym, sec, offset, addend, expr);
+    return;
+  case R_DTPREL:
+    if (toExecRelax)
+      expr = target->adjustTlsExpr(type, R_RELAX_TLS_LD_TO_LE);
+    sec.relocations.push_back({expr, type, offset, addend, &sym});
+    return;
+  case R_TLSDESC_CALL:
+  case R_TLSLD_HINT:
+    if (config->emachine == EM_PPC64) {
+      if (i == end) {
+        errorOrWarn("R_PPC64_TLSGD/R_PPC64_TLSLD may not be the last "
+                    "relocation" +
+                    getLocation(sec, sym, offset));
+        return;
+      }
+
+      // Offset the 4-byte aligned R_PPC64_TLSGD by one byte in the NOTOC case,
+      // so we can discern it later from the toc-case.
+      if (i->getType(false) == R_PPC64_REL24_NOTOC)
+        ++offset;
+    }
+    i += handleTlsRelocation<ELFT>(type, sym, sec, offset, addend, expr);
+    return;
+
+  case R_TLSDESC_GOTPLT:
+  case R_TLSGD_GOTPLT:
+    in.gotPlt->hasGotPltOffRel = true;
+    i += handleTlsRelocation<ELFT>(type, sym, sec, offset, addend, expr);
+    return;
+
+  case R_TLSLD_GOT_OFF:
+    // PPC64 Local-Dynamic sequence where offset of tls variable relative to
+    // dynamic thread pointer is stored in the got. This cannot be relaxed to
+    // Local-Exec.
+    if (!sym.isInGot()) {
+      in.got->addEntry(sym);
+      uint64_t off = sym.getGotOffset();
+      in.got->relocations.push_back(
+          {R_ABS, target->tlsOffsetRel, off, 0, &sym});
+    }
+    break;
+
+  case R_TLS_GOT:
+  case R_TLS_GOT_PC:
+  case R_TLS_GOT_GOTPLT:
+  case R_TLSIE_HINT:
+  case R_AARCH64_TLS_GOT_PAGE_PC:
+  case R_TLS_GOTOFF: {
+    if (expr == R_TLS_GOT_GOTPLT)
+      in.gotPlt->hasGotPltOffRel = true;
+    handleTlsIe<ELFT>(type, sym, sec, offset, addend, expr,
+                      toExecRelax && !sym.isPreemptible);
+    return;
+  }
+
+  default:
+    break;
+  }
+
+  if (action == RELOCATE)
+    sec.relocations.push_back({expr, type, offset, addend, &sym});
+  else if (action != IGNORE)
     processRelocAux<ELFT>(sec, expr, type, offset, sym, addend);
 }
 
 // R_PPC64_TLSGD/R_PPC64_TLSLD is required to mark `bl __tls_get_addr` for
 // General Dynamic/Local Dynamic code sequences. If a GD/LD GOT relocation is
 // found but no R_PPC64_TLSGD/R_PPC64_TLSLD is seen, we assume that the
 // instructions are generated by very old IBM XL compilers. Work around the
 // issue by disabling GD/LD to IE/LE relaxation.
 template <class RelTy>

lld/ELF/Target.h

 // is a global entry point (GEP) which typically is used to initialize the TOC
 // pointer in general purpose register 2.  The second is a local entry
 // point (LEP) which bypasses the TOC pointer initialization code. The
 // offset between GEP and LEP is encoded in a function's st_other flags.
 // This function will return the offset (in bytes) from the global entry-point
 // to the local entry-point.
 unsigned getPPC64GlobalEntryToLocalEntryOffset(uint8_t stOther);
 
-// Returns true if a relocation is a small code model relocation that accesses
-// the .toc section.
-bool isPPC64SmallCodeModelTocReloc(RelType type);
-
 // Write a prefixed instruction, which is a 4-byte prefix followed by a 4-byte
 // instruction (regardless of endianness). Therefore, the prefix is always in
 // lower memory than the instruction.
 void writePrefixedInstruction(uint8_t *loc, uint64_t insn);
 
 void addPPC64SaveRestore();
 uint64_t getPPC64TocBase();
 uint64_t getAArch64Page(uint64_t expr);