[RFC][ELF] Refactor relocation processing

[MaskRay]

Summary

Current relocation processing does:

if (the relocation expr belongs to category A)
  do something;
if (the relocation expr belongs to category B)
  do something;
if (the relocation expr belongs to category C)
  do something;

This refactor flattens it into

switch (expr) {
case R_PC:
case R_...:
  do something;
case R_...:
  do something;
...
}

which may potentially improve performance.
In addition, if some architectures have strange logic, we can add a new RelExpr member and make it interrupt less to the generic code.

[peter.smith]

Will take a look later this week, hopefully tomorrow. Just got back into the office and have a bit of a backlog of stuff to go through.

[peter.smith]

On a first look I like the way this is going. There is more information in a single place. I've made a few suggestions but none are particularly important for an RFC. If I get time I'll keep thinking to see if I can think of any other suggestions.

lld/ELF/Relocations.cpp

214–215

I'm guessing that some of the entries have been taken out as the relocations were not CUSTOM so they would never expect to be called?

Perhaps worth changing the comment
// The following expressions always compute a constant. The list does not contain expressions marked RELOCATE in scanReloc.

214–215

Is it worth moving closer to the only caller processRelocAux. Perhaps worth emphasisng that this only handles relocations that are expected in processRelocaAux. Could do with a comment or perhaps renaming to something like customRelocIsStaticLinkTimeConstant

If there were a way to know if a particular expr code could be CUSTOM outside of scanReloc then we could assert that which would make it clear, but I'm not sure if there is an easily maintainable way to do that.

890

suggest cantUseRelocationAgainstLocal or invalidRelocToLocal

1168–1169

Perhaps worth renaming to handleTlsNonIE

1372

This is also in handleTlsRelocation, worth making an inline function? Something like toExecRelax()

1445

Would be a shame to lose the information in the comment, even if it had to be moved/split up.

[MaskRay]

Thanks for the comments. Applies changes.

I put up this patch because I think our relocation processing still has some problems
wand want explore how we should organize relocations.

lld/ELF/Relocations.cpp

214–215

Thanks for the suggestion. I just pushed the isStaticLinkTimeConstant move and comment adding separately.

890

Used invalidRelocAgainstSym.

I was thinking of extracting this out as a separate function to be shared, but currently it is only used once. I may inline it again.

[stefanp]

I feel like there is a lot going on in this patch. It may be easier if it was just done little by little.
For example: toExecRelax is now a static function. This can be separated out into a small NFC patch where the only thing that is done is the refactoring of that one static function.
Also, handleTlsRelocation returns a slightly different value (what should be skipped vs what was processed). This can also be done as a separate mini patch.

Anyway, those are my 2 cents.

lld/ELF/Relocations.cpp

1269–1271

Do we need this } and the return 0; ?
The if has been changed to an assert so now we just return return target->getTlsGdRelaxSkip(type) - 1;.

You can also remove the starting }.

[MaskRay]

In D113228#3122697, @stefanp wrote:

I feel like there is a lot going on in this patch. It may be easier if it was just done little by little.
For example: toExecRelax is now a static function. This can be separated out into a small NFC patch where the only thing that is done is the refactoring of that one static function.
Also, handleTlsRelocation returns a slightly different value (what should be skipped vs what was processed). This can also be done as a separate mini patch.

Anyway, those are my 2 cents.

Structurally it's difficult to do it piecemeal. This patch explores an alternative way handling relocations.
If we consider the current state and the exploration local optima, some changes (like moving toExecRelax outside) are actually strictly inferior (if we don't split handleTlsRelocations).

For minor details, some changes can be split and I've pushed such commits separately (e.g. moving isStaticLinkTimeConstant).

I have spent much time on this patch (some architectures like Mips and PPC64 add significant complexity) but not entirely decided this is the way to go.

[pkasting]

It's about a year later; I'm curious if you've gained any insight into whether this is a good direction or not.

[MaskRay]

In D113228#3869752, @pkasting wrote:

It's about a year later; I'm curious if you've gained any insight into whether this is a good direction or not.

I think there is value in optimizing relocation scanning. For some workload lld's pass takes 2.x time as mold's.
InputSectionBase::relocations takes some time but other checks contribute to the other slowdown.

We probably need to be more aggressive and possibly moving more relocation scanning into arch-specific ELF/Arch/*.cpp.

lld/ELF/Arch/AArch64.cpp

case R_AARCH64_ADR_PREL_PG_HI21_NC:
     return R_AARCH64_PAGE_PC;
   case R_AARCH64_LD64_GOT_LO12_NC:
     return R_GOT;
   case R_AARCH64_TLSIE_LD64_GOTTPREL_LO12_NC:
     return R_TLS_GOT;
   case R_AARCH64_LD64_GOTPAGE_LO15:
     return R_AARCH64_GOT_PAGE;
   case R_AARCH64_ADR_GOT_PAGE:
-  case R_AARCH64_TLSIE_ADR_GOTTPREL_PAGE21:
     return R_AARCH64_GOT_PAGE_PC;
+  case R_AARCH64_TLSIE_ADR_GOTTPREL_PAGE21:
+    return R_AARCH64_TLS_GOT_PAGE_PC;
   case R_AARCH64_NONE:
     return R_NONE;
   default:
     error(getErrorLocation(loc) + "unknown relocation (" + Twine(type) +
           ") against symbol " + toString(s));
     return R_NONE;
   }
 }

lld/ELF/InputSection.cpp

uint64_t InputSectionBase::getRelocTargetVA(const InputFile *file, RelType type,
   case R_RELAX_TLS_GD_TO_IE_GOTPLT:
     return sym.getGotVA() + a - in.gotPlt->getVA();
   case R_TLSLD_GOT_OFF:
   case R_GOT_OFF:
   case R_TLS_GOTOFF:
   case R_RELAX_TLS_GD_TO_IE_GOT_OFF:
     return sym.getGotOffset() + a;
   case R_AARCH64_GOT_PAGE_PC:
+  case R_AARCH64_TLS_GOT_PAGE_PC:
   case R_AARCH64_RELAX_TLS_GD_TO_IE_PAGE_PC:
     return getAArch64Page(sym.getGotVA() + a) - getAArch64Page(p);
   case R_AARCH64_GOT_PAGE:
     return sym.getGotVA() + a - getAArch64Page(in.got->getVA());
   case R_GOT_PC:
   case R_TLS_GOT_PC:
   case R_RELAX_TLS_GD_TO_IE:
     return sym.getGotVA() + a - p;

lld/ELF/Relocations.h

enum RelExpr {
   R_TLS_GOTOFF,
 
   // The following is abstract relocation types used for only one target.
   //
   // Even though RelExpr is intended to be a target-neutral representation
   // of a relocation type, there are some relocations whose semantics are
   // unique to a target. Such relocation are marked with R_<TARGET_NAME>.
   R_AARCH64_GOT_PAGE_PC,
+  R_AARCH64_TLS_GOT_PAGE_PC,
   R_AARCH64_GOT_PAGE,
   R_AARCH64_PAGE_PC,
   R_AARCH64_RELAX_TLS_GD_TO_IE_PAGE_PC,
   R_AARCH64_TLSDESC_PAGE,
   R_ARM_PCA,
   R_ARM_SBREL,
   R_MIPS_GOTREL,
   R_MIPS_GOT_GP,

lld/ELF/Relocations.cpp

 
 // True if this expression is of the form Sym - X, where X is a position in the
 // file (PC, or GOT for example).
 static bool isRelExpr(RelExpr expr) {
   return oneof<R_PC, R_GOTREL, R_GOTPLTREL, R_MIPS_GOTREL, R_PPC64_CALL,
                R_PPC64_RELAX_TOC, R_AARCH64_PAGE_PC, R_RELAX_GOT_PC,
                R_RISCV_PC_INDIRECT, R_PPC64_RELAX_GOT_PC>(expr);
 }
 
 static RelExpr toPlt(RelExpr expr) {
   switch (expr) {
   case R_PPC64_CALL:
     return R_PPC64_CALL_PLT;
   case R_PC:
     return R_PLT_PC;
   case R_ABS:
     return R_PLT;
   default:
static bool canDefineSymbolInExecutable(Symbol &sym) {
   // If we are allowed to break address equality of functions, defining
   // a plt entry will allow the program to call the function in the
   // .so, but the .so and the executable will no agree on the address
   // of the function. Similar logic for objects.
   return ((sym.isFunc() && config->ignoreFunctionAddressEquality) ||
           (sym.isObject() && config->ignoreDataAddressEquality));
 }
 
+static void invalidRelocAgainstSym(InputSectionBase &sec, uint64_t offset,
+                                   RelType type, Symbol &sym) {
+  errorOrWarn("relocation " + toString(type) + " cannot be used against " +
+              (sym.getName().empty() ? "local symbol"
+                                     : "symbol '" + toString(sym) + "'") +
+              "; recompile with -fPIC" + getLocation(sec, sym, offset));
+}
+
 // Returns true if a given relocation can be computed at link-time.
 // This only handles relocation types expected in processRelocAux.
 //
 // For instance, we know the offset from a relocation to its target at
 // link-time if the relocation is PC-relative and refers a
 // non-interposable function in the same executable. This function
 // will return true for such relocation.
 //
 // If this function returns false, that means we need to emit a
 // dynamic relocation so that the relocation will be fixed at load-time.
-static bool isStaticLinkTimeConstant(RelExpr e, RelType type, const Symbol &sym,
-                                     InputSectionBase &s, uint64_t relOff) {
-  // These expressions always compute a constant
-  if (oneof<R_GOTPLT, R_GOT_OFF, R_MIPS_GOT_LOCAL_PAGE, R_MIPS_GOTREL,
-            R_MIPS_GOT_OFF, R_MIPS_GOT_OFF32, R_MIPS_GOT_GP_PC,
-            R_AARCH64_GOT_PAGE, R_AARCH64_GOT_PAGE_PC, R_GOT_PC, R_GOTONLY_PC,
+static bool customRelocIsLinkTimeConstant(RelExpr e, RelType type,
+                                          const Symbol &sym,
+                                          InputSectionBase &s,
+                                          uint64_t relOff) {
+  // The following expressions always compute a constant. The list does not
+  // contain expressions marked RELOCATE in scanReloc.
+  if (oneof<R_TLSLD_GOT_OFF, R_AARCH64_GOT_PAGE_PC, R_GOT_PC, R_GOTONLY_PC,
             R_GOTPLTONLY_PC, R_PLT_PC, R_PLT_GOTPLT, R_PPC32_PLTREL,
-            R_PPC64_CALL_PLT, R_PPC64_RELAX_TOC, R_RISCV_ADD>(e))
+            R_PPC64_RELAX_TOC>(e))
     return true;
 
   // These never do, except if the entire file is position dependent or if
   // only the low bits are used.
-  if (e == R_GOT || e == R_PLT || e == R_TLS_GOT)
+  if (e == R_GOT || e == R_PLT)
     return target->usesOnlyLowPageBits(type) || !config->isPic;
 
   if (sym.isPreemptible)
     return false;
   if (!config->isPic)
     return true;
 
   // The size of a non preemptible symbol is a constant.
static void processRelocAux(InputSectionBase &sec, RelExpr expr, RelType type,
   // relocate{,Non}Alloc() resolve them) for -no-pie and try producing dynamic
   // relocations for -pie and -shared.
   //
   // The general expectation of -no-pie static linking is that there is no
   // dynamic relocation (except IRELATIVE). Emitting dynamic relocations for
   // -shared matches the spirit of its -z undefs default. -pie has freedom on
   // choices, and we choose dynamic relocations to be consistent with the
   // handling of GOT-generating relocations.
-  if (isStaticLinkTimeConstant(expr, type, sym, sec, offset) ||
+  if (customRelocIsLinkTimeConstant(expr, type, sym, sec, offset) ||
       (!config->isPic && sym.isUndefWeak())) {
     sec.relocations.push_back({expr, type, offset, addend, &sym});
     return;
   }
 
   bool canWrite = (sec.flags & SHF_WRITE) || !config->zText;
   if (canWrite) {
     RelType rel = target->getDynRel(type);
if (sym.isFunc()) {
         }
       }
       sym.needsPltAddr = true;
       sec.relocations.push_back({expr, type, offset, addend, &sym});
       return;
     }
   }
 
-  errorOrWarn("relocation " + toString(type) + " cannot be used against " +
-              (sym.getName().empty() ? "local symbol"
-                                     : "symbol '" + toString(sym) + "'") +
-              "; recompile with -fPIC" + getLocation(sec, sym, offset));
+  invalidRelocAgainstSym(sec, offset, type, sym);
 }
 
-// This function is similar to the `handleTlsRelocation`. MIPS does not
+// This function is similar to the `handleTlsRelocationNonIE`. MIPS does not
 // support any relaxations for TLS relocations so by factoring out MIPS
 // handling in to the separate function we can simplify the code and do not
-// pollute other `handleTlsRelocation` by MIPS `ifs` statements.
+// pollute other `handleTlsRelocationNonIE` by MIPS `ifs` statements.
 // Mips has a custom MipsGotSection that handles the writing of GOT entries
 // without dynamic relocations.
 static unsigned handleMipsTlsRelocation(RelType type, Symbol &sym,
                                         InputSectionBase &c, uint64_t offset,
                                         int64_t addend, RelExpr expr) {
   if (expr == R_MIPS_TLSLD) {
     in.mipsGot->addTlsIndex(*c.file);
     c.relocations.push_back({expr, type, offset, addend, &sym});
     return 1;
   }
   if (expr == R_MIPS_TLSGD) {
     in.mipsGot->addDynTlsEntry(*c.file, sym);
     c.relocations.push_back({expr, type, offset, addend, &sym});
     return 1;
   }
   return 0;
 }
 
+template <class ELFT>
+static void handleTlsIe(RelType type, Symbol &sym, InputSectionBase &c,
+                        typename ELFT::uint offset, int64_t addend,
+                        RelExpr expr, bool relax) {
+  // Initial-Exec relocs can be relaxed to Local-Exec if the symbol is locally
+  // defined.
+  if (relax) {
+    c.relocations.push_back({R_RELAX_TLS_IE_TO_LE, type, offset, addend, &sym});
+  } else if (expr != R_TLSIE_HINT) {
+    if (!sym.isInGot())
+      addTpOffsetGotEntry(sym);
+    // R_TLS_GOT needs a relative relocation for PIC on i386 and Hexagon.
+    if (expr == R_TLS_GOT && config->isPic &&
+        !target->usesOnlyLowPageBits(type))
+      addRelativeReloc(&c, offset, sym, addend, expr, type);
+    else
+      c.relocations.push_back({expr, type, offset, addend, &sym});
+  }
+}
+
+static inline bool toExecRelax(const InputSectionBase &sec) {
+  // ARM, Hexagon and RISC-V do not support GD/LD to IE/LE relaxation.  For
+  // PPC64, if the file has missing R_PPC64_TLSGD/R_PPC64_TLSLD, disable
+  // relaxation as well.
+  auto m = config->emachine;
+  return !config->shared && m != EM_ARM && m != EM_MIPS && m != EM_HEXAGON &&
+         m != EM_RISCV && !sec.file->ppc64DisableTLSRelax;
+}
+
 // Notes about General Dynamic and Local Dynamic TLS models below. They may
 // require the generation of a pair of GOT entries that have associated dynamic
 // relocations. The pair of GOT entries created are of the form GOT[e0] Module
 // Index (Used to find pointer to TLS block at run-time) GOT[e1] Offset of
 // symbol in TLS block.
 //
-// Returns the number of relocations processed.
+// Returns the number of subsequent relocations which should be skipped.
 template <class ELFT>
-static unsigned
-handleTlsRelocation(RelType type, Symbol &sym, InputSectionBase &c,
-                    typename ELFT::uint offset, int64_t addend, RelExpr expr) {
-  if (!sym.isTls())
-    return 0;
-
-  if (config->emachine == EM_MIPS)
-    return handleMipsTlsRelocation(type, sym, c, offset, addend, expr);
-
+static unsigned handleTlsRelocationNonIE(RelType type, Symbol &sym,
+                                         InputSectionBase &c,
+                                         typename ELFT::uint offset,
+                                         int64_t addend, RelExpr expr) {
   if (oneof<R_AARCH64_TLSDESC_PAGE, R_TLSDESC, R_TLSDESC_CALL, R_TLSDESC_PC,
             R_TLSDESC_GOTPLT>(expr) &&
       config->shared) {
     if (in.got->addDynTlsEntry(sym)) {
       uint64_t off = in.got->getGlobalDynOffset(sym);
       mainPart->relaDyn->addAddendOnlyRelocIfNonPreemptible(
           target->tlsDescRel, in.got, off, sym, target->tlsDescRel);
     }
     if (expr != R_TLSDESC_CALL)
       c.relocations.push_back({expr, type, offset, addend, &sym});
-    return 1;
+    return 0;
   }
 
-  // ARM, Hexagon and RISC-V do not support GD/LD to IE/LE relaxation.  For
-  // PPC64, if the file has missing R_PPC64_TLSGD/R_PPC64_TLSLD, disable
-  // relaxation as well.
-  bool toExecRelax = !config->shared && config->emachine != EM_ARM &&
-                     config->emachine != EM_HEXAGON &&
-                     config->emachine != EM_RISCV &&
-                     !c.file->ppc64DisableTLSRelax;
-
   // If we are producing an executable and the symbol is non-preemptable, it
   // must be defined and the code sequence can be relaxed to use Local-Exec.
   //
   // ARM and RISC-V do not support any relaxations for TLS relocations, however,
   // we can omit the DTPMOD dynamic relocations and resolve them at link time
   // because them are always 1. This may be necessary for static linking as
   // DTPMOD may not be expected at load time.
   bool isLocalInExecutable = !sym.isPreemptible && !config->shared;
 
   // Local Dynamic is for access to module local TLS variables, while still
   // being suitable for being dynamically loaded via dlopen. GOT[e0] is the
   // module index, with a special value of 0 for the current module. GOT[e1] is
   // unused. There only needs to be one module index entry.
   if (oneof<R_TLSLD_GOT, R_TLSLD_GOTPLT, R_TLSLD_PC, R_TLSLD_HINT>(
           expr)) {
     // Local-Dynamic relocs can be relaxed to Local-Exec.
-    if (toExecRelax) {
+    if (toExecRelax(c)) {
       c.relocations.push_back(
           {target->adjustTlsExpr(type, R_RELAX_TLS_LD_TO_LE), type, offset,
            addend, &sym});
-      return target->getTlsGdRelaxSkip(type);
+      return target->getTlsGdRelaxSkip(type) - 1;
     }
     if (expr == R_TLSLD_HINT)
-      return 1;
+      return 0;
     if (in.got->addTlsIndex()) {
       if (isLocalInExecutable)
         in.got->relocations.push_back(
             {R_ADDEND, target->symbolicRel, in.got->getTlsIndexOff(), 1, &sym});
       else
         mainPart->relaDyn->addReloc(
             {target->tlsModuleIndexRel, in.got, in.got->getTlsIndexOff()});
     }
     c.relocations.push_back({expr, type, offset, addend, &sym});
-    return 1;
-  }
-
-  // Local-Dynamic relocs can be relaxed to Local-Exec.
-  if (expr == R_DTPREL) {
-    if (toExecRelax)
-      expr = target->adjustTlsExpr(type, R_RELAX_TLS_LD_TO_LE);
-    c.relocations.push_back({expr, type, offset, addend, &sym});
-    return 1;
-  }
-
-  // Local-Dynamic sequence where offset of tls variable relative to dynamic
-  // thread pointer is stored in the got. This cannot be relaxed to Local-Exec.
-  if (expr == R_TLSLD_GOT_OFF) {
-    if (!sym.isInGot()) {
-      in.got->addEntry(sym);
-      uint64_t off = sym.getGotOffset();
-      in.got->relocations.push_back(
-          {R_ABS, target->tlsOffsetRel, off, 0, &sym});
-    }
-    c.relocations.push_back({expr, type, offset, addend, &sym});
-    return 1;
+    return 0;
   }
 
-  if (oneof<R_AARCH64_TLSDESC_PAGE, R_TLSDESC, R_TLSDESC_CALL, R_TLSDESC_PC,
-            R_TLSDESC_GOTPLT, R_TLSGD_GOT, R_TLSGD_GOTPLT, R_TLSGD_PC>(expr)) {
-    if (!toExecRelax) {
+  assert(
+      (oneof<R_AARCH64_TLSDESC_PAGE, R_TLSDESC, R_TLSDESC_CALL, R_TLSDESC_PC,
+             R_TLSDESC_GOTPLT, R_TLSGD_GOT, R_TLSGD_GOTPLT, R_TLSGD_PC>(expr)));
+  {
+    if (!toExecRelax(c)) {
       if (in.got->addDynTlsEntry(sym)) {
         uint64_t off = in.got->getGlobalDynOffset(sym);
 
         if (isLocalInExecutable)
           // Write one to the GOT slot.
           in.got->relocations.push_back(
               {R_ADDEND, target->symbolicRel, off, 1, &sym});
         else
           mainPart->relaDyn->addSymbolReloc(target->tlsModuleIndexRel, in.got,
                                             off, sym);
 
         // If the symbol is preemptible we need the dynamic linker to write
         // the offset too.
         uint64_t offsetOff = off + config->wordsize;
         if (sym.isPreemptible)
           mainPart->relaDyn->addSymbolReloc(target->tlsOffsetRel, in.got,
                                             offsetOff, sym);
         else
           in.got->relocations.push_back(
               {R_ABS, target->tlsOffsetRel, offsetOff, 0, &sym});
       }
       c.relocations.push_back({expr, type, offset, addend, &sym});
-      return 1;
+      return 0;
     }
 
     // Global-Dynamic relocs can be relaxed to Initial-Exec or Local-Exec
     // depending on the symbol being locally defined or not.
     if (sym.isPreemptible) {
       c.relocations.push_back(
           {target->adjustTlsExpr(type, R_RELAX_TLS_GD_TO_IE), type, offset,
            addend, &sym});
       if (!sym.isInGot()) {
         in.got->addEntry(sym);
         mainPart->relaDyn->addSymbolReloc(target->tlsGotRel, in.got,
                                           sym.getGotOffset(), sym);
       }
     } else {
       c.relocations.push_back(
           {target->adjustTlsExpr(type, R_RELAX_TLS_GD_TO_LE), type, offset,
            addend, &sym});
     }
-    return target->getTlsGdRelaxSkip(type);
-  }
-
-  if (oneof<R_TLS_GOT, R_TLS_GOT_GOTPLT, R_TLS_GOT_PC, R_AARCH64_GOT_PAGE_PC,
-            R_TLS_GOTOFF, R_TLSIE_HINT>(expr)) {
-    // Initial-Exec relocs can be relaxed to Local-Exec if the symbol is locally
-    // defined.
-    if (toExecRelax && isLocalInExecutable) {
-      c.relocations.push_back(
-          {R_RELAX_TLS_IE_TO_LE, type, offset, addend, &sym});
-    } else if (expr != R_TLSIE_HINT) {
-      if (!sym.isInGot())
-        addTpOffsetGotEntry(sym);
-      // R_TLS_GOT needs a relative relocation for PIC on i386 and Hexagon.
-      if (expr == R_TLS_GOT && config->isPic &&
-          !target->usesOnlyLowPageBits(type))
-        addRelativeReloc(&c, offset, sym, addend, expr, type);
-      else
-        c.relocations.push_back({expr, type, offset, addend, &sym});
-    }
-    return 1;
+    return target->getTlsGdRelaxSkip(type) - 1;
   }
 
   return 0;
 }
 
 template <class ELFT, class RelTy>
 static void scanReloc(InputSectionBase &sec, OffsetGetter &getOffset, RelTy *&i,
                       RelTy *start, RelTy *end) {
   const RelTy &rel = *i;
   uint32_t symIndex = rel.getSymbol(config->isMips64EL);
   Symbol &sym = sec.getFile<ELFT>()->getSymbol(symIndex);
static void scanReloc(InputSectionBase &sec, OffsetGetter &getOffset, RelTy *&i,
   // Error if the target symbol is undefined. Symbol index 0 may be used by
   // marker relocations, e.g. R_*_NONE and R_ARM_V4BX. Don't error on them.
   if (symIndex != 0 && maybeReportUndefined(sym, sec, rel.r_offset))
     return;
 
   const uint8_t *relocatedAddr = sec.data().begin() + rel.r_offset;
   RelExpr expr = target->getRelExpr(type, sym, relocatedAddr);
 
-  // Ignore R_*_NONE and other marker relocations.
-  if (expr == R_NONE)
-    return;
-
   // Read an addend.
   int64_t addend = computeAddend<ELFT>(rel, end, sec, expr, sym.isLocal());
 
-  if (config->emachine == EM_PPC64) {
-    // We can separate the small code model relocations into 2 categories:
-    // 1) Those that access the compiler generated .toc sections.
-    // 2) Those that access the linker allocated got entries.
-    // lld allocates got entries to symbols on demand. Since we don't try to
-    // sort the got entries in any way, we don't have to track which objects
-    // have got-based small code model relocs. The .toc sections get placed
-    // after the end of the linker allocated .got section and we do sort those
-    // so sections addressed with small code model relocations come first.
-    if (type == R_PPC64_TOC16 || type == R_PPC64_TOC16_DS)
-      sec.file->ppc64SmallCodeModelTocRelocs = true;
-
-    // Record the TOC entry (.toc + addend) as not relaxable. See the comment in
-    // InputSectionBase::relocateAlloc().
-    if (type == R_PPC64_TOC16_LO && sym.isSection() && isa<Defined>(sym) &&
-        cast<Defined>(sym).section->name == ".toc")
-      ppc64noTocRelax.insert({&sym, addend});
-
-    if ((type == R_PPC64_TLSGD && expr == R_TLSDESC_CALL) ||
-        (type == R_PPC64_TLSLD && expr == R_TLSLD_HINT)) {
-      if (i == end) {
-        errorOrWarn("R_PPC64_TLSGD/R_PPC64_TLSLD may not be the last "
-                    "relocation" +
-                    getLocation(sec, sym, offset));
-        return;
-      }
-
-      // Offset the 4-byte aligned R_PPC64_TLSGD by one byte in the NOTOC case,
-      // so we can discern it later from the toc-case.
-      if (i->getType(/*isMips64EL=*/false) == R_PPC64_REL24_NOTOC)
-        ++offset;
-    }
-  }
-
-  // Relax relocations.
-  //
-  // If we know that a PLT entry will be resolved within the same ELF module, we
-  // can skip PLT access and directly jump to the destination function. For
-  // example, if we are linking a main executable, all dynamic symbols that can
-  // be resolved within the executable will actually be resolved that way at
-  // runtime, because the main executable is always at the beginning of a search
-  // list. We can leverage that fact.
-  if (!sym.isPreemptible && (!sym.isGnuIFunc() || config->zIfuncNoplt)) {
-    if (expr != R_GOT_PC) {
-      // The 0x8000 bit of r_addend of R_PPC_PLTREL24 is used to choose call
-      // stub type. It should be ignored if optimized to R_PC.
-      if (config->emachine == EM_PPC && expr == R_PPC32_PLTREL)
+  if (sym.isGnuIFunc()) {
+    if (config->zIfuncNoplt) {
+      if (!sym.isPreemptible) {
+        if (expr == R_PPC32_PLTREL)
           addend &= ~0x8000;
-      // R_HEX_GD_PLT_B22_PCREL (call a@GDPLT) is transformed into
-      // call __tls_get_addr even if the symbol is non-preemptible.
-      if (!(config->emachine == EM_HEXAGON &&
-           (type == R_HEX_GD_PLT_B22_PCREL ||
-            type == R_HEX_GD_PLT_B22_PCREL_X ||
-            type == R_HEX_GD_PLT_B32_PCREL_X)))
         expr = fromPlt(expr);
-    } else if (!isAbsoluteValue(sym)) {
-      expr = target->adjustGotPcExpr(type, addend, relocatedAddr);
-    }
-  }
-
-  // If the relocation does not emit a GOT or GOTPLT entry but its computation
-  // uses their addresses, we need GOT or GOTPLT to be created.
-  //
-  // The 5 types that relative GOTPLT are all x86 and x86-64 specific.
-  if (oneof<R_GOTPLTONLY_PC, R_GOTPLTREL, R_GOTPLT, R_PLT_GOTPLT,
-            R_TLSDESC_GOTPLT, R_TLSGD_GOTPLT, R_TLS_GOT_GOTPLT>(expr)) {
-    in.gotPlt->hasGotPltOffRel = true;
-  } else if (oneof<R_GOTONLY_PC, R_GOTREL, R_PPC64_TOCBASE, R_PPC64_RELAX_TOC>(
-                 expr)) {
-    in.got->hasGotOffRel = true;
       }
-
-  // Process TLS relocations, including relaxing TLS relocations. Note that
-  // R_TPREL and R_TPREL_NEG relocations are resolved in processRelocAux.
-  if (expr == R_TPREL || expr == R_TPREL_NEG) {
-    if (config->shared) {
-      errorOrWarn("relocation " + toString(type) + " against " + toString(sym) +
-                  " cannot be used with -shared" +
-                  getLocation(sec, sym, offset));
-      return;
-    }
-  } else if (unsigned processed = handleTlsRelocation<ELFT>(
-                 type, sym, sec, offset, addend, expr)) {
-    i += (processed - 1);
-    return;
-  }
-
-  // We were asked not to generate PLT entries for ifuncs. Instead, pass the
-  // direct relocation on through.
-  if (sym.isGnuIFunc() && config->zIfuncNoplt) {
+      // TODO adjustGotPcExpr
       sym.exportDynamic = true;
       mainPart->relaDyn->addSymbolReloc(type, &sec, offset, sym, addend, type);
       return;
     }
 
-  // Non-preemptible ifuncs require special handling. First, handle the usual
-  // case where the symbol isn't one of these.
-  if (!sym.isGnuIFunc() || sym.isPreemptible) {
-    // If a relocation needs PLT, we create PLT and GOTPLT slots for the symbol.
-    if (needsPlt(expr) && !sym.isInPlt())
-      addPltEntry(in.plt, in.gotPlt, in.relaPlt, target->pltRel, sym);
-
-    // Create a GOT slot if a relocation needs GOT.
-    if (needsGot(expr)) {
-      if (config->emachine == EM_MIPS) {
-        // MIPS ABI has special rules to process GOT entries and doesn't
-        // require relocation entries for them. A special case is TLS
-        // relocations. In that case dynamic loader applies dynamic
-        // relocations to initialize TLS GOT entries.
-        // See "Global Offset Table" in Chapter 5 in the following document
-        // for detailed description:
-        // ftp://www.linux-mips.org/pub/linux/mips/doc/ABI/mipsabi.pdf
-        in.mipsGot->addEntry(*sec.file, sym, addend, expr);
-      } else if (!sym.isInGot()) {
-        addGotEntry(sym);
-      }
-    }
-  } else {
     // Handle a reference to a non-preemptible ifunc. These are special in a
     // few ways:
     //
     // - Unlike most non-preemptible symbols, non-preemptible ifuncs do not have
     //   a fixed value. But assuming that all references to the ifunc are
     //   GOT-generating or PLT-generating, the handling of an ifunc is
     //   relatively straightforward. We create a PLT entry in Iplt, which is
     //   usually at the end of .plt, which makes an indirect call using a
     //   matching GOT entry in igotPlt, which is usually at the end of .got.plt.
if (sym.isGnuIFunc()) {
     //   GOT-generating relocations.
     //
     // - The fact that these symbols do not have a fixed value makes them an
     //   exception to the general rule that a statically linked executable does
     //   not require any form of dynamic relocation. To handle these relocations
     //   correctly, the IRELATIVE relocations are stored in an array which a
     //   statically linked executable's startup code must enumerate using the
     //   linker-defined symbols __rela?_iplt_{start,end}.
+    if (!sym.isPreemptible) {
       if (!sym.isInPlt()) {
         // Create PLT and GOTPLT slots for the symbol.
         sym.isInIplt = true;
 
         // Create a copy of the symbol to use as the target of the IRELATIVE
         // relocation in the igotPlt. This is in case we make the PLT canonical
         // later, which would overwrite the original symbol.
         //
         // FIXME: Creating a copy of the symbol here is a bit of a hack. All
-      // that's really needed to create the IRELATIVE is the section and value,
-      // so ideally we should just need to copy those.
+        // that's really needed to create the IRELATIVE is the section and
+        // value, so ideally we should just need to copy those.
         auto *directSym = make<Defined>(cast<Defined>(sym));
         addPltEntry(in.iplt, in.igotPlt, in.relaIplt, target->iRelativeRel,
                     *directSym);
         sym.pltIndex = directSym->pltIndex;
       }
       if (needsGot(expr)) {
         // Redirect GOT accesses to point to the Igot.
         //
         // This field is also used to keep track of whether we ever needed a GOT
         // entry. If we did and we make the PLT canonical later, we'll need to
         // create a GOT entry pointing to the PLT entry for Sym.
         sym.gotInIgot = true;
       } else if (!needsPlt(expr)) {
         // Make the ifunc's PLT entry canonical by changing the value of its
         // symbol to redirect all references to point to it.
         auto &d = cast<Defined>(sym);
         d.section = in.iplt;
         d.value = sym.pltIndex * target->ipltEntrySize;
         d.size = 0;
         // It's important to set the symbol type here so that dynamic loaders
         // don't try to call the PLT as if it were an ifunc resolver.
         d.type = STT_FUNC;
 
         if (sym.gotInIgot) {
           // We previously encountered a GOT generating reference that we
           // redirected to the Igot. Now that the PLT entry is canonical we must
           // clear the redirection to the Igot and add a GOT entry. As we've
-        // changed the symbol type to STT_FUNC future GOT generating references
-        // will naturally use this GOT entry.
+          // changed the symbol type to STT_FUNC future GOT generating
+          // references will naturally use this GOT entry.
           //
-        // We don't need to worry about creating a MIPS GOT here because ifuncs
-        // aren't a thing on MIPS.
+          // We don't need to worry about creating a MIPS GOT here because
+          // ifuncs aren't a thing on MIPS.
           sym.gotInIgot = false;
           addGotEntry(sym);
         }
       }
+      processRelocAux<ELFT>(sec, expr, type, offset, sym, addend);
+      return;
+    }
   }
 
+#define OPTIMIZE_PLT (!sym.isPreemptible && !sym.isGnuIFunc())
+
+  enum Action { RELOCATE, IGNORE, CUSTOM };
+  Action action = RELOCATE;
+  switch (expr) {
+  case R_NONE:
+    action = IGNORE;
+    break;
+
+  case R_ABS:
+  case R_SIZE:
+  case R_PC:
+  case R_AARCH64_PAGE_PC:
+    action = CUSTOM;
+    break;
+
+    //*** GOT
+  case R_GOT_PC:
+    if (!sym.isPreemptible && !isAbsoluteValue(sym))
+      expr = target->adjustGotPcExpr(type, addend, relocatedAddr);
+    if (expr != R_GOT_PC)
+      action = CUSTOM;
+    else if (!sym.isInGot())
+      addGotEntry(sym);
+    break;
+
+  case R_GOT:
+    if (!sym.isInGot())
+      addGotEntry(sym);
+    action = CUSTOM;
+    break;
+  case R_GOT_OFF:
+  case R_GOTPLT:
+  case R_AARCH64_GOT_PAGE_PC:
+  case R_AARCH64_GOT_PAGE:
+    if (expr == R_GOTPLT)
+      in.gotPlt->hasGotPltOffRel = true;
+    if (!sym.isInGot())
+      addGotEntry(sym);
+    break;
+
+  case R_GOTONLY_PC:
+    in.got->hasGotOffRel = true;
+    break;
+  case R_PPC64_TOCBASE:
+    in.got->hasGotOffRel = true;
+    action = CUSTOM;
+    break;
+  case R_GOTREL:
+    if (config->emachine == EM_PPC64) {
+      // We can separate the small code model relocations into 2 categories:
+      // 1) Those that access the compiler generated .toc sections.
+      // 2) Those that access the linker allocated got entries.
+      // lld allocates got entries to symbols on demand. Since we don't try to
+      // sort the got entries in any way, we don't have to track which objects
+      // have got-based small code model relocs. The .toc sections get placed
+      // after the end of the linker allocated .got section and we do sort those
+      // so sections addressed with small code model relocations come first.
+      if (type == R_PPC64_TOC16 || type == R_PPC64_TOC16_DS)
+        sec.file->ppc64SmallCodeModelTocRelocs = true;
+      // Record the TOC entry (.toc + addend) as not relaxable. See the comment
+      // in InputSectionBase::relocateAlloc().
+      if (type == R_PPC64_TOC16_LO && sym.isSection() && isa<Defined>(sym) &&
+          cast<Defined>(sym).section->name == ".toc")
+        ppc64noTocRelax.insert({&sym, addend});
+    }
+    in.got->hasGotOffRel = true;
+    break;
+  case R_PPC64_RELAX_TOC:
+    in.got->hasGotOffRel = true;
+    break;
+
+  case R_MIPS_GOT_LOCAL_PAGE:
+  case R_MIPS_GOT_OFF:
+  case R_MIPS_GOT_OFF32:
+    // MIPS ABI has special rules to process GOT entries and doesn't
+    // require relocation entries for them. A special case is TLS
+    // relocations. In that case dynamic loader applies dynamic
+    // relocations to initialize TLS GOT entries.
+    // See "Global Offset Table" in Chapter 5 in the following document
+    // for detailed description:
+    // ftp://www.linux-mips.org/pub/linux/mips/doc/ABI/mipsabi.pdf
+    in.mipsGot->addEntry(*sec.file, sym, addend, expr);
+    break;
+
+    //*** .got.plt
+  case R_GOTPLTONLY_PC:
+  case R_GOTPLTREL:
+    in.gotPlt->hasGotPltOffRel = true;
+    break;
+
+    //*** PLT
+  case R_PLT:
+  case R_PLT_PC:
+  case R_PLT_GOTPLT:
+  case R_PPC64_CALL_PLT:
+    if (expr == R_PLT_GOTPLT)
+      in.gotPlt->hasGotPltOffRel = true;
+    if (OPTIMIZE_PLT) {
+      if (config->emachine == EM_HEXAGON &&
+          (type == R_HEX_GD_PLT_B22_PCREL || type == R_HEX_GD_PLT_B22_PCREL_X ||
+           type == R_HEX_GD_PLT_B32_PCREL_X)) {
+        if (!sym.isInPlt())
+          addPltEntry(in.plt, in.gotPlt, in.relaPlt, target->pltRel, sym);
+      } else {
+        expr = fromPlt(expr);
+        action = CUSTOM; // TODO R_PC vs SHN_ABS
+      }
+    } else {
+      if (!sym.isInPlt())
+        addPltEntry(in.plt, in.gotPlt, in.relaPlt, target->pltRel, sym);
+    }
+    break;
+  case R_PPC32_PLTREL:
+    if (OPTIMIZE_PLT) {
+      expr = R_PC;
+      addend &= ~0x8000;
+    } else if (!sym.isInPlt()) {
+      addPltEntry(in.plt, in.gotPlt, in.relaPlt, target->pltRel, sym);
+    }
+    break;
+
+    //*** TPREL
+  case R_TPREL:
+  case R_TPREL_NEG:
+    if (config->shared) {
+      errorOrWarn("relocation " + toString(type) + " against " + toString(sym) +
+                  " cannot be used with -shared" +
+                  getLocation(sec, sym, offset));
+      return;
+    }
+    if (!sym.isDefined())
+      action = CUSTOM;
+    break;
+
+    //*** TLS (non-TPREL)
+  case R_TLSDESC:
+  case R_TLSDESC_PC:
+  case R_AARCH64_TLSDESC_PAGE:
+  case R_TLSGD_GOT:
+  case R_TLSGD_PC:
+  case R_TLSLD_GOT:
+  case R_TLSLD_GOTPLT:
+  case R_TLSLD_PC:
+  case R_MIPS_TLSGD:
+  case R_MIPS_TLSLD:
+    if (config->emachine == EM_MIPS)
+      handleMipsTlsRelocation(type, sym, sec, offset, addend, expr);
+    else
+      i += handleTlsRelocationNonIE<ELFT>(type, sym, sec, offset, addend, expr);
+    return;
+  case R_DTPREL:
+    if (toExecRelax(sec))
+      expr = target->adjustTlsExpr(type, R_RELAX_TLS_LD_TO_LE);
+    sec.relocations.push_back({expr, type, offset, addend, &sym});
+    return;
+  case R_TLSDESC_CALL:
+  case R_TLSLD_HINT:
+    if (config->emachine == EM_PPC64) {
+      if (i == end) {
+        errorOrWarn("R_PPC64_TLSGD/R_PPC64_TLSLD may not be the last "
+                    "relocation" +
+                    getLocation(sec, sym, offset));
+        return;
+      }
+
+      // Offset the 4-byte aligned R_PPC64_TLSGD by one byte in the NOTOC case,
+      // so we can discern it later from the toc-case.
+      if (i->getType(false) == R_PPC64_REL24_NOTOC)
+        ++offset;
+    }
+    i += handleTlsRelocationNonIE<ELFT>(type, sym, sec, offset, addend, expr);
+    return;
+
+  case R_TLSDESC_GOTPLT:
+  case R_TLSGD_GOTPLT:
+    in.gotPlt->hasGotPltOffRel = true;
+    i += handleTlsRelocationNonIE<ELFT>(type, sym, sec, offset, addend, expr);
+    return;
+
+  case R_TLSLD_GOT_OFF:
+    // PPC64 Local-Dynamic sequence where offset of tls variable relative to
+    // dynamic thread pointer is stored in the got. This cannot be relaxed to
+    // Local-Exec.
+    if (!sym.isInGot()) {
+      in.got->addEntry(sym);
+      uint64_t off = sym.getGotOffset();
+      in.got->relocations.push_back(
+          {R_ABS, target->tlsOffsetRel, off, 0, &sym});
+    }
+    break;
+
+  case R_TLS_GOT:
+  case R_TLS_GOT_PC:
+  case R_TLS_GOT_GOTPLT:
+  case R_TLSIE_HINT:
+  case R_AARCH64_TLS_GOT_PAGE_PC:
+  case R_TLS_GOTOFF: {
+    if (expr == R_TLS_GOT_GOTPLT)
+      in.gotPlt->hasGotPltOffRel = true;
+    handleTlsIe<ELFT>(type, sym, sec, offset, addend, expr,
+                      toExecRelax(sec) && !sym.isPreemptible);
+    return;
+  }
+
+  default:
+    break;
+  }
+
+  if (action == RELOCATE)
+    sec.relocations.push_back({expr, type, offset, addend, &sym});
+  else if (action != IGNORE)
     processRelocAux<ELFT>(sec, expr, type, offset, sym, addend);
 }
 
 // R_PPC64_TLSGD/R_PPC64_TLSLD is required to mark `bl __tls_get_addr` for
 // General Dynamic/Local Dynamic code sequences. If a GD/LD GOT relocation is
 // found but no R_PPC64_TLSGD/R_PPC64_TLSLD is seen, we assume that the
 // instructions are generated by very old IBM XL compilers. Work around the
 // issue by disabling GD/LD to IE/LE relaxation.
 template <class RelTy>