[LLD][ELF] Implement --[no-]apply-dynamic-relocs option.

[peter.smith]

Summary

When resolving dynamic RELA relocations the addend is taken from the relocation and not the place being relocated. Accordingly lld does not write the addend field to the place like it would for a REL relocation. Unfortunately there is some system software, in particlar dynamic loaders such as Bionic's linker64 that use the value of the place prior to
relocation to find the offset that they have been loaded at. Both gold and bfd control this behavior with the --[no-]apply-dynamic-relocs option.

This change implements the option and defaults it to true for compatibility with gold and bfd. I've chosen the same default (--apply-dynamic-relocs) but I don't have a strong opinion.

I found this out the hard way while trying to see if LLD could be used as the linker for Android AOSP. The resulting image wouldn't boot with LLD linking the Bionic dynamic loader linker64. It turns out that the Bionic loader makes use of the addend in the place prior to relocation:

In Bionic linker_main.cpp (https://android.googlesource.com/platform/bionic/+/master/linker/linker_main.cpp)

extern "C" ElfW(Addr) __linker_init(void* raw_args) {
  KernelArgumentBlock args(raw_args);

  // AT_BASE is set to 0 in the case when linker is run by iself
  // so in order to link the linker it needs to calcuate AT_BASE
  // using information at hand. The trick below takes advantage
  // of the fact that the value of linktime_addr before relocations
  // are run is an offset and this can be used to calculate AT_BASE.
  static uintptr_t linktime_addr = reinterpret_cast<uintptr_t>(&linktime_addr);
  ElfW(Addr) linker_addr = reinterpret_cast<uintptr_t>(&linktime_addr) - linktime_addr;

This results in a R*RELATIVE relocation on linktime_addr with the addend field on RELA containing the address of linktime_addr. If 0 is written to the place this calculation goes wrong and the dynamic linker fails to relocate itself. I've raised https://issuetracker.google.com/issues/72789859 on Bionic to see if there is an alternative mechanism that they can use.

Both gold and recent versions of BFD (Binutils 2.26 https://sourceware.org/ml/binutils/2016-04/msg00407.html) write the addend into the place. Due to another bug in Bionic https://android-review.googlesource.com/c/platform/bionic/+/176902 there is an option --[no-]apply-dynamic-relocs that controls whether the addends are written to the place.

For reference it seems like glibc on aarch64 uses a similar trick, but relies on a different convention.

/* Return the link-time address of _DYNAMIC.  Conveniently, this is the
   first element of the GOT. */
static inline ElfW(Addr) __attribute__ ((unused))
elf_machine_dynamic (void)
{
  extern const ElfW(Addr) _GLOBAL_OFFSET_TABLE_[] attribute_hidden;
  return _GLOBAL_OFFSET_TABLE_[0];
}

/* Return the run-time load address of the shared object.  */

static inline ElfW(Addr) __attribute__ ((unused))
elf_machine_load_address (void)
{
  /* To figure out the load address we use the definition that for any symbol:
     dynamic_addr(symbol) = static_addr(symbol) + load_addr

    _DYNAMIC sysmbol is used here as its link-time address stored in
    the special unrelocated first GOT entry.  */

    extern ElfW(Dyn) _DYNAMIC[] attribute_hidden;
    return (ElfW(Addr)) &_DYNAMIC - elf_machine_dynamic ();
}

[ruiu]

ELF/Options.td

52

Remove the full stop.

179

Ditto

ELF/SyntheticSections.cpp

1209–1215

Can you rewrite entire comment to say that we basically store addends both to RELA relocations and to the locations where the relocations are applied, even though it is redundant, because that's what GNU linkers do, and some dynamic linkers such as Bionic's depend on it.

[grimar]

test/ELF/dynamic-got-rela.s

6

Seems it should be:
llvm-readobj -r -s -l -section-data %t2.so

With corresponding change to NOAPPLYDYNREL block below.

[peter.smith]

Thanks to both for the comments. I've incorporated them in the diff.

[ruiu]

LGTM

ELF/SyntheticSections.cpp

1211

I'd prefer to add "even though it's redundant" because it'd be the first thing to wonder when someone read this comment.

ELF/Config.h

struct Configuration {
   std::vector<llvm::StringRef> SymbolOrderingFile;
   std::vector<llvm::StringRef> Undefined;
   std::vector<SymbolVersion> DynamicList;
   std::vector<SymbolVersion> VersionScriptGlobals;
   std::vector<SymbolVersion> VersionScriptLocals;
   std::vector<uint8_t> BuildIdVector;
   bool AllowMultipleDefinition;
   bool AndroidPackDynRelocs = false;
+  bool ApplyDynamicRelocs;
   bool ARMHasBlx = false;
   bool ARMHasMovtMovw = false;
   bool ARMJ1J2BranchEncoding = false;
   bool AsNeeded = false;
   bool Bsymbolic;
   bool BsymbolicFunctions;
   bool CompressDebugSections;
   bool DefineCommon;

ELF/Driver.cpp

error(Arg->getSpelling() + "=" + Arg->getValue() +
           ": number expected, but got '" + S + "'");
   return V;
 }
 
 // Initializes Config members by the command line options.
 void LinkerDriver::readConfigs(opt::InputArgList &Args) {
   Config->AllowMultipleDefinition =
       Args.hasArg(OPT_allow_multiple_definition) || hasZOption(Args, "muldefs");
+  Config->ApplyDynamicRelocs = Args.hasFlag(OPT_apply_dynamic_relocs,
+                                            OPT_no_apply_dynamic_relocs, true);
   Config->AuxiliaryList = args::getStrings(Args, OPT_auxiliary);
   Config->Bsymbolic = Args.hasArg(OPT_Bsymbolic);
   Config->BsymbolicFunctions = Args.hasArg(OPT_Bsymbolic_functions);
   Config->Chroot = Args.getLastArgValue(OPT_chroot);
   Config->CompressDebugSections = getCompressDebugSections(Args);
   Config->DefineCommon = Args.hasFlag(OPT_define_common, OPT_no_define_common,
                                       !Args.hasArg(OPT_relocatable));
   Config->Demangle = Args.hasFlag(OPT_demangle, OPT_no_demangle, true);

ELF/Options.td

defm Tdata: Eq<"Tdata">,
   HelpText<"Same as --section-start with .data as the sectionname">;
 
 defm Ttext: Eq<"Ttext">,
   HelpText<"Same as --section-start with .text as the sectionname">;
 
 def allow_multiple_definition: F<"allow-multiple-definition">,
   HelpText<"Allow multiple definitions">;
 
+def apply_dynamic_relocs: F<"apply-dynamic-relocs">,
+  HelpText<"Apply dynamic relocations to place">;
+
 def as_needed: F<"as-needed">,
   HelpText<"Only set DT_NEEDED for shared libraries if used">;
 
 // -chroot doesn't have a help text because it is an internal option.
 def chroot: S<"chroot">;
 
 def color_diagnostics: F<"color-diagnostics">,
   HelpText<"Use colors in diagnostics">;

 defm Map: Eq<"Map">, HelpText<"Print a link map to the specified file">;
 
 def merge_exidx_entries: F<"merge-exidx-entries">,
   HelpText<"Enable merging .ARM.exidx entries">;
 
 def nostdlib: F<"nostdlib">,
   HelpText<"Only search directories specified on the command line">;
 
+def no_apply_dynamic_relocs: F<"no-apply-dynamic-relocs">,
+  HelpText<"Do not apply dynamic relocations to place">;
+
 def no_as_needed: F<"no-as-needed">,
   HelpText<"Always DT_NEEDED for shared libraries">;
 
 def no_color_diagnostics: F<"no-color-diagnostics">,
   HelpText<"Do not use colors in diagnostics">;
 
 def no_define_common: F<"no-define-common">,
   HelpText<"Do not assign space to common symbols">;

ELF/SyntheticSections.cpp

RelocationBaseSection::RelocationBaseSection(StringRef Name, uint32_t Type,
                                              int32_t SizeDynamicTag)
     : SyntheticSection(SHF_ALLOC, Type, Config->Wordsize, Name),
       DynamicTag(DynamicTag), SizeDynamicTag(SizeDynamicTag) {}
 
 void RelocationBaseSection::addReloc(uint32_t DynType,
                                      InputSectionBase *InputSec,
                                      uint64_t OffsetInSec, bool UseSymVA,
                                      Symbol *Sym, int64_t Addend, RelExpr Expr,
                                      RelType Type) {
-  // REL type relocations don't have addend fields unlike RELAs, and
-  // their addends are stored to the section to which they are applied.
-  // So, store addends if we need to.
-  if (!Config->IsRela && UseSymVA)
+  // We store the addends for dynamic relocations for both REL and RELA
+  // relocations for compatibility with GNU Linkers. There is some system
+  // software such as the Bionic dynamic linker that uses the addend prior
+  // to dynamic relocation resolution.
+  if ((!Config->IsRela || Config->ApplyDynamicRelocs) && UseSymVA)
     InputSec->Relocations.push_back({Expr, Type, OffsetInSec, Addend, Sym});
   addReloc({DynType, InputSec, OffsetInSec, UseSymVA, Sym, Addend});
 }
 
 void RelocationBaseSection::addReloc(const DynamicReloc &Reloc) {
   if (Reloc.Type == Target->RelativeRel)
     ++NumRelativeRelocs;
   Relocs.push_back(Reloc);
 }

test/ELF/dynamic-got-rela.s

 // REQUIRES: x86
 // RUN: llvm-mc -filetype=obj -triple=x86_64-pc-linux %s -o %t.o
 // RUN: ld.lld %t.o -o %t.so -shared
-// RUN: llvm-readobj -r -s -l -section-data %t.so | FileCheck %s
+// RUN: llvm-readobj -r -s -l -section-data %t.so | FileCheck -check-prefix CHECK -check-prefix APPLYDYNREL %s
+// RUN: ld.lld %t.o -o %t2.so -shared --no-apply-dynamic-relocs
+// RUN: llvm-readobj -r -s -l -section-data %t2.so | FileCheck -check-prefix CHECK -check-prefix NOAPPLYDYNREL %s
 
-// CHECK:      Name: .got
-// CHECK-NEXT: Type: SHT_PROGBITS
-// CHECK-NEXT: Flags [
-// CHECK-NEXT:   SHF_ALLOC
-// CHECK-NEXT:   SHF_WRITE
-// CHECK-NEXT: ]
-// CHECK-NEXT: Address: 0x[[GOT:.*]]
-// CHECK-NEXT: Offset:
-// CHECK-NEXT: Size:
-// CHECK-NEXT: Link:
-// CHECK-NEXT: Info:
-// CHECK-NEXT: AddressAlignment:
-// CHECK-NEXT: EntrySize:
-// CHECK-NEXT: SectionData (
-// CHECK-NEXT:   0000: 00000000 00000000                |
-// CHECK-NEXT: )
+// APPLYDYNREL:      Name: .got
+// APPLYDYNREL-NEXT: Type: SHT_PROGBITS
+// APPLYDYNREL-NEXT: Flags [
+// APPLYDYNREL-NEXT:   SHF_ALLOC
+// APPLYDYNREL-NEXT:   SHF_WRITE
+// APPLYDYNREL-NEXT: ]
+// APPLYDYNREL-NEXT: Address: 0x[[GOT:.*]]
+// APPLYDYNREL-NEXT: Offset:
+// APPLYDYNREL-NEXT: Size:
+// APPLYDYNREL-NEXT: Link:
+// APPLYDYNREL-NEXT: Info:
+// APPLYDYNREL-NEXT: AddressAlignment:
+// APPLYDYNREL-NEXT: EntrySize:
+// APPLYDYNREL-NEXT: SectionData (
+// APPLYDYNREL-NEXT:   0000: 00200000 00000000                |
+// APPLYDYNREL-NEXT: )
+
+// NOAPPLYDYNREL:      Name: .got
+// NOAPPLYDYNREL-NEXT: Type: SHT_PROGBITS
+// NOAPPLYDYNREL-NEXT: Flags [
+// NOAPPLYDYNREL-NEXT:   SHF_ALLOC
+// NOAPPLYDYNREL-NEXT:   SHF_WRITE
+// NOAPPLYDYNREL-NEXT: ]
+// NOAPPLYDYNREL-NEXT: Address: 0x[[GOT:.*]]
+// NOAPPLYDYNREL-NEXT: Offset:
+// NOAPPLYDYNREL-NEXT: Size:
+// NOAPPLYDYNREL-NEXT: Link:
+// NOAPPLYDYNREL-NEXT: Info:
+// NOAPPLYDYNREL-NEXT: AddressAlignment:
+// NOAPPLYDYNREL-NEXT: EntrySize:
+// NOAPPLYDYNREL-NEXT: SectionData (
+// NOAPPLYDYNREL-NEXT:   0000: 00000000 00000000                |
+// NOAPPLYDYNREL-NEXT: )
 
 // CHECK:      Relocations [
 // CHECK-NEXT:   Section ({{.*}}) .rela.dyn {
 // CHECK-NEXT:     0x[[GOT]] R_X86_64_RELATIVE - 0x[[ADDEND:.*]]
 // CHECK-NEXT:   }
 // CHECK-NEXT: ]
 
 // CHECK:      Type: PT_DYNAMIC
 // CHECK-NEXT: Offset: 0x[[ADDEND]]
 // CHECK-NEXT: VirtualAddress: 0x[[ADDEND]]
 // CHECK-NEXT: PhysicalAddress: 0x[[ADDEND]]
 
 cmpq    $0, _DYNAMIC@GOTPCREL(%rip)

test/ELF/relocation-non-alloc.s

 // REQUIRES: x86
 
 // RUN: llvm-mc -filetype=obj -triple=x86_64-pc-linux %s -o %t
 // RUN: ld.lld %t -o %t2 -shared
-// RUN: llvm-readobj -s -section-data -r %t2 | FileCheck %s
+// RUN: llvm-readobj -s -section-data -r %t2 | FileCheck -check-prefix CHECK -check-prefix APPLYDYNREL %s
 
-// CHECK:      Name: .data
-// CHECK-NEXT: Type: SHT_PROGBITS
-// CHECK-NEXT: Flags [
-// CHECK-NEXT:   SHF_ALLOC
-// CHECK-NEXT:   SHF_WRITE
-// CHECK-NEXT: ]
-// CHECK-NEXT: Address: 0x1000
-// CHECK-NEXT: Offset: 0x1000
-// CHECK-NEXT: Size: 16
-// CHECK-NEXT: Link: 0
-// CHECK-NEXT: Info: 0
-// CHECK-NEXT: AddressAlignment: 1
-// CHECK-NEXT: EntrySize: 0
-// CHECK-NEXT: SectionData (
-// CHECK-NEXT:   0000: 00000000 00000000 00000000 00000000
-// CHECK-NEXT: )
+// RUN: ld.lld %t -o %t2 -shared --no-apply-dynamic-relocs
+// RUN: llvm-readobj -s -section-data -r %t2 | FileCheck -check-prefix CHECK -check-prefix NOAPPLYDYNREL %s
+
+// APPLYDYNREL:      Name: .data
+// APPLYDYNREL-NEXT: Type: SHT_PROGBITS
+// APPLYDYNREL-NEXT: Flags [
+// APPLYDYNREL-NEXT:   SHF_ALLOC
+// APPLYDYNREL-NEXT:   SHF_WRITE
+// APPLYDYNREL-NEXT: ]
+// APPLYDYNREL-NEXT: Address: 0x1000
+// APPLYDYNREL-NEXT: Offset: 0x1000
+// APPLYDYNREL-NEXT: Size: 16
+// APPLYDYNREL-NEXT: Link: 0
+// APPLYDYNREL-NEXT: Info: 0
+// APPLYDYNREL-NEXT: AddressAlignment: 1
+// APPLYDYNREL-NEXT: EntrySize: 0
+// APPLYDYNREL-NEXT: SectionData (
+// APPLYDYNREL-NEXT:   0000: 00100000 00000000 00000000 00000000
+// APPLYDYNREL-NEXT: )
+
+// NOAPPLYDYNREL:      Name: .data
+// NOAPPLYDYNREL-NEXT: Type: SHT_PROGBITS
+// NOAPPLYDYNREL-NEXT: Flags [
+// NOAPPLYDYNREL-NEXT:   SHF_ALLOC
+// NOAPPLYDYNREL-NEXT:   SHF_WRITE
+// NOAPPLYDYNREL-NEXT: ]
+// NOAPPLYDYNREL-NEXT: Address: 0x1000
+// NOAPPLYDYNREL-NEXT: Offset: 0x1000
+// NOAPPLYDYNREL-NEXT: Size: 16
+// NOAPPLYDYNREL-NEXT: Link: 0
+// NOAPPLYDYNREL-NEXT: Info: 0
+// NOAPPLYDYNREL-NEXT: AddressAlignment: 1
+// NOAPPLYDYNREL-NEXT: EntrySize: 0
+// NOAPPLYDYNREL-NEXT: SectionData (
+// NOAPPLYDYNREL-NEXT:   0000: 00000000 00000000 00000000 00000000
+// NOAPPLYDYNREL-NEXT: )
 
 // CHECK:      Name: foo
 // CHECK-NEXT: Type: SHT_PROGBITS
 // CHECK-NEXT:    Flags [
 // CHECK-NEXT: ]
 // CHECK-NEXT: Address: 0x0
 // CHECK-NEXT: Offset:
 // CHECK-NEXT: Size: 32