[lld-macho] Support static linking of thread-locals

[int3]

Summary

Note: What ELF refers to as "TLS", Mach-O seems to refer to as "TLV", i.e.
thread-local variables.

This diff implements support for TLV relocations that reference defined
symbols. On x86_64, TLV relocations are always used with movq opcodes, so for
defined TLVs, we don't need to create a synthetic section to store the
addresses of the symbols -- we can just convert the movq to a leaq.

One notable quirk of Mach-O's TLVs is that absolute-address relocations
inside TLV-defining sections behave differently -- their addresses are
no longer absolute, but relative to the start of the target section.
(AFAICT, RIP-relative relocations are not allowed in these sections.)

[compnerd]

lld/MachO/Arch/X86_64.cpp

284

What do you think about moving this outside of this function? This way this function remains a read-only operation which is what one would expect as being a getter. Alternatively, you could rename the function if you think that this should be done in a single shot (I don't think that the access pattern is such that it would cause any performance implication).

288

This seems architecture specific. Could you please base this on the architecture via a switch?

lld/MachO/InputSection.h

42

nit: I think isThreadLocalVariable is a better name (the flag is checked on a single symbol)

[int3]

lld/MachO/Arch/X86_64.cpp

284

I think doing it in one shot makes sense since the VA we are returning is valid only if the opcode matches. But I agree the method name is now misleading... maybe resolveSymbolVA() would be better?

288

this entire file is meant to be architecture-specific

lld/MachO/InputSection.h

42

it's being checked on a section (which can contain many TLVs), not a symbol

[int3]

getSymbolVA() -> resolveSymbolVA()

[compnerd]

lld/MachO/InputSection.h

42

Ah, that's confusing. Perhaps it makes sense to make this a member of OutputSection (would be a good follow up change).

[smeenai]

LGTM

lld/MachO/InputSection.cpp

39

I'm assuming this should be resolveSymbolVA?

41

Is this not being put in resolveSymbolVA because it's target-independent?

Also, this is pretty strange ... I wonder why it's set up like this.

[int3]

update

[int3]

lld/MachO/InputSection.cpp

39

yup

41

yup, I think it's target-independent

[smeenai]

lld/MachO/InputSection.cpp

41

Can you double check against ld64 to make sure that's the case? If so, this LGTM.

[int3]

lld/MachO/InputSection.cpp

41

Yeah, src/ld/passes/tlvp.cpp does this in a target-independent manner.

lld/MachO/Arch/X86_64.cpp

struct X86_64 : TargetInfo {
 
   void writeStub(uint8_t *buf, const DylibSymbol &) const override;
   void writeStubHelperHeader(uint8_t *buf) const override;
   void writeStubHelperEntry(uint8_t *buf, const DylibSymbol &,
                             uint64_t entryAddr) const override;
 
   void prepareSymbolRelocation(lld::macho::Symbol &, const InputSection *,
                                const Reloc &) override;
-  uint64_t getSymbolVA(const lld::macho::Symbol &, uint8_t type) const override;
+  uint64_t resolveSymbolVA(uint8_t *buf, const lld::macho::Symbol &,
+                           uint8_t type) const override;
 };
 
 } // namespace
 
 static std::string getErrorLocation(MemoryBufferRef mb, const section_64 &sec,
                                     const relocation_info &rel) {
   return ("invalid relocation at offset " + std::to_string(rel.r_address) +
           " of " + sec.segname + "," + sec.sectname + " in " +
static void validateLength(MemoryBufferRef mb, const section_64 &sec,
   }
   fatal(msg);
 }
 
 uint64_t X86_64::getImplicitAddend(MemoryBufferRef mb, const section_64 &sec,
                                    const relocation_info &rel) const {
   auto *buf = reinterpret_cast<const uint8_t *>(mb.getBufferStart());
   const uint8_t *loc = buf + sec.offset + rel.r_address;
+
+  if (isThreadLocalVariables(sec.flags) && rel.r_type != X86_64_RELOC_UNSIGNED)
+    error("relocations in thread-local variable sections must be "
+          "X86_64_RELOC_UNSIGNED");
+
   switch (rel.r_type) {
   case X86_64_RELOC_BRANCH:
     // XXX: ld64 also supports r_length = 0 here but I'm not sure when such a
     // relocation will actually be generated.
     validateLength(mb, sec, rel, {2});
     break;
   case X86_64_RELOC_SIGNED:
   case X86_64_RELOC_SIGNED_1:
   case X86_64_RELOC_SIGNED_2:
   case X86_64_RELOC_SIGNED_4:
   case X86_64_RELOC_GOT_LOAD:
   case X86_64_RELOC_GOT:
+  case X86_64_RELOC_TLV:
     if (!rel.r_pcrel)
       fatal(getErrorLocation(mb, sec, rel) + ": relocations of type " +
             std::to_string(rel.r_type) + " must be pcrel");
     validateLength(mb, sec, rel, {2});
     break;
   case X86_64_RELOC_UNSIGNED:
     if (rel.r_pcrel)
       fatal(getErrorLocation(mb, sec, rel) + ": relocations of type " +
void X86_64::relocateOne(uint8_t *loc, const Reloc &r, uint64_t val) const {
   switch (r.type) {
   case X86_64_RELOC_BRANCH:
   case X86_64_RELOC_SIGNED:
   case X86_64_RELOC_SIGNED_1:
   case X86_64_RELOC_SIGNED_2:
   case X86_64_RELOC_SIGNED_4:
   case X86_64_RELOC_GOT_LOAD:
   case X86_64_RELOC_GOT:
+  case X86_64_RELOC_TLV:
     // These types are only used for pc-relative relocations, so offset by 4
     // since the RIP has advanced by 4 at this point. This is only valid when
     // r_length = 2, which is enforced by validateLength().
     val -= 4;
     break;
   case X86_64_RELOC_UNSIGNED:
     break;
   default:
case X86_64_RELOC_UNSIGNED: {
     }
     break;
   }
   case X86_64_RELOC_SIGNED:
   case X86_64_RELOC_SIGNED_1:
   case X86_64_RELOC_SIGNED_2:
   case X86_64_RELOC_SIGNED_4:
     break;
-  case X86_64_RELOC_SUBTRACTOR:
   case X86_64_RELOC_TLV:
+    if (auto *dysym = dyn_cast<DylibSymbol>(&sym))
+      error("relocations to thread-local dylib symbols not yet implemented");
+    else
+      assert(isa<Defined>(&sym));
+    break;
+  case X86_64_RELOC_SUBTRACTOR:
     fatal("TODO: handle relocation type " + std::to_string(r.type));
     break;
   default:
     llvm_unreachable("unexpected relocation type");
   }
 }
 
-uint64_t X86_64::getSymbolVA(const lld::macho::Symbol &sym,
+uint64_t X86_64::resolveSymbolVA(uint8_t *buf, const lld::macho::Symbol &sym,
                                  uint8_t type) const {
   switch (type) {
   case X86_64_RELOC_GOT_LOAD:
   case X86_64_RELOC_GOT:
     return in.got->addr + sym.gotIndex * WordSize;
   case X86_64_RELOC_BRANCH:
     if (auto *dysym = dyn_cast<DylibSymbol>(&sym))
       return in.stubs->addr + dysym->stubsIndex * sizeof(stub);
     return sym.getVA();
   case X86_64_RELOC_UNSIGNED:
   case X86_64_RELOC_SIGNED:
   case X86_64_RELOC_SIGNED_1:
   case X86_64_RELOC_SIGNED_2:
   case X86_64_RELOC_SIGNED_4:
     return sym.getVA();
+  case X86_64_RELOC_TLV: {
+    if (auto *dysym = dyn_cast<DylibSymbol>(&sym))
+      error("relocations to thread-local dylib symbols not yet implemented");
+
+    // Convert the movq to a leaq.
+    assert(isa<Defined>(&sym));
+    if (buf[-2] != 0x8b)
+      error("X86_64_RELOC_TLV must be used with movq instructions");
+    buf[-2] = 0x8d;
+    return sym.getVA();
+  }
   case X86_64_RELOC_SUBTRACTOR:
-  case X86_64_RELOC_TLV:
     fatal("TODO: handle relocation type " + std::to_string(type));
   default:
     llvm_unreachable("Unexpected relocation type");
   }
 }
 
 X86_64::X86_64() {
   cpuType = CPU_TYPE_X86_64;

lld/MachO/InputSection.h

struct Reloc {
   // the destination that this relocation refers to.
   uint64_t addend;
   llvm::PointerUnion<Symbol *, InputSection *> target;
 };
 
 inline bool isZeroFill(uint8_t flags) {
   return llvm::MachO::isVirtualSection(flags & llvm::MachO::SECTION_TYPE);
 }
+
+inline bool isThreadLocalVariables(uint8_t flags) {
+  return (flags & llvm::MachO::SECTION_TYPE) ==
+         llvm::MachO::S_THREAD_LOCAL_VARIABLES;
 }
 
 class InputSection {
 public:
   virtual ~InputSection() = default;
   virtual uint64_t getSize() const { return data.size(); }
   virtual uint64_t getFileSize() const {
     return isZeroFill(flags) ? 0 : getSize();

lld/MachO/InputSection.cpp

 void InputSection::writeTo(uint8_t *buf) {
   if (getFileSize() == 0)
     return;
 
   memcpy(buf, data.data(), data.size());
 
   for (Reloc &r : relocs) {
     uint64_t va = 0;
-    if (auto *s = r.target.dyn_cast<Symbol *>())
-      va = target->getSymbolVA(*s, r.type);
-    else if (auto *isec = r.target.dyn_cast<InputSection *>())
+    if (auto *s = r.target.dyn_cast<Symbol *>()) {
+      va = target->resolveSymbolVA(buf + r.offset, *s, r.type);
+
+      if (isThreadLocalVariables(flags)) {
+        // References from thread-local variable sections are treated as
+        // offsets relative to the start of the target section, instead of as
+        // absolute addresses.
+        if (auto *defined = dyn_cast<Defined>(s))
+          va -= defined->isec->parent->addr;
+      }
+    } else if (auto *isec = r.target.dyn_cast<InputSection *>()) {
       va = isec->getVA();
+    }
 
     uint64_t val = va + r.addend;
     if (r.pcrel)
       val -= getVA() + r.offset;
     target->relocateOne(buf + r.offset, r, val);
   }
 }

lld/MachO/SyntheticSections.cpp

void MachHeaderSection::writeTo(uint8_t *buf) const {
   auto *hdr = reinterpret_cast<MachO::mach_header_64 *>(buf);
   hdr->magic = MachO::MH_MAGIC_64;
   hdr->cputype = MachO::CPU_TYPE_X86_64;
   hdr->cpusubtype = MachO::CPU_SUBTYPE_X86_64_ALL | MachO::CPU_SUBTYPE_LIB64;
   hdr->filetype = config->outputType;
   hdr->ncmds = loadCommands.size();
   hdr->sizeofcmds = sizeOfCmds;
   hdr->flags = MachO::MH_NOUNDEFS | MachO::MH_DYLDLINK | MachO::MH_TWOLEVEL;
+
   if (config->outputType == MachO::MH_DYLIB && !config->hasReexports)
     hdr->flags |= MachO::MH_NO_REEXPORTED_DYLIBS;
 
+  for (OutputSegment *seg : outputSegments) {
+    for (OutputSection *osec : seg->getSections()) {
+      if (isThreadLocalVariables(osec->flags)) {
+        hdr->flags |= MachO::MH_HAS_TLV_DESCRIPTORS;
+        break;
+      }
+    }
+  }
+
   uint8_t *p = reinterpret_cast<uint8_t *>(hdr + 1);
   for (LoadCommand *lc : loadCommands) {
     lc->writeTo(p);
     p += lc->getSize();
   }
 }
 
 PageZeroSection::PageZeroSection()

lld/MachO/Target.h

public:
   // details.
   virtual void writeStub(uint8_t *buf, const DylibSymbol &) const = 0;
   virtual void writeStubHelperHeader(uint8_t *buf) const = 0;
   virtual void writeStubHelperEntry(uint8_t *buf, const DylibSymbol &,
                                     uint64_t entryAddr) const = 0;
 
   // Symbols may be referenced via either the GOT or the stubs section,
   // depending on the relocation type. prepareSymbolRelocation() will set up the
-  // GOT/stubs entries, and getSymbolVA() will return the addresses of those
-  // entries.
+  // GOT/stubs entries, and resolveSymbolVA() will return the addresses of those
+  // entries. resolveSymbolVA() may also relax the target instructions to save
+  // on a level of address indirection.
   virtual void prepareSymbolRelocation(Symbol &, const InputSection *,
                                        const Reloc &) = 0;
-  virtual uint64_t getSymbolVA(const Symbol &, uint8_t type) const = 0;
+  virtual uint64_t resolveSymbolVA(uint8_t *buf, const Symbol &,
+                                   uint8_t type) const = 0;
 
   uint32_t cpuType;
   uint32_t cpuSubtype;
 
   size_t stubSize;
   size_t stubHelperHeaderSize;
   size_t stubHelperEntrySize;
 };

lld/test/MachO/Inputs/MacOSX.sdk/usr/lib/libSystem.tbd

 archs:            [ x86_64 ]
 uuids:            [ 'x86_64: 00000000-0000-0000-0000-000000000001' ]
 platform:         macosx
 install-name:     '/usr/lib/system/libdyld.dylib'
 current-version:  0001.001.1
 parent-umbrella:  System
 exports:
   - archs:        [ 'x86_64' ]
-    symbols:      [ dyld_stub_binder ]
+    symbols:      [ dyld_stub_binder, __tlv_bootstrap ]
 --- !tapi-tbd-v3
 archs:            [ x86_64 ]
 uuids:            [ 'x86_64: 00000000-0000-0000-0000-000000000002' ]
 platform:         macosx
 install-name:     '/usr/lib/system/libsystem_c.dylib'
 current-version:  0001.001.1
 parent-umbrella:  System
 exports:

lld/test/MachO/invalid/bad-tlv-def.s

+# REQUIRES: x86
+# RUN: llvm-mc -filetype=obj -triple=x86_64-apple-darwin %s -o %t.o
+# RUN: not lld -flavor darwinnew -o /dev/null %t.o 2>&1 | FileCheck %s
+
+# CHECK: error: relocations in thread-local variable sections must be X86_64_RELOC_UNSIGNED
+
+.text
+.globl _main
+_main:
+  ret
+
+.section  __DATA,__thread_vars,thread_local_variables
+.globl  _foo, _bar
+_foo:
+  movq _bar@GOTPCREL(%rip), %rax

lld/test/MachO/invalid/bad-tlv-opcode.s

+# REQUIRES: x86
+# RUN: llvm-mc -filetype=obj -triple=x86_64-apple-darwin %s -o %t.o
+# RUN: not lld -flavor darwinnew -o /dev/null %t.o 2>&1 | FileCheck %s
+
+# CHECK: error: X86_64_RELOC_TLV must be used with movq instructions
+
+.text
+.globl _main
+_main:
+  leaq _foo@TLVP(%rip), %rax
+  ret
+
+.section  __DATA,__thread_vars,thread_local_variables
+_foo:

lld/test/MachO/tlv.s

+# REQUIRES: x86
+# RUN: mkdir -p %t
+# RUN: llvm-mc -filetype=obj -triple=x86_64-apple-darwin %s -o %t/test.o
+# RUN: lld -flavor darwinnew -L%S/Inputs/MacOSX.sdk/usr/lib -lSystem -o %t/test %t/test.o
+# RUN: llvm-readobj --file-headers %t/test | FileCheck %s --check-prefix=HEADER
+# RUN: llvm-objdump -D %t/test | FileCheck %s
+
+# HEADER: MH_HAS_TLV_DESCRIPTORS
+
+# CHECK:       Disassembly of section __TEXT,__text:
+# CHECK-EMPTY:
+# CHECK-NEXT:  <_main>:
+# CHECK-NEXT:  leaq    {{.*}}(%rip), %rax  # {{.*}} <_foo>
+# CHECK-NEXT:  leaq    {{.*}}(%rip), %rax  # {{.*}} <_bar>
+# CHECK-NEXT:  retq
+# CHECK-EMPTY:
+# CHECK-NEXT:  Disassembly of section __DATA,__thread_data:
+# CHECK-EMPTY:
+# CHECK-NEXT:  <__thread_data>:
+# CHECK-NEXT:  ef
+# CHECK-NEXT:  be ad de be ba
+# CHECK-NEXT:  fe ca
+# CHECK-EMPTY:
+# CHECK-NEXT:  Disassembly of section __DATA,__thread_vars:
+# CHECK-EMPTY:
+# CHECK-NEXT: <_foo>:
+# CHECK-NEXT:          ...
+# CHECK-EMPTY:
+# CHECK-NEXT:  <_bar>:
+# CHECK-NEXT:          ...
+# CHECK-NEXT:  04 00
+# CHECK-NEXT:  00 00
+# CHECK-NEXT:  00 00
+# CHECK-NEXT:  00 00
+
+.globl _main
+_main:
+  mov _foo@TLVP(%rip), %rax
+  mov _bar@TLVP(%rip), %rax
+  ret
+
+.section  __DATA,__thread_data,thread_local_regular
+_foo$tlv$init:
+  .long 0xdeadbeef
+_bar$tlv$init:
+  .long 0xcafebabe
+
+.section  __DATA,__thread_vars,thread_local_variables
+.globl  _foo, _bar
+_foo:
+  .quad __tlv_bootstrap
+  .quad 0
+  .quad _foo$tlv$init
+_bar:
+  .quad __tlv_bootstrap
+  .quad 0
+  .quad _bar$tlv$init