[lld-macho] Support dynamic linking of thread-locals

[int3]

Summary

References to symbols in dylibs work very similarly regardless of
whether the symbol is a TLV. The main difference is that we have a
separate __thread_ptrs section that acts as the GOT for these
thread-locals.

We can identify thread-locals in dylibs by a flag in their export trie
entries, and we cross-check it with the relocations that refer to them
to ensure that we are not using a GOT relocation to reference a
thread-local (or vice versa).

[compnerd]

lld/MachO/Arch/X86_64.cpp

227

Personally I prefer TLV as it is an initialism.

lld/MachO/ExportTrie.cpp

67

What do you think of extracting this into a simple helper function that should be marked as inline?

static inline uint32_t GetSymbolFlags(const Symbol &S) {
  uint32_t flags = 0;
  if (S.isWeakDefa()) flags |= EXPORT_SYMBOL_FLAGS_WEAK_DEFINITION;
  if (S.isTLV()) flags |= EXPORT_SYMBOL_FLAGS_KIND_THREAD_LOCAL;
  ...
  return flags;
}

lld/MachO/InputFiles.cpp

410

Perhaps it makes sense to pass a Symbol & to addDylib rather than having a number of flags to the function?

lld/MachO/SyntheticSections.cpp

141

Why the removal of this assert? Seems unrelated (I suspect its related to the TLV section being filled in).

lld/MachO/SyntheticSections.h

104

Would you mind renaming this type? It is confusing with __nl_symbol_ptr being a specific section.

[int3]

lld/MachO/Arch/X86_64.cpp

227

I picked this to match LLD-ELF's isTls()

lld/MachO/ExportTrie.cpp

67

updateOffset has the potential to query flags many times, so I wanted to avoid doing unnecessary computation. I also don't see how a helper function helps readability here...

lld/MachO/InputFiles.cpp

410

I agree that it's starting to look a bit unwieldy, but it looks like the equivalent addSymbol method in ld64 only takes these two flags, so I don't think we have to worry about a further proliferation of flags...

also passing Symbol & in wouldn't work very elegantly with our placement-new-based replaceSymbol implementation.

lld/MachO/SyntheticSections.h

104

Do you have a suggestion? I picked this name since all these sections will have the S_NON_LAZY_SYMBOL_POINTERS flag set. I initially didn't think the collision with __nl_symbol_ptr was super important since it no longer seems to be used when linking for x86_64 targets, but I've realized that it's still in use for ARM (and i386) targets

[int3]

lld/MachO/SyntheticSections.h

104

I'm thinking NonLazyPointerBase or NonLazyPointerSectionBase

[int3]

lld/MachO/SyntheticSections.cpp

141

yeah there's no guarantee that the TLV section will come after the GOT / before the rest of the sections. Actually there's no guarantee that the GOT comes before either, I think we just got lucky that the assert didn't get triggered earlier...

Anyway, I previously added this assert because I'd thought that BIND_OPCODE_ADD_ADDR_ULEB was adding an unsigned, arbitrary-sized integer to the address offset. But it turns out that it's actually adding a signed 64-bit value, so we can subtract from the offset here.

[int3]

rename

[int3]

clang-tidy

[int3]

Bump. There are a number of diffs stacked on top of this one, hoping to land them soon!

[smeenai]

Can you also add a test with both GOT and TLV bindings, to ensure nothing's getting mixed up between the two?

lld/MachO/Arch/X86_64.cpp

228

Would be nice to include information about the file, section and offset of the incorrect relocation, if possible.

265

Same here.

lld/MachO/Symbols.h

63

It's slightly confusing to have gotIndex possibly refer to the TLV index, but I can't immediately think of a better name (short of gotOrTlvIndex or perhaps something like pointerSectionIndex).

lld/MachO/SyntheticSections.cpp

175

Can we just use idx to make this more generic, since it'll be for the TLV ones too?

lld/MachO/SyntheticSections.h

101

"they are will both be" -> "they will both be"

lld/test/MachO/tlv-dylib.s

5

The split-file utility was landed last week (rGbcea3a7a288). It might be nicer for some of these tests than the long echo lines.

21

Is this DAG just because you don't care about the order of the bind table entries, or because there's a potential for the order to be different? (I would hope the latter isn't the case?)

[int3]

lld/MachO/Arch/X86_64.cpp

228

offset is a bit tricky due to subsection splitting, but including the other two is straightforward enough

lld/test/MachO/tlv-dylib.s

5

ooh, that is a sweet tool

21

The former -- the ordering should be arbitrary (but deterministic)

[int3]

address comments

[smeenai]

Can you also add a test with both GOT and TLV bindings, to ensure nothing's getting mixed up between the two?

I don't see this one? LGTM otherwise.

lld/MachO/Symbols.h

63

Do you have any better naming ideas for this?

lld/test/MachO/tlv-dylib.s

5

CC @MaskRay :)

[int3]

extend test

[int3]

I don't see this one? LGTM otherwise.

Whoops, I carefully checked off all the inline ones but forgot to look at the non-inline one 😅I've extended tlv-dylib to have a GOT entry.

lld/MachO/InputSection.cpp

61

NB: this is the same format used by LLD-ELF

lld/MachO/Symbols.h

63

hm, I think this is fine... I personally think of the TLV section as "GOT for thread locals", so this naming doesn't seem too far off the mark

[smeenai]

In D85081#2214557, @int3 wrote:

I don't see this one? LGTM otherwise.

Whoops, I carefully checked off all the inline ones but forgot to look at the non-inline one 😅I've extended tlv-dylib to have a GOT entry.

Hah, I figured. Thanks!

lld/MachO/Symbols.h

63

Sure, works for me.

lld/MachO/Arch/X86_64.cpp

 
 void X86_64::prepareSymbolRelocation(lld::macho::Symbol &sym,
                                      const InputSection *isec, const Reloc &r) {
   switch (r.type) {
   case X86_64_RELOC_GOT_LOAD:
     // TODO: implement mov -> lea relaxation for non-dynamic symbols
   case X86_64_RELOC_GOT:
     in.got->addEntry(sym);
+    if (sym.isTlv())
+      error("found GOT relocation referencing thread-local variable in " +
+            toString(isec));
     break;
   case X86_64_RELOC_BRANCH: {
     // TODO: weak dysyms should go into the weak binding section instead
     if (auto *dysym = dyn_cast<DylibSymbol>(&sym))
       in.stubs->addEntry(*dysym);
     break;
   }
   case X86_64_RELOC_UNSIGNED: {
     if (auto *dysym = dyn_cast<DylibSymbol>(&sym)) {
       if (r.length != 3) {
         error("X86_64_RELOC_UNSIGNED referencing the dynamic symbol " +
               dysym->getName() + " must have r_length = 3");
         return;
       }
       in.binding->addEntry(dysym, isec, r.offset, r.addend);
     }
     break;
   }
   case X86_64_RELOC_SIGNED:
   case X86_64_RELOC_SIGNED_1:
   case X86_64_RELOC_SIGNED_2:
   case X86_64_RELOC_SIGNED_4:
     break;
   case X86_64_RELOC_TLV:
-    if (isa<DylibSymbol>(&sym))
-      error("relocations to thread-local dylib symbols not yet implemented");
-    else
+    if (isa<DylibSymbol>(&sym)) {
+      in.tlvPointers->addEntry(sym);
+    } else {
       assert(isa<Defined>(&sym));
+      // TLV relocations on x86_64 are always used with a movq opcode, which
+      // can be converted to leaq opcodes if they reference a defined symbol.
+      // (This is in contrast to GOT relocations, which can be used with
+      // non-movq opcodes.) As such, there is no need to add an entry to
+      // tlvPointers here.
+    }
+    if (!sym.isTlv())
+      error(
+          "found X86_64_RELOC_TLV referencing a non-thread-local variable in " +
+          toString(isec));
     break;
   case X86_64_RELOC_SUBTRACTOR:
     fatal("TODO: handle relocation type " + std::to_string(r.type));
     break;
   default:
     llvm_unreachable("unexpected relocation type");
   }
 }
uint64_t X86_64::resolveSymbolVA(uint8_t *buf, const lld::macho::Symbol &sym,
   case X86_64_RELOC_UNSIGNED:
   case X86_64_RELOC_SIGNED:
   case X86_64_RELOC_SIGNED_1:
   case X86_64_RELOC_SIGNED_2:
   case X86_64_RELOC_SIGNED_4:
     return sym.getVA();
   case X86_64_RELOC_TLV: {
     if (isa<DylibSymbol>(&sym))
-      error("relocations to thread-local dylib symbols not yet implemented");
+      return in.tlvPointers->addr + sym.gotIndex * WordSize;
 
     // Convert the movq to a leaq.
     assert(isa<Defined>(&sym));
     if (buf[-2] != 0x8b)
       error("X86_64_RELOC_TLV must be used with movq instructions");
     buf[-2] = 0x8d;
     return sym.getVA();
   }

lld/MachO/ExportTrie.cpp

struct Edge {
   Edge(StringRef s, TrieNode *node) : substring(s), child(node) {}
 
   StringRef substring;
   struct TrieNode *child;
 };
 
 struct ExportInfo {
   uint64_t address;
-  uint8_t flags;
-  explicit ExportInfo(const Symbol &sym)
-      : address(sym.getVA()),
-        flags(sym.isWeakDef() ? EXPORT_SYMBOL_FLAGS_WEAK_DEFINITION : 0) {}
+  uint8_t flags = 0;
+  explicit ExportInfo(const Symbol &sym) : address(sym.getVA()) {
+    if (sym.isWeakDef())
+      flags |= EXPORT_SYMBOL_FLAGS_WEAK_DEFINITION;
+    if (sym.isTlv())
+      flags |= EXPORT_SYMBOL_FLAGS_KIND_THREAD_LOCAL;
     // TODO: Add proper support for re-exports & stub-and-resolver flags.
+  }
 };
 
 } // namespace
 
 struct macho::TrieNode {
   std::vector<Edge> edges;
   Optional<ExportInfo> info;
   // Estimated offset from the start of the serialized trie to the current node.

lld/MachO/InputFiles.cpp

DylibFile::DylibFile(MemoryBufferRef mb, DylibFile *umbrella)
   }
 
   // Initialize symbols.
   if (const load_command *cmd = findCommand(hdr, LC_DYLD_INFO_ONLY)) {
     auto *c = reinterpret_cast<const dyld_info_command *>(cmd);
     parseTrie(buf + c->export_off, c->export_size,
               [&](const Twine &name, uint64_t flags) {
                 bool isWeakDef = flags & EXPORT_SYMBOL_FLAGS_WEAK_DEFINITION;
-                symbols.push_back(
-                    symtab->addDylib(saver.save(name), umbrella, isWeakDef));
+                bool isTlv = flags & EXPORT_SYMBOL_FLAGS_KIND_THREAD_LOCAL;
+                symbols.push_back(symtab->addDylib(saver.save(name), umbrella,
+                                                   isWeakDef, isTlv));
               });
   } else {
     error("LC_DYLD_INFO_ONLY not found in " + getName());
     return;
   }
 
   if (hdr->flags & MH_NO_REEXPORTED_DYLIBS)
     return;

 DylibFile::DylibFile(std::shared_ptr<llvm::MachO::InterfaceFile> interface,
                      DylibFile *umbrella)
     : InputFile(DylibKind, MemoryBufferRef()) {
   if (umbrella == nullptr)
     umbrella = this;
 
   dylibName = saver.save(interface->getInstallName());
   // TODO(compnerd) filter out symbols based on the target platform
-  // TODO: handle weak defs
+  // TODO: handle weak defs, thread locals
   for (const auto symbol : interface->symbols())
     if (symbol->getArchitectures().has(config->arch))
       symbols.push_back(symtab->addDylib(saver.save(symbol->getName()),
-                                         umbrella, /*isWeakDef=*/false));
+                                         umbrella, /*isWeakDef=*/false,
+                                         /*isTlv=*/false));
   // TODO(compnerd) properly represent the hierarchy of the documents as it is
   // in theory possible to have re-exported dylibs from re-exported dylibs which
   // should be parent'ed to the child.
   for (auto document : interface->documents())
     reexported.push_back(make<DylibFile>(document, umbrella));
 }
 
 ArchiveFile::ArchiveFile(std::unique_ptr<llvm::object::Archive> &&f)

lld/MachO/InputSection.h

public:
 
   ArrayRef<uint8_t> data;
   std::vector<Reloc> relocs;
 };
 
 extern std::vector<InputSection *> inputSections;
 
 } // namespace macho
+
+std::string toString(const macho::InputSection *);
+
 } // namespace lld
 
 #endif

lld/MachO/InputSection.cpp

 //===- InputSection.cpp ---------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "InputSection.h"
+#include "InputFiles.h"
 #include "OutputSegment.h"
 #include "Symbols.h"
 #include "Target.h"
 #include "lld/Common/Memory.h"
 #include "llvm/Support/Endian.h"
 
 using namespace llvm;
 using namespace llvm::MachO;
for (Reloc &r : relocs) {
     }
 
     uint64_t val = va + r.addend;
     if (r.pcrel)
       val -= getVA() + r.offset;
     target->relocateOne(buf + r.offset, r, val);
   }
 }
+
+std::string lld::toString(const InputSection *isec) {
+  return (toString(isec->file) + ":(" + isec->name + ")").str();
+}

lld/MachO/SymbolTable.h

  */
 class SymbolTable {
 public:
   Symbol *addDefined(StringRef name, InputSection *isec, uint32_t value,
                      bool isWeakDef);
 
   Symbol *addUndefined(StringRef name);
 
-  Symbol *addDylib(StringRef name, DylibFile *file, bool isWeakDef);
+  Symbol *addDylib(StringRef name, DylibFile *file, bool isWeakDef, bool isTlv);
 
   Symbol *addLazy(StringRef name, ArchiveFile *file,
                   const llvm::object::Archive::Symbol &sym);
 
   Symbol *addDSOHandle(const MachHeaderSection *);
 
   ArrayRef<Symbol *> getSymbols() const { return symVector; }
   Symbol *find(StringRef name);

lld/MachO/SymbolTable.cpp

Symbol *SymbolTable::addUndefined(StringRef name) {
 
   if (wasInserted)
     replaceSymbol<Undefined>(s, name);
   else if (LazySymbol *lazy = dyn_cast<LazySymbol>(s))
     lazy->fetchArchiveMember();
   return s;
 }
 
-Symbol *SymbolTable::addDylib(StringRef name, DylibFile *file, bool isWeakDef) {
+Symbol *SymbolTable::addDylib(StringRef name, DylibFile *file, bool isWeakDef,
+                              bool isTlv) {
   Symbol *s;
   bool wasInserted;
   std::tie(s, wasInserted) = insert(name);
 
   if (wasInserted || isa<Undefined>(s) ||
       (isa<DylibSymbol>(s) && !isWeakDef && s->isWeakDef()))
-    replaceSymbol<DylibSymbol>(s, file, name, isWeakDef);
+    replaceSymbol<DylibSymbol>(s, file, name, isWeakDef, isTlv);
 
   return s;
 }
 
 Symbol *SymbolTable::addLazy(StringRef name, ArchiveFile *file,
                              const llvm::object::Archive::Symbol &sym) {
   Symbol *s;
   bool wasInserted;

lld/MachO/Symbols.h

public:
   virtual uint64_t getVA() const { return 0; }
 
   virtual uint64_t getFileOffset() const {
     llvm_unreachable("attempt to get an offset from a non-defined symbol");
   }
 
   virtual bool isWeakDef() const { llvm_unreachable("cannot be weak"); }
 
+  virtual bool isTlv() const { llvm_unreachable("cannot be TLV"); }
+
+  // The index of this symbol in the GOT or the TLVPointer section, depending
+  // on whether it is a thread-local. A given symbol cannot be referenced by
+  // both these sections at once.
   uint32_t gotIndex = UINT32_MAX;
 
 protected:
   Symbol(Kind k, StringRefZ name) : symbolKind(k), name(name) {}
 
   Kind symbolKind;
   StringRefZ name;
 };
 
 class Defined : public Symbol {
 public:
   Defined(StringRefZ name, InputSection *isec, uint32_t value, bool isWeakDef)
       : Symbol(DefinedKind, name), isec(isec), value(value),
         weakDef(isWeakDef) {}
 
   bool isWeakDef() const override { return weakDef; }
 
+  bool isTlv() const override { return isThreadLocalVariables(isec->flags); }
+
   static bool classof(const Symbol *s) { return s->kind() == DefinedKind; }
 
   uint64_t getVA() const override { return isec->getVA() + value; }
 
   uint64_t getFileOffset() const override {
     return isec->getFileOffset() + value;
   }
 
   InputSection *isec;
   uint32_t value;
 
 private:
   const bool weakDef;
 };
 
 class Undefined : public Symbol {
 public:
   Undefined(StringRefZ name) : Symbol(UndefinedKind, name) {}
 
   static bool classof(const Symbol *s) { return s->kind() == UndefinedKind; }
 };
 
 class DylibSymbol : public Symbol {
 public:
-  DylibSymbol(DylibFile *file, StringRefZ name, bool isWeakDef)
-      : Symbol(DylibKind, name), file(file), weakDef(isWeakDef) {}
+  DylibSymbol(DylibFile *file, StringRefZ name, bool isWeakDef, bool isTlv)
+      : Symbol(DylibKind, name), file(file), weakDef(isWeakDef), tlv(isTlv) {}
 
   bool isWeakDef() const override { return weakDef; }
 
+  bool isTlv() const override { return tlv; }
+
   static bool classof(const Symbol *s) { return s->kind() == DylibKind; }
 
   DylibFile *file;
   uint32_t stubsIndex = UINT32_MAX;
   uint32_t lazyBindOffset = UINT32_MAX;
 
 private:
   const bool weakDef;
+  const bool tlv;
 };
 
 class LazySymbol : public Symbol {
 public:
   LazySymbol(ArchiveFile *file, const llvm::object::Archive::Symbol &sym)
       : Symbol(LazyKind, sym.getName()), file(file), sym(sym) {}
 
   static bool classof(const Symbol *s) { return s->kind() == LazyKind; }

lld/MachO/SyntheticSections.h

 //===- SyntheticSections.h -------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLD_MACHO_SYNTHETIC_SECTIONS_H
 #define LLD_MACHO_SYNTHETIC_SECTIONS_H
 
 #include "Config.h"
 #include "ExportTrie.h"
 #include "InputSection.h"
 #include "OutputSection.h"
+#include "OutputSegment.h"
 #include "Target.h"
 
 #include "llvm/ADT/SetVector.h"
 #include "llvm/Support/raw_ostream.h"
 
 namespace lld {
 namespace macho {
 
 namespace section_names {
 
 constexpr const char pageZero[] = "__pagezero";
 constexpr const char header[] = "__mach_header";
 constexpr const char binding[] = "__binding";
 constexpr const char lazyBinding[] = "__lazy_binding";
 constexpr const char export_[] = "__export";
 constexpr const char symbolTable[] = "__symbol_table";
 constexpr const char stringTable[] = "__string_table";
 constexpr const char got[] = "__got";
+constexpr const char threadPtrs[] = "__thread_ptrs";
 
 } // namespace section_names
 
 class DylibSymbol;
 class LoadCommand;
 
 class SyntheticSection : public OutputSection {
 public:

 public:
   PageZeroSection();
   bool isHidden() const override { return true; }
   uint64_t getSize() const override { return PageZeroSize; }
   uint64_t getFileSize() const override { return 0; }
   void writeTo(uint8_t *buf) const override {}
 };
 
-// This section will be populated by dyld with addresses to non-lazily-loaded
-// dylib symbols.
-class GotSection : public SyntheticSection {
+// This is the base class for the GOT and TLVPointer sections, which are nearly
+// functionally identical -- they will both be populated by dyld with addresses
+// to non-lazily-loaded dylib symbols. The main difference is that the
+// TLVPointerSection stores references to thread-local variables.
+class NonLazyPointerSectionBase : public SyntheticSection {
 public:
-  GotSection();
+  NonLazyPointerSectionBase(const char *segname, const char *name);
 
   const llvm::SetVector<const Symbol *> &getEntries() const { return entries; }
 
   bool isNeeded() const override { return !entries.empty(); }
 
   uint64_t getSize() const override { return entries.size() * WordSize; }
 
   void writeTo(uint8_t *buf) const override;
 
   void addEntry(Symbol &sym);
 
 private:
   llvm::SetVector<const Symbol *> entries;
 };
 
+class GotSection : public NonLazyPointerSectionBase {
+public:
+  GotSection()
+      : NonLazyPointerSectionBase(segment_names::dataConst,
+                                  section_names::got) {
+    // TODO: section_64::reserved1 should be an index into the indirect symbol
+    // table, which we do not currently emit
+  }
+};
+
+class TlvPointerSection : public NonLazyPointerSectionBase {
+public:
+  TlvPointerSection()
+      : NonLazyPointerSectionBase(segment_names::data,
+                                  section_names::threadPtrs) {}
+};
+
 struct BindingEntry {
   const DylibSymbol *dysym;
   const InputSection *isec;
   uint64_t offset;
   int64_t addend;
   BindingEntry(const DylibSymbol *dysym, const InputSection *isec,
                uint64_t offset, int64_t addend)
       : dysym(dysym), isec(isec), offset(offset), addend(addend) {}
private:
   StringTableSection &stringTableSection;
   std::vector<SymtabEntry> symbols;
 };
 
 struct InStruct {
   MachHeaderSection *header = nullptr;
   BindingSection *binding = nullptr;
   GotSection *got = nullptr;
+  TlvPointerSection *tlvPointers = nullptr;
   LazyPointerSection *lazyPointers = nullptr;
   StubsSection *stubs = nullptr;
   StubHelperSection *stubHelper = nullptr;
   ImageLoaderCacheSection *imageLoaderCache = nullptr;
 };
 
 extern InStruct in;
 extern std::vector<SyntheticSection *> syntheticSections;
 
 } // namespace macho
 } // namespace lld
 
 #endif

lld/MachO/SyntheticSections.cpp

for (LoadCommand *lc : loadCommands) {
     lc->writeTo(p);
     p += lc->getSize();
   }
 }
 
 PageZeroSection::PageZeroSection()
     : SyntheticSection(segment_names::pageZero, section_names::pageZero) {}
 
-GotSection::GotSection()
-    : SyntheticSection(segment_names::dataConst, section_names::got) {
+NonLazyPointerSectionBase::NonLazyPointerSectionBase(const char *segname,
+                                                     const char *name)
+    : SyntheticSection(segname, name) {
   align = 8;
   flags = MachO::S_NON_LAZY_SYMBOL_POINTERS;
-
-  // TODO: section_64::reserved1 should be an index into the indirect symbol
-  // table, which we do not currently emit
 }
 
-void GotSection::addEntry(Symbol &sym) {
+void NonLazyPointerSectionBase::addEntry(Symbol &sym) {
   if (entries.insert(&sym)) {
+    assert(sym.gotIndex == UINT32_MAX);
     sym.gotIndex = entries.size() - 1;
   }
 }
 
-void GotSection::writeTo(uint8_t *buf) const {
+void NonLazyPointerSectionBase::writeTo(uint8_t *buf) const {
   for (size_t i = 0, n = entries.size(); i < n; ++i)
     if (auto *defined = dyn_cast<Defined>(entries[i]))
       write64le(&buf[i * WordSize], defined->getVA());
 }
 
 BindingSection::BindingSection()
     : LinkEditSection(segment_names::linkEdit, section_names::binding) {}
 
 bool BindingSection::isNeeded() const {
-  return bindings.size() != 0 || in.got->isNeeded();
+  return bindings.size() != 0 || in.got->isNeeded() ||
+         in.tlvPointers->isNeeded();
 }
 
 namespace {
 struct Binding {
   OutputSegment *segment = nullptr;
   uint64_t offset = 0;
   int64_t addend = 0;
   uint8_t ordinal = 0;
static void encodeBinding(const DylibSymbol &dysym, const OutputSection *osec,
   uint64_t offset = osec->getSegmentOffset() + outSecOff;
   if (lastBinding.segment != seg) {
     os << static_cast<uint8_t>(BIND_OPCODE_SET_SEGMENT_AND_OFFSET_ULEB |
                                seg->index);
     encodeULEB128(offset, os);
     lastBinding.segment = seg;
     lastBinding.offset = offset;
   } else if (lastBinding.offset != offset) {
-    assert(lastBinding.offset <= offset);
     os << static_cast<uint8_t>(BIND_OPCODE_ADD_ADDR_ULEB);
     encodeULEB128(offset - lastBinding.offset, os);
     lastBinding.offset = offset;
   }
 
   if (lastBinding.ordinal != dysym.file->ordinal) {
     if (dysym.file->ordinal <= BIND_IMMEDIATE_MASK) {
       os << static_cast<uint8_t>(BIND_OPCODE_SET_DYLIB_ORDINAL_IMM |
static void encodeBinding(const DylibSymbol &dysym, const OutputSection *osec,
   os << static_cast<uint8_t>(BIND_OPCODE_SET_SYMBOL_TRAILING_FLAGS_IMM)
      << dysym.getName() << '\0'
      << static_cast<uint8_t>(BIND_OPCODE_SET_TYPE_IMM | BIND_TYPE_POINTER)
      << static_cast<uint8_t>(BIND_OPCODE_DO_BIND);
   // DO_BIND causes dyld to both perform the binding and increment the offset
   lastBinding.offset += WordSize;
 }
 
+static bool encodeNonLazyPointerSection(NonLazyPointerSectionBase *osec,
+                                        Binding &lastBinding,
+                                        raw_svector_ostream &os) {
+  bool didEncode = false;
+  size_t idx = 0;
+  for (const Symbol *sym : osec->getEntries()) {
+    if (const auto *dysym = dyn_cast<DylibSymbol>(sym)) {
+      didEncode = true;
+      encodeBinding(*dysym, osec, idx * WordSize, /*addend=*/0, lastBinding,
+                    os);
+    }
+    ++idx;
+  }
+  return didEncode;
+}
+
 // Emit bind opcodes, which are a stream of byte-sized opcodes that dyld
 // interprets to update a record with the following fields:
 //  * segment index (of the segment to write the symbol addresses to, typically
 //    the __DATA_CONST segment which contains the GOT)
 //  * offset within the segment, indicating the next location to write a binding
 //  * symbol type
 //  * symbol library ordinal (the index of its library's LC_LOAD_DYLIB command)
 //  * symbol name
 //  * addend
 // When dyld sees BIND_OPCODE_DO_BIND, it uses the current record state to bind
 // a symbol in the GOT, and increments the segment offset to point to the next
 // entry. It does *not* clear the record state after doing the bind, so
 // subsequent opcodes only need to encode the differences between bindings.
 void BindingSection::finalizeContents() {
   raw_svector_ostream os{contents};
   Binding lastBinding;
-  bool didEncode = false;
-  size_t gotIdx = 0;
-  for (const Symbol *sym : in.got->getEntries()) {
-    if (const auto *dysym = dyn_cast<DylibSymbol>(sym)) {
-      didEncode = true;
-      encodeBinding(*dysym, in.got, gotIdx * WordSize, 0, lastBinding, os);
-    }
-    ++gotIdx;
-  }
+  bool didEncode = encodeNonLazyPointerSection(in.got, lastBinding, os);
+  didEncode |= encodeNonLazyPointerSection(in.tlvPointers, lastBinding, os);
 
   // Sorting the relocations by segment and address allows us to encode them
   // more compactly.
   llvm::sort(bindings, [](const BindingEntry &a, const BindingEntry &b) {
     OutputSegment *segA = a.isec->parent->parent;
     OutputSegment *segB = b.isec->parent->parent;
     if (segA != segB)
       return segA->fileOff < segB->fileOff;

lld/MachO/Writer.cpp

 }
 
 void macho::writeResult() { Writer().run(); }
 
 void macho::createSyntheticSections() {
   in.header = make<MachHeaderSection>();
   in.binding = make<BindingSection>();
   in.got = make<GotSection>();
+  in.tlvPointers = make<TlvPointerSection>();
   in.lazyPointers = make<LazyPointerSection>();
   in.stubs = make<StubsSection>();
   in.stubHelper = make<StubHelperSection>();
   in.imageLoaderCache = make<ImageLoaderCacheSection>();
 }

lld/test/MachO/invalid/bad-got-to-dylib-tlv-reference.s

+# REQUIRES: x86
+# RUN: split-file %s %t
+
+# RUN: llvm-mc -filetype=obj -triple=x86_64-apple-darwin %t/libtlv.s -o %t/libtlv.o
+# RUN: lld -flavor darwinnew -dylib -install_name @executable_path/libtlv.dylib \
+# RUN:   -Z -L%S/../Inputs/MacOSX.sdk/usr/lib -lSystem -o %t/libtlv.dylib %t/libtlv.o
+
+# RUN: llvm-mc -filetype=obj -triple=x86_64-apple-darwin %t/test.s -o %t/test.o
+# RUN: not lld -flavor darwinnew -Z -L%S/../Inputs/MacOSX.sdk/usr/lib -lSystem -L%t -ltlv -o /dev/null %t/test.o 2>&1 | FileCheck %s -DFILE=%t/test.o
+
+# CHECK: error: found GOT relocation referencing thread-local variable in [[FILE]]:(__text)
+
+#--- libtlv.s
+.section  __DATA,__thread_vars,thread_local_variables
+.globl _foo
+_foo:
+
+#--- test.s
+.text
+.globl _main
+_main:
+  movq _foo@GOTPCREL(%rip), %rax
+  ret

lld/test/MachO/invalid/bad-got-to-tlv-reference.s

+# REQUIRES: x86
+# RUN: llvm-mc -filetype=obj -triple=x86_64-apple-darwin %s -o %t.o
+# RUN: not lld -flavor darwinnew -o /dev/null %t.o 2>&1 | FileCheck %s -DFILE=%t.o
+
+# CHECK: error: found GOT relocation referencing thread-local variable in [[FILE]]:(__text)
+
+.text
+.globl _main
+_main:
+  movq _foo@GOTPCREL(%rip), %rax
+  ret
+
+.section __DATA,__thread_vars,thread_local_variables
+_foo:

lld/test/MachO/invalid/bad-tlv-relocation.s

+# REQUIRES: x86
+# RUN: llvm-mc -filetype=obj -triple=x86_64-apple-darwin %s -o %t.o
+# RUN: not lld -flavor darwinnew -o /dev/null %t.o 2>&1 | FileCheck %s -DFILE=%t.o
+
+# CHECK: error: found X86_64_RELOC_TLV referencing a non-thread-local variable in [[FILE]]:(__text)
+
+.text
+.globl _main
+_main:
+  leaq _foo@TLVP(%rip), %rax
+  ret
+
+.data
+_foo:

lld/test/MachO/tlv-dylib.s

+# REQUIRES: x86
+# RUN: split-file %s %t
+
+# RUN: llvm-mc -filetype=obj -triple=x86_64-apple-darwin %t/libtlv.s -o %t/libtlv.o
+# RUN: lld -flavor darwinnew -dylib -install_name @executable_path/libtlv.dylib \
+# RUN:   -Z -L%S/Inputs/MacOSX.sdk/usr/lib -lSystem -o %t/libtlv.dylib %t/libtlv.o
+# RUN: llvm-objdump --exports-trie -d --no-show-raw-insn %t/libtlv.dylib | FileCheck %s --check-prefix=DYLIB
+# DYLIB-DAG: _foo [per-thread]
+# DYLIB-DAG: _bar [per-thread]
+
+# RUN: llvm-mc -filetype=obj -triple=x86_64-apple-darwin %t/test.s -o %t/test.o
+# RUN: lld -flavor darwinnew -Z -L%S/Inputs/MacOSX.sdk/usr/lib -lSystem -L%t -ltlv %t/test.o -o %t/test
+# RUN: llvm-objdump --bind -d --no-show-raw-insn %t/test | FileCheck %s
+
+# CHECK:      movq [[#]](%rip), %rax # [[#%x, FOO:]]
+# CHECK-NEXT: movq [[#]](%rip), %rax # [[#%x, BAR:]]
+# CHECK-NEXT: movq [[#]](%rip), %rax # [[#%x, BAZ:]]
+
+# CHECK-LABEL: Bind table:
+# CHECK-DAG: __DATA       __thread_ptrs  0x{{0*}}[[#%x, FOO]] pointer 0   libtlv   _foo
+# CHECK-DAG: __DATA       __thread_ptrs  0x{{0*}}[[#%x, BAR]] pointer 0   libtlv   _bar
+# CHECK-DAG: __DATA_CONST __got          0x{{0*}}[[#%x, BAZ]] pointer 0   libtlv   _baz
+
+#--- libtlv.s
+.section  __DATA,__thread_vars,thread_local_variables
+.globl _foo, _bar, _baz
+_foo:
+_bar:
+
+.text
+_baz:
+
+#--- test.s
+.globl _main
+_main:
+  mov _foo@TLVP(%rip), %rax
+  mov _bar@TLVP(%rip), %rax
+## Add a GOT entry to make sure we don't mix it up with TLVs
+  mov _baz@GOTPCREL(%rip), %rax
+  ret