[RuntimeDyld] Implemented relocation of TLS symbols in ELF

[MoritzS]

Summary

The tests use a GOTPC32 relocation which is implemented in D95512. So, they
will fail for now.

[lhames]

Hi Moritz,

I'm very sorry that I did not see this earlier -- this is really interesting stuff!

Have you looked at JITLink for x86-64 yet? I think it might be better suited to this. Between JITLink and the just-landed (e256445bfff1) ELFNixPlatform code I think that you could get full cross-process (and even cross-architecture) TLV working, similar to what we have on Darwin / MachO.

We can still discuss this going in to RuntimeDyld in the short term if you particularly need / want it, but the aim is to deprecate and eventually remove RuntimeDyld, so implementing this in JITLink would be the strongly preferred approach if possible.

[MoritzS]

Thanks for looking at this!

I'm aware that RuntimeDyld will eventually be replaced by JITLink and I agree that this should definitely be implemented in JITLink eventually.
Since I'm using this (and my other patch for GNU IFUNCs D105465) in a research project where we currently make heavy use of RuntimeDyld/MCJIT, I would like to get this into main, even if it will be deprecated soon.
However, I think JITLink in general has a much better design and I will try to implement the patches for JITLink as well, soon.

[lhames]

What was the previous behavior when we encountered these relocations?

My only concern here is avoiding regressing existing users if we turn a previously unsupported (but silently accepted) relocation into a potential link-time error. It's perverse, but this kind of improvement to functionality and error checking has caused problems for clients before, and the main aim for MCJIT / RuntimeDyld right now is stability.

If the new support could cause a problem then I think the best solution would be to add a 'bool supportsTLV' method to the memory manager that we can query before running any of the TLV code.

llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldELF.cpp

1861

This function is pretty big already. Could the TLV code be split out into separate functions for readability?

[lhames]

However, I think JITLink in general has a much better design and I will try to implement the patches for JITLink as well, soon.

That would be very cool!

[MoritzS]

In D105466#2968873, @lhames wrote:

What was the previous behavior when we encountered these relocations?

My only concern here is avoiding regressing existing users if we turn a previously unsupported (but silently accepted) relocation into a potential link-time error. It's perverse, but this kind of improvement to functionality and error checking has caused problems for clients before, and the main aim for MCJIT / RuntimeDyld right now is stability.

If the new support could cause a problem then I think the best solution would be to add a 'bool supportsTLV' method to the memory manager that we can query before running any of the TLV code.

All relocations that are not implemented will eventually lead to report_fatal_error("Relocation type not implemented yet!"); in resolveX86_64Relocation(). Whereas with my changes it will reach report_fatal_error("allocation of TLS not implemented"); in MemoryManager::allocateTLSSection(). So in both cases the execution crashes, it just prints a different error message now.

[MoritzS]

llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldELF.cpp

1861

That's a good idea. Maybe this will also make it easier to port this to JITLink in the future.

[MoritzS]

Refactored most of the relocation logic into separate functions.

I added the changes from D95512 so that the tests won't fail.

[MoritzS]

Require x86_64-linux for the TLS test as the execution in llvm-rtdyld is
ifdef'd only for x86_64 and ELF.

[MoritzS]

Added NOLINT comments to avoid clang-tidy warning for the functions called that
have X86_64 in their name.

[lhames]

In D105466#2969076, @MoritzS wrote:

In D105466#2968873, @lhames wrote:

What was the previous behavior when we encountered these relocations?

My only concern here is avoiding regressing existing users if we turn a previously unsupported (but silently accepted) relocation into a potential link-time error. It's perverse, but this kind of improvement to functionality and error checking has caused problems for clients before, and the main aim for MCJIT / RuntimeDyld right now is stability.

If the new support could cause a problem then I think the best solution would be to add a 'bool supportsTLV' method to the memory manager that we can query before running any of the TLV code.

All relocations that are not implemented will eventually lead to report_fatal_error("Relocation type not implemented yet!"); in resolveX86_64Relocation(). Whereas with my changes it will reach report_fatal_error("allocation of TLS not implemented"); in MemoryManager::allocateTLSSection(). So in both cases the execution crashes, it just prints a different error message now.

Ok. I've just realized that there's also a danger on the flip-side too: code containing TLVs will be accepted, but this only support accessing them on one thread, which will be surprising to some clients. I think it would be worth adding an "enablePartialTLVSupport" method to the memory manager (which should default to false), and gating this new feature on that. If you're happy to do that before the code lands then that's great, otherwise I'm happy to do it in-tree after it lands.

Thank you again for all your work on this Moritz -- it's really appreciated!

[lhames]

Side note: @StephenFan just posted a review for initial JITLink / ELFNixPlatform TLV support in https://reviews.llvm.org/D109293, if you're interested in getting involved in that.

[MoritzS]

In D105466#2984334, @lhames wrote:

Ok. I've just realized that there's also a danger on the flip-side too: code containing TLVs will be accepted, but this only support accessing them on one thread, which will be surprising to some clients. I think it would be worth adding an "enablePartialTLVSupport" method to the memory manager (which should default to false), and gating this new feature on that. If you're happy to do that before the code lands then that's great, otherwise I'm happy to do it in-tree after it lands.

I implemented the single-thread only handling of TLS only in llvm-rtdyld.cpp. The default implementation in the memory manager is to always crash with the fatal error. So this should only be a problem if someone uses llvm-rtdyld -execute with an object file that has TLS relocations which previously crashed but now silently succeeds but will lead to incorrect execution when multiple threads are used.

[brad]

This commit appears to have broken the build on OpenBSD..

[23/210] Linking CXX executable bin/llvm-rtdyld
FAILED: bin/llvm-rtdyld
: && /usr/bin/c++ -fPIC -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wc++98-compat-extra-semi -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wstring-conversion -Wmisleading-indentation -fdiagnostics-color -g -Wl,--color-diagnostics tools/llvm-rtdyld/CMakeFiles/llvm-rtdyld.dir/llvm-rtdyld.cpp.o -o bin/llvm-rtdyld -L/usr/lib -Wl,-z,origin,-rpath,"\$ORIGIN/../lib"  lib/libLLVMAArch64Disassembler.a  lib/libLLVMAMDGPUDisassembler.a  lib/libLLVMARMDisassembler.a  lib/libLLVMAVRDisassembler.a  lib/libLLVMBPFDisassembler.a  lib/libLLVMHexagonDisassembler.a  lib/libLLVMLanaiDisassembler.a  lib/libLLVMMipsDisassembler.a  lib/libLLVMMSP430Disassembler.a  lib/libLLVMPowerPCDisassembler.a  lib/libLLVMRISCVDisassembler.a  lib/libLLVMSparcDisassembler.a  lib/libLLVMSystemZDisassembler.a  lib/libLLVMWebAssemblyDisassembler.a  lib/libLLVMX86Disassembler.a  lib/libLLVMXCoreDisassembler.a  lib/libLLVMAArch64Desc.a  lib/libLLVMAMDGPUDesc.a  lib/libLLVMARMDesc.a  lib/libLLVMAVRDesc.a  lib/libLLVMBPFDesc.a  lib/libLLVMHexagonDesc.a  lib/libLLVMLanaiDesc.a  lib/libLLVMMipsDesc.a  lib/libLLVMMSP430Desc.a  lib/libLLVMNVPTXDesc.a  lib/libLLVMPowerPCDesc.a  lib/libLLVMRISCVDesc.a  lib/libLLVMSparcDesc.a  lib/libLLVMSystemZDesc.a  lib/libLLVMWebAssemblyDesc.a  lib/libLLVMX86Desc.a  lib/libLLVMXCoreDesc.a  lib/libLLVMAArch64Info.a  lib/libLLVMAMDGPUInfo.a  lib/libLLVMARMInfo.a  lib/libLLVMAVRInfo.a  lib/libLLVMBPFInfo.a  lib/libLLVMHexagonInfo.a  lib/libLLVMLanaiInfo.a  lib/libLLVMMipsInfo.a  lib/libLLVMMSP430Info.a  lib/libLLVMNVPTXInfo.a  lib/libLLVMPowerPCInfo.a  lib/libLLVMRISCVInfo.a  lib/libLLVMSparcInfo.a  lib/libLLVMSystemZInfo.a  lib/libLLVMWebAssemblyInfo.a  lib/libLLVMX86Info.a  lib/libLLVMXCoreInfo.a  lib/libLLVMDebugInfoDWARF.a  lib/libLLVMExecutionEngine.a  lib/libLLVMMC.a  lib/libLLVMObject.a  lib/libLLVMRuntimeDyld.a  lib/libLLVMSupport.a  -lpthread  lib/libLLVMAArch64Utils.a  lib/libLLVMAMDGPUUtils.a  lib/libLLVMARMUtils.a  lib/libLLVMWebAssemblyUtils.a  lib/libLLVMCodeGen.a  lib/libLLVMBitWriter.a  lib/libLLVMScalarOpts.a  lib/libLLVMAggressiveInstCombine.a  lib/libLLVMInstCombine.a  lib/libLLVMTransformUtils.a  lib/libLLVMMCDisassembler.a  lib/libLLVMOrcTargetProcess.a  lib/libLLVMOrcShared.a  lib/libLLVMTarget.a  lib/libLLVMAnalysis.a  lib/libLLVMObject.a  lib/libLLVMBitReader.a  lib/libLLVMMCParser.a  lib/libLLVMMC.a  lib/libLLVMDebugInfoCodeView.a  lib/libLLVMTextAPI.a  lib/libLLVMProfileData.a  lib/libLLVMCore.a  lib/libLLVMBinaryFormat.a  lib/libLLVMRemarks.a  lib/libLLVMBitstreamReader.a  lib/libLLVMSupport.a  -lexecinfo  -lpthread  -lm  /usr/lib/libz.so.6.0  -lcurses  lib/libLLVMDemangle.a  -Wl,-rpath-link,/usr/X11R6/lib:/usr/local/lib && :
ld: error: undefined symbol: LLVMRTDyldTLSSpace
>>> referenced by llvm-rtdyld.cpp:372 (/home/brad/llvm/llvm-new/llvm/tools/llvm-rtdyld/llvm-rtdyld.cpp:372)
>>>               tools/llvm-rtdyld/CMakeFiles/llvm-rtdyld.dir/llvm-rtdyld.cpp.o:(TrivialMemoryManager::allocateTLSSection(unsigned long, unsigned int, unsigned int, llvm::StringRef))
c++: error: linker command failed with exit code 1 (use -v to see invocation)

[MoritzS]

In D105466#3107759, @brad wrote:

This commit appears to have broken the build on OpenBSD..

ld: error: undefined symbol: LLVMRTDyldTLSSpace

referenced by llvm-rtdyld.cpp:372 (/home/brad/llvm/llvm-new/llvm/tools/llvm-rtdyld/llvm-rtdyld.cpp:372)

This symbol is only used when defined(__x86_64__) && defined(__ELF__) is true. The definition looks like this:

extern "C" {
alignas(16) __attribute__((visibility("hidden"), tls_model("initial-exec"),
                           used)) thread_local char LLVMRTDyldTLSSpace[16];
}

And the symbol is then directly referenced from inline assembly like this:

asm("leaq LLVMRTDyldTLSSpace@tpoff, %0" : "=r"(TLSOffset));

Does thread-local storage in ELF work differently on OpenBSD than it does on Linux? Is there an easy way to fix it or should I try to ifdef this out for OpenBSD?

llvm/include/llvm/ExecutionEngine/RuntimeDyld.h

public:
     /// Allocate a memory block of (at least) the given size suitable for data.
     /// The SectionID is a unique identifier assigned by the JIT engine, and
     /// optionally recorded by the memory manager to access a loaded section.
     virtual uint8_t *allocateDataSection(uintptr_t Size, unsigned Alignment,
                                          unsigned SectionID,
                                          StringRef SectionName,
                                          bool IsReadOnly) = 0;
 
+    /// An allocated TLS section
+    struct TLSSection {
+      /// The pointer to the initialization image
+      uint8_t *InitializationImage;
+      /// The TLS offset
+      intptr_t Offset;
+    };
+
+    /// Allocate a memory block of (at least) the given size to be used for
+    /// thread-local storage (TLS).
+    virtual TLSSection allocateTLSSection(uintptr_t Size, unsigned Alignment,
+                                          unsigned SectionID,
+                                          StringRef SectionName);
+
     /// Inform the memory manager about the total amount of memory required to
     /// allocate all sections to be loaded:
     /// \p CodeSize - the total size of all code sections
     /// \p DataSizeRO - the total size of all read-only data sections
     /// \p DataSizeRW - the total size of all read-write data sections
     ///
     /// Note that by default the callback is disabled. To enable it
     /// redefine the method needsToReserveAllocationSpace to return true.

llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyld.cpp

return COFFObj->getCOFFSection(Section)->Characteristics &
             COFF::IMAGE_SCN_CNT_UNINITIALIZED_DATA;
 
   auto *MachO = cast<MachOObjectFile>(Obj);
   unsigned SectionType = MachO->getSectionType(Section);
   return SectionType == MachO::S_ZEROFILL ||
          SectionType == MachO::S_GB_ZEROFILL;
 }
 
+static bool isTLS(const SectionRef Section) {
+  const ObjectFile *Obj = Section.getObject();
+  if (isa<object::ELFObjectFileBase>(Obj))
+    return ELFSectionRef(Section).getFlags() & ELF::SHF_TLS;
+  return false;
+}
+
 // Compute an upper bound of the memory size that is required to load all
 // sections
 Error RuntimeDyldImpl::computeTotalAllocSize(const ObjectFile &Obj,
                                              uint64_t &CodeSize,
                                              uint32_t &CodeAlign,
                                              uint64_t &RODataSize,
                                              uint32_t &RODataAlign,
                                              uint64_t &RWDataSize,
for (section_iterator SI = Obj.section_begin(), SE = Obj.section_end();
 
     // Consider only the sections that are required to be loaded for execution
     if (IsRequired) {
       uint64_t DataSize = Section.getSize();
       uint64_t Alignment64 = Section.getAlignment();
       unsigned Alignment = (unsigned)Alignment64 & 0xffffffffL;
       bool IsCode = Section.isText();
       bool IsReadOnly = isReadOnlyData(Section);
+      bool IsTLS = isTLS(Section);
 
       Expected<StringRef> NameOrErr = Section.getName();
       if (!NameOrErr)
         return NameOrErr.takeError();
       StringRef Name = *NameOrErr;
 
       uint64_t StubBufSize = computeSectionStubBufSize(Obj, Section);
 
if (IsRequired) {
         SectionSize = 1;
 
       if (IsCode) {
         CodeAlign = std::max(CodeAlign, Alignment);
         CodeSectionSizes.push_back(SectionSize);
       } else if (IsReadOnly) {
         RODataAlign = std::max(RODataAlign, Alignment);
         ROSectionSizes.push_back(SectionSize);
-      } else {
+      } else if (!IsTLS) {
         RWDataAlign = std::max(RWDataAlign, Alignment);
         RWSectionSizes.push_back(SectionSize);
       }
     }
   }
 
   // Compute Global Offset Table size. If it is not zero we
   // also update alignment, which is equal to a size of a
RuntimeDyldImpl::emitSection(const ObjectFile &Obj,
 
   unsigned Alignment = (unsigned)Alignment64 & 0xffffffffL;
   unsigned PaddingSize = 0;
   unsigned StubBufSize = 0;
   bool IsRequired = isRequiredForExecution(Section);
   bool IsVirtual = Section.isVirtual();
   bool IsZeroInit = isZeroInit(Section);
   bool IsReadOnly = isReadOnlyData(Section);
+  bool IsTLS = isTLS(Section);
   uint64_t DataSize = Section.getSize();
 
   // An alignment of 0 (at least with ELF) is identical to an alignment of 1,
   // while being more "polite".  Other formats do not support 0-aligned sections
   // anyway, so we should guarantee that the alignment is always at least 1.
   Alignment = std::max(1u, Alignment);
 
   Expected<StringRef> NameOrErr = Section.getName();
   if (!NameOrErr)
     return NameOrErr.takeError();
   StringRef Name = *NameOrErr;
 
   StubBufSize = computeSectionStubBufSize(Obj, Section);
 
   // The .eh_frame section (at least on Linux) needs an extra four bytes padded
   // with zeroes added at the end.  For MachO objects, this section has a
   // slightly different name, so this won't have any effect for MachO objects.
   if (Name == ".eh_frame")
     PaddingSize = 4;
 
   uintptr_t Allocate;
   unsigned SectionID = Sections.size();
   uint8_t *Addr;
+  uint64_t LoadAddress = 0;
   const char *pData = nullptr;
 
   // If this section contains any bits (i.e. isn't a virtual or bss section),
   // grab a reference to them.
   if (!IsVirtual && !IsZeroInit) {
     // In either case, set the location of the unrelocated section in memory,
     // since we still process relocations for it even if we're not applying them.
     if (Expected<StringRef> E = Section.getContents())
RuntimeDyldImpl::emitSection(const ObjectFile &Obj,
   }
 
   // Some sections, such as debug info, don't need to be loaded for execution.
   // Process those only if explicitly requested.
   if (IsRequired || ProcessAllSections) {
     Allocate = DataSize + PaddingSize + StubBufSize;
     if (!Allocate)
       Allocate = 1;
-    Addr = IsCode ? MemMgr.allocateCodeSection(Allocate, Alignment, SectionID,
-                                               Name)
-                  : MemMgr.allocateDataSection(Allocate, Alignment, SectionID,
-                                               Name, IsReadOnly);
+    if (IsTLS) {
+      auto TLSSection =
+          MemMgr.allocateTLSSection(Allocate, Alignment, SectionID, Name);
+      Addr = TLSSection.InitializationImage;
+      LoadAddress = TLSSection.Offset;
+    } else if (IsCode) {
+      Addr = MemMgr.allocateCodeSection(Allocate, Alignment, SectionID, Name);
+    } else {
+      Addr = MemMgr.allocateDataSection(Allocate, Alignment, SectionID, Name,
+                                        IsReadOnly);
+    }
     if (!Addr)
       report_fatal_error("Unable to allocate section memory!");
 
     // Zero-initialize or copy the data from the image
     if (IsZeroInit || IsVirtual)
       memset(Addr, 0, DataSize);
     else
       memcpy(Addr, pData, DataSize);
LLVM_DEBUG(
                << " obj addr: " << format("%p", data.data()) << " new addr: 0"
                << " DataSize: " << DataSize << " StubBufSize: " << StubBufSize
                << " Allocate: " << Allocate << "\n");
   }
 
   Sections.push_back(
       SectionEntry(Name, Addr, DataSize, Allocate, (uintptr_t)pData));
 
+  // The load address of a TLS section is not equal to the address of its
+  // initialization image
+  if (IsTLS)
+    Sections.back().setLoadAddress(LoadAddress);
   // Debug info sections are linked as if their load address was zero
   if (!IsRequired)
     Sections.back().setLoadAddress(0);
 
   return SectionID;
 }
 
 Expected<unsigned>
uint64_t RuntimeDyld::LoadedObjectInfo::getSectionLoadAddress(
 
   auto I = ObjSecToIDMap.find(Sec);
   if (I != ObjSecToIDMap.end())
     return RTDyld.Sections[I->second].getLoadAddress();
 
   return 0;
 }
 
+RuntimeDyld::MemoryManager::TLSSection
+RuntimeDyld::MemoryManager::allocateTLSSection(uintptr_t Size,
+                                               unsigned Alignment,
+                                               unsigned SectionID,
+                                               StringRef SectionName) {
+  report_fatal_error("allocation of TLS not implemented");
+}
+
 void RuntimeDyld::MemoryManager::anchor() {}
 void JITSymbolResolver::anchor() {}
 void LegacyJITSymbolResolver::anchor() {}
 
 RuntimeDyld::RuntimeDyld(RuntimeDyld::MemoryManager &MemMgr,
                          JITSymbolResolver &Resolver)
     : MemMgr(MemMgr), Resolver(Resolver) {
   // FIXME: There's a potential issue lurking here if a single instance of

llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldELF.h

private:
   SmallVector<SID, 2> UnregisteredEHFrameSections;
 
   // Map between GOT relocation value and corresponding GOT offset
   std::map<RelocationValueRef, uint64_t> GOTOffsetMap;
 
   bool relocationNeedsGot(const RelocationRef &R) const override;
   bool relocationNeedsStub(const RelocationRef &R) const override;
 
+  // Process a GOTTPOFF TLS relocation for x86-64
+  // NOLINTNEXTLINE(readability-identifier-naming)
+  void processX86_64GOTTPOFFRelocation(unsigned SectionID, uint64_t Offset,
+                                       RelocationValueRef Value,
+                                       int64_t Addend);
+  // Process a TLSLD/TLSGD relocation for x86-64
+  // NOLINTNEXTLINE(readability-identifier-naming)
+  void processX86_64TLSRelocation(unsigned SectionID, uint64_t Offset,
+                                  uint64_t RelType, RelocationValueRef Value,
+                                  int64_t Addend,
+                                  const RelocationRef &GetAddrRelocation);
+
 public:
   RuntimeDyldELF(RuntimeDyld::MemoryManager &MemMgr,
                  JITSymbolResolver &Resolver);
   ~RuntimeDyldELF() override;
 
   static std::unique_ptr<RuntimeDyldELF>
   create(Triple::ArchType Arch, RuntimeDyld::MemoryManager &MemMgr,
          JITSymbolResolver &Resolver);

llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldELF.cpp

for (const auto &Section : Sections) {
         break;
       }
     }
     assert(GOTBase != 0 && "missing GOT");
     int64_t GOTOffset = Value - GOTBase + Addend;
     support::ulittle64_t::ref(Section.getAddressWithOffset(Offset)) = GOTOffset;
     break;
   }
+  case ELF::R_X86_64_DTPMOD64: {
+    // We only have one DSO, so the module id is always 1.
+    support::ulittle64_t::ref(Section.getAddressWithOffset(Offset)) = 1;
+    break;
+  }
+  case ELF::R_X86_64_DTPOFF64:
+  case ELF::R_X86_64_TPOFF64: {
+    // DTPOFF64 should resolve to the offset in the TLS block, TPOFF64 to the
+    // offset in the *initial* TLS block. Since we are statically linking, all
+    // TLS blocks already exist in the initial block, so resolve both
+    // relocations equally.
+    support::ulittle64_t::ref(Section.getAddressWithOffset(Offset)) =
+        Value + Addend;
+    break;
+  }
+  case ELF::R_X86_64_DTPOFF32:
+  case ELF::R_X86_64_TPOFF32: {
+    // As for the (D)TPOFF64 relocations above, both DTPOFF32 and TPOFF32 can
+    // be resolved equally.
+    int64_t RealValue = Value + Addend;
+    assert(RealValue >= INT32_MIN && RealValue <= INT32_MAX);
+    int32_t TruncValue = RealValue;
+    support::ulittle32_t::ref(Section.getAddressWithOffset(Offset)) =
+        TruncValue;
+    break;
+  }
   }
 }
 
 void RuntimeDyldELF::resolveX86Relocation(const SectionEntry &Section,
                                           uint64_t Offset, uint32_t Value,
                                           uint32_t Type, int32_t Addend) {
   switch (Type) {
   case ELF::R_386_32: {
if (RelType == ELF::R_X86_64_PLT32) {
       (void)allocateGOTEntries(0);
       processSimpleRelocation(SectionID, Offset, RelType, Value);
     } else if (RelType == ELF::R_X86_64_PC32) {
       Value.Addend += support::ulittle32_t::ref(computePlaceholderAddress(SectionID, Offset));
       processSimpleRelocation(SectionID, Offset, RelType, Value);
     } else if (RelType == ELF::R_X86_64_PC64) {
       Value.Addend += support::ulittle64_t::ref(computePlaceholderAddress(SectionID, Offset));
       processSimpleRelocation(SectionID, Offset, RelType, Value);
+    } else if (RelType == ELF::R_X86_64_GOTTPOFF) {
+      processX86_64GOTTPOFFRelocation(SectionID, Offset, Value, Addend);
+    } else if (RelType == ELF::R_X86_64_TLSGD ||
+               RelType == ELF::R_X86_64_TLSLD) {
+      // The next relocation must be the relocation for __tls_get_addr.
+      ++RelI;
+      auto &GetAddrRelocation = *RelI;
+      processX86_64TLSRelocation(SectionID, Offset, RelType, Value, Addend,
+                                 GetAddrRelocation);
     } else {
       processSimpleRelocation(SectionID, Offset, RelType, Value);
     }
   } else {
     if (Arch == Triple::x86) {
       Value.Addend += support::ulittle32_t::ref(computePlaceholderAddress(SectionID, Offset));
     }
     processSimpleRelocation(SectionID, Offset, RelType, Value);
   }
   return ++RelI;
 }
 
+void RuntimeDyldELF::processX86_64GOTTPOFFRelocation(unsigned SectionID,
+                                                     uint64_t Offset,
+                                                     RelocationValueRef Value,
+                                                     int64_t Addend) {
+  // Use the approach from "x86-64 Linker Optimizations" from the TLS spec
+  // to replace the GOTTPOFF relocation with a TPOFF relocation. The spec
+  // only mentions one optimization even though there are two different
+  // code sequences for the Initial Exec TLS Model. We match the code to
+  // find out which one was used.
+
+  // A possible TLS code sequence and its replacement
+  struct CodeSequence {
+    // The expected code sequence
+    ArrayRef<uint8_t> ExpectedCodeSequence;
+    // The negative offset of the GOTTPOFF relocation to the beginning of
+    // the sequence
+    uint64_t TLSSequenceOffset;
+    // The new code sequence
+    ArrayRef<uint8_t> NewCodeSequence;
+    // The offset of the new TPOFF relocation
+    uint64_t TpoffRelocationOffset;
+  };
+
+  std::array<CodeSequence, 2> CodeSequences;
+
+  // Initial Exec Code Model Sequence
+  {
+    static const std::initializer_list<uint8_t> ExpectedCodeSequenceList = {
+        0x64, 0x48, 0x8b, 0x04, 0x25, 0x00, 0x00, 0x00,
+        0x00,                                    // mov %fs:0, %rax
+        0x48, 0x03, 0x05, 0x00, 0x00, 0x00, 0x00 // add x@gotpoff(%rip),
+                                                 // %rax
+    };
+    CodeSequences[0].ExpectedCodeSequence =
+        ArrayRef<uint8_t>(ExpectedCodeSequenceList);
+    CodeSequences[0].TLSSequenceOffset = 12;
+
+    static const std::initializer_list<uint8_t> NewCodeSequenceList = {
+        0x64, 0x48, 0x8b, 0x04, 0x25, 0x00, 0x00, 0x00, 0x00, // mov %fs:0, %rax
+        0x48, 0x8d, 0x80, 0x00, 0x00, 0x00, 0x00 // lea x@tpoff(%rax), %rax
+    };
+    CodeSequences[0].NewCodeSequence = ArrayRef<uint8_t>(NewCodeSequenceList);
+    CodeSequences[0].TpoffRelocationOffset = 12;
+  }
+
+  // Initial Exec Code Model Sequence, II
+  {
+    static const std::initializer_list<uint8_t> ExpectedCodeSequenceList = {
+        0x48, 0x8b, 0x05, 0x00, 0x00, 0x00, 0x00, // mov x@gotpoff(%rip), %rax
+        0x64, 0x48, 0x8b, 0x00, 0x00, 0x00, 0x00  // mov %fs:(%rax), %rax
+    };
+    CodeSequences[1].ExpectedCodeSequence =
+        ArrayRef<uint8_t>(ExpectedCodeSequenceList);
+    CodeSequences[1].TLSSequenceOffset = 3;
+
+    static const std::initializer_list<uint8_t> NewCodeSequenceList = {
+        0x66, 0x0f, 0x1f, 0x44, 0x00, 0x00,             // 6 byte nop
+        0x64, 0x8b, 0x04, 0x25, 0x00, 0x00, 0x00, 0x00, // mov %fs:x@tpoff, %rax
+    };
+    CodeSequences[1].NewCodeSequence = ArrayRef<uint8_t>(NewCodeSequenceList);
+    CodeSequences[1].TpoffRelocationOffset = 10;
+  }
+
+  bool Resolved = false;
+  auto &Section = Sections[SectionID];
+  for (const auto &C : CodeSequences) {
+    assert(C.ExpectedCodeSequence.size() == C.NewCodeSequence.size() &&
+           "Old and new code sequences must have the same size");
+
+    if (Offset < C.TLSSequenceOffset ||
+        (Offset - C.TLSSequenceOffset + C.NewCodeSequence.size()) >
+            Section.getSize()) {
+      // This can't be a matching sequence as it doesn't fit in the current
+      // section
+      continue;
+    }
+
+    auto TLSSequenceStartOffset = Offset - C.TLSSequenceOffset;
+    auto *TLSSequence = Section.getAddressWithOffset(TLSSequenceStartOffset);
+    if (ArrayRef<uint8_t>(TLSSequence, C.ExpectedCodeSequence.size()) !=
+        C.ExpectedCodeSequence) {
+      continue;
+    }
+
+    memcpy(TLSSequence, C.NewCodeSequence.data(), C.NewCodeSequence.size());
+
+    // The original GOTTPOFF relocation has an addend as it is PC relative,
+    // so it needs to be corrected. The TPOFF32 relocation is used as an
+    // absolute value (which is an offset from %fs:0), so remove the addend
+    // again.
+    RelocationEntry RE(SectionID,
+                       TLSSequenceStartOffset + C.TpoffRelocationOffset,
+                       ELF::R_X86_64_TPOFF32, Value.Addend - Addend);
+
+    if (Value.SymbolName)
+      addRelocationForSymbol(RE, Value.SymbolName);
+    else
+      addRelocationForSection(RE, Value.SectionID);
+
+    Resolved = true;
+    break;
+  }
+
+  if (!Resolved) {
+    // The GOTTPOFF relocation was not used in one of the sequences
+    // described in the spec, so we can't optimize it to a TPOFF
+    // relocation.
+    uint64_t GOTOffset = allocateGOTEntries(1);
+    resolveGOTOffsetRelocation(SectionID, Offset, GOTOffset + Addend,
+                               ELF::R_X86_64_PC32);
+    RelocationEntry RE =
+        computeGOTOffsetRE(GOTOffset, Value.Offset, ELF::R_X86_64_TPOFF64);
+    if (Value.SymbolName)
+      addRelocationForSymbol(RE, Value.SymbolName);
+    else
+      addRelocationForSection(RE, Value.SectionID);
+  }
+}
+
+void RuntimeDyldELF::processX86_64TLSRelocation(
+    unsigned SectionID, uint64_t Offset, uint64_t RelType,
+    RelocationValueRef Value, int64_t Addend,
+    const RelocationRef &GetAddrRelocation) {
+  // Since we are statically linking and have no additional DSOs, we can resolve
+  // the relocation directly without using __tls_get_addr.
+  // Use the approach from "x86-64 Linker Optimizations" from the TLS spec
+  // to replace it with the Local Exec relocation variant.
+
+  // Find out whether the code was compiled with the large or small memory
+  // model. For this we look at the next relocation which is the relocation
+  // for the __tls_get_addr function. If it's a 32 bit relocation, it's the
+  // small code model, with a 64 bit relocation it's the large code model.
+  bool IsSmallCodeModel;
+  // Is the relocation for the __tls_get_addr a PC-relative GOT relocation?
+  bool IsGOTPCRel = false;
+
+  switch (GetAddrRelocation.getType()) {
+  case ELF::R_X86_64_GOTPCREL:
+  case ELF::R_X86_64_REX_GOTPCRELX:
+  case ELF::R_X86_64_GOTPCRELX:
+    IsGOTPCRel = true;
+    LLVM_FALLTHROUGH;
+  case ELF::R_X86_64_PLT32:
+    IsSmallCodeModel = true;
+    break;
+  case ELF::R_X86_64_PLTOFF64:
+    IsSmallCodeModel = false;
+    break;
+  default:
+    report_fatal_error(
+        "invalid TLS relocations for General/Local Dynamic TLS Model: "
+        "expected PLT or GOT relocation for __tls_get_addr function");
+  }
+
+  // The negative offset to the start of the TLS code sequence relative to
+  // the offset of the TLSGD/TLSLD relocation
+  uint64_t TLSSequenceOffset;
+  // The expected start of the code sequence
+  ArrayRef<uint8_t> ExpectedCodeSequence;
+  // The new TLS code sequence that will replace the existing code
+  ArrayRef<uint8_t> NewCodeSequence;
+
+  if (RelType == ELF::R_X86_64_TLSGD) {
+    // The offset of the new TPOFF32 relocation (offset starting from the
+    // beginning of the whole TLS sequence)
+    uint64_t TpoffRelocOffset;
+
+    if (IsSmallCodeModel) {
+      if (!IsGOTPCRel) {
+        static const std::initializer_list<uint8_t> CodeSequence = {
+            0x66, // data16 (no-op prefix)
+            0x48, 0x8d, 0x3d, 0x00, 0x00,
+            0x00, 0x00,                  // lea <disp32>(%rip), %rdi
+            0x66, 0x66,                  // two data16 prefixes
+            0x48,                        // rex64 (no-op prefix)
+            0xe8, 0x00, 0x00, 0x00, 0x00 // call __tls_get_addr@plt
+        };
+        ExpectedCodeSequence = ArrayRef<uint8_t>(CodeSequence);
+        TLSSequenceOffset = 4;
+      } else {
+        // This code sequence is not described in the TLS spec but gcc
+        // generates it sometimes.
+        static const std::initializer_list<uint8_t> CodeSequence = {
+            0x66, // data16 (no-op prefix)
+            0x48, 0x8d, 0x3d, 0x00, 0x00,
+            0x00, 0x00, // lea <disp32>(%rip), %rdi
+            0x66,       // data16 prefix (no-op prefix)
+            0x48,       // rex64 (no-op prefix)
+            0xff, 0x15, 0x00, 0x00, 0x00,
+            0x00 // call *__tls_get_addr@gotpcrel(%rip)
+        };
+        ExpectedCodeSequence = ArrayRef<uint8_t>(CodeSequence);
+        TLSSequenceOffset = 4;
+      }
+
+      // The replacement code for the small code model. It's the same for
+      // both sequences.
+      static const std::initializer_list<uint8_t> SmallSequence = {
+          0x64, 0x48, 0x8b, 0x04, 0x25, 0x00, 0x00, 0x00,
+          0x00,                                    // mov %fs:0, %rax
+          0x48, 0x8d, 0x80, 0x00, 0x00, 0x00, 0x00 // lea x@tpoff(%rax),
+                                                   // %rax
+      };
+      NewCodeSequence = ArrayRef<uint8_t>(SmallSequence);
+      TpoffRelocOffset = 12;
+    } else {
+      static const std::initializer_list<uint8_t> CodeSequence = {
+          0x48, 0x8d, 0x3d, 0x00, 0x00, 0x00, 0x00, // lea <disp32>(%rip),
+                                                    // %rdi
+          0x48, 0xb8, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+          0x00,             // movabs $__tls_get_addr@pltoff, %rax
+          0x48, 0x01, 0xd8, // add %rbx, %rax
+          0xff, 0xd0        // call *%rax
+      };
+      ExpectedCodeSequence = ArrayRef<uint8_t>(CodeSequence);
+      TLSSequenceOffset = 3;
+
+      // The replacement code for the large code model
+      static const std::initializer_list<uint8_t> LargeSequence = {
+          0x64, 0x48, 0x8b, 0x04, 0x25, 0x00, 0x00, 0x00,
+          0x00,                                     // mov %fs:0, %rax
+          0x48, 0x8d, 0x80, 0x00, 0x00, 0x00, 0x00, // lea x@tpoff(%rax),
+                                                    // %rax
+          0x66, 0x0f, 0x1f, 0x44, 0x00, 0x00        // nopw 0x0(%rax,%rax,1)
+      };
+      NewCodeSequence = ArrayRef<uint8_t>(LargeSequence);
+      TpoffRelocOffset = 12;
+    }
+
+    // The TLSGD/TLSLD relocations are PC-relative, so they have an addend.
+    // The new TPOFF32 relocations is used as an absolute offset from
+    // %fs:0, so remove the TLSGD/TLSLD addend again.
+    RelocationEntry RE(SectionID, Offset - TLSSequenceOffset + TpoffRelocOffset,
+                       ELF::R_X86_64_TPOFF32, Value.Addend - Addend);
+    if (Value.SymbolName)
+      addRelocationForSymbol(RE, Value.SymbolName);
+    else
+      addRelocationForSection(RE, Value.SectionID);
+  } else if (RelType == ELF::R_X86_64_TLSLD) {
+    if (IsSmallCodeModel) {
+      if (!IsGOTPCRel) {
+        static const std::initializer_list<uint8_t> CodeSequence = {
+            0x48, 0x8d, 0x3d, 0x00, 0x00, 0x00, // leaq <disp32>(%rip), %rdi
+            0x00, 0xe8, 0x00, 0x00, 0x00, 0x00  // call __tls_get_addr@plt
+        };
+        ExpectedCodeSequence = ArrayRef<uint8_t>(CodeSequence);
+        TLSSequenceOffset = 3;
+
+        // The replacement code for the small code model
+        static const std::initializer_list<uint8_t> SmallSequence = {
+            0x66, 0x66, 0x66, // three data16 prefixes (no-op)
+            0x64, 0x48, 0x8b, 0x04, 0x25,
+            0x00, 0x00, 0x00, 0x00 // mov %fs:0, %rax
+        };
+        NewCodeSequence = ArrayRef<uint8_t>(SmallSequence);
+      } else {
+        // This code sequence is not described in the TLS spec but gcc
+        // generates it sometimes.
+        static const std::initializer_list<uint8_t> CodeSequence = {
+            0x48, 0x8d, 0x3d, 0x00,
+            0x00, 0x00, 0x00, // leaq <disp32>(%rip), %rdi
+            0xff, 0x15, 0x00, 0x00,
+            0x00, 0x00 // call
+                       // *__tls_get_addr@gotpcrel(%rip)
+        };
+        ExpectedCodeSequence = ArrayRef<uint8_t>(CodeSequence);
+        TLSSequenceOffset = 3;
+
+        // The replacement is code is just like above but it needs to be
+        // one byte longer.
+        static const std::initializer_list<uint8_t> SmallSequence = {
+            0x0f, 0x1f, 0x40, 0x00, // 4 byte nop
+            0x64, 0x48, 0x8b, 0x04, 0x25,
+            0x00, 0x00, 0x00, 0x00 // mov %fs:0, %rax
+        };
+        NewCodeSequence = ArrayRef<uint8_t>(SmallSequence);
+      }
+    } else {
+      // This is the same sequence as for the TLSGD sequence with the large
+      // memory model above
+      static const std::initializer_list<uint8_t> CodeSequence = {
+          0x48, 0x8d, 0x3d, 0x00, 0x00, 0x00, 0x00, // lea <disp32>(%rip),
+                                                    // %rdi
+          0x48, 0xb8, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+          0x48,       // movabs $__tls_get_addr@pltoff, %rax
+          0x01, 0xd8, // add %rbx, %rax
+          0xff, 0xd0  // call *%rax
+      };
+      ExpectedCodeSequence = ArrayRef<uint8_t>(CodeSequence);
+      TLSSequenceOffset = 3;
+
+      // The replacement code for the large code model
+      static const std::initializer_list<uint8_t> LargeSequence = {
+          0x66, 0x66, 0x66, // three data16 prefixes (no-op)
+          0x66, 0x66, 0x0f, 0x1f, 0x84, 0x00, 0x00, 0x00, 0x00,
+          0x00,                                                // 10 byte nop
+          0x64, 0x48, 0x8b, 0x04, 0x25, 0x00, 0x00, 0x00, 0x00 // mov %fs:0,%rax
+      };
+      NewCodeSequence = ArrayRef<uint8_t>(LargeSequence);
+    }
+  } else {
+    llvm_unreachable("both TLS relocations handled above");
+  }
+
+  assert(ExpectedCodeSequence.size() == NewCodeSequence.size() &&
+         "Old and new code sequences must have the same size");
+
+  auto &Section = Sections[SectionID];
+  if (Offset < TLSSequenceOffset ||
+      (Offset - TLSSequenceOffset + NewCodeSequence.size()) >
+          Section.getSize()) {
+    report_fatal_error("unexpected end of section in TLS sequence");
+  }
+
+  auto *TLSSequence = Section.getAddressWithOffset(Offset - TLSSequenceOffset);
+  if (ArrayRef<uint8_t>(TLSSequence, ExpectedCodeSequence.size()) !=
+      ExpectedCodeSequence) {
+    report_fatal_error(
+        "invalid TLS sequence for Global/Local Dynamic TLS Model");
+  }
+
+  memcpy(TLSSequence, NewCodeSequence.data(), NewCodeSequence.size());
+}
+
 size_t RuntimeDyldELF::getGOTEntrySize() {
   // We don't use the GOT in all of these cases, but it's essentially free
   // to put them all here.
   size_t Result = 0;
   switch (Arch) {
   case Triple::x86_64:
   case Triple::aarch64:
   case Triple::aarch64_be:

llvm/test/ExecutionEngine/RuntimeDyld/X86/TLS.s

+# REQUIRES: x86_64-linux
+# RUN: rm -rf %t && mkdir -p %t
+# RUN: llvm-mc -triple=x86_64-unknown-linux -filetype=obj -o %t/tls.o %s
+# RUN: llvm-rtdyld -triple=x86_64-unknown-linux -execute %t/tls.o
+
+
+_main:
+
+  push %rbx
+  # load the address of the GOT in rbx for the large code model tests
+  lea _GLOBAL_OFFSET_TABLE_(%rip), %rbx
+
+# Test Local Exec TLS Model
+  mov %fs:tls_foo@tpoff, %eax
+  cmp $0x12, %eax
+  je 1f
+  mov $1, %eax
+  jmp 2f
+1:
+
+  mov %fs:tls_bar@tpoff, %eax
+  cmp $0x34, %eax
+  je 1f
+  mov $2, %eax
+  jmp 2f
+1:
+
+# Test Initial Exec TLS Model
+  mov tls_foo@gottpoff(%rip), %rax
+  mov %fs:(%rax), %eax
+  cmp $0x12, %eax
+  je 1f
+  mov $3, %eax
+  jmp 2f
+1:
+
+  mov tls_bar@gottpoff(%rip), %rax
+  mov %fs:(%rax), %eax
+  cmp $0x34, %eax
+  je 1f
+  mov $4, %eax
+  jmp 2f
+1:
+
+# Test Local Dynamic TLS Model (small code model)
+  lea tls_foo@tlsld(%rip), %rdi
+  call __tls_get_addr@plt
+  mov tls_foo@dtpoff(%rax), %eax
+  cmp $0x12, %eax
+  je 1f
+  mov $5, %eax
+  jmp 2f
+1:
+
+  lea tls_bar@tlsld(%rip), %rdi
+  call __tls_get_addr@plt
+  mov tls_bar@dtpoff(%rax), %eax
+  cmp $0x34, %eax
+  je 1f
+  mov $6, %eax
+  jmp 2f
+1:
+
+# Test Local Dynamic TLS Model (large code model)
+  lea tls_foo@tlsld(%rip), %rdi
+  movabs $__tls_get_addr@pltoff, %rax
+  add %rbx, %rax
+  call *%rax
+  mov tls_foo@dtpoff(%rax), %eax
+  cmp $0x12, %eax
+  je 1f
+  mov $7, %eax
+  jmp 2f
+1:
+
+  lea tls_bar@tlsld(%rip), %rdi
+  movabs $__tls_get_addr@pltoff, %rax
+  add %rbx, %rax
+  call *%rax
+  mov tls_bar@dtpoff(%rax), %eax
+  cmp $0x34, %eax
+  je 1f
+  mov $8, %eax
+  jmp 2f
+1:
+
+# Test Global Dynamic TLS Model (small code model)
+  .byte 0x66
+  leaq tls_foo@tlsgd(%rip), %rdi
+  .byte 0x66, 0x66, 0x48
+  call __tls_get_addr@plt
+  mov (%rax), %eax
+  cmp $0x12, %eax
+  je 1f
+  mov $9, %eax
+  jmp 2f
+1:
+
+  .byte 0x66
+  leaq tls_bar@tlsgd(%rip), %rdi
+  .byte 0x66, 0x66, 0x48
+  call __tls_get_addr@plt
+  mov (%rax), %eax
+  cmp $0x34, %eax
+  je 1f
+  mov $10, %eax
+  jmp 2f
+1:
+
+# Test Global Dynamic TLS Model (large code model)
+  lea tls_foo@tlsgd(%rip), %rdi
+  movabs $__tls_get_addr@pltoff, %rax
+  add %rbx, %rax
+  call *%rax
+  mov (%rax), %eax
+  cmp $0x12, %eax
+  je 1f
+  mov $11, %eax
+  jmp 2f
+1:
+
+  lea tls_bar@tlsgd(%rip), %rdi
+  movabs $__tls_get_addr@pltoff, %rax
+  add %rbx, %rax
+  call *%rax
+  mov (%rax), %eax
+  cmp $0x34, %eax
+  je 1f
+  mov $12, %eax
+  jmp 2f
+1:
+
+  xor %eax, %eax
+
+2:
+  pop %rbx
+  ret
+
+
+  .section .tdata, "awT", @progbits
+
+  .global tls_foo
+  .type tls_foo, @object
+  .size tls_foo, 4
+  .align 4
+tls_foo:
+  .long 0x12
+
+  .global tls_bar
+  .type tls_bar, @object
+  .size tls_bar, 4
+  .align 4
+tls_bar:
+  .long 0x34

llvm/tools/llvm-rtdyld/llvm-rtdyld.cpp

public:
   SmallVector<SectionInfo, 16> DataMemory;
 
   uint8_t *allocateCodeSection(uintptr_t Size, unsigned Alignment,
                                unsigned SectionID,
                                StringRef SectionName) override;
   uint8_t *allocateDataSection(uintptr_t Size, unsigned Alignment,
                                unsigned SectionID, StringRef SectionName,
                                bool IsReadOnly) override;
+  TrivialMemoryManager::TLSSection
+  allocateTLSSection(uintptr_t Size, unsigned Alignment, unsigned SectionID,
+                     StringRef SectionName) override;
 
   /// If non null, records subsequent Name -> SectionID mappings.
   void setSectionIDsMap(SectionIDMap *SecIDMap) {
     this->SecIDMap = SecIDMap;
   }
 
   void *getPointerToNamedFunction(const std::string &Name,
                                   bool AbortOnFailure = true) override {

 
 private:
   std::map<std::string, uint64_t> DummyExterns;
   sys::MemoryBlock PreallocSlab;
   bool UsePreallocation = false;
   uintptr_t SlabSize = 0;
   uintptr_t CurrentSlabOffset = 0;
   SectionIDMap *SecIDMap = nullptr;
+  unsigned UsedTLSStorage = 0;
 };
 
 uint8_t *TrivialMemoryManager::allocateCodeSection(uintptr_t Size,
                                                    unsigned Alignment,
                                                    unsigned SectionID,
                                                    StringRef SectionName) {
   if (PrintAllocationRequests)
     outs() << "allocateCodeSection(Size = " << Size << ", Alignment = "
sys::Memory::allocateMappedMemory(Size, nullptr,
                                       sys::Memory::MF_WRITE,
                                       EC);
   if (!MB.base())
     report_fatal_error("MemoryManager allocation failed: " + EC.message());
   DataMemory.push_back(SectionInfo(SectionName, MB, SectionID));
   return (uint8_t*)MB.base();
 }
 
+// In case the execution needs TLS storage, we define a very small TLS memory
+// area here that will be used in allocateTLSSection().
+#if defined(__x86_64__) && defined(__ELF__)
+extern "C" {
+alignas(16) __attribute__((visibility("hidden"), tls_model("initial-exec"),
+                           used)) thread_local char LLVMRTDyldTLSSpace[16];
+}
+#endif
+
+TrivialMemoryManager::TLSSection
+TrivialMemoryManager::allocateTLSSection(uintptr_t Size, unsigned Alignment,
+                                         unsigned SectionID,
+                                         StringRef SectionName) {
+#if defined(__x86_64__) && defined(__ELF__)
+  if (Size + UsedTLSStorage > sizeof(LLVMRTDyldTLSSpace)) {
+    return {};
+  }
+
+  // Get the offset of the TLSSpace in the TLS block by using a tpoff
+  // relocation here.
+  int64_t TLSOffset;
+  asm("leaq LLVMRTDyldTLSSpace@tpoff, %0" : "=r"(TLSOffset));
+
+  TLSSection Section;
+  // We use the storage directly as the initialization image. This means that
+  // when a new thread is spawned after this allocation, it will not be
+  // initialized correctly. This means, llvm-rtdyld will only support TLS in a
+  // single thread.
+  Section.InitializationImage =
+      reinterpret_cast<uint8_t *>(LLVMRTDyldTLSSpace + UsedTLSStorage);
+  Section.Offset = TLSOffset + UsedTLSStorage;
+
+  UsedTLSStorage += Size;
+
+  return Section;
+#else
+  return {};
+#endif
+}
+
 static const char *ProgramName;
 
 static void ErrorAndExit(const Twine &Msg) {
   errs() << ProgramName << ": error: " << Msg << "\n";
   exit(1);
 }
 
 static void loadDylibs() {