[JITLink] Add initial native TLS support to ELFNix platform

[StephenFan]

Summary

This patch use the same way as the https://reviews.llvm.org/rGfe1fa43f16beac1506a2e73a9f7b3c81179744eb to handle the thread local variable.

It allocates 2 * pointerSize space in GOT to represent the thread key and data address. Instead of using the _tls_get_addr function, I customed a function __orc_rt_elfnix_tls_get_addr to get the address of thread local varible. Currently, this is a wip patch, only one TLS relocation R_X86_64_TLSGD is supported and I need to add the corresponding test cases.

To allocate the TLS descriptor in GOT, I need to get the edge kind information in PerGraphGOTAndPLTStubBuilder, So I add a Edge::Kind K argument in some functions in PerGraphGOTAndPLTStubBuilder.h. If it is not suitable, I can think further to solve this problem.

[StephenFan]

Improve comments

[MaskRay]

TLS descriptors refer to an alternative ABI to traditional general dynamic/local dynamic TLS models. The name cannot be repurposed to a different usage.

[lhames]

To allocate the TLS descriptor in GOT, I need to get the edge kind information in PerGraphGOTAndPLTStubBuilder, So I add a Edge::Kind K argument in some functions in PerGraphGOTAndPLTStubBuilder.h. If it is not
suitable, I can think further to solve this problem.

Is there a good reason to put these in the GOT? I would create and manage a different section for these, rather than re-using the GOT.

Side note: Re-using the GOT in MachO is safe, but was a bit lazy. We could probably come up with a generic TableSection<T> utility that we could re-use for GOTs, PLTs, and these new thread data structures.

In D109293#2984221, @MaskRay wrote:

TLS descriptors refer to an alternative ABI to traditional general dynamic/local dynamic TLS models. The name cannot be repurposed to a different usage.

LinkGraphs are not ELF graphs so we're free to redefine terms to a certain extent, but I agree that it's good to avoid confusion where possible. Do you have a a suggested alternative?

compiler-rt/lib/orc/elfnix_platform.cpp

113

The capitalization of THread should be fixed here.

256

The capitalization of THread should be fixed here too.

[MoritzS]

In D109293#2984338, @lhames wrote:

To allocate the TLS descriptor in GOT, I need to get the edge kind information in PerGraphGOTAndPLTStubBuilder, So I add a Edge::Kind K argument in some functions in PerGraphGOTAndPLTStubBuilder.h. If it is not
suitable, I can think further to solve this problem.

Is there a good reason to put these in the GOT? I would create and manage a different section for these, rather than re-using the GOT.

The ELF TLS spec actually describes using the GOT for that. Each TLSGD relocation is usually converted into two adjacent GOT entries with the DTPMOD64 and DTPOFF64 relocations. I think the practical reason for that is that a TLSGD relocation is 32-bit PC-relative, so using an address of the GOT will guarantee that the 32 bit offset can represent the address.

In general I would suggest not trying to resolve a TLSGD/LD relocation at all but instead converting it to a GOTTPOFF relocation. Then, no runtime function to implement __tls_get_addr is needed. You can find the detailed description of how to do this in Section 5.5 of the ELF TLS spec (https://akkadia.org/drepper/tls.pdf).

I implemented this for RuntimeDyld (D105466) so you can take a look there. Let me know where I can help!

[lhames]

In D109293#2984957, @MoritzS wrote:

The ELF TLS spec actually describes using the GOT for that.

I need to find some time to read the ELF TLS spec. Unless use of the GOT is required (which seems unlikely) I'd rather put these in their own section. In the unlikely event that they really need to be mixed with GOT entries we should spin out an ELF-specific GOT pass to handle this.

I think the practical reason for that is that a TLSGD relocation is 32-bit PC-relative, so using an address of the GOT will guarantee that the 32 bit offset can represent the address.

We can count on JITLink (and the memory manager) to do this for us. It's designed to address RuntimeDyld::MemoryManager's shortcomings in that regard.

In general I would suggest not trying to resolve a TLSGD/LD relocation at all but instead converting it to a GOTTPOFF relocation. Then, no runtime function to implement __tls_get_addr is needed.

Is that compatible with adding extra code at runtime? I suspect we'll need to go the other way: convert things to function calls by default, but make it possible to use direct models where they're safe. That's speculation having not read the TLS spec though.

[MoritzS]

In D109293#2985202, @lhames wrote:

In D109293#2984957, @MoritzS wrote:

In general I would suggest not trying to resolve a TLSGD/LD relocation at all but instead converting it to a GOTTPOFF relocation. Then, no runtime function to implement __tls_get_addr is needed.

Is that compatible with adding extra code at runtime? I suspect we'll need to go the other way: convert things to function calls by default, but make it possible to use direct models where they're safe. That's speculation having not read the TLS spec though.

That depends on how the allocation of TLS sections is implemented, but in general you are right, I didn't think about that.

The problem I faced when implementing the TLS relocations is that we can't chose which types of TLS relocations the object files we try to link use. This means that we need to implement the GOTTPOFF/TPOFF relocations that do not go through the indirection of __tls_get_addr. They just resolve to an offset that will be added to the value stored in %fs:0. This is usually set initially by ld.so when a process starts by using arch_prctl(ARCH_SET_FS), or for newer CPUS by using the wrfsbase instruction. This means that we can't easily extend the storage of the process that runs the runtime linker to add new TLS sections since messing with the fs register will very likely lead to unexpected results. There are several possible solutions for that with different disadvantages:

1. Don't allow GOTTPOFF/TPOFF relocations at all. This means that we will not be able to link many pre-compiled static libraries.
2. Allow all TLS relocations. Since we can't add new TLS storage at runtime, define a global thread-local variable which will then be used by the linker to resolve GOTTPOFF/TPOFF relocations.
3. Rewrite the relocated code so that it does not use the fs register. On Linux we could probably replace all references to fs by gs and then manually set the gs register? I'm not sure if this can break code if we change instructions that did not originally have a relocation.

I implemented 2. in llvm-rtdyld. Since this is mainly used for testing, the size of the pre-allocated TLS section is only 16 B. In a "real" program you could easily allocate an entire page without noticing any runtime overhead. This is more than enough to link an entire static glibc, for example.

[MoritzS]

In D109293#2985435, @MoritzS wrote:

In D109293#2985202, @lhames wrote:

In D109293#2984957, @MoritzS wrote:

In general I would suggest not trying to resolve a TLSGD/LD relocation at all but instead converting it to a GOTTPOFF relocation. Then, no runtime function to implement __tls_get_addr is needed.

Is that compatible with adding extra code at runtime? I suspect we'll need to go the other way: convert things to function calls by default, but make it possible to use direct models where they're safe. That's speculation having not read the TLS spec though.

That depends on how the allocation of TLS sections is implemented, but in general you are right, I didn't think about that.

The problem I faced when implementing the TLS relocations is that we can't chose which types of TLS relocations the object files we try to link use. This means that we need to implement the GOTTPOFF/TPOFF relocations that do not go through the indirection of __tls_get_addr. They just resolve to an offset that will be added to the value stored in %fs:0. This is usually set initially by ld.so when a process starts by using arch_prctl(ARCH_SET_FS), or for newer CPUS by using the wrfsbase instruction. This means that we can't easily extend the storage of the process that runs the runtime linker to add new TLS sections since messing with the fs register will very likely lead to unexpected results. There are several possible solutions for that with different disadvantages:

1. Don't allow GOTTPOFF/TPOFF relocations at all. This means that we will not be able to link many pre-compiled static libraries.
2. Allow all TLS relocations. Since we can't add new TLS storage at runtime, define a global thread-local variable which will then be used by the linker to resolve GOTTPOFF/TPOFF relocations.
3. Rewrite the relocated code so that it does not use the fs register. On Linux we could probably replace all references to fs by gs and then manually set the gs register? I'm not sure if this can break code if we change instructions that did not originally have a relocation.

I implemented 2. in llvm-rtdyld. Since this is mainly used for testing, the size of the pre-allocated TLS section is only 16 B. In a "real" program you could easily allocate an entire page without noticing any runtime overhead. This is more than enough to link an entire static glibc, for example.

The point I wanted to make here is that if we go for 2. (or 3.), we might as well convert all TLSGD/LD relocations to TPOFF. We would implement all the logic for the TPOFF relocations anyway and if we assume that the pre-allocated TLS storage is large enough to link all object files, this will lead to less runtime overhead to access TLS variables.

[StephenFan]

In D109293#2984338, @lhames wrote:

To allocate the TLS descriptor in GOT, I need to get the edge kind information in PerGraphGOTAndPLTStubBuilder, So I add a Edge::Kind K argument in some functions in PerGraphGOTAndPLTStubBuilder.h. If it is not
suitable, I can think further to solve this problem.

Is there a good reason to put these in the GOT? I would create and manage a different section for these, rather than re-using the GOT.

Put these in the GOT is not necessary. I agree with you that we need to create and manage a different section for these.

Side note: Re-using the GOT in MachO is safe, but was a bit lazy. We could probably come up with a generic TableSection<T> utility that we could re-use for GOTs, PLTs, and these new thread data structures.

Emm, I don't know how to come up with a generic TableSection<T> utility. What does the generic type `T' represents?

[StephenFan]

In D109293#2985435, @MoritzS wrote:

In D109293#2985202, @lhames wrote:

In D109293#2984957, @MoritzS wrote:

In general I would suggest not trying to resolve a TLSGD/LD relocation at all but instead converting it to a GOTTPOFF relocation. Then, no runtime function to implement __tls_get_addr is needed.

Is that compatible with adding extra code at runtime? I suspect we'll need to go the other way: convert things to function calls by default, but make it possible to use direct models where they're safe. That's speculation having not read the TLS spec though.

That depends on how the allocation of TLS sections is implemented, but in general you are right, I didn't think about that.

The problem I faced when implementing the TLS relocations is that we can't chose which types of TLS relocations the object files we try to link use. This means that we need to implement the GOTTPOFF/TPOFF relocations that do not go through the indirection of __tls_get_addr. They just resolve to an offset that will be added to the value stored in %fs:0. This is usually set initially by ld.so when a process starts by using arch_prctl(ARCH_SET_FS), or for newer CPUS by using the wrfsbase instruction. This means that we can't easily extend the storage of the process that runs the runtime linker to add new TLS sections since messing with the fs register will very likely lead to unexpected results. There are several possible solutions for that with different disadvantages:

1. Don't allow GOTTPOFF/TPOFF relocations at all. This means that we will not be able to link many pre-compiled static libraries.
2. Allow all TLS relocations. Since we can't add new TLS storage at runtime, define a global thread-local variable which will then be used by the linker to resolve GOTTPOFF/TPOFF relocations.
3. Rewrite the relocated code so that it does not use the fs register. On Linux we could probably replace all references to fs by gs and then manually set the gs register? I'm not sure if this can break code if we change instructions that did not originally have a relocation.

I implemented 2. in llvm-rtdyld. Since this is mainly used for testing, the size of the pre-allocated TLS section is only 16 B. In a "real" program you could easily allocate an entire page without noticing any runtime overhead. This is more than enough to link an entire static glibc, for example.

Thanks, MoritzS!

I will read your implementation.

[MaskRay]

I am well familiar with the TLS implementations in ld.lld and musl and somewhat familiar with FreeBSD rtld and glibc ld.so.
I have been consulted by ghc on its FreeBSD support.

I appreciate if folks who want to add the LLVM JIT ELF support keep me in the loop.
I currently know nearly nothing about JIT but am happy to allocate some time to study.

In D109293#2984957, @MoritzS wrote:

In D109293#2984338, @lhames wrote:

To allocate the TLS descriptor in GOT, I need to get the edge kind information in PerGraphGOTAndPLTStubBuilder, So I add a Edge::Kind K argument in some functions in PerGraphGOTAndPLTStubBuilder.h. If it is not
suitable, I can think further to solve this problem.

Is there a good reason to put these in the GOT? I would create and manage a different section for these, rather than re-using the GOT.

The ELF TLS spec actually describes using the GOT for that. Each TLSGD relocation is usually converted into two adjacent GOT entries with the DTPMOD64 and DTPOFF64 relocations. I think the practical reason for that is that a TLSGD relocation is 32-bit PC-relative, so using an address of the GOT will guarantee that the 32 bit offset can represent the address.

It doesn't need to be the .got section. It can be any section, but ld.so is the one responsible for filling DTPMOD64/DTPOFF64 values.

In general I would suggest not trying to resolve a TLSGD/LD relocation at all but instead converting it to a GOTTPOFF relocation. Then, no runtime function to implement __tls_get_addr is needed. You can find the detailed description of how to do this in Section 5.5 of the ELF TLS spec (https://akkadia.org/drepper/tls.pdf).

I implemented this for RuntimeDyld (D105466) so you can take a look there. Let me know where I can help!

While the psABI documents of x86-32/x86-64/ppc32/ppc64 have defined TLS optimizations, many (arm/aarch64/riscv/...) don't.
So converting to GOTPTOFF is not always feasible.

2. Allow all TLS relocations. Since we can't add new TLS storage at runtime, define a global thread-local variable which will then be used by the linker to resolve GOTTPOFF/TPOFF relocations.

I agree that reserving static TLS blocks is the most plausible approach (before looking at an implementation).

[MoritzS]

compiler-rt/lib/orc/elfnix_tls.x86-64.S

21

Do we need to save all registers here? I saw you took this from the implementation in MachO and I'm not familiar with the ABI there. But for ELFNix I don't think that is necessary. For TLSGD/TLSLD relocations the compiler already emits a regular function call to __tls_get_addr which means that it already takes care of saving the caller saved registers. Also, since you implemented __orc_rt_elfnix_tls_get_addr_impl as a regular function in C++, its generated assembly will also correctly store all callee saved registers.

[StephenFan]

1. Fix typo
2. Rename TLS Descriptor to TLS Info entry
3. Add PerGraphTLSInfoEntryBuilder pass to create and manage the TLS relative data structure
4. Add test file trivial-tls.S
5. Address @MoritzS 's comments

[StephenFan]

clang-format

[StephenFan]

compiler-rt/lib/orc/elfnix_tls.x86-64.S

21

I agree with you, although I am not a ELF x86-64 expert.

[lhames]

In D109293#2985882, @StephenFan wrote:

In D109293#2984338, @lhames wrote:

Side note: Re-using the GOT in MachO is safe, but was a bit lazy. We could probably come up with a generic TableSection<T> utility that we could re-use for GOTs, PLTs, and these new thread data structures.

Emm, I don't know how to come up with a generic TableSection<T> utility. What does the generic type `T' represents?

@StephenFan -- got-pointer, tlv-pointer, plt-stub, tlv-entry-pair, ....
This would generalize the behavior in PerGraphGOTAndPLTStubsBuilder, which is a nice cleanup. I don't think it's important for this discussion though.

In D109293#2985444, @MoritzS wrote:

In D109293#2985435, @MoritzS wrote:

1. Don't allow GOTTPOFF/TPOFF relocations at all. This means that we will not be able to link many pre-compiled static libraries.
2. Allow all TLS relocations. Since we can't add new TLS storage at runtime, define a global thread-local variable which will then be used by the linker to resolve GOTTPOFF/TPOFF relocations.
3. Rewrite the relocated code so that it does not use the fs register. On Linux we could probably replace all references to fs by gs and then manually set the gs register? I'm not sure if this can break code if we change instructions that did not originally have a relocation.

I implemented 2. in llvm-rtdyld. Since this is mainly used for testing, the size of the pre-allocated TLS section is only 16 B. In a "real" program you could easily allocate an entire page without noticing any runtime overhead. This is more than enough to link an entire static glibc, for example.

The point I wanted to make here is that if we go for 2. (or 3.), we might as well convert all TLSGD/LD relocations to TPOFF. We would implement all the logic for the TPOFF relocations anyway and if we assume that the pre-allocated TLS storage is large enough to link all object files, this will lead to less runtime overhead to access TLS variables.

By default we should optimize for full support of TLS and ORC features, at the cost of performance. We can also offer plugins that people can opt in to to go the other way and give up features (e.g. dynamically extensible TLV) to recapture performance.

On x86-64 and arm64 I think that we can synthesize a per-TLV stub and re-write the TLV access as a call to that stub to get the address of the requested TLV. There may be higher performance schemes that still fit our constraints (full TLS and ORC feature support), but specialized-stubs at least offer a baseline solution.

compiler-rt/lib/orc/elfnix_tls.x86-64.S

21

We don't need to save all the registers, that is just a conservative way to get up and running. I've filed https://llvm.org/PR51820 with a rough sketch of the scheme that I would like to move to eventually for MachO (and I think ELFNix could share the same code).

[lhames]

LGTM -- I think any follow up improvements based on this discussion can happen in-tree.

Thank you for working on this Stephen!

compiler-rt/lib/orc/CMakeLists.txt

 # Build for all components of the ORC runtime support library.
 
 # ORC runtime library implementation files.
 set(ORC_SOURCES
   extensible_rtti.cpp
   log_error_to_stderr.cpp
   macho_platform.cpp
   elfnix_platform.cpp
   run_program_wrapper.cpp
   )
 
 # Implementation files for all ORC architectures.
 set(x86_64_SOURCES
 # x86-64 specific assembly files will go here.
   macho_tlv.x86-64.S
+  elfnix_tls.x86-64.S
 )
 
 set(ORC_IMPL_HEADERS
 # Implementation headers will go here.
   adt.h
   c_api.h
   common.h
   compiler.h

compiler-rt/lib/orc/elfnix_platform.cpp

 //===- elfnix_platform.cpp ------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
for (const auto &ModInits : InitArraySections) {
 
     using InitFunc = void (*)();
     for (auto *Init : ModInits.toSpan<InitFunc>())
       (*Init)();
   }
 
   return Error::success();
 }
+struct TLSInfoEntry {
+  unsigned long Key = 0;
+  unsigned long DataAddress = 0;
+};
 
 class ELFNixPlatformRuntimeState {
 private:
   struct AtExitEntry {
     void (*Func)(void *);
     void *Arg;
   };
 
public:
   const char *dlerror();
   void *dlopen(string_view Name, int Mode);
   int dlclose(void *DSOHandle);
   void *dlsym(void *DSOHandle, string_view Symbol);
 
   int registerAtExit(void (*F)(void *), void *Arg, void *DSOHandle);
   void runAtExits(void *DSOHandle);
 
+  /// Returns the base address of the section containing ThreadData.
+  Expected<std::pair<const char *, size_t>>
+  getThreadDataSectionFor(const char *ThreadData);
+
 private:
   PerJITDylibState *getJITDylibStateByHeaderAddr(void *DSOHandle);
   PerJITDylibState *getJITDylibStateByName(string_view Path);
   PerJITDylibState &
   getOrCreateJITDylibState(ELFNixJITDylibInitializers &MOJDIs);
 
+  Error registerThreadDataSection(span<const char> ThreadDataSection);
+
   Expected<ExecutorAddress> lookupSymbolInJITDylib(void *DSOHandle,
                                                    string_view Symbol);
 
   Expected<ELFNixJITDylibInitializerSequence>
   getJITDylibInitializersByName(string_view Path);
   Expected<void *> dlopenInitialize(string_view Path, int Mode);
   Error initializeJITDylib(ELFNixJITDylibInitializers &MOJDIs);
 
   static ELFNixPlatformRuntimeState *MOPS;
 
   using InitSectionHandler =
       Error (*)(const std::vector<ExecutorAddressRange> &Sections,
                 const ELFNixJITDylibInitializers &MOJDIs);
   const std::vector<std::pair<const char *, InitSectionHandler>> InitSections =
       {{".init_array", runInitArray}};
 
   // FIXME: Move to thread-state.
   std::string DLFcnError;
 
   std::recursive_mutex JDStatesMutex;
   std::unordered_map<void *, PerJITDylibState> JDStates;
   std::unordered_map<std::string, void *> JDNameToHeader;
+
+  std::mutex ThreadDataSectionsMutex;
+  std::map<const char *, size_t> ThreadDataSections;
 };
 
 ELFNixPlatformRuntimeState *ELFNixPlatformRuntimeState::MOPS = nullptr;
 
 void ELFNixPlatformRuntimeState::initialize() {
   assert(!MOPS && "ELFNixPlatformRuntimeState should be null");
   MOPS = new ELFNixPlatformRuntimeState();
 }
 
 ELFNixPlatformRuntimeState &ELFNixPlatformRuntimeState::get() {
   assert(MOPS && "ELFNixPlatformRuntimeState not initialized");
   return *MOPS;
 }
 
 void ELFNixPlatformRuntimeState::destroy() {
   assert(MOPS && "ELFNixPlatformRuntimeState not initialized");
   delete MOPS;
 }
 
 Error ELFNixPlatformRuntimeState::registerObjectSections(
     ELFNixPerObjectSectionsToRegister POSR) {
   if (POSR.EHFrameSection.StartAddress)
     __register_frame(POSR.EHFrameSection.StartAddress.toPtr<const char *>());
 
-  // TODO: Register thread data sections.
+  if (POSR.ThreadDataSection.StartAddress) {
+    if (auto Err = registerThreadDataSection(
+            POSR.ThreadDataSection.toSpan<const char>()))
+      return Err;
+  }
 
   return Error::success();
 }
 
 Error ELFNixPlatformRuntimeState::deregisterObjectSections(
     ELFNixPerObjectSectionsToRegister POSR) {
   if (POSR.EHFrameSection.StartAddress)
     __deregister_frame(POSR.EHFrameSection.StartAddress.toPtr<const char *>());
void ELFNixPlatformRuntimeState::runAtExits(void *DSOHandle) {
 
   while (!V.empty()) {
     auto &AE = V.back();
     AE.Func(AE.Arg);
     V.pop_back();
   }
 }
 
+Expected<std::pair<const char *, size_t>>
+ELFNixPlatformRuntimeState::getThreadDataSectionFor(const char *ThreadData) {
+  std::lock_guard<std::mutex> Lock(ThreadDataSectionsMutex);
+  auto I = ThreadDataSections.upper_bound(ThreadData);
+  // Check that we have a valid entry conovering this address.
+  if (I == ThreadDataSections.begin())
+    return make_error<StringError>("No thread local data section for key");
+  I = std::prev(I);
+  if (ThreadData >= I->first + I->second)
+    return make_error<StringError>("No thread local data section for key");
+  return *I;
+}
+
 ELFNixPlatformRuntimeState::PerJITDylibState *
 ELFNixPlatformRuntimeState::getJITDylibStateByHeaderAddr(void *DSOHandle) {
   auto I = JDStates.find(DSOHandle);
   if (I == JDStates.end())
     return nullptr;
   return &I->second;
 }
 
assert(!JDNameToHeader.count(MOJDIs.Name) &&
            "JITDylib has header map entry but no name map entry");
     JDNameToHeader[MOJDIs.Name] = Header;
     JDS.Header = Header;
   }
 
   return JDS;
 }
 
+Error ELFNixPlatformRuntimeState::registerThreadDataSection(
+    span<const char> ThreadDataSection) {
+  std::lock_guard<std::mutex> Lock(ThreadDataSectionsMutex);
+  auto I = ThreadDataSections.upper_bound(ThreadDataSection.data());
+  if (I != ThreadDataSections.begin()) {
+    auto J = std::prev(I);
+    if (J->first + J->second > ThreadDataSection.data())
+      return make_error<StringError>("Overlapping .tdata sections");
+  }
+  ThreadDataSections.insert(
+      I, std::make_pair(ThreadDataSection.data(), ThreadDataSection.size()));
+  return Error::success();
+}
+
 Expected<ExecutorAddress>
 ELFNixPlatformRuntimeState::lookupSymbolInJITDylib(void *DSOHandle,
                                                    string_view Sym) {
   Expected<ExecutorAddress> Result((ExecutorAddress()));
   if (auto Err = WrapperFunction<SPSExpected<SPSExecutorAddress>(
           SPSExecutorAddress,
           SPSString)>::call(&__orc_rt_elfnix_symbol_lookup_tag, Result,
                             ExecutorAddress::fromPtr(DSOHandle), Sym))
for (auto &KV : InitSections) {
     if (I != MOJDIs.InitSections.end()) {
       if (auto Err = Handler(I->second, MOJDIs))
         return Err;
     }
   }
 
   return Error::success();
 }
+class ELFNixPlatformRuntimeTLVManager {
+public:
+  void *getInstance(const char *ThreadData);
+
+private:
+  std::unordered_map<const char *, char *> Instances;
+  std::unordered_map<const char *, std::unique_ptr<char[]>> AllocatedSections;
+};
+
+void *ELFNixPlatformRuntimeTLVManager::getInstance(const char *ThreadData) {
+  auto I = Instances.find(ThreadData);
+  if (I != Instances.end())
+    return I->second;
+  auto TDS =
+      ELFNixPlatformRuntimeState::get().getThreadDataSectionFor(ThreadData);
+  if (!TDS) {
+    __orc_rt_log_error(toString(TDS.takeError()).c_str());
+    return nullptr;
+  }
+
+  auto &Allocated = AllocatedSections[TDS->first];
+  if (!Allocated) {
+    Allocated = std::make_unique<char[]>(TDS->second);
+    memcpy(Allocated.get(), TDS->first, TDS->second);
+  }
+  size_t ThreadDataDelta = ThreadData - TDS->first;
+  assert(ThreadDataDelta <= TDS->second && "ThreadData outside section bounds");
+
+  char *Instance = Allocated.get() + ThreadDataDelta;
+  Instances[ThreadData] = Instance;
+  return Instance;
+}
+
+void destroyELFNixTLVMgr(void *ELFNixTLVMgr) {
+  delete static_cast<ELFNixPlatformRuntimeTLVManager *>(ELFNixTLVMgr);
+}
 
 } // end anonymous namespace
 
 //------------------------------------------------------------------------------
 //                             JIT entry points
 //------------------------------------------------------------------------------
 
 ORC_RT_INTERFACE __orc_rt_CWrapperFunctionResult
return WrapperFunction<SPSError(SPSELFNixPerObjectSectionsToRegister)>::
              [](ELFNixPerObjectSectionsToRegister &POSR) {
                return ELFNixPlatformRuntimeState::get()
                    .deregisterObjectSections(std::move(POSR));
              })
           .release();
 }
 
 //------------------------------------------------------------------------------
+//                           TLV support
+//------------------------------------------------------------------------------
+
+ORC_RT_INTERFACE void *__orc_rt_elfnix_tls_get_addr_impl(TLSInfoEntry *D) {
+  auto *TLVMgr = static_cast<ELFNixPlatformRuntimeTLVManager *>(
+      pthread_getspecific(D->Key));
+  if (!TLVMgr)
+    TLVMgr = new ELFNixPlatformRuntimeTLVManager();
+  if (pthread_setspecific(D->Key, TLVMgr)) {
+    __orc_rt_log_error("Call to pthread_setspecific failed");
+    return nullptr;
+  }
+
+  return TLVMgr->getInstance(
+      reinterpret_cast<char *>(static_cast<uintptr_t>(D->DataAddress)));
+}
+
+ORC_RT_INTERFACE __orc_rt_CWrapperFunctionResult
+__orc_rt_elfnix_create_pthread_key(char *ArgData, size_t ArgSize) {
+  return WrapperFunction<SPSExpected<uint64_t>(void)>::handle(
+             ArgData, ArgSize,
+             []() -> Expected<uint64_t> {
+               pthread_key_t Key;
+               if (int Err = pthread_key_create(&Key, destroyELFNixTLVMgr)) {
+                 __orc_rt_log_error("Call to pthread_key_create failed");
+                 return make_error<StringError>(strerror(Err));
+               }
+               return static_cast<uint64_t>(Key);
+             })
+      .release();
+}
+
+//------------------------------------------------------------------------------
 //                           cxa_atexit support
 //------------------------------------------------------------------------------
 
 int __orc_rt_elfnix_cxa_atexit(void (*func)(void *), void *arg,
                                void *dso_handle) {
   return ELFNixPlatformRuntimeState::get().registerAtExit(func, arg,
                                                           dso_handle);
 }

compiler-rt/lib/orc/elfnix_tls.x86-64.S

+
+//===-- orc_rt_elfnix_tls_x86-64.s -------------------------------*- ASM -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file is a part of the ORC runtime support library.
+//
+//===----------------------------------------------------------------------===//
+
+#define REGISTER_SAVE_SPACE_SIZE        512
+
+        .text
+
+  // returns address of TLV in %rax, all other registers preserved
+  .globl ___orc_rt_elfnix_tls_get_addr
+___orc_rt_elfnix_tls_get_addr:
+        pushq           %rbp
+        movq            %rsp,        %rbp
+        subq            $REGISTER_SAVE_SPACE_SIZE, %rsp
+        movq            %rcx,    -16(%rbp)
+        movq            %rdx,    -24(%rbp)
+        movq            %rsi,    -32(%rbp)
+        movq            %rdi,    -40(%rbp)
+        movq            %r8,     -48(%rbp)
+        movq            %r9,     -56(%rbp)
+        movq            %r10,    -64(%rbp)
+        movq            %r11,    -72(%rbp)
+  movdqa          %xmm0,  -128(%rbp)
+  movdqa          %xmm1,  -144(%rbp)
+  movdqa          %xmm2,  -160(%rbp)
+  movdqa          %xmm3,  -176(%rbp)
+  movdqa          %xmm4,  -192(%rbp)
+  movdqa          %xmm5,  -208(%rbp)
+  movdqa          %xmm6,  -224(%rbp)
+  movdqa          %xmm7,  -240(%rbp)
+        call           __orc_rt_elfnix_tls_get_addr_impl
+        movq            -16(%rbp),      %rcx
+        movq            -24(%rbp),      %rdx
+        movq            -32(%rbp),      %rsi
+        movq            -40(%rbp),      %rdi
+        movq            -48(%rbp),      %r8
+        movq            -56(%rbp),      %r9
+        movq            -64(%rbp),      %r10
+        movq            -72(%rbp),      %r11
+  movdqa          -128(%rbp),     %xmm0
+  movdqa          -144(%rbp),     %xmm1
+  movdqa          -160(%rbp),     %xmm2
+  movdqa          -176(%rbp),     %xmm3
+  movdqa          -192(%rbp),     %xmm4
+  movdqa          -208(%rbp),     %xmm5
+  movdqa          -224(%rbp),     %xmm6
+  movdqa          -240(%rbp),     %xmm7
+        addq            $REGISTER_SAVE_SPACE_SIZE, %rsp
+        popq            %rbp
+        ret

compiler-rt/test/orc/TestCases/Linux/x86-64/trivial-tls.S

+// RUN: %clang -c -o %t %s
+// RUN: %llvm_jitlink %t
+//
+// Test that basic ELF TLS work by adding together TLSs with values
+// 0, 1, and -1, and returning the result (0 for success). This setup
+// tests both zero-initialized (.tbss) and non-zero-initialized
+// (.tdata) sections.
+
+  .text
+  .file "tlstest.cpp"
+  .globl  main                            # -- Begin function main
+  .p2align  4, 0x90
+  .type main,@function
+main:                                   # @main
+# %bb.0:                                # %entry
+  pushq %rbp
+  movq  %rsp, %rbp
+  subq  $32, %rsp
+  movl  $0, -4(%rbp)
+  movl  %edi, -8(%rbp)
+  movq  %rsi, -16(%rbp)
+  data16
+  leaq  x@TLSGD(%rip), %rdi
+  data16
+  data16
+  rex64
+  callq __tls_get_addr@PLT
+  movl  (%rax), %eax
+  movl  %eax, -24(%rbp)                 # 4-byte Spill
+  data16
+  leaq  y@TLSGD(%rip), %rdi
+  data16
+  data16
+  rex64
+  callq __tls_get_addr@PLT
+  movq  %rax, %rcx
+  movl  -24(%rbp), %eax                 # 4-byte Reload
+  movl  (%rcx), %ecx
+  addl  %ecx, %eax
+  movl  %eax, -20(%rbp)                 # 4-byte Spill
+  data16
+  leaq  z@TLSGD(%rip), %rdi
+  data16
+  data16
+  rex64
+  callq __tls_get_addr@PLT
+  movq  %rax, %rcx
+  movl  -20(%rbp), %eax                 # 4-byte Reload
+  movl  (%rcx), %ecx
+  addl  %ecx, %eax
+  addq  $32, %rsp
+  popq  %rbp
+  retq
+.Lfunc_end0:
+  .size main, .Lfunc_end0-main
+                                        # -- End function
+  .type x,@object                       # @x
+  .section  .tbss,"awT",@nobits
+  .globl  x
+  .p2align  2
+x:
+  .long 0                               # 0x0
+  .size x, 4
+
+  .type y,@object                       # @y
+  .section  .tdata,"awT",@progbits
+  .globl  y
+  .p2align  2
+y:
+  .long 1                               # 0x1
+  .size y, 4
+
+  .type z,@object                       # @z
+  .globl  z
+  .p2align  2
+z:
+  .long 4294967295                      # 0xffffffff
+  .size z, 4
+
+  .section  ".note.GNU-stack","",@progbits
+  .addrsig

llvm/include/llvm/ExecutionEngine/JITLink/ELF_x86_64.h

 //===--- ELF_x86_64.h - JIT link functions for ELF/x86-64 ---*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //

 enum ELFX86RelocationKind : Edge::Kind {
   Branch32 = Edge::FirstRelocation,
   Pointer32Signed,
   Pointer64,
   PCRel32,
   PCRel32GOTLoad,
   PCRel32GOTLoadRelaxable,
   PCRel32REXGOTLoadRelaxable,
+  PCRel32TLV,
   PCRel64GOT,
   GOTOFF64,
   GOT64,
   Delta64,
 };
 
 } // end namespace ELF_x86_64_Edges
 

llvm/include/llvm/ExecutionEngine/JITLink/x86_64.h

 //===-- x86_64.h - Generic JITLink x86-64 edge kinds, utilities -*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
enum EdgeKind_x86_64 : Edge::Kind {
   /// Errors:
   ///   - The result of the fixup expression must fit into an int32, otherwise
   ///     an out-of-range error will be returned.
   ///   - The target must be either external, or a TLV entry of the required
   ///     form, otherwise a malformed TLV entry error will be returned.
   ///
   PCRel32TLVPLoadREXRelaxable,
 
+  /// TODO: Explain the generic edge kind
+  RequestTLSDescInGOTAndTransformToDelta32,
+
   /// A TLVP entry getter/constructor, transformed to
   /// Delta32ToTLVPLoadREXRelaxable.
   ///
   /// Indicates that this edge should be transformed into a
   /// Delta32ToTLVPLoadREXRelaxable targeting the TLVP entry for the edge's
   /// current target. A TLVP entry for the target should be created if one does
   /// not already exist.
   ///

llvm/include/llvm/ExecutionEngine/Orc/ELFNixPlatform.h

 //===-- ELFNixPlatform.h -- Utilities for executing ELF in Orc --*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
private:
   // Records the addresses of runtime symbols used by the platform.
   Error bootstrapELFNixRuntime(JITDylib &PlatformJD);
 
   Error registerInitInfo(JITDylib &JD,
                          ArrayRef<jitlink::Section *> InitSections);
 
   Error registerPerObjectSections(const ELFPerObjectSectionsToRegister &POSR);
 
+  Expected<uint64_t> createPThreadKey();
+
   ExecutionSession &ES;
   ObjectLinkingLayer &ObjLinkingLayer;
 
   SymbolStringPtr DSOHandleSymbol;
   std::atomic<bool> RuntimeBootstrapped{false};
 
   ExecutorAddress orc_rt_elfnix_platform_bootstrap;
   ExecutorAddress orc_rt_elfnix_platform_shutdown;

llvm/lib/ExecutionEngine/JITLink/ELFLinkGraphBuilder.h

 //===------- ELFLinkGraphBuilder.h - ELF LinkGraph builder ------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
for (ELFSymbolIndex SymIndex = 0; SymIndex != Symbols->size(); ++SymIndex) {
     if (auto LSOrErr = getSymbolLinkageAndScope(Sym, *Name))
       std::tie(L, S) = *LSOrErr;
     else
       return LSOrErr.takeError();
 
     if (Sym.isDefined() &&
         (Sym.getType() == ELF::STT_NOTYPE || Sym.getType() == ELF::STT_FUNC ||
          Sym.getType() == ELF::STT_OBJECT ||
-         Sym.getType() == ELF::STT_SECTION)) {
+         Sym.getType() == ELF::STT_SECTION || Sym.getType() == ELF::STT_TLS)) {
 
       // FIXME: Handle extended tables.
       if (auto *GraphSec = getGraphSection(Sym.st_shndx)) {
         Block *B = nullptr;
         {
           auto Blocks = GraphSec->blocks();
           assert(Blocks.begin() != Blocks.end() && "No blocks for section");
           assert(std::next(Blocks.begin()) == Blocks.end() &&

llvm/lib/ExecutionEngine/JITLink/ELF_x86_64.cpp

 //===---- ELF_x86_64.cpp -JIT linker implementation for ELF/x86-64 ----===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // ELF/x86-64 jit-link implementation.
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/ExecutionEngine/JITLink/ELF_x86_64.h"
 #include "llvm/ExecutionEngine/JITLink/JITLink.h"
 #include "llvm/ExecutionEngine/JITLink/x86_64.h"
 #include "llvm/Object/ELFObjectFile.h"
 #include "llvm/Support/Endian.h"
 
 #include "DefineExternalSectionStartAndEndSymbols.h"
 #include "EHFrameSupportImpl.h"
 #include "ELFLinkGraphBuilder.h"
 #include "JITLinkGeneric.h"
 #include "PerGraphGOTAndPLTStubsBuilder.h"
+#include "PerGraphTLSInfoEntryBuilder.h"
 
 #define DEBUG_TYPE "jitlink"
 
 using namespace llvm;
 using namespace llvm::jitlink;
 using namespace llvm::jitlink::ELF_x86_64_Edges;
 
 namespace {
 
 constexpr StringRef ELFGOTSectionName = "$__GOT";
 constexpr StringRef ELFGOTSymbolName = "_GLOBAL_OFFSET_TABLE_";
+constexpr StringRef ELFTLSInfoSectionName = "$__TLSINFO";
+
+class PerGraphTLSInfoBuilder_ELF_x86_64
+    : public PerGraphTLSInfoEntryBuilder<PerGraphTLSInfoBuilder_ELF_x86_64> {
+public:
+  static const uint8_t TLSInfoEntryContent[16];
+  using PerGraphTLSInfoEntryBuilder<
+      PerGraphTLSInfoBuilder_ELF_x86_64>::PerGraphTLSInfoEntryBuilder;
+
+  bool isTLSEdgeToFix(Edge &E) {
+    return E.getKind() == x86_64::RequestTLSDescInGOTAndTransformToDelta32;
+  }
+
+  Symbol &createTLSInfoEntry(Symbol &Target) { 
+    // the TLS Info entry's key value will be written by the fixTLVSectionByName pass, so create mutable content.
+    auto &TLSInfoEntry = G.createMutableContentBlock(
+        getTLSInfoSection(), G.allocateContent(getTLSInfoEntryContent()), 0, 8, 0);
+    TLSInfoEntry.addEdge(x86_64::Pointer64, 8, Target, 0);
+    return G.addAnonymousSymbol(TLSInfoEntry, 0, 16, false, false);
+  }
+
+  void fixTLSEdge(Edge &E, Symbol &Target) {
+    if (E.getKind() == x86_64::RequestTLSDescInGOTAndTransformToDelta32) {
+      E.setTarget(Target);
+      E.setKind(x86_64::Delta32);
+    }
+  }
+
+  Section &getTLSInfoSection() const {
+    if (!TLSInfoSection)
+      TLSInfoSection =
+          &G.createSection(ELFTLSInfoSectionName, sys::Memory::MF_READ);
+    return *TLSInfoSection;
+  }
+
+private:
+  ArrayRef<char> getTLSInfoEntryContent() {
+    return {reinterpret_cast<const char *>(TLSInfoEntryContent),
+            sizeof(TLSInfoEntryContent)};
+  }
+
+  mutable Section *TLSInfoSection = nullptr;
+};
+
+const uint8_t PerGraphTLSInfoBuilder_ELF_x86_64::TLSInfoEntryContent[16] = {
+    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /*pthread key */
+    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00  /*data address*/
+};
 
 class PerGraphGOTAndPLTStubsBuilder_ELF_x86_64
     : public PerGraphGOTAndPLTStubsBuilder<
           PerGraphGOTAndPLTStubsBuilder_ELF_x86_64> {
 public:
   static const uint8_t NullGOTEntryContent[8];
   static const uint8_t StubContent[6];
 
getRelocationKind(const uint32_t Type) {
     case ELF::R_X86_64_GOTPCREL64:
       return ELF_x86_64_Edges::ELFX86RelocationKind::PCRel64GOT;
     case ELF::R_X86_64_GOT64:
       return ELF_x86_64_Edges::ELFX86RelocationKind::GOT64;
     case ELF::R_X86_64_GOTOFF64:
       return ELF_x86_64_Edges::ELFX86RelocationKind::GOTOFF64;
     case ELF::R_X86_64_PLT32:
       return ELF_x86_64_Edges::ELFX86RelocationKind::Branch32;
+    case ELF::R_X86_64_TLSGD:
+      return ELF_x86_64_Edges::ELFX86RelocationKind::PCRel32TLV;
     }
     return make_error<JITLinkError>("Unsupported x86-64 relocation type " +
                                     formatv("{0:d}: ", Type) +
                                     getELFX86_64RelocName(Type));
   }
 
   Error addRelocations() override {
     LLVM_DEBUG(dbgs() << "Adding relocations\n");
for (auto &SecRef : Sections) {
           Kind = x86_64::RequestGOTAndTransformToDelta32;
           break;
         }
         case PCRel32REXGOTLoadRelaxable: {
           Kind = x86_64::RequestGOTAndTransformToPCRel32GOTLoadREXRelaxable;
           Addend = 0;
           break;
         }
+        case PCRel32TLV: {
+          Kind = x86_64::RequestTLSDescInGOTAndTransformToDelta32;
+          break;
+        }
         case PCRel32GOTLoadRelaxable: {
           Kind = x86_64::RequestGOTAndTransformToPCRel32GOTLoadRelaxable;
           Addend = 0;
           break;
         }
         case PCRel64GOT: {
           Kind = x86_64::RequestGOTAndTransformToDelta64;
           break;
if (Ctx->shouldAddDefaultTargetPasses(G->getTargetTriple())) {
     // Construct a JITLinker and run the link function.
     // Add a mark-live pass.
     if (auto MarkLive = Ctx->getMarkLivePass(G->getTargetTriple()))
       Config.PrePrunePasses.push_back(std::move(MarkLive));
     else
       Config.PrePrunePasses.push_back(markAllSymbolsLive);
 
     // Add an in-place GOT/Stubs pass.
+
+    Config.PostPrunePasses.push_back(PerGraphTLSInfoBuilder_ELF_x86_64::asPass);
     Config.PostPrunePasses.push_back(
         PerGraphGOTAndPLTStubsBuilder_ELF_x86_64::asPass);
 
     // Resolve any external section start / end symbols.
     Config.PostAllocationPasses.push_back(
         createDefineExternalSectionStartAndEndSymbolsPass(
             identifyELFSectionStartAndEndSymbols));
 

llvm/lib/ExecutionEngine/JITLink/PerGraphTLSInfoEntryBuilder.h

+//===---------------- PerGraphTLSInfoEntryBuilder.h -------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// Construct Thread local storage info entry for each graph.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_EXECUTIONENGINE_JITLINK_PERGRAPHTLSINFOENTRYBUILDER_H
+#define LLVM_EXECUTIONENGINE_JITLINK_PERGRAPHTLSINFOENTRYBUILDER_H
+
+#include "llvm/ExecutionEngine/JITLink/JITLink.h"
+#include "llvm/Support/Debug.h"
+
+#define DEBUG_TYPE "jitlink"
+namespace llvm {
+namespace jitlink {
+
+template <typename BuilderImplT> class PerGraphTLSInfoEntryBuilder {
+public:
+  PerGraphTLSInfoEntryBuilder(LinkGraph &G) : G(G) {}
+  static Error asPass(LinkGraph &G) { return BuilderImplT(G).run(); }
+
+  Error run() {
+    LLVM_DEBUG(dbgs() << "Running Per-Graph TLS Info entry builder:\n ");
+
+    std::vector<Block *> Worklist(G.blocks().begin(), G.blocks().end());
+
+    for (auto *B : Worklist)
+      for (auto &E : B->edges()) {
+        if (impl().isTLSEdgeToFix(E)) {
+          LLVM_DEBUG({
+            dbgs() << "  Fixing " << G.getEdgeKindName(E.getKind())
+                   << " edge at " << formatv("{0:x}", B->getFixupAddress(E))
+                   << " (" << formatv("{0:x}", B->getAddress()) << " + "
+                   << formatv("{0:x}", E.getOffset()) << ")\n";
+          });
+          impl().fixTLSEdge(E, getTLSInfoEntry(E.getTarget()));
+        }
+      }
+    return Error::success();
+  }
+
+protected:
+  LinkGraph &G;
+
+  Symbol &getTLSInfoEntry(Symbol &Target) {
+    assert(Target.hasName() && "TLS edge cannot point to anonymous target");
+    auto TLSInfoEntryI = TLSInfoEntries.find(Target.getName());
+    if (TLSInfoEntryI == TLSInfoEntries.end()) {
+      auto &TLSInfoEntry = impl().createTLSInfoEntry(Target);
+      LLVM_DEBUG({
+        dbgs() << "    Created TLS Info entry for " << Target.getName() << ": "
+               << TLSInfoEntry << "\n";
+      });
+      TLSInfoEntryI =
+          TLSInfoEntries.insert(std::make_pair(Target.getName(), &TLSInfoEntry))
+              .first;
+    }
+    assert(TLSInfoEntryI != TLSInfoEntries.end() &&
+           "Could not get TLSInfo symbol");
+    LLVM_DEBUG({
+      dbgs() << "    Using TLS Info entry" << *TLSInfoEntryI->second << "\n";
+    });
+    return *TLSInfoEntryI->second;
+  }
+
+private:
+  DenseMap<StringRef, Symbol *> TLSInfoEntries;
+  BuilderImplT &impl() { return static_cast<BuilderImplT &>(*this); }
+};
+} // namespace jitlink
+} // namespace llvm
+#endif

llvm/lib/ExecutionEngine/Orc/ELFNixPlatform.cpp

 //===------ ELFNixPlatform.cpp - Utilities for executing MachO in Orc -----===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 

 }
 
 Error ELFNixPlatform::bootstrapELFNixRuntime(JITDylib &PlatformJD) {
 
   std::pair<const char *, ExecutorAddress *> Symbols[] = {
       {"__orc_rt_elfnix_platform_bootstrap", &orc_rt_elfnix_platform_bootstrap},
       {"__orc_rt_elfnix_platform_shutdown", &orc_rt_elfnix_platform_shutdown},
       {"__orc_rt_elfnix_register_object_sections",
-       &orc_rt_elfnix_register_object_sections}};
+       &orc_rt_elfnix_register_object_sections},
+      {"__orc_rt_elfnix_create_pthread_key",
+       &orc_rt_elfnix_create_pthread_key}};
 
   SymbolLookupSet RuntimeSymbols;
   std::vector<std::pair<SymbolStringPtr, ExecutorAddress *>> AddrsToRecord;
   for (const auto &KV : Symbols) {
     auto Name = ES.intern(KV.first);
     RuntimeSymbols.add(Name);
     AddrsToRecord.push_back({std::move(Name), KV.second});
   }
Error ELFNixPlatform::registerPerObjectSections(
   Error ErrResult = Error::success();
   if (auto Err = ES.callSPSWrapper<shared::SPSError(
                      SPSELFPerObjectSectionsToRegister)>(
           orc_rt_elfnix_register_object_sections.getValue(), ErrResult, POSR))
     return Err;
   return ErrResult;
 }
 
+Expected<uint64_t> ELFNixPlatform::createPThreadKey() {
+  if (!orc_rt_elfnix_create_pthread_key)
+    return make_error<StringError>(
+        "Attempting to create pthread key in target, but runtime support has "
+        "not been loaded yet",
+        inconvertibleErrorCode());
+
+  Expected<uint64_t> Result(0);
+  if (auto Err = ES.callSPSWrapper<SPSExpected<uint64_t>(void)>(
+          orc_rt_elfnix_create_pthread_key.getValue(), Result))
+    return std::move(Err);
+  return Result;
+}
+
 void ELFNixPlatform::ELFNixPlatformPlugin::modifyPassConfig(
     MaterializationResponsibility &MR, jitlink::LinkGraph &LG,
     jitlink::PassConfiguration &Config) {
 
   // If the initializer symbol is the __dso_handle symbol then just add
   // the DSO handle support passes.
   if (MR.getInitializerSymbol() == MP.DSOHandleSymbol) {
     addDSOHandleSupportPasses(MR, Config);
void ELFNixPlatform::ELFNixPlatformPlugin::addDSOHandleSupportPasses(
   });
 }
 
 void ELFNixPlatform::ELFNixPlatformPlugin::addEHAndTLVSupportPasses(
     MaterializationResponsibility &MR, jitlink::PassConfiguration &Config) {
 
   // Insert TLV lowering at the start of the PostPrunePasses, since we want
   // it to run before GOT/PLT lowering.
-  Config.PostPrunePasses.insert(
-      Config.PostPrunePasses.begin(),
+
+  // TODO: Check that before the fixTLVSectionsAndEdges pass, the GOT/PLT build
+  // pass has done. Because the TLS descriptor need to be allocate in GOT.
+  Config.PostPrunePasses.push_back(
       [this, &JD = MR.getTargetJITDylib()](jitlink::LinkGraph &G) {
         return fixTLVSectionsAndEdges(G, JD);
       });
 
   // Add a pass to register the final addresses of the eh-frame and TLV sections
   // with the runtime.
   Config.PostFixupPasses.push_back([this](jitlink::LinkGraph &G) -> Error {
     ELFPerObjectSectionsToRegister POSR;
Error ELFNixPlatform::ELFNixPlatformPlugin::registerInitSections(
 
   return MP.registerInitInfo(JD, InitSections);
 }
 
 Error ELFNixPlatform::ELFNixPlatformPlugin::fixTLVSectionsAndEdges(
     jitlink::LinkGraph &G, JITDylib &JD) {
 
   // TODO implement TLV support
+  for (auto *Sym : G.external_symbols())
+    if (Sym->getName() == "__tls_get_addr") {
+      Sym->setName("___orc_rt_elfnix_tls_get_addr");
+    }
+
+  auto *TLSInfoEntrySection = G.findSectionByName("$__TLSINFO");
+
+  if (TLSInfoEntrySection) {
+    Optional<uint64_t> Key;
+    {
+      std::lock_guard<std::mutex> Lock(MP.PlatformMutex);
+      auto I = MP.JITDylibToPThreadKey.find(&JD);
+      if (I != MP.JITDylibToPThreadKey.end())
+        Key = I->second;
+    }
+    if (!Key) {
+      if (auto KeyOrErr = MP.createPThreadKey())
+        Key = *KeyOrErr;
+      else
+        return KeyOrErr.takeError();
+    }
+
+    uint64_t PlatformKeyBits =
+        support::endian::byte_swap(*Key, G.getEndianness());
+
+    for (auto *B : TLSInfoEntrySection->blocks()) {
+      // FIXME: The TLS descriptor byte length may different with different
+      // ISA
+      assert(B->getSize() == (G.getPointerSize() * 2) &&
+             "TLS descriptor must be 2 words length");
+      auto TLSInfoEntryContent = B->getMutableContent(G);
+      memcpy(TLSInfoEntryContent.data(), &PlatformKeyBits, G.getPointerSize());
+    }
+  }
 
   return Error::success();
 }
 
 } // End namespace orc.
 } // End namespace llvm.

llvm/lib/ExecutionEngine/RuntimeDyld/JITSymbol.cpp

 //===----------- JITSymbol.cpp - JITSymbol class implementation -----------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
if (*SymbolFlagsOrErr & object::BasicSymbolRef::SF_Common)
     Flags |= JITSymbolFlags::Common;
   if (*SymbolFlagsOrErr & object::BasicSymbolRef::SF_Exported)
     Flags |= JITSymbolFlags::Exported;
 
   auto SymbolType = Symbol.getType();
   if (!SymbolType)
     return SymbolType.takeError();
 
-  if (*SymbolType & object::SymbolRef::ST_Function)
+  if (*SymbolType == object::SymbolRef::ST_Function)
     Flags |= JITSymbolFlags::Callable;
 
   return Flags;
 }
 
 ARMJITSymbolFlags
 llvm::ARMJITSymbolFlags::fromObjectSymbol(const object::SymbolRef &Symbol) {
   Expected<uint32_t> SymbolFlagsOrErr = Symbol.getFlags();