[lld][elf2] Local Dynamic TLS

[Bigcheese]

Summary

This patch isn't in a finished state, but I wanted some guidance on how to properly do the GOT parts. Right now it's hacked to just look for the TLSLD relocs. We're also going to run into an issue with GOT entries when we do global dynamic TLS. Each TLS symbol gets two entries, each with a different relocation.

I switched compareOutputSections to a numeric based compare because adding proper order for TLS to it over complicated the code.

[ruiu]

ELF/OutputSections.cpp

58

continue;  // skip TLS empty second slot

128

I don't understand this. Is R_x86_64_TLSLD interpositioned? This function is really complicated and needs cleanup.

ELF/OutputSections.h

111

It seems that we don't really have to pass the number to this function because the symbol should know that. If it's STT_TLS, we need two slots. Otherwise one. Can you remove the second parameter?

ELF/Writer.cpp

143

Global uninitialized values should be initialized with zero, so no need to assign the same value again.

220

This is another use of canBePreempted(Body) || Type == R_X86_64_TLSLD

292–327

Can you move this function to top level? This doesn't have to be a lambda.

298

(Flags & SHF_ALLOC) && (Flags | SHF_TLS)

563

(Sec->getFlags() & SHF_ALLOC) && (Sec->getFalgs() & SHF_TLS)

563–586

Please try to reduce the amount of code to add this code, or completely split this function. This function is already too long.

[• rafael]

This also needs to be rebased.

ELF/Target.cpp

372

Please update this. Depending on false -> 0 is just bad.

You probably also want to rename the function.

ELF/Writer.cpp

294

Sorry, but why exactly does it need to be at the start of the PT_LOAD? We need to have a single PT_TLS ,but that would still be the case with us having

• Some stuff
• TLS stuff
• Some other stuff

[ruiu]

ELF/Target.cpp

372

I don't think we need to return int in the first place as the number of got slots a symbol needs can be inferred from the symbol type, as I wrote in the review comment.

[• rafael]

We had a chat "offline".

I asked Michael to split this in 3 patches for easier discussion:

• Ordering the TLS sections.
• Creating the TLS segment.
• Handling the relocations.

For the first part, there a few properties of tls to keep in mind

• There is only one segment, so the NOBITS part of TLS has to be just

after the non NOBITS part.

• All actual TLS sections are RW, so we only need to put the tls logic

once we know we are in a single PT_LOAD.

• If it is NOBITS TLS, it must come before non TLS. If it is a non

NOBITS TLS, it must come after.

Cheers,
Rafael

[Bigcheese]

ELF/OutputSections.cpp

128

R_X86_64_TLSLD can't be preempted, but this is the function that's used to determine if we should emit a dynamic relocation for it or not. As I said in the description, I wanted some guidance on how to restructure the GOT code to handle this.

ELF/Writer.cpp

143

This would make writeResult non-reentrant while it currently is reentrant.

ELF/OutputSections.h

class GotSection final : public OutputSectionBase<ELFT::Is64Bits> {
   typedef typename Base::uintX_t uintX_t;
 
 public:
   GotSection();
   void finalize() override {
     this->Header.sh_size = Entries.size() * this->getAddrSize();
   }
   void writeTo(uint8_t *Buf) override;
-  void addEntry(SymbolBody *Sym);
+  void addEntry(SymbolBody *Sym, int Size = 1);
   bool empty() const { return Entries.empty(); }
   uintX_t getEntryAddr(const SymbolBody &B) const;
 
 private:
   std::vector<const SymbolBody *> Entries;
 };
 
 template <class ELFT>
template <class ELFT> struct Out {
   static InterpSection<ELFT::Is64Bits> *Interp;
   static OutputSection<ELFT> *Bss;
   static PltSection<ELFT> *Plt;
   static RelocationSection<ELFT> *RelaDyn;
   static StringTableSection<ELFT::Is64Bits> *DynStrTab;
   static StringTableSection<ELFT::Is64Bits> *StrTab;
   static SymbolTableSection<ELFT> *DynSymTab;
   static SymbolTableSection<ELFT> *SymTab;
+  static typename llvm::object::ELFFile<ELFT>::uintX_t ThreadDataStart;
 };
 
 template <class ELFT> DynamicSection<ELFT> *Out<ELFT>::Dynamic;
 template <class ELFT> GotSection<ELFT> *Out<ELFT>::Got;
 template <class ELFT> HashTableSection<ELFT> *Out<ELFT>::HashTab;
 template <class ELFT> InterpSection<ELFT::Is64Bits> *Out<ELFT>::Interp;
 template <class ELFT> OutputSection<ELFT> *Out<ELFT>::Bss;
 template <class ELFT> PltSection<ELFT> *Out<ELFT>::Plt;
 template <class ELFT> RelocationSection<ELFT> *Out<ELFT>::RelaDyn;
 template <class ELFT> StringTableSection<ELFT::Is64Bits> *Out<ELFT>::DynStrTab;
 template <class ELFT> StringTableSection<ELFT::Is64Bits> *Out<ELFT>::StrTab;
 template <class ELFT> SymbolTableSection<ELFT> *Out<ELFT>::DynSymTab;
 template <class ELFT> SymbolTableSection<ELFT> *Out<ELFT>::SymTab;
+template <class ELFT>
+typename llvm::object::ELFFile<ELFT>::uintX_t Out<ELFT>::ThreadDataStart;
 }
 }
 #endif

ELF/OutputSections.cpp

 template <class ELFT>
 GotSection<ELFT>::GotSection()
     : OutputSectionBase<ELFT::Is64Bits>(".got", llvm::ELF::SHT_PROGBITS,
                                         llvm::ELF::SHF_ALLOC |
                                             llvm::ELF::SHF_WRITE) {
   this->Header.sh_addralign = this->getAddrSize();
 }
 
-template <class ELFT> void GotSection<ELFT>::addEntry(SymbolBody *Sym) {
+template <class ELFT>
+void GotSection<ELFT>::addEntry(SymbolBody *Sym, int Size) {
   Sym->GotIndex = Entries.size();
   Entries.push_back(Sym);
+  Entries.insert(Entries.end(), Size - 1, nullptr);
 }
 
 template <class ELFT>
 typename GotSection<ELFT>::uintX_t
 GotSection<ELFT>::getEntryAddr(const SymbolBody &B) const {
   return this->getVA() + B.GotIndex * this->getAddrSize();
 }
 
 template <class ELFT> void GotSection<ELFT>::writeTo(uint8_t *Buf) {
   for (const SymbolBody *B : Entries) {
     uint8_t *Entry = Buf;
     Buf += sizeof(uintX_t);
+    if (!B)
+      continue;
     if (canBePreempted(B))
       continue; // The dynamic linker will take care of it.
     uintX_t VA = getSymVA<ELFT>(*B);
     write<uintX_t, ELFT::TargetEndianness, sizeof(uintX_t)>(Entry, VA);
   }
 }
 
 template <class ELFT>
for (const DynamicReloc<ELFT> &Rel : Relocs) {
     OutputSection<ELFT> *OutSec = C.getOutputSection();
     uint32_t SymIndex = RI.getSymbol(IsMips64EL);
     const ObjectFile<ELFT> &File = *C.getFile();
     const SymbolBody *Body = File.getSymbolBody(SymIndex);
     const ELFFile<ELFT> &Obj = File.getObj();
 
     uint32_t Type = RI.getType(IsMips64EL);
 
-    bool CanBePreempted = canBePreempted(Body);
+    bool CanBePreempted = canBePreempted(Body) || Type == R_X86_64_TLSLD;
     uintX_t Addend = 0;
     if (!CanBePreempted) {
       if (IsRela) {
         if (Body)
           Addend += getSymVA<ELFT>(cast<ELFSymbolBody<ELFT>>(*Body));
         else
           Addend += getLocalSymVA(
               Obj.getRelocationSymbol(&RI, File.getSymbolTable()), File);
       }
       P->setSymbolAndType(0, Target->getRelativeReloc(), IsMips64EL);
     }
 
     if (Body && Target->relocNeedsGot(Type, *Body)) {
       P->r_offset = Out<ELFT>::Got->getEntryAddr(*Body);
       if (CanBePreempted)
         P->setSymbolAndType(Body->getDynamicSymbolTableIndex(),
-                            Target->getGotReloc(), IsMips64EL);
+                            Target->getGotReloc(Type), IsMips64EL);
     } else {
       if (IsRela)
         Addend += static_cast<const Elf_Rela &>(RI).r_addend;
       P->r_offset = RI.r_offset + C.getOutputSectionOff() + OutSec->getVA();
       if (CanBePreempted)
         P->setSymbolAndType(Body->getDynamicSymbolTableIndex(), Type,
                             IsMips64EL);
     }
case SymbolBody::DefinedSyntheticKind: {
     return D.Section.getVA() + D.Sym.st_value;
   }
   case SymbolBody::DefinedAbsoluteKind:
     return cast<DefinedAbsolute<ELFT>>(S).Sym.st_value;
   case SymbolBody::DefinedRegularKind: {
     const auto &DR = cast<DefinedRegular<ELFT>>(S);
     const InputSection<ELFT> *SC = &DR.Section;
     OutputSection<ELFT> *OS = SC->getOutputSection();
+    if (DR.Sym.getType() == STT_TLS) {
+      return (OS->getVA() + SC->getOutputSectionOff() + DR.Sym.st_value) -
+             Out<ELFT>::ThreadDataStart;
+    }
     return OS->getVA() + SC->getOutputSectionOff() + DR.Sym.st_value;
   }
   case SymbolBody::DefinedCommonKind:
     return Out<ELFT>::Bss->getVA() + cast<DefinedCommon<ELFT>>(S).OffsetInBSS;
   case SymbolBody::SharedKind:
   case SymbolBody::UndefinedKind:
     return 0;
   case SymbolBody::LazyKind:

ELF/Target.h

 class SymbolBody;
 
 class TargetInfo {
 public:
   llvm::StringRef getDefaultEntry() const { return DefaultEntry; }
   unsigned getPageSize() const { return PageSize; }
   uint64_t getVAStart() const { return VAStart; }
   unsigned getPCRelReloc() const { return PCRelReloc; }
-  unsigned getGotReloc() const { return GotReloc; }
   unsigned getGotRefReloc() const { return GotRefReloc; }
   unsigned getRelativeReloc() const { return RelativeReloc; }
   unsigned getPltEntrySize() const { return PltEntrySize; }
+  virtual uint32_t getGotReloc(uint32_t Type) const { return GotReloc; }
   virtual void writePltEntry(uint8_t *Buf, uint64_t GotEntryAddr,
                              uint64_t PltEntryAddr) const = 0;
   virtual bool isRelRelative(uint32_t Type) const;
-  virtual bool relocNeedsGot(uint32_t Type, const SymbolBody &S) const = 0;
+  /// /returns the number of got slots needed for this relocation, if any. 0
+  ///   means reloc doesn't require a got.
+  virtual int relocNeedsGot(uint32_t Type, const SymbolBody &S) const = 0;
   virtual bool relocPointsToGot(uint32_t Type) const;
   virtual bool relocNeedsPlt(uint32_t Type, const SymbolBody &S) const = 0;
   virtual void relocateOne(uint8_t *Buf, const void *RelP, uint32_t Type,
                            uint64_t BaseAddr, uint64_t SymVA) const = 0;
 
   virtual ~TargetInfo();
 
 protected:
   unsigned PageSize = 4096;
   uint64_t VAStart;
   unsigned PCRelReloc;
   unsigned GotRefReloc;
   unsigned GotReloc;
   unsigned RelativeReloc;
   unsigned PltEntrySize = 8;
   llvm::StringRef DefaultEntry = "_start";
 };
 
 class X86TargetInfo final : public TargetInfo {
 public:
   X86TargetInfo();
   void writePltEntry(uint8_t *Buf, uint64_t GotEntryAddr,
                      uint64_t PltEntryAddr) const override;
-  bool relocNeedsGot(uint32_t Type, const SymbolBody &S) const override;
+  int relocNeedsGot(uint32_t Type, const SymbolBody &S) const override;
   bool relocPointsToGot(uint32_t Type) const override;
   bool relocNeedsPlt(uint32_t Type, const SymbolBody &S) const override;
   void relocateOne(uint8_t *Buf, const void *RelP, uint32_t Type,
                    uint64_t BaseAddr, uint64_t SymVA) const override;
 };
 
 class X86_64TargetInfo final : public TargetInfo {
 public:
   X86_64TargetInfo();
+  uint32_t getGotReloc(uint32_t Type) const override;
   void writePltEntry(uint8_t *Buf, uint64_t GotEntryAddr,
                      uint64_t PltEntryAddr) const override;
-  bool relocNeedsGot(uint32_t Type, const SymbolBody &S) const override;
+  int relocNeedsGot(uint32_t Type, const SymbolBody &S) const override;
   bool relocNeedsPlt(uint32_t Type, const SymbolBody &S) const override;
   void relocateOne(uint8_t *Buf, const void *RelP, uint32_t Type,
                    uint64_t BaseAddr, uint64_t SymVA) const override;
   bool isRelRelative(uint32_t Type) const override;
 };
 
 class PPC64TargetInfo final : public TargetInfo {
 public:
   PPC64TargetInfo();
   void writePltEntry(uint8_t *Buf, uint64_t GotEntryAddr,
                      uint64_t PltEntryAddr) const override;
-  bool relocNeedsGot(uint32_t Type, const SymbolBody &S) const override;
+  int relocNeedsGot(uint32_t Type, const SymbolBody &S) const override;
   bool relocNeedsPlt(uint32_t Type, const SymbolBody &S) const override;
   void relocateOne(uint8_t *Buf, const void *RelP, uint32_t Type,
                    uint64_t BaseAddr, uint64_t SymVA) const override;
 };
 
 class PPCTargetInfo final : public TargetInfo {
 public:
   PPCTargetInfo();
   void writePltEntry(uint8_t *Buf, uint64_t GotEntryAddr,
                      uint64_t PltEntryAddr) const override;
-  bool relocNeedsGot(uint32_t Type, const SymbolBody &S) const override;
+  int relocNeedsGot(uint32_t Type, const SymbolBody &S) const override;
   bool relocNeedsPlt(uint32_t Type, const SymbolBody &S) const override;
   void relocateOne(uint8_t *Buf, const void *RelP, uint32_t Type,
                    uint64_t BaseAddr, uint64_t SymVA) const override;
 };
 
 class ARMTargetInfo final : public TargetInfo {
 public:
   ARMTargetInfo();
   void writePltEntry(uint8_t *Buf, uint64_t GotEntryAddr,
                      uint64_t PltEntryAddr) const override;
-  bool relocNeedsGot(uint32_t Type, const SymbolBody &S) const override;
+  int relocNeedsGot(uint32_t Type, const SymbolBody &S) const override;
   bool relocNeedsPlt(uint32_t Type, const SymbolBody &S) const override;
   void relocateOne(uint8_t *Buf, const void *RelP, uint32_t Type,
                    uint64_t BaseAddr, uint64_t SymVA) const override;
 };
 
 class AArch64TargetInfo final : public TargetInfo {
 public:
   AArch64TargetInfo();
   void writePltEntry(uint8_t *Buf, uint64_t GotEntryAddr,
                      uint64_t PltEntryAddr) const override;
-  bool relocNeedsGot(uint32_t Type, const SymbolBody &S) const override;
+  int relocNeedsGot(uint32_t Type, const SymbolBody &S) const override;
   bool relocNeedsPlt(uint32_t Type, const SymbolBody &S) const override;
   void relocateOne(uint8_t *Buf, const void *RelP, uint32_t Type,
                    uint64_t BaseAddr, uint64_t SymVA) const override;
 };
 
 class MipsTargetInfo final : public TargetInfo {
 public:
   MipsTargetInfo();
   void writePltEntry(uint8_t *Buf, uint64_t GotEntryAddr,
                      uint64_t PltEntryAddr) const override;
-  bool relocNeedsGot(uint32_t Type, const SymbolBody &S) const override;
+  int relocNeedsGot(uint32_t Type, const SymbolBody &S) const override;
   bool relocNeedsPlt(uint32_t Type, const SymbolBody &S) const override;
   void relocateOne(uint8_t *Buf, const void *RelP, uint32_t Type,
                    uint64_t BaseAddr, uint64_t SymVA) const override;
 };
 
 extern std::unique_ptr<TargetInfo> Target;
 }
 }
 
 #endif

ELF/Target.cpp

void X86TargetInfo::writePltEntry(uint8_t *Buf, uint64_t GotEntryAddr,
                                   uint64_t PltEntryAddr) const {
   // jmpl *val; nop; nop
   const uint8_t Inst[] = {0xff, 0x25, 0, 0, 0, 0, 0x90, 0x90};
   memcpy(Buf, Inst, sizeof(Inst));
   assert(isUInt<32>(GotEntryAddr));
   write32le(Buf + 2, GotEntryAddr);
 }
 
-bool X86TargetInfo::relocNeedsGot(uint32_t Type, const SymbolBody &S) const {
+int X86TargetInfo::relocNeedsGot(uint32_t Type, const SymbolBody &S) const {
   return Type == R_386_GOT32 || relocNeedsPlt(Type, S);
 }
 
 bool X86TargetInfo::relocPointsToGot(uint32_t Type) const {
   return Type == R_386_GOTPC;
 }
 
 bool X86TargetInfo::relocNeedsPlt(uint32_t Type, const SymbolBody &S) const {
X86_64TargetInfo::X86_64TargetInfo() {
   // On linux that is controled by vm.mmap_min_addr. At least on some x86_64
   // installs that is 65536, so the first 15 pages cannot be used.
   // Given that, the smallest value that can be used in here is 0x10000.
   // If using 2MB pages, the smallest page aligned address that works is
   // 0x200000, but it looks like every OS uses 4k pages for executables.
   VAStart = 0x10000;
 }
 
+uint32_t X86_64TargetInfo::getGotReloc(uint32_t Type) const {
+  if (Type == R_X86_64_TLSLD)
+    return R_X86_64_DTPMOD64;
+  return GotReloc;
+}
+
 void X86_64TargetInfo::writePltEntry(uint8_t *Buf, uint64_t GotEntryAddr,
                                      uint64_t PltEntryAddr) const {
   // jmpq *val(%rip); nop; nop
   const uint8_t Inst[] = {0xff, 0x25, 0, 0, 0, 0, 0x90, 0x90};
   memcpy(Buf, Inst, sizeof(Inst));
 
   uint64_t NextPC = PltEntryAddr + 6;
   int64_t Delta = GotEntryAddr - NextPC;
   assert(isInt<32>(Delta));
   write32le(Buf + 2, Delta);
 }
 
-bool X86_64TargetInfo::relocNeedsGot(uint32_t Type, const SymbolBody &S) const {
+int X86_64TargetInfo::relocNeedsGot(uint32_t Type, const SymbolBody &S) const {
+  if (Type == R_X86_64_TLSLD)
+    return 2;
   return Type == R_X86_64_GOTPCREL || relocNeedsPlt(Type, S);
 }
 
 bool X86_64TargetInfo::relocNeedsPlt(uint32_t Type, const SymbolBody &S) const {
   switch (Type) {
   default:
     return false;
   case R_X86_64_PC32:

 bool X86_64TargetInfo::isRelRelative(uint32_t Type) const {
   switch (Type) {
   default:
     return false;
   case R_X86_64_PC64:
   case R_X86_64_PC32:
   case R_X86_64_PC16:
   case R_X86_64_PC8:
+  case R_X86_64_DTPOFF32:
     return true;
   }
 }
 
 void X86_64TargetInfo::relocateOne(uint8_t *Buf, const void *RelP,
                                    uint32_t Type, uint64_t BaseAddr,
                                    uint64_t SymVA) const {
   typedef ELFFile<ELF64LE>::Elf_Rela Elf_Rela;
case R_X86_64_32S:
     if (Type == R_X86_64_32 && !isUInt<32>(VA))
       error("R_X86_64_32 out of range");
     else if (!isInt<32>(VA))
       error("R_X86_64_32S out of range");
 
     write32le(Loc, VA);
     break;
   }
+  case R_X86_64_DTPOFF32:
+    write32le(Loc, SymVA);
+    break;
   default:
     error(Twine("unrecognized reloc ") + Twine(Type));
     break;
   }
 }
 
 PPC64TargetInfo::PPC64TargetInfo() {
   // PCRelReloc = FIXME
   // GotReloc = FIXME
   PltEntrySize = 32;
   PageSize = 65536;
   VAStart = 0x10000000;
 }
 void PPC64TargetInfo::writePltEntry(uint8_t *Buf, uint64_t GotEntryAddr,
                                     uint64_t PltEntryAddr) const {}
-bool PPC64TargetInfo::relocNeedsGot(uint32_t Type, const SymbolBody &S) const {
+int PPC64TargetInfo::relocNeedsGot(uint32_t Type, const SymbolBody &S) const {
   return false;
 }
 bool PPC64TargetInfo::relocNeedsPlt(uint32_t Type, const SymbolBody &S) const {
   return false;
 }
 void PPC64TargetInfo::relocateOne(uint8_t *Buf, const void *RelP, uint32_t Type,
                                   uint64_t BaseAddr, uint64_t SymVA) const {
   typedef ELFFile<ELF64BE>::Elf_Rela Elf_Rela;

 PPCTargetInfo::PPCTargetInfo() {
   // PCRelReloc = FIXME
   // GotReloc = FIXME
   PageSize = 65536;
   VAStart = 0x10000000;
 }
 void PPCTargetInfo::writePltEntry(uint8_t *Buf, uint64_t GotEntryAddr,
                                   uint64_t PltEntryAddr) const {}
-bool PPCTargetInfo::relocNeedsGot(uint32_t Type, const SymbolBody &S) const {
+int PPCTargetInfo::relocNeedsGot(uint32_t Type, const SymbolBody &S) const {
   return false;
 }
 bool PPCTargetInfo::relocNeedsPlt(uint32_t Type, const SymbolBody &S) const {
   return false;
 }
 void PPCTargetInfo::relocateOne(uint8_t *Buf, const void *RelP, uint32_t Type,
                                 uint64_t BaseAddr, uint64_t SymVA) const {}
 
 ARMTargetInfo::ARMTargetInfo() {
   // PCRelReloc = FIXME
   // GotReloc = FIXME
   VAStart = 0x8000;
 }
 void ARMTargetInfo::writePltEntry(uint8_t *Buf, uint64_t GotEntryAddr,
                                   uint64_t PltEntryAddr) const {}
-bool ARMTargetInfo::relocNeedsGot(uint32_t Type, const SymbolBody &S) const {
+int ARMTargetInfo::relocNeedsGot(uint32_t Type, const SymbolBody &S) const {
   return false;
 }
 bool ARMTargetInfo::relocNeedsPlt(uint32_t Type, const SymbolBody &S) const {
   return false;
 }
 void ARMTargetInfo::relocateOne(uint8_t *Buf, const void *RelP, uint32_t Type,
                                 uint64_t BaseAddr, uint64_t SymVA) const {}
 
 AArch64TargetInfo::AArch64TargetInfo() {
   // PCRelReloc = FIXME
   // GotReloc = FIXME
   VAStart = 0x400000;
 }
 void AArch64TargetInfo::writePltEntry(uint8_t *Buf, uint64_t GotEntryAddr,
                                       uint64_t PltEntryAddr) const {}
-bool AArch64TargetInfo::relocNeedsGot(uint32_t Type,
+int AArch64TargetInfo::relocNeedsGot(uint32_t Type,
                                       const SymbolBody &S) const {
   return false;
 }
 bool AArch64TargetInfo::relocNeedsPlt(uint32_t Type,
                                       const SymbolBody &S) const {
   return false;
 }
 
MipsTargetInfo::MipsTargetInfo() {
   DefaultEntry = "__start";
   PageSize = 65536;
   VAStart = 0x400000;
 }
 
 void MipsTargetInfo::writePltEntry(uint8_t *Buf, uint64_t GotEntryAddr,
                                    uint64_t PltEntryAddr) const {}
 
-bool MipsTargetInfo::relocNeedsGot(uint32_t Type, const SymbolBody &S) const {
+int MipsTargetInfo::relocNeedsGot(uint32_t Type, const SymbolBody &S) const {
   return false;
 }
 
 bool MipsTargetInfo::relocNeedsPlt(uint32_t Type, const SymbolBody &S) const {
   return false;
 }
 
 void MipsTargetInfo::relocateOne(uint8_t *Buf, const void *RelP, uint32_t Type,
                                  uint64_t BaseAddr, uint64_t SymVA) const {}
 }
 }

ELF/Writer.cpp

private:
 
   llvm::BumpPtrAllocator PAlloc;
   SymbolTable<ELFT> &Symtab;
   std::vector<ProgramHeader<ELFT> *> PHDRs;
   ProgramHeader<ELFT> PhdrPHDR{PT_PHDR, PF_R, 0, 0};
   ProgramHeader<ELFT> FileHeaderPHDR{PT_LOAD, PF_R, 0, 0};
   ProgramHeader<ELFT> InterpPHDR{PT_INTERP, 0, 0, 0};
   ProgramHeader<ELFT> DynamicPHDR{PT_DYNAMIC, 0, 0, 0};
+  ProgramHeader<ELFT> TlsPHDR{ PT_TLS, PF_R, 0, 0 };
 
   uintX_t FileSize;
   uintX_t ProgramHeaderOff;
   uintX_t SectionHeaderOff;
 };
 } // anonymous namespace
 
 template <class ELFT> void lld::elf2::writeResult(SymbolTable<ELFT> *Symtab) {
template <class ELFT> void lld::elf2::writeResult(SymbolTable<ELFT> *Symtab) {
   SymbolTableSection<ELFT> DynSymTab(*Symtab, *Out<ELFT>::DynStrTab);
   Out<ELFT>::DynSymTab = &DynSymTab;
   HashTableSection<ELFT> HashTab;
   Out<ELFT>::HashTab = &HashTab;
   RelocationSection<ELFT> RelaDyn(Symtab->shouldUseRela());
   Out<ELFT>::RelaDyn = &RelaDyn;
   DynamicSection<ELFT> Dynamic(*Symtab);
   Out<ELFT>::Dynamic = &Dynamic;
+  Out<ELFT>::ThreadDataStart = 0;
 
   Writer<ELFT>(*Symtab).run();
 }
 
 // The main function of the writer.
 template <class ELFT> void Writer<ELFT>::run() {
   if (!Config->DiscardAll)
     copyLocalSymbols();
for (const RelType &RI : Rels) {
     SymbolBody *Body = File.getSymbolBody(SymIndex);
     uint32_t Type = RI.getType(IsMips64EL);
     if (Body) {
       if (Target->relocNeedsPlt(Type, *Body)) {
         if (Body->isInPlt())
           continue;
         Out<ELFT>::Plt->addEntry(Body);
       }
-      if (Target->relocNeedsGot(Type, *Body)) {
+      if (int GotCount = Target->relocNeedsGot(Type, *Body)) {
         if (Body->isInGot())
           continue;
-        Out<ELFT>::Got->addEntry(Body);
+        Out<ELFT>::Got->addEntry(Body, GotCount);
       }
     }
-    if (canBePreempted(Body)) {
+    if (canBePreempted(Body) || Type == R_X86_64_TLSLD) {
       Body->setUsedInDynamicReloc();
       Out<ELFT>::RelaDyn->addReloc({C, RI});
     } else if (Config->Shared && !Target->isRelRelative(Type)) {
       Out<ELFT>::RelaDyn->addReloc({C, RI});
     }
   }
 }
 

 }
 
 // Output section ordering is determined by this function.
 template <class ELFT>
 static bool compareOutputSections(OutputSectionBase<ELFT::Is64Bits> *A,
                                   OutputSectionBase<ELFT::Is64Bits> *B) {
   typedef typename ELFFile<ELFT>::uintX_t uintX_t;
 
-  uintX_t AFlags = A->getFlags();
-  uintX_t BFlags = B->getFlags();
-
-  // Allocatable sections go first to reduce the total PT_LOAD size and
-  // so debug info doesn't change addresses in actual code.
-  bool AIsAlloc = AFlags & SHF_ALLOC;
-  bool BIsAlloc = BFlags & SHF_ALLOC;
-  if (AIsAlloc != BIsAlloc)
-    return AIsAlloc;
-
-  // We don't have any special requirements for the relative order of
-  // two non allocatable sections.
-  if (!AIsAlloc)
-    return false;
+  auto CalcRank = [](OutputSectionBase<ELFT::Is64Bits> *Sec) {
+    uintX_t Flags = Sec->getFlags();
+    // The TLS initialization block goes at the start of the read/write PT_LOAD
+    // as it needs to first be loaded into memory so relocations can be applied.
+    // The rest of the flags are ignored as there can only be one TLS
+    // initialization block per binary.
+    if ((Flags & (SHF_ALLOC | SHF_TLS)) == (SHF_ALLOC | SHF_TLS))
+      return 2;
 
+    switch (Flags & (SHF_ALLOC | SHF_WRITE | SHF_EXECINSTR)) {
+    case SHF_ALLOC:
       // We want the read only sections first so that they go in the PT_LOAD
       // covering the program headers at the start of the file.
-  bool AIsWritable = AFlags & SHF_WRITE;
-  bool BIsWritable = BFlags & SHF_WRITE;
-  if (AIsWritable != BIsWritable)
-    return BIsWritable;
-
-  // For a corresponding reason, put non exec sections first (the program
-  // header PT_LOAD is not executable).
-  bool AIsExec = AFlags & SHF_EXECINSTR;
-  bool BIsExec = BFlags & SHF_EXECINSTR;
-  if (AIsExec != BIsExec)
-    return BIsExec;
+      return 0;
+
+    // The relative order of the following 3 cases is arbitrary.
+
+    case SHF_ALLOC | SHF_EXECINSTR:
+      return 1;
+    case SHF_ALLOC | SHF_WRITE:
+      return 3;
+    case SHF_ALLOC | SHF_WRITE | SHF_EXECINSTR:
+      return 4;
+
+    case 0:
+    case SHF_WRITE | SHF_EXECINSTR:
+    case SHF_WRITE:
+    case SHF_EXECINSTR:
+      // Non allocatable sections go last to reduce the total PT_LOAD size and
+      // so debug info doesn't change addresses in actual code. There are also
+      // no special requirements for the relative order of non allocatable
+      // sections.
+      return 5;
+    }
+    llvm_unreachable("Covered switch.");
+  };
+
+  int ARank = CalcRank(A);
+  int BRank = CalcRank(B);
+
+  if (ARank != BRank)
+    return ARank < BRank;
+
+  // There are no special requirements for the relative order of non allocatable
+  // sections.
+  if (ARank == 5)
+    return false;
 
   // If we got here we know that both A and B and in the same PT_LOAD.
   // The last requirement we have is to put nobits section last. The
   // reason is that the only thing the dynamic linker will see about
   // them is a p_memsz that is larger than p_filesz. Seeing that it
   // zeros the end of the PT_LOAD, so that has to correspond to the
   // nobits sections.
   return A->getType() != SHT_NOBITS && B->getType() == SHT_NOBITS;
if (needsInterpSection())
     PHDRs.push_back(&InterpPHDR);
 
   // Create a PHDR for the file header.
   PHDRs.push_back(&FileHeaderPHDR);
   FileHeaderPHDR.Header.p_vaddr = getVAStart();
   FileHeaderPHDR.Header.p_paddr = getVAStart();
   FileHeaderPHDR.Header.p_align = Target->getPageSize();
 
+  // Since .tbss doesn't take up virtual address space, we have to track the
+  // offset from the first .tbss section manually.
+  uintX_t ThreadBSSOffset = 0;
+
   for (OutputSectionBase<ELFT::Is64Bits> *Sec : OutputSections) {
     if (Sec->getSize()) {
       uintX_t Flags = toPHDRFlags(Sec->getFlags());
       ProgramHeader<ELFT> *Last = PHDRs.back();
       if (Last->Header.p_flags != Flags || !needsPHDR<ELFT>(Sec)) {
         // Flags changed. End current PHDR and potentially create a new one.
         if (!Last->Closed) {
           Last->Header.p_filesz = FileOff - Last->Header.p_offset;
           Last->Header.p_memsz = VA - Last->Header.p_vaddr;
           Last->Closed = true;
         }
 
         if (needsPHDR<ELFT>(Sec)) {
           VA = RoundUpToAlignment(VA, Target->getPageSize());
           FileOff = RoundUpToAlignment(FileOff, Target->getPageSize());
           PHDRs.push_back(new (PAlloc)
                               ProgramHeader<ELFT>(PT_LOAD, Flags, FileOff, VA));
         }
       }
     }
 
     uintX_t Align = Sec->getAlign();
     uintX_t Size = Sec->getSize();
-    if (Sec->getFlags() & SHF_ALLOC) {
-      VA = RoundUpToAlignment(VA, Align);
-      Sec->setVA(VA);
-      VA += Size;
-    }
     FileOff = RoundUpToAlignment(FileOff, Align);
     Sec->setFileOffset(FileOff);
     if (Sec->getType() != SHT_NOBITS)
       FileOff += Size;
+    if ((Sec->getFlags() & (SHF_ALLOC | SHF_TLS)) == (SHF_ALLOC | SHF_TLS)) {
+      if (Sec->getType() != SHT_NOBITS)
+        VA = RoundUpToAlignment(VA, Align);
+      uintX_t TVA = RoundUpToAlignment(VA + ThreadBSSOffset, Align);
+      Sec->setVA(TVA);
+      ThreadBSSOffset = TVA - VA;
+      if (Sec->getType() == SHT_NOBITS)
+        ThreadBSSOffset += Size;
+      else
+        VA += Size;
+      if (TlsPHDR.Header.p_offset == 0) {
+        TlsPHDR.Header.p_offset = Sec->getFileOff();
+        TlsPHDR.Header.p_vaddr = Sec->getVA();
+        TlsPHDR.Header.p_paddr = TlsPHDR.Header.p_vaddr;
+        Out<ELFT>::ThreadDataStart = TlsPHDR.Header.p_vaddr;
+      }
+      TlsPHDR.Header.p_filesz += Sec->getType() == SHT_NOBITS ? 0 : Size;
+      TlsPHDR.Header.p_memsz += Size;
+      TlsPHDR.Header.p_align = std::max<uintX_t>(TlsPHDR.Header.p_align, Align);
+    } else if (Sec->getFlags() & SHF_ALLOC) {
+      VA = RoundUpToAlignment(VA, Align);
+      Sec->setVA(VA);
+      VA += Size;
+    }
   }
 
   // Add a PHDR for the dynamic table.
   if (needsDynamicSections())
     PHDRs.push_back(&DynamicPHDR);
+  if (TlsPHDR.Header.p_memsz > 0)
+    PHDRs.push_back(&TlsPHDR);
 
   FileOff += OffsetToAlignment(FileOff, ELFT::Is64Bits ? 8 : 4);
 
   // Fix up the first entry's size.
   PhdrPHDR.Header.p_filesz = sizeof(Elf_Phdr) * PHDRs.size();
   PhdrPHDR.Header.p_memsz = sizeof(Elf_Phdr) * PHDRs.size();
 
   // Add space for section headers.

test/elf2/section-layout.s

 # RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t
 # RUN: ld.lld2 %t -o %tout
 # RUN: llvm-readobj -sections %tout | FileCheck %s
 # REQUIRES: x86
 
 # Check that sections are laid out in the correct order.
 
 .global _start
 .text
 _start:
 
-.section h,""
-.section g,"",@nobits
-.section f,"aw",@nobits
-.section e,"aw"
+.section v,"T",@nobits
+.section u,"T"
+.section t,"x",@nobits
+.section s,"x"
+.section r,"w",@nobits
+.section q,"w"
+.section p,"wx",@nobits
+.section o,"wx"
+.section n,"",@nobits
+.section m,""
+
+.section l,"awx",@nobits
+.section k,"awx"
+.section j,"aw",@nobits
+.section i,"aw"
+.section h,"aT",@nobits
+.section g,"awT",@nobits
+.section f,"aT"
+.section e,"awT"
 .section d,"ax",@nobits
 .section c,"ax"
 .section b,"a",@nobits
 .section a,"a"
 
 // CHECK: Name: a
 // CHECK: Name: b
 // CHECK: Name: c
 // CHECK: Name: d
-// CHECK: Name: e
+
+// TLS sections are only sorted on NOBITS.
 // CHECK: Name: f
+// CHECK: Name: e
 // CHECK: Name: h
 // CHECK: Name: g
+
+// CHECK: Name: i
+// CHECK: Name: j
+// CHECK: Name: k
+// CHECK: Name: l
+
+// Non allocated sections are in input order.
+// CHECK: Name: v
+// CHECK: Name: u
+// CHECK: Name: t
+// CHECK: Name: s
+// CHECK: Name: r
+// CHECK: Name: q
+// CHECK: Name: p
+// CHECK: Name: o
+// CHECK: Name: n
+// CHECK: Name: m

test/elf2/tls-dynamic.s

+// REQUIRES: x86
+// RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t
+// RUN: ld.lld2 -shared %t -o %tout
+// RUN: llvm-readobj -symbols -sections -program-headers -relocations %tout | \
+// RUN:   FileCheck %s
+// RUN: llvm-objdump -d %tout | FileCheck %s --check-prefix=DIS
+
+  .text
+  .globl  _start
+_start:
+  leaq  a@TLSLD(%rip), %rdi
+  callq __tls_get_addr@PLT
+  movl  $1, a@DTPOFF(%rax)
+  movl  $2, c@DTPOFF(%rax)
+  movl  b@DTPOFF(%rax), %esi
+  addl  a@DTPOFF(%rax), %esi
+  addl  c@DTPOFF(%rax), %esi
+  addl  d@DTPOFF(%rax), %esi
+  retq
+
+  .hidden a
+  .section  .tbss,"awT",@nobits
+  .globl  a
+  .align 4
+a:
+  .long 0
+
+  .hidden b
+  .section  .tdata,"awT",@progbits
+  .globl  b
+  .align 4
+b:
+  .long 1
+
+  .hidden c
+  .section  .thread_bss,"awT",@nobits
+  .globl  c
+  .align 4
+c:
+  .long 0
+
+  .hidden d
+  .section  .thread_data,"awT",@progbits
+  .globl  d
+  .align 4
+d:
+  .long 2
+
+// CHECK:          Name: .tdata
+// CHECK-NEXT:     Type: SHT_PROGBITS
+// CHECK-NEXT:     Flags [
+// CHECK-NEXT:       SHF_ALLOC
+// CHECK-NEXT:       SHF_TLS
+// CHECK-NEXT:       SHF_WRITE
+// CHECK-NEXT:     ]
+// CHECK-NEXT:     Address: [[TDATA_ADDR:0x.*]]
+// CHECK-NEXT:     Offset:
+// CHECK-NEXT:     Size: 4
+// CHECK-NEXT:     Link:
+// CHECK-NEXT:     Info:
+// CHECK-NEXT:     AddressAlignment:
+// CHECK-NEXT:     EntrySize:
+// CHECK-NEXT:   }
+// CHECK-NEXT:   Section {
+// CHECK-NEXT:     Index:
+// CHECK-NEXT:     Name: .thread_data
+// CHECK-NEXT:     Type: SHT_PROGBITS
+// CHECK-NEXT:     Flags [
+// CHECK-NEXT:       SHF_ALLOC
+// CHECK-NEXT:       SHF_TLS
+// CHECK-NEXT:       SHF_WRITE
+// CHECK-NEXT:     ]
+// CHECK-NEXT:     Address:
+// CHECK-NEXT:     Offset:
+// CHECK-NEXT:     Size: 4
+// CHECK-NEXT:     Link:
+// CHECK-NEXT:     Info:
+// CHECK-NEXT:     AddressAlignment:
+// CHECK-NEXT:     EntrySize:
+// CHECK-NEXT:   }
+// CHECK-NEXT:   Section {
+// CHECK-NEXT:     Index:
+// CHECK-NEXT:     Name: .tbss
+// CHECK-NEXT:     Type: SHT_NOBITS
+// CHECK-NEXT:     Flags [
+// CHECK-NEXT:       SHF_ALLOC
+// CHECK-NEXT:       SHF_TLS
+// CHECK-NEXT:       SHF_WRITE
+// CHECK-NEXT:     ]
+// CHECK-NEXT:     Address: [[TBSS_ADDR:0x.*]]
+// CHECK-NEXT:     Offset:
+// CHECK-NEXT:     Size: 4
+// CHECK-NEXT:     Link:
+// CHECK-NEXT:     Info:
+// CHECK-NEXT:     AddressAlignment:
+// CHECK-NEXT:     EntrySize:
+// CHECK-NEXT:   }
+// CHECK-NEXT:   Section {
+// CHECK-NEXT:     Index:
+// CHECK-NEXT:     Name: .thread_bss
+// CHECK-NEXT:     Type: SHT_NOBITS
+// CHECK-NEXT:     Flags [
+// CHECK-NEXT:       SHF_ALLOC
+// CHECK-NEXT:       SHF_TLS
+// CHECK-NEXT:       SHF_WRITE
+// CHECK-NEXT:     ]
+// CHECK-NEXT:     Address:
+// CHECK-NEXT:     Offset:
+// CHECK-NEXT:     Size: 4
+// CHECK-NEXT:     Link:
+// CHECK-NEXT:     Info:
+// CHECK-NEXT:     AddressAlignment:
+// CHECK-NEXT:     EntrySize:
+// CHECK-NEXT:   }
+// CHECK-NEXT:   Section {
+// CHECK-NEXT:     Index:
+// CHECK-NEXT:     Name:
+// CHECK-NEXT:     Type:
+// CHECK-NEXT:     Flags [
+// CHECK-NEXT:       SHF_ALLOC
+// CHECK-NEXT:       SHF_WRITE
+// CHECK-NEXT:     ]
+// CHECK-NEXT:     Address: [[TBSS_ADDR]]
+// CHECK:          Name: .got
+// CHECK-NEXT:     Type: SHT_PROGBITS
+// CHECK-NEXT:     Flags [
+// CHECK-NEXT:       SHF_ALLOC
+// CHECK-NEXT:       SHF_WRITE
+// CHECK-NEXT:     ]
+// CHECK-NEXT:     Address: 0x3098
+
+// CHECK: Relocations [
+// CHECK-NOT: R_X86_64_RELATIVE
+// CHECK: {{.*}} R_X86_64_DTPMOD64 - 0x0
+// CHECK-NOT: R_X86_64_RELATIVE
+
+// CHECK:        Symbol {
+// CHECK:          Name: a
+// CHECK-NEXT:     Value: 0x8
+// CHECK-NEXT:     Size: 0
+// CHECK-NEXT:     Binding: Local
+// CHECK-NEXT:     Type: TLS
+// CHECK-NEXT:     Other: 2
+// CHECK-NEXT:     Section: .tbss
+// CHECK-NEXT:   }
+// CHECK-NEXT:   Symbol {
+// CHECK-NEXT:     Name: b
+// CHECK-NEXT:     Value: 0x0
+// CHECK-NEXT:     Size: 0
+// CHECK-NEXT:     Binding: Local
+// CHECK-NEXT:     Type: TLS
+// CHECK-NEXT:     Other: 2
+// CHECK-NEXT:     Section: .tdata
+// CHECK-NEXT:   }
+// CHECK-NEXT:   Symbol {
+// CHECK-NEXT:     Name: c
+// CHECK-NEXT:     Value: 0xC
+// CHECK-NEXT:     Size: 0
+// CHECK-NEXT:     Binding: Local
+// CHECK-NEXT:     Type: TLS
+// CHECK-NEXT:     Other: 2
+// CHECK-NEXT:     Section: .thread_bss
+// CHECK-NEXT:   }
+// CHECK-NEXT:   Symbol {
+// CHECK-NEXT:     Name: d
+// CHECK-NEXT:     Value: 0x4
+// CHECK-NEXT:     Size: 0
+// CHECK-NEXT:     Binding: Local
+// CHECK-NEXT:     Type: TLS
+// CHECK-NEXT:     Other: 2
+// CHECK-NEXT:     Section: .thread_data
+// CHECK-NEXT:   }
+
+// CHECK:          Type: PT_TLS
+// CHECK-NEXT:     Offset:
+// CHECK-NEXT:     VirtualAddress: [[TDATA_ADDR]]
+// CHECK-NEXT:     PhysicalAddress: [[TDATA_ADDR]]
+// CHECK-NEXT:     FileSize: 8
+// CHECK-NEXT:     MemSize: 16
+// CHECK-NEXT:     Flags [
+// CHECK-NEXT:       PF_R
+// CHECK-NEXT:     ]
+// CHECK-NEXT:     Alignment:
+// CHECK-NEXT:   }
+
+// <tls got module slot> = 0x3098, offset = 0x3098 - (0x2000 + 7) = 4241
+
+// DIS: Disassembly of section .text:
+// DIS-NEXT: _start:
+// DIS-NEXT:   2000:  48 8d 3d {{.*}}                 leaq    4241(%rip), %rdi
+// DIS-NEXT:   2007:  e8 {{.*}}                       callq
+// DIS-NEXT:   200c:  c7 80 08 00 00 00 01 00 00 00   movl    $1, 8(%rax)
+// DIS-NEXT:   2016:  c7 80 0c 00 00 00 02 00 00 00   movl    $2, 12(%rax)
+// DIS-NEXT:   2020:  8b b0 00 00 00 00               movl    (%rax), %esi
+// DIS-NEXT:   2026:  03 b0 08 00 00 00               addl    8(%rax), %esi
+// DIS-NEXT:   202c:  03 b0 0c 00 00 00               addl    12(%rax), %esi
+// DIS-NEXT:   2032:  03 b0 04 00 00 00               addl    4(%rax), %esi