[ELF/AArch64] Implemented set of R_AARCH64_TLSDESC_* relocations.

[zatrazz]

Summary

This patch implements next relocations:

R_AARCH64_TLSDESC_ADR_PAGE21, R_AARCH64_TLSDESC_LD64_LO12_NC,
R_AARCH64_TLSDESC_ADD_LO12_NC, R_AARCH64_TLSDESC_CALL.

They have lazy relocation nature, so require entries in got.plt. Each one
is double sized. Also special Plt entry with resolver is created in
addition to single got entry. A module that uses lazy TLSDESC relocations
must define next two entries in dynamic section:

DT_TLSDESC_PLT - Location of PLT entry for TLS descriptor resolver calls.
DT_TLSDESC_GOT - Location of GOT entry used by TLS descriptor resolver PLT entry.

More information can be found in "Thread-Local Storage Descriptors for IA32 and
AMD64/EM64T" and "Thread-Local Storage Descriptors for the ARM platform"
(http://www.fsfla.org/~lxoliva/writeups/TLS/RFC-TLSDESC-x86.txt,
http://www.fsfla.org/~lxoliva/writeups/TLS/RFC-TLSDESC-ARM.txt)

Sample app to see them on AArch64 is:
__thread int foo;
int main() {

foo = 5;

}
aarch64-linux-gnu-g++ -fPIC test.cpp -S -o test.s

PS: this patch is based on http://reviews.llvm.org/D16201

[emaste]

With this and D18331, D18332, D18333 applied, linking FreeBSD/arm64 with lld still gives me:

relocation R_AARCH64_ADR_PREL_PG_HI21 cannot be used when making a shared object; recompile with -fPIC.
relocation R_AARCH64_LDST64_ABS_LO12_NC cannot be used when making a shared object; recompile with -fPIC.
relocation R_AARCH64_ADR_PREL_PG_HI21 cannot be used when making a shared object; recompile with -fPIC.
relocation R_AARCH64_LDST64_ABS_LO12_NC cannot be used when making a shared object; recompile with -fPIC.
relocation R_AARCH64_ADR_PREL_PG_HI21 cannot be used when making a shared object; recompile with -fPIC.
relocation R_AARCH64_LDST64_ABS_LO12_NC cannot be used when making a shared object; recompile with -fPIC.
relocation R_AARCH64_LDST64_ABS_LO12_NC cannot be used when making a shared object; recompile with -fPIC.
relocation R_AARCH64_LDST64_ABS_LO12_NC cannot be used when making a shared object; recompile with -fPIC.

This happens when linking libc.so and is proving slightly tricky to track down.

[zatrazz]

In D18330#380245, @emaste wrote:

With this and D18331, D18332, D18333 applied, linking FreeBSD/arm64 with lld still gives me:

relocation R_AARCH64_ADR_PREL_PG_HI21 cannot be used when making a shared object; recompile with -fPIC.
relocation R_AARCH64_LDST64_ABS_LO12_NC cannot be used when making a shared object; recompile with -fPIC.
relocation R_AARCH64_ADR_PREL_PG_HI21 cannot be used when making a shared object; recompile with -fPIC.
relocation R_AARCH64_LDST64_ABS_LO12_NC cannot be used when making a shared object; recompile with -fPIC.
relocation R_AARCH64_ADR_PREL_PG_HI21 cannot be used when making a shared object; recompile with -fPIC.
relocation R_AARCH64_LDST64_ABS_LO12_NC cannot be used when making a shared object; recompile with -fPIC.
relocation R_AARCH64_LDST64_ABS_LO12_NC cannot be used when making a shared object; recompile with -fPIC.
relocation R_AARCH64_LDST64_ABS_LO12_NC cannot be used when making a shared object; recompile with -fPIC.

This happens when linking libc.so and is proving slightly tricky to track down.

Does this happen while trying to bootstrap or using a smaller testscase?

[emaste]

In D18330#380354, @zatrazz wrote:

Does this happen while trying to bootstrap or using a smaller testscase?

Some small test cases are fine and I can self-host lld and pass ninja check-lld.

[grimar]

So it is really very close with http://reviews.llvm.org/D16201.
Can I abandon my patch then ? If you want and going to support aarch64 stuff - I would be happy with that.

[zatrazz]

In D18330#381237, @grimar wrote:

So it is really very close with http://reviews.llvm.org/D16201.
Can I abandon my patch then ? If you want and going to support aarch64 stuff - I would be happy with that.

I think so, this patch is based directly on yours (I basically rebase against trunk).

[emaste]

Does this happen while trying to bootstrap or using a smaller testscase?

I tracked the issue down by adding an abort() at the "the recompile with -fPIC" warning to find the name of the symbol triggering it. It turned out to be a similar case as I previously had on X86 -- non-hidden symbols in assembly language routines.

A workaround on the FreeBSD side is fairly straightforward and a build is ongoing. I do wonder if we should handle this as GNU ld does, but this can be revisited later on.

[grimar]

In D18330#381272, @zatrazz wrote:

In D18330#381237, @grimar wrote:

So it is really very close with http://reviews.llvm.org/D16201.
Can I abandon my patch then ? If you want and going to support aarch64 stuff - I would be happy with that.

I think so, this patch is based directly on yours (I basically rebase against trunk).

Ok. My last change was rebasing also. But if you are willing to support that, then I am happy to take that from my shoulders :)
I`ll abandon my version.

[emaste]

minor nits and a question

ELF/InputSection.cpp

275

"are specific" seems to be missing information. Maybe "...follow a specific format. They use two..."?

278

Maybe mention "is based on" or such to make it clear this wasn't accidentally copied and pasted from X86-specific stuff.

ELF/OutputSections.cpp

344

I wonder if there's a nicer way to handle this?

ELF/OutputSections.h

184

Comment?

[zatrazz]

ELF/InputSection.cpp

278

What about:

TLS descriptor uses two words in the .got.plt which describes a structure
of a function pointer and its arguments. Both fields will be adjusted at
runtime by the dynamic loader with a single relocation (usually
R_<ARCH>_TLSDESC).

ELF/OutputSections.cpp

344

I though about this, but I think to remove the Tail it would require to maintain two or move relocation vectors (one for normal and another for tlsdesc) and this will just add more complexity IMHO.

However we might change in later if this shows to be a performance issue.

ELF/OutputSections.h

184

I think it would be better to change to Off_GotPltTlsdesc and add a comment such 'the got.plt entry for a tlsdesc Sym'.

[zatrazz]

In D18330#381401, @emaste wrote:

Does this happen while trying to bootstrap or using a smaller testscase?

I tracked the issue down by adding an abort() at the "the recompile with -fPIC" warning to find the name of the symbol triggering it. It turned out to be a similar case as I previously had on X86 -- non-hidden symbols in assembly language routines.

A workaround on the FreeBSD side is fairly straightforward and a build is ongoing. I do wonder if we should handle this as GNU ld does, but this can be revisited later on.

How GNU ld handles it?

[emaste]

ELF/InputSection.cpp

278

Makes sense to me.

ELF/OutputSections.cpp

344

However we might change in later if this shows to be a performance issue.

To me having two vectors seems more clear. Anyway, we can certainly try it both ways alter on and evaluate the performance difference.

[emaste]

In D18330#381755, @zatrazz wrote:

How GNU ld handles it?

I'm investigating exactly what happens, but it seems the two symbols responsible for lld's failure messages (minbrk and curbrk) do not appear in a ld.bfd-linked libc.so and have no relocations.

[• rafael]

ELF/OutputSections.cpp

81

Can you provide a description of what this is?

In general, any reason you cannot just use GlobalDynIndex for the position of the pair of got entries you need?

[grimar]

ELF/OutputSections.cpp

344

Problem here is that 2 vectors are not really a solution. I mean for this place we just need to put these relocations after all others, but actually there are other similar group of relocations also: IRelative, which are also should be placed at tail. If we one day we will want to implement them (I mean dynamic case, for static one we already did it and static case does not have that problem), then 2 vectors probably is not the best design as we will need 3 of them then ?

At the same time, probably, to have two vectors for now would be more clear, I just kept in mind that all stuff above when implemented that in my version of patch and decided to have one vector basing on that thoughts.

[emaste]

This needs to be rebased again; hopefully it can be reviewed soon.

ELF/OutputSections.cpp

344

I tried a quick experiment with a std::vector<int>. First case called push_back 10 times, then insert(v.end() - 10, i)

[zatrazz]

ELF/OutputSections.cpp

81

The 'Tail' basically keep the position of last default relocation since tlsdesc for aarch64 are place after the jump slots and IRELATIVE.

And I think we can GlobalDynIndex instead of using this extra function. I will check that.

[zatrazz]

ELF/OutputSections.cpp

344

Indeed 2 vectors is not a definitely solution, but I think we can use current approach now and think about a more flexible solution later (since now only aarch64 requires tlsdesc support in this layout).

I think a better solution would be to add a variable number of vectors with the addTls* function to add on the specific one (for instance 'jump slot', 'tlsdesc', and 'irelative'). It could be a DenseMap or similar associating an type to a vector. The Target will then provide the desirable layout for the got.plt and the class will write down as specified. What do you think?

[grimar]

ELF/OutputSections.cpp

344

I would not use 2 vectors. Because that is probably not solves the problem in general at all. Having 2 of them now and 3 later just will bring the complication of whole code.
I am sure there should be much simplier solution for that. I have no ideas currently unfortunately though.
So I would leave that as is now, that place anyways needs refactoring, I think we all agree with that.

[zatrazz]

I have rebased and simplified the patch a bit, based on previous comments.
Changes from previous version:

• Reword the comment about TLSDESC description;
• Remove getTailSize function and used internal SymbolBody fields instead;
• Changes PltRev to PltTlsDesc to describe better the idea of the new field.

I have check and bootstrapped the compiler (along with http://reviews.llvm.org/D18332).

[zatrazz]

I forgot to add a fix on AArch64TargetInfo::needsGot and
AArch64TargetInfo::refersToGotEntry in last patch. This one is the
correct update.

[emaste]

ELF/OutputSections.cpp

344

I tried a simple experiment with std::vector<int> to get a sense of the scale of the difference. One case started with 10 elements and then added 13 million more with v.insert(v.end() - 10 , i) while the other case maintained two vectors and concatenated them at the end. I took the 13 million number from the relocation count in @ruiu's note on linking debug Chrome.

The insert approach takes about 200 times as long in this test. Times in ms:

x concat.txt
+ insert.txt
+------------------------------------------------------------------------------+
|x                                                                      ++++  +|
|A                                                                      |_MA_| |
+------------------------------------------------------------------------------+
    N           Min           Max        Median           Avg        Stddev
x   5             2             5             4             4     1.2247449
+   5           770           840           800           802     25.884358
Difference at 95.0% confidence
        798 +/- 26.7237
        19950% +/- 668.094%
        (Student's t, pooled s = 18.3235)

Something like @zatrazz's suggestion above sounds reasonable to me, although I'd be happy enough to have a functional version committed first and the change made in a subsequent commit.

[emaste]

Needs to be rebased again

ELF/Target.cpp

1394–1395

git complained about trailing whitespace here

[zatrazz]

Rebase against master, fixed trailing whitespace and ping.

[emaste]

Moved to D18790

ELF/InputSection.cpp

for (size_t I = 0; I < Num; ++I) {
 
     if (Target->isTlsGlobalDynamicRel(Type) &&
         !Target->canRelaxTls(Type, &Body)) {
       Target->relocateOne(BufLoc, BufEnd, Type, AddrLoc,
                           Out<ELFT>::Got->getGlobalDynAddr(Body) + A);
       continue;
     }
 
+    // TLS descriptor uses two words in the .got.plt which describes a
+    // structure of a function pointer and its arguments. Both fields will
+    // be adjusted at runtime by the dynamic loader with a single
+    // relocation (usually R_<ARCH>_TLSDESC).
+    if (Target->isTlsDescRel(Type)) {
+      Target->relocateOne(BufLoc, BufEnd, Type, AddrLoc,
+                          Body.getGotPltTlsDescVA<ELFT>() + A);
+      continue;
+    }
+
     if (!RelTy::IsRela)
       A += Target->getImplicitAddend(BufLoc, Type);
     if (Config->EMachine == EM_MIPS)
       A += findMipsPairedAddend(Buf, BufLoc, Body, &RI, Rels.end());
     uintX_t SymVA = Body.getVA<ELFT>(A);
 
     if (Target->needsPlt(Type, Body)) {
       SymVA = Body.getPltVA<ELFT>() + A;

ELF/OutputSections.h

template <class ELFT> class GotSection final : public OutputSectionBase<ELFT> {
   typedef typename ELFT::uint uintX_t;
 
 public:
   GotSection();
   void finalize() override;
   void writeTo(uint8_t *Buf) override;
   void addEntry(SymbolBody &Sym);
   bool addDynTlsEntry(SymbolBody &Sym);
+  void addTlsDescEntry();
   bool addTlsIndex();
   bool empty() const { return MipsLocalEntries == 0 && Entries.empty(); }
   uintX_t getMipsLocalFullAddr(const SymbolBody &B);
   uintX_t getMipsLocalPageAddr(uintX_t Addr);
   uintX_t getGlobalDynAddr(const SymbolBody &B) const;
   uintX_t getNumEntries() const { return Entries.size(); }
 
   // Returns the symbol which corresponds to the first entry of the global part
   // of GOT on MIPS platform. It is required to fill up MIPS-specific dynamic
   // table properties.
   // Returns nullptr if the global part is empty.
   const SymbolBody *getMipsFirstGlobalEntry() const;
 
   // Returns the number of entries in the local part of GOT including
   // the number of reserved entries. This method is MIPS-specific.
   unsigned getMipsLocalEntriesNum() const;
 
   uintX_t getTlsIndexVA() { return Base::getVA() + TlsIndexOff; }
+  uintX_t getTlsDescEntryVA() const { return Base::getVA() + TlsDescEntryOff; }
+  bool hasTlsDescEntry() const { return TlsDescEntryOff != (uintX_t)-1; }
 
 private:
   std::vector<const SymbolBody *> Entries;
   uint32_t TlsIndexOff = -1;
+  uintX_t TlsDescEntryOff = -1;
   uint32_t MipsLocalEntries = 0;
   // Output sections referenced by MIPS GOT relocations.
   llvm::SmallPtrSet<const OutputSectionBase<ELFT> *, 10> MipsOutSections;
   llvm::DenseMap<uintX_t, size_t> MipsLocalGotPos;
 
   uintX_t getMipsLocalEntryAddr(uintX_t EntryValue);
 };
 
 template <class ELFT>
 class GotPltSection final : public OutputSectionBase<ELFT> {
   typedef typename ELFT::uint uintX_t;
 
 public:
   GotPltSection();
   void finalize() override;
   void writeTo(uint8_t *Buf) override;
   void addEntry(SymbolBody &Sym);
+  void addTlsDescEntry(SymbolBody &Sym);
   bool empty() const;
 
 private:
+  size_t Tail = 0;
   std::vector<const SymbolBody *> Entries;
 };
 
 template <class ELFT> class PltSection final : public OutputSectionBase<ELFT> {
   typedef OutputSectionBase<ELFT> Base;
   typedef typename ELFT::uint uintX_t;
 
 public:
   PltSection();
   void finalize() override;
   void writeTo(uint8_t *Buf) override;
   void addEntry(SymbolBody &Sym);
-  bool empty() const { return Entries.empty(); }
+  bool empty() const;
+  uintX_t getTlsDescEntryVA() const;
 
 private:
   std::vector<std::pair<const SymbolBody *, unsigned>> Entries;
 };
 
 template <class ELFT> struct DynamicReloc {
   typedef typename ELFT::uint uintX_t;
   uint32_t Type;
 
   // Where the relocation is.
   enum OffsetKind {
     Off_Got,       // The got entry of Sym.
     Off_GotPlt,    // The got.plt entry of Sym.
+    Off_GotPltTlsdesc, // The .got.plt entry for a TLS descriptor.
     Off_Bss,       // The bss entry of Sym (copy reloc).
     Off_Sec,       // The final position of the given input section and offset.
     Off_LTlsIndex, // The local tls index.
     Off_GTlsIndex, // The global tls index of Sym.
     Off_GTlsOffset // The global tls offset of Sym.
   } OKind;
 
   SymbolBody *Sym = nullptr;

 class RelocationSection final : public OutputSectionBase<ELFT> {
   typedef typename ELFT::Rel Elf_Rel;
   typedef typename ELFT::Rela Elf_Rela;
   typedef typename ELFT::uint uintX_t;
 
 public:
   RelocationSection(StringRef Name);
   void addReloc(const DynamicReloc<ELFT> &Reloc);
+  void addTlsDescReloc(const DynamicReloc<ELFT> &Reloc);
   unsigned getRelocOffset();
   void finalize() override;
   void writeTo(uint8_t *Buf) override;
   bool hasRelocs() const { return !Relocs.empty(); }
 
   bool Static = false;
 
 private:
+  size_t Tail = 0;
   std::vector<DynamicReloc<ELFT>> Relocs;
 };
 
 template <class ELFT>
 class OutputSection final : public OutputSectionBase<ELFT> {
 public:
   typedef typename ELFT::Shdr Elf_Shdr;
   typedef typename ELFT::Sym Elf_Sym;
class DynamicSection final : public OutputSectionBase<ELFT> {
   typedef typename ELFT::Rela Elf_Rela;
   typedef typename ELFT::Shdr Elf_Shdr;
   typedef typename ELFT::Sym Elf_Sym;
   typedef typename ELFT::uint uintX_t;
 
   // The .dynamic section contains information for the dynamic linker.
   // The section consists of fixed size entries, which consist of
   // type and value fields. Value are one of plain integers, symbol
-  // addresses, or section addresses. This struct represents the entry.
+  // addresses, or section addresses. Also value can be of special type.
+  // Such ones are handled depending on Tag value in writeTo(). This struct
+  // represents the entry.
   struct Entry {
     int32_t Tag;
     union {
       OutputSectionBase<ELFT> *OutSec;
       uint64_t Val;
       const SymbolBody *Sym;
     };
-    enum KindT { SecAddr, SymAddr, PlainInt } Kind;
+    enum KindT { SecAddr, SymAddr, PlainInt, Special } Kind;
     Entry(int32_t Tag, OutputSectionBase<ELFT> *OutSec)
         : Tag(Tag), OutSec(OutSec), Kind(SecAddr) {}
     Entry(int32_t Tag, uint64_t Val) : Tag(Tag), Val(Val), Kind(PlainInt) {}
     Entry(int32_t Tag, const SymbolBody *Sym)
         : Tag(Tag), Sym(Sym), Kind(SymAddr) {}
+    Entry(int32_t Tag) : Tag(Tag), Kind(Special) {}
   };
 
   // finalize() fills this vector with the section contents. finalize()
   // cannot directly create final section contents because when the
   // function is called, symbol or section addresses are not fixed yet.
   std::vector<Entry> Entries;
 
 public:

ELF/OutputSections.cpp

GotPltSection<ELFT>::GotPltSection()
   this->Header.sh_addralign = sizeof(uintX_t);
 }
 
 template <class ELFT> void GotPltSection<ELFT>::addEntry(SymbolBody &Sym) {
   Sym.GotPltIndex = Target->GotPltHeaderEntriesNum + Entries.size();
   Entries.push_back(&Sym);
 }
 
+template <class ELFT>
+void GotPltSection<ELFT>::addTlsDescEntry(SymbolBody &Sym) {
+  Sym.GlobalDynIndex = Target->GotPltHeaderEntriesNum + Entries.size() - Tail;
+  Entries.push_back(nullptr);
+  Entries.push_back(nullptr);
+  // Some relocations are placed after all others ones.
+  // GotPlt records for them also follows that rule.
+  Sym.GotPltIndex = Tail;
+  Tail += 2;
+}
+
 template <class ELFT> bool GotPltSection<ELFT>::empty() const {
   return Entries.empty();
 }
 
 template <class ELFT> void GotPltSection<ELFT>::finalize() {
   this->Header.sh_size =
       (Target->GotPltHeaderEntriesNum + Entries.size()) * sizeof(uintX_t);
 }
 
 template <class ELFT> void GotPltSection<ELFT>::writeTo(uint8_t *Buf) {
   Target->writeGotPltHeader(Buf);
   Buf += Target->GotPltHeaderEntriesNum * sizeof(uintX_t);
   for (const SymbolBody *B : Entries) {
+    if (B) // This might be TLS descriptor entry.
       Target->writeGotPlt(Buf, B->getPltVA<ELFT>());
     Buf += sizeof(uintX_t);
   }
 }
 
 template <class ELFT>
 GotSection<ELFT>::GotSection()
     : OutputSectionBase<ELFT>(".got", SHT_PROGBITS, SHF_ALLOC | SHF_WRITE) {
   if (Config->EMachine == EM_MIPS)
if (Sym.hasGlobalDynIndex())
     return false;
   Sym.GlobalDynIndex = Target->GotHeaderEntriesNum + Entries.size();
   // Global Dynamic TLS entries take two GOT slots.
   Entries.push_back(&Sym);
   Entries.push_back(nullptr);
   return true;
 }
 
+template <class ELFT> void GotSection<ELFT>::addTlsDescEntry() {
+  if (TlsDescEntryOff != uintX_t(-1))
+    return;
+  Entries.push_back(nullptr);
+  TlsDescEntryOff = (Entries.size() - 1) * sizeof(uintX_t);
+}
+
 // Reserves TLS entries for a TLS module ID and a TLS block offset.
 // In total it takes two GOT slots.
 template <class ELFT> bool GotSection<ELFT>::addTlsIndex() {
   if (TlsIndexOff != uint32_t(-1))
     return false;
   TlsIndexOff = Entries.size() * sizeof(uintX_t);
   Entries.push_back(nullptr);
   Entries.push_back(nullptr);
for (auto &I : Entries) {
     const SymbolBody *B = I.first;
     unsigned RelOff = I.second;
     uint64_t Got =
         Target->UseLazyBinding ? B->getGotPltVA<ELFT>() : B->getGotVA<ELFT>();
     uint64_t Plt = this->getVA() + Off;
     Target->writePlt(Buf + Off, Got, Plt, B->PltIndex, RelOff);
     Off += Target->PltEntrySize;
   }
+
+  // If module uses TLS descriptor relocations, it requires double .got.plt
+  // entry and a special .plt entry as well. This plt entry is used for TLS
+  // descriptor resolver calls, and also dynamic entry DT_TLSDESC_PLT should be
+  // created to hold it's address.
+  if (Out<ELFT>::Got->hasTlsDescEntry())
+    Target->writePltTlsDescEntry(Buf + Off, this->getVA() + Off,
+                                 Out<ELFT>::Got->getTlsDescEntryVA(),
+                                 Out<ELFT>::GotPlt->getVA());
 }
 
 template <class ELFT> void PltSection<ELFT>::addEntry(SymbolBody &Sym) {
   Sym.PltIndex = Entries.size();
   unsigned RelOff = Target->UseLazyBinding
                         ? Out<ELFT>::RelaPlt->getRelocOffset()
                         : Out<ELFT>::RelaDyn->getRelocOffset();
   Entries.push_back(std::make_pair(&Sym, RelOff));
 }
 
+template <class ELFT> bool PltSection<ELFT>::empty() const {
+  return Entries.empty() && !Out<ELFT>::Got->hasTlsDescEntry();
+}
+
+template <class ELFT>
+typename PltSection<ELFT>::uintX_t PltSection<ELFT>::getTlsDescEntryVA() const {
+  return this->getVA() + Target->PltZeroSize +
+         Entries.size() * Target->PltEntrySize;
+}
+
 template <class ELFT> void PltSection<ELFT>::finalize() {
   this->Header.sh_size =
       Target->PltZeroSize + Entries.size() * Target->PltEntrySize;
+  if (Out<ELFT>::Got->hasTlsDescEntry())
+    this->Header.sh_size += Target->PltTlsDescSize;
 }
 
 template <class ELFT>
 RelocationSection<ELFT>::RelocationSection(StringRef Name)
     : OutputSectionBase<ELFT>(Name, Config->Rela ? SHT_RELA : SHT_REL,
                               SHF_ALLOC) {
   this->Header.sh_entsize = Config->Rela ? sizeof(Elf_Rela) : sizeof(Elf_Rel);
   this->Header.sh_addralign = sizeof(uintX_t);
 }
 
-template <class ELFT>
-void RelocationSection<ELFT>::addReloc(const DynamicReloc<ELFT> &Reloc) {
+template <class ELFT> static void checkReloc(const DynamicReloc<ELFT> &Reloc) {
   SymbolBody *Sym = Reloc.Sym;
   if (!Reloc.UseSymVA && Sym)
     Sym->MustBeInDynSym = true;
+}
+
+template <class ELFT>
+void RelocationSection<ELFT>::addReloc(const DynamicReloc<ELFT> &Reloc) {
+  checkReloc(Reloc);
+  // Some relocations should be added after all others.
+  // Like R_*_TLSDESC must go after R_*_JUMP_SLOT ones.
+  // Tail here is amount of such special relocations.
+  Relocs.insert(Relocs.end() - Tail, Reloc);
+}
+
+template <class ELFT>
+void RelocationSection<ELFT>::addTlsDescReloc(const DynamicReloc<ELFT> &Reloc) {
+  checkReloc(Reloc);
   Relocs.push_back(Reloc);
+  ++Tail;
 }
 
 template <class ELFT>
 typename ELFT::uint DynamicReloc<ELFT>::getOffset() const {
   switch (OKind) {
   case Off_GTlsIndex:
     return Out<ELFT>::Got->getGlobalDynAddr(*Sym);
   case Off_GTlsOffset:
     return Out<ELFT>::Got->getGlobalDynAddr(*Sym) + sizeof(uintX_t);
   case Off_LTlsIndex:
     return Out<ELFT>::Got->getTlsIndexVA();
   case Off_Sec:
     return OffsetSec->getOffset(OffsetInSec) + OffsetSec->OutSec->getVA();
   case Off_Bss:
     return cast<SharedSymbol<ELFT>>(Sym)->OffsetInBss + Out<ELFT>::Bss->getVA();
   case Off_Got:
     return Sym->getGotVA<ELFT>();
   case Off_GotPlt:
     return Sym->getGotPltVA<ELFT>();
+  case Off_GotPltTlsdesc:
+    return Sym->getGotPltTlsDescVA<ELFT>();
   }
   llvm_unreachable("invalid offset kind");
 }
 
 template <class ELFT> void RelocationSection<ELFT>::writeTo(uint8_t *Buf) {
   for (const DynamicReloc<ELFT> &Rel : Relocs) {
     auto *P = reinterpret_cast<Elf_Rela *>(Buf);
     Buf += Config->Rela ? sizeof(Elf_Rela) : sizeof(Elf_Rel);
template <class ELFT> void DynamicSection<ELFT>::finalize() {
   if (Out<ELFT>::RelaPlt && Out<ELFT>::RelaPlt->hasRelocs()) {
     Add({DT_JMPREL, Out<ELFT>::RelaPlt});
     Add({DT_PLTRELSZ, Out<ELFT>::RelaPlt->getSize()});
     Add({Config->EMachine == EM_MIPS ? DT_MIPS_PLTGOT : DT_PLTGOT,
          Out<ELFT>::GotPlt});
     Add({DT_PLTREL, uint64_t(Config->Rela ? DT_RELA : DT_REL)});
   }
 
+  if (Out<ELFT>::Got->hasTlsDescEntry()) {
+    Add({DT_TLSDESC_GOT});
+    Add({DT_TLSDESC_PLT});
+  }
+
   Add({DT_SYMTAB, Out<ELFT>::DynSymTab});
   Add({DT_SYMENT, sizeof(Elf_Sym)});
   Add({DT_STRTAB, Out<ELFT>::DynStrTab});
   Add({DT_STRSZ, Out<ELFT>::DynStrTab->getSize()});
   if (Out<ELFT>::GnuHashTab)
     Add({DT_GNU_HASH, Out<ELFT>::GnuHashTab});
   if (Out<ELFT>::HashTab)
     Add({DT_HASH, Out<ELFT>::HashTab});
if (Config->EMachine == EM_MIPS) {
     if (Out<ELFT>::MipsRldMap)
       Add({DT_MIPS_RLD_MAP, Out<ELFT>::MipsRldMap});
   }
 
   // +1 for DT_NULL
   Header.sh_size = (Entries.size() + 1) * Header.sh_entsize;
 }
 
+template <class ELFT, class Entry, class ElfDyn>
+static void handleSpecialDynamic(const Entry &E, ElfDyn *P) {
+  switch (E.Tag) {
+  case DT_TLSDESC_GOT:
+    P->d_un.d_ptr = Out<ELFT>::Got->getTlsDescEntryVA();
+    break;
+  case DT_TLSDESC_PLT:
+    P->d_un.d_ptr = Out<ELFT>::Plt->getTlsDescEntryVA();
+    break;
+  default:
+    assert(false && "Unknown special dynamic section entry");
+  }
+}
+
 template <class ELFT> void DynamicSection<ELFT>::writeTo(uint8_t *Buf) {
   auto *P = reinterpret_cast<Elf_Dyn *>(Buf);
 
   for (const Entry &E : Entries) {
     P->d_tag = E.Tag;
     switch (E.Kind) {
     case Entry::SecAddr:
       P->d_un.d_ptr = E.OutSec->getVA();
       break;
     case Entry::SymAddr:
       P->d_un.d_ptr = E.Sym->template getVA<ELFT>();
       break;
     case Entry::PlainInt:
       P->d_un.d_val = E.Val;
       break;
+    case Entry::Special:
+      handleSpecialDynamic<ELFT>(E, P);
+      break;
     }
     ++P;
   }
 }
 
 template <class ELFT>
 EhFrameHeader<ELFT>::EhFrameHeader()
     : OutputSectionBase<ELFT>(".eh_frame_hdr", llvm::ELF::SHT_PROGBITS,

ELF/Symbols.h

public:
 
   unsigned DynsymIndex = 0;
   uint32_t GlobalDynIndex = -1;
   uint32_t GotIndex = -1;
   uint32_t GotPltIndex = -1;
   uint32_t PltIndex = -1;
   bool hasGlobalDynIndex() { return GlobalDynIndex != uint32_t(-1); }
   bool isInGot() const { return GotIndex != -1U; }
+  bool isInGotPlt() const { return GotPltIndex != -1U; }
   bool isInPlt() const { return PltIndex != -1U; }
 
   void setUsedInRegularObj() { IsUsedInRegularObj = true; }
 
   template <class ELFT>
   typename ELFT::uint getVA(typename ELFT::uint Addend = 0) const;
 
   template <class ELFT> typename ELFT::uint getGotVA() const;
   template <class ELFT> typename ELFT::uint getGotPltVA() const;
+  template <class ELFT> typename ELFT::uint getGotPltTlsDescVA() const;
   template <class ELFT> typename ELFT::uint getPltVA() const;
   template <class ELFT> typename ELFT::uint getSize() const;
 
   // A SymbolBody has a backreference to a Symbol. Originally they are
   // doubly-linked. A backreference will never change. But the pointer
   // in the Symbol may be mutated by the resolver. If you have a
   // pointer P to a SymbolBody and are not sure whether the resolver
   // has chosen the object among other objects having the same name,

ELF/Symbols.cpp

return Out<ELFT>::Got->getVA() +
          (Out<ELFT>::Got->getMipsLocalEntriesNum() + GotIndex) *
              sizeof(typename ELFT::uint);
 }
 
 template <class ELFT> typename ELFT::uint SymbolBody::getGotPltVA() const {
   return Out<ELFT>::GotPlt->getVA() + GotPltIndex * sizeof(typename ELFT::uint);
 }
 
+template <class ELFT>
+typename ELFT::uint SymbolBody::getGotPltTlsDescVA() const {
+  return Out<ELFT>::GotPlt->getVA() +
+           (GlobalDynIndex + GotPltIndex) *
+              sizeof(typename ELFFile<ELFT>::uintX_t);
+}
+
 template <class ELFT> typename ELFT::uint SymbolBody::getPltVA() const {
   return Out<ELFT>::Plt->getVA() + Target->PltZeroSize +
          PltIndex * Target->PltEntrySize;
 }
 
 template <class ELFT> typename ELFT::uint SymbolBody::getSize() const {
   if (auto *B = dyn_cast<DefinedElf<ELFT>>(this))
     return B->Sym.st_size;

 template uint64_t SymbolBody::template getGotVA<ELF64LE>() const;
 template uint64_t SymbolBody::template getGotVA<ELF64BE>() const;
 
 template uint32_t SymbolBody::template getGotPltVA<ELF32LE>() const;
 template uint32_t SymbolBody::template getGotPltVA<ELF32BE>() const;
 template uint64_t SymbolBody::template getGotPltVA<ELF64LE>() const;
 template uint64_t SymbolBody::template getGotPltVA<ELF64BE>() const;
 
+template uint32_t SymbolBody::template getGotPltTlsDescVA<ELF32LE>() const;
+template uint32_t SymbolBody::template getGotPltTlsDescVA<ELF32BE>() const;
+template uint64_t SymbolBody::template getGotPltTlsDescVA<ELF64LE>() const;
+template uint64_t SymbolBody::template getGotPltTlsDescVA<ELF64BE>() const;
+
 template uint32_t SymbolBody::template getPltVA<ELF32LE>() const;
 template uint32_t SymbolBody::template getPltVA<ELF32BE>() const;
 template uint64_t SymbolBody::template getPltVA<ELF64LE>() const;
 template uint64_t SymbolBody::template getPltVA<ELF64BE>() const;
 
 template uint32_t SymbolBody::template getSize<ELF32LE>() const;
 template uint32_t SymbolBody::template getSize<ELF32BE>() const;
 template uint64_t SymbolBody::template getSize<ELF64LE>() const;

ELF/Target.h

 class TargetInfo {
 public:
   uint64_t getVAStart() const;
   virtual bool isTlsInitialExecRel(uint32_t Type) const;
   virtual bool pointsToLocalDynamicGotEntry(uint32_t Type) const;
   virtual bool isTlsLocalDynamicRel(uint32_t Type) const;
   virtual bool isTlsGlobalDynamicRel(uint32_t Type) const;
   virtual uint32_t getDynRel(uint32_t Type) const { return Type; }
+  virtual bool isTlsDescRel(unsigned Type) const {
+    return false;
+  }
   virtual uint32_t getTlsGotRel(uint32_t Type) const { return TlsGotRel; }
   virtual void writeGotHeader(uint8_t *Buf) const {}
   virtual void writeGotPltHeader(uint8_t *Buf) const {}
   virtual void writeGotPlt(uint8_t *Buf, uint64_t Plt) const {};
   virtual uint64_t getImplicitAddend(uint8_t *Buf, uint32_t Type) const;
 
   // If lazy binding is supported, the first entry of the PLT has code
   // to call the dynamic linker to resolve PLT entries the first time
   // they are called. This function writes that code.
   virtual void writePltZero(uint8_t *Buf) const {}
 
   virtual void writePlt(uint8_t *Buf, uint64_t GotEntryAddr,
                         uint64_t PltEntryAddr, int32_t Index,
                         unsigned RelOff) const {}
 
+  virtual void writePltTlsDescEntry(uint8_t *Buf, uint64_t PltEntryAddr,
+                                    uint64_t GotTlsDescEntryAddr,
+                                    uint64_t GotPltVA) const {};
+
   // Returns true if a relocation is just a hint for linker to make for example
   // some code optimization. Such relocations should not be handled as a regular
   // ones and lead to dynamic relocation creation etc.
   virtual bool isHintRel(uint32_t Type) const;
 
   // Returns true if a relocation is relative to the place being relocated,
   // such as relocations used for PC-relative instructions. Such relocations
   // need not be fixed up if an image is loaded to a different address than
   // the link-time address. So we don't have to emit a relocation for the
   // dynamic linker if isRelRelative returns true.
   virtual bool isRelRelative(uint32_t Type) const;
 
   virtual bool isSizeRel(uint32_t Type) const;
   virtual bool needsDynRelative(uint32_t Type) const { return false; }
   virtual bool needsGot(uint32_t Type, const SymbolBody &S) const;
-  virtual bool refersToGotEntry(uint32_t Type) const;
+  virtual bool refersToGotEntry(uint32_t Type, const SymbolBody &S) const;
 
   enum PltNeed { Plt_No, Plt_Explicit, Plt_Implicit };
   PltNeed needsPlt(uint32_t Type, const SymbolBody &S) const;
 
   virtual void relocateOne(uint8_t *Loc, uint8_t *BufEnd, uint32_t Type,
                            uint64_t P, uint64_t SA) const = 0;
   virtual bool isGotRelative(uint32_t Type) const;
   bool canRelaxTls(uint32_t Type, const SymbolBody *S) const;
public:
   // 0x200000, but it looks like every OS uses 4k pages for executables.
   uint64_t VAStart = 0x10000;
 
   uint32_t CopyRel;
   uint32_t GotRel;
   uint32_t PltRel;
   uint32_t RelativeRel;
   uint32_t IRelativeRel;
+  uint32_t TlsDescRel;
   uint32_t TlsGotRel = 0;
   uint32_t TlsModuleIndexRel;
   uint32_t TlsOffsetRel;
   unsigned PltEntrySize = 8;
+  unsigned PltTlsDescSize = 0;
   unsigned PltZeroSize = 0;
   unsigned GotHeaderEntriesNum = 0;
   unsigned GotPltHeaderEntriesNum = 3;
   bool UseLazyBinding = false;
 
 private:
   virtual bool needsCopyRelImpl(uint32_t Type) const;
   virtual bool needsPltImpl(uint32_t Type) const;

ELF/Target.cpp

public:
   size_t relaxTlsGdToLe(uint8_t *Loc, uint8_t *BufEnd, uint32_t Type,
                         uint64_t P, uint64_t SA) const override;
   size_t relaxTlsIeToLe(uint8_t *Loc, uint8_t *BufEnd, uint32_t Type,
                         uint64_t P, uint64_t SA) const override;
   size_t relaxTlsLdToLe(uint8_t *Loc, uint8_t *BufEnd, uint32_t Type,
                         uint64_t P, uint64_t SA) const override;
 
   bool isGotRelative(uint32_t Type) const override;
-  bool refersToGotEntry(uint32_t Type) const override;
+  bool refersToGotEntry(uint32_t Type, const SymbolBody &S) const override;
 };
 
 class X86_64TargetInfo final : public TargetInfo {
 public:
   X86_64TargetInfo();
   uint32_t getDynRel(uint32_t Type) const override;
   uint32_t getTlsGotRel(uint32_t Type) const override;
   bool pointsToLocalDynamicGotEntry(uint32_t Type) const override;
   bool isTlsLocalDynamicRel(uint32_t Type) const override;
   bool isTlsGlobalDynamicRel(uint32_t Type) const override;
   bool isTlsInitialExecRel(uint32_t Type) const override;
   void writeGotPltHeader(uint8_t *Buf) const override;
   void writeGotPlt(uint8_t *Buf, uint64_t Plt) const override;
   void writePltZero(uint8_t *Buf) const override;
   void writePlt(uint8_t *Buf, uint64_t GotEntryAddr, uint64_t PltEntryAddr,
                 int32_t Index, unsigned RelOff) const override;
   bool needsCopyRelImpl(uint32_t Type) const override;
   bool needsGot(uint32_t Type, const SymbolBody &S) const override;
-  bool refersToGotEntry(uint32_t Type) const override;
+  bool refersToGotEntry(uint32_t Type, const SymbolBody &S) const override;
   bool needsPltImpl(uint32_t Type) const override;
   void relocateOne(uint8_t *Loc, uint8_t *BufEnd, uint32_t Type, uint64_t P,
                    uint64_t SA) const override;
   bool isRelRelative(uint32_t Type) const override;
   bool isSizeRel(uint32_t Type) const override;
 
   size_t relaxTlsGdToIe(uint8_t *Loc, uint8_t *BufEnd, uint32_t Type,
                         uint64_t P, uint64_t SA) const override;
void relocateOne(uint8_t *Loc, uint8_t *BufEnd, uint32_t Type, uint64_t P,
                    uint64_t SA) const override;
   bool isRelRelative(uint32_t Type) const override;
 };
 
 class AArch64TargetInfo final : public TargetInfo {
 public:
   AArch64TargetInfo();
   uint32_t getDynRel(uint32_t Type) const override;
-  bool isTlsGlobalDynamicRel(uint32_t Type) const override;
+  bool isTlsDescRel(unsigned Type) const override;
   bool isTlsInitialExecRel(uint32_t Type) const override;
   void writeGotPlt(uint8_t *Buf, uint64_t Plt) const override;
   void writePltZero(uint8_t *Buf) const override;
   void writePlt(uint8_t *Buf, uint64_t GotEntryAddr, uint64_t PltEntryAddr,
                 int32_t Index, unsigned RelOff) const override;
+  void writePltTlsDescEntry(uint8_t *Buf, uint64_t PltEntryAddr,
+                            uint64_t GotTlsDescEntryAddr,
+                            uint64_t GotPltVA) const override;
   uint32_t getTlsGotRel(uint32_t Type) const override;
   bool isRelRelative(uint32_t Type) const override;
   bool needsCopyRelImpl(uint32_t Type) const override;
   bool needsGot(uint32_t Type, const SymbolBody &S) const override;
   bool needsPltImpl(uint32_t Type) const override;
   void relocateOne(uint8_t *Loc, uint8_t *BufEnd, uint32_t Type, uint64_t P,
                    uint64_t SA) const override;
+  bool refersToGotEntry(uint32_t Type, const SymbolBody &) const override;
+
   size_t relaxTlsGdToLe(uint8_t *Loc, uint8_t *BufEnd, uint32_t Type,
                         uint64_t P, uint64_t SA) const override;
   size_t relaxTlsIeToLe(uint8_t *Loc, uint8_t *BufEnd, uint32_t Type,
                         uint64_t P, uint64_t SA) const override;
 
 private:
   static const uint64_t TcbSize = 16;
 };
public:
   void writeGotHeader(uint8_t *Buf) const override;
   bool needsCopyRelImpl(uint32_t Type) const override;
   bool needsGot(uint32_t Type, const SymbolBody &S) const override;
   bool needsPltImpl(uint32_t Type) const override;
   void relocateOne(uint8_t *Loc, uint8_t *BufEnd, uint32_t Type, uint64_t P,
                    uint64_t SA) const override;
   bool isHintRel(uint32_t Type) const override;
   bool isRelRelative(uint32_t Type) const override;
-  bool refersToGotEntry(uint32_t Type) const override;
+  bool refersToGotEntry(uint32_t Type, const SymbolBody &) const override;
 };
 } // anonymous namespace
 
 TargetInfo *createTarget() {
   switch (Config->EMachine) {
   case EM_386:
     return new X86TargetInfo();
   case EM_AARCH64:

 bool TargetInfo::canRelaxTls(uint32_t Type, const SymbolBody *S) const {
   if (Config->Shared || (S && !S->IsTls))
     return false;
 
   // We know we are producing an executable.
 
   // Global-Dynamic relocs can be relaxed to Initial-Exec or Local-Exec
   // depending on the symbol being locally defined or not.
-  if (isTlsGlobalDynamicRel(Type))
+  if (isTlsDescRel(Type) || isTlsGlobalDynamicRel(Type))
     return true;
 
   // Local-Dynamic relocs can be relaxed to Local-Exec.
   if (isTlsLocalDynamicRel(Type))
     return true;
 
   // Initial-Exec relocs can be relaxed to Local-Exec if the symbol is locally
   // defined.

 bool TargetInfo::isSizeRel(uint32_t Type) const { return false; }
 
 bool TargetInfo::needsGot(uint32_t Type, const SymbolBody &S) const {
   return false;
 }
 
 bool TargetInfo::needsPltImpl(uint32_t Type) const { return false; }
 
-bool TargetInfo::refersToGotEntry(uint32_t Type) const { return false; }
+bool TargetInfo::refersToGotEntry(uint32_t Type, const SymbolBody&) const {
+  return false;
+}
 
 TargetInfo::PltNeed TargetInfo::needsPlt(uint32_t Type,
                                          const SymbolBody &S) const {
   if (S.IsGnuIFunc)
     return Plt_Explicit;
   if (S.isPreemptible() && needsPltImpl(Type))
     return Plt_Explicit;
 
TargetInfo::PltNeed TargetInfo::needsPlt(uint32_t Type,
   // a direct reference to a so symbol by creating an undefined symbol with a
   // non zero st_value. Seeing that, the dynamic linker resolves the symbol to
   // the value of the symbol we created. This is true even for got entries, so
   // pointer equality is maintained. To avoid an infinite loop, the only entry
   // that points to the real function is a dedicated got entry used by the
   // plt. That is identified by special relocation types (R_X86_64_JUMP_SLOT,
   // R_386_JMP_SLOT, etc).
   if (S.isShared())
-    if (!Config->Pic && S.IsFunc && !refersToGotEntry(Type))
+    if (!Config->Pic && S.IsFunc && !refersToGotEntry(Type, S))
       return Plt_Implicit;
 
   return Plt_No;
 }
 
 bool TargetInfo::isTlsInitialExecRel(uint32_t Type) const { return false; }
 
 bool TargetInfo::pointsToLocalDynamicGotEntry(uint32_t Type) const {
   return false;
 }
 
 bool TargetInfo::isTlsLocalDynamicRel(uint32_t Type) const { return false; }
 
 bool TargetInfo::isTlsGlobalDynamicRel(uint32_t Type) const {
   return false;
 }
 
 size_t TargetInfo::relaxTls(uint8_t *Loc, uint8_t *BufEnd, uint32_t Type,
                             uint64_t P, uint64_t SA,
                             const SymbolBody &S) const {
-  if (isTlsGlobalDynamicRel(Type)) {
+  if (isTlsDescRel(Type) || isTlsGlobalDynamicRel(Type)) {
     if (S.isPreemptible())
       return relaxTlsGdToIe(Loc, BufEnd, Type, P, SA);
     return relaxTlsGdToLe(Loc, BufEnd, Type, P, SA);
   }
   if (isTlsLocalDynamicRel(Type))
     return relaxTlsLdToLe(Loc, BufEnd, Type, P, SA);
   assert(isTlsInitialExecRel(Type));
   return relaxTlsIeToLe(Loc, BufEnd, Type, P, SA);

 
 bool X86TargetInfo::isGotRelative(uint32_t Type) const {
   // This relocation does not require got entry,
   // but it is relative to got and needs it to be created.
   // Here we request for that.
   return Type == R_386_GOTOFF;
 }
 
-bool X86TargetInfo::refersToGotEntry(uint32_t Type) const {
+bool X86TargetInfo::refersToGotEntry(uint32_t Type, const SymbolBody &S) const {
   return Type == R_386_GOT32;
 }
 
 uint64_t X86TargetInfo::getImplicitAddend(uint8_t *Buf, uint32_t Type) const {
   switch (Type) {
   default:
     return 0;
   case R_386_32:
void X86_64TargetInfo::writePlt(uint8_t *Buf, uint64_t GotEntryAddr,
   write32le(Buf + 12, -Index * PltEntrySize - PltZeroSize - 16);
 }
 
 bool X86_64TargetInfo::needsCopyRelImpl(uint32_t Type) const {
   return Type == R_X86_64_32S || Type == R_X86_64_32 || Type == R_X86_64_PC32 ||
          Type == R_X86_64_64;
 }
 
-bool X86_64TargetInfo::refersToGotEntry(uint32_t Type) const {
+bool X86_64TargetInfo::refersToGotEntry(uint32_t Type,
+                                        const SymbolBody &S) const {
   return Type == R_X86_64_GOTPCREL || Type == R_X86_64_GOTPCRELX ||
          Type == R_X86_64_REX_GOTPCRELX;
 }
 
 bool X86_64TargetInfo::needsGot(uint32_t Type, const SymbolBody &S) const {
   if (Type == R_X86_64_TLSGD)
     return Target->canRelaxTls(Type, &S) && S.isPreemptible();
   if (Type == R_X86_64_GOTTPOFF)
     return !canRelaxTls(Type, &S);
-  return refersToGotEntry(Type) || needsPlt(Type, S);
+  return refersToGotEntry(Type, S) || needsPlt(Type, S);
 }
 
 uint32_t X86_64TargetInfo::getDynRel(uint32_t Type) const {
   if (Type == R_X86_64_PC32 || Type == R_X86_64_32)
     if (Config->Shared)
       error(getELFRelocationTypeName(EM_X86_64, Type) +
             " cannot be a dynamic relocation");
   return Type;

 }
 
 AArch64TargetInfo::AArch64TargetInfo() {
   CopyRel = R_AARCH64_COPY;
   RelativeRel = R_AARCH64_RELATIVE;
   IRelativeRel = R_AARCH64_IRELATIVE;
   GotRel = R_AARCH64_GLOB_DAT;
   PltRel = R_AARCH64_JUMP_SLOT;
+  TlsDescRel = R_AARCH64_TLSDESC;
   TlsGotRel = R_AARCH64_TLS_TPREL64;
   TlsModuleIndexRel = R_AARCH64_TLS_DTPMOD64;
   TlsOffsetRel = R_AARCH64_TLS_DTPREL64;
   UseLazyBinding = true;
   PltEntrySize = 16;
   PltZeroSize = 32;
+  PltTlsDescSize = 32;
+}
+
+bool AArch64TargetInfo::isTlsDescRel(unsigned Type) const {
+  return Type == R_AARCH64_TLSDESC_ADR_PAGE21 ||
+         Type == R_AARCH64_TLSDESC_LD64_LO12_NC ||
+         Type == R_AARCH64_TLSDESC_ADD_LO12_NC ||
+         Type == R_AARCH64_TLSDESC_CALL;
 }
 
 bool AArch64TargetInfo::isRelRelative(uint32_t Type) const {
   switch (Type) {
   default:
     return false;
   case R_AARCH64_PREL32:
   case R_AARCH64_ADR_PREL_LO21:
bool AArch64TargetInfo::isRelRelative(uint32_t Type) const {
   case R_AARCH64_JUMP26:
   case R_AARCH64_CONDBR19:
   case R_AARCH64_TSTBR14:
   case R_AARCH64_PREL64:
     return true;
   }
 }
 
-bool AArch64TargetInfo::isTlsGlobalDynamicRel(uint32_t Type) const {
-  return Type == R_AARCH64_TLSDESC_ADR_PAGE21 ||
-         Type == R_AARCH64_TLSDESC_LD64_LO12_NC ||
-         Type == R_AARCH64_TLSDESC_ADD_LO12_NC ||
-         Type == R_AARCH64_TLSDESC_CALL;
-}
-
 bool AArch64TargetInfo::isTlsInitialExecRel(uint32_t Type) const {
   return Type == R_AARCH64_TLSIE_ADR_GOTTPREL_PAGE21 ||
          Type == R_AARCH64_TLSIE_LD64_GOTTPREL_LO12_NC;
 }
 
 uint32_t AArch64TargetInfo::getDynRel(uint32_t Type) const {
+  if (Type == R_AARCH64_TLSDESC_ADR_PAGE21 ||
+      Type == R_AARCH64_TLSDESC_LD64_LO12_NC ||
+      Type == R_AARCH64_TLSDESC_ADD_LO12_NC ||
+      Type == R_AARCH64_TLSDESC_CALL)
+    return R_AARCH64_TLSDESC;
   if (Type == R_AARCH64_ABS32 || Type == R_AARCH64_ABS64)
     return Type;
   StringRef S = getELFRelocationTypeName(EM_AARCH64, Type);
   error("relocation " + S + " cannot be used when making a shared object; "
                             "recompile with -fPIC.");
   // Keep it going with a dummy value so that we can find more reloc errors.
   return R_AARCH64_ABS32;
 }
void AArch64TargetInfo::writePlt(uint8_t *Buf, uint64_t GotEntryAddr,
   relocateOne(Buf, Buf + 4, R_AARCH64_ADR_PREL_PG_HI21, PltEntryAddr,
               GotEntryAddr);
   relocateOne(Buf + 4, Buf + 8, R_AARCH64_LDST64_ABS_LO12_NC, PltEntryAddr + 4,
               GotEntryAddr);
   relocateOne(Buf + 8, Buf + 12, R_AARCH64_ADD_ABS_LO12_NC, PltEntryAddr + 8,
               GotEntryAddr);
 }
 
+void AArch64TargetInfo::writePltTlsDescEntry(uint8_t *Buf,
+                                             uint64_t PltEntryAddr,
+                                             uint64_t GotTlsDescEntryAddr,
+                                             uint64_t GotPltVA) const {
+  const uint8_t Inst[] = {
+      0xe2, 0x0f, 0xbf, 0xa9, // stp x2, x3, [sp, #-16]!
+      0x02, 0x00, 0x00, 0x90, // adrp x2, Page(DT_TLSDESC_GOT)
+      0x03, 0x00, 0x00, 0x90, // adrp x3, Page(&.got.plt[0])
+      0x42, 0x00, 0x40, 0xf9, // ldr x2, [x2, #0]
+      0x63, 0x00, 0x00, 0x91, // add x3, x3, 0
+      0x40, 0x00, 0x1f, 0xd6, // br x2
+      0x1f, 0x20, 0x03, 0xd5, // nop
+      0x1f, 0x20, 0x03, 0xd5  // nop
+  };
+  memcpy(Buf, Inst, sizeof(Inst));
+  relocateOne(Buf + 4, Buf + 8, R_AARCH64_ADR_PREL_PG_HI21, PltEntryAddr + 4,
+              GotTlsDescEntryAddr);
+  relocateOne(Buf + 8, Buf + 12, R_AARCH64_ADR_PREL_PG_HI21, PltEntryAddr + 8,
+              GotPltVA);
+  relocateOne(Buf + 12, Buf + 16, R_AARCH64_LDST64_ABS_LO12_NC,
+              PltEntryAddr + 12, GotTlsDescEntryAddr);
+  relocateOne(Buf + 16, Buf + 20, R_AARCH64_ADD_ABS_LO12_NC, PltEntryAddr + 16,
+              GotPltVA);
+}
+
 uint32_t AArch64TargetInfo::getTlsGotRel(uint32_t Type) const {
   assert(Type == R_AARCH64_TLSIE_ADR_GOTTPREL_PAGE21 ||
          Type == R_AARCH64_TLSIE_LD64_GOTTPREL_LO12_NC);
   return Type;
 }
 
 bool AArch64TargetInfo::needsCopyRelImpl(uint32_t Type) const {
   switch (Type) {
bool AArch64TargetInfo::needsCopyRelImpl(uint32_t Type) const {
   case R_AARCH64_LDST16_ABS_LO12_NC:
   case R_AARCH64_LDST32_ABS_LO12_NC:
   case R_AARCH64_LDST64_ABS_LO12_NC:
   case R_AARCH64_LDST128_ABS_LO12_NC:
     return true;
   }
 }
 
-bool AArch64TargetInfo::needsGot(uint32_t Type, const SymbolBody &S) const {
+bool AArch64TargetInfo::refersToGotEntry(uint32_t Type,
+                                         const SymbolBody &S) const {
   switch (Type) {
   case R_AARCH64_TLSIE_ADR_GOTTPREL_PAGE21:
   case R_AARCH64_TLSIE_LD64_GOTTPREL_LO12_NC:
     return !canRelaxTls(Type, &S);
   case R_AARCH64_ADR_GOT_PAGE:
   case R_AARCH64_LD64_GOT_LO12_NC:
     return true;
   default:
-    return needsPlt(Type, S);
+    return false;
   }
 }
 
+bool AArch64TargetInfo::needsGot(uint32_t Type, const SymbolBody &S) const {
+  return refersToGotEntry(Type, S) || needsPlt(Type, S);
+}
+
 bool AArch64TargetInfo::needsPltImpl(uint32_t Type) const {
   switch (Type) {
   default:
     return false;
   case R_AARCH64_CALL26:
   case R_AARCH64_CONDBR19:
   case R_AARCH64_JUMP26:
   case R_AARCH64_TSTBR14:
void AArch64TargetInfo::relocateOne(uint8_t *Loc, uint8_t *BufEnd,
   }
   case R_AARCH64_ADR_PREL_LO21: {
     uint64_t X = SA - P;
     checkInt<21>(X, Type);
     updateAArch64Addr(Loc, X & 0x1FFFFF);
     break;
   }
   case R_AARCH64_ADR_PREL_PG_HI21:
-  case R_AARCH64_TLSIE_ADR_GOTTPREL_PAGE21: {
+  case R_AARCH64_TLSIE_ADR_GOTTPREL_PAGE21:
+  case R_AARCH64_TLSDESC_ADR_PAGE21: {
     uint64_t X = getAArch64Page(SA) - getAArch64Page(P);
     checkInt<33>(X, Type);
     updateAArch64Addr(Loc, (X >> 12) & 0x1FFFFF); // X[32:12]
     break;
   }
   case R_AARCH64_CALL26:
   case R_AARCH64_JUMP26: {
     uint64_t X = SA - P;
     checkInt<28>(X, Type);
     or32le(Loc, (X & 0x0FFFFFFC) >> 2);
     break;
   }
   case R_AARCH64_CONDBR19: {
     uint64_t X = SA - P;
     checkInt<21>(X, Type);
     or32le(Loc, (X & 0x1FFFFC) << 3);
     break;
   }
   case R_AARCH64_LD64_GOT_LO12_NC:
   case R_AARCH64_TLSIE_LD64_GOTTPREL_LO12_NC:
+  case R_AARCH64_TLSDESC_LD64_LO12_NC:
     checkAlignment<8>(SA, Type);
     or32le(Loc, (SA & 0xFF8) << 7);
     break;
   case R_AARCH64_LDST128_ABS_LO12_NC:
     or32le(Loc, (SA & 0x0FF8) << 6);
     break;
   case R_AARCH64_LDST16_ABS_LO12_NC:
     or32le(Loc, (SA & 0x0FFC) << 9);
     break;
   case R_AARCH64_LDST8_ABS_LO12_NC:
+  case R_AARCH64_TLSDESC_ADD_LO12_NC:
     or32le(Loc, (SA & 0xFFF) << 10);
     break;
   case R_AARCH64_LDST32_ABS_LO12_NC:
     or32le(Loc, (SA & 0xFFC) << 8);
     break;
   case R_AARCH64_LDST64_ABS_LO12_NC:
     or32le(Loc, (SA & 0xFF8) << 7);
     break;
case R_AARCH64_TLSLE_ADD_TPREL_HI12: {
     updateAArch64Add(Loc, (V & 0xFFF000) >> 12);
     break;
   }
   case R_AARCH64_TLSLE_ADD_TPREL_LO12_NC: {
     uint64_t V = llvm::alignTo(TcbSize, Out<ELF64LE>::TlsPhdr->p_align) + SA;
     updateAArch64Add(Loc, V & 0xFFF);
     break;
   }
+  case R_AARCH64_TLSDESC_CALL:
+    // For relaxation only. Must be used to identify a
+    // BLR instruction which performs an indirect call
+    // to the TLS descriptor function for S + A.
+    break;
   default:
     fatal("unrecognized reloc " + Twine(Type));
   }
 }
 
 size_t AArch64TargetInfo::relaxTlsGdToLe(uint8_t *Loc, uint8_t *BufEnd,
                                          uint32_t Type, uint64_t P,
                                          uint64_t SA) const {
   // TLSDESC Global-Dynamic relocation are in the form:
   //   adrp    x0, :tlsdesc:v             [R_AARCH64_TLSDESC_ADR_PAGE21]
   //   ldr     x1, [x0, #:tlsdesc_lo12:v  [R_AARCH64_TLSDESC_LD64_LO12_NC]
   //   add     x0, x0, :tlsdesc_los:v     [_AARCH64_TLSDESC_ADD_LO12_NC]
   //   .tlsdesccall                       [R_AARCH64_TLSDESC_CALL]
   // And it can optimized to:
   //   movz    x0, #0x0, lsl #16
   //   movk    x0, #0x10
   //   nop
   //   nop
+
+  // AArch64 uses Variant I of TLS data structures. Thread Control Block is
+  // placed before data. Read "ELF Handling For Thread-Local Storage, 3. Run
+  // Time Handling of TLS" by Ulrich Drepper for details.
   uint64_t TPOff = llvm::alignTo(TcbSize, Out<ELF64LE>::TlsPhdr->p_align);
   uint64_t X = SA + TPOff;
   checkUInt<32>(X, Type);
 
   uint32_t NewInst;
   switch (Type) {
   case R_AARCH64_TLSDESC_ADD_LO12_NC:
   case R_AARCH64_TLSDESC_CALL:

 
 template <class ELFT>
 bool MipsTargetInfo<ELFT>::needsCopyRelImpl(uint32_t Type) const {
   return !isRelRelative(Type);
 }
 
 template <class ELFT>
 bool MipsTargetInfo<ELFT>::needsGot(uint32_t Type, const SymbolBody &S) const {
-  return needsPlt(Type, S) || refersToGotEntry(Type);
+  return needsPlt(Type, S) || refersToGotEntry(Type, S);
 }
 
 template <class ELFT>
-bool MipsTargetInfo<ELFT>::refersToGotEntry(uint32_t Type) const {
+bool MipsTargetInfo<ELFT>::refersToGotEntry(uint32_t Type,
+                                            const SymbolBody &S) const {
   return Type == R_MIPS_GOT16 || Type == R_MIPS_CALL16;
 }
 
 template <class ELFT>
 bool MipsTargetInfo<ELFT>::needsPltImpl(uint32_t Type) const {
   return Type == R_MIPS_26;
 }
 

ELF/Writer.cpp

if (Target->isTlsGlobalDynamicRel(Type)) {
     }
     if (!Body.isPreemptible())
       return 1;
     Out<ELFT>::Got->addEntry(Body);
     Out<ELFT>::RelaDyn->addReloc(
         {Target->TlsGotRel, DynamicReloc<ELFT>::Off_Got, false, &Body});
     return 2;
   }
+
+  // If module uses TLS descriptor relocations,
+  // then special entries are created for module:
+  // 1) .got entry to be filled in by the dynamic loader with the
+  //  address of the internal function to be used for lazy relocation of TLS
+  //  descriptors.
+  // 2) Special .plt entry that
+  //  pushes onto the stack the module's link map address, located in the
+  //  GOT portion reserved for the dynamic loader to use, and then jumps to
+  //  the lazy relocation function, using the address stored in the
+  //  TLSDESC_GOT entry.
+  // For each Body itself two words are allocated in .got.plt instead of
+  // usual .got, because these relocations are lazy ones.
+  if (Target->isTlsDescRel(Type)) {
+    if (!Target->canRelaxTls(Type, &Body)) {
+      Out<ELFT>::Got->addTlsDescEntry();
+      if (Body.isInGotPlt())
+        return 1;
+      Out<ELFT>::GotPlt->addTlsDescEntry(Body);
+      Out<ELFT>::RelaPlt->addTlsDescReloc(
+          {Target->TlsDescRel, DynamicReloc<ELFT>::Off_GotPltTlsdesc, &Body});
+      return 2;
+    }
+  }
+
   return 0;
 }
 
 // The reason we have to do this early scan is as follows
 // * To mmap the output file, we need to know the size
 // * For that, we need to know how many dynamic relocs we will have.
 // It might be possible to avoid this by outputting the file with write:
 // * Write the allocated output sections, computing addresses.

test/ELF/aarch64-tls-desc.s

+# RUN: llvm-mc -filetype=obj -triple=aarch64-unknown-freebsd %s -o %tmain.o
+# RUN: ld.lld %tmain.o -shared -o %tout
+# RUN: llvm-objdump -d %tout | FileCheck %s
+# RUN: llvm-readobj -s -r -dynamic-table %tout | FileCheck -check-prefix=READOBJ %s
+# REQUIRES: aarch64
+
+#READOBJ:      Section {
+#READOBJ:       Index:
+#READOBJ:       Name: .plt
+#READOBJ-NEXT:   Type: SHT_PROGBITS
+#READOBJ-NEXT:   Flags [
+#READOBJ-NEXT:     SHF_ALLOC
+#READOBJ-NEXT:     SHF_EXECINSTR
+#READOBJ-NEXT:   ]
+#READOBJ-NEXT:   Address: 0x1030
+#READOBJ-NEXT:   Offset:
+#READOBJ-NEXT:   Size: 80
+#READOBJ-NEXT:   Link:
+#READOBJ-NEXT:   Info:
+#READOBJ-NEXT:   AddressAlignment:
+#READOBJ-NEXT:   EntrySize:
+#READOBJ-NEXT: }
+#READOBJ:      Section {
+#READOBJ:       Index:
+#READOBJ:       Name: .got
+#READOBJ-NEXT:     Type: SHT_PROGBITS
+#READOBJ-NEXT:     Flags [
+#READOBJ-NEXT:       SHF_ALLOC
+#READOBJ-NEXT:       SHF_WRITE
+#READOBJ-NEXT:     ]
+#READOBJ-NEXT:   Address: 0x20C8
+#READOBJ-NEXT:   Offset:
+#READOBJ-NEXT:   Size: 8
+#READOBJ-NEXT:   Link:
+#READOBJ-NEXT:   Info:
+#READOBJ-NEXT:   AddressAlignment:
+#READOBJ-NEXT:   EntrySize:
+#READOBJ-NEXT: }
+#READOBJ:      Section {
+#READOBJ:       Index:
+#READOBJ:       Name: .got.plt
+#READOBJ-NEXT:   Type: SHT_PROGBITS
+#READOBJ-NEXT:   Flags [
+#READOBJ-NEXT:     SHF_ALLOC
+#READOBJ-NEXT:     SHF_WRITE
+#READOBJ-NEXT:   ]
+#READOBJ-NEXT:   Address: 0x3000
+#READOBJ-NEXT:   Offset: 0x3000
+#READOBJ-NEXT:   Size: 64
+#READOBJ-NEXT:   Link: 0
+#READOBJ-NEXT:   Info: 0
+#READOBJ-NEXT:   AddressAlignment: 8
+#READOBJ-NEXT:   EntrySize: 0
+#READOBJ-NEXT: }
+#READOBJ:      Relocations [
+#READOBJ-NEXT:  Section ({{.*}}) .rela.plt {
+## This also checks that SLOT relocations are placed vefore TLSDESC
+## 0x3018 = .got.plt + 0x18 (reserved 3 entries)
+## 0x3020 = next entry (first was single, 8 bytes)
+## 0x3030 = 0x3018 + double entry size (16)
+#READOBJ-NEXT:    0x3018 R_AARCH64_JUMP_SLOT slot 0x0
+#READOBJ-NEXT:    0x3020 R_AARCH64_TLSDESC foo 0x0
+#READOBJ-NEXT:    0x3030 R_AARCH64_TLSDESC bar 0x0
+#READOBJ-NEXT:  }
+#READOBJ-NEXT:]
+#READOBJ:       DynamicSection [
+#READOBJ-NEXT:    Tag                Type                 Name/Value
+#READOBJ-NEXT:    0x0000000000000017 JMPREL               0x2C0
+#READOBJ-NEXT:    0x0000000000000002 PLTRELSZ             72 (bytes)
+#READOBJ-NEXT:    0x0000000000000003 PLTGOT               0x3000
+#READOBJ-NEXT:    0x0000000000000014 PLTREL               RELA
+## 0x20A8 = Location of GOT entry used by TLS descriptor resolver PLT entry
+#READOBJ-NEXT:    0x000000006FFFFEF7 TLSDESC_GOT          0x20C8
+## 0x1040 = Location of PLT entry for TLS descriptor resolver calls.
+#READOBJ-NEXT:    0x000000006FFFFEF6 TLSDESC_PLT          0x1060
+#READOBJ-NEXT:    0x0000000000000006 SYMTAB               0x200
+#READOBJ-NEXT:    0x000000000000000B SYMENT               24 (bytes)
+#READOBJ-NEXT:    0x0000000000000005 STRTAB               0x2A8
+#READOBJ-NEXT:    0x000000000000000A STRSZ                21 (bytes)
+#READOBJ-NEXT:    0x0000000000000004 HASH                 0x278
+#READOBJ-NEXT:    0x0000000000000000 NULL                 0x0
+#READOBJ-NEXT:  ]
+
+#CHECK:      Disassembly of section .text:
+#CHECK-NEXT: _start:
+#CHECK-NEXT: 1000:  80 02 00 54   b.eq  #80
+#CHECK-NEXT: 1004:  00 00 00 d0   adrp  x0, #8192
+## Page(.got.plt[N]) - Page(0x1000) = Page(0x3020) - 0x1000 =
+##   0x3000 - 0x1000 = 0x2000 = 8192
+## 0x20 = 32
+#CHECK-NEXT: 1008:  02 10 40 f9   ldr x2, [x0, #32]
+#CHECK-NEXT: 100c:  00 80 00 91   add x0, x0, #32
+#CHECK-NEXT: 1010:  40 00 3f d6   blr x2
+## Page(.got.plt[N]) - Page(0x1000) = Page(0x3030) - 0x1000 =
+##   0x3000 - 0x1000 = 0x2000 = 8192
+## 0x30 = 48
+#CHECK-NEXT: 1014:  00 00 00 d0   adrp  x0, #8192
+#CHECK-NEXT: 1018:  02 18 40 f9   ldr x2, [x0, #48]
+#CHECK-NEXT: 101c:  00 c0 00 91   add x0, x0, #48
+#CHECK-NEXT: 1020:  40 00 3f d6   blr x2
+#CHECK-NEXT: Disassembly of section .plt:
+#CHECK-NEXT: .plt:
+#CHECK-NEXT:  1030: f0 7b bf a9   stp x16, x30, [sp, #-16]!
+#CHECK-NEXT:  1034: 10 00 00 d0   adrp  x16, #8192
+#CHECK-NEXT:  1038: 11 0a 40 f9   ldr x17, [x16, #16]
+#CHECK-NEXT:  103c: 10 42 00 91   add x16, x16, #16
+#CHECK-NEXT:  1040: 20 02 1f d6   br  x17
+#CHECK-NEXT:  1044: 1f 20 03 d5   nop
+#CHECK-NEXT:  1048: 1f 20 03 d5   nop
+#CHECK-NEXT:  104c: 1f 20 03 d5   nop
+#CHECK-NEXT:  1050: 10 00 00 d0   adrp  x16, #8192
+#CHECK-NEXT:  1054: 11 0e 40 f9   ldr x17, [x16, #24]
+#CHECK-NEXT:  1058: 10 62 00 91   add x16, x16, #24
+#CHECK-NEXT:  105c: 20 02 1f d6   br  x17
+## Page(.got[N]) - Page(P) = Page(0x20C8) - Page(0x1044) =
+##  0x2000 - 0x1000 = 4096
+## Page(.got.plt) - Page(P) = Page(0x3000) - Page(0x1048) =
+##  0x3000 - 0x1000 = 8192
+## 0xC8 = 200
+## 0x0 = 0
+#CHECK-NEXT:  1060: {{.*}} stp  x2, x3, [sp, #-16]!
+#CHECK-NEXT:  1064: {{.*}} adrp x2, #4096
+#CHECK-NEXT:  1068: {{.*}} adrp x3, #8192
+#CHECK-NEXT:  106c: {{.*}} ldr  x2, [x2, #200]
+#CHECK-NEXT:  1070: {{.*}} add  x3, x3, #0
+#CHECK-NEXT:  1074: {{.*}} br   x2
+#CHECK-NEXT:  1078: {{.*}} nop
+#CHECK-NEXT:  107c: {{.*}} nop
+
+.text
+ .global foo
+ .section .tdata,"awT",%progbits
+ .align 2
+ .type foo, %object
+ .size foo, 4
+foo:
+ .word 5
+ .text
+
+.text
+ .global bar
+ .section .tdata,"awT",%progbits
+ .align 2
+ .type bar, %object
+ .size bar, 4
+bar:
+ .word 5
+ .text
+
+.globl _start
+_start:
+ b.eq slot
+
+ adrp x0, :tlsdesc:foo
+ ldr x2, [x0, #:tlsdesc_lo12:foo]
+ add x0, x0, :tlsdesc_lo12:foo
+ .tlsdesccall foo
+ blr x2
+
+ adrp x0, :tlsdesc:bar
+ ldr x2, [x0, #:tlsdesc_lo12:bar]
+ add x0, x0, :tlsdesc_lo12:bar
+ .tlsdesccall bar
+ blr x2