[ELF/AArch64] - Implemented set of R_AARCH64_TLSDESC_* relocations.

[grimar]

Summary

This patch implements next relocations:
R_AARCH64_TLSDESC_ADR_PAGE21, R_AARCH64_TLSDESC_LD64_LO12_NC, R_AARCH64_TLSDESC_ADD_LO12_NC, R_AARCH64_TLSDESC_CALL.

They have lazy relocation nature, so require entries in got.plt. Each one is double sized. Also special Plt entry with resolver is created in addition to single got entry.
A module that uses lazy TLSDESC relocations must define next two entries in dynamic section:

• DT_TLSDESC_PLT - Location of PLT entry for TLS descriptor resolver calls.
• DT_TLSDESC_GOT - Location of GOT entry used by TLS descriptor resolver PLT entry.

More information can be found in "Thread-Local Storage Descriptors for IA32 and AMD64/EM64T" and "Thread-Local Storage Descriptors for the ARM platform" (http://www.fsfla.org/~lxoliva/writeups/TLS/RFC-TLSDESC-x86.txt, http://www.fsfla.org/~lxoliva/writeups/TLS/RFC-TLSDESC-ARM.txt)

Sample app to see them on AArch64 is:
__thread int foo;
int main() {
  foo = 5;
}

aarch64-linux-gnu-g++ -fPIC test.cpp -S -o test.s

[ruiu]

ELF/InputSection.cpp

211

This needs a comment.

ELF/OutputSections.cpp

191

Ditto

253

Ditto

ELF/Writer.cpp

250–273

Ditto

[grimar]

test/ELF/aarch64-tls-desc.s

76

I`ll remove the unknowns once http://reviews.llvm.org/D16224 is commited.

[grimar]

Review comments addressed (added comments).

[grimar]

• Implemented TlsDesc GD->LE relaxation.

[ruiu]

ELF/InputSection.cpp

13–1

... why don't you add

} else if (Body->isTls() && Target->isTlsDescReloc(Type, *Body)) {
  SymVA = Out<ELFT>::GotPlt->getEntryAddr(*Body)
  continue;

here? (I'm not saying that this long if-elseif-elseif is easy to read, but consistency matters.

14

Instead of adding this code,

ELF/Writer.cpp

23

Can you move this code to a new function? This function got too large.

[grimar]

Review comments addressed.

[grimar]

ELF/InputSection.cpp

13–1

In general descriptor relocations are very specific and if for example we have logic for isTlsGlobalDynamicReloc separated (which is also specific case) then why dont do the same for descreloc ?
Code for Target->isTlsLocalDynamicReloc(Type) and Target->isTlsGlobalDynamicReloc(Type) is above it. So I did it exactly for consistency.
Also I think in the similar situation for R_X86_64_GOTTPOFF optimization (http://reviews.llvm.org/D14713?vs=on&id=40303&whitespace=ignore-most#toc, ELF/InputSection.cpp, line 150) you asked me to move the code above uintX_t SymVA :)

ELF/Writer.cpp

23

Created new function for all TLS code in http://reviews.llvm.org/D16354, moved that code.

[ruiu]

Please submit http://reviews.llvm.org/D16354 and sync this patch to HEAD.

[grimar]

Synced with head as requested.

[grimar]

Rui, I know you want to suspend adding complexity to the relocation handler until cleanup, I am not sure if this one does it or not, so this update is mostly to have the lastest vesion of this patch available here as was requested above.

[ruiu]

Yeah, it would be OK if this were a small patch, but this is fairly large and bring in a new concept of TLS descriptor in the linker. I'd like to suspend this. Most of the code will be reusable after the refactoring.

[• rafael]

The refactoring on how dynamic relocations are handled is done (r259829).

We still need to figure out a better way to propagate information for
the relocation scan to relocateOne, but I don't know of anyone doing
that on the immediate future, so this might be a reasonable time to
implement this.

[grimar]

The refactoring on how dynamic relocations are handled is done (r259829).

We still need to figure out a better way to propagate information for
the relocation scan to relocateOne, but I don't know of anyone doing
that on the immediate future, so this might be a reasonable time to
implement this.

it looks r260677 "[ELF/AArch64] Add support to some GD/LE/IS TLS relocations" also implemented something relative,
I didn`t look close on it yet.
Will try to ressurect the patch soon.

George.

[grimar]

• Rebased

[ruiu]

ELF/InputSection.cpp

211

Do you mean special by specific?

215–216

What is not clear in the TLS documents and in LLD's code is what is the ABI requirement and what's an optimization. We probably need to write our own document (read "long comments") to describe the TLS ABI.

[zatrazz]

Although my TLSDESC patch have missing bits (as you pointed out and I am inclided to abandon it), there is something missing on this patch. Although I can bootstrap the compiler itself, the make check-ldd is stuck in:

tools/lld/unittests/DriverTests/DriverTests --gtest_list_tests

(gdb) bt
#0 0x0000007f8842ca04 in lll_lock_wait (futex=0x34c59980, private=0) at lowlevellock.c:46
#1 0x0000007f88428964 in GI___pthread_rwlock_rdlock (rwlock=0x34c59980) at pthread_rwlock_rdlock.c:113
#2 0x0000000001971c54 in llvm::sys::RWMutexImpl::RWMutexImpl() ()
#3 0x0000000001971c54 in llvm::sys::RWMutexImpl::RWMutexImpl() ()
#4 0x0000000001971c54 in llvm::sys::RWMutexImpl::RWMutexImpl() ()
#5 0x0000000001971c54 in llvm::sys::RWMutexImpl::RWMutexImpl() ()
#6 0x0000000001971c54 in llvm::sys::RWMutexImpl::RWMutexImpl() ()
#7 0x0000000001971c54 in llvm::sys::RWMutexImpl::RWMutexImpl() ()
#8 0x0000000001971c54 in llvm::sys::RWMutexImpl::RWMutexImpl() ()
#9 0x0000000001971c54 in llvm::sys::RWMutexImpl::RWMutexImpl() ()
[...]
(gdb) info threads

Id   Target Id         Frame

• 1 Thread 0x7f880c8000 (LWP 14280) "DriverTests" 0x0000007f8842ca04 in __lll_lock_wait (futex=0x34c59980, private=0) at lowlevellock.c:46

Where with my patch I can bootstrap and run all the make-ldd tests without issue. I am debugging this with your patch.

[grimar]

In D16201#359660, @zatrazz wrote:

Although my TLSDESC patch have missing bits (as you pointed out and I am inclided to abandon it), there is something missing on this patch. Although I can bootstrap the compiler itself, the make check-ldd is stuck in:

tools/lld/unittests/DriverTests/DriverTests --gtest_list_tests

(gdb) bt
#0 0x0000007f8842ca04 in lll_lock_wait (futex=0x34c59980, private=0) at lowlevellock.c:46
#1 0x0000007f88428964 in GI___pthread_rwlock_rdlock (rwlock=0x34c59980) at pthread_rwlock_rdlock.c:113
#2 0x0000000001971c54 in llvm::sys::RWMutexImpl::RWMutexImpl() ()
#3 0x0000000001971c54 in llvm::sys::RWMutexImpl::RWMutexImpl() ()
#4 0x0000000001971c54 in llvm::sys::RWMutexImpl::RWMutexImpl() ()
#5 0x0000000001971c54 in llvm::sys::RWMutexImpl::RWMutexImpl() ()
#6 0x0000000001971c54 in llvm::sys::RWMutexImpl::RWMutexImpl() ()
#7 0x0000000001971c54 in llvm::sys::RWMutexImpl::RWMutexImpl() ()
#8 0x0000000001971c54 in llvm::sys::RWMutexImpl::RWMutexImpl() ()
#9 0x0000000001971c54 in llvm::sys::RWMutexImpl::RWMutexImpl() ()
[...]
(gdb) info threads

Id   Target Id         Frame

• 1 Thread 0x7f880c8000 (LWP 14280) "DriverTests" 0x0000007f8842ca04 in __lll_lock_wait (futex=0x34c59980, private=0) at lowlevellock.c:46

Where with my patch I can bootstrap and run all the make-ldd tests without issue. I am debugging this with your patch.

Thanks for info ! I`ll review the output produced by this patch once again.

[grimar]

In D16201#359660, @zatrazz wrote:

Although my TLSDESC patch have missing bits (as you pointed out and I am inclided to abandon it)

Looks your one implements non-lazy case and since your one is already workable, I think there is no real need to abandon it.
This one really can be commited on top if issues with it be resolved.

[grimar]

I found that I can run the next helloworld app fine with this:

main.c

int method();
int main(void) {
  return method();
}

shared.c

int puts(const char *);
__thread int foo;
int method() {
  puts("Hello, World");
  foo = 0;
}

/home/umb/LLVM/build/bin/clang -fuse-ld=lld -target aarch64-linux-gnu -fPIC -shared shared.c -o shared.so
/home/umb/LLVM/build/bin/clang -fuse-ld=lld -target aarch64-linux-gnu -fPIC main.c shared.so -o test
root@ubuntu:/home# ./test
Hello, World

but if I switch lines:

foo = 0;
puts("Hello, World");

Then it crashes:

root@ubuntu:/home# ./test
Inconsistency detected by ld.so: dl-runtime.c: 79: _dl_fixup: Assertion `((reloc->r_info) & 0xffffffff) == 1026' failed!
(source code of dynamic linker for that place: https://github.com/lattera/glibc/blob/master/elf/dl-runtime.c#L86)

My guess that it is because TLSDESC is placed not at the end in that case:
000000003020 000100000402 R_AARCH64_JUMP_SL 0000000000000000 gmon_start + 0
000000003028 000e00000402 R_AARCH64_JUMP_SL 00000000000011b0 __cxa_finalize + 0
000000003030 000c00000407 R_AARCH64_TLSDESC 0000000000000000 foo + 0
000000003040 000500000402 R_AARCH64_JUMP_SL 0000000000000000 puts + 0

I think gold/bfd place it at the end and that is the root of problem of this patch. I am fixing it.
I think this is the first relocation we meet that requires placing after all JUMP_SLOTs ones.
I can remember that also IFunc relocations were placed to the end for dynamic case by gold, but
we did not implement dynamic case for IFunc`s.

[zatrazz]

I am also not sure about the position of the TLSDESC relocation, I will check that.

I also noted that the patches does not create dynamic R_AARCH64_TLSDESC relocations against local symbols. For instance:

__thread int v1;
static __thread int v2;

// We don't use these pointers, but putting them in tests alignment on
// a 64-bit target.
__thread char* p1;
char dummy;
__thread char* p2 = &dummy;

__thread int v3 = 3;
static __thread int v4 = 4;
__thread int v5;
static __thread int v6;

int*
f() {
  return &v2;
}

clang++ -target aarch64-linux-gnu -fpic test.cc -c -o test.o
clang++ -target aarch64-linux-gnu -fpic test.o -shared -o test.so
Improper alignment for relocation R_AARCH64_TLSDESC_LD64_LO12_NC
clang-3.9: error: linker command failed with exit code 1 (use -v to see invocation)

It is due the fact is passing the wrong value on relocateOne (...). Also to avoid extra bogus dynamic relocations with recent changes in trunk I am using:

diff --git a/ELF/Target.cpp b/ELF/Target.cpp
index dbe0cc5..fef5b60 100644
--- a/ELF/Target.cpp
+++ b/ELF/Target.cpp
@@ -1220,11 +1220,14 @@ AArch64TargetInfo::AArch64TargetInfo() {
 }
 
 bool AArch64TargetInfo::isRelRelative(uint32_t Type) const {
-  return Type == R_AARCH64_PREL32 || Type == R_AARCH64_ADR_PREL_PG_HI21 ||
-         Type == R_AARCH64_LDST8_ABS_LO12_NC ||
-         Type == R_AARCH64_LDST32_ABS_LO12_NC ||
-         Type == R_AARCH64_LDST64_ABS_LO12_NC ||
-         Type == R_AARCH64_ADD_ABS_LO12_NC || Type == R_AARCH64_CALL26;
+  switch (Type) {
+  default:
+    return true;
+  case R_AARCH64_ABS64:
+  case R_AARCH64_ADR_GOT_PAGE:
+  case R_AARCH64_LD64_GOT_LO12_NC:
+    return !Config->Shared;
+  }
 }

Which I intend to create a patch to simplify the logic on dynamic relocation creation.

[grimar]

I placed R_AARCH64_TLSDESC after R_AARCH64_JUMP_SLOT ones. And also placed .got.plt entries after common ones for these relocations.
These changes together resolved the issues I observed, helloworld now works fine for me. I hope it is enough for initial version of patch and I am going to update it with this changes soon.

In D16201#363173, @zatrazz wrote:

I am also not sure about the position of the TLSDESC relocation, I will check that.

I also noted that the patches does not create dynamic R_AARCH64_TLSDESC relocations against local symbols.

I`ll check that. But I think that can be fixed in a different patch(s). Changes I wrote about above IMO a good point to commit as this patch is already large.

So I have plans to update it tommorrow.

[grimar]

• Placed R_AARCH64_TLSDESC after R_AARCH64_JUMP_SLOT ones.
• Placed .got.plt entries after common ones for these relocations.
• Updated the testcase.

With that changes helloworld application mentioned above in this thread works fine.

[grimar]

ELF/InputSection.cpp

211

Probably yes. Fixed.

215–216

I am not sure here. There is long comment about how TlsDesc works in Writer.cpp.
If we are talk about optimizations then I removed the aarch64-tls-desc-gdle.s test from this patch, because optimizations for these relocs were commited separately in r260677. And this patch does not contain any mention of optimizations anymore anywhere.

[zatrazz]

I see placing TLSDESC after jump slots seems to be the right way. From elf aarch64 code in glibc (sysdeps/aarch64/dl-machine.h) I see that only TLSDESC is a double GOT, so we should we place it on the end to avoid dynamic relocation handling issues (gold also place the TLSDESC in .got.plt after jump slot and IRELATIVE).

However I still think it is something missing, I am still seeing some issues with bootstrap with an random testcase:

$ gdb ./bin/lld-link
(gdb) r @tools/lld/test/COFF/Output/responsefile.test.tmp.rsp /heap:0x3000
(gdb) bt
#0 0x00454211f0000f10 in ?? ()
#1 0x0000007fb7f52030 in pthread_once_slow (once_control=0x1aac6d0, init_routine=0xf9454211f0000f10) at pthread_once.c:114
#2 0x0000000000900368 in std::future_base::_Async_state_commonV2::_M_complete_async() ()
#3 0x00000000008fd3e4 in std::future<lld::coff::InputFile*>::get() ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) up
[...]
(gdb) disas
[...]

0x0000007fb7f52020 <+132>:   add     x1, x1, #0x7c
0x0000007fb7f52024 <+136>:   mov     x19, x4
0x0000007fb7f52028 <+140>:   bl      0x7fb7f53440 <_pthread_cleanup_push>
0x0000007fb7f5202c <+144>:   blr     x20

> 0x0000007fb7f52030 <+148>: mov x0, x21

ELF/InputSection.cpp

219

I think you could just use the already got addend 'A' at line 235.

ELF/OutputSections.cpp

199

I think this should use the non-specialized 'GotPlt' access (Out<ELT>::GotPlt).

[grimar]

• Addressed Adhemerval's review comments.
• Updated comment.

[grimar]

In D16201#364697, @zatrazz wrote:

However I still think it is something missing, I am still seeing some issues with bootstrap with an random testcase:

That might be not something directly relative with this patch. At least all sample apps that uses TLSDESC relocations are working now. And we probably can land this and continue investigating issues and fix them separatelly. What do you think ?

ELF/InputSection.cpp

219

Done.

ELF/OutputSections.cpp

199

Fixed.

[zatrazz]

Right I think we can investigate it later with this patch as base.

[grimar]

I am sorry that I am pinging this on 4th day and not on 5th day how it declared in llvm polite rules :) I just feel that it is an old enough patch with lot of discussions and it was just forgotten, probably.
But actually it still requires an review and I will appreciate really any comment about it, since it is large and probably needs tweaking.

At least it introduces the approach that adds relocations to the end of .got.plt, after the other ones.
(implemented in void GotPltSection<ELFT>::addTlsDescEntry(SymbolBody *Sym)).
That is the first relocations implemented in lld that requires that.

With this patch the next helloworld apps on AArch64 are running fine:

main.c

int method();
int main(void) {
  return method();
}

shared.c

int puts(const char *);
__thread int foo;
int method() {
  foo = 0;
  puts("Hello, World");
}

And that was not possible before, so I really believe it is useful to have.

[zatrazz]

Sorry if I was not clear, this LGMT to me.

[zatrazz]

Although I think you will need to rebase again, i see that canRelaxTls is now Target agnostic and it will require tlsdesc logic on it.

[grimar]

In D16201#368750, @zatrazz wrote:

Sorry if I was not clear, this LGMT to me.

Yap, but I think we need to wait to know what Rafael and Rui thinks about it.

[ruiu]

Do you know the number of performance gains you can get with this optimization? I'm a bit worried about the complexity of the peephole optimizations we have in the linker as it is probably the most hard-to-read part. Also, as I wrote before, it hard to identify what we have to do and what are optional optimizations (so you can skip that code when reading) by just reading code.

[grimar]

In D16201#371640, @ruiu wrote:

Do you know the number of performance gains you can get with this optimization? I'm a bit worried about the complexity of the peephole optimizations we have in the linker as it is probably the most hard-to-read part. Also, as I wrote before, it hard to identify what we have to do and what are optional optimizations (so you can skip that code when reading) by just reading code.

Probably there is some misunderstanding here. That is not an optimization. I think we actually have no choice about implementing them, It is just the set of relocations that are used on aarch64 for tls. This relocations can be optimized, and that is optional path I believe, that was implemented separatelly (http://reviews.llvm.org/D17980). This patch just implements the basic not optimized way.

Consider the code from my comments above. When we compile it using clang without any additional options it creates such relocations.
test.c

__thread int foo;
int method() {
  foo = 0;
}

clang -target aarch64-linux-gnu -fPIC -c test.c -o test.o
aarch64-linux-gnu-readelf -r test.o

Relocation section '.rela.text' at offset 0x160 contains 4 entries:
  Offset          Info           Type           Sym. Value    Sym. Name + Addend
00000000000c  000600000232 R_AARCH64_TLSDESC 0000000000000000 foo + 0
000000000010  000600000233 R_AARCH64_TLSDESC 0000000000000000 foo + 0
000000000014  000600000234 R_AARCH64_TLSDESC 0000000000000000 foo + 0
000000000018  000600000239 R_AARCH64_TLSDESC 0000000000000000 foo + 0

They requires .got entry, .got.plt and so on but that is not the optimization and just the way how TLSDECS implementation actually works.

test/ELF/aarch64-tls-desc.s

7

I`ll move REQUIRES to be the first.

130

I will also add the space after # to be consistent in this test case.

[zatrazz]

This patch is required because clang only implements the tlsdesc dialect for aarch64. GCC implements both dialects, so you can:

$ gcc test.c -fPIC -o test.o -c -mtls-dialect=trad
$ readelf -r test.o

Relocation section '.rela.text' at offset 0x248 contains 3 entries:

Offset          Info           Type           Sym. Value    Sym. Name + Addend

000000000008 000a00000201 R_AARCH64_TLSGD_A 0000000000000000 foo + 0
00000000000c 000a00000202 R_AARCH64_TLSGD_A 0000000000000000 foo + 0
000000000010 000c0000011b R_AARCH64_CALL26 0000000000000000 __tls_get_addr + 0

Also, even is clang implements the traditional TLS dialect for aarch64, we do need to support tlsdesc one (and it is the default for aarch64).

[grimar]

In D16201#371723, @zatrazz wrote:

This patch is required because clang only implements the tlsdesc dialect for aarch64. GCC implements both dialects, so you can:

$ gcc test.c -fPIC -o test.o -c -mtls-dialect=trad
...
Also, even is clang implements the traditional TLS dialect for aarch64, we do need to support tlsdesc one (and it is the default for aarch64).

So, that mean that without additional options, like "-mtls-dialect=trad", gcc will also produce the output with TLSDESC. That what I was pointing on about clang: it is aarch64 specific tls relocation type that we can't just avoid to support, because it is used often.

[zatrazz]

And as a side note, x86_64 does have tlsdesc abi dialect implemented by gcc. Using the same example:

$ gcc test3.c -fPIC -c -mtls-dialect=gnu2
$ readelf -r test3.o

Relocation section '.rela.text' at offset 0x5b8 contains 2 entries:

Offset          Info           Type           Sym. Value    Sym. Name + Addend

000000000007 000900000022 R_X86_64_GOTPC32_ 0000000000000000 foo - 4
00000000000b 000900000023 R_X86_64_TLSDESC_ 0000000000000000 foo + 0

It does not trigger with object built with clang because it only implementes the traditional TLS argument.

[grimar]

• Rebased
• Cleaned up
• Fixed testcase formatting

[zatrazz]

I have been testing this patch along other that fix aarch64-linux support [1] [2] [3] and I can bootstrap and run both lld own testcases and test-suite with with relative success (test-suite shows 5 runtime failures I am investigating).

This patch also requires a new rebase and some adjustments due addEntry interface changes: instead of using a pointer to SymbolBody, I would recommend to use a reference on addTlsDescEntry (as it is for addEntry).

The only thing I am not sure about this patch is for the architecture that might use the new tlsdesc dialect, saying x86_64, i386, arm, and aarch64; the position seems to be architecture dependent. Using gold implementation as reference:

• x86_64 places TLSDESC relocs, if any, after regular PLT followed by IRELATIVE
• aarch64 and i386 seems to place IRELATIVE, if any, after regular PLT followed by TLSDESC
• arm is not implemented.

So I think this solves aarc64 and possible i386 case (if the idea is to implement it in the future), however I think we will need something more flexible for a possible x86_64 implementation.

[1] http://reviews.llvm.org/D18026
[2] http://reviews.llvm.org/D17980
[3] http://reviews.llvm.org/D18031

[grimar]

In D16201#374826, @zatrazz wrote:

The only thing I am not sure about this patch is for the architecture that might use the new tlsdesc dialect, saying x86_64, i386, arm, and aarch64; the position seems to be architecture dependent. Using gold implementation as reference:

• x86_64 places TLSDESC relocs, if any, after regular PLT followed by IRELATIVE
• aarch64 and i386 seems to place IRELATIVE, if any, after regular PLT followed by TLSDESC

That should not be a problem. We are supporting gnu_ifunc only for static case. Therefore there are no any other than IRELATIVE relocations in rela.plt when they are used.
There is probably no real need in supporting dynamic case in closest future because as far as I know ifunc mostly needed to statically link against multiarch libc.
So for now I would not overcomplicate the solution for that.

[grimar]

ELF/OutputSections.cpp

253

Ah, Relocs is a vector, not list. That is bad line then. I`ll replace it with the list on next iteration.

[zatrazz]

{F1670821}I have rebased this patch against master and checked on an aarch64 machine (using both lld-test and a full bootstrap).

ELF/OutputSections.cpp

253

I do not think this is incorrect since the vector will rearrange the itens by expanding it internally. It might however show performance regression, although I also think changing it to std::list or similar will show less performance and more memory usage (even with itens rearrange by inserting in the middle). If you check this small performance test (http://pastie.org/10764242), std::list append shows pretty much the same performance of the std::vector rearrange (the 13000000 is number of relocation in NewLLD doc).

[grimar]

In D16201#377507, @zatrazz wrote:

{F1670821}I have rebased this patch against master and checked on an aarch64 machine (using both lld-test and a full bootstrap).

Did you mean this, my patch ? Then attached patch is a different one it seems. I am going to rebase this one too soon.

[grimar]

Actually I would put this on hold for now. It is not reviewed for about a month or so, so I think there is a reason for that or this patch just is not "in time" for current lld.
So I am ready to rebase this on request, but will not continue support of that until that.

[zatrazz]

Do you mind if I use this patch as base and repost a similar one for revision? I really want to implement this continue the TLS support on aarch64.

[grimar]

In D16201#378314, @zatrazz wrote:

Do you mind if I use this patch as base and repost a similar one for revision? I really want to implement this continue the TLS support on aarch64.

Sure, please feel free to use this :)

[grimar]

In D16201#378314, @zatrazz wrote:

Do you mind if I use this patch as base and repost a similar one for revision? I really want to implement this continue the TLS support on aarch64.

So since Ed is ready to start investigating lld on FreeBSD/arm64, I think it is reasonable to keep that in rebased state for him. I`ll rebase this in a few hours.
But just in case if you still want to do your own patch based on this that is also fine for me.

[grimar]

• Rebased

ELF/OutputSections.cpp

253

Well, that test uses list of ints, but we have Relocs which is a struct with several fields:

template <class ELFT> struct DynamicReloc {
  typedef typename ELFT::uint uintX_t;
  uint32_t Type;
...
  SymbolBody *Sym = nullptr;
  InputSectionBase<ELFT> *OffsetSec = nullptr;
  uintX_t OffsetInSec = 0;
  bool UseSymVA = false;
  uintX_t Addend = 0;

So shift of one element should cost a bit more.
Perfomance probably will depend a lot on amount of that tail relocations what may vary from application to application.

I don't think we should care much about memory usage here, there are probably not much relocations anyways, every list item should add only 2 pointers for each element, while vector anyways will reserve some unused place too.

I will leave that place of code as is for now.

[emaste]

So since Ed is ready to start investigating lld on FreeBSD/arm64, I think it is reasonable to keep that in rebased state for him. I`ll rebase this in a few hours.

Thank you!

Current status, I've applied this and D18031 and the build fails early on, when linking libc, with:

relocation R_AARCH64_ADR_PREL_PG_HI21 cannot be used when making a shared object; recompile with -fPIC.
relocation R_AARCH64_LDST64_ABS_LO12_NC cannot be used when making a shared object; recompile with -fPIC.
relocation R_AARCH64_ADR_PREL_PG_HI21 cannot be used when making a shared object; recompile with -fPIC.
relocation R_AARCH64_LDST64_ABS_LO12_NC cannot be used when making a shared object; recompile with -fPIC.
relocation R_AARCH64_ADR_PREL_PG_HI21 cannot be used when making a shared object; recompile with -fPIC.
relocation R_AARCH64_LDST64_ABS_LO12_NC cannot be used when making a shared object; recompile with -fPIC.
relocation R_AARCH64_LDST64_ABS_LO12_NC cannot be used when making a shared object; recompile with -fPIC.
relocation R_AARCH64_LDST64_ABS_LO12_NC cannot be used when making a shared object; recompile with -fPIC.
cc: error: linker command failed with exit code 1 (use -v to see invocation)
*** [libc.so.7.full] Error code 1

I'll move on to trying self-hosted clang/llvm/lld builds on FreeBSD first.

[zatrazz]

In D16201#379217, @emaste wrote:

So since Ed is ready to start investigating lld on FreeBSD/arm64, I think it is reasonable to keep that in rebased state for him. I`ll rebase this in a few hours.

Thank you!

Current status, I've applied this and D18031 and the build fails early on, when linking libc, with:

relocation R_AARCH64_ADR_PREL_PG_HI21 cannot be used when making a shared object; recompile with -fPIC.
relocation R_AARCH64_LDST64_ABS_LO12_NC cannot be used when making a shared object; recompile with -fPIC.
relocation R_AARCH64_ADR_PREL_PG_HI21 cannot be used when making a shared object; recompile with -fPIC.
relocation R_AARCH64_LDST64_ABS_LO12_NC cannot be used when making a shared object; recompile with -fPIC.
relocation R_AARCH64_ADR_PREL_PG_HI21 cannot be used when making a shared object; recompile with -fPIC.
relocation R_AARCH64_LDST64_ABS_LO12_NC cannot be used when making a shared object; recompile with -fPIC.
relocation R_AARCH64_LDST64_ABS_LO12_NC cannot be used when making a shared object; recompile with -fPIC.
relocation R_AARCH64_LDST64_ABS_LO12_NC cannot be used when making a shared object; recompile with -fPIC.
cc: error: linker command failed with exit code 1 (use -v to see invocation)
*** [libc.so.7.full] Error code 1

I'll move on to trying self-hosted clang/llvm/lld builds on FreeBSD first.

Yes, with master you will hit this issue. I fixed with http://reviews.llvm.org/D18031 and you will also need http://reviews.llvm.org/D17980 and http://reviews.llvm.org/D18026 (which are not rebased yet).

I plan to resend all the patches mentioned (including this one) today rebased against master (I am currently bootstrapping and checking the test-suite on aarch64).

ELF/OutputSections.cpp

243

Although I did use a vector of ints, if you check the push_back is dominated by internal libc++ calls where insert it is dominated by memmove. And if you use the same benchmark and increase the total number of elements you will see that it still somewhat faster. It is due for general usage of lld, bulk memmove yields much more CPI and performance than pointer chasing and function call from std::list.

[llvm-commits]

I would really ask the reviewers then if they will have a time to take a look on current aarch64 patches.
As updating them in every day lld changing world is really painfull sometimes.

We have multiple aarch64 patches like "on hold" now, I think any progress here would be really appreciated.

Best regards,
George.

---

От: Adhemerval Zanella <[e-mail]>
Отправлено: 21 марта 2016 г. 19:07
Кому: George Rimar; [e-mail]; [e-mail]
Копия: [e-mail]; [e-mail]; [e-mail]; [e-mail]; [e-mail]
Тема: Re: [PATCH] D16201: [ELF/AArch64] - Implemented set of R_AARCH64_TLSDESC_* relocations.

zatrazz added a comment.

In http://reviews.llvm.org/D16201#379217, @emaste wrote:

So since Ed is ready to start investigating lld on FreeBSD/arm64, I think it is reasonable to keep that in rebased state for him. I`ll rebase this in a few hours.

Thank you!

Current status, I've applied this and http://reviews.llvm.org/D18031 and the build fails early on, when linking libc, with:

relocation R_AARCH64_ADR_PREL_PG_HI21 cannot be used when making a shared object; recompile with -fPIC.
relocation R_AARCH64_LDST64_ABS_LO12_NC cannot be used when making a shared object; recompile with -fPIC.
relocation R_AARCH64_ADR_PREL_PG_HI21 cannot be used when making a shared object; recompile with -fPIC.
relocation R_AARCH64_LDST64_ABS_LO12_NC cannot be used when making a shared object; recompile with -fPIC.
relocation R_AARCH64_ADR_PREL_PG_HI21 cannot be used when making a shared object; recompile with -fPIC.
relocation R_AARCH64_LDST64_ABS_LO12_NC cannot be used when making a shared object; recompile with -fPIC.
relocation R_AARCH64_LDST64_ABS_LO12_NC cannot be used when making a shared object; recompile with -fPIC.
relocation R_AARCH64_LDST64_ABS_LO12_NC cannot be used when making a shared object; recompile with -fPIC.
cc: error: linker command failed with exit code 1 (use -v to see invocation)
*** [libc.so.7.full] Error code 1

I'll move on to trying self-hosted clang/llvm/lld builds on FreeBSD first.

Yes, with master you will hit this issue. I fixed with http://reviews.llvm.org/D18031 and you will also need http://reviews.llvm.org/D17980 and http://reviews.llvm.org/D18026 (which are not rebased yet).

I plan to resend all the patches mentioned (including this one) today rebased against master (I am currently bootstrapping and checking the test-suite on aarch64).

Comment at: ELF/OutputSections.cpp:294
@@ +293,3 @@
+ Relocs.insert(Relocs.end() - Tail, Reloc);
+}

+

Although I did use a vector of ints, if you check the push_back is dominated by internal libc++ calls where insert it is dominated by memmove. And if you use the same benchmark and increase the total number of elements you will see that it still somewhat faster. It is due for general usage of lld, bulk memmove yields much more CPI and performance than pointer chasing and function call from std::list.

http://reviews.llvm.org/D16201

---

llvm-commits mailing list
[e-mail]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits

[emaste]

I plan to resend all the patches mentioned (including this one) today rebased against master (I am currently bootstrapping and checking the test-suite on aarch64).

Ok, once that's ready I will apply all of those and try the FreeBSD/arm64 build.

[grimar]

Let support all of that in one place: http://reviews.llvm.org/D18330

[emaste]

In D16201#381416, @grimar wrote:

Let support all of that in one place: http://reviews.llvm.org/D18330

If you put just the review ID phabricator will add a cross-reference to the target also: D18330.

[grimar]

In D16201#381424, @emaste wrote:

In D16201#381416, @grimar wrote:

Let support all of that in one place: http://reviews.llvm.org/D18330

If you put just the review ID phabricator will add a cross-reference to the target also: D18330.

Sorry, what I actually wanted it was to abandon this. I forgot to choose the status it seems.

ELF/InputSection.cpp

 //===- InputSection.cpp ---------------------------------------------------===//
 //
 //                             The LLVM Linker
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #include "InputSection.h"
 #include "Config.h"
 #include "Error.h"
 #include "InputFiles.h"
 #include "OutputSections.h"
 #include "Target.h"
 
 using namespace llvm;
 using namespace llvm::ELF;
 using namespace llvm::object;
 
 using namespace lld;
 using namespace lld::elf2;
for (size_t I = 0; I < Num; ++I) {
     if (Target->isTlsGlobalDynamicReloc(Type) &&
         !Target->isTlsOptimized(Type, Body)) {
       Target->relocateOne(BufLoc, BufEnd, Type, AddrLoc,
                           Out<ELFT>::Got->getGlobalDynAddr(*Body) +
                               getAddend<ELFT>(RI));
       continue;
     }
 
+    // TLS descriptor lazy relocations are specific. They
+    // uses two words in the .got.plt. A single relocation is to be used to
+    // compute the value of the two words of the TLS descriptor.
+    // Overall design can be found in
+    // "Thread-Local Storage Descriptors for IA32 and AMD64/EM64T"
+    // http://www.fsfla.org/~lxoliva/writeups/TLS/RFC-TLSDESC-x86.txt
+    if (Body->isTls() && Target->isTlsDescReloc(Type, *Body)) {
+      Target->relocateOne(BufLoc, BufEnd, Type, AddrLoc,
+                          Out<ELFT>::GotPlt->getEntryAddr(*Body) +
+                              getAddend<ELFT>(RI));
+      continue;
+    }
+
     uintX_t SymVA = getSymVA<ELFT>(*Body);
     if (Target->relocNeedsPlt(Type, *Body)) {
       SymVA = Out<ELFT>::Plt->getEntryAddr(*Body);
     } else if (Target->relocNeedsGot(Type, *Body)) {
       SymVA = Out<ELFT>::Got->getEntryAddr(*Body);
       if (Body->isTls())
         Type = Target->getTlsGotReloc(Type);
     } else if (!Target->needsCopyRel(Type, *Body) &&

ELF/OutputSections.h

 
 public:
   GotSection();
   void finalize() override;
   void writeTo(uint8_t *Buf) override;
   void addEntry(SymbolBody *Sym);
   bool addDynTlsEntry(SymbolBody *Sym);
   bool addCurrentModuleTlsIndex();
+  void addTlsDescEntry();
   bool empty() const { return Entries.empty(); }
   uintX_t getEntryAddr(const SymbolBody &B) const;
   uintX_t getGlobalDynAddr(const SymbolBody &B) const;
   uintX_t getNumEntries() const { return Entries.size(); }
 
   // Returns the symbol which corresponds to the first entry of the global part
   // of GOT on MIPS platform. It is required to fill up MIPS-specific dynamic
   // table properties.
   // Returns nullptr if the global part is empty.
   const SymbolBody *getMipsFirstGlobalEntry() const;
 
   // Returns the number of entries in the local part of GOT including
   // the number of reserved entries. This method is MIPS-specific.
   unsigned getMipsLocalEntriesNum() const;
 
   uint32_t getLocalTlsIndexVA() { return Base::getVA() + LocalTlsIndexOff; }
+  uintX_t getTlsDescEntryVA() const { return Base::getVA() + TlsDescEntryOff; }
+  bool hasTlsDescEntry() const { return TlsDescEntryOff != (uintX_t)-1; }
 
 private:
   std::vector<const SymbolBody *> Entries;
   uint32_t LocalTlsIndexOff = -1;
+  uintX_t TlsDescEntryOff = -1;
 };
 
 template <class ELFT>
 class GotPltSection final : public OutputSectionBase<ELFT> {
   typedef typename llvm::object::ELFFile<ELFT>::uintX_t uintX_t;
 
 public:
   GotPltSection();
   void finalize() override;
   void writeTo(uint8_t *Buf) override;
   void addEntry(SymbolBody *Sym);
+  void addTlsDescEntry(SymbolBody *Sym);
   bool empty() const;
   uintX_t getEntryAddr(const SymbolBody &B) const;
 
 private:
   std::vector<const SymbolBody *> Entries;
 };
 
 template <class ELFT> class PltSection final : public OutputSectionBase<ELFT> {
   typedef OutputSectionBase<ELFT> Base;
   typedef typename Base::uintX_t uintX_t;
 
 public:
   PltSection();
   void finalize() override;
   void writeTo(uint8_t *Buf) override;
   void addEntry(SymbolBody *Sym);
-  bool empty() const { return Entries.empty(); }
+  bool empty() const;
   uintX_t getEntryAddr(const SymbolBody &B) const;
+  uintX_t getTlsDescEntryVA() const;
 
 private:
   std::vector<std::pair<const SymbolBody *, unsigned>> Entries;
 };
 
 template <class ELFT> struct DynamicReloc {
   typedef typename llvm::object::ELFFile<ELFT>::Elf_Rel Elf_Rel;
   InputSectionBase<ELFT> *C;

ELF/OutputSections.cpp

GotPltSection<ELFT>::GotPltSection()
   this->Header.sh_addralign = sizeof(uintX_t);
 }
 
 template <class ELFT> void GotPltSection<ELFT>::addEntry(SymbolBody *Sym) {
   Sym->GotPltIndex = Target->getGotPltHeaderEntriesNum() + Entries.size();
   Entries.push_back(Sym);
 }
 
+template <class ELFT> void GotPltSection<ELFT>::addTlsDescEntry(SymbolBody *Sym) {
+  Sym->GotPltIndex = Target->getGotPltHeaderEntriesNum() + Entries.size();
+  Entries.push_back(nullptr);
+  Entries.push_back(nullptr);
+}
+
 template <class ELFT> bool GotPltSection<ELFT>::empty() const {
   return Entries.empty();
 }
 
 template <class ELFT>
 typename GotPltSection<ELFT>::uintX_t
 GotPltSection<ELFT>::getEntryAddr(const SymbolBody &B) const {
   return this->getVA() + B.GotPltIndex * sizeof(uintX_t);
 }
 
 template <class ELFT> void GotPltSection<ELFT>::finalize() {
   this->Header.sh_size =
       (Target->getGotPltHeaderEntriesNum() + Entries.size()) * sizeof(uintX_t);
 }
 
 template <class ELFT> void GotPltSection<ELFT>::writeTo(uint8_t *Buf) {
   Target->writeGotPltHeaderEntries(Buf);
   Buf += Target->getGotPltHeaderEntriesNum() * sizeof(uintX_t);
   for (const SymbolBody *B : Entries) {
+    if (B)
       Target->writeGotPltEntry(Buf, Out<ELFT>::Plt->getEntryAddr(*B));
     Buf += sizeof(uintX_t);
   }
 }
 
 template <class ELFT>
 GotSection<ELFT>::GotSection()
     : OutputSectionBase<ELFT>(".got", SHT_PROGBITS, SHF_ALLOC | SHF_WRITE) {
   if (Config->EMachine == EM_MIPS)
template <class ELFT> bool GotSection<ELFT>::addCurrentModuleTlsIndex() {
   if (LocalTlsIndexOff != uint32_t(-1))
     return false;
   Entries.push_back(nullptr);
   Entries.push_back(nullptr);
   LocalTlsIndexOff = (Entries.size() - 2) * sizeof(uintX_t);
   return true;
 }
 
+template <class ELFT> void GotSection<ELFT>::addTlsDescEntry() {
+  if (TlsDescEntryOff != uintX_t(-1))
+    return;
+  Entries.push_back(nullptr);
+  TlsDescEntryOff = (Entries.size() - 1) * sizeof(uintX_t);
+}
+
 template <class ELFT>
 typename GotSection<ELFT>::uintX_t
 GotSection<ELFT>::getEntryAddr(const SymbolBody &B) const {
   return this->getVA() + B.GotIndex * sizeof(uintX_t);
 }
 
 template <class ELFT>
 typename GotSection<ELFT>::uintX_t
for (auto &I : Entries) {
     uint64_t GotVA =
         LazyReloc ? Out<ELFT>::GotPlt->getVA() : Out<ELFT>::Got->getVA();
     uint64_t GotE = LazyReloc ? Out<ELFT>::GotPlt->getEntryAddr(*E)
                               : Out<ELFT>::Got->getEntryAddr(*E);
     uint64_t Plt = this->getVA() + Off;
     Target->writePltEntry(Buf + Off, GotVA, GotE, Plt, E->PltIndex, RelOff);
     Off += Target->getPltEntrySize();
   }
+  // If module uses TLS descriptor relocations, it requires .got entry
+  // and a special .plt entry as well. This plt entry is used for TLS descriptor
+  // resolver calls, and also dynamic entry DT_TLSDESC_PLT should be created to
+  // hold it's address.
+  if (Out<ELFT>::Got->hasTlsDescEntry())
+    Target->writePltTlsDescEntry(Buf + Off, this->getVA() + Off,
+                                 Out<ELFT>::Got->getTlsDescEntryVA(),
+                                 Out<ELF64LE>::GotPlt->getVA());
 }
 
 template <class ELFT> void PltSection<ELFT>::addEntry(SymbolBody *Sym) {
   Sym->PltIndex = Entries.size();
   unsigned RelOff = Target->supportsLazyRelocations()
                         ? Out<ELFT>::RelaPlt->getRelocOffset()
                         : Out<ELFT>::RelaDyn->getRelocOffset();
   Entries.push_back(std::make_pair(Sym, RelOff));
 }
 
+template <class ELFT> bool PltSection<ELFT>::empty() const {
+  return Entries.empty() && !Out<ELFT>::Got->hasTlsDescEntry();
+}
+
 template <class ELFT>
 typename PltSection<ELFT>::uintX_t
 PltSection<ELFT>::getEntryAddr(const SymbolBody &B) const {
   return this->getVA() + Target->getPltZeroEntrySize() +
          B.PltIndex * Target->getPltEntrySize();
 }
 
+template <class ELFT>
+typename PltSection<ELFT>::uintX_t PltSection<ELFT>::getTlsDescEntryVA() const {
+  return this->getVA() + Target->getPltZeroEntrySize() +
+         Entries.size() * Target->getPltEntrySize();
+}
+
 template <class ELFT> void PltSection<ELFT>::finalize() {
   this->Header.sh_size = Target->getPltZeroEntrySize() +
                          Entries.size() * Target->getPltEntrySize();
+  if (Out<ELFT>::Got->hasTlsDescEntry())
+    this->Header.sh_size += Target->getPltTlsDescEntrySize();
 }
 
 template <class ELFT>
 RelocationSection<ELFT>::RelocationSection(StringRef Name, bool IsRela)
     : OutputSectionBase<ELFT>(Name, IsRela ? SHT_RELA : SHT_REL, SHF_ALLOC),
       IsRela(IsRela) {
   this->Header.sh_entsize = IsRela ? sizeof(Elf_Rela) : sizeof(Elf_Rel);
   this->Header.sh_addralign = ELFT::Is64Bits ? 8 : 4;
 }
 
 // Applies corresponding symbol and type for dynamic tls relocation.
 // Returns true if relocation was handled.
 template <class ELFT>
 bool RelocationSection<ELFT>::applyTlsDynamicReloc(SymbolBody *Body,
                                                    uint32_t Type, Elf_Rel *P,
                                                    Elf_Rel *N) {
   if (Target->isTlsLocalDynamicReloc(Type)) {
     P->setSymbolAndType(0, Target->getTlsModuleIndexReloc(), Config->Mips64EL);
     P->r_offset = Out<ELFT>::Got->getLocalTlsIndexVA();
     return true;
   }
 
+  // All TLS descriptor relocations produces single dynamic one which is
+  // R_*_TLSDESC usually. Because of dynamic nature, .got.plt is used
+  // to store double word for them.
+  if (Body && Target->isTlsDescReloc(Type, *Body)) {
+    P->setSymbolAndType(Body->DynamicSymbolTableIndex,
+                        Target->getTlsDescDynReloc(), Config->Mips64EL);
+    P->r_offset = Out<ELFT>::GotPlt->getEntryAddr(*Body);
+    return true;
+  }
+
   if (!Body || !Target->isTlsGlobalDynamicReloc(Type))
     return false;
 
   if (Target->isTlsOptimized(Type, Body)) {
     P->setSymbolAndType(Body->DynamicSymbolTableIndex,
                         Target->getTlsGotReloc(), Config->Mips64EL);
     P->r_offset = Out<ELFT>::Got->getEntryAddr(*Body);
     return true;
if (Out<ELFT>::RelaPlt && Out<ELFT>::RelaPlt->hasRelocs()) {
     // the DT_MIPS_PLTGOT entry because the DT_PLTGOT entry points to
     // the .got section. See "Dynamic Section" in the following document:
     // https://sourceware.org/ml/binutils/2008-07/txt00000.txt
     WritePtr((Config->EMachine == EM_MIPS) ? DT_MIPS_PLTGOT : DT_PLTGOT,
              Out<ELFT>::GotPlt->getVA());
     WriteVal(DT_PLTREL, Out<ELFT>::RelaPlt->isRela() ? DT_RELA : DT_REL);
   }
 
+  if (Out<ELFT>::Got->hasTlsDescEntry()) {
+    WritePtr(DT_TLSDESC_GOT, Out<ELFT>::Got->getTlsDescEntryVA());
+    WritePtr(DT_TLSDESC_PLT, Out<ELFT>::Plt->getTlsDescEntryVA());
+  }
+
   WritePtr(DT_SYMTAB, Out<ELFT>::DynSymTab->getVA());
   WritePtr(DT_SYMENT, sizeof(Elf_Sym));
   WritePtr(DT_STRTAB, Out<ELFT>::DynStrTab->getVA());
   WriteVal(DT_STRSZ, Out<ELFT>::DynStrTab->getSize());
   if (Out<ELFT>::GnuHashTab)
     WritePtr(DT_GNU_HASH, Out<ELFT>::GnuHashTab->getVA());
   if (Out<ELFT>::HashTab)
     WritePtr(DT_HASH, Out<ELFT>::HashTab->getVA());

ELF/Target.h

public:
   bool isTlsLocalDynamicReloc(unsigned Type) const {
     return Type == TlsLocalDynamicReloc;
   }
   bool isTlsGlobalDynamicReloc(unsigned Type) const {
     return Type == TlsGlobalDynamicReloc;
   }
   unsigned getTlsModuleIndexReloc() const { return TlsModuleIndexReloc; }
   unsigned getTlsOffsetReloc() const { return TlsOffsetReloc; }
+  unsigned getTlsDescDynReloc() const { return TlsDescDynReloc; }
   unsigned getPltZeroEntrySize() const { return PltZeroEntrySize; }
   unsigned getPltEntrySize() const { return PltEntrySize; }
+  unsigned getPltTlsDescEntrySize() const { return PltTlsDescEntrySize; }
   bool supportsLazyRelocations() const { return LazyRelocations; }
   unsigned getGotHeaderEntriesNum() const { return GotHeaderEntriesNum; }
   unsigned getGotPltHeaderEntriesNum() const { return GotPltHeaderEntriesNum; }
   virtual unsigned getDynReloc(unsigned Type) const { return Type; }
+  virtual bool isTlsDescReloc(unsigned Type, const SymbolBody &S) const {
+    return false;
+  }
   virtual bool isTlsDynReloc(unsigned Type, const SymbolBody &S) const {
     return false;
   }
   virtual unsigned getTlsGotReloc(unsigned Type = -1) const {
     return TlsGotReloc;
   }
   virtual void writeGotHeaderEntries(uint8_t *Buf) const;
   virtual void writeGotPltHeaderEntries(uint8_t *Buf) const;
   virtual void writeGotPltEntry(uint8_t *Buf, uint64_t Plt) const = 0;
   virtual void writePltZeroEntry(uint8_t *Buf, uint64_t GotEntryAddr,
                                  uint64_t PltEntryAddr) const = 0;
   virtual void writePltEntry(uint8_t *Buf, uint64_t GotAddr,
                              uint64_t GotEntryAddr, uint64_t PltEntryAddr,
                              int32_t Index, unsigned RelOff) const = 0;
+  virtual void writePltTlsDescEntry(uint8_t *Buf, uint64_t PltEntryAddr,
+                                    uint64_t GotTlsDescEntryAddr,
+                                    uint64_t GotPltVA) const;
 
   // Returns true if a relocation is just a hint for linker to make for example
   // some code optimization. Such relocations should not be handled as a regular
   // ones and lead to dynamic relocation creation etc.
   virtual bool isHintReloc(uint32_t Type) const;
 
   // Returns true if a relocation is relative to the place being relocated,
   // such as relocations used for PC-relative instructions. Such relocations
protected:
   unsigned PltReloc;
   unsigned RelativeReloc;
   unsigned IRelativeReloc;
   unsigned TlsGotReloc = 0;
   unsigned TlsLocalDynamicReloc = 0;
   unsigned TlsGlobalDynamicReloc = 0;
   unsigned TlsModuleIndexReloc;
   unsigned TlsOffsetReloc;
+  unsigned TlsDescDynReloc;
   unsigned PltEntrySize = 8;
   unsigned PltZeroEntrySize = 0;
+  unsigned PltTlsDescEntrySize = 0;
   unsigned GotHeaderEntriesNum = 0;
   unsigned GotPltHeaderEntriesNum = 3;
   bool LazyRelocations = false;
 };
 
 uint64_t getPPC64TocBase();
 
 template <class ELFT>

ELF/Target.cpp

public:
   AArch64TargetInfo();
   unsigned getDynReloc(unsigned Type) const override;
   void writeGotPltEntry(uint8_t *Buf, uint64_t Plt) const override;
   void writePltZeroEntry(uint8_t *Buf, uint64_t GotEntryAddr,
                          uint64_t PltEntryAddr) const override;
   void writePltEntry(uint8_t *Buf, uint64_t GotAddr, uint64_t GotEntryAddr,
                      uint64_t PltEntryAddr, int32_t Index,
                      unsigned RelOff) const override;
+  void writePltTlsDescEntry(uint8_t *Buf, uint64_t PltEntryAddr,
+                            uint64_t GotTlsDescEntryAddr,
+                            uint64_t GotPltVA) const override;
   unsigned getTlsGotReloc(unsigned Type = -1) const override;
+  bool isTlsDescReloc(unsigned Type, const SymbolBody &S) const override;
   bool isTlsDynReloc(unsigned Type, const SymbolBody &S) const override;
   bool needsCopyRel(uint32_t Type, const SymbolBody &S) const override;
   bool relocNeedsGot(uint32_t Type, const SymbolBody &S) const override;
   bool relocNeedsPlt(uint32_t Type, const SymbolBody &S) const override;
   void relocateOne(uint8_t *Loc, uint8_t *BufEnd, uint32_t Type, uint64_t P,
                    uint64_t SA, uint64_t ZA = 0,
                    uint8_t *PairedLoc = nullptr) const override;
 };
unsigned TargetInfo::relocateTlsOptimize(uint8_t *Loc, uint8_t *BufEnd,
                                          const SymbolBody &S) const {
   return 0;
 }
 
 void TargetInfo::writeGotHeaderEntries(uint8_t *Buf) const {}
 
 void TargetInfo::writeGotPltHeaderEntries(uint8_t *Buf) const {}
 
+void TargetInfo::writePltTlsDescEntry(uint8_t *Buf, uint64_t PltEntryAddr,
+                                      uint64_t GotTlsDescEntryAddr,
+                                      uint64_t GotPltVA) const {}
+
 X86TargetInfo::X86TargetInfo() {
   CopyReloc = R_386_COPY;
   PCRelReloc = R_386_PC32;
   GotReloc = R_386_GLOB_DAT;
   PltReloc = R_386_JUMP_SLOT;
   IRelativeReloc = R_386_IRELATIVE;
   RelativeReloc = R_386_RELATIVE;
   TlsGotReloc = R_386_TLS_TPOFF;
void PPC64TargetInfo::relocateOne(uint8_t *Loc, uint8_t *BufEnd, uint32_t Type,
   }
 }
 
 AArch64TargetInfo::AArch64TargetInfo() {
   CopyReloc = R_AARCH64_COPY;
   IRelativeReloc = R_AARCH64_IRELATIVE;
   GotReloc = R_AARCH64_GLOB_DAT;
   PltReloc = R_AARCH64_JUMP_SLOT;
+  TlsDescDynReloc = R_AARCH64_TLSDESC;
   TlsGotReloc = R_AARCH64_TLS_TPREL64;
   LazyRelocations = true;
   PltEntrySize = 16;
   PltZeroEntrySize = 32;
+  PltTlsDescEntrySize = 32;
 }
 
 unsigned AArch64TargetInfo::getDynReloc(unsigned Type) const {
   if (Type == R_AARCH64_ABS32 || Type == R_AARCH64_ABS64)
     return Type;
   StringRef S = getELFRelocationTypeName(EM_AARCH64, Type);
   error("Relocation " + S + " cannot be used when making a shared object; "
                             "recompile with -fPIC.");
void AArch64TargetInfo::writePltEntry(uint8_t *Buf, uint64_t GotAddr,
   relocateOne(Buf, Buf + 4, R_AARCH64_ADR_PREL_PG_HI21, PltEntryAddr,
               GotEntryAddr);
   relocateOne(Buf + 4, Buf + 8, R_AARCH64_LDST64_ABS_LO12_NC, PltEntryAddr + 4,
               GotEntryAddr);
   relocateOne(Buf + 8, Buf + 12, R_AARCH64_ADD_ABS_LO12_NC, PltEntryAddr + 8,
               GotEntryAddr);
 }
 
+void AArch64TargetInfo::writePltTlsDescEntry(uint8_t *Buf,
+                                             uint64_t PltEntryAddr,
+                                             uint64_t GotTlsDescEntryAddr,
+                                             uint64_t GotPltVA) const {
+  const uint8_t Inst[] = {
+      0xe2, 0x0f, 0xbf, 0xa9, // stp x2, x3, [sp, #-16]!
+      0x02, 0x00, 0x00, 0x90, // adrp x2, Page(DT_TLSDESC_GOT)
+      0x03, 0x00, 0x00, 0x90, // adrp x3, Page(&.got.plt[0])
+      0x42, 0x00, 0x40, 0xf9, // ldr x2, [x2, #0]
+      0x63, 0x00, 0x00, 0x91, // add x3, x3, 0
+      0x40, 0x00, 0x1f, 0xd6, // br x2
+      0x1f, 0x20, 0x03, 0xd5, // nop
+      0x1f, 0x20, 0x03, 0xd5  // nop
+  };
+  memcpy(Buf, Inst, sizeof(Inst));
+  relocateOne(Buf + 4, Buf + 8, R_AARCH64_ADR_PREL_PG_HI21, PltEntryAddr + 4,
+              GotTlsDescEntryAddr);
+  relocateOne(Buf + 8, Buf + 12, R_AARCH64_ADR_PREL_PG_HI21, PltEntryAddr + 8,
+              GotPltVA);
+  relocateOne(Buf + 12, Buf + 16, R_AARCH64_LDST64_ABS_LO12_NC,
+              PltEntryAddr + 12, GotTlsDescEntryAddr);
+  relocateOne(Buf + 16, Buf + 20, R_AARCH64_ADD_ABS_LO12_NC, PltEntryAddr + 16,
+              GotPltVA);
+}
+
 unsigned AArch64TargetInfo::getTlsGotReloc(unsigned Type) const {
   if (Type == R_AARCH64_TLSIE_ADR_GOTTPREL_PAGE21 ||
       Type == R_AARCH64_TLSIE_LD64_GOTTPREL_LO12_NC)
     return Type;
   return TlsGotReloc;
 }
 
+bool AArch64TargetInfo::isTlsDescReloc(unsigned Type,
+                                       const SymbolBody &S) const {
+  switch (Type) {
+  case R_AARCH64_TLSDESC_ADR_PAGE21:
+  case R_AARCH64_TLSDESC_LD64_LO12_NC:
+  case R_AARCH64_TLSDESC_ADD_LO12_NC:
+  case R_AARCH64_TLSDESC_CALL:
+    return true;
+  default:
+    return false;
+  }
+}
+
 bool AArch64TargetInfo::isTlsDynReloc(unsigned Type,
                                       const SymbolBody &S) const {
   return Type == R_AARCH64_TLSIE_ADR_GOTTPREL_PAGE21 ||
          Type == R_AARCH64_TLSIE_LD64_GOTTPREL_LO12_NC;
 }
 
 bool AArch64TargetInfo::needsCopyRel(uint32_t Type, const SymbolBody &S) const {
   if (Config->Shared)
void AArch64TargetInfo::relocateOne(uint8_t *Loc, uint8_t *BufEnd,
   }
   case R_AARCH64_ADR_PREL_LO21: {
     uint64_t X = SA - P;
     checkInt<21>(X, Type);
     updateAArch64Adr(Loc, X & 0x1FFFFF);
     break;
   }
   case R_AARCH64_ADR_PREL_PG_HI21:
-  case R_AARCH64_TLSIE_ADR_GOTTPREL_PAGE21: {
+  case R_AARCH64_TLSIE_ADR_GOTTPREL_PAGE21:
+  case R_AARCH64_TLSDESC_ADR_PAGE21: {
     uint64_t X = getAArch64Page(SA) - getAArch64Page(P);
     checkInt<33>(X, Type);
     updateAArch64Adr(Loc, (X >> 12) & 0x1FFFFF); // X[32:12]
     break;
   }
   case R_AARCH64_CALL26:
   case R_AARCH64_JUMP26: {
     uint64_t X = SA - P;
     checkInt<28>(X, Type);
     or32le(Loc, (X & 0x0FFFFFFC) >> 2);
     break;
   }
   case R_AARCH64_CONDBR19: {
     uint64_t X = SA - P;
     checkInt<21>(X, Type);
     or32le(Loc, (X & 0x1FFFFC) << 3);
     break;
   }
   case R_AARCH64_LD64_GOT_LO12_NC:
   case R_AARCH64_TLSIE_LD64_GOTTPREL_LO12_NC:
+  case R_AARCH64_TLSDESC_LD64_LO12_NC:
     checkAlignment<8>(SA, Type);
     or32le(Loc, (SA & 0xFF8) << 7);
     break;
   case R_AARCH64_LDST128_ABS_LO12_NC:
     or32le(Loc, (SA & 0x0FF8) << 6);
     break;
   case R_AARCH64_LDST16_ABS_LO12_NC:
     or32le(Loc, (SA & 0x0FFC) << 9);
     break;
   case R_AARCH64_LDST8_ABS_LO12_NC:
+  case R_AARCH64_TLSDESC_ADD_LO12_NC:
     or32le(Loc, (SA & 0xFFF) << 10);
     break;
   case R_AARCH64_LDST32_ABS_LO12_NC:
     or32le(Loc, (SA & 0xFFC) << 8);
     break;
   case R_AARCH64_LDST64_ABS_LO12_NC:
     or32le(Loc, (SA & 0xFF8) << 7);
     break;
case R_AARCH64_PREL64:
     write64le(Loc, SA - P);
     break;
   case R_AARCH64_TSTBR14: {
     uint64_t X = SA - P;
     checkInt<16>(X, Type);
     or32le(Loc, (X & 0xFFFC) << 3);
     break;
   }
+  case R_AARCH64_TLSDESC_CALL:
+    // For relaxation only. Must be used to identify a
+    // BLR instruction which performs an indirect call
+    // to the TLS descriptor function for S + A.
+    break;
   default:
     error("unrecognized reloc " + Twine(Type));
   }
 }
 
 AMDGPUTargetInfo::AMDGPUTargetInfo() {}
 
 void AMDGPUTargetInfo::writeGotPltEntry(uint8_t *Buf, uint64_t Plt) const {

ELF/Writer.cpp

 
 #include "llvm/ADT/StringMap.h"
 #include "llvm/ADT/StringSwitch.h"
 #include "llvm/Support/FileOutputBuffer.h"
 #include "llvm/Support/raw_ostream.h"
 #include "llvm/Support/StringSaver.h"
 
 using namespace llvm;
 using namespace llvm::ELF;
 using namespace llvm::object;
 
 using namespace lld;
 using namespace lld::elf2;
 
 namespace {
 // The writer writes a SymbolTable result to a file.
 template <class ELFT> class Writer {
if (Body && Body->isTls() && Target->isTlsGlobalDynamicReloc(Type)) {
         Out<ELFT>::RelaDyn->addReloc({nullptr, nullptr});
         Body->setUsedInDynamicReloc();
         continue;
       }
       if (!canBePreempted(Body, true))
         continue;
     }
 
-    if (Body && Body->isTls() && !Target->isTlsDynReloc(Type, *Body))
+    if (Body && Body->isTls()) {
+      // If module uses TLS descriptor relocations,
+      // then special entries are created for module:
+      // 1) .got entry to be filled in by the dynamic loader with the
+      //  address of the internal function to be used for lazy relocation of TLS
+      //  descriptors.
+      // 2) Special .plt entry that
+      //  pushes onto the stack the module's link map address, located in the
+      //  GOT portion reserved for the dynamic loader to use, and then jumps to
+      //  the lazy relocation function, using the address stored in the
+      //  TLSDESC_GOT entry.
+      // For each Body itself two words are allocated in .got.plt instead of
+      // usual .got, because these relocations are lazy ones.
+      if (Target->isTlsDescReloc(Type, *Body)) {
+        Out<ELFT>::Got->addTlsDescEntry();
+        if (Body->isInGotPlt())
           continue;
+        Out<ELFT>::GotPlt->addTlsDescEntry(Body);
+        Out<ELFT>::RelaPlt->addReloc({&C, &RI});
+        continue;
+      }
+      if (!Target->isTlsDynReloc(Type, *Body))
+        continue;
+    }
 
     if (Target->relocNeedsDynRelative(Type)) {
       RelType *Rel = new (Alloc) RelType;
       Rel->setSymbolAndType(0, Target->getRelativeReloc(), Config->Mips64EL);
       Rel->r_offset = RI.r_offset;
       Out<ELFT>::RelaDyn->addReloc({&C, Rel});
     }
 

test/ELF/aarch64-tls-desc.s

+# RUN: llvm-mc -filetype=obj -triple=aarch64-unknown-freebsd %s -o %tmain.o
+# RUN: ld.lld %tmain.o -shared -o %tout
+# RUN: llvm-objdump -d %tout | FileCheck %s
+# RUN: llvm-readobj -s -r -dynamic-table %tout | FileCheck -check-prefix=READOBJ %s
+# REQUIRES: aarch64
+
+#READOBJ:      Section {
+#READOBJ:       Index:
+#READOBJ:       Name: .plt
+#READOBJ-NEXT:   Type: SHT_PROGBITS
+#READOBJ-NEXT:   Flags [
+#READOBJ-NEXT:     SHF_ALLOC
+#READOBJ-NEXT:     SHF_EXECINSTR
+#READOBJ-NEXT:   ]
+#READOBJ-NEXT:   Address: 0x1020
+#READOBJ-NEXT:   Offset:
+#READOBJ-NEXT:   Size: 64
+#READOBJ-NEXT:   Link:
+#READOBJ-NEXT:   Info:
+#READOBJ-NEXT:   AddressAlignment:
+#READOBJ-NEXT:   EntrySize:
+#READOBJ-NEXT: }
+#READOBJ:      Section {
+#READOBJ:       Index:
+#READOBJ:       Name: .got
+#READOBJ-NEXT:     Type: SHT_PROGBITS
+#READOBJ-NEXT:     Flags [
+#READOBJ-NEXT:       SHF_ALLOC
+#READOBJ-NEXT:       SHF_WRITE
+#READOBJ-NEXT:     ]
+#READOBJ-NEXT:   Address: 0x20A8
+#READOBJ-NEXT:   Offset:
+#READOBJ-NEXT:   Size: 8
+#READOBJ-NEXT:   Link:
+#READOBJ-NEXT:   Info:
+#READOBJ-NEXT:   AddressAlignment:
+#READOBJ-NEXT:   EntrySize:
+#READOBJ-NEXT: }
+#READOBJ:      Section {
+#READOBJ:       Index:
+#READOBJ:       Name: .got.plt
+#READOBJ-NEXT:   Type: SHT_PROGBITS
+#READOBJ-NEXT:   Flags [
+#READOBJ-NEXT:     SHF_ALLOC
+#READOBJ-NEXT:     SHF_WRITE
+#READOBJ-NEXT:   ]
+#READOBJ-NEXT:   Address: 0x3000
+#READOBJ-NEXT:   Offset: 0x3000
+#READOBJ-NEXT:   Size: 56
+#READOBJ-NEXT:   Link: 0
+#READOBJ-NEXT:   Info: 0
+#READOBJ-NEXT:   AddressAlignment: 8
+#READOBJ-NEXT:   EntrySize: 0
+#READOBJ-NEXT: }
+#READOBJ:      Relocations [
+#READOBJ-NEXT:  Section ({{.*}}) .rela.plt {
+# 0x3018 = .got.plt + 0x18 (reserved 3 entries)
+# 0x3028 = 0x3018 + double entry size (16)
+#READOBJ-NEXT:    0x3018 R_AARCH64_TLSDESC foo 0x0
+#READOBJ-NEXT:    0x3028 R_AARCH64_TLSDESC bar 0x0
+#READOBJ-NEXT:  }
+#READOBJ-NEXT:]
+# Todo: unknowns should be replaced with TLSDESC_PLT and TLSDESC_GOT
+# once llvm-readobj is updated to support it.
+#READOBJ:       DynamicSection [
+#READOBJ-NEXT:    Tag                Type                 Name/Value
+#READOBJ-NEXT:    0x0000000000000017 JMPREL               0x298
+#READOBJ-NEXT:    0x0000000000000002 PLTRELSZ             48 (bytes)
+#READOBJ-NEXT:    0x0000000000000003 PLTGOT               0x3000
+#READOBJ-NEXT:    0x0000000000000014 PLTREL               RELA
+# 0x20A8 = Location of GOT entry used by TLS descriptor resolver PLT entry
+#READOBJ-NEXT:    0x000000006FFFFEF7 unknown              0x20A8
+# 0x1040 = Location of PLT entry for TLS descriptor resolver calls.
+#READOBJ-NEXT:    0x000000006FFFFEF6 unknown              0x1040
+#READOBJ-NEXT:    0x0000000000000006 SYMTAB               0x200
+#READOBJ-NEXT:    0x000000000000000B SYMENT               24 (bytes)
+#READOBJ-NEXT:    0x0000000000000005 STRTAB               0x288
+#READOBJ-NEXT:    0x000000000000000A STRSZ                16 (bytes)
+#READOBJ-NEXT:    0x0000000000000004 HASH                 0x260
+#READOBJ-NEXT:  ]
+
+#CHECK:      Disassembly of section .text:
+#CHECK-NEXT: _start:
+# Page(.got.plt[N]) - Page(0x1000) = Page(0x3018) - 0x1000 =
+#   0x3000 - 0x1000 = 0x2000 = 8192
+# 0x18 = 24
+#CHECK-NEXT:  1000: 00 00 00 d0 adrp x0, #8192
+#CHECK-NEXT:  1004: 02 0c 40 f9 ldr  x2, [x0, #24]
+#CHECK-NEXT:  1008: 00 60 00 91 add  x0, x0, #24
+#CHECK-NEXT:  100c: 40 00 3f d6 blr  x2
+# Page(.got.plt[N]) - Page(0x1000) = Page(0x3028) - 0x1000 =
+#   0x3000 - 0x1000 = 0x2000 = 8192
+# 0x28 = 40
+#CHECK-NEXT:  1010: 00 00 00 d0 adrp x0, #8192
+#CHECK-NEXT:  1014: 02 14 40 f9 ldr  x2, [x0, #40]
+#CHECK-NEXT:  1018: 00 a0 00 91 add  x0, x0, #40
+#CHECK-NEXT:  101c: 40 00 3f d6 blr x2
+#CHECK-NEXT: Disassembly of section .plt:
+#CHECK-NEXT: .plt:
+#CHECK-NEXT:  1020: f0 7b bf a9 stp  x16, x30, [sp, #-16]!
+#CHECK-NEXT:  1024: 10 00 00 d0 adrp x16, #8192
+#CHECK-NEXT:  1028: 11 0a 40 f9 ldr  x17, [x16, #16]
+#CHECK-NEXT:  102c: 10 42 00 91 add  x16, x16, #16
+#CHECK-NEXT:  1030: 20 02 1f d6 br   x17
+#CHECK-NEXT:  1034: 1f 20 03 d5 nop
+#CHECK-NEXT:  1038: 1f 20 03 d5 nop
+#CHECK-NEXT:  103c: 1f 20 03 d5 nop
+# Page(.got[N]) - Page(P) = Page(0x20A8) - Page(0x1044) =
+#  0x2000 - 0x1000 = 4096
+# Page(.got.plt) - Page(P) = Page(0x3000) - Page(0x1048) =
+#  0x3000 - 0x1000 = 8192
+# 0xA8 = 168
+# 0x0 = 0
+#CHECK-NEXT:  1040: e2 0f bf a9 stp  x2, x3, [sp, #-16]!
+#CHECK-NEXT:  1044: 02 00 00 b0 adrp x2, #4096
+#CHECK-NEXT:  1048: 03 00 00 d0 adrp x3, #8192
+#CHECK-NEXT:  104c: 42 54 40 f9 ldr  x2, [x2, #168]
+#CHECK-NEXT:  1050: 63 00 00 91 add  x3, x3, #0
+#CHECK-NEXT:  1054: 40 00 1f d6 br   x2
+#CHECK-NEXT:  1058: 1f 20 03 d5 nop
+#CHECK-NEXT:  105c: 1f 20 03 d5 nop
+
+.text
+ .global foo
+ .section .tdata,"awT",%progbits
+ .align 2
+ .type foo, %object
+ .size foo, 4
+foo:
+ .word 5
+ .text
+
+.text
+ .global bar
+ .section .tdata,"awT",%progbits
+ .align 2
+ .type bar, %object
+ .size bar, 4
+bar:
+ .word 5
+ .text
+
+.globl _start
+_start:
+ adrp x0, :tlsdesc:foo
+ ldr x2, [x0, #:tlsdesc_lo12:foo]
+ add x0, x0, :tlsdesc_lo12:foo
+ .tlsdesccall foo
+ blr x2
+
+ adrp x0, :tlsdesc:bar
+ ldr x2, [x0, #:tlsdesc_lo12:bar]
+ add x0, x0, :tlsdesc_lo12:bar
+ .tlsdesccall bar
+ blr x2