[LLD] Remove global state in lldCommon

[aganea]

Summary

As discussed in our last Windows/COFF call, and as proposed by Reid in [1], this patch implements a first iteration to make LLD free of global initializers & global state. Here we only remove global state in lldCommon, however the same concept can be applied to other LLD libs.

The idea is to move all variables at file-scope or function-static-scope into a hosting structure (lld::CommonLinkingContext) that lives at lldMain()-scope. Drivers will inherit from this structure and add their own global state, in the same way as for the existing LinkingContext and MachOLinkingContext.

Before, code was directly referencing state in the .data section. It now goes through an indirection to a global **LinkingContext returned by lld::context(). In the future, we plan on supporting several instances of lld::**LinkingContext that would coexist in memory (when the hosting application is similar to D86351).

[1] https://lists.llvm.org/pipermail/llvm-dev/2021-June/151184.html

[int3]

lld/MachO/Driver.cpp

148

why not just saver()?

[amccarth]

This is a larger change than I anticipated. I was surprised by the need to have the pointer be thread local (and then ensure that the thread local pointers for new threads are set up, too). If I'm understanding correctly, the idea is that a single program could have n threads doing n links at the same time. Is that right? I'm not arguing against that. I'm just trying to make sure I understand why I was surprised.

It also introduces more indirection than I expected. I believe Rui (and others) favored globals in order to go as fast as possible. Are there lld benchmarks to give us an idea of the cost of this? If there's pushback from the performance die-hards, it would be nice to have an actual measurement or two to show (I hope) that the cost for this flexibility is low.

Anyway, this looks pretty good to me. I'm not well-versed in lld, and--honestly--I didn't scrutinize all the changes that looked mostly mechanical.

lld/COFF/Chunks.cpp

11

I guess we're using ASCIIbetical rather than alphabetical ordering. :-)

lld/MachO/DriverUtils.cpp

262

In most cases, it's just saver(). Is there something special about these instances where you explicitly pass a pointer to the common globals?

lld/include/lld/Common/Globals.h

68

s/beween/between/

lld/tools/lld/lld.cpp

165

Given that this is the not-exiting-early path, should this thread detach from the global state before deleting it? That might help catch instances where something in the chain of destructors inadvertently tries to reference a global.

[MaskRay]

lld/Common/ErrorHandler.cpp

63

add empty line

66

add empty line

lld/Common/Globals.cpp

75

81

lld/include/lld/Common/Globals.h

52

[aganea]

Rebase + changes as suggested.

[aganea]

Thanks for taking a look!

In D108850#2970244, @amccarth wrote:

This is a larger change than I anticipated.

It's a first basis for discussions. Please let me know if I should take this in a different direction.

I was surprised by the need to have the pointer be thread local (and then ensure that the thread local pointers for new threads are set up, too). If I'm understanding correctly, the idea is that a single program could have n threads doing n links at the same time. Is that right? I'm not arguing against that. I'm just trying to make sure I understand why I was surprised.

Yes that is correct.

The dream is, getting to a point where LLVM_PARALLEL_LINK_JOBS wouldn't be needed (if linking in-process). Currently each LLD process spawns its own thread pool, and the applications don't talk to each other. That increases the memory usage and (surprisingly) sometimes can starve the CPU, for example when doing ThinLTO. I feel that we could have better control over resources if we had a single "build/compile/link" application.

It also introduces more indirection than I expected.

Indeed, more memory loads are now required to get to the global. In the vast majority of cases it shouldn't matter, but in tight loops we will need to pass down the globals pointer.

I believe Rui (and others) favored globals in order to go as fast as possible. Are there lld benchmarks to give us an idea of the cost of this? If there's pushback from the performance die-hards, it would be nice to have an actual measurement or two to show (I hope) that the cost for this flexibility is low.

There seems to be a really tiny departure from baseline after this change, but it's hard to tell since LLD is mostly I/O-bound.
The data set is ~5 GB input .OBJs, approx. 2.5 GB .PDB, 475 MB executable.
Sometimes just running the same command line again exchanges the outcome, "before" comes before "after" or vice versa:

Intel(R) Xeon(R) W-2135 CPU @ 3.70GHz (6 cores)
>hyperfine "before\lld-link.exe @link.rsp" "after\lld-link.exe @link.rsp"
Benchmark #1: before\lld-link.exe @link.rsp                                                                                     
  Time (mean ± σ):     13.740 s ±  0.578 s    [User: 0.0 ms, System: 53.5 ms]                                                         
  Range (min … max):   13.391 s … 15.321 s    10 runs

Benchmark #2: after\lld-link.exe @link.rsp
  Time (mean ± σ):     13.737 s ±  0.198 s    [User: 4.4 ms, System: 55.8 ms]                                                         
  Range (min … max):   13.566 s … 14.262 s    10 runs

Summary
  'after\lld-link.exe @link.rsp' ran
    1.00 ± 0.04 times faster than 'before\lld-link.exe @link.rsp'


AMD Ryzen 9 5950X 16-Core Processor @ 3.40 GHz
>hyperfine "before\lld-link.exe @link.rsp" "after\lld-link.exe @link.rsp"
Benchmark #1: before\lld-link.exe @link.rsp                                                                      
  Time (mean ± σ):     10.178 s ±  0.091 s    [User: 3.0 ms, System: 5.2 ms]                                           
  Range (min … max):    9.978 s … 10.303 s    10 runs

Benchmark #2: after\lld-link.exe @link.rsp
  Time (mean ± σ):     10.243 s ±  0.088 s    [User: 4.5 ms, System: 5.2 ms]                                           
  Range (min … max):   10.115 s … 10.357 s    10 runs

Summary
  'before\lld-link.exe @link.rsp' ran
    1.01 ± 0.01 times faster than 'after\lld-link.exe @link.rsp'


AMD Ryzen Threadripper PRO 3975WX 32-Cores @ 3.50 GHz
>hyperfine "before\lld-link.exe @link.rsp" "after\lld-link.exe @link.rsp"
Benchmark #1: before\lld-link.exe @link.rsp                                                                          
  Time (mean ± σ):     15.451 s ±  0.545 s    [User: 1.4 ms, System: 8.6 ms]                                               
  Range (min … max):   15.024 s … 16.849 s    10 runs

Benchmark #2: after\lld-link.exe @link.rsp
  Time (mean ± σ):     15.243 s ±  0.087 s    [User: 0.0 ms, System: 8.6 ms]                                               
  Range (min … max):   15.109 s … 15.392 s    10 runs

Summary
  'after\lld-link.exe @link.rsp' ran
    1.01 ± 0.04 times faster than 'before\lld-link.exe @link.rsp'


2x Intel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz (48 cores, dual CPU)
>hyperfine "before\lld-link.exe @link.rsp" "after\lld-link.exe @link.rsp"
Benchmark #1: before\lld-link.exe @link.rsp                                                                                                   
  Time (mean ± σ):     17.433 s ±  0.852 s    [User: 0.0 ms, System: 6.5 ms]                                                                        
  Range (min … max):   15.202 s … 18.267 s    10 runs

Benchmark #2: after\lld-link.exe @link.rsp
  Time (mean ± σ):     17.661 s ±  0.876 s    [User: 1.4 ms, System: 2.6 ms]                                                                        
  Range (min … max):   15.304 s … 18.442 s    10 runs

Summary
  'before\lld-link.exe @link.rsp' ran
    1.01 ± 0.07 times faster than 'after\lld-link.exe @link.rsp'

Profiling shows a difference in functions that seem to read from memory-mapped files, doing hard page faults (last column is milliseconds):

First line (the lambda) is the first loop in GSIHashStreamBuilder::finalizeBuckets.
The second line is the JamCRC calculation in createSectionContrib in lld/COFF/PDB.cpp.

I thought some cache lines could evicted by the read of CommonGlobls TLS. However two random runs of "before" builds are exhibiting the same difference as abolve. It seems really we're only seeing statistical fluctuations. I'd be happy to run tests on anything that you judge a representative data set.

lld/COFF/Chunks.cpp

11

;-)

lld/MachO/Driver.cpp

148

Changed. Please see my other comment at lld/MachO/DriverUtils.cpp, L263.

lld/MachO/DriverUtils.cpp

262

Only the fact that it's a loop and I wanted to clarify the intent, rather than relying on the optimizer to (maybe) prevent the call to commonGlobals(). But I switched to just saver(), since other operations in this function would dominate.

Ideally we should be passing state down the callstack and not rely on TLS. This could be done in time-critical functions.

lld/tools/lld/lld.cpp

165

It's detaching just at the end of ~CommonGlobals().

[rnk]

Thanks!

Regarding performance, yes, the memory mapped I/O makes measurement very noisy. I'm not worried about this change regressing performance.

lld/include/lld/Common/Globals.h

31

I feel like the LLVM-y naming convention for this would be to use the word Context somehow. Either Link(|ing|er)Context, LldContext, LldContextBase. I could imagine inheriting the COFF context from the linker context, or something like that.

I don't like bikeshedding, but I also feel like the best time to get the name right is at the beginning.

See also the prior art of MachOLinkingContext in the old MachO linker.

[aganea]

Changed CommonGlobals to CommonLinkingContext. Use it along with MachOLinkingContext -- this demonstrates how it will be used for other drivers.

[sbc100]

I have to say I think I will miss the simplicity of the current lld implementation: No relocations, no indirection, just global data. However I do see the advantages you are going for here .. and the discussion about the overall direction has mostly been had on the mailing list at this point.

[akhuang]

This is cool, I was also looking at refactoring out some of the globals in lldCOFF (for now just trying to move some globals into existing classes and passing the classes around). Looks good to me, although I'm not very familiar with structuring code in this way / making things work with threads.

lld/Common/Globals.cpp

46

does static make the anonymous namespace unnecessary?

[aganea]

Remove anon namespace.

[aganea]

In D108850#2978142, @akhuang wrote:

This is cool, I was also looking at refactoring out some of the globals in lldCOFF (for now just trying to move some globals into existing classes and passing the classes around). Looks good to me, although I'm not very familiar with structuring code in this way / making things work with threads.

Both concepts can coexist, passing references to objects across calls, or accesing them through a global (essentially what the mechanism this patch provides). In some cases it just isn't practical to pass references (lld::outs for example, or anything related to the lld::ErrorHandler).

lld/Common/Globals.cpp

46

Yes indeed, fixed!

[akhuang]

seems like everyone's happy with the direction and there are no more comments -- perhaps this is good to be committed now?

[rnk]

I think this should be staged into two patches:

1. LLD-only: includes all of the Context changes, saver(), etc, but the context is still kept in a global pointer.
2. LTO.h and Parallel.h: make the context pointer thread local, allowing the possibility of two linker instances in the same process, and add the task wrapper API

The goal is to make the patch that changes Support as small as possible. MLIR uses the parallel APIs, so we should ask them (@mehdi_amini, @rriddle, maybe others) to review the changes to llvm/Support/Parallel.h.

[rnk]

lld/tools/lld/lld.cpp

182

This probably needs to be initialized or static analysis tools may complain, even though the invalid case is handled above.

[mehdi_amini]

Nice work!

lld/Common/ErrorHandler.cpp

109

I'm curious about this change: you're protecting what seems to be global methods with a non-global mutex. The code comment for the mutex says `

// The functions defined in this file can be called from multiple threads,
// but lld::outs() or lld::errs() are not thread-safe. We protect them using a
// mutex.

This made sense when the mutex was global, less so now. Can you clarify this? (and update the comment)

[mehdi_amini]

The goal is to make the patch that changes Support as small as possible. MLIR uses the parallel APIs, so we should ask them (@mehdi_amini, @rriddle, maybe others) to review the changes to llvm/Support/Parallel.h.

Actually MLIR does not use Parallel.h anymore, especially to avoid relying on any global. We're using instead a ThreadPool owned by the MLIRContext, the client applications can inject a thread pool in the context to manage sharing threads how they see fit.

Our wrappers are in https://github.com/llvm/llvm-project/blob/main/mlir/include/mlir/IR/Threading.h and operate around llvm/Support/ThreadPool.h.

The mechanism introduced in Parallel.h in this revision (wrapThreadState) seems mostly like some sort of "workaround" to be able to continue to rely on some global way to pass state around in lld instead of having something more explicit, which we favored in MLIR (where like in LLVM, the Context is fairly centric and not accessed through a global).

llvm/lib/Support/Parallel.cpp

87

Seems like worth documenting.

[aganea]

Simplify as suggested in https://reviews.llvm.org/D108850#3000527

[aganea]

In D108850#3000699, @mehdi_amini wrote:

The mechanism introduced in Parallel.h in this revision (wrapThreadState) seems mostly like some sort of "workaround" to be able to continue to rely on some global way to pass state around in lld instead of having something more explicit, which we favored in MLIR (where like in LLVM, the Context is fairly centric and not accessed through a global).

It was totally a temporary workaround, until we explicitly pass contexts around. I've tried passing contexts around, but it is a big change and I'm not sure it's worth it. Removed this code for now.

In D108850#3000527, @rnk wrote:

I think this should be staged into two patches:

1. LLD-only: includes all of the Context changes, saver(), etc, but the context is still kept in a global pointer.

Updated the patch to only do 1. like you suggested.

lld/Common/ErrorHandler.cpp

109

Before, the mutex and the functions were global. They both have the same lifetime as the call to lldMain(). I was planning on supporting several concurrent calls to lldMain() in the future. Updated the comment.

llvm/lib/Support/Parallel.cpp

87

Removed.

[rnk]

In D108850#3000699, @mehdi_amini wrote:

Actually MLIR does not use Parallel.h anymore, especially to avoid relying on any global. We're using instead a ThreadPool owned by the MLIRContext, the client applications can inject a thread pool in the context to manage sharing threads how they see fit.

Our wrappers are in https://github.com/llvm/llvm-project/blob/main/mlir/include/mlir/IR/Threading.h and operate around llvm/Support/ThreadPool.h.

The mechanism introduced in Parallel.h in this revision (wrapThreadState) seems mostly like some sort of "workaround" to be able to continue to rely on some global way to pass state around in lld instead of having something more explicit, which we favored in MLIR (where like in LLVM, the Context is fairly centric and not accessed through a global).

I see, I guess that is new as of June, so news to me. :)

I think there are really two key operations done throughout the linker that need the context:

• memory allocation (via BumpPtrAllocator)
• diagnostics (warnings & errors)

Conventional C++ programs use a global heap because passing around a pool or arena is too much work.

I think I would like to see central LLD context classes, but I'm open to using thread locals on the way to that restructuring.

---

As far as this iteration of the patch goes, I think it looks good, but I think other linker port owners should sign off on it too.

[aganea]

Fix linter suggestions.

[aganea]

Ping.

lld/Common/ErrorHandler.cpp

109

I meant now, they both have the same lifetime as the call to lldMain().

[smeenai]

Sorry for the slow review from the Mach-O end.

This seems like a pretty clean solution overall. I was concerned that removing global state would mean we have to thread a ton of context through our entire call stack, but switching to function calls instead is easy enough.

I'll measure the performance of this change this week; we care heavily about linking speed because it dominates incremental build speeds (so fast links can be a huge developer experience boost). I'm not expecting any change from the global variable access changes (especially with LTO), but I'm curious if switching the type-specific bump allocators to using a DenseMap for lookup will have any measurable effect.

I see a bunch of discussions about thread-locals; was that for an earlier version of this patch, or am I just missing something? (I don't see anything thread-local related in the current version of the patch.)

To make sure I'm understanding this correctly, can the calls to freeArena be removed because the destructor for the context will take care of that now?

I'd like @int3 and/or @thakis to take a look on the Mach-O side as well (since they're far more active), and I imagine @MaskRay is the best person for the ELF side.

lld/COFF/InputFiles.cpp

1069

What was the motivation for saving to a local variable here? Is it the use of saver in the loop below? (It makes sense to me; I just wanted to confirm my understanding.)

lld/Common/CommonLinkingContext.cpp

22 ↗

(On Diff #373420)

It'd be nice if there was a way to put this in the header, so that it could be inlined even in non-LTO builds, but that'd prevent lctx from being file-static, which seems undesirable. I imagine LTO or ThinLTO would be happy to inline this (and I'll confirm when I do performance measurements), so it shouldn't matter too much in the end.

23 ↗

(On Diff #373420)

I'm a bit concerned about the potential performance implications of this assert, but I'll measure first.

lld/Common/Memory.cpp

15–46

Can these be put in the header, so that they can be inlined even in non-LTO builds?

lld/include/lld/Common/Memory.h

31–32

This comment should probably be updated? The tracking is happening in the context now.

49

The contents don't matter at all, just the address (which is used as a key for the instances map inside the context), right? Might be helpful to have a comment to that effect, otherwise the introduction of what seems like new global state is confusing.

51–53

This comment needs to be updated (or perhaps just removed).

[aganea]

Rebase.
Integrate with @akhuang's COFFLinkerContext.
Address comments from @smeenai.

[aganea]

Thanks for taking a look!

In D108850#3099397, @smeenai wrote:

I see a bunch of discussions about thread-locals; was that for an earlier version of this patch, or am I just missing something? (I don't see anything thread-local related in the current version of the patch.)

Yes, please take a look at "Diff 4" and before, lctx used to be a THREAD_LOCAL to accommodate for several instances of LLD running in the same process. But @rnk suggested to go in steps, and first move all scattered statics into a global static context.

To make sure I'm understanding this correctly, can the calls to freeArena be removed because the destructor for the context will take care of that now?

Yes.

lld/COFF/InputFiles.cpp

1069

Yes, precisely, to clarify the intention that saver shouldn't change in that loop.

lld/Common/CommonLinkingContext.cpp

22 ↗

(On Diff #373420)

Did you have the chance to run any tests?

23 ↗

(On Diff #373420)

Since this would only trigger in LLVM_ENABLE_ASSERTIONS=ON builds, the performance shouldn't matter that much? Or should it? Otherwise we could only have it in LLVM_ENABLE_EXPENSIVE_CHECKS builds?

[MaskRay]

Will llvm::DenseMap<void *, SpecificAllocBase *> instances; make make<XXX> too slow?

For ELF, I have measured that symbol/section initialization time matters. As an alternative, lld/ELF can do bulk initialization instead.
If an ELF .o file has N sections, 1/3 are discarded sections (SHT_RELA/SHT_SYMTAB/etc) while 1/7 are discarded COMDAT sections.
When linking chrome, perhaps 6/7 are discarded COMDAT sections (too many templates?). It wastes some memory if we don't compute the number of needed sections, but that may be fine.

[aganea]

In D108850#3211509, @MaskRay wrote:

Will llvm::DenseMap<void *, SpecificAllocBase *> instances; make make<XXX> too slow?

I measured a previous version of this patch on Windows with COFF ​and a Chromium-sized output, and I only saw a very tiny departure (the fluctuations could be attributed to mmap'ing). Please see: https://reviews.llvm.org/D108850#2975472
Would you be able to try this patch on Linux linking Chromium?

[MaskRay]

The patch makes my Linux x86-64 lld (-DCMAKE_BUILD_TYPE=Release -DLLVM_TARGETS_TO_BUILD=X86 -DLLVM_ENABLE_PIC=off -DCMAKE_CXX_FLAGS='-fno-asynchronous-unwind-tables') 130+KiB smaller.

I tried linking chrome and clang release/relwithdebinfo with this patch. No significant difference. Here is chrome's --threads=8 output

% hyperfine --warmup 2 --min-runs 20 "numactl -C 20-27 "{/tmp/c/0,/tmp/c/1}" -flavor gnu @response.txt --threads=8 -o lld.8"
Benchmark 1: numactl -C 20-27 /tmp/c/0 -flavor gnu @response.txt --threads=8 -o lld.8
  Time (mean ± σ):      6.699 s ±  0.032 s    [User: 8.815 s, System: 2.439 s]
  Range (min … max):    6.637 s …  6.745 s    20 runs
 
Benchmark 2: numactl -C 20-27 /tmp/c/1 -flavor gnu @response.txt --threads=8 -o lld.8
  Time (mean ± σ):      6.691 s ±  0.027 s    [User: 8.856 s, System: 2.396 s]
  Range (min … max):    6.647 s …  6.743 s    20 runs
 
Summary
  'numactl -C 20-27 /tmp/c/1 -flavor gnu @response.txt --threads=8 -o lld.8' ran
    1.00 ± 0.01 times faster than 'numactl -C 20-27 /tmp/c/0 -flavor gnu @response.txt --threads=8 -o lld.8'

Checked a large internal program, no difference.

Hope that https://maskray.me/blog/2021-12-19-why-isnt-ld.lld-faster#parallel-initialization-of-sections and parallel initialization of symbol tables are not harder to refactor :)

[MaskRay]

-DLLVM_USE_SANITIZER=Address has some leaks.

[aganea]

Fix memory leaks.

[MaskRay]

LGTM.

lld/tools/lld/lld.cpp

164

https://llvm.org/docs/CodingStandards.html#use-early-exits-and-continue-to-simplify-code

177

If an error occurs, link() return false.

After r becomes int, return may be ambitious on whether it applies to the function or to the r variable.

178

You can make r an int and avoid repeated ? 1 : 0 below.

180

Call _Exit to skip destructors if exitEarly is true.

---

The original comment was

// Exit immediately if we don't need to return to the caller.
// This saves time because the overhead of calling destructors
// for all globally-allocated objects is not negligible.

Is it worth copying some words to the new comment?

[aganea]

As suggested by @MaskRay

[MaskRay]

lld/tools/lld/lld.cpp

143

static

[MaskRay]

lld/COFF/Driver.cpp

67

Just raw_ostream

[aganea]

As suggested by @MaskRay

[aganea]

@smeenai @int3 @thakis Any further comments? Can I go ahead with this?

[smeenai]

I wasn't able to get the perf testing done at the time, and I won't have access to one of my testing machines for the next few days, but I'm not anticipating anything dramatic. If @int3 / @thevinster / @oontvoo / @thakis can run tests on their end, that's great, but this looks good to me regardless.

[oontvoo]

In D108850#3226036, @smeenai wrote:

I wasn't able to get the perf testing done at the time, and I won't have access to one of my testing machines for the next few days, but I'm not anticipating anything dramatic. If @int3 / @thevinster / @oontvoo / @thakis can run tests on their end, that's great, but this looks good to me regardless.

Thanks for the headsup. Yeah, I can run a few bms - will share the results in a couple of hours

[oontvoo]

perf-impact seems minimal to me. LGTM!

x base.txt
+ with_patch.txt

SYSTEM CPU time: 
    N           Min           Max        Median           Avg        Stddev
x  10          0.99          1.01          1.01         1.003   0.008232726
+  10          0.98          1.04          1.02         1.014   0.022705848
No difference proven at 95.0% confidence

USER CPU time: 
    N           Min           Max        Median           Avg        Stddev
x  10          3.56          3.61          3.57         3.575   0.018408935
+  10          3.56          3.71          3.62         3.625   0.053176854
Difference at 95.0% confidence
	0.05 +/- 0.0373876
	1.3986% +/- 1.04581%
	(Student's t, pooled s = 0.0397911)

WALL time: 
    N           Min           Max        Median           Avg        Stddev
x  10          4.06          4.12          4.08         4.085   0.024152295
+  10          4.04          4.23          4.14          4.14   0.070867639
Difference at 95.0% confidence
	0.055 +/- 0.0497434
	1.34639% +/- 1.21771%
	(Student's t, pooled s = 0.0529413)

[aganea]

Thanks for testing @oontvoo!

[krzysz00]

`@aganea This change broke SerializeToHsaco in MLIR, can you help resolve this issue or revert? See https://github.com/llvm/llvm-project/blob/main/mlir/lib/Dialect/GPU/Transforms/SerializeToHsaco.cpp#L440

I can't quite tell what the new boolean arguments mean.

[aganea]

In D108850#3280778, @krzysz00 wrote:

`@aganea This change broke SerializeToHsaco in MLIR, can you help resolve this issue or revert? See https://github.com/llvm/llvm-project/blob/main/mlir/lib/Dialect/GPU/Transforms/SerializeToHsaco.cpp#L440

I can't quite tell what the new boolean arguments mean.

Fixing now.

[yaxunl]

This patch caused regressions for HIP on Windows. Basically, any HIP programs using device variables will fail to link on Windows. The issue does not happen on Linux.

To reproduce,

$ cat a-hip-amdgcn-amd-amdhsa-gfx1031.s
        .text
        .amdgcn_target "amdgcn-amd-amdhsa--gfx1031"
        .p2alignl 6, 3214868480
        .fill 48, 4, 3214868480
        .protected      xxx                     ; @xxx
        .type   xxx,@object
        .data
        .globl  xxx
        .p2align        2
xxx:
        .long   123                             ; 0x7b
        .size   xxx, 4

        .ident  "clang version 14.0.0 ([e-mail]:Compute-Mirrors/llvm-project.git ce53ba9aaee8d053604883d5befaf07bad60ba84)"
        .section        ".note.GNU-stack"
        .addrsig
        .amdgpu_metadata
---
amdhsa.kernels:  []
amdhsa.target:   amdgcn-amd-amdhsa--gfx1031
amdhsa.version:
  - 1
  - 1
...

        .end_amdgpu_metadata

$  "C:\\hip\\bin\\clang++.exe" -cc1as -triple amdgcn-amd-amdhsa -filetype obj -target-cpu gfx1031 -mrelocation-model pic -mincremental-linker-compatible --mrelax-relocations -o a-hip-amdgcn-amd-amdhsa-gfx1031.o a-hip-amdgcn-amd-amdhsa-gfx1031.s

$ "C:\\hip\\bin\\lld.exe" -flavor gnu -shared a-hip-amdgcn-amd-amdhsa-gfx1031.o
lld: error: duplicate symbol: xxx
>>> defined at a-hip-amdgcn-amd-amdhsa-gfx1031.o:(xxx)
>>> defined at a-hip-amdgcn-amd-amdhsa-gfx1031.o:(.data+0x0)

[aganea]

@yaxunl I have a fix, I will send a patch later today.

lld/COFF/Chunks.cpp

 //===- Chunks.cpp ---------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "Chunks.h"
 #include "InputFiles.h"
 #include "Symbols.h"
 #include "Writer.h"
 #include "SymbolTable.h"
 #include "lld/Common/ErrorHandler.h"
 #include "llvm/ADT/Twine.h"
 #include "llvm/BinaryFormat/COFF.h"
 #include "llvm/Object/COFF.h"
 #include "llvm/Support/Debug.h"
 #include "llvm/Support/Endian.h"

 void SectionChunk::sortRelocations() {
   auto cmpByVa = [](const coff_relocation &l, const coff_relocation &r) {
     return l.VirtualAddress < r.VirtualAddress;
   };
   if (llvm::is_sorted(getRelocs(), cmpByVa))
     return;
   warn("some relocations in " + file->getName() + " are not sorted");
   MutableArrayRef<coff_relocation> newRelocs(
-      bAlloc.Allocate<coff_relocation>(relocsSize), relocsSize);
+      bAlloc().Allocate<coff_relocation>(relocsSize), relocsSize);
   memcpy(newRelocs.data(), relocsData, relocsSize * sizeof(coff_relocation));
   llvm::sort(newRelocs, cmpByVa);
   setRelocs(newRelocs);
 }
 
 // Similar to writeTo, but suitable for relocating a subsection of the overall
 // section.
 void SectionChunk::writeAndRelocateSubsection(ArrayRef<uint8_t> sec,

lld/COFF/DLL.cpp

for (DefinedImportData *s : syms) {
         names.push_back(make<OrdinalOnlyChunk>(s->getOrdinal()));
       } else {
         auto *c = make<HintNameChunk>(extName, 0);
         names.push_back(make<LookupChunk>(c));
         hintNames.push_back(c);
         // Add a syntentic symbol for this load thunk, using the "__imp_load"
         // prefix, in case this thunk needs to be added to the list of valid
         // call targets for Control Flow Guard.
-        StringRef symName = saver.save("__imp_load_" + extName);
+        StringRef symName = saver().save("__imp_load_" + extName);
         s->loadThunkSym =
             cast<DefinedSynthetic>(symtab->addSynthetic(symName, t));
       }
     }
     thunks.push_back(tm);
     StringRef tmName =
-        saver.save("__tailMerge_" + syms[0]->getDLLName().lower());
+        saver().save("__tailMerge_" + syms[0]->getDLLName().lower());
     symtab->addSynthetic(tmName, tm);
     // Terminate with null values.
     addresses.push_back(make<NullChunk>(8));
     names.push_back(make<NullChunk>(8));
 
     for (int i = 0, e = syms.size(); i < e; ++i)
       syms[i]->setLocation(addresses[base + i]);
     auto *mh = make<NullChunk>(8);

lld/COFF/Driver.cpp

 namespace lld {
 namespace coff {
 
 static Timer inputFileTimer("Input File Reading", Timer::root());
 
 Configuration *config;
 LinkerDriver *driver;
 
-bool link(ArrayRef<const char *> args, bool canExitEarly, raw_ostream &stdoutOS,
-          raw_ostream &stderrOS) {
-  lld::stdoutOS = &stdoutOS;
-  lld::stderrOS = &stderrOS;
-
+bool link(ArrayRef<const char *> args) {
   errorHandler().cleanupCallback = []() {
     TpiSource::clear();
-    freeArena();
     ObjFile::instances.clear();
     PDBInputFile::instances.clear();
     ImportFile::instances.clear();
     BitcodeFile::instances.clear();
     memset(MergeChunk::instances, 0, sizeof(MergeChunk::instances));
     OutputSection::clear();
   };
 
   errorHandler().logName = args::getFilenameWithoutExe(args[0]);
   errorHandler().errorLimitExceededMsg =
       "too many errors emitted, stopping now"
       " (use /errorlimit:0 to see all errors)";
-  errorHandler().exitEarly = canExitEarly;
-  stderrOS.enable_colors(stderrOS.has_colors());
 
   config = make<Configuration>();
   symtab = make<SymbolTable>();
   driver = make<LinkerDriver>();
 
   driver->linkerMain(args);
 
-  // Call exit() if we can to avoid calling destructors.
-  if (canExitEarly)
-    exitLld(errorCount() ? 1 : 0);
-
-  bool ret = errorCount() == 0;
-  if (!canExitEarly)
-    errorHandler().reset();
-  return ret;
+  return errorCount() == 0;
 }
 
 // Parse options of the form "old;new".
 static std::pair<StringRef, StringRef> getOldNewOptions(opt::InputArgList &args,
                                                         unsigned id) {
   auto *arg = args.getLastArg(id);
   if (!arg)
     return {"", ""};
return std::async(strategy, [=]() {
     return MBErrPair{std::move(*mbOrErr), std::error_code()};
   });
 }
 
 // Symbol names are mangled by prepending "_" on x86.
 static StringRef mangle(StringRef sym) {
   assert(config->machine != IMAGE_FILE_MACHINE_UNKNOWN);
   if (config->machine == I386)
-    return saver.save("_" + sym);
+    return saver().save("_" + sym);
   return sym;
 }
 
 static bool findUnderscoreMangle(StringRef sym) {
   Symbol *s = symtab->findMangle(mangle(sym));
   return s && !isa<Undefined>(s);
 }
 
for (StringRef e : directives.exports) {
     // same /EXPORT options. In order to save cost of parsing them,
     // we dedup them first.
     if (!directivesExports.insert(e).second)
       continue;
 
     Export exp = parseExport(e);
     if (config->machine == I386 && config->mingw) {
       if (!isDecorated(exp.name))
-        exp.name = saver.save("_" + exp.name);
+        exp.name = saver().save("_" + exp.name);
       if (!exp.extName.empty() && !isDecorated(exp.extName))
-        exp.extName = saver.save("_" + exp.extName);
+        exp.extName = saver().save("_" + exp.extName);
     }
     exp.directives = true;
     config->exports.push_back(exp);
   }
 
   // Handle /include: in bulk.
   for (StringRef inc : directives.includes)
     addUndefined(inc);
StringRef LinkerDriver::doFindFile(StringRef filename) {
   bool hasPathSep = (filename.find_first_of("/\\") != StringRef::npos);
   if (hasPathSep)
     return filename;
   bool hasExt = filename.contains('.');
   for (StringRef dir : searchPaths) {
     SmallString<128> path = dir;
     sys::path::append(path, filename);
     if (sys::fs::exists(path.str()))
-      return saver.save(path.str());
+      return saver().save(path.str());
     if (!hasExt) {
       path.append(".obj");
       if (sys::fs::exists(path.str()))
-        return saver.save(path.str());
+        return saver().save(path.str());
     }
   }
   return filename;
 }
 
 static Optional<sys::fs::UniqueID> getUniqueID(StringRef path) {
   sys::fs::UniqueID ret;
   if (sys::fs::getUniqueID(path, ret))

 // MinGW specific. If an embedded directive specified to link to
 // foo.lib, but it isn't found, try libfoo.a instead.
 StringRef LinkerDriver::doFindLibMinGW(StringRef filename) {
   if (filename.contains('/') || filename.contains('\\'))
     return filename;
 
   SmallString<128> s = filename;
   sys::path::replace_extension(s, ".a");
-  StringRef libName = saver.save("lib" + s.str());
+  StringRef libName = saver().save("lib" + s.str());
   return doFindFile(libName);
 }
 
 // Find library file from search path.
 StringRef LinkerDriver::doFindLib(StringRef filename) {
   // Add ".lib" to Filename if that has no file extension.
   bool hasExt = filename.contains('.');
   if (!hasExt)
-    filename = saver.save(filename + ".lib");
+    filename = saver().save(filename + ".lib");
   StringRef ret = doFindFile(filename);
   // For MinGW, if the find above didn't turn up anything, try
   // looking for a MinGW formatted library name.
   if (config->mingw && ret == filename)
     return doFindLibMinGW(filename);
   return ret;
 }
 
Optional<StringRef> LinkerDriver::findLib(StringRef filename) {
   return path;
 }
 
 // Parses LIB environment which contains a list of search paths.
 void LinkerDriver::addLibSearchPaths() {
   Optional<std::string> envOpt = Process::GetEnv("LIB");
   if (!envOpt.hasValue())
     return;
-  StringRef env = saver.save(*envOpt);
+  StringRef env = saver().save(*envOpt);
   while (!env.empty()) {
     StringRef path;
     std::tie(path, env) = env.split(';');
     searchPaths.push_back(path);
   }
 }
 
 Symbol *LinkerDriver::addUndefined(StringRef name) {
std::unique_ptr<MemoryBuffer> mb =
             "could not open " + path);
   COFFModuleDefinition m = check(parseCOFFModuleDefinition(
       mb->getMemBufferRef(), config->machine, config->mingw));
 
   // Include in /reproduce: output if applicable.
   driver->takeBuffer(std::move(mb));
 
   if (config->outputFile.empty())
-    config->outputFile = std::string(saver.save(m.OutputFile));
-  config->importName = std::string(saver.save(m.ImportName));
+    config->outputFile = std::string(saver().save(m.OutputFile));
+  config->importName = std::string(saver().save(m.ImportName));
   if (m.ImageBase)
     config->imageBase = m.ImageBase;
   if (m.StackReserve)
     config->stackReserve = m.StackReserve;
   if (m.StackCommit)
     config->stackCommit = m.StackCommit;
   if (m.HeapReserve)
     config->heapReserve = m.HeapReserve;
static void parseModuleDefs(StringRef path) {
   for (COFFShortExport e1 : m.Exports) {
     Export e2;
     // In simple cases, only Name is set. Renamed exports are parsed
     // and set as "ExtName = Name". If Name has the form "OtherDll.Func",
     // it shouldn't be a normal exported function but a forward to another
     // DLL instead. This is supported by both MS and GNU linkers.
     if (!e1.ExtName.empty() && e1.ExtName != e1.Name &&
         StringRef(e1.Name).contains('.')) {
-      e2.name = saver.save(e1.ExtName);
-      e2.forwardTo = saver.save(e1.Name);
+      e2.name = saver().save(e1.ExtName);
+      e2.forwardTo = saver().save(e1.Name);
       config->exports.push_back(e2);
       continue;
     }
-    e2.name = saver.save(e1.Name);
-    e2.extName = saver.save(e1.ExtName);
+    e2.name = saver().save(e1.Name);
+    e2.extName = saver().save(e1.ExtName);
     e2.ordinal = e1.Ordinal;
     e2.noname = e1.Noname;
     e2.data = e1.Data;
     e2.isPrivate = e1.Private;
     e2.constant = e1.Constant;
     config->exports.push_back(e2);
   }
 }
if (!config->dynamicBase &&
     error("/dynamicbase:no is not compatible with " +
           machineToStr(config->machine));
 
   // Handle /export
   for (auto *arg : args.filtered(OPT_export)) {
     Export e = parseExport(arg->getValue());
     if (config->machine == I386) {
       if (!isDecorated(e.name))
-        e.name = saver.save("_" + e.name);
+        e.name = saver().save("_" + e.name);
       if (!e.extName.empty() && !isDecorated(e.extName))
-        e.extName = saver.save("_" + e.extName);
+        e.extName = saver().save("_" + e.extName);
     }
     config->exports.push_back(e);
   }
 
   // Handle /def
   if (auto *arg = args.getLastArg(OPT_deffile)) {
     // parseModuleDefs mutates Config object.
     parseModuleDefs(arg->getValue());

lld/COFF/DriverUtils.cpp

 namespace coff {
 namespace {
 
 const uint16_t SUBLANG_ENGLISH_US = 0x0409;
 const uint16_t RT_MANIFEST = 24;
 
 class Executor {
 public:
-  explicit Executor(StringRef s) : prog(saver.save(s)) {}
-  void add(StringRef s) { args.push_back(saver.save(s)); }
-  void add(std::string &s) { args.push_back(saver.save(s)); }
-  void add(Twine s) { args.push_back(saver.save(s)); }
-  void add(const char *s) { args.push_back(saver.save(s)); }
+  explicit Executor(StringRef s) : prog(saver().save(s)) {}
+  void add(StringRef s) { args.push_back(saver().save(s)); }
+  void add(std::string &s) { args.push_back(saver().save(s)); }
+  void add(Twine s) { args.push_back(saver().save(s)); }
+  void add(const char *s) { args.push_back(saver().save(s)); }
 
   void run() {
     ErrorOr<std::string> exeOrErr = sys::findProgramByName(prog);
     if (auto ec = exeOrErr.getError())
       fatal("unable to find " + prog + " in PATH: " + ec.message());
-    StringRef exe = saver.save(*exeOrErr);
+    StringRef exe = saver().save(*exeOrErr);
     args.insert(args.begin(), exe);
 
     if (sys::ExecuteAndWait(args[0], args) != 0)
       fatal("ExecuteAndWait failed: " +
             llvm::join(args.begin(), args.end(), " "));
   }
 
 private:

 // with or without a leading underscore. (MinGW specific.)
 static StringRef killAt(StringRef sym, bool prefix) {
   if (sym.empty())
     return sym;
   // Strip any trailing stdcall suffix
   sym = sym.substr(0, sym.find('@', 1));
   if (!sym.startswith("@")) {
     if (prefix && !sym.startswith("_"))
-      return saver.save("_" + sym);
+      return saver().save("_" + sym);
     return sym;
   }
   // For fastcall, remove the leading @ and replace it with an
   // underscore, if prefixes are used.
   sym = sym.substr(1);
   if (prefix)
-    sym = saver.save("_" + sym);
+    sym = saver().save("_" + sym);
   return sym;
 }
 
 // Performs error checking on all /export arguments.
 // It also sets ordinals.
 void fixupExports() {
   // Symbol ordinals must be unique.
   std::set<uint16_t> ords;
opt::InputArgList ArgParser::parse(ArrayRef<const char *> argv) {
   opt::InputArgList args = optTable.ParseArgs(argv, missingIndex, missingCount);
 
   // Expand response files (arguments in the form of @<filename>) and insert
   // flags from %LINK% and %_LINK_%, and then parse the argument again.
   SmallVector<const char *, 256> expandedArgv(argv.data(),
                                               argv.data() + argv.size());
   if (!args.hasArg(OPT_lldignoreenv))
     addLINK(expandedArgv);
-  cl::ExpandResponseFiles(saver, getQuotingStyle(args), expandedArgv);
+  cl::ExpandResponseFiles(saver(), getQuotingStyle(args), expandedArgv);
   args = optTable.ParseArgs(makeArrayRef(expandedArgv).drop_front(),
                             missingIndex, missingCount);
 
   // Print the real command line if response files are expanded.
   if (args.hasArg(OPT_verbose) && argv.size() != expandedArgv.size()) {
     std::string msg = "Command line:";
     for (const char *s : expandedArgv)
       msg += " " + std::string(s);

 // Tokenizes and parses a given string as command line in .drective section.
 ParsedDirectives ArgParser::parseDirectives(StringRef s) {
   ParsedDirectives result;
   SmallVector<const char *, 16> rest;
 
   // Handle /EXPORT and /INCLUDE in a fast path. These directives can appear for
   // potentially every symbol in the object, so they must be handled quickly.
   SmallVector<StringRef, 16> tokens;
-  cl::TokenizeWindowsCommandLineNoCopy(s, saver, tokens);
+  cl::TokenizeWindowsCommandLineNoCopy(s, saver(), tokens);
   for (StringRef tok : tokens) {
     if (tok.startswith_insensitive("/export:") ||
         tok.startswith_insensitive("-export:"))
       result.exports.push_back(tok.substr(strlen("/export:")));
     else if (tok.startswith_insensitive("/include:") ||
              tok.startswith_insensitive("-include:"))
       result.includes.push_back(tok.substr(strlen("/include:")));
     else {
       // Copy substrings that are not valid C strings. The tokenizer may have
       // already copied quoted arguments for us, so those do not need to be
       // copied again.
       bool HasNul = tok.end() != s.end() && tok.data()[tok.size()] == '\0';
-      rest.push_back(HasNul ? tok.data() : saver.save(tok).data());
+      rest.push_back(HasNul ? tok.data() : saver().save(tok).data());
     }
   }
 
   // Make InputArgList from unparsed string vectors.
   unsigned missingIndex;
   unsigned missingCount;
 
   result.args = optTable.ParseArgs(rest, missingIndex, missingCount);
void ArgParser::addLINK(SmallVector<const char *, 256> &argv) {
   if (Optional<std::string> s = Process::GetEnv("_LINK_")) {
     std::vector<const char *> v = tokenize(*s);
     argv.insert(std::next(argv.begin()), v.begin(), v.end());
   }
 }
 
 std::vector<const char *> ArgParser::tokenize(StringRef s) {
   SmallVector<const char *, 16> tokens;
-  cl::TokenizeWindowsCommandLine(s, saver, tokens);
+  cl::TokenizeWindowsCommandLine(s, saver(), tokens);
   return std::vector<const char *>(tokens.begin(), tokens.end());
 }
 
 void printHelp(const char *argv0) {
   optTable.printHelp(lld::outs(),
                      (std::string(argv0) + " [options] file...").c_str(),
                      "LLVM Linker", false);
 }
 
 } // namespace coff
 } // namespace lld

lld/COFF/InputFiles.cpp

if (!dwarf) {
     if (!dwarf)
       return None;
   }
   if (config->machine == I386)
     var.consume_front("_");
   Optional<std::pair<std::string, unsigned>> ret = dwarf->getVariableLoc(var);
   if (!ret)
     return None;
-  return std::make_pair(saver.save(ret->first), ret->second);
+  return std::make_pair(saver().save(ret->first), ret->second);
 }
 
 // Used only for DWARF debug info, which is not common (except in MinGW
 // environments).
 Optional<DILineInfo> ObjFile::getDILineInfo(uint32_t offset,
                                             uint32_t sectionIndex) {
   if (!dwarf) {
     dwarf = make<DWARFCache>(DWARFContext::create(*getCOFFObj()));
     if (!dwarf)
       return None;
   }
 
   return dwarf->getDILineInfo(offset, sectionIndex);
 }
 
 void ImportFile::parse() {
   const char *buf = mb.getBufferStart();
   const auto *hdr = reinterpret_cast<const coff_import_header *>(buf);
 
   // Check if the total size is valid.
   if (mb.getBufferSize() != sizeof(*hdr) + hdr->SizeOfData)
     fatal("broken import library");
 
   // Read names and create an __imp_ symbol.
-  StringRef name = saver.save(StringRef(buf + sizeof(*hdr)));
-  StringRef impName = saver.save("__imp_" + name);
+  StringRef name = saver().save(StringRef(buf + sizeof(*hdr)));
+  StringRef impName = saver().save("__imp_" + name);
   const char *nameStart = buf + sizeof(coff_import_header) + name.size() + 1;
   dllName = std::string(StringRef(nameStart));
   StringRef extName;
   switch (hdr->getNameType()) {
   case IMPORT_ORDINAL:
     extName = "";
     break;
   case IMPORT_NAME:
if (config->thinLTOIndexOnly)
     path = replaceThinLTOSuffix(mb.getBufferIdentifier());
 
   // ThinLTO assumes that all MemoryBufferRefs given to it have a unique
   // name. If two archives define two members with the same name, this
   // causes a collision which result in only one of the objects being taken
   // into consideration at LTO time (which very likely causes undefined
   // symbols later in the link stage). So we append file offset to make
   // filename unique.
-  MemoryBufferRef mbref(
-      mb.getBuffer(),
-      saver.save(archiveName.empty() ? path
-                                     : archiveName + sys::path::filename(path) +
+  MemoryBufferRef mbref(mb.getBuffer(),
+                        saver().save(archiveName.empty()
+                                         ? path
+                                         : archiveName +
+                                               sys::path::filename(path) +
                                                utostr(offsetInArchive)));
 
   obj = check(lto::InputFile::create(mbref));
 }
 
 BitcodeFile::~BitcodeFile() = default;
 
 namespace {
 // Convenience class for initializing a coff_section with specific flags.

 
 FakeSection ltoTextSection(IMAGE_SCN_MEM_EXECUTE);
 FakeSection ltoDataSection(IMAGE_SCN_CNT_INITIALIZED_DATA);
 FakeSectionChunk ltoTextSectionChunk(&ltoTextSection.section);
 FakeSectionChunk ltoDataSectionChunk(&ltoDataSection.section);
 } // namespace
 
 void BitcodeFile::parse() {
+  llvm::StringSaver &saver = lld::saver();
   std::vector<std::pair<Symbol *, bool>> comdat(obj->getComdatTable().size());
   for (size_t i = 0; i != obj->getComdatTable().size(); ++i)
     // FIXME: Check nodeduplicate
     comdat[i] =
         symtab->addComdat(this, saver.save(obj->getComdatTable()[i].first));
   for (const lto::InputFile::Symbol &objSym : obj->symbols()) {
     StringRef symName = saver.save(objSym.getName());
     int comdatIndex = objSym.getComdatIndex();
for (const auto &exp : coffObj->export_directories()) {
 
     Symbol *s = make<Symbol>();
     s->dllName = dllName;
     s->symbolName = symbolName;
     s->importType = code ? ImportType::IMPORT_CODE : ImportType::IMPORT_DATA;
     s->nameType = ImportNameType::IMPORT_NAME;
 
     if (coffObj->getMachine() == I386) {
-      s->symbolName = symbolName = saver.save("_" + symbolName);
+      s->symbolName = symbolName = saver().save("_" + symbolName);
       s->nameType = ImportNameType::IMPORT_NAME_NOPREFIX;
     }
 
-    StringRef impName = saver.save("__imp_" + symbolName);
+    StringRef impName = saver().save("__imp_" + symbolName);
     symtab->addLazyDLLSymbol(this, s, impName);
     if (code)
       symtab->addLazyDLLSymbol(this, s, symbolName);
   }
 }
 
 MachineTypes DLLFile::getMachineType() {
   if (coffObj)
     return static_cast<MachineTypes>(coffObj->getMachine());
   return IMAGE_FILE_MACHINE_UNKNOWN;
 }
 
 void DLLFile::makeImport(DLLFile::Symbol *s) {
   if (!seen.insert(s->symbolName).second)
     return;
 
   size_t impSize = s->dllName.size() + s->symbolName.size() + 2; // +2 for NULs
   size_t size = sizeof(coff_import_header) + impSize;
-  char *buf = bAlloc.Allocate<char>(size);
+  char *buf = bAlloc().Allocate<char>(size);
   memset(buf, 0, size);
   char *p = buf;
   auto *imp = reinterpret_cast<coff_import_header *>(p);
   p += sizeof(*imp);
   imp->Sig2 = 0xFFFF;
   imp->Machine = coffObj->getMachine();
   imp->SizeOfData = impSize;
   imp->OrdinalHint = 0; // Only linking by name

lld/COFF/LTO.cpp

std::vector<InputFile *> BitcodeCompiler::compile() {
   // specified, configure LTO to use it as the cache directory.
   lto::NativeObjectCache cache;
   if (!config->ltoCache.empty())
     cache = check(lto::localCache(
         config->ltoCache, [&](size_t task, std::unique_ptr<MemoryBuffer> mb) {
           files[task] = std::move(mb);
         }));
 
+  CommonLinkingContext &ctx = context();
+
   checkError(ltoObj->run(
       [&](size_t task) {
         return std::make_unique<lto::NativeObjectStream>(
             std::make_unique<raw_svector_ostream>(buf[task]));
       },
-      cache));
+      cache, [&ctx] { attachContext(ctx); }));
 
   // Emit empty index files for non-indexed files
   for (StringRef s : thinIndices) {
     std::string path = getThinLTOOutputFile(s);
     openFile(path + ".thinlto.bc");
     if (config->thinLTOEmitImportsFiles)
       openFile(path + ".imports");
   }
std::vector<InputFile *> BitcodeCompiler::compile() {
   std::vector<InputFile *> ret;
   for (unsigned i = 0; i != maxTasks; ++i) {
     // Assign unique names to LTO objects. This ensures they have unique names
     // in the PDB if one is produced. The names should look like:
     // - foo.exe.lto.obj
     // - foo.exe.lto.1.obj
     // - ...
     StringRef ltoObjName =
-        saver.save(Twine(config->outputFile) + ".lto" +
+        saver().save(Twine(config->outputFile) + ".lto" +
                      (i == 0 ? Twine("") : Twine('.') + Twine(i)) + ".obj");
 
     // Get the native object contents either from the cache or from memory.  Do
     // not use the cached MemoryBuffer directly, or the PDB will not be
     // deterministic.
     StringRef objBuf;
     if (files[i])
       objBuf = files[i]->getBuffer();
     else

lld/COFF/MinGW.cpp

for (Export &e : config->exports) {
     }
     os << "\n";
   }
 }
 
 static StringRef mangle(Twine sym) {
   assert(config->machine != IMAGE_FILE_MACHINE_UNKNOWN);
   if (config->machine == I386)
-    return saver.save("_" + sym);
-  return saver.save(sym);
+    return saver().save("_" + sym);
+  return saver().save(sym);
 }
 
 // Handles -wrap option.
 //
 // This function instantiates wrapper symbols. At this point, they seem
 // like they are not being used at all, so we explicitly set some flags so
 // that LTO won't eliminate them.
 std::vector<WrappedSymbol>
for (const WrappedSymbol &w : wrapped) {
     if (Defined *d = dyn_cast<Defined>(w.wrap)) {
       Symbol *imp = symtab->find(("__imp_" + w.sym->getName()).str());
       // Create a new defined local import for the wrap symbol. If
       // no imp prefixed symbol existed, there's no need for it.
       // (We can't easily distinguish whether any object file actually
       // referenced it or not, though.)
       if (imp) {
         DefinedLocalImport *wrapimp = make<DefinedLocalImport>(
-            saver.save("__imp_" + w.wrap->getName()), d);
+            saver().save("__imp_" + w.wrap->getName()), d);
         symtab->localImportChunks.push_back(wrapimp->getChunk());
         map[imp] = wrapimp;
       }
     }
   }
 
   // Update pointers in input files.
   parallelForEach(ObjFile::instances, [&](ObjFile *file) {
     MutableArrayRef<Symbol *> syms = file->getMutableSymbols();
     for (size_t i = 0, e = syms.size(); i != e; ++i)
       if (Symbol *s = map.lookup(syms[i]))
         syms[i] = s;
   });
 }

lld/COFF/PDB.cpp

 namespace {
 class DebugSHandler;
 
 class PDBLinker {
   friend DebugSHandler;
 
 public:
   PDBLinker(SymbolTable *symtab)
-      : symtab(symtab), builder(bAlloc), tMerger(bAlloc) {
+      : symtab(symtab), builder(bAlloc()), tMerger(bAlloc()) {
     // This isn't strictly necessary, but link.exe usually puts an empty string
     // as the first "valid" string in the string table, so we do the same in
     // order to maintain as much byte-for-byte compatibility as possible.
     pdbStrTab.insert("");
   }
 
   /// Emit the basic PDB structure: initial streams, headers, etc.
   void initialize(llvm::codeview::DebugInfo *buildId);
static void addGlobalSymbol(pdb::GSIStreamBuilder &builder, uint16_t modIndex,
   case SymbolKind::S_GDATA32:
   case SymbolKind::S_GTHREAD32:
   case SymbolKind::S_LTHREAD32:
   case SymbolKind::S_LDATA32:
   case SymbolKind::S_PROCREF:
   case SymbolKind::S_LPROCREF: {
     // sym is a temporary object, so we have to copy and reallocate the record
     // to stabilize it.
-    uint8_t *mem = bAlloc.Allocate<uint8_t>(sym.length());
+    uint8_t *mem = bAlloc().Allocate<uint8_t>(sym.length());
     memcpy(mem, sym.data().data(), sym.length());
     builder.addGlobalSymbol(CVSymbol(makeArrayRef(mem, sym.length())));
     break;
   }
   case SymbolKind::S_GPROC32:
   case SymbolKind::S_LPROC32: {
     SymbolRecordKind k = SymbolRecordKind::ProcRefSym;
     if (sym.kind() == SymbolKind::S_LPROC32)
static void warnUnusable(InputFile *f, Error e) {
   if (e)
     warn(msg + "\n>>> failed to load reference " + toString(std::move(e)));
   else
     warn(msg);
 }
 
 // Allocate memory for a .debug$S / .debug$F section and relocate it.
 static ArrayRef<uint8_t> relocateDebugChunk(SectionChunk &debugChunk) {
-  uint8_t *buffer = bAlloc.Allocate<uint8_t>(debugChunk.getSize());
+  uint8_t *buffer = bAlloc().Allocate<uint8_t>(debugChunk.getSize());
   assert(debugChunk.getOutputSectionIdx() == 0 &&
          "debug sections should not be in output sections");
   debugChunk.writeTo(buffer);
   return makeArrayRef(buffer, debugChunk.getSize());
 }
 
 void PDBLinker::addDebugSymbols(TpiSource *source) {
   // If this TpiSource doesn't have an object file, it must be from a type
static void addCommonLinkerModuleSymbols(StringRef path,
   ebs.Fields.push_back("exe");
   SmallString<64> exe = config->argv[0];
   pdbMakeAbsolute(exe);
   ebs.Fields.push_back(exe);
   ebs.Fields.push_back("pdb");
   ebs.Fields.push_back(path);
   ebs.Fields.push_back("cmd");
   ebs.Fields.push_back(argStr);
+  llvm::BumpPtrAllocator &bAlloc = lld::bAlloc();
   mod.addSymbol(codeview::SymbolSerializer::writeOneSymbol(
       ons, bAlloc, CodeViewContainer::Pdb));
   mod.addSymbol(codeview::SymbolSerializer::writeOneSymbol(
       cs, bAlloc, CodeViewContainer::Pdb));
   mod.addSymbol(codeview::SymbolSerializer::writeOneSymbol(
       ebs, bAlloc, CodeViewContainer::Pdb));
 }
 
static void addLinkerModuleCoffGroup(PartialSection *sec,
 
   // Somehow .idata sections & sections groups in the debug symbol stream have
   // the "write" flag set. However the section header for the corresponding
   // .idata section doesn't have it.
   if (cgs.Name.startswith(".idata"))
     cgs.Characteristics |= llvm::COFF::IMAGE_SCN_MEM_WRITE;
 
   mod.addSymbol(codeview::SymbolSerializer::writeOneSymbol(
-      cgs, bAlloc, CodeViewContainer::Pdb));
+      cgs, bAlloc(), CodeViewContainer::Pdb));
 }
 
 static void addLinkerModuleSectionSymbol(pdb::DbiModuleDescriptorBuilder &mod,
                                          OutputSection &os) {
   SectionSym sym(SymbolRecordKind::SectionSym);
   sym.Alignment = 12; // 2^12 = 4KB
   sym.Characteristics = os.header.Characteristics;
   sym.Length = os.getVirtualSize();
   sym.Name = os.name;
   sym.Rva = os.getRVA();
   sym.SectionNumber = os.sectionIndex;
   mod.addSymbol(codeview::SymbolSerializer::writeOneSymbol(
-      sym, bAlloc, CodeViewContainer::Pdb));
+      sym, bAlloc(), CodeViewContainer::Pdb));
 
   // Skip COFF groups in MinGW because it adds a significant footprint to the
   // PDB, due to each function being in its own section
   if (config->mingw)
     return;
 
   // Output COFF groups for individual chunks of this section.
   for (PartialSection *sec : os.contribSections) {
for (ImportFile *file : ImportFile::instances) {
     ts.Parent = 0;
     ts.End = 0;
     ts.Next = 0;
     ts.Thunk = ThunkOrdinal::Standard;
     ts.Length = thunkChunk->getSize();
     ts.Segment = thunkOS->sectionIndex;
     ts.Offset = thunkChunk->getRVA() - thunkOS->getRVA();
 
+    llvm::BumpPtrAllocator &bAlloc = lld::bAlloc();
     mod->addSymbol(codeview::SymbolSerializer::writeOneSymbol(
         ons, bAlloc, CodeViewContainer::Pdb));
     mod->addSymbol(codeview::SymbolSerializer::writeOneSymbol(
         cs, bAlloc, CodeViewContainer::Pdb));
 
     CVSymbol newSym = codeview::SymbolSerializer::writeOneSymbol(
         ts, bAlloc, CodeViewContainer::Pdb);
 

lld/COFF/SymbolTable.cpp

 getFileLineDwarf(const SectionChunk *c, uint32_t addr) {
   Optional<DILineInfo> optionalLineInfo =
       c->file->getDILineInfo(addr, c->getSectionNumber() - 1);
   if (!optionalLineInfo)
     return None;
   const DILineInfo &lineInfo = *optionalLineInfo;
   if (lineInfo.FileName == DILineInfo::BadString)
     return None;
-  return std::make_pair(saver.save(lineInfo.FileName), lineInfo.Line);
+  return std::make_pair(saver().save(lineInfo.FileName), lineInfo.Line);
 }
 
 static Optional<std::pair<StringRef, uint32_t>>
 getFileLine(const SectionChunk *c, uint32_t addr) {
   // MinGW can optionally use codeview, even if the default is dwarf.
   Optional<std::pair<StringRef, uint32_t>> fileLine =
       getFileLineCodeView(c, addr);
   // If codeview didn't yield any result, check dwarf in MinGW mode.

lld/COFF/Writer.cpp

for (size_t i = 0; i != os->chunks.size(); ++i) {
     // Get a writable copy of this section's relocations so they can be
     // modified. If the relocations point into the object file, allocate new
     // memory. Otherwise, this must be previously allocated memory that can be
     // modified in place.
     ArrayRef<coff_relocation> curRelocs = sc->getRelocs();
     MutableArrayRef<coff_relocation> newRelocs;
     if (originalRelocs.data() == curRelocs.data()) {
       newRelocs = makeMutableArrayRef(
-          bAlloc.Allocate<coff_relocation>(originalRelocs.size()),
+          bAlloc().Allocate<coff_relocation>(originalRelocs.size()),
           originalRelocs.size());
     } else {
       newRelocs = makeMutableArrayRef(
           const_cast<coff_relocation *>(curRelocs.data()), curRelocs.size());
     }
 
     // Copy each relocation, but replace the symbol table indices which need
     // thunks.

lld/Common/CMakeLists.txt

add_custom_command(OUTPUT "${version_inc}"
   "-DHEADER_FILE=${version_inc}"
   -P "${generate_vcs_version_script}")
 
 # Mark the generated header as being generated.
 set_source_files_properties("${version_inc}"
   PROPERTIES GENERATED TRUE
   HEADER_FILE_ONLY TRUE)
 
+# Enable errors for any global constructors.
+add_flag_if_supported("-Werror=global-constructors" WERROR_GLOBAL_CONSTRUCTOR)
+
 add_lld_library(lldCommon
   Args.cpp
   DWARF.cpp
   ErrorHandler.cpp
   Filesystem.cpp
+  Globals.cpp
   Memory.cpp
   Reproduce.cpp
   Strings.cpp
   TargetOptionsCommandFlags.cpp
   Timer.cpp
   VCSVersion.inc
   Version.cpp
 

lld/Common/ErrorHandler.cpp

 //===- ErrorHandler.cpp ---------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "lld/Common/ErrorHandler.h"
-
-#include "llvm/Support/Parallel.h"
-
+#include "lld/Common/Globals.h"
 #include "llvm/ADT/Twine.h"
 #include "llvm/IR/DiagnosticInfo.h"
 #include "llvm/IR/DiagnosticPrinter.h"
 #include "llvm/Support/CrashRecoveryContext.h"
 #include "llvm/Support/ManagedStatic.h"
+#include "llvm/Support/Parallel.h"
 #include "llvm/Support/Process.h"
 #include "llvm/Support/Program.h"
 #include "llvm/Support/raw_ostream.h"
 #include <mutex>
 #include <regex>
 
 using namespace llvm;
 using namespace lld;
 
+namespace lld {
+struct ErrorState {
   // The functions defined in this file can be called from multiple threads,
   // but lld::outs() or lld::errs() are not thread-safe. We protect them using a
   // mutex.
-static std::mutex mu;
+  std::mutex mu;
 
   // We want to separate multi-line messages with a newline. `sep` is "\n"
   // if the last messages was multi-line. Otherwise "".
-static StringRef sep;
+  StringRef sep;
+
+  // We wrap stdout and stderr so that you can pass alternative stdout/stderr as
+  // arguments to lld::*::link() functions.
+  llvm::raw_ostream *stdoutOS{};
+  llvm::raw_ostream *stderrOS{};
+
+  ErrorHandler handler;
+};
+
+ErrorState *initError(char *const start, char *&next, unsigned &offset) {
+  return initContextHelper<ErrorState>(start, next, offset);
+}
+
+void freeError(ErrorState *e) { e->~ErrorState(); }
+} // namespace lld
 
 static StringRef getSeparator(const Twine &msg) {
   if (StringRef(msg.str()).contains('\n'))
     return "\n";
   return "";
 }
 
-raw_ostream *lld::stdoutOS;
-raw_ostream *lld::stderrOS;
+ErrorHandler::~ErrorHandler() {
+  if (cleanupCallback)
+    cleanupCallback();
+}
 
-ErrorHandler &lld::errorHandler() {
-  static ErrorHandler handler;
-  return handler;
+ErrorHandler &lld::errorHandler() { return context().e().handler; }
+
+void lld::setStdPipes(llvm::raw_ostream &stdoutOS,
+                      llvm::raw_ostream &stderrOS) {
+  CommonLinkingContext &ctx = context();
+  ctx.e().stdoutOS = &stdoutOS;
+  ctx.e().stderrOS = &stderrOS;
 }
 
 raw_ostream &lld::outs() {
-  if (errorHandler().disableOutput)
+  ErrorState &e = context().e();
+  if (e.handler.disableOutput)
     return llvm::nulls();
-  return stdoutOS ? *stdoutOS : llvm::outs();
+  return e.stdoutOS ? *e.stdoutOS : llvm::outs();
 }
 
 raw_ostream &lld::errs() {
-  if (errorHandler().disableOutput)
+  ErrorState &e = context().e();
+  if (e.handler.disableOutput)
     return llvm::nulls();
-  return stderrOS ? *stderrOS : llvm::errs();
+  return e.stderrOS ? *e.stderrOS : llvm::errs();
 }
 
 void lld::exitLld(int val) {
+  if (hasContext()) {
+    ErrorState &e = context().e();
     // Delete any temporary file, while keeping the memory mapping open.
-  if (errorHandler().outputBuffer)
-    errorHandler().outputBuffer->discard();
+    if (e.handler.outputBuffer)
+      e.handler.outputBuffer->discard();
+  }
 
   // Re-throw a possible signal or exception once/if it was catched by
   // safeLldMain().
   CrashRecoveryContext::throwIfCrash(val);
 
   // Dealloc/destroy ManagedStatic variables before calling _exit().
   // In an LTO build, allows us to get the output of -time-passes.
   // Ensures that the thread pool for the parallel algorithms is stopped to
   // avoid intermittent crashes on Windows when exiting.
   if (!CrashRecoveryContext::GetCurrent())
     llvm_shutdown();
 
-  {
-    std::lock_guard<std::mutex> lock(mu);
+  if (hasContext()) {
+    ErrorState &e = context().e();
+    std::lock_guard<std::mutex> lock(e.mu);
     lld::outs().flush();
     lld::errs().flush();
   }
+
   // When running inside safeLldMain(), restore the control flow back to the
   // CrashRecoveryContext. Otherwise simply use _exit(), meanning no cleanup,
   // since we want to avoid further crashes on shutdown.
   llvm::sys::Process::Exit(val, /*NoCleanup=*/true);
 }
 
 void lld::diagnosticHandler(const DiagnosticInfo &di) {
   SmallString<128> s;
std::string ErrorHandler::getLocation(const Twine &msg) {
   }
 
   return std::string(logName);
 }
 
 void ErrorHandler::log(const Twine &msg) {
   if (!verbose || disableOutput)
     return;
-  std::lock_guard<std::mutex> lock(mu);
+  ErrorState &e = context().e();
+  std::lock_guard<std::mutex> lock(e.mu);
   lld::errs() << logName << ": " << msg << "\n";
 }
 
 void ErrorHandler::message(const Twine &msg) {
   if (disableOutput)
     return;
-  std::lock_guard<std::mutex> lock(mu);
+  ErrorState &e = context().e();
+  std::lock_guard<std::mutex> lock(e.mu);
   lld::outs() << msg << "\n";
   lld::outs().flush();
 }
 
 void ErrorHandler::warn(const Twine &msg) {
   if (fatalWarnings) {
     error(msg);
     return;
   }
 
-  std::lock_guard<std::mutex> lock(mu);
-  lld::errs() << sep << getLocation(msg) << ": " << Colors::MAGENTA
+  ErrorState &e = context().e();
+  std::lock_guard<std::mutex> lock(e.mu);
+  lld::errs() << e.sep << getLocation(msg) << ": " << Colors::MAGENTA
               << "warning: " << Colors::RESET << msg << "\n";
-  sep = getSeparator(msg);
+  e.sep = getSeparator(msg);
 }
 
 void ErrorHandler::error(const Twine &msg) {
+  ErrorState &e = context().e();
   // If Visual Studio-style error message mode is enabled,
   // this particular error is printed out as two errors.
   if (vsDiagnostics) {
     static std::regex re(R"(^(duplicate symbol: .*))"
                          R"((\n>>> defined at \S+:\d+.*\n>>>.*))"
                          R"((\n>>> defined at \S+:\d+.*\n>>>.*))");
     std::string str = msg.str();
     std::smatch m;
 
     if (std::regex_match(str, m, re)) {
       error(m.str(1) + m.str(2));
       error(m.str(1) + m.str(3));
       return;
     }
   }
 
   bool exit = false;
   {
-    std::lock_guard<std::mutex> lock(mu);
+    std::lock_guard<std::mutex> lock(e.mu);
 
     if (errorLimit == 0 || errorCount < errorLimit) {
-      lld::errs() << sep << getLocation(msg) << ": " << Colors::RED
+      lld::errs() << e.sep << getLocation(msg) << ": " << Colors::RED
                   << "error: " << Colors::RESET << msg << "\n";
     } else if (errorCount == errorLimit) {
-      lld::errs() << sep << getLocation(msg) << ": " << Colors::RED
+      lld::errs() << e.sep << getLocation(msg) << ": " << Colors::RED
                   << "error: " << Colors::RESET << errorLimitExceededMsg
                   << "\n";
       exit = exitEarly;
     }
 
-    sep = getSeparator(msg);
+    e.sep = getSeparator(msg);
     ++errorCount;
   }
 
   if (exit)
     exitLld(1);
 }
 
 void ErrorHandler::error(const Twine &msg, ErrorTag tag,

lld/Common/Globals.cpp

+//===- Globals.cpp --------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "lld/Common/Globals.h"
+#include "lld/Common/ErrorHandler.h"
+#include "lld/Common/Memory.h"
+
+using namespace llvm;
+using namespace lld;
+
+namespace lld {
+MemoryState *initMemoryState(char *const start, char *&next, unsigned &offset,
+                             llvm::BumpPtrAllocator &bAlloc);
+TargetOptState *initTargetOpt(char *const start, char *&next, unsigned &offset);
+ErrorState *initError(char *const start, char *&next, unsigned &offset);
+TimerState *initTimer(char *const start, char *&next, unsigned &offset);
+
+void freeMemory(MemoryState *m);
+void freeTargetOpt(TargetOptState *to);
+void freeError(ErrorState *e);
+void freeTimer(TimerState *t);
+} // namespace lld
+
+MemoryState &lld::CommonLinkingContext::m() {
+  return *(MemoryState *)((char *)storage + memoryOffset);
+}
+TargetOptState &lld::CommonLinkingContext::to() {
+  return *(TargetOptState *)((char *)storage + targetOptOffset);
+}
+ErrorState &lld::CommonLinkingContext::e() {
+  return *(ErrorState *)((char *)storage + errorOffset);
+}
+TimerState &lld::CommonLinkingContext::t() {
+  return *(TimerState *)((char *)storage + timerOffset);
+}
+
+// Reference to the current LLD instance. Each thread must be associated to a
+// valid LLD instance, through `attachContext()`.
+static LLVM_THREAD_LOCAL CommonLinkingContext *lctx;
+
+unsigned lld::CommonLinkingContext::id = 0;
+
+lld::CommonLinkingContext::CommonLinkingContext() {
+  char *start = (char *)storage;
+  char *next = (char *)storage;
+#ifndef NDEBUG
+  char *end = next + sizeof(storage);
+#endif
+  initMemoryState(start, next, memoryOffset, bAlloc);
+  assert(next <= end);
+  initTargetOpt(start, next, targetOptOffset);
+  assert(next <= end);
+  initError(start, next, errorOffset);
+  assert(next <= end);
+  initTimer(start, next, timerOffset);
+  assert(next <= end);
+
+  valid = &id;
+
+  // The calling thread shall have a reference to the global state.
+  lctx = this;
+}
+
+lld::CommonLinkingContext::~CommonLinkingContext() {
+  if (valid != &id)
+    return;
+  assert(lctx && "The current thread must reference a global state!");
+
+  // Release states in reverse order.
+  freeTimer(&lctx->t());
+  freeError(&lctx->e());
+  freeTargetOpt(&lctx->to());
+  freeMemory(&lctx->m());
+  valid = nullptr;
+
+  // Detach the current thread from the global state.
+  lctx = nullptr;
+}
+
+void lld::CommonLinkingContext::shutdown() { delete lctx; }
+
+bool lld::CommonLinkingContext::isValid() { return valid == &id; }
+
+void lld::attachContext(CommonLinkingContext &ctx) {
+  assert(ctx.isValid());
+  lctx = &ctx;
+}
+
+void lld::detachContext() { lctx = nullptr; }
+
+CommonLinkingContext &lld::commonContext() {
+  assert(lctx && lctx->isValid());
+  return *lctx;
+}
+
+bool lld::hasContext() { return lctx != nullptr; }

lld/Common/Memory.cpp

 //===- Memory.cpp ---------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "lld/Common/Memory.h"
+#include "lld/Common/Globals.h"
 
 using namespace llvm;
 using namespace lld;
 
-BumpPtrAllocator lld::bAlloc;
-StringSaver lld::saver{bAlloc};
-std::vector<SpecificAllocBase *> lld::SpecificAllocBase::instances;
-
-void lld::freeArena() {
-  for (SpecificAllocBase *alloc : SpecificAllocBase::instances)
-    alloc->reset();
-  bAlloc.Reset();
+namespace lld {
+struct MemoryState {
+  MemoryState(llvm::BumpPtrAllocator &bAlloc) : saver(bAlloc) {}
+  llvm::StringSaver saver;
+  llvm::DenseMap<void *, SpecificAllocBase *> instances;
+};
+MemoryState *initMemoryState(char *const start, char *&next, unsigned &offset,
+                             llvm::BumpPtrAllocator &bAlloc) {
+  return initContextHelper<MemoryState>(start, next, offset, bAlloc);
+}
+
+void freeMemory(MemoryState *m) {
+  for (auto &instance : m->instances)
+    instance.second->reset();
+  m->~MemoryState();
+}
+} // namespace lld
+
+llvm::StringSaver &lld::saver() { return context().m().saver; }
+
+llvm::BumpPtrAllocator &lld::bAlloc() { return context().bAlloc; }
+
+SpecificAllocBase *
+lld::SpecificAllocBase::getOrCreate(void *tag, size_t size, size_t align,
+                                    SpecificAllocBase *(&creator)(void *)) {
+  auto &instances = context().m().instances;
+  auto &instance = instances[tag];
+  if (instance == nullptr) {
+    void *storage = context().bAlloc.Allocate(size, align);
+    instance = creator(storage);
+  }
+  return instance;
 }

lld/Common/TargetOptionsCommandFlags.cpp

 //===-- TargetOptionsCommandFlags.cpp ---------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "lld/Common/TargetOptionsCommandFlags.h"
-
+#include "lld/Common/Globals.h"
 #include "llvm/CodeGen/CommandFlags.h"
 #include "llvm/Target/TargetOptions.h"
 
-static llvm::codegen::RegisterCodeGenFlags CGF;
+using namespace lld;
+
+namespace lld {
+struct TargetOptState {
+  llvm::codegen::RegisterCodeGenFlags cgf;
+};
+TargetOptState *initTargetOpt(char *const start, char *&next,
+                              unsigned &offset) {
+  return initContextHelper<TargetOptState>(start, next, offset);
+}
+void freeTargetOpt(TargetOptState *to) { to->~TargetOptState(); }
+} // namespace lld
 
 llvm::TargetOptions lld::initTargetOptionsFromCodeGenFlags() {
   return llvm::codegen::InitTargetOptionsFromCodeGenFlags(llvm::Triple());
 }
 
 llvm::Optional<llvm::Reloc::Model> lld::getRelocModelFromCMModel() {
   return llvm::codegen::getExplicitRelocModel();
 }
 
 llvm::Optional<llvm::CodeModel::Model> lld::getCodeModelFromCMModel() {
   return llvm::codegen::getExplicitCodeModel();
 }
 
 std::string lld::getCPUStr() { return llvm::codegen::getCPUStr(); }
 
 std::vector<std::string> lld::getMAttrs() { return llvm::codegen::getMAttrs(); }

lld/Common/Timer.cpp

 //===- Timer.cpp ----------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "lld/Common/Timer.h"
 #include "lld/Common/ErrorHandler.h"
 #include "llvm/Support/Format.h"
 
 using namespace lld;
 using namespace llvm;
 
+namespace lld {
+struct TimerState {
+  Timer rootTimer{"Total Link Time"};
+};
+TimerState *initTimer(char *const start, char *&next, unsigned &offset) {
+  return initContextHelper<TimerState>(start, next, offset);
+}
+void freeTimer(TimerState *t) { t->~TimerState(); }
+} // namespace lld
+
 ScopedTimer::ScopedTimer(Timer &t) : t(&t) {
   startTime = std::chrono::high_resolution_clock::now();
 }
 
 void ScopedTimer::stop() {
   if (!t)
     return;
   t->addToTotal(std::chrono::high_resolution_clock::now() - startTime);
   t = nullptr;
 }
 
 ScopedTimer::~ScopedTimer() { stop(); }
 
 Timer::Timer(llvm::StringRef name) : name(std::string(name)) {}
 Timer::Timer(llvm::StringRef name, Timer &parent) : name(std::string(name)) {
   parent.children.push_back(this);
 }
 
 Timer &Timer::root() {
   static Timer rootTimer("Total Link Time");
   return rootTimer;
+  // FIXME: Can't use the "commonGlobals" root timer just yet since globals in
+  // other LLD libs are using this root timer, but context() isn't
+  // available before main(), by design.
+  // return context().t().rootTimer;
 }
 
 void Timer::print() {
-  double totalDuration = static_cast<double>(root().millis());
+  double totalDuration = static_cast<double>(millis());
 
   // We want to print the grand total under all the intermediate phases, so we
   // print all children first, then print the total under that.
   for (const auto &child : children)
     if (child->total > 0)
       child->print(1, totalDuration);
 
   message(std::string(50, '-'));
 
-  root().print(0, root().millis(), false);
+  print(0, millis(), false);
 }
 
 double Timer::millis() const {
   return std::chrono::duration_cast<std::chrono::duration<double, std::milli>>(
              std::chrono::nanoseconds(total))
       .count();
 }
 

lld/ELF/AArch64ErrataFix.cpp

 };
 
 Patch843419Section::Patch843419Section(InputSection *p, uint64_t off)
     : SyntheticSection(SHF_ALLOC | SHF_EXECINSTR, SHT_PROGBITS, 4,
                        ".text.patch"),
       patchee(p), patcheeOffset(off) {
   this->parent = p->getParent();
   patchSym = addSyntheticLocal(
-      saver.save("__CortexA53843419_" + utohexstr(getLDSTAddr())), STT_FUNC, 0,
-      getSize(), *this);
-  addSyntheticLocal(saver.save("$x"), STT_NOTYPE, 0, 0, *this);
+      saver().save("__CortexA53843419_" + utohexstr(getLDSTAddr())), STT_FUNC,
+      0, getSize(), *this);
+  addSyntheticLocal(saver().save("$x"), STT_NOTYPE, 0, 0, *this);
 }
 
 uint64_t Patch843419Section::getLDSTAddr() const {
   return patchee->getVA(patcheeOffset);
 }
 
 void Patch843419Section::writeTo(uint8_t *buf) {
   // Copy the instruction that we will be replacing with a branch in the

lld/ELF/ARMErrataFix.cpp

 
 Patch657417Section::Patch657417Section(InputSection *p, uint64_t off,
                                        uint32_t instr, bool isARM)
     : SyntheticSection(SHF_ALLOC | SHF_EXECINSTR, SHT_PROGBITS, 4,
                        ".text.patch"),
       patchee(p), patcheeOffset(off), instr(instr), isARM(isARM) {
   parent = p->getParent();
   patchSym = addSyntheticLocal(
-      saver.save("__CortexA8657417_" + utohexstr(getBranchAddr())), STT_FUNC,
+      saver().save("__CortexA8657417_" + utohexstr(getBranchAddr())), STT_FUNC,
       isARM ? 0 : 1, getSize(), *this);
-  addSyntheticLocal(saver.save(isARM ? "$a" : "$t"), STT_NOTYPE, 0, 0, *this);
+  addSyntheticLocal(saver().save(isARM ? "$a" : "$t"), STT_NOTYPE, 0, 0, *this);
 }
 
 uint64_t Patch657417Section::getBranchAddr() const {
   return patchee->getVA(patcheeOffset);
 }
 
 // Given a branch instruction instr at sourceAddr work out its destination
 // address. This is only used when the branch instruction has no relocation.

lld/ELF/Arch/PPC64.cpp

void elf::writePrefixedInstruction(uint8_t *loc, uint64_t insn) {
   write64(loc, insn);
 }
 
 static bool addOptional(StringRef name, uint64_t value,
                         std::vector<Defined *> &defined) {
   Symbol *sym = symtab->find(name);
   if (!sym || sym->isDefined())
     return false;
-  sym->resolve(Defined{/*file=*/nullptr, saver.save(name), STB_GLOBAL,
+  sym->resolve(Defined{/*file=*/nullptr, saver().save(name), STB_GLOBAL,
                        STV_HIDDEN, STT_FUNC, value,
                        /*size=*/0, /*section=*/nullptr});
   defined.push_back(cast<Defined>(sym));
   return true;
 }
 
 // If from is 14, write ${prefix}14: firstInsn; ${prefix}15:
 // firstInsn+0x200008; ...; ${prefix}31: firstInsn+(31-14)*0x200008; $tail

lld/ELF/Driver.cpp

 using namespace lld::elf;
 
 Configuration *elf::config;
 LinkerDriver *elf::driver;
 
 static void setConfigs(opt::InputArgList &args);
 static void readConfigs(opt::InputArgList &args);
 
-bool elf::link(ArrayRef<const char *> args, bool canExitEarly,
-               raw_ostream &stdoutOS, raw_ostream &stderrOS) {
-  lld::stdoutOS = &stdoutOS;
-  lld::stderrOS = &stderrOS;
-
+bool elf::link(ArrayRef<const char *> args) {
   errorHandler().cleanupCallback = []() {
-    freeArena();
-
     inputSections.clear();
     outputSections.clear();
     archiveFiles.clear();
     binaryFiles.clear();
     bitcodeFiles.clear();
     lazyObjFiles.clear();
     objectFiles.clear();
     sharedFiles.clear();
     backwardReferences.clear();
 
     tar = nullptr;
     memset(&in, 0, sizeof(in));
 
     partitions = {Partition()};
 
     SharedFile::vernauxNum = 0;
   };
 
   errorHandler().logName = args::getFilenameWithoutExe(args[0]);
   errorHandler().errorLimitExceededMsg =
       "too many errors emitted, stopping now (use "
       "-error-limit=0 to see all errors)";
-  errorHandler().exitEarly = canExitEarly;
-  stderrOS.enable_colors(stderrOS.has_colors());
 
   config = make<Configuration>();
   driver = make<LinkerDriver>();
   script = make<LinkerScript>();
   symtab = make<SymbolTable>();
 
   partitions = {Partition()};
 
   config->progName = args[0];
 
   driver->linkerMain(args);
 
-  // Exit immediately if we don't need to return to the caller.
-  // This saves time because the overhead of calling destructors
-  // for all globally-allocated objects is not negligible.
-  if (canExitEarly)
-    exitLld(errorCount() ? 1 : 0);
-
-  bool ret = errorCount() == 0;
-  if (!canExitEarly)
-    errorHandler().reset();
-  return ret;
+  return errorCount() == 0;
 }
 
 // Parses a linker -m option.
 static std::tuple<ELFKind, uint16_t, uint8_t> parseEmulation(StringRef emul) {
   uint8_t osabi = 0;
   StringRef s = emul;
   if (s.endswith("_fbsd")) {
     s = s.drop_back(5);
for (opt::Arg *arg : args.filtered(OPT_z)) {
     else
       error(errPrefix + toString(pat.takeError()));
   }
 
   cl::ResetAllOptionOccurrences();
 
   // Parse LTO options.
   if (auto *arg = args.getLastArg(OPT_plugin_opt_mcpu_eq))
-    parseClangOption(saver.save("-mcpu=" + StringRef(arg->getValue())),
+    parseClangOption(saver().save("-mcpu=" + StringRef(arg->getValue())),
                      arg->getSpelling());
 
   for (opt::Arg *arg : args.filtered(OPT_plugin_opt_eq_minus))
     parseClangOption(std::string("-") + arg->getValue(), arg->getSpelling());
 
   // GCC collect2 passes -plugin-opt=path/to/lto-wrapper with an absolute or
   // relative path. Just ignore. If not ended with "lto-wrapper", consider it an
   // unsupported LLVMgold.so option and error.
for (auto *arg : args.filtered(OPT_wrap)) {
     StringRef name = arg->getValue();
     if (!seen.insert(name).second)
       continue;
 
     Symbol *sym = symtab->find(name);
     if (!sym)
       continue;
 
-    Symbol *real = addUnusedUndefined(saver.save("__real_" + name));
+    Symbol *real = addUnusedUndefined(saver().save("__real_" + name));
     Symbol *wrap =
-        addUnusedUndefined(saver.save("__wrap_" + name), sym->binding);
+        addUnusedUndefined(saver().save("__wrap_" + name), sym->binding);
     v.push_back({sym, real, wrap});
 
     // We want to tell LTO not to inline symbols to be overwritten
     // because LTO doesn't know the final symbol contents after renaming.
     real->canInline = false;
     sym->canInline = false;
 
     // Tell LTO not to eliminate these symbols.

lld/ELF/DriverUtils.cpp

 // `--plugin-opt <foo>` is converted to `--plugin-opt=<foo>`. This is a
 // bit hacky, but looks like it is still better than handling --plugin-opt
 // options by hand.
 static void concatLTOPluginOptions(SmallVectorImpl<const char *> &args) {
   SmallVector<const char *, 256> v;
   for (size_t i = 0, e = args.size(); i != e; ++i) {
     StringRef s = args[i];
     if ((s == "-plugin-opt" || s == "--plugin-opt") && i + 1 != e) {
-      v.push_back(saver.save(s + "=" + args[i + 1]).data());
+      v.push_back(saver().save(s + "=" + args[i + 1]).data());
       ++i;
     } else {
       v.push_back(args[i]);
     }
   }
   args = std::move(v);
 }
 
 // Parses a given list of options.
 opt::InputArgList ELFOptTable::parse(ArrayRef<const char *> argv) {
   // Make InputArgList from string vectors.
   unsigned missingIndex;
   unsigned missingCount;
   SmallVector<const char *, 256> vec(argv.data(), argv.data() + argv.size());
 
   // We need to get the quoting style for response files before parsing all
   // options so we parse here before and ignore all the options but
   // --rsp-quoting.
   opt::InputArgList args = this->ParseArgs(vec, missingIndex, missingCount);
 
   // Expand response files (arguments in the form of @<filename>)
   // and then parse the argument again.
-  cl::ExpandResponseFiles(saver, getQuotingStyle(args), vec);
+  cl::ExpandResponseFiles(saver(), getQuotingStyle(args), vec);
   concatLTOPluginOptions(vec);
   args = this->ParseArgs(vec, missingIndex, missingCount);
 
   handleColorDiagnostics(args);
   if (missingCount)
     error(Twine(args.getArgString(missingIndex)) + ": missing argument");
 
   for (opt::Arg *arg : args.filtered(OPT_UNKNOWN)) {

lld/ELF/InputFiles.cpp

 }
 
 Optional<MemoryBufferRef> elf::readFile(StringRef path) {
   llvm::TimeTraceScope timeScope("Load input files", path);
 
   // The --chroot option changes our virtual root directory.
   // This is useful when you are dealing with files created by --reproduce.
   if (!config->chroot.empty() && path.startswith("/"))
-    path = saver.save(config->chroot + path);
+    path = saver().save(config->chroot + path);
 
   log(path);
   config->dependencyFiles.insert(llvm::CachedHashString(path));
 
   auto mbOrErr = MemoryBuffer::getFile(path, /*IsText=*/false,
                                        /*RequiresNullTerminator=*/false);
   if (auto ec = mbOrErr.getError()) {
     error("cannot open " + path + ": " + ec.message());
if (sym.isUndefined()) {
         if (idx >= verneeds.size()) {
           error("corrupt input file: version need index " + Twine(idx) +
                 " for symbol " + name + " is out of bounds\n>>> defined in " +
                 toString(this));
           continue;
         }
         StringRef verName = this->stringTable.data() + verneeds[idx];
         versionedNameBuffer.clear();
-        name =
-            saver.save((name + "@" + verName).toStringRef(versionedNameBuffer));
+        name = saver().save(
+            (name + "@" + verName).toStringRef(versionedNameBuffer));
       }
       Symbol *s = symtab->addSymbol(
           Undefined{this, name, sym.getBinding(), sym.st_other, sym.getType()});
       s->exportDynamic = true;
       if (s->isUndefined() && !s->isWeak() &&
           config->unresolvedSymbolsInShlib != UnresolvedPolicy::Ignore)
         requiredSymbols.push_back(s);
       continue;
if (idx >= verdefs.size() || idx == VER_NDX_LOCAL) {
       continue;
     }
 
     StringRef verName =
         this->stringTable.data() +
         reinterpret_cast<const Elf_Verdef *>(verdefs[idx])->getAux()->vda_name;
     versionedNameBuffer.clear();
     name = (name + "@" + verName).toStringRef(versionedNameBuffer);
-    symtab->addSymbol(SharedSymbol{*this, saver.save(name), sym.getBinding(),
+    symtab->addSymbol(SharedSymbol{*this, saver().save(name), sym.getBinding(),
                                    sym.st_other, sym.getType(), sym.st_value,
                                    sym.st_size, alignment, idx});
   }
 }
 
 static ELFKind getBitcodeELFKind(const Triple &t) {
   if (t.isLittleEndian())
     return t.isArch64Bit() ? ELF64LEKind : ELF32LEKind;
if (config->thinLTOIndexOnly)
     path = replaceThinLTOSuffix(mb.getBufferIdentifier());
 
   // ThinLTO assumes that all MemoryBufferRefs given to it have a unique
   // name. If two archives define two members with the same name, this
   // causes a collision which result in only one of the objects being taken
   // into consideration at LTO time (which very likely causes undefined
   // symbols later in the link stage). So we append file offset to make
   // filename unique.
-  StringRef name =
-      archiveName.empty()
-          ? saver.save(path)
-          : saver.save(archiveName + "(" + path::filename(path) + " at " +
-                       utostr(offsetInArchive) + ")");
+  StringRef name = archiveName.empty()
+                       ? saver().save(path)
+                       : saver().save(archiveName + "(" + path::filename(path) +
+                                      " at " + utostr(offsetInArchive) + ")");
   MemoryBufferRef mbref(mb.getBuffer(), name);
 
   obj = CHECK(lto::InputFile::create(mbref), this);
 
   Triple t(obj->getTargetTriple());
   ekind = getBitcodeELFKind(t);
   emachine = getBitcodeMachineKind(mb.getBufferIdentifier(), t);
   osabi = getOsAbi(t);
static uint8_t mapVisibility(GlobalValue::VisibilityTypes gvVisibility) {
   }
   llvm_unreachable("unknown visibility");
 }
 
 template <class ELFT>
 static Symbol *createBitcodeSymbol(const std::vector<bool> &keptComdats,
                                    const lto::InputFile::Symbol &objSym,
                                    BitcodeFile &f) {
-  StringRef name = saver.save(objSym.getName());
+  StringRef name = saver().save(objSym.getName());
   uint8_t binding = objSym.isWeak() ? STB_WEAK : STB_GLOBAL;
   uint8_t type = objSym.isTLS() ? STT_TLS : STT_NOTYPE;
   uint8_t visibility = mapVisibility(objSym.getVisibility());
   bool canOmitFromDynSym = objSym.canBeOmittedFromSymbolTable();
 
   int c = objSym.getComdatIndex();
   if (objSym.isUndefined() || (c != -1 && !keptComdats[c])) {
     Undefined newSym(&f, name, binding, visibility, type);
void BinaryFile::parse() {
   // blob, we define _binary_foo_{start,end,size} symbols, so that
   // user programs can access blobs by name. Non-alphanumeric
   // characters in a filename are replaced with underscore.
   std::string s = "_binary_" + mb.getBufferIdentifier().str();
   for (size_t i = 0; i < s.size(); ++i)
     if (!isAlnum(s[i]))
       s[i] = '_';
 
+  llvm::StringSaver &saver = lld::saver();
+
   symtab->addSymbol(Defined{nullptr, saver.save(s + "_start"), STB_GLOBAL,
                             STV_DEFAULT, STT_OBJECT, 0, 0, section});
   symtab->addSymbol(Defined{nullptr, saver.save(s + "_end"), STB_GLOBAL,
                             STV_DEFAULT, STT_OBJECT, data.size(), 0, section});
   symtab->addSymbol(Defined{nullptr, saver.save(s + "_size"), STB_GLOBAL,
                             STV_DEFAULT, STT_OBJECT, data.size(), 0, nullptr});
 }
 
template <class ELFT> void LazyObjFile::parse() {
 
   // A lazy object file wraps either a bitcode file or an ELF file.
   if (isBitcode(this->mb)) {
     std::unique_ptr<lto::InputFile> obj =
         CHECK(lto::InputFile::create(this->mb), this);
     for (const lto::InputFile::Symbol &sym : obj->symbols()) {
       if (sym.isUndefined())
         continue;
-      symtab->addSymbol(LazyObject{*this, saver.save(sym.getName())});
+      symtab->addSymbol(LazyObject{*this, saver().save(sym.getName())});
     }
     return;
   }
 
   if (getELFKind(this->mb, archiveName) != config->ekind) {
     error("incompatible file: " + this->mb.getBufferIdentifier());
     return;
   }

lld/ELF/InputSection.cpp

 }
 
 void InputSectionBase::uncompress() const {
   size_t size = uncompressedSize;
   char *uncompressedBuf;
   {
     static std::mutex mu;
     std::lock_guard<std::mutex> lock(mu);
-    uncompressedBuf = bAlloc.Allocate<char>(size);
+    uncompressedBuf = bAlloc().Allocate<char>(size);
   }
 
   if (Error e = zlib::uncompress(toStringRef(rawData), uncompressedBuf, size))
     fatal(toString(this) +
           ": uncompress failed: " + llvm::toString(std::move(e)));
   rawData = makeArrayRef((uint8_t *)uncompressedBuf, size);
   uncompressedSize = -1;
 }
if (rawData.size() < 8) {
       return;
     }
 
     uncompressedSize = read64be(rawData.data());
     rawData = rawData.slice(8);
 
     // Restore the original section name.
     // (e.g. ".zdebug_info" -> ".debug_info")
-    name = saver.save("." + name.substr(2));
+    name = saver().save("." + name.substr(2));
     return;
   }
 
   assert(flags & SHF_COMPRESSED);
   flags &= ~(uint64_t)SHF_COMPRESSED;
 
   // New-style header
   if (rawData.size() < sizeof(typename ELFT::Chdr)) {

lld/ELF/LTO.cpp

std::vector<InputFile *> BitcodeCompiler::compile() {
   lto::NativeObjectCache cache;
   if (!config->thinLTOCacheDir.empty())
     cache = check(
         lto::localCache(config->thinLTOCacheDir,
                         [&](size_t task, std::unique_ptr<MemoryBuffer> mb) {
                           files[task] = std::move(mb);
                         }));
 
+  CommonLinkingContext &ctx = context();
+
   if (!bitcodeFiles.empty())
     checkError(ltoObj->run(
         [&](size_t task) {
           return std::make_unique<lto::NativeObjectStream>(
               std::make_unique<raw_svector_ostream>(buf[task]));
         },
-        cache));
+        cache, [&ctx] { attachContext(ctx); }));
 
   // Emit empty index files for non-indexed files but not in single-module mode.
   if (config->thinLTOModulesToCompile.empty()) {
     for (StringRef s : thinIndices) {
       std::string path = getThinLTOOutputFile(s);
       openFile(path + ".thinlto.bc");
       if (config->thinLTOEmitImportsFiles)
         openFile(path + ".imports");

lld/ELF/MarkLive.cpp

for (InputSectionBase *sec : inputSections) {
 
     if (isReserved(sec) || script->shouldKeep(sec)) {
       enqueue(sec, 0);
     } else if ((!config->zStartStopGC || sec->name.startswith("__libc_")) &&
                isValidCIdentifier(sec->name)) {
       // As a workaround for glibc libc.a before 2.34
       // (https://sourceware.org/PR27492), retain __libc_atexit and similar
       // sections regardless of zStartStopGC.
-      cNamedSections[saver.save("__start_" + sec->name)].push_back(sec);
-      cNamedSections[saver.save("__stop_" + sec->name)].push_back(sec);
+      cNamedSections[saver().save("__start_" + sec->name)].push_back(sec);
+      cNamedSections[saver().save("__stop_" + sec->name)].push_back(sec);
     }
   }
 
   mark();
 }
 
 template <class ELFT> void MarkLive<ELFT>::mark() {
   // Mark all reachable sections.

lld/ELF/ScriptParser.cpp

void ScriptParser::readDefsym(StringRef name) {
   script->sectionCommands.push_back(cmd);
 }
 
 void ScriptParser::addFile(StringRef s) {
   if (isUnderSysroot && s.startswith("/")) {
     SmallString<128> pathData;
     StringRef path = (config->sysroot + s).toStringRef(pathData);
     if (sys::fs::exists(path))
-      driver->addFile(saver.save(path), /*withLOption=*/false);
+      driver->addFile(saver().save(path), /*withLOption=*/false);
     else
       setError("cannot find " + s + " inside " + config->sysroot);
     return;
   }
 
   if (s.startswith("/")) {
     // Case 1: s is an absolute path. Just open it.
     driver->addFile(s, /*withLOption=*/false);
   } else if (s.startswith("=")) {
     // Case 2: relative to the sysroot.
     if (config->sysroot.empty())
       driver->addFile(s.substr(1), /*withLOption=*/false);
     else
-      driver->addFile(saver.save(config->sysroot + "/" + s.substr(1)),
+      driver->addFile(saver().save(config->sysroot + "/" + s.substr(1)),
                       /*withLOption=*/false);
   } else if (s.startswith("-l")) {
     // Case 3: search in the list of library paths.
     driver->addLibrary(s.substr(2));
   } else {
     // Case 4: s is a relative path. Search in the directory of the script file.
     std::string filename = std::string(getCurrentMB().getBufferIdentifier());
     StringRef directory = sys::path::parent_path(filename);
     if (!directory.empty()) {
       SmallString<0> path(directory);
       sys::path::append(path, s);
       if (sys::fs::exists(path)) {
         driver->addFile(path, /*withLOption=*/false);
         return;
       }
     }
     // Then search in the current working directory.
     if (sys::fs::exists(s)) {
       driver->addFile(s, /*withLOption=*/false);
     } else {
       // Finally, search in the list of library paths.
       if (Optional<std::string> path = findFromSearchPaths(s))
-        driver->addFile(saver.save(*path), /*withLOption=*/true);
+        driver->addFile(saver().save(*path), /*withLOption=*/true);
       else
         setError("unable to find " + s);
     }
   }
 }
 
 void ScriptParser::readAsNeeded() {
   expect("(");

lld/ELF/SyntheticSections.cpp

 
 // Returns an LLD version string.
 static ArrayRef<uint8_t> getVersion() {
   // Check LLD_VERSION first for ease of testing.
   // You can get consistent output by using the environment variable.
   // This is only for testing.
   StringRef s = getenv("LLD_VERSION");
   if (s.empty())
-    s = saver.save(Twine("Linker: ") + getLLDVersion());
+    s = saver().save(Twine("Linker: ") + getLLDVersion());
 
   // +1 to include the terminating '\0'.
   return {(const uint8_t *)s.data(), s.size() + 1};
 }
 
 // Creates a .comment section containing LLD version info.
 // With this feature, you can identify LLD-generated binaries easily
 // by "readelf --string-dump .comment <file>".
for (InputSectionBase *sec : sections) {
     sec->getFile<ELFT>()->mipsGp0 = r->ri_gp_value;
   };
 
   return make<MipsReginfoSection<ELFT>>(reginfo);
 }
 
 InputSection *elf::createInterpSection() {
   // StringSaver guarantees that the returned string ends with '\0'.
-  StringRef s = saver.save(config->dynamicLinker);
+  StringRef s = saver().save(config->dynamicLinker);
   ArrayRef<uint8_t> contents = {(const uint8_t *)s.data(), s.size() + 1};
 
   return make<InputSection>(nullptr, SHF_ALLOC, SHT_PROGBITS, 1, contents,
                             ".interp");
 }
 
 Defined *elf::addSyntheticLocal(StringRef name, uint8_t type, uint64_t value,
                                 uint64_t size, InputSectionBase &section) {

lld/ELF/Thunks.cpp

const uint8_t data[] = {
     0x00, 0x00, 0x00, 0x00,
   };
   uint64_t s = getAArch64ThunkDestVA(destination, addend);
   memcpy(buf, data, sizeof(data));
   target->relocateNoSym(buf + 8, R_AARCH64_ABS64, s);
 }
 
 void AArch64ABSLongThunk::addSymbols(ThunkSection &isec) {
-  addSymbol(saver.save("__AArch64AbsLongThunk_" + destination.getName()),
+  addSymbol(saver().save("__AArch64AbsLongThunk_" + destination.getName()),
             STT_FUNC, 0, isec);
   addSymbol("$x", STT_NOTYPE, 0, isec);
   addSymbol("$d", STT_NOTYPE, 8, isec);
 }
 
 // This Thunk has a maximum range of 4Gb, this is sufficient for all programs
 // using the small code model, including pc-relative ones. At time of writing
 // clang and gcc do not support the large code model for position independent
void AArch64ADRPThunk::writeTo(uint8_t *buf) {
   uint64_t p = getThunkTargetSym()->getVA();
   memcpy(buf, data, sizeof(data));
   target->relocateNoSym(buf, R_AARCH64_ADR_PREL_PG_HI21,
                         getAArch64Page(s) - getAArch64Page(p));
   target->relocateNoSym(buf + 4, R_AARCH64_ADD_ABS_LO12_NC, s);
 }
 
 void AArch64ADRPThunk::addSymbols(ThunkSection &isec) {
-  addSymbol(saver.save("__AArch64ADRPThunk_" + destination.getName()), STT_FUNC,
-            0, isec);
+  addSymbol(saver().save("__AArch64ADRPThunk_" + destination.getName()),
+            STT_FUNC, 0, isec);
   addSymbol("$x", STT_NOTYPE, 0, isec);
 }
 
 // ARM Target Thunks
 static uint64_t getARMThunkDestVA(const Symbol &s) {
   uint64_t v = s.isInPlt() ? s.getPltVA() : s.getVA();
   return SignExtend64<32>(v);
 }
void ARMV7ABSLongThunk::writeLong(uint8_t *buf) {
   };
   uint64_t s = getARMThunkDestVA(destination);
   memcpy(buf, data, sizeof(data));
   target->relocateNoSym(buf, R_ARM_MOVW_ABS_NC, s);
   target->relocateNoSym(buf + 4, R_ARM_MOVT_ABS, s);
 }
 
 void ARMV7ABSLongThunk::addSymbols(ThunkSection &isec) {
-  addSymbol(saver.save("__ARMv7ABSLongThunk_" + destination.getName()),
+  addSymbol(saver().save("__ARMv7ABSLongThunk_" + destination.getName()),
             STT_FUNC, 0, isec);
   addSymbol("$a", STT_NOTYPE, 0, isec);
 }
 
 void ThumbV7ABSLongThunk::writeLong(uint8_t *buf) {
   const uint8_t data[] = {
       0x40, 0xf2, 0x00, 0x0c, // movw         ip, :lower16:S
       0xc0, 0xf2, 0x00, 0x0c, // movt         ip, :upper16:S
       0x60, 0x47,             // bx   ip
   };
   uint64_t s = getARMThunkDestVA(destination);
   memcpy(buf, data, sizeof(data));
   target->relocateNoSym(buf, R_ARM_THM_MOVW_ABS_NC, s);
   target->relocateNoSym(buf + 4, R_ARM_THM_MOVT_ABS, s);
 }
 
 void ThumbV7ABSLongThunk::addSymbols(ThunkSection &isec) {
-  addSymbol(saver.save("__Thumbv7ABSLongThunk_" + destination.getName()),
+  addSymbol(saver().save("__Thumbv7ABSLongThunk_" + destination.getName()),
             STT_FUNC, 1, isec);
   addSymbol("$t", STT_NOTYPE, 0, isec);
 }
 
 void ARMV7PILongThunk::writeLong(uint8_t *buf) {
   const uint8_t data[] = {
       0xf0, 0xcf, 0x0f, 0xe3, // P:  movw ip,:lower16:S - (P + (L1-P) + 8)
       0x00, 0xc0, 0x40, 0xe3, //     movt ip,:upper16:S - (P + (L1-P) + 8)
       0x0f, 0xc0, 0x8c, 0xe0, // L1: add  ip, ip, pc
       0x1c, 0xff, 0x2f, 0xe1, //     bx   ip
   };
   uint64_t s = getARMThunkDestVA(destination);
   uint64_t p = getThunkTargetSym()->getVA();
   int64_t offset = s - p - 16;
   memcpy(buf, data, sizeof(data));
   target->relocateNoSym(buf, R_ARM_MOVW_PREL_NC, offset);
   target->relocateNoSym(buf + 4, R_ARM_MOVT_PREL, offset);
 }
 
 void ARMV7PILongThunk::addSymbols(ThunkSection &isec) {
-  addSymbol(saver.save("__ARMV7PILongThunk_" + destination.getName()), STT_FUNC,
-            0, isec);
+  addSymbol(saver().save("__ARMV7PILongThunk_" + destination.getName()),
+            STT_FUNC, 0, isec);
   addSymbol("$a", STT_NOTYPE, 0, isec);
 }
 
 void ThumbV7PILongThunk::writeLong(uint8_t *buf) {
   const uint8_t data[] = {
       0x4f, 0xf6, 0xf4, 0x7c, // P:  movw ip,:lower16:S - (P + (L1-P) + 4)
       0xc0, 0xf2, 0x00, 0x0c, //     movt ip,:upper16:S - (P + (L1-P) + 4)
       0xfc, 0x44,             // L1: add  ip, pc
       0x60, 0x47,             //     bx   ip
   };
   uint64_t s = getARMThunkDestVA(destination);
   uint64_t p = getThunkTargetSym()->getVA() & ~0x1;
   int64_t offset = s - p - 12;
   memcpy(buf, data, sizeof(data));
   target->relocateNoSym(buf, R_ARM_THM_MOVW_PREL_NC, offset);
   target->relocateNoSym(buf + 4, R_ARM_THM_MOVT_PREL, offset);
 }
 
 void ThumbV7PILongThunk::addSymbols(ThunkSection &isec) {
-  addSymbol(saver.save("__ThumbV7PILongThunk_" + destination.getName()),
+  addSymbol(saver().save("__ThumbV7PILongThunk_" + destination.getName()),
             STT_FUNC, 1, isec);
   addSymbol("$t", STT_NOTYPE, 0, isec);
 }
 
 void ARMV5ABSLongThunk::writeLong(uint8_t *buf) {
   const uint8_t data[] = {
       0x04, 0xf0, 0x1f, 0xe5, //     ldr pc, [pc,#-4] ; L1
       0x00, 0x00, 0x00, 0x00, // L1: .word S
   };
   memcpy(buf, data, sizeof(data));
   target->relocateNoSym(buf + 4, R_ARM_ABS32, getARMThunkDestVA(destination));
 }
 
 void ARMV5ABSLongThunk::addSymbols(ThunkSection &isec) {
-  addSymbol(saver.save("__ARMv5ABSLongThunk_" + destination.getName()),
+  addSymbol(saver().save("__ARMv5ABSLongThunk_" + destination.getName()),
             STT_FUNC, 0, isec);
   addSymbol("$a", STT_NOTYPE, 0, isec);
   addSymbol("$d", STT_NOTYPE, 4, isec);
 }
 
 bool ARMV5ABSLongThunk::isCompatibleWith(const InputSection &isec,
                                          const Relocation &rel) const {
   // Thumb branch relocations can't use BLX
void ARMV5PILongThunk::writeLong(uint8_t *buf) {
   };
   uint64_t s = getARMThunkDestVA(destination);
   uint64_t p = getThunkTargetSym()->getVA() & ~0x1;
   memcpy(buf, data, sizeof(data));
   target->relocateNoSym(buf + 12, R_ARM_REL32, s - p - 12);
 }
 
 void ARMV5PILongThunk::addSymbols(ThunkSection &isec) {
-  addSymbol(saver.save("__ARMV5PILongThunk_" + destination.getName()), STT_FUNC,
-            0, isec);
+  addSymbol(saver().save("__ARMV5PILongThunk_" + destination.getName()),
+            STT_FUNC, 0, isec);
   addSymbol("$a", STT_NOTYPE, 0, isec);
   addSymbol("$d", STT_NOTYPE, 12, isec);
 }
 
 bool ARMV5PILongThunk::isCompatibleWith(const InputSection &isec,
                                         const Relocation &rel) const {
   // Thumb branch relocations can't use BLX
   return rel.type != R_ARM_THM_JUMP19 && rel.type != R_ARM_THM_JUMP24;
const uint8_t data[] = {
       0x00, 0x00, 0x00, 0x00 // L1: .word S
   };
   uint64_t s = getARMThunkDestVA(destination);
   memcpy(buf, data, sizeof(data));
   target->relocateNoSym(buf + 8, R_ARM_ABS32, s);
 }
 
 void ThumbV6MABSLongThunk::addSymbols(ThunkSection &isec) {
-  addSymbol(saver.save("__Thumbv6MABSLongThunk_" + destination.getName()),
+  addSymbol(saver().save("__Thumbv6MABSLongThunk_" + destination.getName()),
             STT_FUNC, 1, isec);
   addSymbol("$t", STT_NOTYPE, 0, isec);
   addSymbol("$d", STT_NOTYPE, 8, isec);
 }
 
 void ThumbV6MPILongThunk::writeLong(uint8_t *buf) {
   // Most Thumb instructions cannot access the high registers r8 - r15. As the
   // only register we can corrupt is ip (r12) we must instead spill a low
void ThumbV6MPILongThunk::writeLong(uint8_t *buf) {
   };
   uint64_t s = getARMThunkDestVA(destination);
   uint64_t p = getThunkTargetSym()->getVA() & ~0x1;
   memcpy(buf, data, sizeof(data));
   target->relocateNoSym(buf + 12, R_ARM_REL32, s - p - 12);
 }
 
 void ThumbV6MPILongThunk::addSymbols(ThunkSection &isec) {
-  addSymbol(saver.save("__Thumbv6MPILongThunk_" + destination.getName()),
+  addSymbol(saver().save("__Thumbv6MPILongThunk_" + destination.getName()),
             STT_FUNC, 1, isec);
   addSymbol("$t", STT_NOTYPE, 0, isec);
   addSymbol("$d", STT_NOTYPE, 12, isec);
 }
 
 // Write MIPS LA25 thunk code to call PIC function from the non-PIC one.
 void MipsThunk::writeTo(uint8_t *buf) {
   uint64_t s = destination.getVA();
   write32(buf, 0x3c190000); // lui   $25, %hi(func)
   write32(buf + 4, 0x08000000 | (s >> 2)); // j     func
   write32(buf + 8, 0x27390000); // addiu $25, $25, %lo(func)
   write32(buf + 12, 0x00000000); // nop
   target->relocateNoSym(buf, R_MIPS_HI16, s);
   target->relocateNoSym(buf + 8, R_MIPS_LO16, s);
 }
 
 void MipsThunk::addSymbols(ThunkSection &isec) {
-  addSymbol(saver.save("__LA25Thunk_" + destination.getName()), STT_FUNC, 0,
+  addSymbol(saver().save("__LA25Thunk_" + destination.getName()), STT_FUNC, 0,
             isec);
 }
 
 InputSection *MipsThunk::getTargetInputSection() const {
   auto &dr = cast<Defined>(destination);
   return dyn_cast<InputSection>(dr.section);
 }
 
 // Write microMIPS R2-R5 LA25 thunk code
 // to call PIC function from the non-PIC one.
 void MicroMipsThunk::writeTo(uint8_t *buf) {
   uint64_t s = destination.getVA();
   write16(buf, 0x41b9);       // lui   $25, %hi(func)
   write16(buf + 4, 0xd400);   // j     func
   write16(buf + 8, 0x3339);   // addiu $25, $25, %lo(func)
   write16(buf + 12, 0x0c00);  // nop
   target->relocateNoSym(buf, R_MICROMIPS_HI16, s);
   target->relocateNoSym(buf + 4, R_MICROMIPS_26_S1, s);
   target->relocateNoSym(buf + 8, R_MICROMIPS_LO16, s);
 }
 
 void MicroMipsThunk::addSymbols(ThunkSection &isec) {
-  Defined *d = addSymbol(
-      saver.save("__microLA25Thunk_" + destination.getName()), STT_FUNC, 0, isec);
+  Defined *d =
+      addSymbol(saver().save("__microLA25Thunk_" + destination.getName()),
+                STT_FUNC, 0, isec);
   d->stOther |= STO_MIPS_MICROMIPS;
 }
 
 InputSection *MicroMipsThunk::getTargetInputSection() const {
   auto &dr = cast<Defined>(destination);
   return dyn_cast<InputSection>(dr.section);
 }
 
 // Write microMIPS R6 LA25 thunk code
 // to call PIC function from the non-PIC one.
 void MicroMipsR6Thunk::writeTo(uint8_t *buf) {
   uint64_t s = destination.getVA();
   uint64_t p = getThunkTargetSym()->getVA();
   write16(buf, 0x1320);       // lui   $25, %hi(func)
   write16(buf + 4, 0x3339);   // addiu $25, $25, %lo(func)
   write16(buf + 8, 0x9400);   // bc    func
   target->relocateNoSym(buf, R_MICROMIPS_HI16, s);
   target->relocateNoSym(buf + 4, R_MICROMIPS_LO16, s);
   target->relocateNoSym(buf + 8, R_MICROMIPS_PC26_S1, s - p - 12);
 }
 
 void MicroMipsR6Thunk::addSymbols(ThunkSection &isec) {
-  Defined *d = addSymbol(
-      saver.save("__microLA25Thunk_" + destination.getName()), STT_FUNC, 0, isec);
+  Defined *d =
+      addSymbol(saver().save("__microLA25Thunk_" + destination.getName()),
+                STT_FUNC, 0, isec);
   d->stOther |= STO_MIPS_MICROMIPS;
 }
 
 InputSection *MicroMipsR6Thunk::getTargetInputSection() const {
   auto &dr = cast<Defined>(destination);
   return dyn_cast<InputSection>(dr.section);
 }
 
void PPC32PltCallStub::addSymbols(ThunkSection &isec) {
   os << format_hex_no_prefix(addend, 8);
   if (!config->isPic)
     os << ".plt_call32.";
   else if (addend >= 0x8000)
     os << ".got2.plt_pic32.";
   else
     os << ".plt_pic32.";
   os << destination.getName();
-  addSymbol(saver.save(os.str()), STT_FUNC, 0, isec);
+  addSymbol(saver().save(os.str()), STT_FUNC, 0, isec);
 }
 
 bool PPC32PltCallStub::isCompatibleWith(const InputSection &isec,
                                         const Relocation &rel) const {
   return !config->isPic || (isec.file == file && rel.addend == addend);
 }
 
 void PPC32LongThunk::addSymbols(ThunkSection &isec) {
-  addSymbol(saver.save("__LongThunk_" + destination.getName()), STT_FUNC, 0,
+  addSymbol(saver().save("__LongThunk_" + destination.getName()), STT_FUNC, 0,
             isec);
 }
 
 void PPC32LongThunk::writeTo(uint8_t *buf) {
   auto ha = [](uint32_t v) -> uint16_t { return (v + 0x8000) >> 16; };
   auto lo = [](uint32_t v) -> uint16_t { return v; };
   uint32_t d = destination.getVA(addend);
   if (config->isPic) {

 void PPC64PltCallStub::writeTo(uint8_t *buf) {
   int64_t offset = destination.getGotPltVA() - getPPC64TocBase();
   // Save the TOC pointer to the save-slot reserved in the call frame.
   write32(buf + 0, 0xf8410018); // std     r2,24(r1)
   writePPC64LoadAndBranch(buf + 4, offset);
 }
 
 void PPC64PltCallStub::addSymbols(ThunkSection &isec) {
-  Defined *s = addSymbol(saver.save("__plt_" + destination.getName()), STT_FUNC,
-                         0, isec);
+  Defined *s = addSymbol(saver().save("__plt_" + destination.getName()),
+                         STT_FUNC, 0, isec);
   s->needsTocRestore = true;
   s->file = destination.file;
 }
 
 bool PPC64PltCallStub::isCompatibleWith(const InputSection &isec,
                                         const Relocation &rel) const {
   return rel.type == R_PPC64_REL24 || rel.type == R_PPC64_REL14;
 }
if (getMayUseShortThunk()) {
     const int64_t offsetFromTOC =
         in.ppc64LongBranchTarget->getEntryVA(&destination, addend) -
         getPPC64TocBase();
     writePPC64LoadAndBranch(buf + 4, offsetFromTOC);
   }
 }
 
 void PPC64R2SaveStub::addSymbols(ThunkSection &isec) {
-  Defined *s = addSymbol(saver.save("__toc_save_" + destination.getName()),
+  Defined *s = addSymbol(saver().save("__toc_save_" + destination.getName()),
                          STT_FUNC, 0, isec);
   s->needsTocRestore = true;
 }
 
 bool PPC64R2SaveStub::isCompatibleWith(const InputSection &isec,
                                        const Relocation &rel) const {
   return rel.type == R_PPC64_REL24 || rel.type == R_PPC64_REL14;
 }
if (!config->Power10Stub) {
     writePrefixedInstruction(buf + 0, paddi); // paddi r12, 0, func@pcrel, 1
     nextInstOffset = 8;
   }
   write32(buf + nextInstOffset, MTCTR_R12); // mtctr r12
   write32(buf + nextInstOffset + 4, BCTR);  // bctr
 }
 
 void PPC64R12SetupStub::addSymbols(ThunkSection &isec) {
-  addSymbol(saver.save("__gep_setup_" + destination.getName()), STT_FUNC, 0,
+  addSymbol(saver().save("__gep_setup_" + destination.getName()), STT_FUNC, 0,
             isec);
 }
 
 bool PPC64R12SetupStub::isCompatibleWith(const InputSection &isec,
                                          const Relocation &rel) const {
   return rel.type == R_PPC64_REL24_NOTOC;
 }
 
if (config->Power10Stub) {
     write32(buf + 20, 0x398c0000 | (off & 0xffff)); // addi r12,r12,off@l
     nextInstOffset = 24;
   }
   write32(buf + nextInstOffset, MTCTR_R12); // mtctr r12
   write32(buf + nextInstOffset + 4, BCTR);  // bctr
 }
 
 void PPC64PCRelPLTStub::addSymbols(ThunkSection &isec) {
-  addSymbol(saver.save("__plt_pcrel_" + destination.getName()), STT_FUNC, 0,
+  addSymbol(saver().save("__plt_pcrel_" + destination.getName()), STT_FUNC, 0,
             isec);
 }
 
 bool PPC64PCRelPLTStub::isCompatibleWith(const InputSection &isec,
                                          const Relocation &rel) const {
   return rel.type == R_PPC64_REL24_NOTOC;
 }
 
 void PPC64LongBranchThunk::writeTo(uint8_t *buf) {
   int64_t offset = in.ppc64LongBranchTarget->getEntryVA(&destination, addend) -
                    getPPC64TocBase();
   writePPC64LoadAndBranch(buf, offset);
 }
 
 void PPC64LongBranchThunk::addSymbols(ThunkSection &isec) {
-  addSymbol(saver.save("__long_branch_" + destination.getName()), STT_FUNC, 0,
+  addSymbol(saver().save("__long_branch_" + destination.getName()), STT_FUNC, 0,
             isec);
 }
 
 bool PPC64LongBranchThunk::isCompatibleWith(const InputSection &isec,
                                             const Relocation &rel) const {
   return rel.type == R_PPC64_REL24 || rel.type == R_PPC64_REL14;
 }
 
if (!config->Power10Stub) {
     writePrefixedInstruction(buf + 0, paddi); // paddi r12, 0, func@pcrel, 1
     nextInstOffset = 8;
   }
   write32(buf + nextInstOffset, MTCTR_R12); // mtctr r12
   write32(buf + nextInstOffset + 4, BCTR);  // bctr
 }
 
 void PPC64PCRelLongBranchThunk::addSymbols(ThunkSection &isec) {
-  addSymbol(saver.save("__long_branch_pcrel_" + destination.getName()),
+  addSymbol(saver().save("__long_branch_pcrel_" + destination.getName()),
             STT_FUNC, 0, isec);
 }
 
 bool PPC64PCRelLongBranchThunk::isCompatibleWith(const InputSection &isec,
                                                  const Relocation &rel) const {
   return rel.type == R_PPC64_REL24_NOTOC;
 }
 

lld/ELF/Writer.cpp

StringRef elf::getOutputSectionName(const InputSectionBase *s) {
 
   // This is for --emit-relocs. If .text.foo is emitted as .text.bar, we want
   // to emit .rela.text.foo as .rela.text.bar for consistency (this is not
   // technically required, but not doing it is odd). This code guarantees that.
   if (auto *isec = dyn_cast<InputSection>(s)) {
     if (InputSectionBase *rel = isec->getRelocatedSection()) {
       OutputSection *out = rel->getOutputSection();
       if (s->type == SHT_RELA)
-        return saver.save(".rela" + out->name);
-      return saver.save(".rel" + out->name);
+        return saver().save(".rela" + out->name);
+      return saver().save(".rel" + out->name);
     }
   }
 
   // A BssSection created for a common symbol is identified as "COMMON" in
   // linker scripts. It should go to .bss section.
   if (s->name == "COMMON")
     return ".bss";
 

 // __stop_<secname> symbols. They are at beginning and end of the section,
 // respectively. This is not requested by the ELF standard, but GNU ld and
 // gold provide the feature, and used by many programs.
 template <class ELFT>
 void Writer<ELFT>::addStartStopSymbols(OutputSection *sec) {
   StringRef s = sec->name;
   if (!isValidCIdentifier(s))
     return;
-  addOptionalRegular(saver.save("__start_" + s), sec, 0,
+  addOptionalRegular(saver().save("__start_" + s), sec, 0,
                      config->zStartStopVisibility);
-  addOptionalRegular(saver.save("__stop_" + s), sec, -1,
+  addOptionalRegular(saver().save("__stop_" + s), sec, -1,
                      config->zStartStopVisibility);
 }
 
 static bool needsPtLoad(OutputSection *sec) {
   if (!(sec->flags & SHF_ALLOC))
     return false;
 
   // Don't allocate VA space for TLS NOBITS sections. The PT_TLS PHDR is

lld/MachO/ConcatOutputSection.cpp

for (Reloc &r : reverse(relocs)) {
           make<ConcatInputSection>(isec->getSegName(), isec->getName());
       thunkInfo.isec->parent = this;
 
       // This code runs after dead code removal. Need to set the `live` bit
       // on the thunk isec so that asserts that check that only live sections
       // get written are happy.
       thunkInfo.isec->live = true;
 
-      StringRef thunkName = saver.save(funcSym->getName() + ".thunk." +
+      StringRef thunkName = saver().save(funcSym->getName() + ".thunk." +
                                          std::to_string(thunkInfo.sequence++));
       r.referent = thunkInfo.sym = symtab->addDefined(
           thunkName, /*file=*/nullptr, thunkInfo.isec, /*value=*/0,
           /*size=*/thunkSize, /*isWeakDef=*/false, /*isPrivateExtern=*/true,
           /*isThumb=*/false, /*isReferencedDynamically=*/false,
           /*noDeadStrip=*/false);
       target->populateThunk(thunkInfo.isec, funcSym);
       finalizeOne(thunkInfo.isec);
       thunks.push_back(thunkInfo.isec);

lld/MachO/Driver.cpp

for (StringRef path : args::getStrings(args, optionCode)) {
     // NOTE: only absolute paths are re-rooted to syslibroot(s)
     bool found = false;
     if (path::is_absolute(path, path::Style::posix)) {
       for (StringRef root : roots) {
         SmallString<261> buffer(root);
         path::append(buffer, path);
         // Do not warn about paths that are computed via the syslib roots
         if (fs::is_directory(buffer)) {
-          paths.push_back(saver.save(buffer.str()));
+          paths.push_back(saver().save(buffer.str()));
           found = true;
         }
       }
     }
     if (!found && warnIfNotDirectory(optionLetter, path))
       paths.push_back(path);
   }
 
   // `-Z` suppresses the standard "system" search paths.
   if (args.hasArg(OPT_Z))
     return paths;
 
   for (const StringRef &path : systemPaths) {
     for (const StringRef &root : roots) {
       SmallString<261> buffer(root);
       path::append(buffer, path);
       if (fs::is_directory(buffer))
-        paths.push_back(saver.save(buffer.str()));
+        paths.push_back(saver().save(buffer.str()));
     }
   }
   return paths;
 }
 
 static std::vector<StringRef> getSystemLibraryRoots(InputArgList &args) {
   std::vector<StringRef> roots;
   for (const Arg *arg : args.filtered(OPT_syslibroot))
static void referenceStubBinder() {
   // dyld_stub_binder is in libSystem.dylib, which is usually linked in. This
   // isn't needed for correctness, but the presence of that symbol suppresses
   // "no symbols" diagnostics from `nm`.
   // StubHelperSection::setup() adds a reference and errors out if
   // dyld_stub_binder doesn't exist in case it is actually needed.
   symtab->addUndefined("dyld_stub_binder", /*file=*/nullptr, /*isWeak=*/false);
 }
 
-bool macho::link(ArrayRef<const char *> argsArr, bool canExitEarly,
-                 raw_ostream &stdoutOS, raw_ostream &stderrOS) {
-  lld::stdoutOS = &stdoutOS;
-  lld::stderrOS = &stderrOS;
-
-  errorHandler().cleanupCallback = []() { freeArena(); };
-
+bool macho::link(ArrayRef<const char *> argsArr) {
   errorHandler().logName = args::getFilenameWithoutExe(argsArr[0]);
-  stderrOS.enable_colors(stderrOS.has_colors());
 
   MachOOptTable parser;
   InputArgList args = parser.parse(argsArr.slice(1));
 
   errorHandler().errorLimitExceededMsg =
       "too many errors emitted, stopping now "
       "(use --error-limit=0 to see all errors)";
   errorHandler().errorLimit = args::getInteger(args, OPT_error_limit_eq, 20);
// re-exported.
 
       extensions.push_back(".dylib");
       for (const Arg *arg : args.filtered(OPT_sub_library))
         reexportHandler(arg, extensions);
     }
 
     // Parse LTO options.
     if (const Arg *arg = args.getLastArg(OPT_mcpu))
-      parseClangOption(saver.save("-mcpu=" + StringRef(arg->getValue())),
+      parseClangOption(saver().save("-mcpu=" + StringRef(arg->getValue())),
                        arg->getSpelling());
 
     for (const Arg *arg : args.filtered(OPT_mllvm))
       parseClangOption(arg->getValue(), arg->getSpelling());
 
     compileBitcodeFiles();
     replaceCommonSymbols();
 
if (auto E = timeTraceProfilerWrite(
             args.getLastArgValue(OPT_time_trace_file_eq).str(),
             config->outputFile)) {
       handleAllErrors(std::move(E),
                       [&](const StringError &SE) { error(SE.getMessage()); });
     }
 
     timeTraceProfilerCleanup();
   }
-
-  if (canExitEarly)
-    exitLld(errorCount() ? 1 : 0);
-
-  return !errorCount();
+  return errorCount() == 0;
 }

lld/MachO/DriverUtils.cpp

 InputArgList MachOOptTable::parse(ArrayRef<const char *> argv) {
   // Make InputArgList from string vectors.
   unsigned missingIndex;
   unsigned missingCount;
   SmallVector<const char *, 256> vec(argv.data(), argv.data() + argv.size());
 
   // Expand response files (arguments in the form of @<filename>)
   // and then parse the argument again.
-  cl::ExpandResponseFiles(saver, cl::TokenizeGNUCommandLine, vec);
+  cl::ExpandResponseFiles(saver(), cl::TokenizeGNUCommandLine, vec);
   InputArgList args = ParseArgs(vec, missingIndex, missingCount);
 
   // Handle -fatal_warnings early since it converts missing argument warnings
   // to errors.
   errorHandler().fatalWarnings = args.hasArg(OPT_fatal_warnings);
 
   if (missingCount)
     error(Twine(args.getArgString(missingIndex)) + ": missing argument");
macho::findPathCombination(const Twine &name,
   for (StringRef dir : roots) {
     base = dir;
     path::append(base, name);
     for (StringRef ext : extensions) {
       Twine location = base + ext;
       bool exists = fs::exists(location);
       searchedDylib(location, exists);
       if (exists)
-        return saver.save(location.str());
+        return saver().save(location.str());
     }
   }
   return {};
 }
 
 StringRef macho::rerootPath(StringRef path) {
   if (!path::is_absolute(path, path::Style::posix) || path.endswith(".o"))
     return path;

lld/MachO/InputFiles.cpp

Optional<MemoryBufferRef> macho::readFile(StringRef path) {
   const auto *hdr = reinterpret_cast<const fat_header *>(buf);
   if (mbref.getBufferSize() < sizeof(uint32_t) ||
       read32be(&hdr->magic) != FAT_MAGIC) {
     if (tar)
       tar->append(relativeToRoot(path), mbref.getBuffer());
     return mbref;
   }
 
+  llvm::BumpPtrAllocator &bAlloc = lld::bAlloc();
+
   // Object files and archive files may be fat files, which contain multiple
   // real files for different CPU ISAs. Here, we search for a file that matches
   // with the current link target and returns it as a MemoryBufferRef.
   const auto *arch = reinterpret_cast<const fat_arch *>(buf + sizeof(*hdr));
 
   for (uint32_t i = 0, n = read32be(&hdr->nfat_arch); i < n; ++i) {
     if (reinterpret_cast<const char *>(arch + i + 1) >
         buf + mbref.getBufferSize()) {
for (uint32_t i = 0, n = read32be(&hdr->nfat_arch); i < n; ++i) {
     return MemoryBufferRef(StringRef(buf + offset, size), path.copy(bAlloc));
   }
 
   error("unable to find matching architecture in " + path);
   return None;
 }
 
 InputFile::InputFile(Kind kind, const InterfaceFile &interface)
-    : id(idCount++), fileKind(kind), name(saver.save(interface.getPath())) {}
+    : id(idCount++), fileKind(kind), name(saver().save(interface.getPath())) {}
 
 template <class Section>
 void ObjFile::parseSections(ArrayRef<Section> sections) {
   subsections.reserve(sections.size());
   auto *buf = reinterpret_cast<const uint8_t *>(mb.getBufferStart());
 
   for (const Section &sec : sections) {
     StringRef name =
DylibFile::DylibFile(MemoryBufferRef mb, DylibFile *umbrella,
   }
 
   // Initialize symbols.
   exportingFile = isImplicitlyLinked(installName) ? this : this->umbrella;
   if (const load_command *cmd = findCommand(hdr, LC_DYLD_INFO_ONLY)) {
     auto *c = reinterpret_cast<const dyld_info_command *>(cmd);
     parseTrie(buf + c->export_off, c->export_size,
               [&](const Twine &name, uint64_t flags) {
-                StringRef savedName = saver.save(name);
+                StringRef savedName = saver().save(name);
                 if (handleLDSymbol(savedName))
                   return;
                 bool isWeakDef = flags & EXPORT_SYMBOL_FLAGS_WEAK_DEFINITION;
                 bool isTlv = flags & EXPORT_SYMBOL_FLAGS_KIND_THREAD_LOCAL;
                 symbols.push_back(symtab->addDylib(savedName, exportingFile,
                                                    isWeakDef, isTlv));
               });
   } else {
DylibFile::DylibFile(const InterfaceFile &interface, DylibFile *umbrella,
     : InputFile(DylibKind, interface), refState(RefState::Unreferenced),
       isBundleLoader(isBundleLoader) {
   // FIXME: Add test for the missing TBD code path.
 
   if (umbrella == nullptr)
     umbrella = this;
   this->umbrella = umbrella;
 
-  installName = saver.save(interface.getInstallName());
+  installName = saver().save(interface.getInstallName());
   compatibilityVersion = interface.getCompatibilityVersion().rawValue();
   currentVersion = interface.getCurrentVersion().rawValue();
 
   if (config->printEachFile)
     message(toString(this));
   inputFiles.insert(this);
 
   if (!is_contained(skipPlatformChecks, installName) &&
       !is_contained(interface.targets(), config->platformInfo.target)) {
     error(toString(this) + " is incompatible with " +
           std::string(config->platformInfo.target));
     return;
   }
 
   checkAppExtensionSafety(interface.isApplicationExtensionSafe());
 
   exportingFile = isImplicitlyLinked(installName) ? this : umbrella;
   auto addSymbol = [&](const Twine &name) -> void {
-    symbols.push_back(symtab->addDylib(saver.save(name), exportingFile,
+    symbols.push_back(symtab->addDylib(saver().save(name), exportingFile,
                                        /*isWeakDef=*/false,
                                        /*isTlv=*/false));
   };
   // TODO(compnerd) filter out symbols based on the target platform
   // TODO: handle weak defs, thread locals
   for (const auto *symbol : interface.symbols()) {
     if (!symbol->getArchitectures().has(config->arch()))
       continue;
void DylibFile::handleLDPreviousSymbol(StringRef name, StringRef originalName) {
   if (end.tryParse(endVersion)) {
     warn("failed to parse end version, symbol '" + originalName + "' ignored");
     return;
   }
   if (config->platformInfo.minimum < start ||
       config->platformInfo.minimum >= end)
     return;
 
-  this->installName = saver.save(installName);
+  this->installName = saver().save(installName);
 
   if (!compatVersion.empty()) {
     VersionTuple cVersion;
     if (cVersion.tryParse(compatVersion)) {
       warn("failed to parse compatibility version, symbol '" + originalName +
            "' ignored");
       return;
     }
     compatibilityVersion = encodeVersion(cVersion);
   }
 }
 
 void DylibFile::handleLDInstallNameSymbol(StringRef name,
                                           StringRef originalName) {
   // originalName: $ld$ install_name $ os<version> $ install_name
   StringRef condition, installName;
   std::tie(condition, installName) = name.split('$');
   VersionTuple version;
   if (!condition.consume_front("os") || version.tryParse(condition))
     warn("failed to parse os version, symbol '" + originalName + "' ignored");
   else if (version == config->platformInfo.minimum)
-    this->installName = saver.save(installName);
+    this->installName = saver().save(installName);
 }
 
 void DylibFile::checkAppExtensionSafety(bool dylibIsAppExtensionSafe) const {
   if (config->applicationExtension && !dylibIsAppExtensionSafe)
     warn("using '-application_extension' with unsafe dylib: " + toString(this));
 }
 
 ArchiveFile::ArchiveFile(std::unique_ptr<object::Archive> &&f)
void ArchiveFile::fetch(const object::Archive::Symbol &sym) {
   // Match that: intentionally don't call toMachOString().
   if (Error e = fetch(c, symCopy.getName()))
     error(toString(this) + ": could not get the member defining symbol " +
           toMachOString(symCopy) + ": " + toString(std::move(e)));
 }
 
 static macho::Symbol *createBitcodeSymbol(const lto::InputFile::Symbol &objSym,
                                           BitcodeFile &file) {
-  StringRef name = saver.save(objSym.getName());
+  StringRef name = saver().save(objSym.getName());
 
   // TODO: support weak references
   if (objSym.isUndefined())
     return symtab->addUndefined(name, &file, /*isWeakRef=*/false);
 
   // TODO: Write a test demonstrating why computing isPrivateExtern before
   // LTO compilation is important.
   bool isPrivateExtern = false;
BitcodeFile::BitcodeFile(MemoryBufferRef mb, StringRef archiveName,
     : InputFile(BitcodeKind, mb) {
   std::string path = mb.getBufferIdentifier().str();
   // ThinLTO assumes that all MemoryBufferRefs given to it have a unique
   // name. If two members with the same name are provided, this causes a
   // collision and ThinLTO can't proceed.
   // So, we append the archive name to disambiguate two members with the same
   // name from multiple different archives, and offset within the archive to
   // disambiguate two members of the same name from a single archive.
-  MemoryBufferRef mbref(
-      mb.getBuffer(),
-      saver.save(archiveName.empty() ? path
-                                     : archiveName + sys::path::filename(path) +
+  MemoryBufferRef mbref(mb.getBuffer(),
+                        saver().save(archiveName.empty()
+                                         ? path
+                                         : archiveName +
+                                               sys::path::filename(path) +
                                                utostr(offsetInArchive)));
 
   obj = check(lto::InputFile::create(mbref));
 
   // Convert LTO Symbols to LLD Symbols in order to perform resolution. The
   // "winning" symbol will then be marked as Prevailing at LTO compilation
   // time.
   for (const lto::InputFile::Symbol &objSym : obj->symbols())
     symbols.push_back(createBitcodeSymbol(objSym, *this));
 }
 
 template void ObjFile::parse<LP64>();

lld/MachO/LTO.cpp

std::vector<ObjFile *> BitcodeCompiler::compile() {
   lto::NativeObjectCache cache;
   if (!config->thinLTOCacheDir.empty())
     cache = check(
         lto::localCache(config->thinLTOCacheDir,
                         [&](size_t task, std::unique_ptr<MemoryBuffer> mb) {
                           files[task] = std::move(mb);
                         }));
 
+  CommonLinkingContext &ctx = context();
+
   checkError(ltoObj->run(
       [&](size_t task) {
         return std::make_unique<lto::NativeObjectStream>(
             std::make_unique<raw_svector_ostream>(buf[task]));
       },
-      cache));
+      cache, [&ctx] { attachContext(ctx); }));
 
   if (!config->thinLTOCacheDir.empty())
     pruneCache(config->thinLTOCacheDir, config->thinLTOCachePolicy);
 
   if (config->saveTemps) {
     if (!buf[0].empty())
       saveBuffer(buf[0], config->outputFile + ".lto.o");
     for (unsigned i = 1; i != maxTasks; ++i)
if (!config->ltoObjPath.empty()) {
       filePath = config->ltoObjPath;
       path::append(filePath, Twine(i) + "." +
                                  getArchitectureName(config->arch()) +
                                  ".lto.o");
       saveBuffer(buf[i], filePath);
       modTime = getModTime(filePath);
     }
     ret.push_back(make<ObjFile>(
-        MemoryBufferRef(buf[i], saver.save(filePath.str())), modTime, ""));
+        MemoryBufferRef(buf[i], saver().save(filePath.str())), modTime, ""));
   }
   for (std::unique_ptr<MemoryBuffer> &file : files)
     if (file)
       ret.push_back(make<ObjFile>(*file, 0, ""));
   return ret;
 }

lld/MachO/SyntheticSections.cpp

void SymtabSection::emitBeginSourceStab(DWARFUnit *compileUnit) {
   StabsEntry stab(N_SO);
   SmallString<261> dir(compileUnit->getCompilationDir());
   StringRef sep = sys::path::get_separator();
   // We don't use `path::append` here because we want an empty `dir` to result
   // in an absolute path. `append` would give us a relative path for that case.
   if (!dir.endswith(sep))
     dir += sep;
   stab.strx = stringTableSection.addString(
-      saver.save(dir + compileUnit->getUnitDIE().getShortName()));
+      saver().save(dir + compileUnit->getUnitDIE().getShortName()));
   stabs.emplace_back(std::move(stab));
 }
 
 void SymtabSection::emitEndSourceStab() {
   StabsEntry stab(N_SO);
   stab.sect = 1;
   stabs.emplace_back(std::move(stab));
 }
 
 void SymtabSection::emitObjectFileStab(ObjFile *file) {
   StabsEntry stab(N_OSO);
   stab.sect = target->cpuSubtype;
   SmallString<261> path(!file->archiveName.empty() ? file->archiveName
                                                    : file->getName());
   std::error_code ec = sys::fs::make_absolute(path);
   if (ec)
     fatal("failed to get absolute path for " + path);
 
   if (!file->archiveName.empty())
     path.append({"(", file->getName(), ")"});
 
-  stab.strx = stringTableSection.addString(saver.save(path.str()));
+  stab.strx = stringTableSection.addString(saver().save(path.str()));
   stab.desc = 1;
   stab.value = file->modTime;
   stabs.emplace_back(std::move(stab));
 }
 
 void SymtabSection::emitEndFunStab(Defined *defined) {
   StabsEntry stab(N_FUN);
   stab.value = defined->size;

lld/MachO/Writer.cpp

 static size_t getSymbolPriority(const SymbolPriorityEntry &entry,
                                 const InputFile *f) {
   // We don't use toString(InputFile *) here because it returns the full path
   // for object files, and we only want the basename.
   StringRef filename;
   if (f->archiveName.empty())
     filename = path::filename(f->getName());
   else
-    filename = saver.save(path::filename(f->archiveName) + "(" +
+    filename = saver().save(path::filename(f->archiveName) + "(" +
                             path::filename(f->getName()) + ")");
   return std::max(entry.objectFiles.lookup(filename), entry.anyObjectFile);
 }
 
 // Each section gets assigned the priority of the highest-priority symbol it
 // contains.
 static DenseMap<const InputSection *, size_t> buildInputSectionPriorities() {
   DenseMap<const InputSection *, size_t> sectionPriorities;
 
void macho::createSyntheticSections() {
   in.tlvPointers = make<TlvPointerSection>();
   in.lazyPointers = make<LazyPointerSection>();
   in.stubs = make<StubsSection>();
   in.stubHelper = make<StubHelperSection>();
   in.unwindInfo = makeUnwindInfoSection();
 
   // This section contains space for just a single word, and will be used by
   // dyld to cache an address to the image loader it uses.
-  uint8_t *arr = bAlloc.Allocate<uint8_t>(target->wordSize);
+  uint8_t *arr = bAlloc().Allocate<uint8_t>(target->wordSize);
   memset(arr, 0, target->wordSize);
   in.imageLoaderCache = make<ConcatInputSection>(
       segment_names::data, section_names::data, /*file=*/nullptr,
       ArrayRef<uint8_t>{arr, target->wordSize},
       /*align=*/target->wordSize, /*flags=*/S_REGULAR);
   // References from dyld are not visible to us, so ensure this section is
   // always treated as live.
   in.imageLoaderCache->live = true;
 }
 
 OutputSection *macho::firstTLVDataSection = nullptr;
 
 template void macho::writeResult<LP64>();
 template void macho::writeResult<ILP32>();

lld/MinGW/Driver.cpp

static cl::TokenizerCallback getQuotingStyle() {
   return cl::TokenizeGNUCommandLine;
 }
 
 opt::InputArgList MinGWOptTable::parse(ArrayRef<const char *> argv) {
   unsigned missingIndex;
   unsigned missingCount;
 
   SmallVector<const char *, 256> vec(argv.data(), argv.data() + argv.size());
-  cl::ExpandResponseFiles(saver, getQuotingStyle(), vec);
+  cl::ExpandResponseFiles(saver(), getQuotingStyle(), vec);
   opt::InputArgList args = this->ParseArgs(vec, missingIndex, missingCount);
 
   if (missingCount)
     error(StringRef(args.getArgString(missingIndex)) + ": missing argument");
   for (auto *arg : args.filtered(OPT_UNKNOWN))
     error("unknown argument: " + arg->getAsString(args));
   return args;
 }
for (StringRef dir : searchPaths) {
     }
   }
   error("unable to find library -l" + name);
   return "";
 }
 
 // Convert Unix-ish command line arguments to Windows-ish ones and
 // then call coff::link.
-bool mingw::link(ArrayRef<const char *> argsArr, bool canExitEarly,
-                 raw_ostream &stdoutOS, raw_ostream &stderrOS) {
-  lld::stdoutOS = &stdoutOS;
-  lld::stderrOS = &stderrOS;
-
-  stderrOS.enable_colors(stderrOS.has_colors());
-
+bool mingw::link(ArrayRef<const char *> argsArr) {
   MinGWOptTable parser;
   opt::InputArgList args = parser.parse(argsArr.slice(1));
 
   if (errorCount())
     return false;
 
   if (args.hasArg(OPT_help)) {
     printHelp(argsArr[0]);
bool mingw::link(ArrayRef<const char *> argsArr) {
 
   // Repack vector of strings to vector of const char pointers for coff::link.
   std::vector<const char *> vec;
   for (const std::string &s : linkArgs)
     vec.push_back(s.c_str());
   // Pass the actual binary name, to make error messages be printed with
   // the right prefix.
   vec[0] = argsArr[0];
-  return coff::link(vec, canExitEarly, stdoutOS, stderrOS);
+  return coff::link(vec);
 }

lld/include/lld/Common/Driver.h

 //===- lld/Common/Driver.h - Linker Driver Emulator -----------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLD_COMMON_DRIVER_H
 #define LLD_COMMON_DRIVER_H
 
+#include "lld/Common/Globals.h"
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/Support/raw_ostream.h"
 
 namespace lld {
 struct SafeReturn {
   int ret;
   bool canRunAgain;
 };
 
 // Generic entry point when using LLD as a library, safe for re-entry, supports
 // crash recovery. Returns a general completion code and a boolean telling
 // whether it can be called again. In some cases, a crash could corrupt memory
 // and re-entry would not be possible anymore. Use exitLld() in that case to
 // properly exit your application and avoid intermittent crashes on exit caused
 // by cleanup.
 SafeReturn safeLldMain(int argc, const char **argv, llvm::raw_ostream &stdoutOS,
                        llvm::raw_ostream &stderrOS);
 
 namespace coff {
-bool link(llvm::ArrayRef<const char *> args, bool canExitEarly,
-          llvm::raw_ostream &stdoutOS, llvm::raw_ostream &stderrOS);
+bool link(llvm::ArrayRef<const char *> args);
 }
 
 namespace mingw {
-bool link(llvm::ArrayRef<const char *> args, bool canExitEarly,
-          llvm::raw_ostream &stdoutOS, llvm::raw_ostream &stderrOS);
+bool link(llvm::ArrayRef<const char *> args);
 }
 
 namespace elf {
-bool link(llvm::ArrayRef<const char *> args, bool canExitEarly,
-          llvm::raw_ostream &stdoutOS, llvm::raw_ostream &stderrOS);
+bool link(llvm::ArrayRef<const char *> args);
 }
 
 namespace mach_o {
-bool link(llvm::ArrayRef<const char *> args, bool canExitEarly,
-          llvm::raw_ostream &stdoutOS, llvm::raw_ostream &stderrOS);
+bool link(llvm::ArrayRef<const char *> args);
 }
 
 namespace macho {
-bool link(llvm::ArrayRef<const char *> args, bool canExitEarly,
-          llvm::raw_ostream &stdoutOS, llvm::raw_ostream &stderrOS);
+bool link(llvm::ArrayRef<const char *> args);
 }
 
 namespace wasm {
-bool link(llvm::ArrayRef<const char *> args, bool canExitEarly,
-          llvm::raw_ostream &stdoutOS, llvm::raw_ostream &stderrOS);
+bool link(llvm::ArrayRef<const char *> args);
 }
 }
 
 #endif

lld/include/lld/Common/ErrorHandler.h

 // because they are not thread-safe. The functions declared in this file are
 // thread-safe.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLD_COMMON_ERRORHANDLER_H
 #define LLD_COMMON_ERRORHANDLER_H
 
+#include "lld/Common/Globals.h"
 #include "lld/Common/LLVM.h"
-
 #include "llvm/ADT/STLExtras.h"
 #include "llvm/Support/Error.h"
 #include "llvm/Support/FileOutputBuffer.h"
 
 namespace llvm {
 class DiagnosticInfo;
 class raw_ostream;
 }
 
 namespace lld {
 
-// We wrap stdout and stderr so that you can pass alternative stdout/stderr as
-// arguments to lld::*::link() functions.
-extern llvm::raw_ostream *stdoutOS;
-extern llvm::raw_ostream *stderrOS;
-
 llvm::raw_ostream &outs();
 llvm::raw_ostream &errs();
 
 enum class ErrorTag { LibNotFound, SymbolNotFound };
 
-class ErrorHandler {
+class ErrorHandler final {
 public:
   uint64_t errorCount = 0;
   uint64_t errorLimit = 20;
   StringRef errorLimitExceededMsg = "too many errors emitted, stopping now";
   StringRef errorHandlingScript;
   StringRef logName = "lld";
   bool exitEarly = true;
   bool fatalWarnings = false;
   bool verbose = false;
   bool vsDiagnostics = false;
   bool disableOutput = false;
   std::function<void()> cleanupCallback;
 
   void error(const Twine &msg);
   void error(const Twine &msg, ErrorTag tag, ArrayRef<StringRef> args);
   [[noreturn]] void fatal(const Twine &msg);
   void log(const Twine &msg);
   void message(const Twine &msg);
   void warn(const Twine &msg);
 
-  void reset() {
-    if (cleanupCallback)
-      cleanupCallback();
-    *this = ErrorHandler();
-  }
+  ~ErrorHandler();
 
   std::unique_ptr<llvm::FileOutputBuffer> outputBuffer;
 
 private:
   using Colors = raw_ostream::Colors;
 
   std::string getLocation(const Twine &msg);
 };
 
 /// Returns the default error handler.
 ErrorHandler &errorHandler();
 
+void setStdPipes(llvm::raw_ostream &stdoutOS, llvm::raw_ostream &stderrOS);
+
 inline void error(const Twine &msg) { errorHandler().error(msg); }
 inline void error(const Twine &msg, ErrorTag tag, ArrayRef<StringRef> args) {
   errorHandler().error(msg, tag, args);
 }
 [[noreturn]] inline void fatal(const Twine &msg) { errorHandler().fatal(msg); }
 inline void log(const Twine &msg) { errorHandler().log(msg); }
 inline void message(const Twine &msg) { errorHandler().message(msg); }
 inline void warn(const Twine &msg) { errorHandler().warn(msg); }

lld/include/lld/Common/Globals.h

+//===- Globals.h ------------------------------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// Entry point for all global state in lldCommon. The objective is for LLD to be
+// used "as a library" in a thread-safe manner.
+//
+// Instead of program-wide globals or function-local statics, we prefer
+// aggregating all "global" states into a heap-based structure
+// (CommonLinkingContext). This also achieves deterministic initialization &
+// shutdown for all "global" states.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLD_COMMON_GLOBALS_H
+#define LLD_COMMON_GLOBALS_H
+
+#include "llvm/Support/Alignment.h"
+#include "llvm/Support/Allocator.h"
+
+namespace lld {
+
+struct MemoryState;
+struct TargetOptState;
+struct ErrorState;
+struct TimerState;
+
+class CommonLinkingContext {
+public:
+  CommonLinkingContext();
+  virtual ~CommonLinkingContext();
+
+  static void shutdown();
+
+  bool isValid();
+
+  llvm::BumpPtrAllocator bAlloc;
+
+  // Obtain global states as incomplete types. Client code needs to #include the
+  // corresponding headers to access their APIs.
+  MemoryState &m();
+  TargetOptState &to();
+  ErrorState &e();
+  TimerState &t();
+
+private:
+  // Provide storage so that all globals are aggregated on the stack closer to
+  // each other, since they should be accessed quite frequently.
+  uint64_t storage[50]{};
+
+  unsigned memoryOffset{};
+  unsigned targetOptOffset{};
+  unsigned errorOffset{};
+  unsigned timerOffset{};
+
+  static unsigned id;
+  unsigned *valid;
+};
+
+// Attach global state to a thread. This should be usually be called just after
+// the thread has been created, and before any calls to any LLD APIs.
+void attachContext(CommonLinkingContext &ctx);
+
+// Detach the global state to prevent accidental re-use betwen tasks scheduled
+// on the same thread. The thread pool could possibly be shared between LLD
+// instances in the future.
+void detachContext();
+
+// Retrieve the global state associated to the current thread. There could be
+// several global states, one for each running LLD instance in memory.
+CommonLinkingContext &commonContext();
+
+template <typename T = CommonLinkingContext> T &context() {
+  return static_cast<T &>(commonContext());
+}
+
+// Tells whether this thread has a binding to a global state structure.
+bool hasContext();
+
+// Helper for building the *State classes. This is only to keep the *State
+// classes private and only expose APIs that act on them (a bit like pImpl)
+template <typename T, typename... U>
+T *initContextHelper(char *const start, char *&addr, unsigned &offset,
+                     U &&...args) {
+  void *storage = (char *)llvm::alignAddr(addr, llvm::Align(alignof(T)));
+  T *global = new (storage) T(std::forward<U>(args)...);
+  addr = (char *)storage + sizeof(T);
+  offset = (char *)global - start;
+  return global;
+}
+
+} // namespace lld
+
+#endif

lld/include/lld/Common/Memory.h

 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLD_COMMON_MEMORY_H
 #define LLD_COMMON_MEMORY_H
 
 #include "llvm/Support/Allocator.h"
 #include "llvm/Support/StringSaver.h"
-#include <vector>
 
 namespace lld {
-
-// Use this arena if your object doesn't have a destructor.
-extern llvm::BumpPtrAllocator bAlloc;
-extern llvm::StringSaver saver;
-
-void freeArena();
+llvm::StringSaver &saver();
+llvm::BumpPtrAllocator &bAlloc();
 
 // These two classes are hack to keep track of all
 // SpecificBumpPtrAllocator instances.
 struct SpecificAllocBase {
-  SpecificAllocBase() { instances.push_back(this); }
   virtual ~SpecificAllocBase() = default;
   virtual void reset() = 0;
-  static std::vector<SpecificAllocBase *> instances;
+  static SpecificAllocBase *getOrCreate(void *tag, size_t size, size_t align,
+                                        SpecificAllocBase *(&creator)(void *));
 };
 
 template <class T> struct SpecificAlloc : public SpecificAllocBase {
+  static SpecificAllocBase *create(void *storage) {
+    return new (storage) SpecificAlloc<T>();
+  }
   void reset() override { alloc.DestroyAll(); }
   llvm::SpecificBumpPtrAllocator<T> alloc;
+  static int id;
 };
 
+template <class T> int SpecificAlloc<T>::id = 0;
+
 // Use a static local for these singletons so they are only registered if an
 // object of this instance is ever constructed. Otherwise we will create and
 // register ELF allocators for COFF and the reverse.
 template <typename T>
 inline llvm::SpecificBumpPtrAllocator<T> &getSpecificAllocSingleton() {
-  static SpecificAlloc<T> instance;
-  return instance.alloc;
+  SpecificAllocBase *instance = SpecificAllocBase::getOrCreate(
+      &SpecificAlloc<T>::id, sizeof(SpecificAlloc<T>),
+      alignof(SpecificAlloc<T>), SpecificAlloc<T>::create);
+  return ((SpecificAlloc<T> *)instance)->alloc;
 }
 
 // Use this arena if your object has a destructor.
 // Your destructor will be invoked from freeArena().
 template <typename T, typename... U> T *make(U &&... args) {
   return new (getSpecificAllocSingleton<T>().Allocate())
       T(std::forward<U>(args)...);
 }
 
 } // namespace lld
 
 #endif

lld/include/lld/Common/Timer.h

public:
   static Timer &root();
 
   void addToTotal(std::chrono::nanoseconds time) { total += time.count(); }
   void print();
 
   double millis() const;
 
 private:
+  friend struct TimerState;
   explicit Timer(llvm::StringRef name);
   void print(int depth, double totalDuration, bool recurse = true) const;
 
   std::atomic<std::chrono::nanoseconds::rep> total;
   std::vector<Timer *> children;
   std::string name;
 };
 
 } // namespace lld
 
 #endif

lld/include/lld/Core/LinkingContext.h

 //===- lld/Core/LinkingContext.h - Linker Target Info Interface -*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLD_CORE_LINKING_CONTEXT_H
 #define LLD_CORE_LINKING_CONTEXT_H
 
+#include "lld/Common/Globals.h"
 #include "lld/Core/Node.h"
 #include "lld/Core/Reader.h"
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/Support/Allocator.h"
 #include "llvm/Support/Error.h"
 #include <cassert>
 #include <cstdint>

 class Node;
 class SharedLibraryFile;
 
 /// The LinkingContext class encapsulates "what and how" to link.
 ///
 /// The base class LinkingContext contains the options needed by core linking.
 /// Subclasses of LinkingContext have additional options needed by specific
 /// Writers.
-class LinkingContext {
+class LinkingContext : public CommonLinkingContext {
 public:
   virtual ~LinkingContext();
 
   /// \name Methods needed by core linking
   /// @{
 
   /// Name of symbol linker should use as "entry point" to program,
   /// usually "main" or "start".

lld/lib/Driver/DarwinLdDriver.cpp

 //===- lib/Driver/DarwinLdDriver.cpp --------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 ///
 /// \file
 ///
 /// Concrete instance of the Driver for darwin's ld.
 ///
 //===----------------------------------------------------------------------===//
 
 #include "lld/Common/Args.h"
 #include "lld/Common/ErrorHandler.h"
+#include "lld/Common/Globals.h"
 #include "lld/Common/LLVM.h"
 #include "lld/Core/ArchiveLibraryFile.h"
 #include "lld/Core/Error.h"
 #include "lld/Core/File.h"
 #include "lld/Core/Instrumentation.h"
 #include "lld/Core/LinkingContext.h"
 #include "lld/Core/Node.h"
 #include "lld/Core/PassManager.h"
else
     ctx.createInternalFiles(Files);
   for (auto i = Files.rbegin(), e = Files.rend(); i != e; ++i) {
     auto &members = ctx.getNodes();
     members.insert(members.begin(), std::make_unique<FileNode>(std::move(*i)));
   }
 }
 
 /// This is where the link is actually performed.
-bool link(llvm::ArrayRef<const char *> args, bool CanExitEarly,
-          raw_ostream &StdoutOS, raw_ostream &StderrOS) {
-  lld::stdoutOS = &StdoutOS;
-  lld::stderrOS = &StderrOS;
-
+bool link(llvm::ArrayRef<const char *> args) {
   errorHandler().logName = args::getFilenameWithoutExe(args[0]);
   errorHandler().errorLimitExceededMsg =
       "too many errors emitted, stopping now (use "
       "'-error-limit 0' to see all errors)";
-  errorHandler().exitEarly = CanExitEarly;
-  StderrOS.enable_colors(StderrOS.has_colors());
 
-  MachOLinkingContext ctx;
+  MachOLinkingContext &ctx = context<MachOLinkingContext>();
   if (!parse(args, ctx))
     return false;
   if (ctx.doNothing())
     return true;
   if (ctx.getNodes().empty())
     return false;
 
   for (std::unique_ptr<Node> &ie : ctx.getNodes())
bool link(llvm::ArrayRef<const char *> args) {
   ScopedTask writeTask(getDefaultDomain(), "Write");
   if (auto ec = ctx.writeFile(*merged)) {
     // FIXME: This should be passed to logAllUnhandledErrors but it needs
     // to be passed a Twine instead of a string.
     lld::errs() << "Failed to write file '" << ctx.outputPath() << "': ";
     logAllUnhandledErrors(std::move(ec), lld::errs(), std::string());
     return false;
   }
-
-  // Call exit() if we can to avoid calling destructors.
-  if (CanExitEarly)
-    exitLld(errorCount() ? 1 : 0);
-
-
-  return true;
+  return errorCount() == 0;
 }
 
 } // end namespace mach_o
 } // end namespace lld

lld/tools/lld/lld.cpp

 // lld can be invoked as "lld" along with "-flavor" option. This is for
 // backward compatibility and not recommended.
 //
 //===----------------------------------------------------------------------===//
 
 #include "lld/Common/Driver.h"
 #include "lld/Common/ErrorHandler.h"
 #include "lld/Common/Memory.h"
+#include "lld/ReaderWriter/MachOLinkingContext.h"
 #include "llvm/ADT/STLExtras.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/ADT/StringSwitch.h"
 #include "llvm/ADT/Triple.h"
 #include "llvm/ADT/Twine.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/CrashRecoveryContext.h"
 #include "llvm/Support/Host.h"
 #include "llvm/Support/InitLLVM.h"
+#include "llvm/Support/Parallel.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/PluginLoader.h"
 #include <cstdlib>
 
 using namespace lld;
 using namespace llvm;
 using namespace llvm::sys;
 
static bool isPETarget(std::vector<const char *> &v) {
   for (auto it = v.begin(); it + 1 != v.end(); ++it) {
     if (StringRef(*it) != "-m")
       continue;
     return isPETargetName(*(it + 1));
   }
   // Expand response files (arguments in the form of @<filename>)
   // to allow detecting the -m argument from arguments in them.
   SmallVector<const char *, 256> expandedArgs(v.data(), v.data() + v.size());
-  cl::ExpandResponseFiles(saver, getDefaultQuotingStyle(), expandedArgs);
+  BumpPtrAllocator A;
+  StringSaver Saver(A);
+  cl::ExpandResponseFiles(Saver, getDefaultQuotingStyle(), expandedArgs);
   for (auto it = expandedArgs.begin(); it + 1 != expandedArgs.end(); ++it) {
     if (StringRef(*it) != "-m")
       continue;
     return isPETargetName(*(it + 1));
   }
 
 #ifdef LLD_DEFAULT_LD_LLD_IS_MINGW
   return true;
static Flavor parseFlavor(std::vector<const char *> &v) {
 
   // Deduct the flavor from argv[0].
   StringRef arg0 = path::filename(v[0]);
   if (arg0.endswith_insensitive(".exe"))
     arg0 = arg0.drop_back(4);
   return parseProgname(arg0);
 }
 
+bool inTestOutputDisabled = false;
+
 /// Universal linker main(). This linker emulates the gnu, darwin, or
 /// windows linker based on the argv[0] or -flavor option.
 static int lldMain(int argc, const char **argv, llvm::raw_ostream &stdoutOS,
                    llvm::raw_ostream &stderrOS, bool exitEarly = true) {
+
+  CommonLinkingContext *ctx = nullptr;
+
   std::vector<const char *> args(argv, argv + argc);
-  switch (parseFlavor(args)) {
-  case Gnu:
-    if (isPETarget(args))
-      return !mingw::link(args, exitEarly, stdoutOS, stderrOS);
-    return !elf::link(args, exitEarly, stdoutOS, stderrOS);
-  case WinLink:
-    return !coff::link(args, exitEarly, stdoutOS, stderrOS);
-  case Darwin:
-    return !macho::link(args, exitEarly, stdoutOS, stderrOS);
-  case DarwinOld:
-    return !mach_o::link(args, exitEarly, stdoutOS, stderrOS);
-  case Wasm:
-    return !lld::wasm::link(args, exitEarly, stdoutOS, stderrOS);
-  default:
+  Flavor f = parseFlavor(args);
+  bool isMinGW = f == Gnu ? isPETarget(args) : false;
+
+  if (f == Invalid)
     die("lld is a generic driver.\n"
         "Invoke ld.lld (Unix), ld64.lld (macOS), lld-link (Windows), wasm-ld"
         " (WebAssembly) instead");
-  }
+
+  // Create the global context for the corresponding driver. Currently most
+  // drivers use the "base" common context, but this will change as we
+  // progressively move global state to their driver-specific context.
+  if (f == DarwinOld)
+    ctx = new MachOLinkingContext;
+  else
+    ctx = new CommonLinkingContext;
+
+  setStdPipes(stdoutOS, stderrOS);
+  errorHandler().exitEarly = exitEarly;
+  errorHandler().disableOutput = inTestOutputDisabled;
+  stderrOS.enable_colors(stderrOS.has_colors());
+
+  // FIXME: Temporary, bind all TaskGroup threads with the current LLD context.
+  llvm::parallel::detail::TaskGroup::wrapThreadState(
+      [&ctx](std::function<void()> &f) {
+        attachContext(*ctx);
+        f();
+        detachContext();
+      });
+
+  bool r; // The driver will be returning false if an error occurs.
+
+  // Run the driver.
+  if (isMinGW)
+    r = mingw::link(args);
+  else if (f == Gnu)
+    r = elf::link(args);
+  else if (f == WinLink)
+    r = coff::link(args);
+  else if (f == Darwin)
+    r = macho::link(args);
+  else if (f == DarwinOld)
+    r = mach_o::link(args);
+  else if (f == Wasm)
+    r = lld::wasm::link(args);
+
+  // Call exit() if we can to avoid calling destructors.
+  if (exitEarly)
+    exitLld(!r ? 1 : 0);
+
+  // Delete global state & detach from current thread.
+  delete ctx;
+
+  return !r ? 1 : 0;
 }
 
 // Similar to lldMain except that exceptions are caught.
 SafeReturn lld::safeLldMain(int argc, const char **argv,
                             llvm::raw_ostream &stdoutOS,
                             llvm::raw_ostream &stderrOS) {
   int r = 0;
   {
     // The crash recovery is here only to be able to recover from arbitrary
     // control flow when fatal() is called (through setjmp/longjmp or
     // __try/__except).
     llvm::CrashRecoveryContext crc;
     if (!crc.RunSafely([&]() {
           r = lldMain(argc, argv, stdoutOS, stderrOS, /*exitEarly=*/false);
         }))
       return {crc.RetCode, /*canRunAgain=*/false};
   }
 
   // Cleanup memory and reset everything back in pristine condition. This path
   // is only taken when LLD is in test, or when it is used as a library.
   llvm::CrashRecoveryContext crc;
-  if (!crc.RunSafely([&]() { errorHandler().reset(); })) {
+  if (!crc.RunSafely([&]() { CommonLinkingContext::shutdown(); })) {
     // The memory is corrupted beyond any possible recovery.
     return {r, /*canRunAgain=*/false};
   }
   return {r, /*canRunAgain=*/true};
 }
 
 // When in lit tests, tells how many times the LLD tool should re-execute the
 // main loop with the same inputs. When not in test, returns a value of 0 which
int main(int argc, const char **argv) {
   if (!inTestVerbosity())
     return lldMain(argc, argv, llvm::outs(), llvm::errs());
 
   Optional<int> mainRet;
   CrashRecoveryContext::Enable();
 
   for (unsigned i = inTestVerbosity(); i > 0; --i) {
     // Disable stdout/stderr for all iterations but the last one.
-    if (i != 1)
-      errorHandler().disableOutput = true;
+    inTestOutputDisabled = (i != 1);
 
     // Execute one iteration.
     auto r = safeLldMain(argc, argv, llvm::outs(), llvm::errs());
     if (!r.canRunAgain)
       exitLld(r.ret); // Exit now, can't re-execute again.
 
     if (!mainRet) {
       mainRet = r.ret;
     } else if (r.ret != *mainRet) {
       // Exit now, to fail the tests if the result is different between runs.
       return r.ret;
     }
   }
   return *mainRet;
 }

lld/wasm/Driver.cpp

private:
 
   // True if we are in --whole-archive and --no-whole-archive.
   bool inWholeArchive = false;
 
   std::vector<InputFile *> files;
 };
 } // anonymous namespace
 
-bool link(ArrayRef<const char *> args, bool canExitEarly, raw_ostream &stdoutOS,
-          raw_ostream &stderrOS) {
-  lld::stdoutOS = &stdoutOS;
-  lld::stderrOS = &stderrOS;
-
-  errorHandler().cleanupCallback = []() { freeArena(); };
-
+bool link(ArrayRef<const char *> args) {
   errorHandler().logName = args::getFilenameWithoutExe(args[0]);
   errorHandler().errorLimitExceededMsg =
       "too many errors emitted, stopping now (use "
       "-error-limit=0 to see all errors)";
-  stderrOS.enable_colors(stderrOS.has_colors());
 
   config = make<Configuration>();
   symtab = make<SymbolTable>();
 
   initLLVM();
   LinkerDriver().linkerMain(args);
 
-  // Exit immediately if we don't need to return to the caller.
-  // This saves time because the overhead of calling destructors
-  // for all globally-allocated objects is not negligible.
-  if (canExitEarly)
-    exitLld(errorCount() ? 1 : 0);
-
-  return !errorCount();
+  return errorCount() == 0;
 }
 
 // Create prefix string literals used in Options.td
 #define PREFIX(NAME, VALUE) const char *const NAME[] = VALUE;
 #include "Options.inc"
 #undef PREFIX
 
 // Create table mapping all options defined in Options.td
opt::InputArgList WasmOptTable::parse(ArrayRef<const char *> argv) {
 
   // We need to get the quoting style for response files before parsing all
   // options so we parse here before and ignore all the options but
   // --rsp-quoting.
   opt::InputArgList args = this->ParseArgs(vec, missingIndex, missingCount);
 
   // Expand response files (arguments in the form of @<filename>)
   // and then parse the argument again.
-  cl::ExpandResponseFiles(saver, getQuotingStyle(args), vec);
+  cl::ExpandResponseFiles(saver(), getQuotingStyle(args), vec);
   args = this->ParseArgs(vec, missingIndex, missingCount);
 
   handleColorDiagnostics(args);
   for (auto *arg : args.filtered(OPT_UNKNOWN))
     error("unknown argument: " + arg->getAsString(args));
   return args;
 }
 
for (auto *arg : args.filtered(OPT_wrap)) {
     StringRef name = arg->getValue();
     if (!seen.insert(name).second)
       continue;
 
     Symbol *sym = symtab->find(name);
     if (!sym)
       continue;
 
-    Symbol *real = addUndefined(saver.save("__real_" + name));
-    Symbol *wrap = addUndefined(saver.save("__wrap_" + name));
+    Symbol *real = addUndefined(saver().save("__real_" + name));
+    Symbol *wrap = addUndefined(saver().save("__wrap_" + name));
     v.push_back({sym, real, wrap});
 
     // We want to tell LTO not to inline symbols to be overwritten
     // because LTO doesn't know the final symbol contents after renaming.
     real->canInline = false;
     sym->canInline = false;
 
     // Tell LTO not to eliminate these symbols.

lld/wasm/InputFiles.cpp

case GlobalValue::ProtectedVisibility:
     return WASM_SYMBOL_VISIBILITY_HIDDEN;
   }
   llvm_unreachable("unknown visibility");
 }
 
 static Symbol *createBitcodeSymbol(const std::vector<bool> &keptComdats,
                                    const lto::InputFile::Symbol &objSym,
                                    BitcodeFile &f) {
-  StringRef name = saver.save(objSym.getName());
+  StringRef name = saver().save(objSym.getName());
 
   uint32_t flags = objSym.isWeak() ? WASM_SYMBOL_BINDING_WEAK : 0;
   flags |= mapVisibility(objSym.getVisibility());
 
   int c = objSym.getComdatIndex();
   bool excludedByComdat = c != -1 && !keptComdats[c];
 
   if (objSym.isUndefined() || excludedByComdat) {

 
 void BitcodeFile::parse() {
   if (doneLTO) {
     error(toString(this) + ": attempt to add bitcode file after LTO.");
     return;
   }
 
   obj = check(lto::InputFile::create(MemoryBufferRef(
-      mb.getBuffer(), saver.save(archiveName + mb.getBufferIdentifier()))));
+      mb.getBuffer(), saver().save(archiveName + mb.getBufferIdentifier()))));
   Triple t(obj->getTargetTriple());
   if (!t.isWasm()) {
     error(toString(this) + ": machine type must be wasm32 or wasm64");
     return;
   }
   checkArch(t.getArch());
   std::vector<bool> keptComdats;
   // TODO Support nodeduplicate https://bugs.llvm.org/show_bug.cgi?id=50531

lld/wasm/LTO.cpp

std::vector<StringRef> BitcodeCompiler::compile() {
   lto::NativeObjectCache cache;
   if (!config->thinLTOCacheDir.empty())
     cache = check(
         lto::localCache(config->thinLTOCacheDir,
                         [&](size_t task, std::unique_ptr<MemoryBuffer> mb) {
                           files[task] = std::move(mb);
                         }));
 
+  CommonLinkingContext &ctx = context();
+
   checkError(ltoObj->run(
       [&](size_t task) {
         return std::make_unique<lto::NativeObjectStream>(
             std::make_unique<raw_svector_ostream>(buf[task]));
       },
-      cache));
+      cache, [&ctx] { attachContext(ctx); }));
 
   if (!config->thinLTOCacheDir.empty())
     pruneCache(config->thinLTOCacheDir, config->thinLTOCachePolicy);
 
   std::vector<StringRef> ret;
   for (unsigned i = 0; i != maxTasks; ++i) {
     if (buf[i].empty())
       continue;

lld/wasm/SymbolTable.cpp

InputFunction *SymbolTable::replaceWithUnreachable(Symbol *sym,
   // should always compare equal to the null pointer.
   sym->isStub = true;
   return func;
 }
 
 void SymbolTable::replaceWithUndefined(Symbol *sym) {
   // Add a synthetic dummy for weak undefined functions.  These dummies will
   // be GC'd if not used as the target of any "call" instructions.
-  StringRef debugName = saver.save("undefined_weak:" + toString(*sym));
+  StringRef debugName = saver().save("undefined_weak:" + toString(*sym));
   replaceWithUnreachable(sym, *sym->getSignature(), debugName);
   // Hide our dummy to prevent export.
   sym->setHidden(true);
 }
 
 // For weak undefined functions, there may be "call" instructions that reference
 // the symbol. In this case, we need to synthesise a dummy/stub function that
 // will abort at runtime, so that relocations can still provided an operand to
if (!defined) {
                                       cast<FunctionSymbol>(variants[1]), true);
       return;
     }
 
     for (auto *symbol : variants) {
       if (symbol != defined) {
         auto *f = cast<FunctionSymbol>(symbol);
         reportFunctionSignatureMismatch(symName, f, defined, false);
-        StringRef debugName = saver.save("signature_mismatch:" + toString(*f));
+        StringRef debugName =
+            saver().save("signature_mismatch:" + toString(*f));
         replaceWithUnreachable(f, *f->signature, debugName);
       }
     }
   }
 }
 
 } // namespace wasm
 } // namespace lld

lld/wasm/Writer.cpp

if (!count)
       continue;
 
     StringRef name;
     if (sec->type == WASM_SEC_DATA)
       name = "reloc.DATA";
     else if (sec->type == WASM_SEC_CODE)
       name = "reloc.CODE";
     else if (sec->type == WASM_SEC_CUSTOM)
-      name = saver.save("reloc." + sec->name);
+      name = saver().save("reloc." + sec->name);
     else
       llvm_unreachable(
           "relocations only supported for code, data, or custom sections");
 
     addSection(make<RelocSection>(name, sec));
   }
 }
 

 // gold provide the feature, and used by many programs.
 static void addStartStopSymbols(const OutputSegment *seg) {
   StringRef name = seg->name;
   if (!isValidCIdentifier(name))
     return;
   LLVM_DEBUG(dbgs() << "addStartStopSymbols: " << name << "\n");
   uint64_t start = seg->startVA;
   uint64_t stop = start + seg->size;
-  symtab->addOptionalDataSymbol(saver.save("__start_" + name), start);
-  symtab->addOptionalDataSymbol(saver.save("__stop_" + name), stop);
+  symtab->addOptionalDataSymbol(saver().save("__start_" + name), start);
+  symtab->addOptionalDataSymbol(saver().save("__stop_" + name), stop);
 }
 
 void Writer::addSections() {
   addSection(out.dylinkSec);
   addSection(out.typeSec);
   addSection(out.importSec);
   addSection(out.functionSec);
   addSection(out.tableSec);

 
 static void createFunction(DefinedFunction *func, StringRef bodyContent) {
   std::string functionBody;
   {
     raw_string_ostream os(functionBody);
     writeUleb128(os, bodyContent.size(), "function size");
     os << bodyContent;
   }
-  ArrayRef<uint8_t> body = arrayRefFromStringRef(saver.save(functionBody));
+  ArrayRef<uint8_t> body = arrayRefFromStringRef(saver().save(functionBody));
   cast<SyntheticFunction>(func->function)->setBody(body);
 }
 
 bool Writer::needsPassiveInitialization(const OutputSegment *segment) {
   return segment->initFlags & WASM_DATA_SEGMENT_IS_PASSIVE &&
          !segment->isTLS() && !segment->isBss;
 }
 

llvm/include/llvm/LTO/LTO.h

 ///
 /// Clients generally look like this:
 ///
 /// if (AddStreamFn AddStream = Cache(Task, Key))
 ///   ProduceContent(AddStream);
 using NativeObjectCache =
     std::function<AddStreamFn(unsigned Task, StringRef Key)>;
 
+/// This lets callers insert a behavior whenever a LTO thread starts. This is
+/// typically used to bind thread_local pointers to an application-wide global
+/// state that lives on the stack.
+using ThreadInitFn = std::function<void(void)>;
+
 /// A ThinBackend defines what happens after the thin-link phase during ThinLTO.
 /// The details of this type definition aren't important; clients can only
 /// create a ThinBackend using one of the create*ThinBackend() functions below.
 using ThinBackend = std::function<std::unique_ptr<ThinBackendProc>(
     const Config &C, ModuleSummaryIndex &CombinedIndex,
     StringMap<GVSummaryMapTy> &ModuleToDefinedGVSummaries,
-    AddStreamFn AddStream, NativeObjectCache Cache)>;
+    AddStreamFn AddStream, NativeObjectCache Cache, ThreadInitFn ThreadInit)>;
 
 /// This ThinBackend runs the individual backend jobs in-process.
 /// The default value means to use one job per hardware core (not hyper-thread).
 ThinBackend createInProcessThinBackend(ThreadPoolStrategy Parallelism);
 
 /// This ThinBackend writes individual module indexes to files, instead of
 /// running the individual backend jobs. This backend is for distributed builds
 /// where separate processes will invoke the real backends.
public:
   /// Runs the LTO pipeline. This function calls the supplied AddStream
   /// function to add native object files to the link.
   ///
   /// The Cache parameter is optional. If supplied, it will be used to cache
   /// native object files and add them to the link.
   ///
   /// The client will receive at most one callback (via either AddStream or
   /// Cache) for each task identifier.
-  Error run(AddStreamFn AddStream, NativeObjectCache Cache = nullptr);
+  Error run(AddStreamFn AddStream, NativeObjectCache Cache = nullptr,
+            ThreadInitFn ThreadInit = nullptr);
 
   /// Static method that returns a list of libcall symbols that can be generated
   /// by LTO but might not be visible from bitcode symbol table.
   static ArrayRef<const char*> getRuntimeLibcallSymbols();
 
 private:
   Config Conf;
 
private:
   Error linkRegularLTO(RegularLTOState::AddedModule Mod,
                        bool LivenessFromIndex);
 
   Error addThinLTO(BitcodeModule BM, ArrayRef<InputFile::Symbol> Syms,
                    const SymbolResolution *&ResI, const SymbolResolution *ResE);
 
   Error runRegularLTO(AddStreamFn AddStream);
   Error runThinLTO(AddStreamFn AddStream, NativeObjectCache Cache,
+                   ThreadInitFn ThreadInit,
                    const DenseSet<GlobalValue::GUID> &GUIDPreservedSymbols);
 
   Error checkPartiallySplit();
 
   mutable bool CalledGetMaxTasks = false;
 
   // Use Optional to distinguish false from not yet initialized.
   Optional<bool> EnableSplitLTOUnit;

llvm/include/llvm/Support/Parallel.h

class TaskGroup {
   bool Parallel;
 
 public:
   TaskGroup();
   ~TaskGroup();
 
   void spawn(std::function<void()> f);
 
+  // Provide a hook to execute behavior before & after the spawned functions.
+  // This is needed to setup thread_local state.
+  static void wrapThreadState(std::function<void(std::function<void()> &)> f);
+
   void sync() const { L.sync(); }
 };
 
 const ptrdiff_t MinParallelSize = 1024;
 
 /// Inclusive median.
 template <class RandomAccessIterator, class Comparator>
 RandomAccessIterator medianOf3(RandomAccessIterator Start,

llvm/lib/LTO/LTO.cpp

for (auto &S : P.second.SummaryList) {
         return make_error<StringError>(
             "inconsistent LTO Unit splitting (recompile with -fsplit-lto-unit)",
             inconvertibleErrorCode());
     }
   }
   return Error::success();
 }
 
-Error LTO::run(AddStreamFn AddStream, NativeObjectCache Cache) {
+Error LTO::run(AddStreamFn AddStream, NativeObjectCache Cache,
+               ThreadInitFn ThreadInit) {
   // Compute "dead" symbols, we don't want to import/export these!
   DenseSet<GlobalValue::GUID> GUIDPreservedSymbols;
   DenseMap<GlobalValue::GUID, PrevailingType> GUIDPrevailingResolutions;
   for (auto &Res : GlobalResolutions) {
     // Normally resolution have IR name of symbol. We can do nothing here
     // otherwise. See comments in GlobalResolution struct for more details.
     if (Res.second.IRName.empty())
       continue;
Error LTO::run(AddStreamFn AddStream, NativeObjectCache Cache,
   // Setup output file to emit statistics.
   auto StatsFileOrErr = setupStatsFile(Conf.StatsFile);
   if (!StatsFileOrErr)
     return StatsFileOrErr.takeError();
   std::unique_ptr<ToolOutputFile> StatsFile = std::move(StatsFileOrErr.get());
 
   Error Result = runRegularLTO(AddStream);
   if (!Result)
-    Result = runThinLTO(AddStream, Cache, GUIDPreservedSymbols);
+    Result = runThinLTO(AddStream, Cache, ThreadInit, GUIDPreservedSymbols);
 
   if (StatsFile)
     PrintStatisticsJSON(StatsFile->os());
 
   return Result;
 }
 
 Error LTO::runRegularLTO(AddStreamFn AddStream) {
public:
   virtual unsigned getThreadCount() = 0;
 };
 
 namespace {
 class InProcessThinBackend : public ThinBackendProc {
   ThreadPool BackendThreadPool;
   AddStreamFn AddStream;
   NativeObjectCache Cache;
+  ThreadInitFn ThreadInit;
   std::set<GlobalValue::GUID> CfiFunctionDefs;
   std::set<GlobalValue::GUID> CfiFunctionDecls;
 
   Optional<Error> Err;
   std::mutex ErrMu;
 
 public:
   InProcessThinBackend(
       const Config &Conf, ModuleSummaryIndex &CombinedIndex,
       ThreadPoolStrategy ThinLTOParallelism,
       const StringMap<GVSummaryMapTy> &ModuleToDefinedGVSummaries,
-      AddStreamFn AddStream, NativeObjectCache Cache)
+      AddStreamFn AddStream, NativeObjectCache Cache, ThreadInitFn ThreadInit)
       : ThinBackendProc(Conf, CombinedIndex, ModuleToDefinedGVSummaries),
         BackendThreadPool(ThinLTOParallelism), AddStream(std::move(AddStream)),
-        Cache(std::move(Cache)) {
+        Cache(std::move(Cache)), ThreadInit(std::move(ThreadInit)) {
     for (auto &Name : CombinedIndex.cfiFunctionDefs())
       CfiFunctionDefs.insert(
           GlobalValue::getGUID(GlobalValue::dropLLVMManglingEscape(Name)));
     for (auto &Name : CombinedIndex.cfiFunctionDecls())
       CfiFunctionDecls.insert(
           GlobalValue::getGUID(GlobalValue::dropLLVMManglingEscape(Name)));
   }
 
   Error runThinLTOBackendThread(
-      AddStreamFn AddStream, NativeObjectCache Cache, unsigned Task,
-      BitcodeModule BM, ModuleSummaryIndex &CombinedIndex,
+      AddStreamFn AddStream, NativeObjectCache Cache, ThreadInitFn ThreadInit,
+      unsigned Task, BitcodeModule BM, ModuleSummaryIndex &CombinedIndex,
       const FunctionImporter::ImportMapTy &ImportList,
       const FunctionImporter::ExportSetTy &ExportList,
       const std::map<GlobalValue::GUID, GlobalValue::LinkageTypes> &ResolvedODR,
       const GVSummaryMapTy &DefinedGlobals,
       MapVector<StringRef, BitcodeModule> &ModuleMap) {
+
+    // Ensure the client thread init function is called before anything else.
+    if (ThreadInit)
+      ThreadInit();
+
     auto RunThinBackend = [&](AddStreamFn AddStream) {
       LTOLLVMContext BackendContext(Conf);
       Expected<std::unique_ptr<Module>> MOrErr = BM.parseModule(BackendContext);
       if (!MOrErr)
         return MOrErr.takeError();
 
       return thinBackend(Conf, Task, AddStream, **MOrErr, CombinedIndex,
                          ImportList, DefinedGlobals, &ModuleMap);
BackendThreadPool.async(
             const std::map<GlobalValue::GUID, GlobalValue::LinkageTypes>
                 &ResolvedODR,
             const GVSummaryMapTy &DefinedGlobals,
             MapVector<StringRef, BitcodeModule> &ModuleMap) {
           if (LLVM_ENABLE_THREADS && Conf.TimeTraceEnabled)
             timeTraceProfilerInitialize(Conf.TimeTraceGranularity,
                                         "thin backend");
           Error E = runThinLTOBackendThread(
-              AddStream, Cache, Task, BM, CombinedIndex, ImportList, ExportList,
-              ResolvedODR, DefinedGlobals, ModuleMap);
+              AddStream, Cache, ThreadInit, Task, BM, CombinedIndex, ImportList,
+              ExportList, ResolvedODR, DefinedGlobals, ModuleMap);
           if (E) {
             std::unique_lock<std::mutex> L(ErrMu);
             if (Err)
               Err = joinErrors(std::move(*Err), std::move(E));
             else
               Err = std::move(E);
           }
           if (LLVM_ENABLE_THREADS && Conf.TimeTraceEnabled)
unsigned getThreadCount() override {
     return BackendThreadPool.getThreadCount();
   }
 };
 } // end anonymous namespace
 
 ThinBackend lto::createInProcessThinBackend(ThreadPoolStrategy Parallelism) {
   return [=](const Config &Conf, ModuleSummaryIndex &CombinedIndex,
              const StringMap<GVSummaryMapTy> &ModuleToDefinedGVSummaries,
-             AddStreamFn AddStream, NativeObjectCache Cache) {
+             AddStreamFn AddStream, NativeObjectCache Cache,
+             ThreadInitFn ThreadInit) {
     return std::make_unique<InProcessThinBackend>(
         Conf, CombinedIndex, Parallelism, ModuleToDefinedGVSummaries, AddStream,
-        Cache);
+        Cache, ThreadInit);
   };
 }
 
 // Given the original \p Path to an output file, replace any path
 // prefix matching \p OldPrefix with \p NewPrefix. Also, create the
 // resulting directory if it does not yet exist.
 std::string lto::getThinLTOOutputFile(const std::string &Path,
                                       const std::string &OldPrefix,

 };
 } // end anonymous namespace
 
 ThinBackend lto::createWriteIndexesThinBackend(
     std::string OldPrefix, std::string NewPrefix, bool ShouldEmitImportsFiles,
     raw_fd_ostream *LinkedObjectsFile, IndexWriteCallback OnWrite) {
   return [=](const Config &Conf, ModuleSummaryIndex &CombinedIndex,
              const StringMap<GVSummaryMapTy> &ModuleToDefinedGVSummaries,
-             AddStreamFn AddStream, NativeObjectCache Cache) {
+             AddStreamFn AddStream, NativeObjectCache Cache,
+             ThreadInitFn ThreadInit) {
     return std::make_unique<WriteIndexesThinBackend>(
         Conf, CombinedIndex, ModuleToDefinedGVSummaries, OldPrefix, NewPrefix,
         ShouldEmitImportsFiles, LinkedObjectsFile, OnWrite);
   };
 }
 
 Error LTO::runThinLTO(AddStreamFn AddStream, NativeObjectCache Cache,
+                      ThreadInitFn ThreadInit,
                       const DenseSet<GlobalValue::GUID> &GUIDPreservedSymbols) {
   timeTraceProfilerBegin("ThinLink", StringRef(""));
   auto TimeTraceScopeExit = llvm::make_scope_exit([]() {
     if (llvm::timeTraceProfilerEnabled())
       llvm::timeTraceProfilerEnd();
   });
   if (ThinLTO.ModuleMap.empty())
     return Error::success();
Error LTO::runThinLTO(AddStreamFn AddStream, NativeObjectCache Cache,
 
   if (llvm::timeTraceProfilerEnabled())
     llvm::timeTraceProfilerEnd();
 
   TimeTraceScopeExit.release();
 
   std::unique_ptr<ThinBackendProc> BackendProc =
       ThinLTO.Backend(Conf, ThinLTO.CombinedIndex, ModuleToDefinedGVSummaries,
-                      AddStream, Cache);
+                      AddStream, Cache, ThreadInit);
 
   auto &ModuleMap =
       ThinLTO.ModulesToCompile ? *ThinLTO.ModulesToCompile : ThinLTO.ModuleMap;
 
   auto ProcessOneModule = [&](int I) -> Error {
     auto &Mod = *(ModuleMap.begin() + I);
     // Tasks 0 through ParallelCodeGenParallelismLevel-1 are reserved for
     // combined module and parallel code generation partitions.

llvm/lib/Support/Parallel.cpp

 #if LLVM_ENABLE_THREADS
 
 namespace llvm {
 namespace parallel {
 namespace detail {
 
 namespace {
 
-/// An abstract class that takes closures and runs them asynchronously.
-class Executor {
-public:
-  virtual ~Executor() = default;
-  virtual void add(std::function<void()> func) = 0;
-
-  static Executor *getDefaultExecutor();
-};
-
-/// An implementation of an Executor that runs closures on a thread pool
-///   in filo order.
-class ThreadPoolExecutor : public Executor {
+/// Takes closures and runs them asynchronously on a thread pool in FILO order.
+class ThreadPoolExecutor final {
 public:
   explicit ThreadPoolExecutor(ThreadPoolStrategy S = hardware_concurrency()) {
     unsigned ThreadCount = S.compute_thread_count();
     // Spawn all but one of the threads in another thread as spawning threads
     // can take a while.
     Threads.reserve(ThreadCount);
     Threads.resize(1);
     std::lock_guard<std::mutex> Lock(Mutex);
void stop() {
       if (Stop)
         return;
       Stop = true;
     }
     Cond.notify_all();
     ThreadsCreated.get_future().wait();
   }
 
-  ~ThreadPoolExecutor() override {
+  ~ThreadPoolExecutor() {
     stop();
     std::thread::id CurrentThreadId = std::this_thread::get_id();
     for (std::thread &T : Threads)
       if (T.get_id() == CurrentThreadId)
         T.detach();
       else
         T.join();
   }
 
   struct Creator {
     static void *call() { return new ThreadPoolExecutor(strategy); }
   };
   struct Deleter {
     static void call(void *Ptr) { ((ThreadPoolExecutor *)Ptr)->stop(); }
   };
 
-  void add(std::function<void()> F) override {
+  void add(std::function<void()> F) {
     {
       std::lock_guard<std::mutex> Lock(Mutex);
       WorkStack.push(F);
     }
     Cond.notify_one();
   }
 
+  std::function<void(std::function<void()> &)> ThreadStateFn;
+
 private:
   void work(ThreadPoolStrategy S, unsigned ThreadID) {
     S.apply_thread_strategy(ThreadID);
     while (true) {
       std::unique_lock<std::mutex> Lock(Mutex);
       Cond.wait(Lock, [&] { return Stop || !WorkStack.empty(); });
       if (Stop)
         break;
       auto Task = WorkStack.top();
       WorkStack.pop();
       Lock.unlock();
-      Task();
+      ThreadStateFn ? ThreadStateFn(Task) : Task();
     }
   }
 
   std::atomic<bool> Stop{false};
   std::stack<std::function<void()>> WorkStack;
   std::mutex Mutex;
   std::condition_variable Cond;
   std::promise<void> ThreadsCreated;
   std::vector<std::thread> Threads;
 };
 
-Executor *Executor::getDefaultExecutor() {
+static ThreadPoolExecutor *getDefaultExecutor() {
   // The ManagedStatic enables the ThreadPoolExecutor to be stopped via
   // llvm_shutdown() which allows a "clean" fast exit, e.g. via _exit(). This
   // stops the thread pool and waits for any worker thread creation to complete
   // but does not wait for the threads to finish. The wait for worker thread
   // creation to complete is important as it prevents intermittent crashes on
   // Windows due to a race condition between thread creation and process exit.
   //
   // The ThreadPoolExecutor will only be destroyed when the static unique_ptr to

 // lock, only allow the first TaskGroup to run tasks parallelly. In the scenario
 // of nested parallel_for_each(), only the outermost one runs parallelly.
 TaskGroup::TaskGroup() : Parallel(TaskGroupInstances++ == 0) {}
 TaskGroup::~TaskGroup() { --TaskGroupInstances; }
 
 void TaskGroup::spawn(std::function<void()> F) {
   if (Parallel) {
     L.inc();
-    Executor::getDefaultExecutor()->add([&, F] {
+    getDefaultExecutor()->add([&, F] {
       F();
       L.dec();
     });
   } else {
     F();
   }
 }
 
+void TaskGroup::wrapThreadState(
+    std::function<void(std::function<void()> &)> F) {
+  getDefaultExecutor()->ThreadStateFn = F;
+}
+
 } // namespace detail
 } // namespace parallel
 } // namespace llvm
 #endif // LLVM_ENABLE_THREADS