[WebAssembly] Implement thread-local storage (local-exec model)

[quantum]

Summary

Thread local variables are placed inside a .tdata segment. Their symbols are
offsets from the start of the segment. The address of a thread local variable
is computed as __tls_base + the offset from the start of the segment.

.tdata segment is a passive segment and memory.init is used once per thread
to initialize the thread local storage.

__tls_base is a wasm global. Since each thread has its own wasm instance,
it is effectively thread local. Currently, __tls_base must be initialized
at thread startup, and so cannot be used with dynamic libraries.

__tls_base is to be initialized with a new linker-synthesized function,
__wasm_init_tls, which takes as an argument a block of memory to use as the
storage for thread locals. It then initializes the block of memory and sets
__tls_base. As __wasm_init_tls will handle the memory initialization,
the memory does not have to be zeroed.

To help allocating memory for thread-local storage, a new compiler intrinsic
is introduced: __builtin_wasm_tls_size(). This instrinsic function returns
the size of the thread-local storage for the current function.

The expected usage is to run something like the following upon thread startup:

__wasm_init_tls(malloc(__builtin_wasm_tls_size()));

[quantum]

Need to add tests for the compiler intrinsic __builtin_wasm_tls_size.

[quantum]

Add tests for __builtin_wasm_tls_size.

[quantum]

Fix fail call.

[sunfish]

This looks nice!

wasm_init_tls(calloc(builtin_wasm_tls_size(), 1));

Would it make sense to change the API contract for __wasm_init_tls to guarantee that initializes all bytes (with zeros as needed)? __wasm_init_tls knows what bytes it's initializing, so we could use plain malloc instead of calloc and avoid redundant zero initialization of those bytes.

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

1111 ↗

(On Diff #209077)

It looks like this supports local-exec, but would need to be extended to handle initial-exec or the other TLS models. Assuming this is correct, could you add a check for the TLS model and report_fatal_error on unsupported models?

[quantum]

Fix linker warning.

[quantum]

In D64537#1579626, @sunfish wrote:

This looks nice!

wasm_init_tls(calloc(builtin_wasm_tls_size(), 1));

Would it make sense to change the API contract for __wasm_init_tls to guarantee that initializes all bytes (with zeros as needed)? __wasm_init_tls knows what bytes it's initializing, so we could use plain malloc instead of calloc and avoid redundant zero initialization of those bytes.

Updated the summary. I just realized that the way the code is written, the plain malloc would already work.

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

1111 ↗

(On Diff #209077)

I'll add an error for non-local-exec for now.

[aheejin]

Nice!
Then where should we call __wasm_init_tls, in case within a library?

clang/include/clang/Basic/BuiltinsWebAssembly.def

33

Why is it c(const)? According to this comment, this is true if this function has no side effects and doesn't read memory, i.e., the result should be only dependent on its arguments. Can't wasm globals be memory locations in machines?

lld/test/wasm/tls.ll

58

Hmm, I think there might be a way to actually print disassembly results. There are '*.test' files in test directory, in which we have some examples. For example, this test has a sequence of commands you want to run, and you can put input files in a separate directory. That test uses obj2yaml, but can we possibly use llvm-objdump or something to disassemble?

lld/wasm/Driver.cpp

540

Does this TLS thing work when Config->Shared == true, i.e., do we create TLS globals even when this is a library?

lld/wasm/Writer.cpp

431

Should we check for "mutable-global" feature too?

514

Now __tls_base is also mutable, I think we should fix the comment

769

Any reason for the new block scope?

777

Is it guaranteed that there's only one TLS segment?

llvm/include/llvm/IR/IntrinsicsWebAssembly.td

134

• Why is it speculatable?
• I'm not sure if it's InstNoMem, because wasm globals can be memory locations after all. What do other people think?

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

1095 ↗

(On Diff #209102)

If you do the conversion in WebAssemblyISelDAGToDAG.cpp, you don't need to create WebAssemblyISD node, SDTypeProfile, and SDNode for every single instruction you want to generate. This WebAssemblyISelLowering.cpp is a part of legalization so it cannot generate machine instructions directly, whereas WebAssemblyISelDAGToDAG.cpp is a part of instruction selection so you have direct access to machine instructions. I think moving routines there can be cleaner?

1109 ↗

(On Diff #209102)

Does this offset work when there are non-thread-local globals too?

llvm/lib/Target/WebAssembly/WebAssemblyInstrInfo.td

86 ↗

(On Diff #209102)

Nit: indentation

124 ↗

(On Diff #209102)

Nit: indentations look weird for both defs

llvm/lib/Target/WebAssembly/WebAssemblyTargetMachine.cpp

192

Looks like this feature requires both bulk memory and mutable global. Shouldn't we strip thread locals if either is not enabled?

llvm/test/CodeGen/WebAssembly/target-features-tls.ll

21–27

Is thread local feature still dependent on atomics? If not, do we still need to test this with both atomics on and off? This now looks rather dependent on bulk memory and mutable global. Should we test this feature with those options instead? (I can be mistaken, so please let me know if so)

llvm/test/CodeGen/WebAssembly/tls.ll

15

If the only difference of O0 and O2 results is the order of global.get and i32.const, you can use [[ https://llvm.org/docs/CommandGuide/FileCheck.html#the-check-dag-directive | CHECK-DAG ]] directive, like

; CHECK-DAG: global.get __tls_base
; CHECK-DAG: i32.const tls
...

The same for other functions too. Maybe we don't need separate O0 and O2 testing in this file.

[quantum]

Add error for non-local-exec TLS models. Fix unit tests.

[quantum]

I'll clean up the table gen stuff.

clang/include/clang/Basic/BuiltinsWebAssembly.def

33

I was thinking that since the global is immutable, so the value is always a constant.

lld/test/wasm/tls.ll

58

We already know that we can do something like

Run: obj2yaml %t.wasm | sed -n '/Body:/{s/^\s*Body:\s*//;s/../0x& /gp}'  | llvm-mc -disassemble -triple=wasm32

to compare the actual assembly. As for llvm-objdump, it seems to be unable to disassemble the WASM properly:

.../tools/lld/test/wasm/Output/tls.ll.tmp.wasm:	file format WASM


Disassembly of section CODE:

00000000 CODE:
        # 4 functions in section.
       1: 02 00                        	block   	invalid_type
       3: 0b                           	end
       4: 10 00                        	call	0
       6: 20 00                        	local.get	0
       8: 24 01                        	global.set	1
       a: 20 00                        	local.get	0
       c: 41 00                        	i32.const	0
       e: 41 08                        	i32.const	8
      10: fc 08 00 00                  	memory.init	0, 0
      14: 0b                           	end
      15: 0f                           	return
      16: 00                           	llvm-objdump: lib/Target/WebAssembly/WebAssemblyGenAsmWriter.inc:2032: void llvm::WebAssemblyInstPrinter::printInstruction(const llvm::MCInst *, llvm::raw_ostream &): Assertion `Bits != 0 && "Cannot print this instruction."' failed.

lld/wasm/Driver.cpp

540

Since TLS is module specific (we can't allocate memory for other modules), we must in fact generate this symbol for every module. Shared library support will not be implemented in this diff, however.

lld/wasm/Writer.cpp

431

Do we need to? I thought it's always available since we use it for the stack pointer.

514

Will do.

769

Yes, the raw_string_ostream must destruct by the end of the scope. This is the pattern used in other functions above.

777

Yes, all the TLS input segments will be merged into .tdata.

llvm/include/llvm/IR/IntrinsicsWebAssembly.td

134

@tlively suggested that I do this. It should be speculatable because it has no side effects (since it returns a constant). As for InstrNoMem, I am not sure if globals are memory, and whether reading a constant counts as memory access.

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

1095 ↗

(On Diff #209102)

It seems most other architectures do it in *TargetLowering, so I decided to do the same.

1109 ↗

(On Diff #209102)

Yes, the linker handles the offsets. All thread locals are grouped together by the linker.

llvm/lib/Target/WebAssembly/WebAssemblyTargetMachine.cpp

192

Do we want to do that? I think having your thread locals vanish if you didn't pass -mbulk-memory is surprising. It might make more sense to strip thread locals if -pthread is not passed.

llvm/test/CodeGen/WebAssembly/target-features-tls.ll

21–27

On second thought, this test is somewhat meaningless now, since the thread local is always created regardless of whether atomics are enabled. What does @tlively think?

llvm/test/CodeGen/WebAssembly/tls.ll

15

Will do. I didn't know CHECK-DAG existed.

[quantum]

Update comments regarding multable globals. Use CHECK-DAG.

[quantum]

Clean up table gen.

[quantum]

clang/include/clang/Basic/BuiltinsWebAssembly.def

33

According to @tlively, there is no solid definition on whether a global (especially a constant one), counts as memory access. For now, I am going to change this to not constant. We can always change it back later.

llvm/include/llvm/IR/IntrinsicsWebAssembly.td

134

I think I am going to remove these attributes for now. We can add them back if we end up deciding that immutable globals are not memory accesses.

[quantum]

Treat global.get of an immutable global as a memory access for now.

[tlively]

clang/include/clang/Basic/BuiltinsWebAssembly.def

33

I think this requires more conversation.

lld/test/wasm/data-layout.ll

43 ↗

(On Diff #209274)

These globals don't have enough information to tell the reader what they even are, and they don't have anything to do with the data layout, so how about skipping these in the test with a comment saying what is being skipped?

lld/test/wasm/tls.ll

58

It might be worth filing an LLVM bug for this (or possibly fixing in a separate CL).

lld/wasm/Symbols.cpp

207

It would be great to have an explanatory comment here.

lld/wasm/Writer.cpp

243

What happens when there are multiple TLS sections and --no-merge-data-segments is used? I assume their sizes should be added together?

643

Does this mean we can't control whether .tdata or .tbss comes first? Is that important for anything?

777

--no-merge-data-segments!

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

1095 ↗

(On Diff #209102)

It would be good to try @aheejin's suggestion out and see if it leads to cleaner code. Cargo culting other targets is a good way to get started, but we should still try to make our code as simple as possible.

llvm/lib/Target/WebAssembly/WebAssemblyInstrInfo.td

124 ↗

(On Diff #209102)

I agree with @aheejin that it would be ideal if we didn't need to add these new WebAssemblyISD items. We've come this far without needing them, so it would be good to keep that going.

llvm/lib/Target/WebAssembly/WebAssemblyTargetMachine.cpp

192

I have a patch up to make -pthread imply -mbulk-memory, so this shouldn't be a problem from a UI perspective. I agree with @aheejin that we should strip TLS if the necessary features aren't present to preserver feature-correctness.

All the errors for using TLS without bulk memory should also apply to mutable-globals, which I hadn't considered before. I will update my patch so -pthread implies -mmutable-globals as well.

llvm/test/CodeGen/WebAssembly/target-features-tls.ll

21–27

I agree with @aheejin that we should change the test to toggle bulk-memory and mutable-global and strip TLS if those are not both present.

llvm/test/CodeGen/WebAssembly/tls.ll

15

If the point of testing with -O0 and -O2 is to test both with and without FastIsel, it might be better to pass -fast-isel to explicitly enable it instead.

[quantum]

Rebased

[quantum]

clang/include/clang/Basic/BuiltinsWebAssembly.def

33

We decided that this is really a constant, since this global is immutable.

lld/test/wasm/data-layout.ll

43 ↗

(On Diff #209274)

Going to skip over __tls_base and __tls_size.

lld/test/wasm/tls.ll

58

Going to fix this at some point in the future.

lld/wasm/Symbols.cpp

207

Adding a comment.

lld/wasm/Writer.cpp

243

I'll merge the TLS segments anyways despite --no-merge-data-segments.

431

On second thought, the mutable-global feature is for import/export of mutable globals. TLS does not need to do this.

643

Yes, it does mean that. The only reason why .tbss is supposed to be separate is so that its memory can just be zeroed whereas .tdata has to have the bytes stored in the program image. Currently, we just explicitly store the zero bytes, so this won't be a problem.

777

I am making --no-merge-data-segments merge TLS segments anyway.

llvm/include/llvm/IR/IntrinsicsWebAssembly.td

134

We decided that this is a constant after all, since the global is immutable. This also simplifies the implementation of the lowering code.

llvm/lib/Target/WebAssembly/WebAssemblyInstrInfo.td

124 ↗

(On Diff #209102)

I'll try out an implementation with ISelDAGtoDAG later.

llvm/lib/Target/WebAssembly/WebAssemblyTargetMachine.cpp

192

Going to strip out TLS if bulk-memory is not enabled.

llvm/test/CodeGen/WebAssembly/target-features-tls.ll

21–27

Changed test to test +/-bulk-memory.

llvm/test/CodeGen/WebAssembly/tls.ll

15

Changing to use -fast-isel.

[quantum]

Apply review feedback

[sbc100]

I wonder if __tls_base should be allocated by the embedder (or by the parent/creator thread). Then it could be an *immutable* global import that is allocated up front. I guess __stack_pointer should be treated in the same way (except mutable).

I don't want to block this change on this, but just putting the idea out there.

lld/test/wasm/data-layout.ll

43 ↗

(On Diff #209274)

Is there any reason these symbols need to exist in non-threaded builds. The cost of two extra globals is small but not nothing.

I'd also rather not skip over them here. If TLS changes I'd like to see this test need updating.

lld/test/wasm/data-segment-merging.ll

32 ↗

(On Diff #209391)

Again, can we avoid creating this completely for non-threaded builds?

lld/test/wasm/tls.ll

58

This is known issue with the disassembler in that it doesn't know where the functions start and stop in executable output, only in object files.

lld/wasm/Writer.cpp

629

Maybe write this as a single conditional so its clear even without this comment?

[sbc100]

Its really great to see this change BTW! Thanks.

[quantum]

Guard __wasm_init_tls and TLS globals behind a flag (currently --shared-memory).

@tlively and I discussed offline and agreed that it's probably best to use the existing flag instead of adding another flag that needs to be passed when linking multi-threaded executables.

[quantum]

lld/wasm/Writer.cpp

629

Changed in latest diff.

[quantum]

Switch to using ISelDAGToDAG instead of ISelLowering

[quantum]

Don't show a crash report for incorrect flags to use TLS

[quantum]

@sbc100 As for __tls_base being immutable, I am not sure how this will interact with dynamic linking once we implement it. On native platforms, dlopened libraries have their TLS blocks allocated on first use.

What exactly happens to dlopened libraries when there are already threads around? Would it be feasible to create TLS blocks for when the library is loaded on existing thread?

[dschuff]

If there's any chance this TLS ABI could be useful for WASI (I don't know if there's been any WASI work on threads yet, but it seems like there's no reason it couldn't be), then we should start a doc in tool-conventions for it. If not then we should get it behind the emscripten OS in LLVM. (and document it anyway; either in tool-conventions or somewhere in the emscripten site).

[dschuff]

Oh, btw, any reason these have to be passive segments? Why can't we just make them active segments and let the VM initialize them for us?

[quantum]

In D64537#1584069, @dschuff wrote:

Oh, btw, any reason these have to be passive segments? Why can't we just make them active segments and let the VM initialize them for us?

These need to be passive segments so that we can initialize it in different locations in linear memory for every thread.

If we had multiple memories, TLS could have been implemented as a separate linear memory, but we are stuck with mallocing and initializing for now.

In D64537#1584067, @dschuff wrote:

If there's any chance this TLS ABI could be useful for WASI (I don't know if there's been any WASI work on threads yet, but it seems like there's no reason it couldn't be), then we should start a doc in tool-conventions for it. If not then we should get it behind the emscripten OS in LLVM. (and document it anyway; either in tool-conventions or somewhere in the emscripten site).

Documenting this in tool-conventions is planned.

[quantum]

Use signed LEB128 for i32.const. Basically, do what D64612 did.

[aheejin]

lld/test/wasm/data-segments.ll

7

What is the difference between ACTIVE and ACTIVE-TLS? It looks we don't have different build processes for them. And as what @sbc100 said, can we exclude TLS from build?

lld/test/wasm/tls.ll

6

Can we possibly mix one non-tls global variable between tls1 and tls2, just to see they work together?

lld/wasm/Writer.cpp

431

Ah you're right.

514

PIng. It doesn't look like fixed yet

629

Has it been reflected? It says it has been changed but it doesn't look like it has

llvm/lib/Target/WebAssembly/WebAssemblyISelDAGToDAG.cpp

203

These two lines can be shortened to

ReplaceNode(Node, TLSAddress);

The same applies to the code below for __tls_size too.

208

Nit: Might be a bit cleaner to extract the expression, which is complicated, like

unsigned IntNo = cast<ConstantSDNode>(Node->getOperand(0))->getZExtValue());
switch (IntNo) {
 ...

[quantum]

Change per review feedback.

lld/test/wasm/data-segments.ll

7

ACTIVE-TLS is for builds with TLS enabled. Currently, we use --shared-memory to determine that, per @tlively's recommendation. The rationale is that we don't want even more flags that need to be passed in a proper threaded build.

lld/test/wasm/tls.ll

6

Done.

lld/wasm/Writer.cpp

514

Looks like the fix got rebasing against the change that made everything lowerCamelCase. Going to do it again.

629

I think you are commenting on an old version? It is a single condition on the latest version.

llvm/lib/Target/WebAssembly/WebAssemblyISelDAGToDAG.cpp

203

Nice. I didn't know that was possible. Changing.

208

Done.

[aheejin]

lld/test/wasm/data-segments.ll

7

Then if we don't enable --shared-memory, we don't generate those globals? Do we have a test for that?

[tlively]

lld/wasm/Driver.cpp

540

Until we do implement TLS for shared modules, would it make sense to omit these globals and function from shared modules, since we can't use them in that context anyway?

lld/wasm/Writer.cpp

643

It would be really great if we could find a way to elide the .bss 0 bytes as a code size optimization. Since we can't assume that the memory is already zeroed the way we can with passive segments, perhaps we can use a memory.fill instruction to zero the memory? Pursuing this in a follow-on CL should be fine.

805

You could avoid duplicating these lines by making them unconditional.

905

Can you remind me how the InitTLSFunction interacts with relocatable code? I'm wondering if this should be called up in the condition with the other synthetic functions.

llvm/lib/Target/WebAssembly/WebAssemblyTargetMachine.cpp

193

I just realized that if we have atomics but not bulk memory and TLS is stripped, then we will be in the awkward situation of both using atomics and disallowing atomics because the module is not thread safe. I think the best solution would be to go back and forcibly strip both atomics and TLS if either of them would be stripped.

llvm/test/CodeGen/WebAssembly/tls.ll

2

It would be good to check the negative case, too, i.e with bulk-memory disabled.

6

Is CHECK still used as a prefix if it not listed in the invocation of FileCheck?

[quantum]

Deal with review comments.

lld/test/wasm/data-segments.ll

7

Yes. In this file, the cases with --check-prefix ACTIVE will ensure that the fields are not generated.

lld/wasm/Driver.cpp

540

That would make sense for now.

lld/wasm/Writer.cpp

643

Yes, this is a good idea for a follow-on CL.

805

Right.

905

I don't think it should. __wasm_init_tls initializes the TLS block for the current module only, so every shared library needs to have its own __wasm_init_tls.

llvm/lib/Target/WebAssembly/WebAssemblyTargetMachine.cpp

193

Done.

llvm/test/CodeGen/WebAssembly/tls.ll

2

Done.

6

Yes, the default prefix is CHECK.

[quantum]

Fix duplicate end function

[dschuff]

The offset field of a segment can be a constant expression which can be a global.get of an imported global. So we could have an imported global __tls_base which is different for each thread, and have an active segment with that as its segment offset?

[dschuff]

Another high-level question (based just on reading the CL description): The TLS-size intrinsic is per-function, does that mean that the tls-init function is called for every function? are there just multiple TLS sections per object file?

[quantum]

In D64537#1586577, @dschuff wrote:

Another high-level question (based just on reading the CL description): The TLS-size intrinsic is per-function, does that mean that the tls-init function is called for every function? are there just multiple TLS sections per object file?

The TLS-size intrinsic returns the total size of TLS for the module it's called from. The initialization function initializes the TLS for the entire module.

In D64537#1586556, @dschuff wrote:

The offset field of a segment can be a constant expression which can be a global.get of an imported global. So we could have an imported global __tls_base which is different for each thread, and have an active segment with that as its segment offset?

I didn't know that it could have been a constant expression. I don't think this would have worked very well on the main thread though, since we need to run malloc before we can compute __tls_base. I think this requires the global to be mutable, if only because we need to be able to initialize it on the main thread.

[aheejin]

Where should we call __wasm_init_tls, in case within a library?

[quantum]

In D64537#1586809, @aheejin wrote:

Where should we call __wasm_init_tls, in case within a library?

Dynamic libraries are not supported by the local-exec TLS model. Support will be implemented later. The idea is that we'll have an __ensure_tls_initialized function that will malloc and __wasm_init_tls if __tls_base == 0.

[tlively]

LGTM apart from one last comment

llvm/lib/Target/WebAssembly/WebAssemblyTargetMachine.cpp

193

We can be more fine grained than this. We should strip atomics or TLS only if the relevant feature is missing or the other one is stripped. It is possible to have thread-locals and bulk memory but not have any atomics to strip.

[quantum]

More fine-grainted stripping.

[sbc100]

lld/wasm/Writer.cpp

400

Right now we always compiler with -fdata-sections, but if we didn't then object files might have just one ".tdata" sections. Perhaps drop the final "."?

[quantum]

Removed the trailing .. I would add a test to make sure things work without -fdata-sections, but this->Options.DataSections = true; is hard-coded in WebAssemblyTargetMachine.cpp.

[quantum]

Remove extra braces

[quantum]

Disable atomics when TLS is stripped

[tlively]

Nice work!

[dschuff]

I had a reply that got eaten here, so I'm going to keep trolling you on your CL since we don't have a design doc for this.
The offset field of a data segment initializer can be a global.get on an imported global. (https://webassembly.github.io/spec/core/valid/instructions.html#constant-expressions). Since each thread is separately instantiated with separate JS, we could have a global import like __tls_base which has a different value in each thread. Then we wouldn't need to manually call the init code anywhere. Would there be other advantages or disadvantages for that?

[quantum]

In D64537#1588699, @dschuff wrote:

I had a reply that got eaten here, so I'm going to keep trolling you on your CL since we don't have a design doc for this.
The offset field of a data segment initializer can be a global.get on an imported global. (https://webassembly.github.io/spec/core/valid/instructions.html#constant-expressions). Since each thread is separately instantiated with separate JS, we could have a global import like __tls_base which has a different value in each thread. Then we wouldn't need to manually call the init code anywhere. Would there be other advantages or disadvantages for that?

I already answered this one:

In D64537#1586614, @quantum wrote:

In D64537#1586556, @dschuff wrote:

The offset field of a segment can be a constant expression which can be a global.get of an imported global. So we could have an imported global __tls_base which is different for each thread, and have an active segment with that as its segment offset?

I didn't know that it could have been a constant expression. I don't think this would have worked very well on the main thread though, since we need to run malloc before we can compute __tls_base. I think this requires the global to be mutable, if only because we need to be able to initialize it on the main thread.

The problem I found with the import approach is that the main thread cannot use global.get of an imported global to get the location of TLS, mainly because it needs to be malloc'd. To use malloc, the main thread needs to be initialized. This will result in a circular dependency.

[sbc100]

In D64537#1586614, @quantum wrote:

In D64537#1586577, @dschuff wrote:

Another high-level question (based just on reading the CL description): The TLS-size intrinsic is per-function, does that mean that the tls-init function is called for every function? are there just multiple TLS sections per object file?

The TLS-size intrinsic returns the total size of TLS for the module it's called from. The initialization function initializes the TLS for the entire module.

In D64537#1586556, @dschuff wrote:

The offset field of a segment can be a constant expression which can be a global.get of an imported global. So we could have an imported global __tls_base which is different for each thread, and have an active segment with that as its segment offset?

I didn't know that it could have been a constant expression. I don't think this would have worked very well on the main thread though, since we need to run malloc before we can compute __tls_base. I think this requires the global to be mutable, if only because we need to be able to initialize it on the main thread.

Interesting. I was assuming we could be clever and make the main thread special and have the linker allocate its tls space upfront (like we do for main thread stack space) rather than do it a runtime, but maybe that needlessly complicated. Seems like ideally we could want to import __tls_base but not for the main thread but I don't think of a way to express that.

clang/include/clang/Basic/BuiltinsWebAssembly.def

 // with respect to memory.grow calls.
 BUILTIN(__builtin_wasm_memory_size, "zIi", "n")
 BUILTIN(__builtin_wasm_memory_grow, "zIiz", "n")
 
 // Bulk memory builtins
 TARGET_BUILTIN(__builtin_wasm_memory_init, "vIUiIUiv*UiUi", "", "bulk-memory")
 TARGET_BUILTIN(__builtin_wasm_data_drop, "vIUi", "", "bulk-memory")
 
+// Thread-local storage
+TARGET_BUILTIN(__builtin_wasm_tls_size, "z", "nc", "bulk-memory")
+
 // Floating point min/max
 BUILTIN(__builtin_wasm_min_f32, "fff", "nc")
 BUILTIN(__builtin_wasm_max_f32, "fff", "nc")
 BUILTIN(__builtin_wasm_min_f64, "ddd", "nc")
 BUILTIN(__builtin_wasm_max_f64, "ddd", "nc")
 
 // Exception handling builtins.
 TARGET_BUILTIN(__builtin_wasm_throw, "vIUiv*", "r", "exception-handling")

clang/lib/CodeGen/CGBuiltin.cpp

Value *CodeGenFunction::EmitWebAssemblyBuiltinExpr(unsigned BuiltinID,
   case WebAssembly::BI__builtin_wasm_data_drop: {
     llvm::APSInt SegConst;
     if (!E->getArg(0)->isIntegerConstantExpr(SegConst, getContext()))
       llvm_unreachable("Constant arg isn't actually constant?");
     Value *Arg = llvm::ConstantInt::get(getLLVMContext(), SegConst);
     Function *Callee = CGM.getIntrinsic(Intrinsic::wasm_data_drop);
     return Builder.CreateCall(Callee, {Arg});
   }
+  case WebAssembly::BI__builtin_wasm_tls_size: {
+    llvm::Type *ResultType = ConvertType(E->getType());
+    Function *Callee = CGM.getIntrinsic(Intrinsic::wasm_tls_size, ResultType);
+    return Builder.CreateCall(Callee);
+  }
   case WebAssembly::BI__builtin_wasm_throw: {
     Value *Tag = EmitScalarExpr(E->getArg(0));
     Value *Obj = EmitScalarExpr(E->getArg(1));
     Function *Callee = CGM.getIntrinsic(Intrinsic::wasm_throw);
     return Builder.CreateCall(Callee, {Tag, Obj});
   }
   case WebAssembly::BI__builtin_wasm_rethrow_in_catch: {
     Function *Callee = CGM.getIntrinsic(Intrinsic::wasm_rethrow_in_catch);

clang/test/CodeGen/builtins-wasm.c

 }
 
 void data_drop() {
   __builtin_wasm_data_drop(3);
   // WEBASSEMBLY32: call void @llvm.wasm.data.drop(i32 3)
   // WEBASSEMBLY64: call void @llvm.wasm.data.drop(i32 3)
 }
 
+__SIZE_TYPE__ tls_size() {
+  return __builtin_wasm_tls_size();
+  // WEBASSEMBLY32: call i32 @llvm.wasm.tls.size.i32()
+  // WEBASSEMBLY64: call i64 @llvm.wasm.tls.size.i64()
+}
+
 void throw(void *obj) {
   return __builtin_wasm_throw(0, obj);
   // WEBASSEMBLY32: call void @llvm.wasm.throw(i32 0, i8* %{{.*}})
   // WEBASSEMBLY64: call void @llvm.wasm.throw(i32 0, i8* %{{.*}})
 }
 
 void rethrow_in_catch(void) {
   return __builtin_wasm_rethrow_in_catch();

lld/test/wasm/data-segments.ll

 ; RUN: llc -filetype=obj %s -o %t.atomics.o -mattr=+atomics
 ; RUN: llc -filetype=obj %s -o %t.bulk-mem.o -mattr=+bulk-memory
 ; RUN: llc -filetype=obj %s -o %t.atomics.bulk-mem.o -mattr=+atomics,+bulk-memory
 
 ; atomics => active segments (TODO: error)
 ; RUN: wasm-ld -no-gc-sections --no-entry --shared-memory --max-memory=131072 %t.atomics.o -o %t.atomics.wasm
-; RUN: obj2yaml %t.atomics.wasm | FileCheck %s --check-prefix ACTIVE
+; RUN: obj2yaml %t.atomics.wasm | FileCheck %s --check-prefixes ACTIVE,ACTIVE-TLS
 
 ; atomics, active segments => active segments
 ; RUN: wasm-ld -no-gc-sections --no-entry --shared-memory --max-memory=131072 --active-segments %t.atomics.o -o %t.atomics.active.wasm
-; RUN: obj2yaml %t.atomics.active.wasm | FileCheck %s --check-prefix ACTIVE
+; RUN: obj2yaml %t.atomics.active.wasm | FileCheck %s --check-prefixes ACTIVE,ACTIVE-TLS
 
 ; atomics, passive segments => error
 ; RUN: not wasm-ld -no-gc-sections --no-entry --shared-memory --max-memory=131072 --passive-segments %t.atomics.o -o %t.atomics.passive.wasm 2>&1 | FileCheck %s --check-prefix ERROR
 
 ; bulk memory => active segments
 ; RUN: wasm-ld -no-gc-sections --no-entry %t.bulk-mem.o -o %t.bulk-mem.wasm
 ; RUN: obj2yaml %t.bulk-mem.wasm | FileCheck %s --check-prefix ACTIVE
 
 ; bulk-memory, active segments => active segments
 ; RUN: wasm-ld -no-gc-sections --no-entry --active-segments %t.bulk-mem.o -o %t.bulk-mem.active.wasm
 ; RUN: obj2yaml %t.bulk-mem.active.wasm | FileCheck %s --check-prefix ACTIVE
 
 ; bulk memory, passive segments => passive segments
 ; RUN: wasm-ld -no-gc-sections --no-entry --passive-segments %t.bulk-mem.o -o %t.bulk-mem.passive.wasm
 ; RUN: obj2yaml %t.bulk-mem.passive.wasm | FileCheck %s --check-prefix PASSIVE
 
 ; atomics, bulk memory => active segments (TODO: passive)
 ; RUN: wasm-ld -no-gc-sections --no-entry --shared-memory --max-memory=131072 %t.atomics.bulk-mem.o -o %t.atomics.bulk-mem.wasm
-; RUN: obj2yaml %t.atomics.bulk-mem.wasm | FileCheck %s --check-prefix ACTIVE
+; RUN: obj2yaml %t.atomics.bulk-mem.wasm | FileCheck %s --check-prefixes ACTIVE,ACTIVE-TLS
 
 ; atomics, bulk memory, active segments   => active segments
 ; RUN: wasm-ld -no-gc-sections --no-entry --shared-memory --max-memory=131072 --active-segments %t.atomics.bulk-mem.o -o %t.atomics.bulk-mem.active.wasm
-; RUN: obj2yaml %t.atomics.bulk-mem.active.wasm | FileCheck %s --check-prefix ACTIVE
+; RUN: obj2yaml %t.atomics.bulk-mem.active.wasm | FileCheck %s --check-prefixes ACTIVE,ACTIVE-TLS
 
 ; atomics, bulk memory, passive segments  => passive segments
 ; RUN: wasm-ld -no-gc-sections --no-entry --shared-memory --max-memory=131072 --passive-segments %t.atomics.bulk-mem.o -o %t.atomics.bulk-mem.passive.wasm
-; RUN: obj2yaml %t.atomics.bulk-mem.passive.wasm | FileCheck %s --check-prefix PASSIVE
+; RUN: obj2yaml %t.atomics.bulk-mem.passive.wasm | FileCheck %s --check-prefixes PASSIVE,PASSIVE-TLS
 
 target triple = "wasm32-unknown-unknown"
 
 @a = hidden global [6 x i8] c"hello\00", align 1
 @b = hidden global [8 x i8] c"goodbye\00", align 1
 @c = hidden global [9 x i8] c"whatever\00", align 1
 @d = hidden global i32 42, align 4
 
 @e = private constant [9 x i8] c"constant\00", align 1
 @f = private constant i8 43, align 4
 
 ; ERROR: 'bulk-memory' feature must be used in order to emit passive segments
 
 ; ACTIVE-LABEL: - Type:            CODE
 ; ACTIVE-NEXT:    Functions:
 ; ACTIVE-NEXT:      - Index:           0
 ; ACTIVE-NEXT:        Locals:          []
 ; ACTIVE-NEXT:        Body:            0B
+; ACTIVE-TLS-NEXT:  - Index:           1
+; ACTIVE-TLS-NEXT:    Locals:          []
+; ACTIVE-TLS-NEXT:    Body:            0B
 ; ACTIVE-NEXT:  - Type:            DATA
 ; ACTIVE-NEXT:    Segments:
 ; ACTIVE-NEXT:      - SectionOffset:   7
 ; ACTIVE-NEXT:        InitFlags:       0
 ; ACTIVE-NEXT:        Offset:
 ; ACTIVE-NEXT:        Opcode:          I32_CONST
 ; ACTIVE-NEXT:        Value:           1024
 ; ACTIVE-NEXT:        Content:         68656C6C6F00676F6F6462796500776861746576657200002A000000
 ; ACTIVE-NEXT:      - SectionOffset:   41
 ; ACTIVE-NEXT:        InitFlags:       0
 ; ACTIVE-NEXT:        Offset:
 ; ACTIVE-NEXT:        Opcode:          I32_CONST
 ; ACTIVE-NEXT:        Value:           1052
 ; ACTIVE-NEXT:        Content:         636F6E7374616E74000000002B
 ; ACTIVE-NEXT:  - Type:            CUSTOM
 ; ACTIVE-NEXT:    Name:            name
 ; ACTIVE-NEXT:    FunctionNames:
 ; ACTIVE-NEXT:      - Index:           0
 ; ACTIVE-NEXT:        Name:            __wasm_call_ctors
+; ACTIVE-TLS-NEXT:  - Index:           1
+; ACTIVE-TLS-NEXT:    Name:            __wasm_init_tls
 
 ; PASSIVE-LABEL: - Type:            CODE
 ; PASSIVE-NEXT:    Functions:
 ; PASSIVE-NEXT:      - Index:           0
 ; PASSIVE-NEXT:        Locals:          []
 ; PASSIVE-NEXT:        Body:            10010B
 ; PASSIVE-NEXT:      - Index:           1
 ; PASSIVE-NEXT:        Locals:          []
 ; PASSIVE-NEXT:        Body:            4180084100411CFC080000FC0900419C084100410DFC080100FC09010B
+; PASSIVE-TLS-NEXT:  - Index:           2
+; PASSIVE-TLS-NEXT:    Locals:          []
+; PASSIVE-TLS-NEXT:    Body:            0B
 ; PASSIVE-NEXT:  - Type:            DATA
 ; PASSIVE-NEXT:    Segments:
 ; PASSIVE-NEXT:      - SectionOffset:   3
 ; PASSIVE-NEXT:        InitFlags:       1
 ; PASSIVE-NEXT:        Content:         68656C6C6F00676F6F6462796500776861746576657200002A000000
 ; PASSIVE-NEXT:      - SectionOffset:   33
 ; PASSIVE-NEXT:        InitFlags:       1
 ; PASSIVE-NEXT:        Content:         636F6E7374616E74000000002B
 ; PASSIVE-NEXT:  - Type:            CUSTOM
 ; PASSIVE-NEXT:    Name:            name
 ; PASSIVE-NEXT:    FunctionNames:
 ; PASSIVE-NEXT:      - Index:           0
 ; PASSIVE-NEXT:        Name:            __wasm_call_ctors
 ; PASSIVE-NEXT:      - Index:           1
 ; PASSIVE-NEXT:        Name:            __wasm_init_memory
+; PASSIVE-TLS-NEXT:  - Index:           2
+; PASSIVE-TLS-NEXT:    Name:            __wasm_init_tls

lld/test/wasm/tls.ll

+; RUN: llc -mattr=+bulk-memory -filetype=obj %s -o %t.o
+
+target triple = "wasm32-unknown-unknown"
+
+@tls1 = thread_local(localexec) global i32 1, align 4
+@tls2 = thread_local(localexec) global i32 1, align 4
+
+define i32* @tls1_addr() {
+  ret i32* @tls1
+}
+
+define i32* @tls2_addr() {
+  ret i32* @tls2
+}
+
+; RUN: wasm-ld -no-gc-sections --shared-memory --max-memory=131072 --no-entry -o %t.wasm %t.o
+; RUN: obj2yaml %t.wasm | FileCheck %s
+
+; RUN: wasm-ld -no-gc-sections --shared-memory --max-memory=131072 --no-merge-data-segments --no-entry -o %t.wasm %t.o
+; RUN: obj2yaml %t.wasm | FileCheck %s
+
+; CHECK:      - Type:            GLOBAL
+; CHECK-NEXT:   Globals:
+; CHECK-NEXT:     - Index:           0
+; CHECK-NEXT:       Type:            I32
+; CHECK-NEXT:       Mutable:         true
+; CHECK-NEXT:       InitExpr:
+; CHECK-NEXT:         Opcode:          I32_CONST
+; CHECK-NEXT:         Value:           66576
+; CHECK-NEXT:     - Index:           1
+; CHECK-NEXT:       Type:            I32
+; CHECK-NEXT:       Mutable:         true
+; CHECK-NEXT:       InitExpr:
+; CHECK-NEXT:         Opcode:          I32_CONST
+; CHECK-NEXT:         Value:           0
+; CHECK-NEXT:     - Index:           2
+; CHECK-NEXT:       Type:            I32
+; CHECK-NEXT:       Mutable:         false
+; CHECK-NEXT:       InitExpr:
+; CHECK-NEXT:         Opcode:          I32_CONST
+; CHECK-NEXT:         Value:           8
+
+
+; CHECK:      - Type:            CODE
+; CHECK-NEXT:   Functions:
+; CHECK-NEXT:     - Index:           0
+; CHECK-NEXT:       Locals:          []
+; CHECK-NEXT:       Body:            0B
+; CHECK-NEXT:     - Index:           1
+; CHECK-NEXT:       Locals:          []
+; CHECK-NEXT:       Body:            20002401200041004108FC0800000B
+
+; Expected body of __wasm_init_tls:
+;   local.get 0
+;   global.set  1
+;   local.get 0
+;   i32.const 0
+;   i32.const 8
+;   memory.init 0, 0
+;   end
+
+; CHECK-NEXT:     - Index:           2
+; CHECK-NEXT:       Locals:          []
+; CHECK-NEXT:       Body:            2381808080004180808080006A0B
+
+; Expected body of tls1_addr:
+;   global.get 1
+;   i32.const 0
+;   i32.add
+;   end
+
+; CHECK-NEXT:     - Index:           3
+; CHECK-NEXT:       Locals:          []
+; CHECK-NEXT:       Body:            2381808080004184808080006A0B
+
+; Expected body of tls1_addr:
+;   global.get 1
+;   i32.const 4
+;   i32.add
+;   end

lld/wasm/Driver.cpp

createUndefinedGlobal(StringRef name, llvm::wasm::WasmGlobalType *type) {
   config->allowUndefinedSymbols.insert(sym->getName());
   sym->isUsedInRegularObj = true;
   return sym;
 }
 
 // Create ABI-defined synthetic symbols
 static void createSyntheticSymbols() {
   static WasmSignature nullSignature = {{}, {}};
+  static WasmSignature i32ArgSignature = {{}, {ValType::I32}};
   static llvm::wasm::WasmGlobalType globalTypeI32 = {WASM_TYPE_I32, false};
   static llvm::wasm::WasmGlobalType mutableGlobalTypeI32 = {WASM_TYPE_I32,
                                                             true};
 
   if (!config->relocatable) {
     WasmSym::callCtors = symtab->addSyntheticFunction(
         "__wasm_call_ctors", WASM_SYMBOL_VISIBILITY_HIDDEN,
         make<SyntheticFunction>(nullSignature, "__wasm_call_ctors"));
if (config->isPic) {
     // spec proposal is implemented in all major browsers.
     // See: https://github.com/WebAssembly/mutable-global
     WasmSym::stackPointer = symtab->addSyntheticGlobal(
         "__stack_pointer", WASM_SYMBOL_VISIBILITY_HIDDEN, stackPointer);
     WasmSym::globalBase = symtab->addOptionalDataSymbol("__global_base");
     WasmSym::heapBase = symtab->addOptionalDataSymbol("__heap_base");
   }
 
+  if (config->sharedMemory) {
+    llvm::wasm::WasmGlobal tlsBaseGlobal;
+    tlsBaseGlobal.Type = {WASM_TYPE_I32, true};
+    tlsBaseGlobal.InitExpr.Value.Int32 = 0;
+    tlsBaseGlobal.InitExpr.Opcode = WASM_OPCODE_I32_CONST;
+    tlsBaseGlobal.SymbolName = "__tls_base";
+    WasmSym::tlsBase =
+        symtab->addSyntheticGlobal("__tls_base", WASM_SYMBOL_VISIBILITY_HIDDEN,
+                                   make<InputGlobal>(tlsBaseGlobal, nullptr));
+
+    llvm::wasm::WasmGlobal tlsSizeGlobal;
+    tlsSizeGlobal.Type = {WASM_TYPE_I32, false};
+    tlsSizeGlobal.InitExpr.Value.Int32 = 0;
+    tlsSizeGlobal.InitExpr.Opcode = WASM_OPCODE_I32_CONST;
+    tlsSizeGlobal.SymbolName = "__tls_size";
+    WasmSym::tlsSize =
+        symtab->addSyntheticGlobal("__tls_size", WASM_SYMBOL_VISIBILITY_HIDDEN,
+                                   make<InputGlobal>(tlsSizeGlobal, nullptr));
+
+    WasmSym::initTLS = symtab->addSyntheticFunction(
+        "__wasm_init_tls", WASM_SYMBOL_VISIBILITY_HIDDEN,
+        make<SyntheticFunction>(i32ArgSignature, "__wasm_init_tls"));
+  }
+
   WasmSym::dsoHandle = symtab->addSyntheticDataSymbol(
       "__dso_handle", WASM_SYMBOL_VISIBILITY_HIDDEN);
 }
 
 // Reconstructs command line arguments so that so that you can re-run
 // the same command with the same inputs. This is for --reproduce.
 static std::string createResponseFile(const opt::InputArgList &args) {
   SmallString<0> data;

lld/wasm/Symbols.h

struct WasmSym {
   // Symbol marking the start of the global section.
   static DefinedData *globalBase;
 
   // __stack_pointer
   // Global that holds the address of the top of the explicit value stack in
   // linear memory.
   static GlobalSymbol *stackPointer;
 
+  // __tls_base
+  // Global that holds the address of the base of the current thread's
+  // TLS block.
+  static GlobalSymbol *tlsBase;
+
+  // __tls_size
+  // Symbol whose value is the size of the TLS block.
+  static GlobalSymbol *tlsSize;
+
   // __data_end
   // Symbol marking the end of the data and bss.
   static DefinedData *dataEnd;
 
   // __heap_base
   // Symbol marking the end of the data, bss and explicit stack.  Any linear
   // memory following this address is not used by the linked code and can
   // therefore be used as a backing store for brk()/malloc() implementations.
   static DefinedData *heapBase;
 
   // __wasm_call_ctors
   // Function that directly calls all ctors in priority order.
   static DefinedFunction *callCtors;
 
   // __wasm_init_memory
   // Function that initializes passive data segments post-instantiation.
   static DefinedFunction *initMemory;
 
   // __wasm_apply_relocs
   // Function that applies relocations to data segment post-instantiation.
   static DefinedFunction *applyRelocs;
 
+  // __wasm_init_tls
+  // Function that allocates thread-local storage and initializes it.
+  static DefinedFunction *initTLS;
+
   // __dso_handle
   // Symbol used in calls to __cxa_atexit to determine current DLL
   static DefinedData *dsoHandle;
 
   // __table_base
   // Used in PIC code for offset of indirect function table
   static UndefinedGlobal *tableBase;
 

lld/wasm/Symbols.cpp

 using namespace llvm;
 using namespace llvm::wasm;
 using namespace lld;
 using namespace lld::wasm;
 
 DefinedFunction *WasmSym::callCtors;
 DefinedFunction *WasmSym::initMemory;
 DefinedFunction *WasmSym::applyRelocs;
+DefinedFunction *WasmSym::initTLS;
 DefinedData *WasmSym::dsoHandle;
 DefinedData *WasmSym::dataEnd;
 DefinedData *WasmSym::globalBase;
 DefinedData *WasmSym::heapBase;
 GlobalSymbol *WasmSym::stackPointer;
+GlobalSymbol *WasmSym::tlsBase;
+GlobalSymbol *WasmSym::tlsSize;
 UndefinedGlobal *WasmSym::tableBase;
 UndefinedGlobal *WasmSym::memoryBase;
 
 WasmSymbolType Symbol::getWasmType() const {
   if (isa<FunctionSymbol>(this))
     return WASM_SYMBOL_TYPE_FUNCTION;
   if (isa<DataSymbol>(this))
     return WASM_SYMBOL_TYPE_DATA;

 DefinedFunction::DefinedFunction(StringRef name, uint32_t flags, InputFile *f,
                                  InputFunction *function)
     : FunctionSymbol(name, DefinedFunctionKind, flags, f,
                      function ? &function->signature : nullptr),
       function(function) {}
 
 uint32_t DefinedData::getVirtualAddress() const {
   LLVM_DEBUG(dbgs() << "getVirtualAddress: " << getName() << "\n");
-  if (segment)
+  if (segment) {
+    // For thread local data, the symbol location is relative to the start of
+    // the .tdata section, since they are used as offsets from __tls_base.
+    // Hence, we do not add in segment->outputSeg->startVA.
+    if (segment->outputSeg->name == ".tdata")
+      return segment->outputSegmentOffset + offset;
     return segment->outputSeg->startVA + segment->outputSegmentOffset + offset;
+  }
   return offset;
 }
 
 void DefinedData::setVirtualAddress(uint32_t value) {
   LLVM_DEBUG(dbgs() << "setVirtualAddress " << name << " -> " << value << "\n");
   assert(!segment);
   offset = value;
 }

lld/wasm/Writer.cpp

public:
   void run();
 
 private:
   void openFile();
 
   void createInitMemoryFunction();
   void createApplyRelocationsFunction();
   void createCallCtorsFunction();
+  void createInitTLSFunction();
 
   void assignIndexes();
   void populateSymtab();
   void populateProducers();
   void populateTargetFeatures();
   void calculateInitFunctions();
   void calculateImports();
   void calculateExports();
void Writer::layoutMemory() {
   if (WasmSym::dsoHandle)
     WasmSym::dsoHandle->setVirtualAddress(dataStart);
 
   out.dylinkSec->memAlign = 0;
   for (OutputSegment *seg : segments) {
     out.dylinkSec->memAlign = std::max(out.dylinkSec->memAlign, seg->alignment);
     memoryPtr = alignTo(memoryPtr, 1ULL << seg->alignment);
     seg->startVA = memoryPtr;
     log(formatv("mem: {0,-15} offset={1,-8} size={2,-8} align={3}", seg->name,
                 memoryPtr, seg->size, seg->alignment));
     memoryPtr += seg->size;
+
+    if (WasmSym::tlsSize && seg->name == ".tdata") {
+      auto *tlsSize = cast<DefinedGlobal>(WasmSym::tlsSize);
+      tlsSize->global->global.InitExpr.Value.Int32 = seg->size;
+    }
   }
 
   // TODO: Add .bss space here.
   if (WasmSym::dataEnd)
     WasmSym::dataEnd->setVirtualAddress(memoryPtr);
 
   log("mem: static data = " + Twine(memoryPtr - dataStart));
 
for (OutputSection *s : outputSections) {
     fileSize += s->getSize();
   }
 }
 
 void Writer::populateTargetFeatures() {
   StringMap<std::string> used;
   StringMap<std::string> required;
   StringMap<std::string> disallowed;
+  bool tlsUsed = false;
 
   // Only infer used features if user did not specify features
   bool inferFeatures = !config->features.hasValue();
 
   if (!inferFeatures) {
     for (auto &feature : config->features.getValue())
       out.targetFeaturesSec->features.insert(feature);
     // No need to read or check features
for (auto &feature : file->getWasmObj()->getTargetFeatures()) {
       case WASM_FEATURE_PREFIX_DISALLOWED:
         disallowed.insert({feature.Name, fileName});
         break;
       default:
         error("Unrecognized feature policy prefix " +
               std::to_string(feature.Prefix));
       }
     }
+
+    for (InputSegment *segment : file->segments) {
+      if (!segment->live)
+        continue;
+      StringRef name = segment->getName();
+      if (name.startswith(".tdata.") || name.startswith(".tbss."))
+        tlsUsed = true;
+    }
   }
 
   if (inferFeatures)
     out.targetFeaturesSec->features.insert(used.keys().begin(),
                                            used.keys().end());
 
   if (out.targetFeaturesSec->features.count("atomics") &&
       !config->sharedMemory) {
void Writer::populateTargetFeatures() {
   if (disallowed.count("atomics") && config->sharedMemory)
     error("'atomics' feature is disallowed by " + disallowed["atomics"] +
           ", so --shared-memory must not be used");
 
   if (!used.count("bulk-memory") && config->passiveSegments)
     error("'bulk-memory' feature must be used in order to emit passive "
           "segments");
 
+  if (!used.count("bulk-memory") && tlsUsed)
+    error("'bulk-memory' feature must be used in order to use thread-local "
+          "storage");
+
   // Validate that used features are allowed in output
   if (!inferFeatures) {
     for (auto &feature : used.keys()) {
       if (!out.targetFeaturesSec->features.count(feature))
         error(Twine("Target feature '") + feature + "' used by " +
               used[feature] + " is not allowed.");
     }
   }
for (Symbol *sym : symtab->getSymbols()) {
     WasmExport export_;
     if (auto *f = dyn_cast<DefinedFunction>(sym)) {
       export_ = {name, WASM_EXTERNAL_FUNCTION, f->getFunctionIndex()};
     } else if (auto *g = dyn_cast<DefinedGlobal>(sym)) {
       // TODO(sbc): Remove this check once to mutable global proposal is
       // implement in all major browsers.
       // See: https://github.com/WebAssembly/mutable-global
       if (g->getGlobalType()->Mutable) {
         // Only the __stack_pointer should ever be create as mutable.
-        assert(g == WasmSym::stackPointer);
+        assert(g == WasmSym::stackPointer || g == WasmSym::tlsBase);
         continue;
       }
       export_ = {name, WASM_EXTERNAL_GLOBAL, g->getGlobalIndex()};
     } else if (auto *e = dyn_cast<DefinedEvent>(sym)) {
       export_ = {name, WASM_EXTERNAL_EVENT, e->getEventIndex()};
     } else {
       auto *d = cast<DefinedData>(sym);
       out.globalSec->definedFakeGlobals.emplace_back(d);
void Writer::assignIndexes() {
   }
 }
 
 static StringRef getOutputDataSegmentName(StringRef name) {
   // With PIC code we currently only support a single data segment since
   // we only have a single __memory_base to use as our base address.
   if (config->isPic)
     return ".data";
+  // We only support one thread-local segment, so we must merge the segments
+  // despite --no-merge-data-segments.
+  // We also need to merge .tbss into .tdata so they share the same offsets.
+  if (name.startswith(".tdata.") || name.startswith(".tbss."))
+    return ".tdata";
   if (!config->mergeDataSegments)
     return name;
   if (name.startswith(".text."))
     return ".text";
   if (name.startswith(".data."))
     return ".data";
   if (name.startswith(".bss."))
     return ".bss";
   if (name.startswith(".rodata."))
     return ".rodata";
   return name;
 }
 
 void Writer::createOutputSegments() {
   for (ObjFile *file : symtab->objectFiles) {
     for (InputSegment *segment : file->segments) {
       if (!segment->live)
         continue;
       StringRef name = getOutputDataSegmentName(segment->getName());
       OutputSegment *&s = segmentMap[name];
       if (s == nullptr) {
         LLVM_DEBUG(dbgs() << "new segment: " << name << "\n");
         s = make<OutputSegment>(name, segments.size());
-        if (config->passiveSegments)
+        if (config->passiveSegments || name == ".tdata")
           s->initFlags = WASM_SEGMENT_IS_PASSIVE;
         segments.push_back(s);
       }
       s->addInputSegment(segment);
       LLVM_DEBUG(dbgs() << "added data: " << name << ": " << s->size << "\n");
     }
   }
 }
void Writer::createInitMemoryFunction() {
   LLVM_DEBUG(dbgs() << "createInitMemoryFunction\n");
   std::string bodyContent;
   {
     raw_string_ostream os(bodyContent);
     writeUleb128(os, 0, "num locals");
 
     // initialize passive data segments
     for (const OutputSegment *s : segments) {
-      if (s->initFlags & WASM_SEGMENT_IS_PASSIVE) {
+      if (s->initFlags & WASM_SEGMENT_IS_PASSIVE && s->name != ".tdata") {
         // destination address
         writeU8(os, WASM_OPCODE_I32_CONST, "i32.const");
         writeUleb128(os, s->startVA, "destination address");
         // source segment offset
         writeU8(os, WASM_OPCODE_I32_CONST, "i32.const");
         writeUleb128(os, 0, "segment offset");
         // memory region size
         writeU8(os, WASM_OPCODE_I32_CONST, "i32.const");
for (const WasmInitEntry &f : initFunctions) {
       writeUleb128(os, f.sym->getFunctionIndex(), "function index");
     }
     writeU8(os, WASM_OPCODE_END, "END");
   }
 
   createFunction(WasmSym::callCtors, bodyContent);
 }
 
+void Writer::createInitTLSFunction() {
+  if (!WasmSym::initTLS->isLive())
+    return;
+
+  std::string bodyContent;
+  {
+    raw_string_ostream os(bodyContent);
+
+    OutputSegment *tlsSeg = nullptr;
+    for (auto *seg : segments) {
+      if (seg->name == ".tdata")
+        tlsSeg = seg;
+      break;
+    }
+
+    if (tlsSeg) {
+      writeUleb128(os, 0, "num locals");
+
+      writeU8(os, WASM_OPCODE_LOCAL_GET, "local.get");
+      writeUleb128(os, 0, "local index");
+
+      writeU8(os, WASM_OPCODE_GLOBAL_SET, "global.set");
+      writeUleb128(os, WasmSym::tlsBase->getGlobalIndex(), "global index");
+
+      writeU8(os, WASM_OPCODE_LOCAL_GET, "local.get");
+      writeUleb128(os, 0, "local index");
+
+      writeU8(os, WASM_OPCODE_I32_CONST, "i32.const");
+      writeUleb128(os, 0, "segment offset");
+
+      writeU8(os, WASM_OPCODE_I32_CONST, "i32.const");
+      writeUleb128(os, tlsSeg->size, "memory region size");
+
+      writeU8(os, WASM_OPCODE_MISC_PREFIX, "bulk-memory prefix");
+      writeUleb128(os, WASM_OPCODE_MEMORY_INIT, "MEMORY.INIT");
+      writeUleb128(os, tlsSeg->index, "segment index immediate");
+      writeU8(os, 0, "memory index immediate");
+
+      writeU8(os, WASM_OPCODE_END, "end function");
+    } else {
+      writeUleb128(os, 0, "num locals");
+      writeU8(os, WASM_OPCODE_END, "end function");
+    }
+  }
+
+  createFunction(WasmSym::initTLS, bodyContent);
+}
+
 // Populate InitFunctions vector with init functions from all input objects.
 // This is then used either when creating the output linking section or to
 // synthesize the "__wasm_call_ctors" function.
 void Writer::calculateInitFunctions() {
   if (!config->relocatable && !WasmSym::callCtors->isLive())
     return;
 
   for (ObjFile *file : symtab->objectFiles) {
if (!config->relocatable) {
     // Create linker synthesized functions
     if (config->passiveSegments)
       createInitMemoryFunction();
     if (config->isPic)
       createApplyRelocationsFunction();
     createCallCtorsFunction();
   }
 
+  if (config->sharedMemory)
+    createInitTLSFunction();
+
+  if (errorCount())
+    return;
+
   log("-- calculateTypes");
   calculateTypes();
   log("-- calculateExports");
   calculateExports();
   log("-- calculateCustomSections");
   calculateCustomSections();
   log("-- populateSymtab");
   populateSymtab();

llvm/include/llvm/BinaryFormat/Wasm.h

enum : unsigned {
   WASM_EXTERNAL_GLOBAL = 0x3,
   WASM_EXTERNAL_EVENT = 0x4,
 };
 
 // Opcodes used in initializer expressions.
 enum : unsigned {
   WASM_OPCODE_END = 0x0b,
   WASM_OPCODE_CALL = 0x10,
+  WASM_OPCODE_LOCAL_GET = 0x20,
   WASM_OPCODE_GLOBAL_GET = 0x23,
+  WASM_OPCODE_GLOBAL_SET = 0x24,
   WASM_OPCODE_I32_STORE = 0x36,
   WASM_OPCODE_I32_CONST = 0x41,
   WASM_OPCODE_I64_CONST = 0x42,
   WASM_OPCODE_F32_CONST = 0x43,
   WASM_OPCODE_F64_CONST = 0x44,
   WASM_OPCODE_I32_ADD = 0x6a,
   WASM_OPCODE_MISC_PREFIX = 0xfc,
   WASM_OPCODE_MEMORY_INIT = 0x08,

llvm/include/llvm/IR/IntrinsicsWebAssembly.td

Intrinsic<[],
             [llvm_i32_ty, llvm_i32_ty, llvm_ptr_ty, llvm_i32_ty, llvm_i32_ty],
             [IntrWriteMem, IntrInaccessibleMemOrArgMemOnly, WriteOnly<2>,
              IntrHasSideEffects, ImmArg<0>, ImmArg<1>]>;
 def int_wasm_data_drop :
   Intrinsic<[],
             [llvm_i32_ty],
             [IntrNoDuplicate, IntrHasSideEffects, ImmArg<0>]>;
 
+//===----------------------------------------------------------------------===//
+// Thread-local storage intrinsics
+//===----------------------------------------------------------------------===//
+
+def int_wasm_tls_size :
+  Intrinsic<[llvm_anyint_ty],
+            [],
+            [IntrNoMem, IntrSpeculatable]>;
+
 } // TargetPrefix = "wasm"

llvm/include/llvm/MC/MCSectionWasm.h

public:
 
   void PrintSwitchToSection(const MCAsmInfo &MAI, const Triple &T,
                             raw_ostream &OS,
                             const MCExpr *Subsection) const override;
   bool UseCodeAlign() const override;
   bool isVirtualSection() const override;
 
   bool isWasmData() const {
-    return Kind.isGlobalWriteableData() || Kind.isReadOnly();
+    return Kind.isGlobalWriteableData() || Kind.isReadOnly() ||
+           Kind.isThreadLocal();
   }
 
   bool isUnique() const { return UniqueID != ~0U; }
   unsigned getUniqueID() const { return UniqueID; }
 
   uint64_t getSectionOffset() const { return SectionOffset; }
   void setSectionOffset(uint64_t Offset) { SectionOffset = Offset; }
 

llvm/lib/Target/WebAssembly/WebAssemblyFastISel.cpp

if (Ty->getAddressSpace() > 255)
       // address spaces.
       return false;
 
   if (const auto *GV = dyn_cast<GlobalValue>(Obj)) {
     if (TLI.isPositionIndependent())
       return false;
     if (Addr.getGlobalValue())
       return false;
+    if (GV->isThreadLocal())
+      return false;
     Addr.setGlobalValue(GV);
     return true;
   }
 
   switch (Opcode) {
   default:
     break;
   case Instruction::BitCast: {
unsigned WebAssemblyFastISel::fastMaterializeAlloca(const AllocaInst *AI) {
 
   return 0;
 }
 
 unsigned WebAssemblyFastISel::fastMaterializeConstant(const Constant *C) {
   if (const GlobalValue *GV = dyn_cast<GlobalValue>(C)) {
     if (TLI.isPositionIndependent())
       return 0;
+    if (GV->isThreadLocal())
+      return 0;
     unsigned ResultReg =
         createResultReg(Subtarget->hasAddr64() ? &WebAssembly::I64RegClass
                                                : &WebAssembly::I32RegClass);
     unsigned Opc = Subtarget->hasAddr64() ? WebAssembly::CONST_I64
                                           : WebAssembly::CONST_I32;
     BuildMI(*FuncInfo.MBB, FuncInfo.InsertPt, DbgLoc, TII.get(Opc), ResultReg)
         .addGlobalAddress(GV);
     return ResultReg;

llvm/lib/Target/WebAssembly/WebAssemblyISelDAGToDAG.cpp

 /// This file defines an instruction selector for the WebAssembly target.
 ///
 //===----------------------------------------------------------------------===//
 
 #include "MCTargetDesc/WebAssemblyMCTargetDesc.h"
 #include "WebAssembly.h"
 #include "WebAssemblyTargetMachine.h"
 #include "llvm/CodeGen/SelectionDAGISel.h"
+#include "llvm/IR/DiagnosticInfo.h"
 #include "llvm/IR/Function.h" // To access function attributes.
 #include "llvm/Support/Debug.h"
 #include "llvm/Support/KnownBits.h"
 #include "llvm/Support/MathExtras.h"
 #include "llvm/Support/raw_ostream.h"
 using namespace llvm;
 
 #define DEBUG_TYPE "wasm-isel"
case SyncScope::System: {
       CurDAG->RemoveDeadNode(Node);
       return;
     }
     default:
       llvm_unreachable("Unknown scope!");
     }
   }
 
+  case ISD::GlobalTLSAddress: {
+    const auto *GA = cast<GlobalAddressSDNode>(Node);
+
+    if (!MF.getSubtarget<WebAssemblySubtarget>().hasBulkMemory())
+      report_fatal_error("cannot use thread-local storage without bulk memory",
+                         false);
+
+    if (GA->getGlobal()->getThreadLocalMode() !=
+        GlobalValue::LocalExecTLSModel) {
+      report_fatal_error("only -ftls-model=local-exec is supported for now",
+                         false);
+    }
+
+    MVT PtrVT = TLI->getPointerTy(CurDAG->getDataLayout());
+    assert(PtrVT == MVT::i32 && "only wasm32 is supported for now");
+
+    SDValue TLSBaseSym = CurDAG->getTargetExternalSymbol("__tls_base", PtrVT);
+    SDValue TLSOffsetSym = CurDAG->getTargetGlobalAddress(
+        GA->getGlobal(), DL, PtrVT, GA->getOffset(), 0);
+
+    MachineSDNode *TLSBase = CurDAG->getMachineNode(WebAssembly::GLOBAL_GET_I32,
+                                                    DL, MVT::i32, TLSBaseSym);
+    MachineSDNode *TLSOffset = CurDAG->getMachineNode(
+        WebAssembly::CONST_I32, DL, MVT::i32, TLSOffsetSym);
+    MachineSDNode *TLSAddress =
+        CurDAG->getMachineNode(WebAssembly::ADD_I32, DL, MVT::i32,
+                               SDValue(TLSBase, 0), SDValue(TLSOffset, 0));
+    ReplaceUses(SDValue(Node, 0), SDValue(TLSAddress, 0));
+    CurDAG->RemoveDeadNode(Node);
+    return;
+  }
+
+  case ISD::INTRINSIC_WO_CHAIN:
+    switch (cast<ConstantSDNode>(Node->getOperand(0))->getZExtValue()) {
+    case Intrinsic::wasm_tls_size: {
+      MVT PtrVT = TLI->getPointerTy(CurDAG->getDataLayout());
+      assert(PtrVT == MVT::i32 && "only wasm32 is supported for now");
+
+      MachineSDNode *TLSSize = CurDAG->getMachineNode(
+          WebAssembly::GLOBAL_GET_I32, DL, PtrVT,
+          CurDAG->getTargetExternalSymbol("__tls_size", MVT::i32));
+      ReplaceUses(SDValue(Node, 0), SDValue(TLSSize, 0));
+      CurDAG->RemoveDeadNode(Node);
+      return;
+    }
+    }
+    break;
+
   default:
     break;
   }
 
   // Select the default instruction.
   SelectCode(Node);
 }
 

llvm/lib/Target/WebAssembly/WebAssemblyMCInstLower.cpp

MCSymbol *WebAssemblyMCInstLower::GetExternalSymbolSymbol(
   const char *Name = MO.getSymbolName();
   auto *WasmSym = cast<MCSymbolWasm>(Printer.GetExternalSymbolSymbol(Name));
   const WebAssemblySubtarget &Subtarget = Printer.getSubtarget();
 
   // Except for certain known symbols, all symbols used by CodeGen are
   // functions. It's OK to hardcode knowledge of specific symbols here; this
   // method is precisely there for fetching the signatures of known
   // Clang-provided symbols.
-  if (strcmp(Name, "__stack_pointer") == 0 ||
-      strcmp(Name, "__memory_base") == 0 || strcmp(Name, "__table_base") == 0) {
-    bool Mutable = strcmp(Name, "__stack_pointer") == 0;
+  if (strcmp(Name, "__stack_pointer") == 0 || strcmp(Name, "__tls_base") == 0 ||
+      strcmp(Name, "__memory_base") == 0 || strcmp(Name, "__table_base") == 0 ||
+      strcmp(Name, "__tls_size") == 0) {
+    bool Mutable =
+        strcmp(Name, "__stack_pointer") == 0 || strcmp(Name, "__tls_base") == 0;
     WasmSym->setType(wasm::WASM_SYMBOL_TYPE_GLOBAL);
     WasmSym->setGlobalType(wasm::WasmGlobalType{
         uint8_t(Subtarget.hasAddr64() ? wasm::WASM_TYPE_I64
                                       : wasm::WASM_TYPE_I32),
         Mutable});
     return WasmSym;
   }
 

llvm/lib/Target/WebAssembly/WebAssemblyTargetMachine.cpp

public:
   bool runOnModule(Module &M) override {
     FeatureBitset Features = coalesceFeatures(M);
 
     std::string FeatureStr = getFeatureString(Features);
     for (auto &F : M)
       replaceFeatures(F, FeatureStr);
 
     bool Stripped = false;
-    if (!Features[WebAssembly::FeatureAtomics]) {
+    if (!Features[WebAssembly::FeatureAtomics])
       Stripped |= stripAtomics(M);
+
+    if (!Features[WebAssembly::FeatureBulkMemory])
       Stripped |= stripThreadLocals(M);
-    }
 
     recordFeatures(M, Features, Stripped);
 
     // Conservatively assume we have made some change
     return true;
   }
 
 private:

llvm/test/CodeGen/WebAssembly/target-features-tls.ll

-; RUN: llc < %s -mattr=-atomics | FileCheck %s --check-prefixes CHECK,NO-ATOMICS
-; RUN: llc < %s -mattr=+atomics | FileCheck %s --check-prefixes CHECK,ATOMICS
+; RUN: llc < %s -mattr=-bulk-memory | FileCheck %s --check-prefixes NO-BULK-MEM
+; RUN: llc < %s -mattr=+bulk-memory | FileCheck %s --check-prefixes BULK-MEM
 
 ; Test that the target features section contains -atomics or +atomics
 ; for modules that have thread local storage in their source.
 
 target datalayout = "e-m:e-p:32:32-i64:64-n32:64-S128"
 target triple = "wasm32-unknown-unknown"
 
 @foo = internal thread_local global i32 0
 
-; CHECK-LABEL: .custom_section.target_features,"",@
 
-; -atomics
-; NO-ATOMICS-NEXT: .int8 1
-; NO-ATOMICS-NEXT: .int8 45
-; NO-ATOMICS-NEXT: .int8 7
-; NO-ATOMICS-NEXT: .ascii "atomics"
-; NO-ATOMICS-NEXT: .bss.foo,"",@
-
-; +atomics
-; ATOMICS-NEXT: .int8 1
-; ATOMICS-NEXT: .int8 43
-; ATOMICS-NEXT: .int8 7
-; ATOMICS-NEXT: .ascii "atomics"
-; ATOMICS-NEXT: .tbss.foo,"",@
+; -bulk-memory
+; NO-BULK-MEM-LABEL: .custom_section.target_features,"",@
+; NO-BULK-MEM-NEXT: .int8 1
+; NO-BULK-MEM-NEXT: .int8 45
+; NO-BULK-MEM-NEXT: .int8 7
+; NO-BULK-MEM-NEXT: .ascii "atomics"
+; NO-BULK-MEM-NEXT: .bss.foo,"",@
+
+; +bulk-memory
+; BULK-MEM-LABEL: .custom_section.target_features,"",@
+; BULK-MEM-NEXT: .int8 1
+; BULK-MEM-NEXT: .int8 43
+; BULK-MEM-NEXT: .int8 11
+; BULK-MEM-NEXT: .ascii "bulk-memory"
+; BULK-MEM-NEXT: .tbss.foo,"",@

llvm/test/CodeGen/WebAssembly/tls.ll

-; RUN: llc < %s -asm-verbose=false -disable-wasm-fallthrough-return-opt -wasm-disable-explicit-locals -wasm-keep-registers | FileCheck --check-prefix=SINGLE %s
+; RUN: llc < %s -asm-verbose=false -disable-wasm-fallthrough-return-opt -wasm-disable-explicit-locals -mattr=+bulk-memory | FileCheck %s
+; RUN: llc < %s -asm-verbose=false -disable-wasm-fallthrough-return-opt -wasm-disable-explicit-locals -mattr=+bulk-memory -fast-isel | FileCheck %s
 target datalayout = "e-m:e-p:32:32-i64:64-n32:64-S128"
 target triple = "wasm32-unknown-unknown"
 
-; SINGLE-LABEL: address_of_tls:
+; CHECK-LABEL: address_of_tls:
 define i32 @address_of_tls() {
-  ; SINGLE: i32.const $push0=, tls
-  ; SINGLE-NEXT: return $pop0
+  ; CHECK-DAG: global.get __tls_base
+  ; CHECK-DAG: i32.const tls
+  ; CHECK-NEXT: i32.add
+  ; CHECK-NEXT: return
   ret i32 ptrtoint(i32* @tls to i32)
 }
 
-; SINGLE: .type tls,@object
-; SINGLE-NEXT: .section .bss.tls,"",@
-; SINGLE-NEXT: .p2align 2
-; SINGLE-NEXT: tls:
-; SINGLE-NEXT: .int32 0
-@tls = internal thread_local global i32 0
+; CHECK-LABEL: ptr_to_tls
+define i32* @ptr_to_tls() {
+  ; CHECK-DAG: global.get __tls_base
+  ; CHECK-DAG: i32.const tls
+  ; CHECK-NEXT: i32.add
+  ; CHECK-NEXT: return
+  ret i32* @tls
+}
+
+; CHECK-LABEL: tls_load
+define i32 @tls_load() {
+  ; CHECK-DAG: global.get __tls_base
+  ; CHECK-DAG: i32.const tls
+  ; CHECK-NEXT: i32.add
+  ; CHECK-NEXT: i32.load 0
+  ; CHECK-NEXT: return
+  %tmp = load i32, i32* @tls, align 4
+  ret i32 %tmp
+}
+
+; CHECK-LABEL: tls_store
+define void @tls_store(i32 %x) {
+  ; CHECK-DAG: global.get __tls_base
+  ; CHECK-DAG: i32.const tls
+  ; CHECK-NEXT: i32.add
+  ; CHECK-NEXT: i32.store 0
+  ; CHECK-NEXT: return
+  store i32 %x, i32* @tls, align 4
+  ret void
+}
+
+; CHECK-LABEL: tls_size:
+; CHECK-NEXT: .functype tls_size () -> (i32)
+; CHECK-NEXT: global.get __tls_size
+; CHECK-NEXT: return
+declare i32 @llvm.wasm.tls.size.i32()
+define i32 @tls_size() {
+  %1 = call i32 @llvm.wasm.tls.size.i32()
+  ret i32 %1
+}
+
+; CHECK: .type tls,@object
+; CHECK-NEXT: .section .tbss.tls,"",@
+; CHECK-NEXT: .p2align 2
+; CHECK-NEXT: tls:
+; CHECK-NEXT: .int32 0
+@tls = internal thread_local(localexec) global i32 0