[WebAssembly] Ban all TLS on non-emscripten targets (even local-exec)

[sbc100]

Summary

Without this change it's possible for non-emscripten users (e.g. wasi-sdk
users) o end up with TLS usage in their object files. e.g:

$ clang -pthread --target=wasm32-wasi tls_test.c -ftls-model=local-exec

[sbc100]

typo

[tlively]

LGTM if we can verify that neither Rust nor WASI use this functionality.

@alexcrichton, do non-Emscripten Rust targets use the current TLS implementation?

[alexcrichton]

Thanks for the cc! By default rustc doesn't use TLS vars this should be fine. With the atomics feature enabled, however, it does use TLS but I imagine this doesn't affect that?

[tlively]

Thanks, Alex! I suspected that might be the case. Follow-up question: What kind of binary compatibility guarantees does Rust have when mixing compiler versions? Would it be a problem for example if we rewrote the TLS mechanism from scratch so that it worked differently in LLVM 12 than it does in LLVM 11?

Sam, I don't think we can land this change because it would prevent Rust from working with tip-of-tree LLVM.

[alexcrichton]

Currently Rust is able to get away with 0 binary compatibility between rustc versions, so if LLVM 12 is entirely different from LLVM 11, that's totally fine by us!

[sbc100]

But surely rust doesn't build with +bulk-memory?

The only way that rust could be hitting this code bath is by passing both +bulk-memory and -ftls-model=local-exec` right? ... without and explicit ftls-model rust already hits this error.

Does rust actually support building multi-threaded code? Or do you rely on the TLS / atomics lowering pass?

[sbc100]

WASI for sure doesn't support any kind of threading yet.

[alexcrichton]

Currently we specify the TLS model as "local-exec" for non-threaded wasm code, but we never emit thread_local for globals. When the atomics feature is enabled we do use thread_local for globals as needed. Rust does not by default enable bulk-memory and we still rely on users when compiling with atomics to also enable bulk-memory.

We don't have a precompiled target for threads and wasm but we do have support for recompiling the standard library and running the result. There's an example at https://rustwasm.github.io/wasm-bindgen/examples/raytrace.html.

It's also probably worth pointing out that we rely on the atomics lowering pass by default for wasm targets, and only with -Ctarget-feature=+atomics do we disable that.

[sbc100]

In D91274#2392490, @alexcrichton wrote:

Currently we specify the TLS model as "local-exec" for non-threaded wasm code, but we never emit thread_local for globals. When the atomics feature is enabled we do use thread_local for globals as needed. Rust does not by default enable bulk-memory and we still rely on users when compiling with atomics to also enable bulk-memory.

Ah so you do actually end up with real TLS variables and in that case you would hit this error if I land this change. Do you have code for setting up TLS on your newly created threads (like emscripten does)?

What triple do you compile with?

We don't have a precompiled target for threads and wasm but we do have support for recompiling the standard library and running the result. There's an example at https://rustwasm.github.io/wasm-bindgen/examples/raytrace.html.

It's also probably worth pointing out that we rely on the atomics lowering pass by default for wasm targets, and only with -Ctarget-feature=+atomics do we disable that.

[alexcrichton]

Currently rustc's threading-in-wasm story is still very DIY in that you're expected to take the low-level primitives that LLD emits and build your own tooling around initializing thread-locals and such. For example in wasm-bindgen we've got code to glue everything together, but that's all wasm-bindgen-specific and there's nothing more general AFAIK in Rust

For compilation we compile with wasm32-unknown-unknown, which is another big drawback of our story today because we don't have a dedicated target for threads, you just have to compile with the right flags. This is a reflection of how under-baked the story is in Rust today. FWIW I'd be happy to change whatever is necessary in rustc since I think we have very few users of threaded-wasm right now. I suspect whatever Emscripten is doing aligns pretty closely with where we'd want to end up!

llvm/lib/Target/WebAssembly/WebAssemblyISelDAGToDAG.cpp

void WebAssemblyDAGToDAGISel::Select(SDNode *Node) {
 
   case ISD::GlobalTLSAddress: {
     const auto *GA = cast<GlobalAddressSDNode>(Node);
 
     if (!MF.getSubtarget<WebAssemblySubtarget>().hasBulkMemory())
       report_fatal_error("cannot use thread-local storage without bulk memory",
                          false);
 
-    // Currently Emscripten does not support dynamic linking with threads.
-    // Therefore, if we have thread-local storage, only the local-exec model
-    // is possible.
-    // TODO: remove this and implement proper TLS models once Emscripten
-    // supports dynamic linking with threads.
-    if (GA->getGlobal()->getThreadLocalMode() !=
-            GlobalValue::LocalExecTLSModel &&
-        !Subtarget->getTargetTriple().isOSEmscripten()) {
-      report_fatal_error("only -ftls-model=local-exec is supported for now on "
-                         "non-Emscripten OSes: variable " +
+    if (!Subtarget->getTargetTriple().isOSEmscripten()) {
+      report_fatal_error("WebAssembly does not yet support TLS outside of "
+                         "emscripten: variable " +
                              GA->getGlobal()->getName(),
                          false);
     }
 
     SDValue TLSBaseSym = CurDAG->getTargetExternalSymbol("__tls_base", PtrVT);
     SDValue TLSOffsetSym = CurDAG->getTargetGlobalAddress(
         GA->getGlobal(), DL, PtrVT, GA->getOffset(), 0);
 

llvm/test/CodeGen/WebAssembly/tls-general-dynamic.ll

+; Check that using TLS without emscripten OS causes and error
 ; RUN: not --crash llc < %s -asm-verbose=false -disable-wasm-fallthrough-return-opt -wasm-disable-explicit-locals -mattr=+bulk-memory 2>&1 | FileCheck %s --check-prefix=ERROR
 ; RUN: not --crash llc < %s -asm-verbose=false -disable-wasm-fallthrough-return-opt -wasm-disable-explicit-locals -mattr=+bulk-memory -fast-isel 2>&1 | FileCheck %s --check-prefix=ERROR
+
+; ERROR: LLVM ERROR: WebAssembly does not yet support TLS outside of emscripten: variable tls
+
 ; RUN: llc < %s -asm-verbose=false -disable-wasm-fallthrough-return-opt -wasm-disable-explicit-locals -mattr=+bulk-memory --mtriple wasm32-unknown-emscripten | FileCheck %s --check-prefixes=CHECK,TLS
 ; RUN: llc < %s -asm-verbose=false -disable-wasm-fallthrough-return-opt -wasm-disable-explicit-locals -mattr=+bulk-memory --mtriple wasm32-unknown-emscripten -fast-isel | FileCheck %s --check-prefixes=CHECK,TLS
+
+; Check that TLS operations are lowered away when bulk-memory is not enabled.  Even on non-emscripten targets
 ; RUN: llc < %s -asm-verbose=false -disable-wasm-fallthrough-return-opt -wasm-disable-explicit-locals -mattr=-bulk-memory | FileCheck %s --check-prefixes=CHECK,NO-TLS
 target datalayout = "e-m:e-p:32:32-i64:64-n32:64-S128"
 target triple = "wasm32-unknown-unknown"
 
-; ERROR: LLVM ERROR: only -ftls-model=local-exec is supported for now on non-Emscripten OSes: variable tls
 
 ; CHECK-LABEL: address_of_tls:
 ; CHECK-NEXT: .functype  address_of_tls () -> (i32)
 define i32 @address_of_tls() {
   ; TLS-DAG: global.get __tls_base
   ; TLS-DAG: i32.const tls
   ; TLS-NEXT: i32.add
   ; TLS-NEXT: return

llvm/test/CodeGen/WebAssembly/tls-local-exec.ll

-; RUN: llc < %s -asm-verbose=false -disable-wasm-fallthrough-return-opt -wasm-disable-explicit-locals -mattr=+bulk-memory | FileCheck %s --check-prefixes=CHECK,TLS
-; RUN: llc < %s -asm-verbose=false -disable-wasm-fallthrough-return-opt -wasm-disable-explicit-locals -mattr=+bulk-memory -fast-isel | FileCheck %s --check-prefixes=CHECK,TLS
+; Check that using TLS without emscripten OS causes and error
+; RUN: not --crash llc < %s -asm-verbose=false -disable-wasm-fallthrough-return-opt -wasm-disable-explicit-locals -mattr=+bulk-memory 2>&1 | FileCheck %s --check-prefix=ERROR
+; RUN: not --crash llc < %s -asm-verbose=false -disable-wasm-fallthrough-return-opt -wasm-disable-explicit-locals -mattr=+bulk-memory -fast-isel 2>&1 | FileCheck %s --check-prefix=ERROR
+
+; ERROR: LLVM ERROR: WebAssembly does not yet support TLS outside of emscripten: variable tls
+
+; RUN: llc < %s -asm-verbose=false -disable-wasm-fallthrough-return-opt -wasm-disable-explicit-locals -mattr=+bulk-memory --mtriple wasm32-unknown-emscripten | FileCheck %s --check-prefixes=CHECK,TLS
+; RUN: llc < %s -asm-verbose=false -disable-wasm-fallthrough-return-opt -wasm-disable-explicit-locals -mattr=+bulk-memory --mtriple wasm32-unknown-emscripten -fast-isel | FileCheck %s --check-prefixes=CHECK,TLS
+
+; Check that TLS operations are lowered away when bulk-memory is not enabled.  Even on non-emscripten targets
 ; RUN: llc < %s -asm-verbose=false -disable-wasm-fallthrough-return-opt -wasm-disable-explicit-locals -mattr=-bulk-memory | FileCheck %s --check-prefixes=CHECK,NO-TLS
+
 target datalayout = "e-m:e-p:32:32-i64:64-n32:64-S128"
 target triple = "wasm32-unknown-unknown"
 
 ; CHECK-LABEL: address_of_tls:
 ; CHECK-NEXT: .functype  address_of_tls () -> (i32)
 define i32 @address_of_tls() {
   ; TLS-DAG: global.get __tls_base
   ; TLS-DAG: i32.const tls