[WebAssembly] Always include <sysroot>/lib in library path

[sbc100]

Summary

Even though the toolchain supports multiarch, we still want to
be able to support single-arch sysroots.

This also helps for the case where libraries don't correctly install to
the multi-arch directory.

This matches the behavior of the linux toolchain and of gcc.

[sbc100]

tests

[sbc100]

• update comment

[sunfish]

If libraries don't correctly install into multi-arch directories, can we fix the libraries? We do this in the wasi-sdk repo to fix up the libc++ and libc++abi libraries, for example.

[sbc100]

In D61452#1488330, @sunfish wrote:

If libraries don't correctly install into multi-arch directories, can we fix the libraries? We do this in the wasi-sdk repo to fix up the libc++ and libc++abi libraries, for example.

Sure, yes. I've done that on the wasm waterfall too. But its not just that. We might want to make a single-arch sysroot too. Falling back to non-multi-arch seems like a normal thing to do.

[sunfish]

The value of supporting single-arch sysroots is unclear to me. It's always possible to have a sysroot with libraries for just one architecture installed, even with multi-arch paths. Is this just about compatibility with build scripts and tools which are hard-coded to "$sysroot/lib"?

We'll eventually want the ability to have sysroots that support both wasm32 and wasm64. They can share headers, but they'll need their own library directories. If we have libraries installed in "$sysroot/lib", it'll complicate that.

[sbc100]

Even in the case of wasm64 it might well be that somebody wants to build two different sysroots (as they would be arm and arm64 on linux). Even if we don't choose to ship a single arch sysroot for wasi somebody else might.

This doesn't seem like it does any harm to me. It arguable I suppose that it is allowing library authors to delay their transition to mulity-arch install. I don't see why we (WebAssembly driver) should be forcing them to make that transition. This just allows more flexibility for downstream consumers of the compiler.

[sunfish]

If "$sysroot/lib" ends up coming to mean "wasm32" because people come to depend on that, then wasm64 may end up needing to be different in a gratuitous way, which I'd like to avoid.

I'd like to keep our sysroots tidy when we can. If some libraries are installed in lib/wasm32-wasi and others lib for no reason other than build script inertia, that's untidy.

It's not an absolute for me, but I am inclined to see if we can understand the need better first. Single-arch sysroots are possible either way, for example.

[sbc100]

In D61452#1488789, @sunfish wrote:

If "$sysroot/lib" ends up coming to mean "wasm32" because people come to depend on that, then wasm64 may end up needing to be different in a gratuitous way, which I'd like to avoid.

I'd like to keep our sysroots tidy when we can. If some libraries are installed in lib/wasm32-wasi and others lib for no reason other than build script inertia, that's untidy.

It's not an absolute for me, but I am inclined to see if we can understand the need better first. Single-arch sysroots are possible either way, for example.

I think are you wrong here for a couple of reasons.

Firstly, we already look in both the multi-arch directory for include files:

sysroot/include/wasm32-wasi
sysroot/include

Amd for C++ includes:

sysroot/include/wasm32-wasi/c++/v1
sysroot/include/c++/v1

This allows for us to fall back from arch-specific to generic headers as needed. The same can be true of libraries. Not all libraries contains compiled code. .so files can also be linker scripts that reference other libraries in which case they can be arch-neutral.

Secondly, not everyone is going to want to use a multi-arch sysroot. For example I would argue that the wasi SDK would make more sense being single arch since it has exactly one purpose. Its only when installing wasi into an existing system that one really needs to be multi-arch. In any case we should not dictate this. We've already have one person trying to build a wasi-sdk without using multi-arch paths. I'm not sure what you mean by "single arch sysroots are possible either way". The point of a single arch sysroot would be avoid the unnecessary extra path components and I thats exactly what this change is allowing for.

Thirdly, its what the linux driver does, which is the example we are following of how to build a multi-arch toolchain.

[sunfish]

This allows for us to fall back from arch-specific to generic headers as needed. The same can be true of libraries. Not all libraries contains compiled code. .so files can also be linker scripts that reference other libraries in which case they can be arch-neutral.

If we add support for implicit linker scripts in wasm-ld, we can look into this. That said, it's not clear we'll want to. They're rare, and when they are used they're another moving part in the middle of a complex system. There are usually other ways to do what they do.

Secondly, not everyone is going to want to use a multi-arch sysroot. For example I would argue that the wasi SDK would make more sense being single arch since it has exactly one purpose. Its only when installing wasi into an existing system that one really needs to be multi-arch. In any case we should not dictate this.

We're anticipating wasm64. And it's possible WASI could have other triple variants in the future too.

I'm aware some users will sometimes know that they only care about wasm32-wasi. Some of those users may write code, and perhaps some of those that do will share it with others. If such code assumes it can install into $SYSROOT/lib, it may break things for others using a multiarch setup. Unless there are reasons otherwise, it seems valuable to minimize fragmentation between multiarch and non-multiarch users. That seems more valuable than enabling some users to have shorter library paths.

We've already have one person trying to build a wasi-sdk without using multi-arch paths.

They ended up deciding they misunderstood the project for other reasons and left, so it's unclear what conclusion to draw.

I'm not sure what you mean by "single arch sysroots are possible either way". The point of a single arch sysroot would be avoid the unnecessary extra path components and I thats exactly what this change is allowing for.

I just mean you can have a multi-arch-style directory tree with a single arch installed in it.

Thirdly, its what the linux driver does, which is the example we are following of how to build a multi-arch toolchain.

The Linux driver has to stay compatible with existing pre-multi-arch systems; we don't.

clang/lib/Driver/ToolChains/WebAssembly.cpp

 WebAssembly::WebAssembly(const Driver &D, const llvm::Triple &Triple,
                          const llvm::opt::ArgList &Args)
     : ToolChain(D, Triple, Args) {
 
   assert(Triple.isArch32Bit() != Triple.isArch64Bit());
 
   getProgramPaths().push_back(getDriver().getInstalledDir());
 
-  if (getTriple().getOS() == llvm::Triple::UnknownOS) {
-    // Theoretically an "unknown" OS should mean no standard libraries, however
-    // it could also mean that a custom set of libraries is in use, so just add
-    // /lib to the search path. Disable multiarch in this case, to discourage
-    // paths containing "unknown" from acquiring meanings.
-    getFilePaths().push_back(getDriver().SysRoot + "/lib");
-  } else {
+  if (getTriple().getOS() != llvm::Triple::UnknownOS) {
+    // Disable multiarch for UnknownOS, to discourage paths containing "unknown"
+    // from acquiring meanings.
     const std::string MultiarchTriple =
         getMultiarchTriple(getDriver(), Triple, getDriver().SysRoot);
     getFilePaths().push_back(getDriver().SysRoot + "/lib/" + MultiarchTriple);
   }
+
+  getFilePaths().push_back(getDriver().SysRoot + "/lib");
 }
 
 bool WebAssembly::IsMathErrnoDefault() const { return false; }
 
 bool WebAssembly::IsObjCNonFragileABIDefault() const { return true; }
 
 bool WebAssembly::UseObjCMixedDispatch() const { return true; }
 

clang/test/Driver/wasm-toolchain.c

 // RUN: %clang -### -O2 -no-canonical-prefixes -target wasm32-unknown-unknown --sysroot=/foo %s 2>&1 | FileCheck -check-prefix=LINK_OPT %s
 // LINK_OPT: clang{{.*}}" "-cc1" {{.*}} "-o" "[[temp:[^"]*]]"
 // LINK_OPT: wasm-ld{{.*}}" "-L/foo/lib" "crt1.o" "[[temp]]" "-lc" "{{.*[/\\]}}libclang_rt.builtins-wasm32.a" "-o" "a.out"
 
 // A basic C link command-line with known OS.
 
 // RUN: %clang -### -no-canonical-prefixes -target wasm32-wasi --sysroot=/foo %s 2>&1 | FileCheck -check-prefix=LINK_KNOWN %s
 // LINK_KNOWN: clang{{.*}}" "-cc1" {{.*}} "-o" "[[temp:[^"]*]]"
-// LINK_KNOWN: wasm-ld{{.*}}" "-L/foo/lib/wasm32-wasi" "crt1.o" "[[temp]]" "-lc" "{{.*[/\\]}}libclang_rt.builtins-wasm32.a" "-o" "a.out"
+// LINK_KNOWN: wasm-ld{{.*}}" "-L/foo/lib/wasm32-wasi" "-L/foo/lib" "crt1.o" "[[temp]]" "-lc" "{{.*[/\\]}}libclang_rt.builtins-wasm32.a" "-o" "a.out"
 
 // A basic C link command-line with optimization with known OS.
 
 // RUN: %clang -### -O2 -no-canonical-prefixes -target wasm32-wasi --sysroot=/foo %s 2>&1 | FileCheck -check-prefix=LINK_OPT_KNOWN %s
 // LINK_OPT_KNOWN: clang{{.*}}" "-cc1" {{.*}} "-o" "[[temp:[^"]*]]"
-// LINK_OPT_KNOWN: wasm-ld{{.*}}" "-L/foo/lib/wasm32-wasi" "crt1.o" "[[temp]]" "-lc" "{{.*[/\\]}}libclang_rt.builtins-wasm32.a" "-o" "a.out"
+// LINK_OPT_KNOWN: wasm-ld{{.*}}" "-L/foo/lib/wasm32-wasi" "-L/foo/lib" "crt1.o" "[[temp]]" "-lc" "{{.*[/\\]}}libclang_rt.builtins-wasm32.a" "-o" "a.out"
 
 // A basic C compile command-line with known OS.
 
 // RUN: %clang -### -no-canonical-prefixes -target wasm32-wasi --sysroot=/foo %s 2>&1 | FileCheck -check-prefix=COMPILE %s
 // COMPILE: clang{{.*}}" "-cc1" {{.*}} "-internal-isystem" "/foo/include/wasm32-wasi" "-internal-isystem" "/foo/include"
 
 // Thread-related command line tests.
 
 // '-pthread' sets '-target-feature +atomics' and '--shared-memory'
 // RUN: %clang -### -no-canonical-prefixes -target wasm32-unknown-unknown --sysroot=/foo %s -fuse-ld=wasm-ld -pthread 2>&1 | FileCheck -check-prefix=PTHREAD %s
 // PTHREAD: clang{{.*}}" "-cc1" {{.*}} "-target-feature" "+atomics"
 // PTHREAD: wasm-ld{{.*}}" "-lpthread" "--shared-memory"
 
 // '-pthread' not allowed with '-mno-atomics'
 // RUN: %clang -### -no-canonical-prefixes -target wasm32-unknown-unknown --sysroot=/foo %s -pthread -mno-atomics 2>&1 | FileCheck -check-prefix=PTHREAD_NO_ATOMICS %s
 // PTHREAD_NO_ATOMICS: invalid argument '-pthread' not allowed with '-mno-atomics'

clang/test/Driver/wasm-toolchain.cpp

 // RUN: %clangxx -### -O2 -no-canonical-prefixes -target wasm32-unknown-unknown --sysroot=/foo %s --stdlib=c++ 2>&1 | FileCheck -check-prefix=LINK_OPT %s
 // LINK_OPT: clang{{.*}}" "-cc1" {{.*}} "-o" "[[temp:[^"]*]]"
 // LINK_OPT: wasm-ld{{.*}}" "-L/foo/lib" "crt1.o" "[[temp]]" "-lc++" "-lc++abi" "-lc" "{{.*[/\\]}}libclang_rt.builtins-wasm32.a" "-o" "a.out"
 
 // A basic C++ link command-line with known OS.
 
 // RUN: %clangxx -### -no-canonical-prefixes -target wasm32-wasi --sysroot=/foo --stdlib=c++ %s 2>&1 | FileCheck -check-prefix=LINK_KNOWN %s
 // LINK_KNOWN: clang{{.*}}" "-cc1" {{.*}} "-o" "[[temp:[^"]*]]"
-// LINK_KNOWN: wasm-ld{{.*}}" "-L/foo/lib/wasm32-wasi" "crt1.o" "[[temp]]" "-lc++" "-lc++abi" "-lc" "{{.*[/\\]}}libclang_rt.builtins-wasm32.a" "-o" "a.out"
+// LINK_KNOWN: wasm-ld{{.*}}" "-L/foo/lib/wasm32-wasi" "-L/foo/lib" "crt1.o" "[[temp]]" "-lc++" "-lc++abi" "-lc" "{{.*[/\\]}}libclang_rt.builtins-wasm32.a" "-o" "a.out"
 
 // A basic C++ link command-line with optimization with known OS.
 
 // RUN: %clangxx -### -O2 -no-canonical-prefixes -target wasm32-wasi --sysroot=/foo %s --stdlib=c++ 2>&1 | FileCheck -check-prefix=LINK_OPT_KNOWN %s
 // LINK_OPT_KNOWN: clang{{.*}}" "-cc1" {{.*}} "-o" "[[temp:[^"]*]]"
-// LINK_OPT_KNOWN: wasm-ld{{.*}}" "-L/foo/lib/wasm32-wasi" "crt1.o" "[[temp]]" "-lc++" "-lc++abi" "-lc" "{{.*[/\\]}}libclang_rt.builtins-wasm32.a" "-o" "a.out"
+// LINK_OPT_KNOWN: wasm-ld{{.*}}" "-L/foo/lib/wasm32-wasi" "-L/foo/lib" "crt1.o" "[[temp]]" "-lc++" "-lc++abi" "-lc" "{{.*[/\\]}}libclang_rt.builtins-wasm32.a" "-o" "a.out"
 
 // A basic C++ compile command-line with known OS.
 
 // RUN: %clangxx -### -no-canonical-prefixes -target wasm32-wasi --sysroot=/foo --stdlib=c++ %s 2>&1 | FileCheck -check-prefix=COMPILE %s
 // COMPILE: clang{{.*}}" "-cc1" {{.*}} "-internal-isystem" "/foo/include/wasm32-wasi/c++/v1" "-internal-isystem" "/foo/include/c++/v1" "-internal-isystem" "/foo/include/wasm32-wasi" "-internal-isystem" "/foo/include"