ld.lld: Respect -fallback-gnu-driver.

[shlevy]

Summary

This is essentially a run-time version of LLD_DEFAULT_LD_LLD_IS_MINGW.

-fallback-gnu-driver must be the first argument (after -flavor, if it exists).

[MaskRay]

The description does not have information to justify why such an option is needed.
I am concerned with adding more modes to the lld driver with no selected flavor.

I am on a trip so my response my be slow.

[shlevy]

In D124999#3495825, @MaskRay wrote:

The description does not have information to justify why such an option is needed.

We can split up the justification into two parts:

1. Why enable changing the default driver at all? D87418, introducing LLD_DEFAULT_LD_LLD_IS_MINGW, gives the primary justification: there are build systems which grep through --help for certain flags. E.g. any package using libtool (so far I've hit this with gmp and libffi but there are certainly countless others) will think lld can't produce shared libraries. As a secondary consideration, if I'm manually invoking e.g. x86_64-w64-mingw32-ld --help I want to see the usage for the actual command I care about. As a tertiary consideration, if no machine type is set (with -m) then both the ELF driver and the COFF driver (which the MinGW driver uses under the hood) have logic to detect the appropriate machine type from input files, but without the ability to change the default driver there is no way to take advantage of that autodetection via ld.lld for COFF.
2. Why make this knob configurable at run-time? As @Ericson2314 touches on in https://github.com/ClangBuiltLinux/llvm-distributors-conf-2021/issues/20, the LLVM toolchain has a great (and IME unique amongst serious compilers) virtue of target-agnosticism in the build process. This has design benefits, but the immediate cash value is that if I fix build and host I can build one core toolchain and use the same underlying binaries to compile for all of the targets that LLVM supports. Without run-time configuration, to achieve the benefits of point 1 we would have to break this invariant and build a different lld for MinGW targets than for non-MinGW targets. With this (or an analogue that uses env vars, if that's preferred), we can instead simply ensure that the build environment for the specific target passes the right flags (as I do here).

[mstorsjo]

In D124999#3496013, @shlevy wrote:

In D124999#3495825, @MaskRay wrote:

The description does not have information to justify why such an option is needed.

We can split up the justification into two parts:

1. Why enable changing the default driver at all? D87418, introducing LLD_DEFAULT_LD_LLD_IS_MINGW, gives the primary justification: there are build systems which grep through --help for certain flags. E.g. any package using libtool (so far I've hit this with gmp and libffi but there are certainly countless others) will think lld can't produce shared libraries. As a secondary consideration, if I'm manually invoking e.g. x86_64-w64-mingw32-ld --help I want to see the usage for the actual command I care about. As a tertiary consideration, if no machine type is set (with -m) then both the ELF driver and the COFF driver (which the MinGW driver uses under the hood) have logic to detect the appropriate machine type from input files, but without the ability to change the default driver there is no way to take advantage of that autodetection via ld.lld for COFF.
2. Why make this knob configurable at run-time? As @Ericson2314 touches on in https://github.com/ClangBuiltLinux/llvm-distributors-conf-2021/issues/20, the LLVM toolchain has a great (and IME unique amongst serious compilers) virtue of target-agnosticism in the build process. This has design benefits, but the immediate cash value is that if I fix build and host I can build one core toolchain and use the same underlying binaries to compile for all of the targets that LLVM supports. Without run-time configuration, to achieve the benefits of point 1 we would have to break this invariant and build a different lld for MinGW targets than for non-MinGW targets. With this (or an analogue that uses env vars, if that's preferred), we can instead simply ensure that the build environment for the specific target passes the right flags (as I do here).

I guess the main question is that if you configure the default by passing an option (as opposed to an environment variable), why isn't it possible to achieve the same by just passing e.g. -m i386pep? I guess that's somewhat answered by 1. above - doing that loses the chance of letting the linker autodetect things, for the cases where that could be done.

In practice though, I haven't yet used lld in a mingw context where the intended target architecture wasn't known at that point. I'm not familiar with NixOS, but for the cases where you'd want to set this default to make lld default to mingw behaviour, wouldn't you also know the intended architecture at the same time?

[shlevy]

In D124999#3496021, @mstorsjo wrote:

In D124999#3496013, @shlevy wrote:

In D124999#3495825, @MaskRay wrote:

The description does not have information to justify why such an option is needed.

We can split up the justification into two parts:

1. Why enable changing the default driver at all? D87418, introducing LLD_DEFAULT_LD_LLD_IS_MINGW, gives the primary justification: there are build systems which grep through --help for certain flags. E.g. any package using libtool (so far I've hit this with gmp and libffi but there are certainly countless others) will think lld can't produce shared libraries. As a secondary consideration, if I'm manually invoking e.g. x86_64-w64-mingw32-ld --help I want to see the usage for the actual command I care about. As a tertiary consideration, if no machine type is set (with -m) then both the ELF driver and the COFF driver (which the MinGW driver uses under the hood) have logic to detect the appropriate machine type from input files, but without the ability to change the default driver there is no way to take advantage of that autodetection via ld.lld for COFF.
2. Why make this knob configurable at run-time? As @Ericson2314 touches on in https://github.com/ClangBuiltLinux/llvm-distributors-conf-2021/issues/20, the LLVM toolchain has a great (and IME unique amongst serious compilers) virtue of target-agnosticism in the build process. This has design benefits, but the immediate cash value is that if I fix build and host I can build one core toolchain and use the same underlying binaries to compile for all of the targets that LLVM supports. Without run-time configuration, to achieve the benefits of point 1 we would have to break this invariant and build a different lld for MinGW targets than for non-MinGW targets. With this (or an analogue that uses env vars, if that's preferred), we can instead simply ensure that the build environment for the specific target passes the right flags (as I do here).

I guess the main question is that if you configure the default by passing an option (as opposed to an environment variable), why isn't it possible to achieve the same by just passing e.g. -m i386pep? I guess that's somewhat answered by 1. above - doing that loses the chance of letting the linker autodetect things, for the cases where that could be done.

In practice though, I haven't yet used lld in a mingw context where the intended target architecture wasn't known at that point. I'm not familiar with NixOS, but for the cases where you'd want to set this default to make lld default to mingw behaviour, wouldn't you also know the intended architecture at the same time?

I just saw your wrapper-script in llvm-mingw now. If the machine type is truly fixed once you fix the target triple prefix, then this is probably fine for my use case.

[MaskRay]

If there is a way to achieve what you need without adding a new option, I assume this diff can be abandoned?

[shlevy]

I still think it makes sense in parallel to LLD_DEFAULT_LD_LLD_IS_MINGW, but my use case is met.

lld/test/MinGW/driver.test

 APPCONTAINER: -appcontainer
 
 RUN: ld.lld -m i386pep --version 2>&1 | FileCheck -check-prefix=VERSION %s
 RUN: ld.lld -m i386pep -v 2>&1 | FileCheck -check-prefix=VERSION %s
 RUN: not ld.lld -m i386pep -v xyz 2>&1 | FileCheck -check-prefix=VERSION %s
 VERSION: LLD {{.*}} (compatible with GNU linkers)
 
 RUN: ld.lld -m i386pep --help 2>&1 | FileCheck -check-prefix=HELP %s
+RUN: ld.lld -fallback-gnu-driver mingw --help 2>&1 | FileCheck -check-prefix=HELP %s
 HELP: USAGE:
 HELP: --enable-auto-import
 
 RUN: ld.lld -### -m i386pep foo.o -delayload user32.dll --delayload shell32.dll 2>&1 | FileCheck -check-prefix DELAYLOAD %s
 RUN: ld.lld -### -m i386pep foo.o -delayload=user32.dll --delayload=shell32.dll 2>&1 | FileCheck -check-prefix DELAYLOAD %s
 DELAYLOAD: -delayload:user32.dll -delayload:shell32.dll
 
 RUN: not ld.lld -m i386pep -entry 2>&1 | FileCheck -check-prefix MISSING_ARG %s

lld/tools/lld/lld.cpp

 //===- tools/lld/lld.cpp - Linker Driver Dispatcher -----------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
if (Triple(sys::getProcessTriple()).getOS() == Triple::Win32)
     return cl::TokenizeWindowsCommandLine;
   return cl::TokenizeGNUCommandLine;
 }
 
 static bool isPETargetName(StringRef s) {
   return s == "i386pe" || s == "i386pep" || s == "thumb2pe" || s == "arm64pe";
 }
 
+enum GnuDriver { MinGWDriver, ELFDriver, InvalidDriver };
+
+static GnuDriver getDriver(StringRef s) {
+  return StringSwitch<GnuDriver>(s)
+      .CaseLower("elf", ELFDriver)
+      .CaseLower("mingw", MinGWDriver)
+      .Default(InvalidDriver);
+}
+
 static bool isPETarget(std::vector<const char *> &v) {
+  GnuDriver fallbackDriver =
+#ifdef LLD_DEFAULT_LD_LLD_IS_MINGW
+      MinGWDriver;
+#else
+      ELFDriver;
+#endif
+
+  // Parse -fallback-gnu-driver option.
+  if (v.size() > 1 && v[1] == StringRef("-fallback-gnu-driver")) {
+    if (v.size() <= 2)
+      die("missing arg value for '-fallback-gnu-driver'");
+    fallbackDriver = getDriver(v[2]);
+    if (fallbackDriver == InvalidDriver)
+      die("Unknown fallback gnu driver: " + StringRef(v[2]));
+    v.erase(v.begin() + 1, v.begin() + 3);
+  }
+
   for (auto it = v.begin(); it + 1 != v.end(); ++it) {
     if (StringRef(*it) != "-m")
       continue;
     return isPETargetName(*(it + 1));
   }
   // Expand response files (arguments in the form of @<filename>)
   // to allow detecting the -m argument from arguments in them.
   SmallVector<const char *, 256> expandedArgs(v.data(), v.data() + v.size());
   BumpPtrAllocator a;
   StringSaver saver(a);
   cl::ExpandResponseFiles(saver, getDefaultQuotingStyle(), expandedArgs);
   for (auto it = expandedArgs.begin(); it + 1 != expandedArgs.end(); ++it) {
     if (StringRef(*it) != "-m")
       continue;
     return isPETargetName(*(it + 1));
   }
 
-#ifdef LLD_DEFAULT_LD_LLD_IS_MINGW
-  return true;
-#else
-  return false;
-#endif
+  return fallbackDriver == MinGWDriver;
 }
 
 static Flavor parseProgname(StringRef progname) {
   // Use GNU driver for "ld" by default.
   if (progname == "ld")
     return Gnu;
 
   // Progname may be something like "lld-gnu". Parse it.