[Driver] Add --gcc-install-dir=

[MaskRay]

Summary

This option specifies a GCC installation directory such as
/usr/lib/gcc/x86_64-linux-gnu/12, /usr/lib/gcc/x86_64-gentoo-linux-musl/11.2.0 .

It is intended to replace --gcc-toolchain=, which specifies a directory where
lib/gcc{,-cross} can be found. When --gcc-toolchain= is specified, the
selected lib/gcc/$triple/$version installation uses complex logic and the
largest GCC version is picked. There is no way to specify another version in the
presence of multiple GCC versions.

D25661 added gcc-config detection for Gentoo: ScanGentooConfigs.
The implementation may be simplified by using --gcc-install-dir=.

[MaskRay]

improve test

[mgorny]

At a first glance, it works. I'm now building GCC:11 here to test it against two real installations.

[mgorny]

Ok, this works fine for me (though I haven't run the tests yet). If D109621 and this one lands, I think the old Gentoo logic can be removed. We'll still have to update gcc-config to keep updating the config files but I'm pretty sure we can do that in time for 16.0.0.

[mgorny]

Unfortunately, we have some test failures after this change, e.g.:

FAIL: Clang :: CodeGenHLSL/basic_types.hlsl (6525 of 15795)
******************** TEST 'Clang :: CodeGenHLSL/basic_types.hlsl' FAILED ********************
Script:
--
: 'RUN: at line 1';   /var/tmp/portage/sys-devel/clang-16.0.0_pre20220915/work/x/y/clang-abi_x86_32.x86/bin/clang --driver-mode=dxc  -Tlib_6_7 -fcgl -Fo - /var/tmp/portage/sys-devel/clang-16.0.0_pre20220915/work/clang/test/CodeGenHLSL/basic_types.hlsl | /usr/lib/llvm/16/bin/FileCheck /var/tmp/portage/sys-devel/clang-16.0.0_pre20220915/work/clang/test/CodeGenHLSL/basic_types.hlsl
--
Exit Code: 2

Command Output (stderr):
--
+ : 'RUN: at line 1'
+ /usr/lib/llvm/16/bin/FileCheck /var/tmp/portage/sys-devel/clang-16.0.0_pre20220915/work/clang/test/CodeGenHLSL/basic_types.hlsl
+ /var/tmp/portage/sys-devel/clang-16.0.0_pre20220915/work/x/y/clang-abi_x86_32.x86/bin/clang --driver-mode=dxc -Tlib_6_7 -fcgl -Fo - /var/tmp/portage/sys-devel/clang-16.0.0_pre20220915/work/clang/test/CodeGenHLSL/basic_types.hlsl
clang-16: error: unknown argument: '--gcc-install-dir=/usr/lib/gcc/x86_64-pc-linux-gnu/12.2.1'
FileCheck error: '<stdin>' is empty.
FileCheck command line:  /usr/lib/llvm/16/bin/FileCheck /var/tmp/portage/sys-devel/clang-16.0.0_pre20220915/work/clang/test/CodeGenHLSL/basic_types.hlsl

--

********************

Unfortunately, within the current config framework, it's impossible to set options per --driver-mode=... (and i'm not convinced we really want to pursue keeping a bunch of config files to figure out which driver modes use GCC and which do not). Would it be feasible to add this option (and other similar options — e.g. -rtlib, -unwindlib, -stdlib) to all drivers, and make them emit argument unused warnings like -fuse-ld?

[mgorny]

Hmm, apparently this could be achieved by making it a "CoreOption".

[MaskRay]

In D133329#3798166, @mgorny wrote:

Hmm, apparently this could be achieved by making it a "CoreOption".

I don't think it is appropriate to have CoreOption. The Windows dxc support appears to have nothing to do with gcc installation unless I am mistaken.
It's for configuration file support to figure out how to work with --driver-mode, not this patch which causes no test failure if you don't place --driver-mode in a default config file (which isn't a supported case).

In general I am not sure we can reasonably support placing arbitrary options into a default config and expect all test suite to pass.
For --rtlib= --unwindlib= --stdlib= etc, yeah, we currently add the option to fix many parameters, but it is infeasible if this expands to more configurable defaults.

[mgorny]

I think you misunderstood me. The aforementioned tests fail if I put --gcc-install-dir= in the config file. There is no way to put it in the config file that wouldn't affect these tests right now.

[MaskRay]

In D133329#3798371, @mgorny wrote:

I think you misunderstood me. The aforementioned tests fail if I put --gcc-install-dir= in the config file. There is no way to put it in the config file that wouldn't affect these tests right now.

This is unsupported. It's like if you put -W in a default config file it may change a lot of things. We cannot reasonably support such tests without introducing a lot of complexity to a lot of driver tests.

[mgorny]

Let's tackle the config changes independently and not block this on them.

clang/include/clang/Basic/DiagnosticDriverKinds.td

 
 def err_drv_invalid_value : Error<"invalid value '%1' in '%0'">;
 def err_drv_invalid_int_value : Error<"invalid integral value '%1' in '%0'">;
 def err_drv_invalid_value_with_suggestion : Error<
     "invalid value '%1' in '%0', expected one of: %2">;
 def err_drv_alignment_not_power_of_two : Error<"alignment is not a power of 2 in '%0'">;
 def err_drv_invalid_remap_file : Error<
     "invalid option '%0' not of the form <from-file>;<to-file>">;
+def err_drv_invalid_gcc_install_dir : Error<"'%0' does not contain a GCC installation">;
 def err_drv_invalid_gcc_output_type : Error<
     "invalid output type '%0' for use with gcc tool">;
 def err_drv_cc_print_options_failure : Error<
     "unable to open CC_PRINT_OPTIONS file: %0">;
 def err_drv_lto_without_lld : Error<"LTO requires -fuse-ld=lld">;
 def err_drv_preamble_format : Error<
     "incorrect format for -preamble-bytes=N,END">;
 def err_drv_header_unit_extra_inputs : Error<

clang/include/clang/Driver/Options.td

 def _HASH_HASH_HASH : Flag<["-"], "###">, Flags<[NoXarchOption, CoreOption, FlangOption]>,
     HelpText<"Print (but do not run) the commands to run for this compilation">;
 def _DASH_DASH : Option<["--"], "", KIND_REMAINING_ARGS>,
     Flags<[NoXarchOption, CoreOption]>;
 def A : JoinedOrSeparate<["-"], "A">, Flags<[RenderJoined]>, Group<gfortran_Group>;
 def B : JoinedOrSeparate<["-"], "B">, MetaVarName<"<prefix>">,
     HelpText<"Search $prefix$file for executables, libraries, and data files. "
     "If $prefix is a directory, search $prefix/$file">;
+def gcc_install_dir_EQ : Joined<["--"], "gcc-install-dir=">,
+  HelpText<"Use GCC installation in the specified directory. The directory ends with path components like 'lib{,32,64}/gcc{,-cross}/$triple/$version'. "
+  "Note: executables (e.g. ld) used by the compiler are not overridden by the selected GCC installation">;
 def gcc_toolchain : Joined<["--"], "gcc-toolchain=">, Flags<[NoXarchOption]>,
-  HelpText<"Search for GCC installation in the specified directory on targets which commonly use GCC. "
-  "The directory usually contains 'lib{,32,64}/gcc{,-cross}/$triple' and 'include'. If specified, "
-  "sysroot is skipped for GCC detection. Note: executables (e.g. ld) used by the compiler are not "
-  "overridden by the selected GCC installation">;
+  HelpText<"Specify a directory where Clang can find 'include' and 'lib{,32,64}/gcc{,-cross}/$triple/$version'. "
+  "Clang will use the GCC installation with the largest version">;
 def CC : Flag<["-"], "CC">, Flags<[CC1Option]>, Group<Preprocessor_Group>,
     HelpText<"Include comments from within macros in preprocessed output">,
     MarshallingInfoFlag<PreprocessorOutputOpts<"ShowMacroComments">>;
 def C : Flag<["-"], "C">, Flags<[CC1Option]>, Group<Preprocessor_Group>,
     HelpText<"Include comments in preprocessed output">,
     MarshallingInfoFlag<PreprocessorOutputOpts<"ShowComments">>;
 def D : JoinedOrSeparate<["-"], "D">, Group<Preprocessor_Group>,
     Flags<[CC1Option, FlangOption, FC1Option]>, MetaVarName<"<macro>=<value>">,

clang/lib/Driver/ToolChains/Gnu.cpp

void Generic_GCC::GCCInstallationDetector::init(
   SmallVector<StringRef, 4> CandidateLibDirs, CandidateBiarchLibDirs;
   // The compatible GCC triples for this particular architecture.
   SmallVector<StringRef, 16> CandidateTripleAliases;
   SmallVector<StringRef, 16> CandidateBiarchTripleAliases;
   CollectLibDirsAndTriples(TargetTriple, BiarchVariantTriple, CandidateLibDirs,
                            CandidateTripleAliases, CandidateBiarchLibDirs,
                            CandidateBiarchTripleAliases);
 
+  // If --gcc-install-dir= is specified, skip filesystem detection.
+  if (const Arg *A =
+          Args.getLastArg(clang::driver::options::OPT_gcc_install_dir_EQ);
+      A && A->getValue()[0]) {
+    StringRef InstallDir = A->getValue();
+    if (!ScanGCCForMultilibs(TargetTriple, Args, InstallDir, false)) {
+      D.Diag(diag::err_drv_invalid_gcc_install_dir) << InstallDir;
+    } else {
+      (void)InstallDir.consume_back("/");
+      StringRef VersionText = llvm::sys::path::filename(InstallDir);
+      StringRef TripleText =
+          llvm::sys::path::filename(llvm::sys::path::parent_path(InstallDir));
+
+      Version = GCCVersion::Parse(VersionText);
+      GCCTriple.setTriple(TripleText);
+      GCCInstallPath = std::string(InstallDir);
+      GCCParentLibPath = GCCInstallPath + "/../../..";
+      IsValid = true;
+    }
+    return;
+  }
+
   // Compute the set of prefixes for our search.
   SmallVector<std::string, 8> Prefixes;
   StringRef GCCToolchainDir = getGCCToolchainDir(Args, D.SysRoot);
   if (GCCToolchainDir != "") {
     if (GCCToolchainDir.back() == '/')
       GCCToolchainDir = GCCToolchainDir.drop_back(); // remove the /
 
     Prefixes.push_back(std::string(GCCToolchainDir));

clang/test/Driver/gcc-install-dir.cpp

+// UNSUPPORTED: system-windows
+
+/// Test native GCC installation on Arch Linux i686.
+// RUN: %clang -### %s --target=i686-linux-gnu --sysroot=%S/Inputs/archlinux_i686_tree -ccc-install-dir %S/Inputs/basic_linux_tree/usr/bin \
+// RUN:   --stdlib=platform --rtlib=platform \
+// RUN:   --gcc-install-dir=%S/Inputs/archlinux_i686_tree/usr/lib/gcc/i686-pc-linux-gnu/11.1.0 2>&1 | FileCheck %s --check-prefix=ARCH_I686
+// ARCH_I686:      "-internal-isystem"
+// ARCH_I686-SAME: {{^}} "[[SYSROOT:[^"]+]]/usr/lib/gcc/i686-pc-linux-gnu/11.1.0/../../../../include/c++/11.1.0"
+// ARCH_I686-SAME: {{^}} "-internal-isystem" "[[SYSROOT:[^"]+]]/usr/lib/gcc/i686-pc-linux-gnu/11.1.0/../../../../include/c++/11.1.0/i686-pc-linux-gnu"
+// ARCH_I686:      "-L
+// ARCH_I686-SAME: {{^}}[[SYSROOT]]/usr/lib/gcc/i686-pc-linux-gnu/11.1.0"
+// ARCH_I686-SAME: {{^}} "-L[[SYSROOT]]/lib"
+// ARCH_I686-SAME: {{^}} "-L[[SYSROOT]]/usr/lib"
+
+/// Test native GCC installation on Debian amd64. --gcc-install-dir= may end with /.
+// RUN: %clangxx %s -### --target=x86_64-unknown-linux-gnu --sysroot=%S/Inputs/debian_multiarch_tree \
+// RUN:   -ccc-install-dir %S/Inputs/basic_linux_tree/usr/bin -resource-dir=%S/Inputs/resource_dir --stdlib=platform --rtlib=platform \
+// RUN:   --gcc-install-dir=%S/Inputs/debian_multiarch_tree/usr/lib/gcc/x86_64-linux-gnu/10/ 2>&1 | FileCheck %s --check-prefix=DEBIAN_X86_64
+// DEBIAN_X86_64:      "-internal-isystem"
+// DEBIAN_X86_64-SAME: {{^}} "[[SYSROOT:[^"]+]]/usr/lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10"
+// DEBIAN_X86_64-SAME: {{^}} "-internal-isystem" "[[SYSROOT:[^"]+]]/usr/lib/gcc/x86_64-linux-gnu/10/../../../../include/x86_64-linux-gnu/c++/10"
+// DEBIAN_X86_64-SAME: {{^}} "-internal-isystem" "[[SYSROOT]]/usr/lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/backward"
+/// We set explicit -ccc-install-dir ensure that Clang does not pick up extra
+/// library directories which may be present in the runtimes build.
+// DEBIAN_X86_64:      "-L
+// DEBIAN_X86_64-SAME: {{^}}[[SYSROOT]]/usr/lib/gcc/x86_64-linux-gnu/10"
+// DEBIAN_X86_64-SAME: {{^}} "-L[[SYSROOT]]/usr/lib/gcc/x86_64-linux-gnu/10/../../../../lib64"
+
+/// Test -m32.
+// RUN: %clangxx %s -### --target=x86_64-unknown-linux-gnu -m32 --sysroot=%S/Inputs/debian_multiarch_tree \
+// RUN:   -ccc-install-dir %S/Inputs/basic_linux_tree/usr/bin -resource-dir=%S/Inputs/resource_dir --stdlib=platform --rtlib=platform \
+// RUN:   --gcc-install-dir=%S/Inputs/debian_multiarch_tree/usr/lib/gcc/x86_64-linux-gnu/10/ 2>&1 | FileCheck %s --check-prefix=DEBIAN_X86_64_M32
+// DEBIAN_X86_64_M32:      "-internal-isystem"
+// DEBIAN_X86_64_M32-SAME: {{^}} "[[SYSROOT:[^"]+]]/usr/lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10"
+// DEBIAN_X86_64_M32-SAME: {{^}} "-internal-isystem" "[[SYSROOT:[^"]+]]/usr/lib/gcc/x86_64-linux-gnu/10/../../../../include/x86_64-linux-gnu/c++/10/32"
+// DEBIAN_X86_64_M32:      "-L
+// DEBIAN_X86_64_M32-SAME: {{^}}[[SYSROOT]]/usr/lib/gcc/x86_64-linux-gnu/10/32"
+// DEBIAN_X86_64_M32-SAME: {{^}} "-L[[SYSROOT]]/usr/lib/gcc/x86_64-linux-gnu/10/../../../../lib32"
+
+// RUN: %clangxx %s -### --target=x86_64-unknown-linux-gnu --sysroot=%S/Inputs/debian_multiarch_tree \
+// RUN:   -ccc-install-dir %S/Inputs/basic_linux_tree/usr/bin -resource-dir=%S/Inputs/resource_dir --stdlib=platform --rtlib=platform \
+// RUN:   --gcc-install-dir=%S/Inputs/debian_multiarch_tree/usr/lib/gcc/x86_64-linux-gnu 2>&1 | FileCheck %s --check-prefix=INVALID
+// INVALID: error: '{{.*}}/usr/lib/gcc/x86_64-linux-gnu' does not contain a GCC installation