[clang] [Driver] Fall back to default.cfg when calling clang w/o prefix

[mgorny]

Summary

Complement the configuration file search logic to use
default-<mode-suffix>.cfg or default.cfg if clang is called
via the executable without a prefix specifying target. This makes it
possible for vendors to supply a full set of configuration files
matching all installed executables.

[Arfrever]

clang/docs/UsersManual.rst

911–912

This says armv7l.cfg...

clang/lib/Driver/Driver.cpp

1026–1030

But this says armv7l-clang.cfg.
Could you also fix this inconsistency?

[asymptotically]

Updated users' manual following Arfrever's feedback.

[tstellar]

A similar RFC for this was proposed here: https://discourse.llvm.org/t/rfc-adding-a-default-file-location-to-config-file-support/63606

[mgorny]

As @sepavloff points out in the linked discussion, we probably need an option to disable loading default configuration files too, and we should use it in the test suite (via modifying the %clang substitutions, I guess), as otherwise someone enabling config file support is going to see random test failures.

[mstorsjo]

On this topic, it would be great if we could pick up a per-target default config file too, if clang is invoked with clang -target <triple>. Currently this is done automatically if clang is invoked as <triple>-clang, but not with an explicit -target parameter.

[mgorny]

After discussing privately with @asymptotically, I'm going to try to finish this.

[mgorny]

So far my plan is to go for default-{ModeSuffix}.cfg instead (e.g. default-clang.cfg, default-clang++.cfg, etc.). If I'm not mistaken, this will trigger the existing fallback logic stripping mode suffix and trying default.cfg if suffixed config is not found. Also add tests ;-).

[MaskRay]

clang/docs/UsersManual.rst

911–912

These single backsticks probably should be double backsticks.

[MaskRay]

Thanks for taking over the patch!

I've CCed (in https://discourse.llvm.org/t/rfc-adding-a-default-file-location-to-config-file-support/63606/33) some folks who had strong opinions in the initial support patch.
(Cc @joerg manually whom I cannot find a discourse username for)

[mgorny]

Changed the base config filename to default.cfg, and included looking up default-<mode-suffix>.cfg first. Added tests.

clang/docs/UsersManual.rst

911–912

To be honest, I was wondering why it was done like that.

[mgorny]

As an example, we are currently installing the following set of executables + symlinks for clang in Gentoo on amd64 with full multilib:

lrwxrwxrwx 1 root root    8 Sep 17 12:31 /usr/lib/llvm/16/bin/clang -> clang-16
lrwxrwxrwx 1 root root   10 Sep 17 12:31 /usr/lib/llvm/16/bin/clang++ -> clang++-16
lrwxrwxrwx 1 root root    8 Sep 17 12:31 /usr/lib/llvm/16/bin/clang++-16 -> clang-16
-rwxr-xr-x 1 root root 170K Sep 17 12:31 /usr/lib/llvm/16/bin/clang-16
lrwxrwxrwx 1 root root   11 Sep 17 12:31 /usr/lib/llvm/16/bin/clang-cl -> clang-cl-16
lrwxrwxrwx 1 root root    8 Sep 17 12:31 /usr/lib/llvm/16/bin/clang-cl-16 -> clang-16
lrwxrwxrwx 1 root root   12 Sep 17 12:31 /usr/lib/llvm/16/bin/clang-cpp -> clang-cpp-16
lrwxrwxrwx 1 root root    8 Sep 17 12:31 /usr/lib/llvm/16/bin/clang-cpp-16 -> clang-16
lrwxrwxrwx 1 root root   26 Sep 17 12:31 /usr/lib/llvm/16/bin/i686-pc-linux-gnu-clang -> i686-pc-linux-gnu-clang-16
lrwxrwxrwx 1 root root   28 Sep 17 12:31 /usr/lib/llvm/16/bin/i686-pc-linux-gnu-clang++ -> i686-pc-linux-gnu-clang++-16
lrwxrwxrwx 1 root root   10 Sep 17 12:31 /usr/lib/llvm/16/bin/i686-pc-linux-gnu-clang++-16 -> clang++-16
lrwxrwxrwx 1 root root    8 Sep 17 12:31 /usr/lib/llvm/16/bin/i686-pc-linux-gnu-clang-16 -> clang-16
lrwxrwxrwx 1 root root   29 Sep 17 12:31 /usr/lib/llvm/16/bin/i686-pc-linux-gnu-clang-cl -> i686-pc-linux-gnu-clang-cl-16
lrwxrwxrwx 1 root root   11 Sep 17 12:31 /usr/lib/llvm/16/bin/i686-pc-linux-gnu-clang-cl-16 -> clang-cl-16
lrwxrwxrwx 1 root root   30 Sep 17 12:31 /usr/lib/llvm/16/bin/i686-pc-linux-gnu-clang-cpp -> i686-pc-linux-gnu-clang-cpp-16
lrwxrwxrwx 1 root root   12 Sep 17 12:31 /usr/lib/llvm/16/bin/i686-pc-linux-gnu-clang-cpp-16 -> clang-cpp-16
lrwxrwxrwx 1 root root   28 Sep 17 12:31 /usr/lib/llvm/16/bin/x86_64-pc-linux-gnu-clang -> x86_64-pc-linux-gnu-clang-16
lrwxrwxrwx 1 root root   30 Sep 17 12:31 /usr/lib/llvm/16/bin/x86_64-pc-linux-gnu-clang++ -> x86_64-pc-linux-gnu-clang++-16
lrwxrwxrwx 1 root root   10 Sep 17 12:31 /usr/lib/llvm/16/bin/x86_64-pc-linux-gnu-clang++-16 -> clang++-16
lrwxrwxrwx 1 root root    8 Sep 17 12:31 /usr/lib/llvm/16/bin/x86_64-pc-linux-gnu-clang-16 -> clang-16
lrwxrwxrwx 1 root root   31 Sep 17 12:31 /usr/lib/llvm/16/bin/x86_64-pc-linux-gnu-clang-cl -> x86_64-pc-linux-gnu-clang-cl-16
lrwxrwxrwx 1 root root   11 Sep 17 12:31 /usr/lib/llvm/16/bin/x86_64-pc-linux-gnu-clang-cl-16 -> clang-cl-16
lrwxrwxrwx 1 root root   32 Sep 17 12:31 /usr/lib/llvm/16/bin/x86_64-pc-linux-gnu-clang-cpp -> x86_64-pc-linux-gnu-clang-cpp-16
lrwxrwxrwx 1 root root   12 Sep 17 12:31 /usr/lib/llvm/16/bin/x86_64-pc-linux-gnu-clang-cpp-16 -> clang-cpp-16
lrwxrwxrwx 1 root root   31 Sep 17 12:31 /usr/lib/llvm/16/bin/x86_64-pc-linux-gnux32-clang -> x86_64-pc-linux-gnux32-clang-16
lrwxrwxrwx 1 root root   33 Sep 17 12:31 /usr/lib/llvm/16/bin/x86_64-pc-linux-gnux32-clang++ -> x86_64-pc-linux-gnux32-clang++-16
lrwxrwxrwx 1 root root   10 Sep 17 12:31 /usr/lib/llvm/16/bin/x86_64-pc-linux-gnux32-clang++-16 -> clang++-16
lrwxrwxrwx 1 root root    8 Sep 17 12:31 /usr/lib/llvm/16/bin/x86_64-pc-linux-gnux32-clang-16 -> clang-16
lrwxrwxrwx 1 root root   34 Sep 17 12:31 /usr/lib/llvm/16/bin/x86_64-pc-linux-gnux32-clang-cl -> x86_64-pc-linux-gnux32-clang-cl-16
lrwxrwxrwx 1 root root   11 Sep 17 12:31 /usr/lib/llvm/16/bin/x86_64-pc-linux-gnux32-clang-cl-16 -> clang-cl-16
lrwxrwxrwx 1 root root   35 Sep 17 12:31 /usr/lib/llvm/16/bin/x86_64-pc-linux-gnux32-clang-cpp -> x86_64-pc-linux-gnux32-clang-cpp-16
lrwxrwxrwx 1 root root   12 Sep 17 12:31 /usr/lib/llvm/16/bin/x86_64-pc-linux-gnux32-clang-cpp-16 -> clang-cpp-16

…and we can cover all of them using four config files:

-rw-r--r-- 1 root root  0 Sep 17 13:30 default.cfg
lrwxrwxrwx 1 root root 11 Sep 17 12:31 i686-pc-linux-gnu.cfg -> default.cfg
lrwxrwxrwx 1 root root 11 Sep 17 12:31 x86_64-pc-linux-gnu.cfg -> default.cfg
lrwxrwxrwx 1 root root 11 Sep 17 12:31 x86_64-pc-linux-gnux32.cfg -> default.cfg

[mgorny]

Fix documentation formatting. While at it, fix a doc mistake that referenced x86_64-cl.cfg instead of the correct x86_64-clang-cl.cfg.

[sepavloff]

clang/docs/UsersManual.rst

915

What about using default config based on driver mode (clang.cfg, gcc.cfg etc)? It looks like a more flexible solution, - different driver modes may need different default settings. Even if these setting are identical, the default config files may be links to one, say clang.cfg or just include that file with @file.

[mgorny]

clang/docs/UsersManual.rst

915

It's actually described below — you can have default-clang.cfg, default-clang++.cfg, etc. Using this naming makes it possible to use the same logic for default as for prefixed execs.

[sepavloff]

In D109621#3791511, @mstorsjo wrote:

On this topic, it would be great if we could pick up a per-target default config file too, if clang is invoked with clang -target <triple>. Currently this is done automatically if clang is invoked as <triple>-clang, but not with an explicit -target parameter.

Would enabling config files in such invocations be a solution for your use case?

[mgorny]

In D109621#3799133, @sepavloff wrote:

In D109621#3791511, @mstorsjo wrote:

On this topic, it would be great if we could pick up a per-target default config file too, if clang is invoked with clang -target <triple>. Currently this is done automatically if clang is invoked as <triple>-clang, but not with an explicit -target parameter.

Would enabling config files in such invocations be a solution for your use case?

Not really, we don't strictly need per-target config files at the moment. There might be some use for them in the future but right now our most dire need is to be able to ensure that our config file is included in all invocations (except for explicit --no-default-config).

With this patch, this is handled by having files for all installed prefixes + default* for unprefixed executables.

I suppose an alternative option would be to do a major refactoring to support a wider range of naming variants with fallbacks. Would you prefer if I tried to do that?

[mgorny]

Abandoning in favor of D134337.

clang/docs/UsersManual.rst

 Configuration files
 -------------------
 
 Configuration files group command-line options and allow all of them to be
 specified just by referencing the configuration file. They may be used, for
 example, to collect options required to tune compilation for particular
 target, such as -L, -I, -l, --sysroot, codegen options, etc.
 
-The command line option `--config` can be used to specify configuration
+The command line option ``--config`` can be used to specify configuration
 file in a Clang invocation. For example:
 
 ::
 
     clang --config /home/user/cfgs/testing.txt
     clang --config debug.cfg
 
 If the provided argument contains a directory separator, it is considered as
 a file path, and options are read from that file. Otherwise the argument is
 treated as a file name and is searched for sequentially in the directories:
 
     - user directory,
     - system directory,
     - the directory where Clang executable resides.
 
 Both user and system directories for configuration files are specified during
 clang build using CMake parameters, CLANG_CONFIG_FILE_USER_DIR and
 CLANG_CONFIG_FILE_SYSTEM_DIR respectively. The first file found is used. It is
 an error if the required file cannot be found.
 
 If no explicit configuration file is specified, Clang searches for a default
 configuration file following the rules described in the next paragraphs.
-To disable this behavior, `--no-default-config` flag can be used.
+To disable this behavior, ``--no-default-config`` flag can be used.
 
 Another way to specify a configuration file is to encode it in executable name.
-For example, if the Clang executable is named `armv7l-clang` (it may be a
-symbolic link to `clang`), then Clang will search for file `armv7l.cfg` in the
-directory where Clang resides.
+For example, if the Clang executable is named ``armv7l-clang`` (it may be a
+symbolic link to ``clang``), then Clang will search for file ``armv7l.cfg``
+in the directory where Clang resides. If Clang is started via an executable
+without target prefix, Clang will use ``default.cfg`` instead.
 
 If a driver mode is specified in invocation, Clang tries to find a file specific
 for the specified mode. For example, if the executable file is named
-`x86_64-clang-cl`, Clang first looks for `x86_64-cl.cfg` and if it is not found,
-looks for `x86_64.cfg`.
+``x86_64-clang-cl``, Clang first looks for ``x86_64-clang-cl.cfg`` and if it is
+not found, looks for ``x86_64.cfg``. Similarly, for ``clang-cl``, it looks for
+``default-clang-cl.cfg`` before trying ``default.cfg``.
 
 If the command line contains options that effectively change target architecture
 (these are -m32, -EL, and some others) and the configuration file starts with an
 architecture name, Clang tries to load the configuration file for the effective
 architecture. For example, invocation:
 
 ::
 
     x86_64-clang -m32 abc.c
 
-causes Clang search for a file `i368.cfg` first, and if no such file is found,
-Clang looks for the file `x86_64.cfg`.
+causes Clang search for a file ``i368.cfg`` first, and if no such file is found,
+Clang looks for the file ``x86_64.cfg``.
 
 The configuration file consists of command-line options specified on one or
 more lines. Lines composed of whitespace characters only are ignored as well as
-lines in which the first non-blank character is `#`. Long options may be split
+lines in which the first non-blank character is ``#``. Long options may be split
 between several lines by a trailing backslash. Here is example of a
 configuration file:
 
 ::
 
     # Several options on line
     -c --target=x86_64-unknown-linux-gnu
 
     # Long option split between lines
     -I/usr/lib/gcc/x86_64-linux-gnu/5.4.0/../../../../\
     include/c++/5.4.0
 
     # other config files may be included
     @linux.options
 
-Files included by `@file` directives in configuration files are resolved
+Files included by ``@file`` directives in configuration files are resolved
 relative to the including file. For example, if a configuration file
-`~/.llvm/target.cfg` contains the directive `@os/linux.opts`, the file
-`linux.opts` is searched for in the directory `~/.llvm/os`.
+``~/.llvm/target.cfg`` contains the directive ``@os/linux.opts``, the file
+``linux.opts`` is searched for in the directory ``~/.llvm/os``.
 
-To generate paths relative to the configuration file, the `<CFGDIR>` token may
+To generate paths relative to the configuration file, the ``<CFGDIR>`` token may
 be used. This will expand to the absolute path of the directory containing the
 configuration file.
 
 In cases where a configuration file is deployed alongside SDK contents, the
-SDK directory can remain fully portable by using `<CFGDIR>` prefixed paths.
+SDK directory can remain fully portable by using ``<CFGDIR>`` prefixed paths.
 In this way, the user may only need to specify a root configuration file with
-`--config` to establish every aspect of the SDK with the compiler:
+``--config`` to establish every aspect of the SDK with the compiler:
 
 ::
 
     --target=foo
     -isystem <CFGDIR>/include
     -L <CFGDIR>/lib
     -T <CFGDIR>/ldscripts/link.ld
 

clang/lib/Driver/Driver.cpp

if (!ConfigFiles.empty()) {
           }
         }
         return readConfigFile(CfgFilePath);
       }
 
       FileSpecifiedExplicitly = true;
     }
   }
 
-  if (!(CLOptions && CLOptions->hasArg(options::OPT_no_default_config))) {
+  if (CfgFileName.empty()) {
+    if (CLOptions && CLOptions->hasArg(options::OPT_no_default_config))
+      return false;
+
     // If config file is not specified explicitly, try to deduce configuration
     // from executable name. For instance, an executable 'armv7l-clang' will
-    // search for config file 'armv7l-clang.cfg'.
-    if (CfgFileName.empty() && !ClangNameParts.TargetPrefix.empty())
+    // search for config file 'armv7l-clang.cfg'. If we are called without
+    // a specific prefix, use e.g. 'default-clang.cfg' instead.
+    if (!ClangNameParts.TargetPrefix.empty())
       CfgFileName =
           ClangNameParts.TargetPrefix + '-' + ClangNameParts.ModeSuffix;
+    else
+      CfgFileName = "default-" + ClangNameParts.ModeSuffix;
   }
 
-  if (CfgFileName.empty())
-    return false;
-
   // Determine architecture part of the file name, if it is present.
   StringRef CfgFileArch = CfgFileName;
   size_t ArchPrefixLen = CfgFileArch.find('-');
   if (ArchPrefixLen == StringRef::npos)
     ArchPrefixLen = CfgFileArch.size();
   llvm::Triple CfgTriple;
   CfgFileArch = CfgFileArch.take_front(ArchPrefixLen);
   CfgTriple = llvm::Triple(llvm::Triple::normalize(CfgFileArch));
if (searchForFile(CfgFilePath, CfgFileSearchDirs, FixedConfigFile,
       return readConfigFile(CfgFilePath);
   }
 
   // Then try original file name.
   if (searchForFile(CfgFilePath, CfgFileSearchDirs, CfgFileName, getVFS()))
     return readConfigFile(CfgFilePath);
 
   // Finally try removing driver mode part: 'x86_64-clang.cfg' -> 'x86_64.cfg'.
-  if (!ClangNameParts.ModeSuffix.empty() &&
-      !ClangNameParts.TargetPrefix.empty()) {
-    CfgFileName.assign(ClangNameParts.TargetPrefix);
+  if (!FileSpecifiedExplicitly && !ClangNameParts.ModeSuffix.empty()) {
+    CfgFileName.resize(CfgFileName.size() - ClangNameParts.ModeSuffix.size() -
+                       5);
     CfgFileName.append(".cfg");
     if (searchForFile(CfgFilePath, CfgFileSearchDirs, CfgFileName, getVFS()))
       return readConfigFile(CfgFilePath);
   }
 
   // Report error but only if config file was specified explicitly, by option
   // --config. If it was deduced from executable name, it is not an error.
   if (FileSpecifiedExplicitly) {

clang/test/Driver/config-file3.c

 
 
 //--- Invocation qqq-clang-g++ tries to find config file qqq-clang-g++.cfg first.
 //
 // RUN: mkdir %t/testdmode
 // RUN: ln -s %clang %t/testdmode/qqq-clang-g++
 // RUN: echo "-Wundefined-func-template" > %t/testdmode/qqq-clang-g++.cfg
 // RUN: echo "-Werror" > %t/testdmode/qqq.cfg
+// RUN: ln -s %clang %t/testdmode/clang-g++
+// RUN: ln -s %clang %t/testdmode/clang
+// RUN: : > %t/testdmode/default-clang.cfg
+// RUN: : > %t/testdmode/default.cfg
 // RUN: %t/testdmode/qqq-clang-g++ --config-system-dir= --config-user-dir= -c -no-canonical-prefixes -### %s 2>&1 | FileCheck %s -check-prefix FULL-NAME
 //
 // FULL-NAME: Configuration file: {{.*}}/testdmode/qqq-clang-g++.cfg
 // FULL-NAME: -Wundefined-func-template
 // FULL-NAME-NOT: -Werror
 //
 //--- Invocation qqq-clang-g++ tries to find config file qqq-clang-g++.cfg even without -no-canonical-prefixes.
 // (As the clang executable and symlink are in different directories, this
 // requires specifying the path via --config-*-dir= though.)
 //
 // RUN: %t/testdmode/qqq-clang-g++ --config-system-dir= --config-user-dir=%t/testdmode -c -### %s 2>&1 | FileCheck %s -check-prefix SYMLINK
 //
 // SYMLINK: Configuration file: {{.*}}/testdmode/qqq-clang-g++.cfg
 //
 //--- File specified by --config overrides config inferred from clang executable.
 //
 // RUN: %t/testdmode/qqq-clang-g++ --config-system-dir=%S/Inputs/config --config-user-dir= --config i386-qqq -c -no-canonical-prefixes -### %s 2>&1 | FileCheck %s -check-prefix CHECK-EXPLICIT
 //
 // CHECK-EXPLICIT: Configuration file: {{.*}}/Inputs/config/i386-qqq.cfg
 
+//--- Invoking clang falls back to default-clang.cfg.
+// RUN: %t/testdmode/clang --config-system-dir= --config-user-dir= -c -no-canonical-prefixes -### %s 2>&1 | FileCheck %s -check-prefix DEFAULT-CLANG
+//
+// DEFAULT-CLANG: Configuration file: {{.*}}/testdmode/default-clang.cfg
+
+//--- Invoking clang-g++ falls back to default.cfg.
+// RUN: %t/testdmode/clang-g++ --config-system-dir= --config-user-dir= -c -no-canonical-prefixes -### %s 2>&1 | FileCheck %s -check-prefix DEFAULT
+//
+// DEFAULT: Configuration file: {{.*}}/testdmode/default.cfg
+
 //--- --no-default-config disables config search.
 //
 // RUN: %t/testdmode/qqq-clang-g++ --config-system-dir= --config-user-dir=%t/testdmode --no-default-config -c -### %s 2>&1 | FileCheck %s -check-prefix NO-DEFAULT-CONFIG
 //
 // NO-DEFAULT-CONFIG-NOT: Configuration file:
 
 //--- Invocation qqq-clang-g++ tries to find config file qqq.cfg if qqq-clang-g++.cfg is not found.
 //