[clang] [Driver] Fall back to default.cfg when calling clang w/o prefix

[mgorny]

Summary

Complement the configuration file search logic to use
default-<mode-suffix>.cfg or default.cfg if clang is called
via the executable without a prefix specifying target. This makes it
possible for vendors to supply a full set of configuration files
matching all installed executables.

[Arfrever]

clang/docs/UsersManual.rst

913

This says armv7l.cfg...

clang/lib/Driver/Driver.cpp

1026–1030

But this says armv7l-clang.cfg.
Could you also fix this inconsistency?

[asymptotically]

Updated users' manual following Arfrever's feedback.

[tstellar]

A similar RFC for this was proposed here: https://discourse.llvm.org/t/rfc-adding-a-default-file-location-to-config-file-support/63606

[mgorny]

As @sepavloff points out in the linked discussion, we probably need an option to disable loading default configuration files too, and we should use it in the test suite (via modifying the %clang substitutions, I guess), as otherwise someone enabling config file support is going to see random test failures.

[mstorsjo]

On this topic, it would be great if we could pick up a per-target default config file too, if clang is invoked with clang -target <triple>. Currently this is done automatically if clang is invoked as <triple>-clang, but not with an explicit -target parameter.

[mgorny]

After discussing privately with @asymptotically, I'm going to try to finish this.

[mgorny]

So far my plan is to go for default-{ModeSuffix}.cfg instead (e.g. default-clang.cfg, default-clang++.cfg, etc.). If I'm not mistaken, this will trigger the existing fallback logic stripping mode suffix and trying default.cfg if suffixed config is not found. Also add tests ;-).

[MaskRay]

clang/docs/UsersManual.rst

912–913

These single backsticks probably should be double backsticks.

[MaskRay]

Thanks for taking over the patch!

I've CCed (in https://discourse.llvm.org/t/rfc-adding-a-default-file-location-to-config-file-support/63606/33) some folks who had strong opinions in the initial support patch.
(Cc @joerg manually whom I cannot find a discourse username for)

[mgorny]

Changed the base config filename to default.cfg, and included looking up default-<mode-suffix>.cfg first. Added tests.

clang/docs/UsersManual.rst

912–913

To be honest, I was wondering why it was done like that.

[mgorny]

As an example, we are currently installing the following set of executables + symlinks for clang in Gentoo on amd64 with full multilib:

lrwxrwxrwx 1 root root    8 Sep 17 12:31 /usr/lib/llvm/16/bin/clang -> clang-16
lrwxrwxrwx 1 root root   10 Sep 17 12:31 /usr/lib/llvm/16/bin/clang++ -> clang++-16
lrwxrwxrwx 1 root root    8 Sep 17 12:31 /usr/lib/llvm/16/bin/clang++-16 -> clang-16
-rwxr-xr-x 1 root root 170K Sep 17 12:31 /usr/lib/llvm/16/bin/clang-16
lrwxrwxrwx 1 root root   11 Sep 17 12:31 /usr/lib/llvm/16/bin/clang-cl -> clang-cl-16
lrwxrwxrwx 1 root root    8 Sep 17 12:31 /usr/lib/llvm/16/bin/clang-cl-16 -> clang-16
lrwxrwxrwx 1 root root   12 Sep 17 12:31 /usr/lib/llvm/16/bin/clang-cpp -> clang-cpp-16
lrwxrwxrwx 1 root root    8 Sep 17 12:31 /usr/lib/llvm/16/bin/clang-cpp-16 -> clang-16
lrwxrwxrwx 1 root root   26 Sep 17 12:31 /usr/lib/llvm/16/bin/i686-pc-linux-gnu-clang -> i686-pc-linux-gnu-clang-16
lrwxrwxrwx 1 root root   28 Sep 17 12:31 /usr/lib/llvm/16/bin/i686-pc-linux-gnu-clang++ -> i686-pc-linux-gnu-clang++-16
lrwxrwxrwx 1 root root   10 Sep 17 12:31 /usr/lib/llvm/16/bin/i686-pc-linux-gnu-clang++-16 -> clang++-16
lrwxrwxrwx 1 root root    8 Sep 17 12:31 /usr/lib/llvm/16/bin/i686-pc-linux-gnu-clang-16 -> clang-16
lrwxrwxrwx 1 root root   29 Sep 17 12:31 /usr/lib/llvm/16/bin/i686-pc-linux-gnu-clang-cl -> i686-pc-linux-gnu-clang-cl-16
lrwxrwxrwx 1 root root   11 Sep 17 12:31 /usr/lib/llvm/16/bin/i686-pc-linux-gnu-clang-cl-16 -> clang-cl-16
lrwxrwxrwx 1 root root   30 Sep 17 12:31 /usr/lib/llvm/16/bin/i686-pc-linux-gnu-clang-cpp -> i686-pc-linux-gnu-clang-cpp-16
lrwxrwxrwx 1 root root   12 Sep 17 12:31 /usr/lib/llvm/16/bin/i686-pc-linux-gnu-clang-cpp-16 -> clang-cpp-16
lrwxrwxrwx 1 root root   28 Sep 17 12:31 /usr/lib/llvm/16/bin/x86_64-pc-linux-gnu-clang -> x86_64-pc-linux-gnu-clang-16
lrwxrwxrwx 1 root root   30 Sep 17 12:31 /usr/lib/llvm/16/bin/x86_64-pc-linux-gnu-clang++ -> x86_64-pc-linux-gnu-clang++-16
lrwxrwxrwx 1 root root   10 Sep 17 12:31 /usr/lib/llvm/16/bin/x86_64-pc-linux-gnu-clang++-16 -> clang++-16
lrwxrwxrwx 1 root root    8 Sep 17 12:31 /usr/lib/llvm/16/bin/x86_64-pc-linux-gnu-clang-16 -> clang-16
lrwxrwxrwx 1 root root   31 Sep 17 12:31 /usr/lib/llvm/16/bin/x86_64-pc-linux-gnu-clang-cl -> x86_64-pc-linux-gnu-clang-cl-16
lrwxrwxrwx 1 root root   11 Sep 17 12:31 /usr/lib/llvm/16/bin/x86_64-pc-linux-gnu-clang-cl-16 -> clang-cl-16
lrwxrwxrwx 1 root root   32 Sep 17 12:31 /usr/lib/llvm/16/bin/x86_64-pc-linux-gnu-clang-cpp -> x86_64-pc-linux-gnu-clang-cpp-16
lrwxrwxrwx 1 root root   12 Sep 17 12:31 /usr/lib/llvm/16/bin/x86_64-pc-linux-gnu-clang-cpp-16 -> clang-cpp-16
lrwxrwxrwx 1 root root   31 Sep 17 12:31 /usr/lib/llvm/16/bin/x86_64-pc-linux-gnux32-clang -> x86_64-pc-linux-gnux32-clang-16
lrwxrwxrwx 1 root root   33 Sep 17 12:31 /usr/lib/llvm/16/bin/x86_64-pc-linux-gnux32-clang++ -> x86_64-pc-linux-gnux32-clang++-16
lrwxrwxrwx 1 root root   10 Sep 17 12:31 /usr/lib/llvm/16/bin/x86_64-pc-linux-gnux32-clang++-16 -> clang++-16
lrwxrwxrwx 1 root root    8 Sep 17 12:31 /usr/lib/llvm/16/bin/x86_64-pc-linux-gnux32-clang-16 -> clang-16
lrwxrwxrwx 1 root root   34 Sep 17 12:31 /usr/lib/llvm/16/bin/x86_64-pc-linux-gnux32-clang-cl -> x86_64-pc-linux-gnux32-clang-cl-16
lrwxrwxrwx 1 root root   11 Sep 17 12:31 /usr/lib/llvm/16/bin/x86_64-pc-linux-gnux32-clang-cl-16 -> clang-cl-16
lrwxrwxrwx 1 root root   35 Sep 17 12:31 /usr/lib/llvm/16/bin/x86_64-pc-linux-gnux32-clang-cpp -> x86_64-pc-linux-gnux32-clang-cpp-16
lrwxrwxrwx 1 root root   12 Sep 17 12:31 /usr/lib/llvm/16/bin/x86_64-pc-linux-gnux32-clang-cpp-16 -> clang-cpp-16

…and we can cover all of them using four config files:

-rw-r--r-- 1 root root  0 Sep 17 13:30 default.cfg
lrwxrwxrwx 1 root root 11 Sep 17 12:31 i686-pc-linux-gnu.cfg -> default.cfg
lrwxrwxrwx 1 root root 11 Sep 17 12:31 x86_64-pc-linux-gnu.cfg -> default.cfg
lrwxrwxrwx 1 root root 11 Sep 17 12:31 x86_64-pc-linux-gnux32.cfg -> default.cfg

[mgorny]

Fix documentation formatting. While at it, fix a doc mistake that referenced x86_64-cl.cfg instead of the correct x86_64-clang-cl.cfg.

[sepavloff]

clang/docs/UsersManual.rst

917

What about using default config based on driver mode (clang.cfg, gcc.cfg etc)? It looks like a more flexible solution, - different driver modes may need different default settings. Even if these setting are identical, the default config files may be links to one, say clang.cfg or just include that file with @file.

[mgorny]

clang/docs/UsersManual.rst

917

It's actually described below — you can have default-clang.cfg, default-clang++.cfg, etc. Using this naming makes it possible to use the same logic for default as for prefixed execs.

[sepavloff]

In D109621#3791511, @mstorsjo wrote:

On this topic, it would be great if we could pick up a per-target default config file too, if clang is invoked with clang -target <triple>. Currently this is done automatically if clang is invoked as <triple>-clang, but not with an explicit -target parameter.

Would enabling config files in such invocations be a solution for your use case?

[mgorny]

In D109621#3799133, @sepavloff wrote:

In D109621#3791511, @mstorsjo wrote:

On this topic, it would be great if we could pick up a per-target default config file too, if clang is invoked with clang -target <triple>. Currently this is done automatically if clang is invoked as <triple>-clang, but not with an explicit -target parameter.

Would enabling config files in such invocations be a solution for your use case?

Not really, we don't strictly need per-target config files at the moment. There might be some use for them in the future but right now our most dire need is to be able to ensure that our config file is included in all invocations (except for explicit --no-default-config).

With this patch, this is handled by having files for all installed prefixes + default* for unprefixed executables.

I suppose an alternative option would be to do a major refactoring to support a wider range of naming variants with fallbacks. Would you prefer if I tried to do that?

[mgorny]

Abandoning in favor of D134337.

clang/docs/UsersManual.rst

 CLANG_CONFIG_FILE_SYSTEM_DIR respectively. The first file found is used. It is
 an error if the required file cannot be found.
 
 If no explicit configuration file is specified, Clang searches for a default
 configuration file following the rules described in the next paragraphs.
 To disable this behavior, `--no-default-config` flag can be used.
 
 Another way to specify a configuration file is to encode it in executable name.
 For example, if the Clang executable is named `armv7l-clang` (it may be a
 symbolic link to `clang`), then Clang will search for file `armv7l.cfg` in the
-directory where Clang resides.
+directory where Clang resides. If Clang is started via an executable without
+target prefix, Clang will use `default.cfg` instead.
 
 If a driver mode is specified in invocation, Clang tries to find a file specific
 for the specified mode. For example, if the executable file is named
 `x86_64-clang-cl`, Clang first looks for `x86_64-cl.cfg` and if it is not found,
-looks for `x86_64.cfg`.
+looks for `x86_64.cfg`. Similarly, for `clang-cl`, it looks for `default-cl.cfg`
+before trying `default.cfg`.
 
 If the command line contains options that effectively change target architecture
 (these are -m32, -EL, and some others) and the configuration file starts with an
 architecture name, Clang tries to load the configuration file for the effective
 architecture. For example, invocation:
 
 ::
 

clang/lib/Driver/Driver.cpp

if (!ConfigFiles.empty()) {
           }
         }
         return readConfigFile(CfgFilePath);
       }
 
       FileSpecifiedExplicitly = true;
     }
   }
 
-  if (!(CLOptions && CLOptions->hasArg(options::OPT_no_default_config))) {
+  if (CfgFileName.empty()) {
+    if (CLOptions && CLOptions->hasArg(options::OPT_no_default_config))
+      return false;
+
     // If config file is not specified explicitly, try to deduce configuration
     // from executable name. For instance, an executable 'armv7l-clang' will
-    // search for config file 'armv7l-clang.cfg'.
-    if (CfgFileName.empty() && !ClangNameParts.TargetPrefix.empty())
+    // search for config file 'armv7l-clang.cfg'. If we are called without
+    // a specific prefix, use e.g. 'default-clang.cfg' instead.
+    if (!ClangNameParts.TargetPrefix.empty())
       CfgFileName =
           ClangNameParts.TargetPrefix + '-' + ClangNameParts.ModeSuffix;
+    else
+      CfgFileName = "default-" + ClangNameParts.ModeSuffix;
   }
 
-  if (CfgFileName.empty())
-    return false;
-
   // Determine architecture part of the file name, if it is present.
   StringRef CfgFileArch = CfgFileName;
   size_t ArchPrefixLen = CfgFileArch.find('-');
   if (ArchPrefixLen == StringRef::npos)
     ArchPrefixLen = CfgFileArch.size();
   llvm::Triple CfgTriple;
   CfgFileArch = CfgFileArch.take_front(ArchPrefixLen);
   CfgTriple = llvm::Triple(llvm::Triple::normalize(CfgFileArch));
if (searchForFile(CfgFilePath, CfgFileSearchDirs, FixedConfigFile,
       return readConfigFile(CfgFilePath);
   }
 
   // Then try original file name.
   if (searchForFile(CfgFilePath, CfgFileSearchDirs, CfgFileName, getVFS()))
     return readConfigFile(CfgFilePath);
 
   // Finally try removing driver mode part: 'x86_64-clang.cfg' -> 'x86_64.cfg'.
-  if (!ClangNameParts.ModeSuffix.empty() &&
-      !ClangNameParts.TargetPrefix.empty()) {
-    CfgFileName.assign(ClangNameParts.TargetPrefix);
+  if (!FileSpecifiedExplicitly && !ClangNameParts.ModeSuffix.empty()) {
+    CfgFileName.resize(CfgFileName.size() - ClangNameParts.ModeSuffix.size() -
+                       5);
     CfgFileName.append(".cfg");
     if (searchForFile(CfgFilePath, CfgFileSearchDirs, CfgFileName, getVFS()))
       return readConfigFile(CfgFilePath);
   }
 
   // Report error but only if config file was specified explicitly, by option
   // --config. If it was deduced from executable name, it is not an error.
   if (FileSpecifiedExplicitly) {

clang/test/Driver/config-file3.c

 
 
 //--- Invocation qqq-clang-g++ tries to find config file qqq-clang-g++.cfg first.
 //
 // RUN: mkdir %t/testdmode
 // RUN: ln -s %clang %t/testdmode/qqq-clang-g++
 // RUN: echo "-Wundefined-func-template" > %t/testdmode/qqq-clang-g++.cfg
 // RUN: echo "-Werror" > %t/testdmode/qqq.cfg
+// RUN: ln -s %clang %t/testdmode/clang-g++
+// RUN: ln -s %clang %t/testdmode/clang
+// RUN: : > %t/testdmode/default-clang.cfg
+// RUN: : > %t/testdmode/default.cfg
 // RUN: %t/testdmode/qqq-clang-g++ --config-system-dir= --config-user-dir= -c -no-canonical-prefixes -### %s 2>&1 | FileCheck %s -check-prefix FULL-NAME
 //
 // FULL-NAME: Configuration file: {{.*}}/testdmode/qqq-clang-g++.cfg
 // FULL-NAME: -Wundefined-func-template
 // FULL-NAME-NOT: -Werror
 //
 //--- Invocation qqq-clang-g++ tries to find config file qqq-clang-g++.cfg even without -no-canonical-prefixes.
 // (As the clang executable and symlink are in different directories, this
 // requires specifying the path via --config-*-dir= though.)
 //
 // RUN: %t/testdmode/qqq-clang-g++ --config-system-dir= --config-user-dir=%t/testdmode -c -### %s 2>&1 | FileCheck %s -check-prefix SYMLINK
 //
 // SYMLINK: Configuration file: {{.*}}/testdmode/qqq-clang-g++.cfg
 //
 //--- File specified by --config overrides config inferred from clang executable.
 //
 // RUN: %t/testdmode/qqq-clang-g++ --config-system-dir=%S/Inputs/config --config-user-dir= --config i386-qqq -c -no-canonical-prefixes -### %s 2>&1 | FileCheck %s -check-prefix CHECK-EXPLICIT
 //
 // CHECK-EXPLICIT: Configuration file: {{.*}}/Inputs/config/i386-qqq.cfg
 
+//--- Invoking clang falls back to default-clang.cfg.
+// RUN: %t/testdmode/clang --config-system-dir= --config-user-dir= -c -no-canonical-prefixes -### %s 2>&1 | FileCheck %s -check-prefix DEFAULT-CLANG
+//
+// DEFAULT-CLANG: Configuration file: {{.*}}/testdmode/default-clang.cfg
+
+//--- Invoking clang-g++ falls back to default.cfg.
+// RUN: %t/testdmode/clang-g++ --config-system-dir= --config-user-dir= -c -no-canonical-prefixes -### %s 2>&1 | FileCheck %s -check-prefix DEFAULT
+//
+// DEFAULT: Configuration file: {{.*}}/testdmode/default.cfg
+
 //--- --no-default-config disables config search.
 //
 // RUN: %t/testdmode/qqq-clang-g++ --config-system-dir= --config-user-dir=%t/testdmode --no-default-config -c -### %s 2>&1 | FileCheck %s -check-prefix NO-DEFAULT-CONFIG
 //
 // NO-DEFAULT-CONFIG-NOT: Configuration file:
 
 //--- Invocation qqq-clang-g++ tries to find config file qqq.cfg if qqq-clang-g++.cfg is not found.
 //