[Driver] Select newest GCC installation on Solaris

[ro]

Summary

As described in Issue #53709, since 28d58d8fe2094af6902dee7b4d68ec30a3e9d737 clang doesn't find the latest of several parallel GCC installations on Solaris, but only the first in directory order, which is pretty random.

Since this breaks basic functionality of the Solaris port, this patch restores scanning all prefixes for GCC installations.

Tested on sparcv9-sun-solaris2.11, amd64-pc-solaris2.11, and x86_64-pc-linux-gnu.

[MaskRay]

clang/lib/Driver/ToolChains/Gnu.cpp

2193

Adding a Solaris special case here seems strange.

Do you know what the typical Prefixes values are on Solaris? Is it possible to remove some elements as a special case for Solaris instead?

[ro]

clang/lib/Driver/ToolChains/Gnu.cpp

2193

Agreed. This patch in its current form is just meant as a stop-gap measure to unbreak basic Solaris functionality while avoiding to affect other targets.

I've described the value of Prefixes on Solaris in the Issue: its an array of /usr/gcc/<N> for each installed version of GCC, however in directory order and thus random. The exact set is variable since every version can be installed or uninstalled at any time, old versions get deprecated and removed, and new ones added over time.

My fundamental problem here is that I haven't yet been able to reproduce the failure your original patch is meant to fix, so I don't really know what to guard against.

[ro]

Updated based on discussions in Issue #53709:

• Sort Solaris GCC prefixes in reverse version order so the latest version is picked.
• Update testcase to match.

Tested on amd64-pc-solaris2.11 and x86_64-pc-linux-gnu.

I'm seeing weird testsuite failures in 2-stage builds on Solaris/amd64 with gcc 12, but not with clang 16, which I don't yet understand.

[MaskRay]

Thanks for the update. Regarding testing, it seems that unittests will be more convenience than creating so many placeholder files in Inputs/. clang/unittests/Driver/ToolChainTest.cpp TEST(ToolChainTest, VFSGCCInstallation) has an example for Linux. You can add another for Solaris.

clang/lib/Driver/ToolChains/Gnu.cpp

2248–2249

It's better to use https://en.wikipedia.org/wiki/Schwartzian_transform here to avoid calling llvm::sys::path::filename and Generic_GCC::GCCVersion::Parse in the comparator. Then the comparator is just A < B and can just be removed.

[ro]

In D157275#4585184, @MaskRay wrote:

Thanks for the update. Regarding testing, it seems that unittests will be more convenience than creating so many placeholder files in Inputs/. clang/unittests/Driver/ToolChainTest.cpp TEST(ToolChainTest, VFSGCCInstallation) has an example for Linux. You can add another for Solaris.

TBH, I'm quite sceptical about the unittest framework here: we already have 2300+ empty files in clang/test/Driver, so adding 12 more shouldn't be such a big deal. With a real in-filesystem tree, I can quickly test various scenarious with clang -v --sysroot, while this isn't possible with the unittest framework (which I find way harder to read). I'm adding the unittest nonetheless, but would prefer to keep the lit-based test instead.

clang/lib/Driver/ToolChains/Gnu.cpp

2248–2249

This is way shorter and efficient indeed. Patch updated.

[ro]

• Simplify SolarisPrefixes sorting.
• Switch to unitest`.

[MaskRay]

Thanks!

clang/lib/Driver/ToolChains/Gnu.cpp

2248–2249

Use emplace_back

clang/lib/Driver/ToolChains/Gnu.cpp

for (StringRef Suffix : CandidateBiarchLibDirs) {
       bool GCCDirExists = VFS.exists(LibDir + "/gcc");
       bool GCCCrossDirExists = VFS.exists(LibDir + "/gcc-cross");
       for (StringRef Candidate : CandidateBiarchTripleAliases)
         ScanLibDirForGCCTriple(TargetTriple, Args, LibDir, Candidate, true,
                                GCCDirExists, GCCCrossDirExists);
     }
 
     // Skip other prefixes once a GCC installation is found.
     if (Version > VersionZero)
       break;
   }
 }
 
 void Generic_GCC::GCCInstallationDetector::print(raw_ostream &OS) const {
   for (const auto &InstallPath : CandidateGCCInstallPaths)
     OS << "Found candidate GCC installation: " << InstallPath << "\n";
 
void Generic_GCC::GCCInstallationDetector::AddDefaultGCCPrefixes(
     StringRef SysRoot) {
   if (TargetTriple.getOS() == llvm::Triple::Solaris) {
     // Solaris is a special case.
     // The GCC installation is under
     //   /usr/gcc/<major>.<minor>/lib/gcc/<triple>/<major>.<minor>.<patch>/
     // so we need to find those /usr/gcc/*/lib/gcc libdirs and go with
     // /usr/gcc/<version> as a prefix.
 
+    SmallVector<std::pair<GCCVersion, std::string>, 8> SolarisPrefixes;
     std::string PrefixDir = concat(SysRoot, "/usr/gcc");
     std::error_code EC;
     for (llvm::vfs::directory_iterator LI = D.getVFS().dir_begin(PrefixDir, EC),
                                        LE;
          !EC && LI != LE; LI = LI.increment(EC)) {
       StringRef VersionText = llvm::sys::path::filename(LI->path());
       GCCVersion CandidateVersion = GCCVersion::Parse(VersionText);
 
       // Filter out obviously bad entries.
       if (CandidateVersion.Major == -1 || CandidateVersion.isOlderThan(4, 1, 1))
         continue;
 
       std::string CandidatePrefix = PrefixDir + "/" + VersionText.str();
       std::string CandidateLibPath = CandidatePrefix + "/lib/gcc";
       if (!D.getVFS().exists(CandidateLibPath))
         continue;
 
-      Prefixes.push_back(CandidatePrefix);
+      SolarisPrefixes.emplace_back(
+          std::make_pair(CandidateVersion, CandidatePrefix));
     }
+    // Sort in reverse order so GCCInstallationDetector::init picks the latest.
+    std::sort(SolarisPrefixes.rbegin(), SolarisPrefixes.rend());
+    for (auto p : SolarisPrefixes)
+      Prefixes.emplace_back(p.second);
     return;
   }
 
   // For Linux, if --sysroot is not specified, look for RHEL/CentOS devtoolsets
   // and gcc-toolsets.
   if (SysRoot.empty() && TargetTriple.getOS() == llvm::Triple::Linux &&
       D.getVFS().exists("/opt/rh")) {
     // TODO: We may want to remove this, since the functionality

clang/unittests/Driver/ToolChainTest.cpp

EXPECT_EQ("Found candidate GCC installation: "
             "/home/test/bin/../lib/gcc/arm-linux-gnueabi/4.6.1\n"
             "Selected GCC installation: "
             "/home/test/bin/../lib/gcc/arm-linux-gnueabi/4.6.1\n"
             "Candidate multilib: .;@m32\n"
             "Selected multilib: .;@m32\n",
             S);
 }
 
+TEST(ToolChainTest, VFSSolarisMultiGCCInstallation) {
+  IntrusiveRefCntPtr<DiagnosticOptions> DiagOpts = new DiagnosticOptions();
+
+  IntrusiveRefCntPtr<DiagnosticIDs> DiagID(new DiagnosticIDs());
+  struct TestDiagnosticConsumer : public DiagnosticConsumer {};
+  IntrusiveRefCntPtr<llvm::vfs::InMemoryFileSystem> InMemoryFileSystem(
+      new llvm::vfs::InMemoryFileSystem);
+
+  const char *EmptyFiles[] = {
+      // Sort entries so the latest version doesn't come first.
+      "/usr/gcc/7/lib/gcc/sparcv9-sun-solaris2.11/7.5.0/32/crtbegin.o",
+      "/usr/gcc/7/lib/gcc/sparcv9-sun-solaris2.11/7.5.0/crtbegin.o",
+      "/usr/gcc/7/lib/gcc/x86_64-pc-solaris2.11/7.5.0/32/crtbegin.o",
+      "/usr/gcc/7/lib/gcc/x86_64-pc-solaris2.11/7.5.0/crtbegin.o",
+      "/usr/gcc/11/lib/gcc/sparcv9-sun-solaris2.11/11.4.0/crtbegin.o",
+      "/usr/gcc/11/lib/gcc/sparcv9-sun-solaris2.11/11.4.0/sparcv8plus/"
+      "crtbegin.o",
+      "/usr/gcc/11/lib/gcc/x86_64-pc-solaris2.11/11.4.0/32/crtbegin.o",
+      "/usr/gcc/11/lib/gcc/x86_64-pc-solaris2.11/11.4.0/crtbegin.o",
+      "/usr/gcc/4.7/lib/gcc/i386-pc-solaris2.11/4.7.3/amd64/crtbegin.o",
+      "/usr/gcc/4.7/lib/gcc/i386-pc-solaris2.11/4.7.3/crtbegin.o",
+      "/usr/gcc/4.7/lib/gcc/sparc-sun-solaris2.11/4.7.3/crtbegin.o",
+      "/usr/gcc/4.7/lib/gcc/sparc-sun-solaris2.11/4.7.3/sparcv9/crtbegin.o",
+  };
+
+  for (const char *Path : EmptyFiles)
+    InMemoryFileSystem->addFile(Path, 0,
+                                llvm::MemoryBuffer::getMemBuffer("\n"));
+
+  {
+    DiagnosticsEngine Diags(DiagID, &*DiagOpts, new TestDiagnosticConsumer);
+    Driver TheDriver("/bin/clang", "i386-pc-solaris2.11", Diags,
+                     "clang LLVM compiler", InMemoryFileSystem);
+    std::unique_ptr<Compilation> C(
+        TheDriver.BuildCompilation({"-v", "--gcc-toolchain=", "--sysroot="}));
+    ASSERT_TRUE(C);
+    std::string S;
+    {
+      llvm::raw_string_ostream OS(S);
+      C->getDefaultToolChain().printVerboseInfo(OS);
+    }
+    if (is_style_windows(llvm::sys::path::Style::native))
+      std::replace(S.begin(), S.end(), '\\', '/');
+    EXPECT_EQ("Found candidate GCC installation: "
+              "/usr/gcc/11/lib/gcc/x86_64-pc-solaris2.11/11.4.0\n"
+              "Selected GCC installation: "
+              "/usr/gcc/11/lib/gcc/x86_64-pc-solaris2.11/11.4.0\n"
+              "Candidate multilib: .;@m64\n"
+              "Candidate multilib: 32;@m32\n"
+              "Selected multilib: 32;@m32\n",
+              S);
+  }
+
+  {
+    DiagnosticsEngine Diags(DiagID, &*DiagOpts, new TestDiagnosticConsumer);
+    Driver TheDriver("/bin/clang", "amd64-pc-solaris2.11", Diags,
+                     "clang LLVM compiler", InMemoryFileSystem);
+    std::unique_ptr<Compilation> C(
+        TheDriver.BuildCompilation({"-v", "--gcc-toolchain=", "--sysroot="}));
+    ASSERT_TRUE(C);
+    std::string S;
+    {
+      llvm::raw_string_ostream OS(S);
+      C->getDefaultToolChain().printVerboseInfo(OS);
+    }
+    if (is_style_windows(llvm::sys::path::Style::native))
+      std::replace(S.begin(), S.end(), '\\', '/');
+    EXPECT_EQ("Found candidate GCC installation: "
+              "/usr/gcc/11/lib/gcc/x86_64-pc-solaris2.11/11.4.0\n"
+              "Selected GCC installation: "
+              "/usr/gcc/11/lib/gcc/x86_64-pc-solaris2.11/11.4.0\n"
+              "Candidate multilib: .;@m64\n"
+              "Candidate multilib: 32;@m32\n"
+              "Selected multilib: .;@m64\n",
+              S);
+  }
+
+  {
+    DiagnosticsEngine Diags(DiagID, &*DiagOpts, new TestDiagnosticConsumer);
+    Driver TheDriver("/bin/clang", "x86_64-pc-solaris2.11", Diags,
+                     "clang LLVM compiler", InMemoryFileSystem);
+    std::unique_ptr<Compilation> C(
+        TheDriver.BuildCompilation({"-v", "--gcc-toolchain=", "--sysroot="}));
+    ASSERT_TRUE(C);
+    std::string S;
+    {
+      llvm::raw_string_ostream OS(S);
+      C->getDefaultToolChain().printVerboseInfo(OS);
+    }
+    if (is_style_windows(llvm::sys::path::Style::native))
+      std::replace(S.begin(), S.end(), '\\', '/');
+    EXPECT_EQ("Found candidate GCC installation: "
+              "/usr/gcc/11/lib/gcc/x86_64-pc-solaris2.11/11.4.0\n"
+              "Selected GCC installation: "
+              "/usr/gcc/11/lib/gcc/x86_64-pc-solaris2.11/11.4.0\n"
+              "Candidate multilib: .;@m64\n"
+              "Candidate multilib: 32;@m32\n"
+              "Selected multilib: .;@m64\n",
+              S);
+  }
+
+  {
+    DiagnosticsEngine Diags(DiagID, &*DiagOpts, new TestDiagnosticConsumer);
+    Driver TheDriver("/bin/clang", "sparc-sun-solaris2.11", Diags,
+                     "clang LLVM compiler", InMemoryFileSystem);
+    std::unique_ptr<Compilation> C(
+        TheDriver.BuildCompilation({"-v", "--gcc-toolchain=", "--sysroot="}));
+    ASSERT_TRUE(C);
+    std::string S;
+    {
+      llvm::raw_string_ostream OS(S);
+      C->getDefaultToolChain().printVerboseInfo(OS);
+    }
+    if (is_style_windows(llvm::sys::path::Style::native))
+      std::replace(S.begin(), S.end(), '\\', '/');
+    EXPECT_EQ("Found candidate GCC installation: "
+              "/usr/gcc/11/lib/gcc/sparcv9-sun-solaris2.11/11.4.0\n"
+              "Selected GCC installation: "
+              "/usr/gcc/11/lib/gcc/sparcv9-sun-solaris2.11/11.4.0\n"
+              "Candidate multilib: .;@m64\n"
+              "Candidate multilib: sparcv8plus;@m32\n"
+              "Selected multilib: sparcv8plus;@m32\n",
+              S);
+  }
+  {
+    DiagnosticsEngine Diags(DiagID, &*DiagOpts, new TestDiagnosticConsumer);
+    Driver TheDriver("/bin/clang", "sparcv9-sun-solaris2.11", Diags,
+                     "clang LLVM compiler", InMemoryFileSystem);
+    std::unique_ptr<Compilation> C(
+        TheDriver.BuildCompilation({"-v", "--gcc-toolchain=", "--sysroot="}));
+    ASSERT_TRUE(C);
+    std::string S;
+    {
+      llvm::raw_string_ostream OS(S);
+      C->getDefaultToolChain().printVerboseInfo(OS);
+    }
+    if (is_style_windows(llvm::sys::path::Style::native))
+      std::replace(S.begin(), S.end(), '\\', '/');
+    EXPECT_EQ("Found candidate GCC installation: "
+              "/usr/gcc/11/lib/gcc/sparcv9-sun-solaris2.11/11.4.0\n"
+              "Selected GCC installation: "
+              "/usr/gcc/11/lib/gcc/sparcv9-sun-solaris2.11/11.4.0\n"
+              "Candidate multilib: .;@m64\n"
+              "Candidate multilib: sparcv8plus;@m32\n"
+              "Selected multilib: .;@m64\n",
+              S);
+  }
+}
+
 TEST(ToolChainTest, DefaultDriverMode) {
   IntrusiveRefCntPtr<DiagnosticOptions> DiagOpts = new DiagnosticOptions();
 
   IntrusiveRefCntPtr<DiagnosticIDs> DiagID(new DiagnosticIDs());
   struct TestDiagnosticConsumer : public DiagnosticConsumer {};
   DiagnosticsEngine Diags(DiagID, &*DiagOpts, new TestDiagnosticConsumer);
   IntrusiveRefCntPtr<llvm::vfs::InMemoryFileSystem> InMemoryFileSystem(
       new llvm::vfs::InMemoryFileSystem);