[Driver] Use normalized triples for multiarch runtime path

[phosek]

Summary

Previously we used target triple as provided which matches the GCC
behavior, but it also means that all clients have to be consistent
in their spelling of target triples since e.g. x86_64-linux-gnu and
x86_64-unknown-linux-gnu will result in Clang driver looking at two
different paths when searching for runtime libraries.

Unfortunately, as it turned out many clients aren't consistent in
their spelling of target triples, e.g. many Linux distributions use
the shorter spelling but config.guess and rustc insist on using the
normalized variant which is causing issues. To avoid having to ship
multiple copies of runtimes for different triple spelling or rely on
symlinks which are not portable, we should use the normalized triple
when constructing paths for multiarch runtimes.

[rnk]

Would it be to much to check both the normalized and non-normalized?

[phosek]

In D50547#1198218, @rnk wrote:

Would it be to much to check both the normalized and non-normalized?

Done, currently I'm only checking the provided target triple followed by the normalized one, I'm not checking the effective triple but it'd be easy to add if we decide to do that as well.

[beanz]

LGTM!

[beanz]

Just want to bring an IRC conversation that I had with @phosek into the proper code review.

This patch is great as-is, but if we want to extend this to Darwin there are some problems because of how Darwin handles triples. Specifically Darwin has multiple potentially valid triples that can mean the same things. For example x86_64-apple-darwin17.7.0 and x86_64-apple-macos10.13.6 are 100% interchangeable (even down to referring to the same OS version). Things get even stranger when you start talking about the -simulator triples...

This means that to support this in Darwin we need to support iterating over a list of triples. We also probably want to cache that list so that we don't run getVFS().exists(...) over and over again. @phosek said on IRC he will update the patch to reflect our conversation.

Thanks @phosek for all the great work on this!

[phosek]

I also need to create the following empty files for tests to pass (somehow those are not displayed by Phabricator):

clang/test/Driver/Inputs/resource_dir_with_per_target_subdir/aarch64-fuchsia/lib/libclang_rt.asan-preinit.a
clang/test/Driver/Inputs/resource_dir_with_per_target_subdir/aarch64-fuchsia/lib/libclang_rt.asan.so
clang/test/Driver/Inputs/resource_dir_with_per_target_subdir/aarch64-fuchsia/lib/libclang_rt.builtins.a
clang/test/Driver/Inputs/resource_dir_with_per_target_subdir/aarch64-fuchsia/lib/libclang_rt.fuzzer.a
clang/test/Driver/Inputs/resource_dir_with_per_target_subdir/aarch64-fuchsia/lib/libclang_rt.scudo.so
clang/test/Driver/Inputs/resource_dir_with_per_target_subdir/i386-linux-gnu/lib/libclang_rt.builtins.a
clang/test/Driver/Inputs/resource_dir_with_per_target_subdir/x86_64-fuchsia/lib/libclang_rt.asan-preinit.a
clang/test/Driver/Inputs/resource_dir_with_per_target_subdir/x86_64-fuchsia/lib/libclang_rt.asan.so
clang/test/Driver/Inputs/resource_dir_with_per_target_subdir/x86_64-fuchsia/lib/libclang_rt.builtins.a
clang/test/Driver/Inputs/resource_dir_with_per_target_subdir/x86_64-fuchsia/lib/libclang_rt.fuzzer.a
clang/test/Driver/Inputs/resource_dir_with_per_target_subdir/x86_64-fuchsia/lib/libclang_rt.scudo.so
clang/test/Driver/Inputs/resource_dir_with_per_target_subdir/x86_64-linux-gnu/lib/libclang_rt.builtins.a

[phosek]

clang/lib/Driver/ToolChain.cpp

372 ↗

(On Diff #161821)

One possible alternative I've considered would be to simply return -lclang_rt.<component>.<suffix> instead of returning the full path when getLibraryPaths() is not empty. That should work because we add all paths in this list as -L in AddFilePathLibArgs and it's more consistent with e.g. -lgcc_s.

[beanz]

LGTM as-is

clang/lib/Driver/ToolChain.cpp

372 ↗

(On Diff #161821)

I'd prefer not doing that since Darwin doesn't call AddFilePathLibArgs, and adjusting to that is a larger change on the Darwin side.

include/clang/Driver/ToolChain.h

private:
   llvm::Triple Triple;
   const llvm::opt::ArgList &Args;
 
   // We need to initialize CachedRTTIArg before CachedRTTIMode
   const llvm::opt::Arg *const CachedRTTIArg;
 
   const RTTIMode CachedRTTIMode;
 
+  /// The list of toolchain specific path prefixes to search for libraries.
+  path_list LibraryPaths;
+
   /// The list of toolchain specific path prefixes to search for files.
   path_list FilePaths;
 
   /// The list of toolchain specific path prefixes to search for programs.
   path_list ProgramPaths;
 
   mutable std::unique_ptr<Tool> Clang;
   mutable std::unique_ptr<Tool> Assemble;
public:
   }
 
   /// Get the toolchain's effective clang triple.
   const llvm::Triple &getEffectiveTriple() const {
     assert(!EffectiveTriple.getTriple().empty() && "No effective triple");
     return EffectiveTriple;
   }
 
+  path_list &getLibraryPaths() { return LibraryPaths; }
+  const path_list &getLibraryPaths() const { return LibraryPaths; }
+
   path_list &getFilePaths() { return FilePaths; }
   const path_list &getFilePaths() const { return FilePaths; }
 
   path_list &getProgramPaths() { return ProgramPaths; }
   const path_list &getProgramPaths() const { return ProgramPaths; }
 
   const MultilibSet &getMultilibs() const { return Multilibs; }
 

lib/Driver/ToolChain.cpp

static ToolChain::RTTIMode CalculateRTTIMode(const ArgList &Args,
   // -frtti is default, except for the PS4 CPU.
   return (Triple.isPS4CPU()) ? ToolChain::RM_Disabled : ToolChain::RM_Enabled;
 }
 
 ToolChain::ToolChain(const Driver &D, const llvm::Triple &T,
                      const ArgList &Args)
     : D(D), Triple(T), Args(Args), CachedRTTIArg(GetRTTIArgument(Args)),
       CachedRTTIMode(CalculateRTTIMode(Args, Triple, CachedRTTIArg)) {
-  SmallString<128> P(D.ResourceDir);
+  SmallString<128> P;
+
+  P.assign(D.ResourceDir);
   llvm::sys::path::append(P, D.getTargetTriple(), "lib");
   if (getVFS().exists(P))
-    getFilePaths().push_back(P.str());
+    getLibraryPaths().push_back(P.str());
+
+  P.assign(D.ResourceDir);
+  llvm::sys::path::append(P, Triple.str(), "lib");
+  if (getVFS().exists(P))
+    getLibraryPaths().push_back(P.str());
 
   std::string CandidateLibPath = getArchSpecificLibPath();
   if (getVFS().exists(CandidateLibPath))
     getFilePaths().push_back(CandidateLibPath);
 }
 
 void ToolChain::setTripleEnvironment(llvm::Triple::EnvironmentType Env) {
   Triple.setEnvironment(Env);
std::string ToolChain::getCompilerRT(const ArgList &Args, StringRef Component,
   const llvm::Triple &TT = getTriple();
   bool IsITANMSVCWindows =
       TT.isWindowsMSVCEnvironment() || TT.isWindowsItaniumEnvironment();
 
   const char *Prefix = IsITANMSVCWindows ? "" : "lib";
   const char *Suffix = Shared ? (Triple.isOSWindows() ? ".dll" : ".so")
                               : (IsITANMSVCWindows ? ".lib" : ".a");
 
-  const Driver &D = getDriver();
-  SmallString<128> P(D.ResourceDir);
-  llvm::sys::path::append(P, D.getTargetTriple(), "lib");
-  if (getVFS().exists(P)) {
+  for (const auto &LibPath : getLibraryPaths()) {
+    SmallString<128> P(LibPath);
     llvm::sys::path::append(P, Prefix + Twine("clang_rt.") + Component + Suffix);
+    if (getVFS().exists(P))
       return P.str();
   }
 
   StringRef Arch = getArchNameForCompilerRTLib(*this, Args);
   const char *Env = TT.isAndroid() ? "-android" : "";
   SmallString<128> Path(getCompilerRTPath());
   llvm::sys::path::append(Path, Prefix + Twine("clang_rt.") + Component + "-" +
                                     Arch + Env + Suffix);
   return Path.str();
void ToolChain::AddCXXStdlibLibArgs(const ArgList &Args,
   case ToolChain::CST_Libstdcxx:
     CmdArgs.push_back("-lstdc++");
     break;
   }
 }
 
 void ToolChain::AddFilePathLibArgs(const ArgList &Args,
                                    ArgStringList &CmdArgs) const {
+  for (const auto &LibPath : getLibraryPaths())
+    if(LibPath.length() > 0)
+      CmdArgs.push_back(Args.MakeArgString(StringRef("-L") + LibPath));
+
   for (const auto &LibPath : getFilePaths())
     if(LibPath.length() > 0)
       CmdArgs.push_back(Args.MakeArgString(StringRef("-L") + LibPath));
 }
 
 void ToolChain::AddCCKextLibArgs(const ArgList &Args,
                                  ArgStringList &CmdArgs) const {
   CmdArgs.push_back("-lcc_kext");