[Driver] Use normalized triples for multiarch runtime path

[phosek]

Summary

Previously we used target triple as provided which matches the GCC
behavior, but it also means that all clients have to be consistent
in their spelling of target triples since e.g. x86_64-linux-gnu and
x86_64-unknown-linux-gnu will result in Clang driver looking at two
different paths when searching for runtime libraries.

Unfortunately, as it turned out many clients aren't consistent in
their spelling of target triples, e.g. many Linux distributions use
the shorter spelling but config.guess and rustc insist on using the
normalized variant which is causing issues. To avoid having to ship
multiple copies of runtimes for different triple spelling or rely on
symlinks which are not portable, we should use the normalized triple
when constructing paths for multiarch runtimes.

[rnk]

Would it be to much to check both the normalized and non-normalized?

[phosek]

In D50547#1198218, @rnk wrote:

Would it be to much to check both the normalized and non-normalized?

Done, currently I'm only checking the provided target triple followed by the normalized one, I'm not checking the effective triple but it'd be easy to add if we decide to do that as well.

[beanz]

LGTM!

[beanz]

Just want to bring an IRC conversation that I had with @phosek into the proper code review.

This patch is great as-is, but if we want to extend this to Darwin there are some problems because of how Darwin handles triples. Specifically Darwin has multiple potentially valid triples that can mean the same things. For example x86_64-apple-darwin17.7.0 and x86_64-apple-macos10.13.6 are 100% interchangeable (even down to referring to the same OS version). Things get even stranger when you start talking about the -simulator triples...

This means that to support this in Darwin we need to support iterating over a list of triples. We also probably want to cache that list so that we don't run getVFS().exists(...) over and over again. @phosek said on IRC he will update the patch to reflect our conversation.

Thanks @phosek for all the great work on this!

[phosek]

I also need to create the following empty files for tests to pass (somehow those are not displayed by Phabricator):

clang/test/Driver/Inputs/resource_dir_with_per_target_subdir/aarch64-fuchsia/lib/libclang_rt.asan-preinit.a
clang/test/Driver/Inputs/resource_dir_with_per_target_subdir/aarch64-fuchsia/lib/libclang_rt.asan.so
clang/test/Driver/Inputs/resource_dir_with_per_target_subdir/aarch64-fuchsia/lib/libclang_rt.builtins.a
clang/test/Driver/Inputs/resource_dir_with_per_target_subdir/aarch64-fuchsia/lib/libclang_rt.fuzzer.a
clang/test/Driver/Inputs/resource_dir_with_per_target_subdir/aarch64-fuchsia/lib/libclang_rt.scudo.so
clang/test/Driver/Inputs/resource_dir_with_per_target_subdir/i386-linux-gnu/lib/libclang_rt.builtins.a
clang/test/Driver/Inputs/resource_dir_with_per_target_subdir/x86_64-fuchsia/lib/libclang_rt.asan-preinit.a
clang/test/Driver/Inputs/resource_dir_with_per_target_subdir/x86_64-fuchsia/lib/libclang_rt.asan.so
clang/test/Driver/Inputs/resource_dir_with_per_target_subdir/x86_64-fuchsia/lib/libclang_rt.builtins.a
clang/test/Driver/Inputs/resource_dir_with_per_target_subdir/x86_64-fuchsia/lib/libclang_rt.fuzzer.a
clang/test/Driver/Inputs/resource_dir_with_per_target_subdir/x86_64-fuchsia/lib/libclang_rt.scudo.so
clang/test/Driver/Inputs/resource_dir_with_per_target_subdir/x86_64-linux-gnu/lib/libclang_rt.builtins.a

[phosek]

clang/lib/Driver/ToolChain.cpp

363–364

One possible alternative I've considered would be to simply return -lclang_rt.<component>.<suffix> instead of returning the full path when getLibraryPaths() is not empty. That should work because we add all paths in this list as -L in AddFilePathLibArgs and it's more consistent with e.g. -lgcc_s.

[beanz]

LGTM as-is

clang/lib/Driver/ToolChain.cpp

363–364

I'd prefer not doing that since Darwin doesn't call AddFilePathLibArgs, and adjusting to that is a larger change on the Darwin side.

clang/include/clang/Driver/ToolChain.h

public:
   virtual std::string getCompilerRT(const llvm::opt::ArgList &Args,
                                     StringRef Component,
                                     bool Shared = false) const;
 
   const char *getCompilerRTArgString(const llvm::opt::ArgList &Args,
                                      StringRef Component,
                                      bool Shared = false) const;
 
+  // Returns <ResourceDir>/lib/<Triple>. This is used by runtimes to find
+  // per-target libraries.
+  Optional<std::string> getTargetLibPath() const;
+
   // Returns <ResourceDir>/lib/<OSName>/<arch>.  This is used by runtimes (such
   // as OpenMP) to find arch-specific libraries.
   std::string getArchSpecificLibPath() const;
 
   // Returns <OSname> part of above.
   StringRef getOSLibName() const;
 
   /// needsProfileRT - returns true if instrumentation profile is on.

clang/lib/Driver/ToolChain.cpp

static ToolChain::RTTIMode CalculateRTTIMode(const ArgList &Args,
   // -frtti is default, except for the PS4 CPU.
   return (Triple.isPS4CPU()) ? ToolChain::RM_Disabled : ToolChain::RM_Enabled;
 }
 
 ToolChain::ToolChain(const Driver &D, const llvm::Triple &T,
                      const ArgList &Args)
     : D(D), Triple(T), Args(Args), CachedRTTIArg(GetRTTIArgument(Args)),
       CachedRTTIMode(CalculateRTTIMode(Args, Triple, CachedRTTIArg)) {
-  SmallString<128> P(D.ResourceDir);
-  llvm::sys::path::append(P, D.getTargetTriple(), "lib");
-  if (getVFS().exists(P))
-    getFilePaths().push_back(P.str());
+  if (llvm::Optional<std::string> TargetLibPath = getTargetLibPath())
+    getFilePaths().push_back(*TargetLibPath);
 
   std::string CandidateLibPath = getArchSpecificLibPath();
   if (getVFS().exists(CandidateLibPath))
     getFilePaths().push_back(CandidateLibPath);
 }
 
 void ToolChain::setTripleEnvironment(llvm::Triple::EnvironmentType Env) {
   Triple.setEnvironment(Env);
std::string ToolChain::getCompilerRT(const ArgList &Args, StringRef Component,
   const llvm::Triple &TT = getTriple();
   bool IsITANMSVCWindows =
       TT.isWindowsMSVCEnvironment() || TT.isWindowsItaniumEnvironment();
 
   const char *Prefix = IsITANMSVCWindows ? "" : "lib";
   const char *Suffix = Shared ? (Triple.isOSWindows() ? ".dll" : ".so")
                               : (IsITANMSVCWindows ? ".lib" : ".a");
 
-  const Driver &D = getDriver();
-  SmallString<128> P(D.ResourceDir);
-  llvm::sys::path::append(P, D.getTargetTriple(), "lib");
-  if (getVFS().exists(P)) {
+  if (llvm::Optional<std::string> TargetLibPath = getTargetLibPath()) {
+    SmallString<128> P(*TargetLibPath);
     llvm::sys::path::append(P, Prefix + Twine("clang_rt.") + Component + Suffix);
     return P.str();
   }
 
   StringRef Arch = getArchNameForCompilerRTLib(*this, Args);
   const char *Env = TT.isAndroid() ? "-android" : "";
   SmallString<128> Path(getCompilerRTPath());
   llvm::sys::path::append(Path, Prefix + Twine("clang_rt.") + Component + "-" +
                                     Arch + Env + Suffix);
   return Path.str();
 }
 
 const char *ToolChain::getCompilerRTArgString(const llvm::opt::ArgList &Args,
                                               StringRef Component,
                                               bool Shared) const {
   return Args.MakeArgString(getCompilerRT(Args, Component, Shared));
 }
 
+Optional<std::string> ToolChain::getTargetLibPath() const {
+  const Driver &D = getDriver();
+  SmallString<128> P(D.ResourceDir);
+  llvm::sys::path::append(P, D.getTargetTriple(), "lib");
+  if (getVFS().exists(P)) {
+    return std::string(P.str());
+  } else {
+    P.assign(D.ResourceDir);
+    llvm::sys::path::append(P, Triple.str(), "lib");
+    if (getVFS().exists(P))
+      return std::string(P.str());
+  }
+  return None;
+}
+
 std::string ToolChain::getArchSpecificLibPath() const {
   SmallString<128> Path(getDriver().ResourceDir);
   llvm::sys::path::append(Path, "lib", getOSLibName(),
                           llvm::Triple::getArchTypeName(getArch()));
   return Path.str();
 }
 
 bool ToolChain::needsProfileRT(const ArgList &Args) {