First-pass at ARM hard/soft-float multilib driver (NOT WORKING YET)

[kelledin]

Summary

This is my first crack at implementing working ARM EABI multilib support
(where multilib support is between hard/soft float ONLY, not between 32-bit
and 64-bit).  This is currently NOT suitable for merging; I'm only posting
it for guidance as to what exactly I'm missing.  Specifically, for the
default-hardfloat target (arm-linux-gnueabihf), clang selects the correct
GCC suffix (/sf vs /hf) but consistently selects /usr/lib/arm-linux-gnueabihf
as the library directory, even if soft-float flags are passed.  This leaves
binutils trying to mix hard-float/soft-float objects, which of course fails
miserably.  AIUI the multilib driver *should* trigger the selection of
/usr/lib/arm-linux-gnueabi instead.

I can produce verbose output from clang's behavior upon request.  Be aware
that this changeset depends on https://reviews.llvm.org/D52149 for compiler-rt.

[kristina]

Accepted blocking differential D52149, updated reviewers to include ARM code owner and @rsmith.

[peter.smith]

I think that you may be better off posting a RFC to llvm-dev to get discussion on the best approach here. It would also be good to step back a bit and consider the requirements, as it stands it looks like this might be a solution for just one particular multilib layout and we might be able to get some input on some other use cases that others may have. I personally would like to enable clang to be able to query a gcc installation to find out (and potentially cache) its multlib directories as I've been told for the Arm embedded toolchain that the directory structure is not fixed and may change at any point in the future.

Another unrelated thing that may be worth doing is to check for other places where the target might affect the code-generation independently of -mfloat-abi, I'm thinking in particular of builtin functions as this has changed a bit over the past few years.

lib/Driver/ToolChains/Gnu.cpp

1441

When I saw EABIMultilibs I first thought of multilib support for arm-none-eabi , I think that this may be worth renaming to something like ArmLinuxGNUEABI as this could be specific to Linux Distributions.

1451

AltTriple doesn't seem to be used anywhere in the rest of the function?

1454

Another place where this is used, and clang can't yet handle, is the GCC Embedded toolchain arm-none-eabi which includes both hard and soft float abi libraries. These have a different structure to the one presented here though.

1458

I definitely agree here. It seems to me to be quite fragile to hard-code in directory structures of gcc toolchains or linux distributions that can, and do, vary over time.

lib/Driver/ToolChains/Arch/ARM.cpp

default:
       case llvm::Triple::MuslEABIHF:
       case llvm::Triple::EABIHF:
         ABI = FloatABI::Hard;
         break;
       case llvm::Triple::GNUEABI:
       case llvm::Triple::MuslEABI:
       case llvm::Triple::EABI:
         // EABI is always AAPCS, and if it was not marked 'hard', it's softfp
-        ABI = FloatABI::SoftFP;
+        ABI = FloatABI::Soft;
         break;
       case llvm::Triple::Android:
         ABI = (SubArch == 7) ? FloatABI::SoftFP : FloatABI::Soft;
         break;
       default:
         // Assume "soft", but warn the user we are guessing.
         if (Triple.isOSBinFormatMachO() &&
             Triple.getSubArch() == llvm::Triple::ARMSubArch_v7em)

lib/Driver/ToolChains/Gnu.cpp

 } // end anonymous namespace
 
 static bool isSoftFloatABI(const ArgList &Args) {
   Arg *A = Args.getLastArg(options::OPT_msoft_float, options::OPT_mhard_float,
                            options::OPT_mfloat_abi_EQ);
   if (!A)
     return false;
 
+  // ARM GNUEABI allows a "softfp" ABI (compatible with soft-float calling
+  // conventions, but can still generate FPU instructions inside functions).
   return A->getOption().matches(options::OPT_msoft_float) ||
          (A->getOption().matches(options::OPT_mfloat_abi_EQ) &&
-          A->getValue() == StringRef("soft"));
+          (A->getValue() == StringRef("soft") || A->getValue() == StringRef("softfp")));
+}
+
+static bool isHardFloatABI(const ArgList &Args) {
+  Arg *A = Args.getLastArg(options::OPT_msoft_float, options::OPT_mhard_float,
+                           options::OPT_mfloat_abi_EQ);
+  if (!A)
+    return false;
+
+  return A->getOption().matches(options::OPT_mhard_float) ||
+         (A->getOption().matches(options::OPT_mfloat_abi_EQ) &&
+          A->getValue() == StringRef("hard"));
 }
 
 /// \p Flag must be a flag accepted by the driver with its leading '-' removed,
 //     otherwise '-print-multi-lib' will not emit them correctly.
 static void addMultilibFlag(bool Enabled, const char *const Flag,
                             std::vector<std::string> &Flags) {
   if (Enabled)
     Flags.push_back(std::string("+") + Flag);
static void findRISCVMultilibs(const Driver &D,
   addMultilibFlag(ABIName == "lp64", "mabi=lp64", Flags);
   addMultilibFlag(ABIName == "lp64f", "mabi=lp64f", Flags);
   addMultilibFlag(ABIName == "lp64d", "mabi=lp64d", Flags);
 
   if (RISCVMultilibs.select(Flags, Result.SelectedMultilib))
     Result.Multilibs = RISCVMultilibs;
 }
 
+static bool findArmEABIMultilibs(const Driver &D,
+                                 const llvm::Triple &TargetTriple,
+                                 StringRef Path, const ArgList &Args,
+                                 DetectedMultilibs &Result) {
+  // Find multilibs with subdirectories like hf (for hard-float) or sf (for
+  // soft-float).  gcc also allows for "-mfloat-abi=softfp": ABI-equivalent
+  // to soft-float, but can still generate FPU instuctions inside functions.
+  // softfp usually has the same multilib subdirectory as soft-float.
+  Multilib Default;
+  bool DefaultHardFloat = TargetTriple.isHardFloatEABI();
+  llvm::Triple AltTriple(DefaultHardFloat ?
+                             TargetTriple.getSoftFloatArchVariant() :
+                             TargetTriple.getHardFloatArchVariant());
+  // We assume the Debian libdir/includedir arrangement for armel/armhf,
+  // since Debian and its descendents are apparently the only common Linux
+  // distros that have anything resembling multilib support for this scenario.
+  // SuSE and RedHat seem to stick with hard-float only and no libdir suffix.
+  // TODO: this (and a lot of other code here) could be generalized via the
+  // output of "gcc <flags> --print-multi-os-dir".
+  Multilib ArmHFMultilib = makeMultilib("/hf")
+                               .osSuffix("/" + TargetTriple.getHardFloatArchVariant().str())
+                               .includeSuffix("/" + TargetTriple.getHardFloatArchVariant().str())
+                               .flag("+mhard-float")
+                               .flag("+mfloat-abi=hard")
+                               .flag("-msoft-float")
+                               .flag("-mfloat-abi=soft")
+                               .flag("-mfloat-abi=softfp");
+  Multilib ArmSFMultilib = makeMultilib("/sf")
+                               .osSuffix("/" + TargetTriple.getSoftFloatArchVariant().str())
+                               .includeSuffix("/" + TargetTriple.getSoftFloatArchVariant().str())
+                               .flag("+msoft-float")
+                               .flag("+mfloat-abi=soft")
+                               .flag("+mfloat-abi=softfp")
+                               .flag("-mhard-float")
+                               .flag("-mfloat-abi=hard");
+
+  FilterNonExistent NonExistent(Path, "/crtbegin.o", D.getVFS());
+
+  if (DefaultHardFloat)
+    Default.flag("+mhard-float")
+           .flag("+mfloat-abi=hard")
+           .flag("-msoft-float")
+           .flag("-mfloat-abi=soft")
+           .flag("-mfloat-abi=softfp");
+  else
+    Default.flag("+msoft-float")
+           .flag("+mfloat-abi=soft")
+           .flag("+mfloat-abi=softfp")
+           .flag("-mhard-float")
+           .flag("-mfloat-abi=hard");
+
+  Result.Multilibs.push_back(Default);
+  Result.Multilibs.push_back(ArmHFMultilib);
+  Result.Multilibs.push_back(ArmSFMultilib);
+  Result.Multilibs.FilterOut(NonExistent);
+
+  Multilib::flags_list Flags;
+  addMultilibFlag(isSoftFloatABI(Args), "msoft-float", Flags);
+  addMultilibFlag(isHardFloatABI(Args), "mhard-float", Flags);
+
+  if (!Result.Multilibs.select(Flags, Result.SelectedMultilib))
+    return false;
+
+  if (Result.SelectedMultilib == ArmSFMultilib || Result.SelectedMultilib == ArmHFMultilib)
+    Result.BiarchSibling = Default;
+
+  return true;
+}
+
 static bool findBiarchMultilibs(const Driver &D,
                                 const llvm::Triple &TargetTriple,
                                 StringRef Path, const ArgList &Args,
                                 bool NeedsBiarchSuffix,
                                 DetectedMultilibs &Result) {
   Multilib Default;
 
   // Some versions of SUSE and Fedora on ppc64 put 32-bit libs

 ///
 /// FIXME: We shouldn't need an explicit TargetTriple parameter here, and
 /// should instead pull the target out of the driver. This is currently
 /// necessary because the driver doesn't store the final version of the target
 /// triple.
 void Generic_GCC::GCCInstallationDetector::init(
     const llvm::Triple &TargetTriple, const ArgList &Args,
     ArrayRef<std::string> ExtraTripleAliases) {
-  llvm::Triple BiarchVariantTriple = TargetTriple.isArch32Bit()
+  llvm::Triple BiarchVariantTriple;
+
+  // Triples with an EABI environment are normally biarch-by-FPU rather
+  // than biarch-by-wordsize.
+  if (TargetTriple.isEABI()) {
+    BiarchVariantTriple = TargetTriple.isHardFloatEABI()
+                                           ? TargetTriple.getSoftFloatArchVariant()
+                                           : TargetTriple.getHardFloatArchVariant();
+  }
+  else {
+    BiarchVariantTriple = TargetTriple.isArch32Bit()
                                            ? TargetTriple.get64BitArchVariant()
                                            : TargetTriple.get32BitArchVariant();
+  }
+
   // The library directories which may contain GCC installations.
   SmallVector<StringRef, 4> CandidateLibDirs, CandidateBiarchLibDirs;
   // The compatible GCC triples for this particular architecture.
   SmallVector<StringRef, 16> CandidateTripleAliases;
   SmallVector<StringRef, 16> CandidateBiarchTripleAliases;
   CollectLibDirsAndTriples(TargetTriple, BiarchVariantTriple, CandidateLibDirs,
                            CandidateTripleAliases, CandidateBiarchLibDirs,
                            CandidateBiarchTripleAliases);
bool Generic_GCC::GCCInstallationDetector::ScanGCCForMultilibs(
   DetectedMultilibs Detected;
 
   // Android standalone toolchain could have multilibs for ARM and Thumb.
   // Debian mips multilibs behave more like the rest of the biarch ones,
   // so handle them there
   if (isArmOrThumbArch(TargetArch) && TargetTriple.isAndroid()) {
     // It should also work without multilibs in a simplified toolchain.
     findAndroidArmMultilibs(D, TargetTriple, Path, Args, Detected);
+  } else if (isArmOrThumbArch(TargetArch) && TargetTriple.isEABI()) {
+    if (!findArmEABIMultilibs(D, TargetTriple, Path, Args, Detected))
+      return false;
   } else if (TargetTriple.isMIPS()) {
     if (!findMIPSMultilibs(D, TargetTriple, Path, Args, Detected))
       return false;
   } else if (isRISCV(TargetArch)) {
     findRISCVMultilibs(D, TargetTriple, Path, Args, Detected);
   } else if (!findBiarchMultilibs(D, TargetTriple, Path, Args,
                                   NeedsBiarchSuffix, Detected)) {
     return false;

lib/Driver/ToolChains/Linux.cpp

std::string Linux::getDynamicLinker(const ArgList &Args) const {
   case llvm::Triple::aarch64_be:
     LibDir = "lib";
     Loader = "ld-linux-aarch64_be.so.1";
     break;
   case llvm::Triple::arm:
   case llvm::Triple::thumb:
   case llvm::Triple::armeb:
   case llvm::Triple::thumbeb: {
+    // NOTE: If no float-ABI selector flags are in use, getARMFloatABI()
+    // will fall back on determining ABI by Triple.getEnvironment().
     const bool HF =
-        Triple.getEnvironment() == llvm::Triple::GNUEABIHF ||
         tools::arm::getARMFloatABI(*this, Args) == tools::arm::FloatABI::Hard;
 
     LibDir = "lib";
     Loader = HF ? "ld-linux-armhf.so.3" : "ld-linux.so.3";
     break;
   }
   case llvm::Triple::mips:
   case llvm::Triple::mipsel: