[CUDA] Fix libdevice selection.

[tra]

Summary

This makes clang's libdevice selection match that of NVCC as described in
http://docs.nvidia.com/cuda/libdevice-users-guide/basic-usage.html#version-selection

If required libdevice variant is not found, driver now fails with an error.

[jlebar]

lib/Driver/ToolChains.cpp

4773

I *think* march is never empty here. We should add it when we create the CudaActions.

Same below.

If I'm right or if I'm wrong, let's make sure there are tests. :)

test/Driver/cuda-detect.cu

66

Should we test the new sm_XX --> libdevice file mappings added?

[tra]

Added tests for libdevice mapping correctness.

lib/Driver/ToolChains.cpp

4773

It's guaranteed to be non-empty here only when we're constructing device-side pipeline which *is* normally the case.

Alas, it's possible for user to specify a nonsensical set of arguments that would trigger this assertion:

For instance: clang --cuda-host-only -target nvptx-nvidia-cuda -x cuda used by test/Preprocessor/cuda-types.cu.

Because target is CUDA, we do pick CudaToolchain, but because it's a host compilation, we never get to add -march=sm_XX (TC->TranslateArgs is called with BoundArch=nullptr).

Dafaulting to somewhat safe sm_20 on user error seems better than crashing compiler with an assertion.

test/Driver/cuda-detect.cu

66

OK.

[jlebar]

lib/Driver/ToolChains.cpp

4773

Hm. I totally agree that we shouldn't crash (especially because assertions are disabled in production builds!). But I'm not sure that defaulting to sm_20 every time we read this option is right. "This option must always be set" seems like an easier invariant to enforce than "whenever you read this option, you must use a default of sm_20, unless you're really sure that it's always set. And if you think you're really sure, you're probably wrong." :)

[tra]

Now that D23042 / r277537 makes sure that CudaToolchain is only used on device side, we can
remove defaults for -march and restore asserts() ensuring that -march is added by the driver.

include/clang/Basic/DiagnosticDriverKinds.td

def err_drv_unknown_stdin_type_clang_cl : Error<
   "use /Tc or /Tp to set input type for standard input">;
 def err_drv_unknown_language : Error<"language not recognized: '%0'">;
 def err_drv_invalid_arch_name : Error<
   "invalid arch name '%0'">;
 def err_drv_cuda_bad_gpu_arch : Error<"Unsupported CUDA gpu architecture: %0">;
 def err_drv_no_cuda_installation : Error<
   "cannot find CUDA installation.  Provide its path via --cuda-path, or pass "
   "-nocudainc to build without CUDA includes.">;
+def err_drv_no_cuda_libdevice : Error<
+  "cannot find libdevice for %0. Provide path to different CUDA installation "
+  "via --cuda-path, or pass -nocudalib to build without linking with libdevice.">;
 def err_drv_cuda_version_too_low : Error<
   "GPU arch %1 requires CUDA version at least %3, but installation at %0 is %2. "
   "Use --cuda-path to specify a different CUDA install, or pass "
   "--no-cuda-version-check.">;
 def err_drv_invalid_thread_model_for_target : Error<
   "invalid thread model '%0' in '%1' for this target">;
 def err_drv_invalid_linker_name : Error<
   "invalid linker name in argument '%0'">;

lib/Driver/ToolChains.cpp

for (llvm::sys::fs::directory_iterator LI(LibDevicePath, EC), LE;
       // Process all bitcode filenames that look like libdevice.compute_XX.YY.bc
       const StringRef LibDeviceName = "libdevice.";
       if (!(FileName.startswith(LibDeviceName) && FileName.endswith(".bc")))
         continue;
       StringRef GpuArch = FileName.slice(
           LibDeviceName.size(), FileName.find('.', LibDeviceName.size()));
       LibDeviceMap[GpuArch] = FilePath.str();
       // Insert map entries for specifc devices with this compute capability.
+      // NVCC's choice of libdevice library version is rather peculiar:
+      // http://docs.nvidia.com/cuda/libdevice-users-guide/basic-usage.html#version-selection
+      // TODO: this will need to be updated once CUDA-8 is released.
       if (GpuArch == "compute_20") {
         LibDeviceMap["sm_20"] = FilePath;
         LibDeviceMap["sm_21"] = FilePath;
+        LibDeviceMap["sm_32"] = FilePath;
       } else if (GpuArch == "compute_30") {
         LibDeviceMap["sm_30"] = FilePath;
-        LibDeviceMap["sm_32"] = FilePath;
-      } else if (GpuArch == "compute_35") {
-        LibDeviceMap["sm_35"] = FilePath;
-        LibDeviceMap["sm_37"] = FilePath;
-      } else if (GpuArch == "compute_50") {
+        // compute_30 is the fallback libdevice variant for sm_30+,
+        // unless CUDA specifies different version for specific GPU
+        // arch.
         LibDeviceMap["sm_50"] = FilePath;
         LibDeviceMap["sm_52"] = FilePath;
         LibDeviceMap["sm_53"] = FilePath;
+        // sm_6? are currently all aliases for sm_53 in LLVM and
+        // should use compute_30.
         LibDeviceMap["sm_60"] = FilePath;
         LibDeviceMap["sm_61"] = FilePath;
         LibDeviceMap["sm_62"] = FilePath;
+      } else if (GpuArch == "compute_35") {
+        LibDeviceMap["sm_35"] = FilePath;
+        LibDeviceMap["sm_37"] = FilePath;
+      } else if (GpuArch == "compute_50") {
+        // NVCC does not use compute_50 libdevice at all at the moment.
+        // The version that's shipped with CUDA-7.5 is a copy of compute_30.
       }
     }
 
     llvm::ErrorOr<std::unique_ptr<llvm::MemoryBuffer>> VersionFile =
         FS.getBufferForFile(InstallPath + "/version.txt");
     if (!VersionFile) {
       // CUDA 7.0 doesn't have a version.txt, so guess that's our version if
       // version.txt isn't present.
CudaToolChain::addClangTargetOptions(const llvm::opt::ArgList &DriverArgs,
 
   if (DriverArgs.hasFlag(options::OPT_fcuda_approx_transcendentals,
                          options::OPT_fno_cuda_approx_transcendentals, false))
     CC1Args.push_back("-fcuda-approx-transcendentals");
 
   if (DriverArgs.hasArg(options::OPT_nocudalib))
     return;
 
-  std::string LibDeviceFile = CudaInstallation.getLibDeviceFile(
-      DriverArgs.getLastArgValue(options::OPT_march_EQ));
-  if (!LibDeviceFile.empty()) {
-    CC1Args.push_back("-mlink-cuda-bitcode");
-    CC1Args.push_back(DriverArgs.MakeArgString(LibDeviceFile));
+  StringRef GpuArch =
+      DriverArgs.getLastArgValue(options::OPT_march_EQ, "sm_20");
+  std::string LibDeviceFile = CudaInstallation.getLibDeviceFile(GpuArch);
+
+  if (LibDeviceFile.empty()) {
+    getDriver().Diag(diag::err_drv_no_cuda_libdevice) << GpuArch;
+    return;
+  }
+
 
-    // Libdevice in CUDA-7.0 requires PTX version that's more recent
-    // than LLVM defaults to. Use PTX4.2 which is the PTX version that
-    // came with CUDA-7.0.
-    CC1Args.push_back("-target-feature");
-    CC1Args.push_back("+ptx42");
-  }
-}
 
 void CudaToolChain::AddCudaIncludeArgs(const ArgList &DriverArgs,
                                        ArgStringList &CC1Args) const {
   // Check our CUDA version if we're going to include the CUDA headers.
   if (!DriverArgs.hasArg(options::OPT_nocudainc) &&
       !DriverArgs.hasArg(options::OPT_no_cuda_version_check)) {
-    StringRef Arch = DriverArgs.getLastArgValue(options::OPT_march_EQ);
-    assert(!Arch.empty() && "Must have an explicit GPU arch.");
+    StringRef Arch = DriverArgs.getLastArgValue(options::OPT_march_EQ, "sm_20");
     CudaInstallation.CheckCudaVersionSupportsArch(StringToCudaArch(Arch));
   }
   Linux::AddCudaIncludeArgs(DriverArgs, CC1Args);
 }
 
 llvm::opt::DerivedArgList *
 CudaToolChain::TranslateArgs(const llvm::opt::DerivedArgList &Args,
                              const char *BoundArch) const {

test/Driver/cuda-detect.cu

 // RUN:   -nocudainc --cuda-path=%S/Inputs/CUDA/usr/local/cuda %s 2>&1 \
 // RUN:   | FileCheck %s -check-prefix COMMON -check-prefix NOCUDAINC \
 // RUN:     -check-prefix LIBDEVICE -check-prefix LIBDEVICE35
 // We should not add any CUDA include paths if there's no valid CUDA installation
 // RUN: %clang -### -v --target=i386-unknown-linux --cuda-gpu-arch=sm_35 \
 // RUN:   --cuda-path=%S/no-cuda-there %s 2>&1 \
 // RUN:   | FileCheck %s -check-prefix COMMON -check-prefix NOCUDAINC
 
-// Verify that no options related to bitcode linking are passes if
-// there's no bitcode file.
+// Verify that we get an error if there's no libdevice library to link with.
 // RUN: %clang -### -v --target=i386-unknown-linux --cuda-gpu-arch=sm_30 \
 // RUN:   --cuda-path=%S/Inputs/CUDA/usr/local/cuda %s 2>&1 \
-// RUN:   | FileCheck %s -check-prefix COMMON -check-prefix NOLIBDEVICE
-// .. or if we explicitly passed -nocudalib
+// RUN:   | FileCheck %s -check-prefix COMMON -check-prefix MISSINGLIBDEVICE
+
+// Verify that  -nocudalib prevents linking libdevice bitcode in.
 // RUN: %clang -### -v --target=i386-unknown-linux --cuda-gpu-arch=sm_35 \
 // RUN:   -nocudalib --cuda-path=%S/Inputs/CUDA/usr/local/cuda %s 2>&1 \
 // RUN:   | FileCheck %s -check-prefix COMMON -check-prefix NOLIBDEVICE
 // Verify that we don't add include paths, link with libdevice or
 // -include __clang_cuda_runtime_wrapper.h without valid CUDA installation.
 // RUN: %clang -### -v --target=i386-unknown-linux --cuda-gpu-arch=sm_35 \
 // RUN:   --cuda-path=%S/no-cuda-there %s 2>&1 \
 // RUN:   | FileCheck %s -check-prefix COMMON \
 // RUN:     -check-prefix NOCUDAINC -check-prefix NOLIBDEVICE
 
 // CHECK: Found CUDA installation: {{.*}}/Inputs/CUDA/usr/local/cuda
 // NOCUDA-NOT: Found CUDA installation:
 
+// MISSINGLIBDEVICE: error: cannot find libdevice for sm_30.
+
 // COMMON: "-triple" "nvptx-nvidia-cuda"
 // COMMON-SAME: "-fcuda-is-device"
 // LIBDEVICE-SAME: "-mlink-cuda-bitcode"
 // NOLIBDEVICE-NOT: "-mlink-cuda-bitcode"
 // LIBDEVICE21-SAME: libdevice.compute_20.10.bc
 // LIBDEVICE35-SAME: libdevice.compute_35.10.bc
 // NOLIBDEVICE-NOT: libdevice.compute_{{.*}}.bc
 // LIBDEVICE-SAME: "-target-feature" "+ptx42"
 // NOLIBDEVICE-NOT: "-target-feature" "+ptx42"
 // CUDAINC-SAME: "-internal-isystem" "{{.*}}/Inputs/CUDA/usr/local/cuda/include"
 // NOCUDAINC-NOT: "-internal-isystem" "{{.*}}/cuda/include"
 // CUDAINC-SAME: "-include" "__clang_cuda_runtime_wrapper.h"
 // NOCUDAINC-NOT: "-include" "__clang_cuda_runtime_wrapper.h"
 // COMMON-SAME: "-x" "cuda"