[NVPTX] Emit debug info in DWARF-2 by default for Cuda devices.

[ABataev]

Summary

NVPTX target supports debug info in DWARF-2 format. Patch adds emission
of debug info in DWARF-2 by default.

[tra]

lib/Driver/ToolChains/Cuda.cpp

353 ↗

(On Diff #131602)

There's more than one -g option. Presumably you still want -g here for -g{1,2,3} or -ggdb{1,2,3} or -gdwarf2.

Speaking of dwarf, what's going to happen if someone passes -gdwarf5? Should we downgrade it to -gdwarf2 on device side and issue a warning?

test/Driver/cuda-dwarf-2.cu

6–11 ↗

(On Diff #131602)

Nit: you could probably collapse all test cases into two: HAS_DEBUG/NO_DEBUG

[ABataev]

lib/Driver/ToolChains/Cuda.cpp

353 ↗

(On Diff #131602)

That's exactly what I want to avoid. I think we can ignore these options and handle only -g option.
Or I can add a function to ToolChain class to force using the default settings for debugger tuning and DWARF version rather than take them from the driver options.

test/Driver/cuda-dwarf-2.cu

6–11 ↗

(On Diff #131602)

Ok, will do

[tra]

lib/Driver/ToolChains/Cuda.cpp

353 ↗

(On Diff #131602)

You should handle at least -gN/-gline-tables-only/-g. Now that we can emit dwarf debug info, I would want to be able to control it to some degree -- off/line-info-only/full.

I'm still not sure what's the plan for -gdwarfN unsupported on device side?

• do we ignore it and always emit dwarf2?
• do we accept it, emit requested version and let user stare at ptxas (or cuda-gdb) errors when it fails to deal with it?

[ABataev]

lib/Driver/ToolChains/Cuda.cpp

353 ↗

(On Diff #131602)

1. Ok, will look at this
2. -gdwarfN should be ignored, we should always emit DWARF2, otherwise, it will break ptxas.

[tra]

lib/Driver/ToolChains/Cuda.cpp

353 ↗

(On Diff #131602)

Works for me. Please add a test to verify that it's always dwarf2 on device side.

[probinson]

If you want to force DWARF 2, probably clamping the version in LLVM would be simpler? Although most of the debug-info tests are architecture-specific and wouldn't run for an NVPTX target anyway.

[ABataev]

In D42581#989760, @probinson wrote:

If you want to force DWARF 2, probably clamping the version in LLVM would be simpler? Although most of the debug-info tests are architecture-specific and wouldn't run for an NVPTX target anyway.

Hi, I think it would be better to do both: make it default in the frontend and in the backend.

[ABataev]

lib/Driver/ToolChains/Cuda.cpp

353 ↗

(On Diff #131602)

After some thoughts, I think it would better to follow Paul's suggestion, i.e. force DWARF2 for NVPTX in the backend. In this case, -gdwarfN will not break anything in the debug info for NVPTX.

[ABataev]

1. Updated after review.
2. Changed the default behavior of --[no]cuda-noopt-device-debug. If the optimization level is not specified or is O0 and debug info must be emitted, the device debug info is emitted. If optimization level is specified and >O0 and debug info must be emitted, the debug info for device is emitted only if --cuda-noopt-device-debug is specified.

[tra]

lib/Driver/ToolChains/Cuda.cpp

436–437 ↗

(On Diff #132016)

Do we need to pass -g to make lineinfo debugging work?

[ABataev]

lib/Driver/ToolChains/Cuda.cpp

436–437 ↗

(On Diff #132016)

I checked with nvcc, it does not send -g in line table only mode.

[tra]

LGTM

[ABataev]

Thanks, will commit it after the commit of the LLVM part

cfe/trunk/lib/Driver/ToolChains/Cuda.h

void AddIAMCUIncludeArgs(const llvm::opt::ArgList &DriverArgs,
                            llvm::opt::ArgStringList &CC1Args) const override;
 
   SanitizerMask getSupportedSanitizers() const override;
 
   VersionTuple
   computeMSVCVersion(const Driver *D,
                      const llvm::opt::ArgList &Args) const override;
 
+  unsigned GetDefaultDwarfVersion() const override { return 2; }
+
   const ToolChain &HostTC;
   CudaInstallationDetector CudaInstallation;
 
 protected:
   Tool *buildAssembler() const override;  // ptxas
   Tool *buildLinker() const override;     // fatbinary (ok, not really a linker)
 
 private:
   const Action::OffloadKind OK;
 };
 
 } // end namespace toolchains
 } // end namespace driver
 } // end namespace clang
 
 #endif // LLVM_CLANG_LIB_DRIVER_TOOLCHAINS_CUDA_H

cfe/trunk/lib/Driver/ToolChains/Cuda.cpp

 }
 
 void CudaInstallationDetector::print(raw_ostream &OS) const {
   if (isValid())
     OS << "Found CUDA installation: " << InstallPath << ", version "
        << CudaVersionToString(Version) << "\n";
 }
 
+namespace {
+  /// Debug info kind.
+enum DebugInfoKind {
+  NoDebug,       /// No debug info.
+  LineTableOnly, /// Line tables only.
+  FullDebug      /// Full debug info.
+};
+} // anonymous namespace
+
+static DebugInfoKind mustEmitDebugInfo(const ArgList &Args) {
+  Arg *A = Args.getLastArg(options::OPT_O_Group);
+  if (Args.hasFlag(options::OPT_cuda_noopt_device_debug,
+                   options::OPT_no_cuda_noopt_device_debug,
+                   !A || A->getOption().matches(options::OPT_O0))) {
+    if (const Arg *A = Args.getLastArg(options::OPT_g_Group)) {
+      const Option &Opt = A->getOption();
+      if (Opt.matches(options::OPT_gN_Group)) {
+        if (Opt.matches(options::OPT_g0) || Opt.matches(options::OPT_ggdb0))
+          return NoDebug;
+        if (Opt.matches(options::OPT_gline_tables_only) ||
+            Opt.matches(options::OPT_ggdb1))
+          return LineTableOnly;
+      }
+      return FullDebug;
+    }
+  }
+  return NoDebug;
+}
+
 void NVPTX::Assembler::ConstructJob(Compilation &C, const JobAction &JA,
                                     const InputInfo &Output,
                                     const InputInfoList &Inputs,
                                     const ArgList &Args,
                                     const char *LinkingOutput) const {
   const auto &TC =
       static_cast<const toolchains::CudaToolChain &>(getToolChain());
   assert(TC.getTriple().isNVPTX() && "Wrong platform");
void NVPTX::Assembler::ConstructJob(Compilation &C, const JobAction &JA,
 
   // Check that our installation's ptxas supports gpu_arch.
   if (!Args.hasArg(options::OPT_no_cuda_version_check)) {
     TC.CudaInstallation.CheckCudaVersionSupportsArch(gpu_arch);
   }
 
   ArgStringList CmdArgs;
   CmdArgs.push_back(TC.getTriple().isArch64Bit() ? "-m64" : "-m32");
-  if (Args.hasFlag(options::OPT_cuda_noopt_device_debug,
-                   options::OPT_no_cuda_noopt_device_debug, false)) {
+  DebugInfoKind DIKind = mustEmitDebugInfo(Args);
+  if (DIKind == FullDebug) {
     // ptxas does not accept -g option if optimization is enabled, so
     // we ignore the compiler's -O* options if we want debug info.
     CmdArgs.push_back("-g");
     CmdArgs.push_back("--dont-merge-basicblocks");
     CmdArgs.push_back("--return-at-end");
   } else if (Arg *A = Args.getLastArg(options::OPT_O_Group)) {
     // Map the -O we received to -O{0,1,2,3}.
     //
else if (A->getOption().matches(options::OPT_O)) {
                  .Default("2");
     }
     CmdArgs.push_back(Args.MakeArgString(llvm::Twine("-O") + OOpt));
   } else {
     // If no -O was passed, pass -O0 to ptxas -- no opt flag should correspond
     // to no optimizations, but ptxas's default is -O3.
     CmdArgs.push_back("-O0");
   }
+  if (DIKind == LineTableOnly)
+    CmdArgs.push_back("-lineinfo");
 
   // Pass -v to ptxas if it was passed to the driver.
   if (Args.hasArg(options::OPT_v))
     CmdArgs.push_back("-v");
 
   CmdArgs.push_back("--gpu-name");
   CmdArgs.push_back(Args.MakeArgString(CudaArchToString(gpu_arch)));
   CmdArgs.push_back("--output-file");
const auto &TC =
       static_cast<const toolchains::CudaToolChain &>(getToolChain());
   assert(TC.getTriple().isNVPTX() && "Wrong platform");
 
   ArgStringList CmdArgs;
   CmdArgs.push_back("--cuda");
   CmdArgs.push_back(TC.getTriple().isArch64Bit() ? "-64" : "-32");
   CmdArgs.push_back(Args.MakeArgString("--create"));
   CmdArgs.push_back(Args.MakeArgString(Output.getFilename()));
+  if (mustEmitDebugInfo(Args) == FullDebug)
+    CmdArgs.push_back("-g");
 
   for (const auto& II : Inputs) {
     auto *A = II.getAction();
     assert(A->getInputs().size() == 1 &&
            "Device offload action is expected to have a single input");
     const char *gpu_arch_str = A->getOffloadingArch();
     assert(gpu_arch_str &&
            "Device action expected to have associated a GPU architecture!");
void NVPTX::OpenMPLinker::ConstructJob(Compilation &C, const JobAction &JA,
   assert(!JA.isHostOffloading(Action::OFK_OpenMP) &&
          "CUDA toolchain not expected for an OpenMP host device.");
 
   if (Output.isFilename()) {
     CmdArgs.push_back("-o");
     CmdArgs.push_back(Output.getFilename());
   } else
     assert(Output.isNothing() && "Invalid output.");
-  if (Args.hasArg(options::OPT_g_Flag))
+  if (mustEmitDebugInfo(Args) == FullDebug)
     CmdArgs.push_back("-g");
 
   if (Args.hasArg(options::OPT_v))
     CmdArgs.push_back("-v");
 
   StringRef GPUArch =
       Args.getLastArgValue(options::OPT_march_EQ);
   assert(!GPUArch.empty() && "At least one GPU Arch required for ptxas.");

cfe/trunk/test/Driver/cuda-dwarf-2.cu

+// REQUIRES: clang-driver
+//
+// RUN: %clang -### -target x86_64-linux-gnu -c --cuda-gpu-arch=sm_20 %s -g -O0 --no-cuda-noopt-device-debug 2>&1 | \
+// RUN:   FileCheck %s -check-prefix NO_DEBUG
+// RUN: %clang -### -target x86_64-linux-gnu -c --cuda-gpu-arch=sm_20 %s -g -O3 2>&1 | \
+// RUN:   FileCheck %s -check-prefix NO_DEBUG
+// RUN: %clang -### -target x86_64-linux-gnu -c --cuda-gpu-arch=sm_20 %s -g -O3 --no-cuda-noopt-device-debug 2>&1 | \
+// RUN:   FileCheck %s -check-prefix NO_DEBUG
+// RUN: %clang -### -target x86_64-linux-gnu -c --cuda-gpu-arch=sm_20 %s -g0 2>&1 | \
+// RUN:   FileCheck %s -check-prefix NO_DEBUG
+// RUN: %clang -### -target x86_64-linux-gnu -c --cuda-gpu-arch=sm_20 %s -ggdb0 -O3 --cuda-noopt-device-debug 2>&1 | \
+// RUN:   FileCheck %s -check-prefix NO_DEBUG
+// RUN: %clang -### -target x86_64-linux-gnu -c --cuda-gpu-arch=sm_20 %s -ggdb1 2>&1 | \
+// RUN:   FileCheck %s -check-prefix NO_DEBUG -check-prefix LINE_TABLE
+// RUN: %clang -### -target x86_64-linux-gnu -c --cuda-gpu-arch=sm_20 %s -gline-tables-only -O2 --cuda-noopt-device-debug 2>&1 | \
+// RUN:   FileCheck %s -check-prefix NO_DEBUG -check-prefix LINE_TABLE
+
+// NO_DEBUG: ptxas
+// NO_DEBUG-NOT: "-g"
+// LINE_TABLE: "-lineinfo"
+// NO_DEBUG: fatbinary
+// NO_DEBUG-NOT: "-g"
+
+// RUN: %clang -### -target x86_64-linux-gnu -c --cuda-gpu-arch=sm_20 %s -g 2>&1 | \
+// RUN:   FileCheck %s -check-prefix HAS_DEBUG
+// RUN: %clang -### -target x86_64-linux-gnu -c --cuda-gpu-arch=sm_20 %s -g -O0 --cuda-noopt-device-debug 2>&1 | \
+// RUN:   FileCheck %s -check-prefix HAS_DEBUG
+// RUN: %clang -### -target x86_64-linux-gnu -c --cuda-gpu-arch=sm_20 %s -g -O3 --cuda-noopt-device-debug 2>&1 | \
+// RUN:   FileCheck %s -check-prefix HAS_DEBUG
+// RUN: %clang -### -target x86_64-linux-gnu -c --cuda-gpu-arch=sm_20 %s -g2 2>&1 | \
+// RUN:   FileCheck %s -check-prefix HAS_DEBUG
+// RUN: %clang -### -target x86_64-linux-gnu -c --cuda-gpu-arch=sm_20 %s -ggdb2 -O0 2>&1 | \
+// RUN:   FileCheck %s -check-prefix HAS_DEBUG
+// RUN: %clang -### -target x86_64-linux-gnu -c --cuda-gpu-arch=sm_20 %s -g3 -O2 --cuda-noopt-device-debug 2>&1 | \
+// RUN:   FileCheck %s -check-prefix HAS_DEBUG
+// RUN: %clang -### -target x86_64-linux-gnu -c --cuda-gpu-arch=sm_20 %s -ggdb3 -O3 --cuda-noopt-device-debug 2>&1 | \
+// RUN:   FileCheck %s -check-prefix HAS_DEBUG
+
+// HAS_DEBUG: "-fcuda-is-device"
+// HAS_DEBUG-SAME: "-dwarf-version=2"
+// HAS_DEBUG: ptxas
+// HAS_DEBUG-SAME: "-g"
+// HAS_DEBUG-SAME: "--dont-merge-basicblocks"
+// HAS_DEBUG-SAME: "--return-at-end"
+// HAS_DEBUG: fatbinary
+// HAS_DEBUG-SAME: "-g"
+

cfe/trunk/test/Driver/cuda-external-tools.cu

 // RUN: | FileCheck -check-prefixes=CHECK,ARCH64,SM20,OPT3 %s
 // RUN: %clang -### -target x86_64-linux-gnu -Ofast -c %s 2>&1 \
 // RUN: | FileCheck -check-prefixes=CHECK,ARCH64,SM20,OPT3 %s
 // Generating relocatable device code
 // RUN: %clang -### -target x86_64-linux-gnu -fcuda-rdc -c %s 2>&1 \
 // RUN: | FileCheck -check-prefixes=CHECK,ARCH64,SM20,RDC %s
 
 // With debugging enabled, ptxas should be run with with no ptxas optimizations.
-// RUN: %clang -### -target x86_64-linux-gnu --cuda-noopt-device-debug -O2 -c %s 2>&1 \
+// RUN: %clang -### -target x86_64-linux-gnu --cuda-noopt-device-debug -O2 -g -c %s 2>&1 \
 // RUN: | FileCheck -check-prefixes=CHECK,ARCH64,SM20,DBG %s
 
 // --no-cuda-noopt-device-debug overrides --cuda-noopt-device-debug.
 // RUN: %clang -### -target x86_64-linux-gnu --cuda-noopt-device-debug \
 // RUN:   --no-cuda-noopt-device-debug -O2 -c %s 2>&1 \
 // RUN: | FileCheck -check-prefixes=CHECK,ARCH64,SM20,OPT2 %s
 
 // Regular compile without -O.  This should result in us passing -O0 to ptxas.

cfe/trunk/test/Driver/openmp-offload-gpu.c

 /// Check that the warning is thrown when the libomptarget bitcode library is not found.
 /// Libomptarget requires sm_35 or newer so an sm_20 bitcode library should never exist.
 // RUN:   %clang -### -fopenmp=libomp -fopenmp-targets=nvptx64-nvidia-cuda \
 // RUN:   -Xopenmp-target -march=sm_20 --cuda-path=%S/Inputs/CUDA_80/usr/local/cuda \
 // RUN:   -fopenmp-relocatable-target -save-temps -no-canonical-prefixes %s 2>&1 \
 // RUN:   | FileCheck -check-prefix=CHK-BCLIB-WARN %s
 
 // CHK-BCLIB-WARN: No library 'libomptarget-nvptx-sm_20.bc' found in the default clang lib directory or in LIBRARY_PATH. Expect degraded performance due to no inlining of runtime functions on target devices.
+
+/// Check that debug info is emitted in dwarf-2
+// RUN:   %clang -### -no-canonical-prefixes -fopenmp=libomp -fopenmp-targets=nvptx64-nvidia-cuda -Xopenmp-target -march=sm_60 %s -g -O0 --no-cuda-noopt-device-debug 2>&1 \
+// RUN:   | FileCheck -check-prefix=NO_DEBUG %s
+// RUN:   %clang -### -no-canonical-prefixes -fopenmp=libomp -fopenmp-targets=nvptx64-nvidia-cuda -Xopenmp-target -march=sm_60 %s -g -O3 2>&1 \
+// RUN:   | FileCheck -check-prefix=NO_DEBUG %s
+// RUN:   %clang -### -no-canonical-prefixes -fopenmp=libomp -fopenmp-targets=nvptx64-nvidia-cuda -Xopenmp-target -march=sm_60 %s -g -O3 --no-cuda-noopt-device-debug 2>&1 \
+// RUN:   | FileCheck -check-prefix=NO_DEBUG %s
+// RUN:   %clang -### -no-canonical-prefixes -fopenmp=libomp -fopenmp-targets=nvptx64-nvidia-cuda -Xopenmp-target -march=sm_60 %s -g0 2>&1 \
+// RUN:   | FileCheck -check-prefix=NO_DEBUG %s
+// RUN:   %clang -### -no-canonical-prefixes -fopenmp=libomp -fopenmp-targets=nvptx64-nvidia-cuda -Xopenmp-target -march=sm_60 %s -ggdb0 -O3 --cuda-noopt-device-debug 2>&1 \
+// RUN:   | FileCheck -check-prefix=NO_DEBUG %s
+// RUN:   %clang -### -no-canonical-prefixes -fopenmp=libomp -fopenmp-targets=nvptx64-nvidia-cuda -Xopenmp-target -march=sm_60 %s -gline-tables-only 2>&1 \
+// RUN:   | FileCheck -check-prefix=NO_DEBUG -check-prefix=LINE_TABLE %s
+// RUN:   %clang -### -no-canonical-prefixes -fopenmp=libomp -fopenmp-targets=nvptx64-nvidia-cuda -Xopenmp-target -march=sm_60 %s -ggdb1 -O2 --cuda-noopt-device-debug 2>&1 \
+// RUN:   | FileCheck -check-prefix=NO_DEBUG -check-prefix=LINE_TABLE %s
+
+// NO_DEBUG: ptxas
+// LINE_TABLE: "-lineinfo"
+// NO_DEBUG-NOT: "-g"
+// NO_DEBUG: nvlink
+// NO_DEBUG-NOT: "-g"
+
+// RUN:   %clang -### -no-canonical-prefixes -fopenmp=libomp -fopenmp-targets=nvptx64-nvidia-cuda -Xopenmp-target -march=sm_60 %s -g 2>&1 \
+// RUN:   | FileCheck -check-prefix=HAS_DEBUG %s
+// RUN:   %clang -### -no-canonical-prefixes -fopenmp=libomp -fopenmp-targets=nvptx64-nvidia-cuda -Xopenmp-target -march=sm_60 %s -g -O0 --cuda-noopt-device-debug 2>&1 \
+// RUN:   | FileCheck -check-prefix=HAS_DEBUG %s
+// RUN:   %clang -### -no-canonical-prefixes -fopenmp=libomp -fopenmp-targets=nvptx64-nvidia-cuda -Xopenmp-target -march=sm_60 %s -g -O3 --cuda-noopt-device-debug 2>&1 \
+// RUN:   | FileCheck -check-prefix=HAS_DEBUG %s
+// RUN:   %clang -### -no-canonical-prefixes -fopenmp=libomp -fopenmp-targets=nvptx64-nvidia-cuda -Xopenmp-target -march=sm_60 %s -g2 2>&1 \
+// RUN:   | FileCheck -check-prefix=HAS_DEBUG %s
+// RUN:   %clang -### -no-canonical-prefixes -fopenmp=libomp -fopenmp-targets=nvptx64-nvidia-cuda -Xopenmp-target -march=sm_60 %s -ggdb2 -O0 --cuda-noopt-device-debug 2>&1 \
+// RUN:   | FileCheck -check-prefix=HAS_DEBUG %s
+// RUN:   %clang -### -no-canonical-prefixes -fopenmp=libomp -fopenmp-targets=nvptx64-nvidia-cuda -Xopenmp-target -march=sm_60 %s -g3 -O3 --cuda-noopt-device-debug 2>&1 \
+// RUN:   | FileCheck -check-prefix=HAS_DEBUG %s
+// RUN:   %clang -### -no-canonical-prefixes -fopenmp=libomp -fopenmp-targets=nvptx64-nvidia-cuda -Xopenmp-target -march=sm_60 %s -ggdb3 -O2 --cuda-noopt-device-debug 2>&1 \
+// RUN:   | FileCheck -check-prefix=HAS_DEBUG %s
+
+// HAS_DEBUG: "-triple" "nvptx64-nvidia-cuda"
+// HAS_DEBUG-SAME: "-dwarf-version=2"
+// HAS_DEBUG-SAME: "-fopenmp-is-device"
+// HAS_DEBUG: ptxas
+// HAS_DEBUG-SAME: "-g"
+// HAS_DEBUG-SAME: "--dont-merge-basicblocks"
+// HAS_DEBUG-SAME: "--return-at-end"
+// HAS_DEBUG: nvlink
+// HAS_DEBUG-SAME: "-g"
+