Make -gsplit-dwarf generally available

[trixirt]

Summary

Change CLANG_DEFAULT_OBJCOPY from objcopy to llvm-objcopy
Remove is-linux checks
Add dwarf splitting to FreeBSD's assembler job.

[jakehehrlich]

lib/Driver/ToolChains/Clang.cpp

4818–4819

Maybe re-add a TODO to point out that *some* objcopy is needed be it GNU objcopy or llvm-objcopy? Ideally this would be done by the compiler and not by an external tool.

test/Driver/split-debug.c

7

I think I may have missed something here. I thought this would change all of clang's objcopy invocations regardless of the target triple. What chooses "objcopy" if the target system is linux or linux-gnu? I think this might be passing because that line still matches "llvm-objcopy ..."

14

When I wrote this I wanted this test to pass regardless of which objcopy was used so I just left it as "objcopy" because that would still pass if "llvm-objcopy" or "objcopy" was used. For instance my team builds the toolchain and we consider a build a fail of all tests don't pass. If someone wanted to use a GNU objcopy and built and tested toolchains the same way they would run into this issue.

[jakehehrlich]

I'm generally ok with this but I think some important checks need to occur first. Namely I think I'd like to check that Chromium still builds with this option since I'd imagine they'd be affected by this.

[trixirt]

Improve comment in CLANG_DEFAULT_OBJCOPY to specify what is required of objcopy.
Remove the llvm- prefix from testing to allow for override of default value.

[echristo]

FWIW Peter has some patches to move object emission away from objcopy that I'm on the hook to review here shortly so the objcopy part of this should become unnecessary and can just have us able to emit dwarf5 compatible split dwarf on any platform.

-eric

[jakehehrlich]

Yeah I haven't heard back from Lexan nor have I tried reproducing such a build myself and if we're that close to not relaying on this sort of hack I'd much rather wait on this than risk breaking someone for the sake of a stopgap for such a short period of time. @echristo Can you link the review to that?

[pcc]

There were a bunch of them but the last one was D47093 which has already landed :)

Probably all that needs to happen on this change is to replace the isLinux() check with isELF().

[echristo]

In D46791#1107168, @pcc wrote:

There were a bunch of them but the last one was D47093 which has already landed :)

Probably all that needs to happen on this change is to replace the isLinux() check with isELF().

Be good if it didn't even require the isELF check anymore, but that probably requires testing on a few platforms. :)

[compnerd]

test/Driver/split-debug.c

23

NIT: macOS or the legacy spelling of Mac OS X.

[emaste]

Ping

[aprantl]

test/Driver/split-debug.c

28

split dwarf does not make sense on macOS since the debug info is never linked into the binary to begin with, so you might as well not test this configuration, or even better error out when it is chosen.

[jakehehrlich]

This is obsolete I think. Clang doesn't use llvm-objcopy for this anymore if I understand correct. @pcc should be able to confirm.

[emaste]

running clang -target x86_64-unknown-freebsd13.0 -split-dwarf foo.c indeed produces a foo.dwo and foo.o w/o invoking objcopy

[pcc]

Correct, clang no longer uses objcopy for this as of D47093.

CMakeLists.txt

 if (NOT(CLANG_DEFAULT_RTLIB STREQUAL "" OR
         CLANG_DEFAULT_RTLIB STREQUAL "libgcc" OR
         CLANG_DEFAULT_RTLIB STREQUAL "compiler-rt"))
   message(WARNING "Resetting default rtlib to use platform default")
   set(CLANG_DEFAULT_RTLIB "" CACHE STRING
     "Default runtime library to use (\"libgcc\" or \"compiler-rt\", empty for platform default)" FORCE)
 endif()
 
-set(CLANG_DEFAULT_OBJCOPY "objcopy" CACHE STRING
+# The CLANG_DEFAULT_OBJCOPY program must support these interfaces
+# For the -gsplit-dwarf :
+#   --extract-dwo <file>
+#   --strip-dwo <file>
+#
+# And to work with the testsuite it should be called '*objcopy'
+#
+# llvm-objcopy is the most universal program.
+# newer gnu objcopy also works.
+set(CLANG_DEFAULT_OBJCOPY "llvm-objcopy" CACHE STRING
   "Default objcopy executable to use.")
 
 set(CLANG_DEFAULT_OPENMP_RUNTIME "libomp" CACHE STRING
   "Default OpenMP runtime used by -fopenmp.")
 
 # OpenMP offloading requires at least sm_35 because we use shuffle instructions
 # to generate efficient code for reductions and the atomicMax instruction on
 # 64-bit integers in the implementation of conditional lastprivate.

lib/Driver/ToolChains/Clang.cpp

       Args.hasArg(options::OPT_gmodules)) {
     DebugInfoKind = codegenoptions::LimitedDebugInfo;
     CmdArgs.push_back("-dwarf-ext-refs");
     CmdArgs.push_back("-fmodule-format=obj");
   }
 
   // -gsplit-dwarf should turn on -g and enable the backend dwarf
   // splitting and extraction.
-  // FIXME: Currently only works on Linux.
-  if (T.isOSLinux()) {
-    if (!SplitDWARFInlining)
-      CmdArgs.push_back("-fno-split-dwarf-inlining");
-
-    if (SplitDWARFArg) {
-      if (DebugInfoKind == codegenoptions::NoDebugInfo)
-        DebugInfoKind = codegenoptions::LimitedDebugInfo;
-      CmdArgs.push_back("-enable-split-dwarf");
-    }
+  if (!SplitDWARFInlining)
+    CmdArgs.push_back("-fno-split-dwarf-inlining");
+
+  if (SplitDWARFArg) {
+    if (DebugInfoKind == codegenoptions::NoDebugInfo)
+      DebugInfoKind = codegenoptions::LimitedDebugInfo;
+    CmdArgs.push_back("-enable-split-dwarf");
   }
 
   // After we've dealt with all combinations of things that could
   // make DebugInfoKind be other than None or DebugLineTablesOnly,
   // figure out if we need to "upgrade" it to standalone debug info.
   // We parse these two '-f' options whether or not they will be used,
   // to claim them even if you wrote "-fstandalone-debug -gline-tables-only"
   bool NeedFullDebug = Args.hasFlag(options::OPT_fstandalone_debug,

     EmitCodeView = Args.hasArg(options::OPT_gcodeview);
 
   const Arg *SplitDWARFArg = nullptr;
   RenderDebugOptions(getToolChain(), D, RawTriple, Args, EmitCodeView,
                      IsWindowsMSVC, CmdArgs, DebugInfoKind, SplitDWARFArg);
 
   // Add the split debug info name to the command lines here so we
   // can propagate it to the backend.
-  bool SplitDWARF = SplitDWARFArg && RawTriple.isOSLinux() &&
+  bool SplitDWARF = SplitDWARFArg &&
                     (isa<AssembleJobAction>(JA) || isa<CompileJobAction>(JA) ||
                      isa<BackendJobAction>(JA));
   const char *SplitDWARFOut;
   if (SplitDWARF) {
     CmdArgs.push_back("-split-dwarf-file");
     SplitDWARFOut = SplitDebugName(Args, Input);
     CmdArgs.push_back(SplitDWARFOut);
   }

     // fails, so that the main compilation's fallback to cl.exe runs.
     C.addCommand(llvm::make_unique<ForceSuccessCommand>(JA, *this, Exec,
                                                         CmdArgs, Inputs));
   } else {
     C.addCommand(llvm::make_unique<Command>(JA, *this, Exec, CmdArgs, Inputs));
   }
 
   // Handle the debug info splitting at object creation time if we're
   // creating an object.
-  // TODO: Currently only works on linux with newer objcopy.
   if (SplitDWARF && Output.getType() == types::TY_Object)
     SplitDebugInfo(getToolChain(), C, *this, JA, Args, Output, SplitDWARFOut);
 
   if (Arg *A = Args.getLastArg(options::OPT_pg))
     if (Args.hasArg(options::OPT_fomit_frame_pointer))
       D.Diag(diag::err_drv_argument_not_allowed_with) << "-fomit-frame-pointer"
                                                       << A->getAsString(Args);
 
   // Claim some arguments which clang supports automatically.

   assert(Input.isFilename() && "Invalid input.");
   CmdArgs.push_back(Input.getFilename());
 
   const char *Exec = getToolChain().getDriver().getClangProgramPath();
   C.addCommand(llvm::make_unique<Command>(JA, *this, Exec, CmdArgs, Inputs));
 
   // Handle the debug info splitting at object creation time if we're
   // creating an object.
-  // TODO: Currently only works on linux with newer objcopy.
-  if (Args.hasArg(options::OPT_gsplit_dwarf) &&
-      getToolChain().getTriple().isOSLinux())
+  if (Args.hasArg(options::OPT_gsplit_dwarf))
     SplitDebugInfo(getToolChain(), C, *this, JA, Args, Output,
                    SplitDebugName(Args, Input));
 }
 
 // Begin OffloadBundler
 
 void OffloadBundler::ConstructJob(Compilation &C, const JobAction &JA,
                                   const InputInfo &Output,

lib/Driver/ToolChains/FreeBSD.cpp

void freebsd::Assembler::ConstructJob(Compilation &C, const JobAction &JA,
   CmdArgs.push_back("-o");
   CmdArgs.push_back(Output.getFilename());
 
   for (const auto &II : Inputs)
     CmdArgs.push_back(II.getFilename());
 
   const char *Exec = Args.MakeArgString(getToolChain().GetProgramPath("as"));
   C.addCommand(llvm::make_unique<Command>(JA, *this, Exec, CmdArgs, Inputs));
+
+  // Handle the debug info splitting at object creation time if we're
+  // creating an object.
+  if (Args.hasArg(options::OPT_gsplit_dwarf))
+    SplitDebugInfo(getToolChain(), C, *this, JA, Args, Output,
+                   SplitDebugName(Args, Inputs[0]));
 }
 
 void freebsd::Linker::ConstructJob(Compilation &C, const JobAction &JA,
                                    const InputInfo &Output,
                                    const InputInfoList &Inputs,
                                    const ArgList &Args,
                                    const char *LinkingOutput) const {
   const toolchains::FreeBSD &ToolChain =

lib/Driver/ToolChains/Gnu.cpp

   for (const auto &II : Inputs)
     CmdArgs.push_back(II.getFilename());
 
   const char *Exec = Args.MakeArgString(getToolChain().GetProgramPath("as"));
   C.addCommand(llvm::make_unique<Command>(JA, *this, Exec, CmdArgs, Inputs));
 
   // Handle the debug info splitting at object creation time if we're
   // creating an object.
-  // TODO: Currently only works on linux with newer objcopy.
-  if (Args.hasArg(options::OPT_gsplit_dwarf) &&
-      getToolChain().getTriple().isOSLinux())
+  if (Args.hasArg(options::OPT_gsplit_dwarf))
     SplitDebugInfo(getToolChain(), C, *this, JA, Args, Output,
                    SplitDebugName(Args, Inputs[0]));
 }
 
 namespace {
 // Filter to remove Multilibs that don't exist as a suffix to Path
 class FilterNonExistent {
   StringRef Base, File;

test/Driver/split-debug.c

 // Check that we split debug output properly
 //
+// Linux
 // RUN: %clang -target x86_64-unknown-linux-gnu -gsplit-dwarf -c -### %s 2> %t
 // RUN: FileCheck -check-prefix=CHECK-ACTIONS < %t %s
 //
 // CHECK-ACTIONS: objcopy{{.*}}--extract-dwo{{.*}}"split-debug.dwo"
 // CHECK-ACTIONS: objcopy{{.*}}--strip-dwo{{.*}}"split-debug.o"
 
+// FreeBSD
+// RUN: %clang -fno-integrated-as -target x86_64-unknown-freebsd11.0 -gsplit-dwarf -c -### %s 2> %t
+// RUN: FileCheck -check-prefix=FREEBSD-NOIA-CHECK-ACTIONS < %t %s
+//
+// FREEBSD-NOIA-CHECK-ACTIONS: objcopy{{.*}}--extract-dwo{{.*}}"split-debug.dwo"
+// FREEBSD-NOIA-CHECK-ACTIONS: objcopy{{.*}}--strip-dwo{{.*}}"split-debug.o"
+//
+// RUN: %clang -fintegrated-as -target x86_64-unknown-freebsd11.0 -gsplit-dwarf -c -### %s 2> %t
+// RUN: FileCheck -check-prefix=FREEBSD-IA-CHECK-ACTIONS < %t %s
+//
+// FREEBSD-IA-CHECK-ACTIONS: objcopy{{.*}}--extract-dwo{{.*}}"split-debug.dwo"
+// FREEBSD-IA-CHECK-ACTIONS: objcopy{{.*}}--strip-dwo{{.*}}"split-debug.o"
 
+// Macosx
 // RUN: %clang -target x86_64-macosx -gsplit-dwarf -c -### %s 2> %t
-// RUN: FileCheck -check-prefix=CHECK-NO-ACTIONS < %t %s
+// RUN: FileCheck -check-prefix=MACOSX-CHECK-ACTIONS < %t %s
 //
-// CHECK-NO-ACTIONS-NOT: -split-dwarf
-
+// MACOSX-CHECK-ACTIONS: objcopy{{.*}}--extract-dwo{{.*}}"split-debug.dwo"
+// MACOSX-CHECK-ACTIONS: objcopy{{.*}}--strip-dwo{{.*}}"split-debug.o"
 
 // RUN: %clang -target x86_64-unknown-linux-gnu -gsplit-dwarf -o Bad.x -### %s 2> %t
 // RUN: FileCheck -check-prefix=CHECK-BAD < %t %s
 //
 // CHECK-BAD-NOT: "Bad.dwo"
 
 // RUN: %clang -target x86_64-unknown-linux-gnu -gsplit-dwarf -c -### %s 2> %t
 // RUN: FileCheck -check-prefix=CHECK-OPTION < %t %s

test/Driver/split-debug.s

 // Check that we split debug output properly
 //
 // RUN: %clang -target x86_64-unknown-linux-gnu -gsplit-dwarf -c -### %s 2> %t
 // RUN: FileCheck -check-prefix=CHECK-ACTIONS < %t %s
 //
 // CHECK-ACTIONS: objcopy{{.*}}--extract-dwo{{.*}}"split-debug.dwo"
 // CHECK-ACTIONS: objcopy{{.*}}--strip-dwo{{.*}}"split-debug.o"
 
+// RUN: %clang -target x86_64-unknown-freebsd11.0 -gsplit-dwarf -c -### %s 2> %t
+// RUN: FileCheck -check-prefix=FREEBSD-CHECK-ACTIONS < %t %s
+//
+// FREEBSD-CHECK-ACTIONS: objcopy{{.*}}--extract-dwo{{.*}}"split-debug.dwo"
+// FREEBSD-CHECK-ACTIONS: objcopy{{.*}}--strip-dwo{{.*}}"split-debug.o"
 
 // RUN: %clang -target x86_64-macosx -gsplit-dwarf -c -### %s 2> %t
-// RUN: FileCheck -check-prefix=CHECK-NO-ACTIONS < %t %s
+// RUN: FileCheck -check-prefix=MACOSX-CHECK-ACTIONS < %t %s
 //
-// CHECK-NO-ACTIONS-NOT: -split-dwarf
-
+// MACOSX-CHECK-ACTIONS: objcopy{{.*}}--extract-dwo{{.*}}"split-debug.dwo"
+// MACOSX-CHECK-ACTIONS: objcopy{{.*}}--strip-dwo{{.*}}"split-debug.o"
 
 // RUN: %clang -target x86_64-unknown-linux-gnu -gsplit-dwarf -o Bad.x -### %s 2> %t
 // RUN: FileCheck -check-prefix=CHECK-BAD < %t %s
 //
 // CHECK-BAD-NOT: "Bad.dwo"