[llvm-objcopy] Add support for dwarf fission

[jakehehrlich]

Summary

This change adds support for dwarf fission.

[jakehehrlich]

I wanted to put this up for review a bit early for people to look at. I haven't confirmed that gdb/lldb are happy with this output yet but I think my basic test here is correct. It only appears to disagree with GNU objcopy on points that don't really matter.

My understanding of dwarf-fission is that the .dwo file should wind up with all the .dwo sections, the section header table, the .note section and nothing else (including the symbol table). I don't really have a spec to work off of to confirm that though. I'm just guessing based on what I've seen GNU objcopy and clang spit out. My understanding is also that the stripped file should wind up with all the sections it had before minus the .dwo sections. If anyone could confirm/deny or point me towards a better spec that would be awesome. If that is the case then I think this is behavior is correct.

[aprantl]

tools/llvm-objcopy/llvm-objcopy.cpp

72 ↗

(On Diff #119942)

equivliant
gsplit-dwaf

[jhenderson]

I've made a number of inline comments, but it's mostly fine-detail. The overall approach looks fine to me.

test/tools/llvm-objcopy/drawf-fission.test

6 ↗

(On Diff #119942)

I'm not sure you need the -rs here? Also, do you really need to check the output? Doesn't diff exit with a non-zero exit code if the files differ?

9 ↗

(On Diff #119942)

I think explicitly listing all the sections here and their properties is unnecessary here. Probably better would be to a) check the number of sections is as expected, and then simply do something like:

#DWARF: Name: (0)
#DWARF: Name: .debug_loc.dwo
#DWARF: Name: .debug_str.dwo
...

If you want to check specific properties (e.g. the Size), then just do Name, Size, Name, Size checks without the NEXT, since that will capture the ones that are needed in the right places (checking the number of sections means that you don't accidentally start checking another section's properties).

tools/llvm-objcopy/llvm-objcopy.cpp

68 ↗

(On Diff #119942)

"non DWARF" should be hyphenated, I believe: "non-DWARF".

71 ↗

(On Diff #119942)

What is "only-keep-dwo"?

76 ↗

(On Diff #119942)

The has the -> This has the

87–90 ↗

(On Diff #119942)

Why do we need this? According to the DWARF 5 appendix, there are tags in the debug information that allow matching the two up. It certainly doesn't mention anything about the SHT_NOTE sections. Note that there are many different kinds of SHT_NOTE sections as well, and not just one single one.

96 ↗

(On Diff #119942)

File should be a StringRef, I believe?

I'd also avoid the unnecessary abbreviation here. Either call the function WriteObject or WriteObjectFile.

112 ↗

(On Diff #119942)

StringRef

113 ↗

(On Diff #119942)

"... for the DWO sections", maybe? Also full stop.

166 ↗

(On Diff #119942)

Do you need the explicit template argument list here? Can't it derive it from (*Obj)'s type?

[jakehehrlich]

1. Fixed typos and StringRef/std::string mixup
2. Changed the way that SHT_NOTE sections were handled.
3. Made the test shorter because I included a ridiculous amount of information in that test.
4. Responded to a few other little things from James

[jakehehrlich]

test/tools/llvm-objcopy/drawf-fission.test

6 ↗

(On Diff #119942)

For some reason, until this comment, I was under the impression that llvm-lit and FileCheck had a special relationship and that if you wanted a test to fail the failure had to come from FileCheck but thinking about it that makes no sense. I prefer doing things this way much better.

tools/llvm-objcopy/llvm-objcopy.cpp

71 ↗

(On Diff #119942)

I had the option wrong to begin with and this is a remnant of that mistake. I meant "extract-dwo"

87–90 ↗

(On Diff #119942)

Yeah I think I mislead myself on that one reading into this further. Nothing even seems to produce a SHT_NOTE section. I'll remove that.

[jhenderson]

How do you expect -gsplit-dwarf to interact with the other two options? In other words, if a user specifies it with other or both of the other options, what do you expect the result to be?

I ask because I think the behaviour might be a little unexpected in some combinations of these three switches.

test/tools/llvm-objcopy/drawf-fission.test

8 ↗

(On Diff #120479)

I think you've gone too far in stripping things down, because now there's no way of knowing that only the expected sections are in the output. You also need to check the number of sections as well (i.e. the corresponding elf header field). This test might still pass if objcopy were to continue to emit the other sections!

tools/llvm-objcopy/llvm-objcopy.cpp

66 ↗

(On Diff #120479)

This help text is incorrect - we don't remove all DWARF info, just the DWO sections, leaving the skeleton debug information.

68 ↗

(On Diff #120479)

There's a similar issue with this help text - we do remove some DWARF information, but keep the DWO info.

75 ↗

(On Diff #120479)

Obj is an unneeded argument (and therefore so is the templating, I believe).

Also, I think this function may be better off being called IsDWOSection or similar, since that's all it's doing. It's more general purpose than a predicate for some other function.

Given that, the comment probably needs moving somewhere else.

83 ↗

(On Diff #120479)

Dwo should be capitalized for consistency, since it is an abbreviation.

87 ↗

(On Diff #120479)

I think this comment needs rewording, since there may not be another file, if using extract-dwo directly.

71 ↗

(On Diff #119942)

Looking at the online documentation, gsplit-dwarf is a compiler switch, rather than an objcopy switch. I want to be sure with this switch name that you have chosen it deliberately to match this, because I'd have probably just called it "split-dwarf" or even "split-dwo" (to match the other two options).

It's also not clear to me what the "gsplit-dwarf file" is. I think you need to reword the help text here.

[jakehehrlich]

In D39207#911606, @jhenderson wrote:

How do you expect -gsplit-dwarf to interact with the other two options? In other words, if a user specifies it with other or both of the other options, what do you expect the result to be?

I ask because I think the behaviour might be a little unexpected in some combinations of these three switches.

I'm fine renaming this to split-dwo. I think that name is better. I took the name gsplit-dwarf from clang but there really is no reason to do that.

As for using these in conjunction...man I have no clue what I expect out of that. Even if I just think about -split-dwo and -extract-dwo being used in combination I'm a bit confused as to what should happen. GNU objcopy's answer depends on the order that you give the arguments in. I would interpret the result of requesting both be done as a contradiction. e.g. it is demanded that the result of --strip-dwo should not have any .dwo sections. Likewise the result of --extract-dwo should be as small as possible while still maintaining any .dwo sections. I suppose one consistent way of interpreting this is that just the string table should be left. The other way would be to say this is an error. I think throwing errors like that checking for consistent sets of flags is kind of a nightmare and in this case there is a consistent result it's just somewhat unexpected (but so is what GNU objcopy does IMO). Right now as is llvm-objcopy will just give you the string table which seems consistent with "--strip-dwo guarantees no .dwo sections and --extract-dwo will preserve any .dwo sections it can but nothing else". I'm happy with that.

The other issue is how options in general should effect things like gsplit-dwarf (or now split-dwo). One option is to make no options apply to output to a separate file. This would mean such file outputting options would never compose with other options. In the case of GNU objcopy's --dump-section this is acceptable. The other idea is to make all options apply to them. This has the consequence of being too course and not allowing different options to apply to each file. I also think this could produce some extremely unexpected results now that you point it out. Neither of these seem like good ideas to me. In fact this seems like a general reason not to support these kind of options (options that output object files). Is removing this flag fine with you or do you think one of these options is consistent? GNU objcopy just has --strip-dwo and --extract-dwo. I think that's fine even though it's somewhat non-ideal in the number of invocations the compiler has to make.

[jakehehrlich]

1. Added section count to tests to avoid false positives.
2. Fixed option descriptions.
3. Repalced "Dwo" with "DWO" everywhere including in cl::opt types.
4. Changed "-gsplit-dwarf" to "-split-dwo" but I think this should actually be removed. I'm waiting to hear back before I remove it.
5. Moved some comments around and altered them.
6. Removed extraneous passing of Obj and additionally removed the template.

[jhenderson]

In D39207#912415, @jakehehrlich wrote:

As for using these in conjunction...man I have no clue what I expect out of that. Even if I just think about -split-dwo and -extract-dwo being used in combination I'm a bit confused as to what should happen. GNU objcopy's answer depends on the order that you give the arguments in. I would interpret the result of requesting both be done as a contradiction. e.g. it is demanded that the result of --strip-dwo should not have any .dwo sections. Likewise the result of --extract-dwo should be as small as possible while still maintaining any .dwo sections. I suppose one consistent way of interpreting this is that just the string table should be left. The other way would be to say this is an error. I think throwing errors like that checking for consistent sets of flags is kind of a nightmare and in this case there is a consistent result it's just somewhat unexpected (but so is what GNU objcopy does IMO). Right now as is llvm-objcopy will just give you the string table which seems consistent with "--strip-dwo guarantees no .dwo sections and --extract-dwo will preserve any .dwo sections it can but nothing else". I'm happy with that.

I agree. This sounds like a reasonable plan.

In D39207#912415, @jakehehrlich wrote:

The other issue is how options in general should effect things like gsplit-dwarf (or now split-dwo). One option is to make no options apply to output to a separate file. This would mean such file outputting options would never compose with other options. In the case of GNU objcopy's --dump-section this is acceptable. The other idea is to make all options apply to them. This has the consequence of being too course and not allowing different options to apply to each file. I also think this could produce some extremely unexpected results now that you point it out. Neither of these seem like good ideas to me. In fact this seems like a general reason not to support these kind of options (options that output object files). Is removing this flag fine with you or do you think one of these options is consistent? GNU objcopy just has --strip-dwo and --extract-dwo. I think that's fine even though it's somewhat non-ideal in the number of invocations the compiler has to make.

I think it would be a shame to drop the extra switch, if we can come up with some sort of sensible scheme to implement it. I think my preference of the two ideas (apply all or no options to the secondary output) is for the options to not be applied to the DWO file. If a user wants to apply additional options to it, they can invoke objcopy again on the file.

However, that all being said, it could be dropped for now, if you prefer, since there isn't a compelling use case, so that the other switches can go in, since those are uncontroversially useful.

tools/llvm-objcopy/llvm-objcopy.cpp

107 ↗

(On Diff #121089)

If you keep split-dwo, then GSplitObj -> SplitObj, or perhaps more clearly DWOFile.

71 ↗

(On Diff #119942)

Still doesn't read right. Maybe "equivalent to -strip-dwo on the input file to the specified file, then -extract-dwo on the input file to the output file". Does the command-line code have the ability to set the display name of the option's argument? If so, it might be nice to use that in the help text instead of "specified file" to make it unambiguous. I don't think there is a "strip-dwo file".

Also, if you keep this option, you should rename the variable to SplitDWO, to match its new name.

[jakehehrlich]

1. Cleaned up command line argument description. I decided to keep the split-dwo option. For the one use case of dwarf fission that I'm aware of (clang) that's a better option to use. I agree that not applying options to the DWO file is more sensible.
2. removed/replaced all references to "GSplitDwarf"

[jhenderson]

Couple more small things, and then I think we're done.

tools/llvm-objcopy/llvm-objcopy.cpp

72–73 ↗

(On Diff #121163)

Reading again, I think these are the wrong way around - the extract needs to happen first, otherwise there's nothing to extract!

Text should read "equivalent to extract-dwo on the input file to <dwo-file>, then strip-dwo on the input file".

148–149 ↗

(On Diff #121163)

To be absolutely clear, I'd move this clause to before the RemovePred assignments, either to about line 128 (in the current diff), or to line 137, depending on your preference. The reason for this is to maintain the conceptual order of things - split the DWO into a separate file needs to happen before the stripping. Really, it's an independent operation, so should not be mixed in with all the remove section stuff.

[jakehehrlich]

Also, as more stripping options become available I think coming up with a system to handle the removal predicate would be nice. I might write that patch soon.

[jhenderson]

LGTM. The change as it stands is fine, but I don't understand why you've made a couple of changes, that I don't remember seeing before (see inline comments). If there's a good reason for them, feel free to put the changes in as part of this, or to remove them if they're not supposed to be there.

In D39207#914752, @jakehehrlich wrote:

Also, as more stripping options become available I think coming up with a system to handle the removal predicate would be nice. I might write that patch soon.

Sounds good.

tools/llvm-objcopy/llvm-objcopy.cpp

81 ↗

(On Diff #121422)

Why the removal of static here and in the following lines?

139 ↗

(On Diff #121422)

Why the change here?

[jakehehrlich]

It would appear some kind stranger made some sweeping improvements to the code base that caused conflicts with the code I wrote. I never added those 'static' clauses (but they're good) and I never used the "using" keyword (but I support it). See here: https://reviews.llvm.org/rL317123

I'll make sure those improvements are kept.

llvm/trunk/test/tools/llvm-objcopy/drawf-fission.test

+# RUN: llvm-objcopy -extract-dwo %p/Inputs/dwarf.dwo %t
+# RUN: llvm-objcopy -strip-dwo %p/Inputs/dwarf.dwo %t2
+# RUN: llvm-objcopy -split-dwo=%t3 %p/Inputs/dwarf.dwo %t4
+# RUN: llvm-readobj -file-headers -sections %t | FileCheck %s -check-prefix=DWARF
+# RUN: llvm-readobj -file-headers -sections %t2 | FileCheck %s -check-prefix=STRIP
+# RUN: diff %t %t3
+# RUN: diff %t2 %t4
+
+#DWARF:     SectionHeaderCount: 8
+
+#DWARF:     Name: .debug_loc.dwo
+#DWARF:     Name: .debug_str.dwo
+#DWARF:     Name: .debug_str_offsets.dwo
+#DWARF:     Name: .debug_info.dwo
+#DWARF:     Name: .debug_abbrev.dwo
+#DWARF:     Name: .debug_line.dwo
+#DWARF:     Name: .strtab
+
+#STRIP:     SectionHeaderCount: 24
+
+#STRIP:    Name: .text
+#STRIP:    Name: .rodata.str1.1
+#STRIP:    Name: .debug_str
+#STRIP:    Name: .debug_abbrev
+#STRIP:    Name: .debug_info
+#STRIP:    Name: .debug_ranges
+#STRIP:    Name: .debug_macinfo
+#STRIP:    Name: .debug_addr
+#STRIP:    Name: .debug_pubnames
+#STRIP:    Name: .debug_pubtypes
+#STRIP:    Name: .comment
+#STRIP:    Name: .note.GNU-stack
+#STRIP:    Name: .debug_frame
+#STRIP:    Name: .debug_line
+#STRIP:    Name: .symtab
+#STRIP:    Name: .rela.text
+#STRIP:    Name: .rela.debug_info
+#STRIP:    Name: .rela.debug_addr
+#STRIP:    Name: .rela.debug_pubnames
+#STRIP:    Name: .rela.debug_pubtypes
+#STRIP:    Name: .rela.debug_frame
+#STRIP:    Name: .rela.debug_line
+#STRIP:    Name: .strtab

llvm/trunk/tools/llvm-objcopy/Object.h

public:
   uint32_t Machine;
   uint32_t Version;
   uint32_t Flags;
   bool WriteSectionHeaders = true;
 
   Object(const object::ELFObjectFile<ELFT> &Obj);
   virtual ~Object() = default;
 
+  const SectionBase *getSectionHeaderStrTab() const { return SectionNames; }
   void removeSections(std::function<bool(const SectionBase &)> ToRemove);
   virtual size_t totalSize() const = 0;
   virtual void finalize() = 0;
   virtual void write(FileOutputBuffer &Out) const = 0;
 };
 
 template <class ELFT> class ELFObject : public Object<ELFT> {
 private:

llvm/trunk/tools/llvm-objcopy/llvm-objcopy.cpp

static cl::opt<std::string>
     OutputFormat("O", cl::desc("set output format to one of the following:"
                                "\n\tbinary"));
 static cl::list<std::string> ToRemove("remove-section",
                                       cl::desc("Remove a specific section"));
 static cl::alias ToRemoveA("R", cl::desc("Alias for remove-section"),
                            cl::aliasopt(ToRemove));
 static cl::opt<bool> StripSections("strip-sections",
                                    cl::desc("Remove all section headers"));
+static cl::opt<bool>
+    StripDWO("strip-dwo", cl::desc("remove all DWARF .dwo sections from file"));
+static cl::opt<bool> ExtractDWO(
+    "extract-dwo",
+    cl::desc("remove all sections that are not DWARF .dwo sections from file"));
+static cl::opt<std::string>
+    SplitDWO("split-dwo",
+             cl::desc("equivalent to extract-dwo on the input file to "
+                      "<dwo-file>, then strip-dwo on the input file"),
+             cl::value_desc("dwo-file"));
 
 using SectionPred = std::function<bool(const SectionBase &Sec)>;
 
-void CopyBinary(const ELFObjectFile<ELF64LE> &ObjFile) {
+bool IsDWOSection(const SectionBase &Sec) {
+  return Sec.Name.endswith(".dwo");
+}
+
+template <class ELFT>
+bool OnlyKeepDWOPred(const Object<ELFT> &Obj, const SectionBase &Sec) {
+  // We can't remove the section header string table.
+  if (&Sec == Obj.getSectionHeaderStrTab())
+    return false;
+  // Short of keeping the string table we want to keep everything that is a DWO
+  // section and remove everything else.
+  return !IsDWOSection(Sec);
+}
+
+template <class ELFT>
+void WriteObjectFile(const Object<ELFT> &Obj, StringRef File) {
   std::unique_ptr<FileOutputBuffer> Buffer;
+  ErrorOr<std::unique_ptr<FileOutputBuffer>> BufferOrErr =
+      FileOutputBuffer::create(File, Obj.totalSize(),
+                               FileOutputBuffer::F_executable);
+  if (BufferOrErr.getError())
+    error("failed to open " + OutputFilename);
+  else
+    Buffer = std::move(*BufferOrErr);
+  Obj.write(*Buffer);
+  if (auto EC = Buffer->commit())
+    reportError(File, EC);
+}
+
+template <class ELFT>
+void SplitDWOToFile(const ELFObjectFile<ELFT> &ObjFile, StringRef File) {
+  // Construct a second output file for the DWO sections.
+  ELFObject<ELFT> DWOFile(ObjFile);
+
+  DWOFile.removeSections([&](const SectionBase &Sec) {
+    return OnlyKeepDWOPred<ELFT>(DWOFile, Sec);
+  });
+  DWOFile.finalize();
+  WriteObjectFile(DWOFile, File);
+}
+
+void CopyBinary(const ELFObjectFile<ELF64LE> &ObjFile) {
   std::unique_ptr<Object<ELF64LE>> Obj;
+
   if (!OutputFormat.empty() && OutputFormat != "binary")
     error("invalid output format '" + OutputFormat + "'");
   if (!OutputFormat.empty() && OutputFormat == "binary")
     Obj = llvm::make_unique<BinaryObject<ELF64LE>>(ObjFile);
   else
     Obj = llvm::make_unique<ELFObject<ELF64LE>>(ObjFile);
 
+  if (!SplitDWO.empty())
+    SplitDWOToFile<ELF64LE>(ObjFile, SplitDWO.getValue());
+
   SectionPred RemovePred = [](const SectionBase &) { return false; };
 
   if (!ToRemove.empty()) {
     RemovePred = [&](const SectionBase &Sec) {
       return std::find(std::begin(ToRemove), std::end(ToRemove), Sec.Name) !=
              std::end(ToRemove);
     };
   }
 
+  if (StripDWO || !SplitDWO.empty())
+    RemovePred = [RemovePred, &Obj](const SectionBase &Sec) {
+      return IsDWOSection(Sec) || RemovePred(Sec);
+    };
+
+  if (ExtractDWO)
+    RemovePred = [RemovePred, &Obj](const SectionBase &Sec) {
+      return OnlyKeepDWOPred(*Obj, Sec) || RemovePred(Sec);
+    };
+
   if (StripSections) {
     RemovePred = [RemovePred](const SectionBase &Sec) {
       return RemovePred(Sec) || (Sec.Flags & SHF_ALLOC) == 0;
     };
     Obj->WriteSectionHeaders = false;
   }
 
   Obj->removeSections(RemovePred);
-
   Obj->finalize();
-  ErrorOr<std::unique_ptr<FileOutputBuffer>> BufferOrErr =
-      FileOutputBuffer::create(OutputFilename, Obj->totalSize(),
-                               FileOutputBuffer::F_executable);
-  if (BufferOrErr.getError())
-    error("failed to open " + OutputFilename);
-  else
-    Buffer = std::move(*BufferOrErr);
-  std::error_code EC;
-  if (EC)
-    report_fatal_error(EC.message());
-  Obj->write(*Buffer);
-  if (auto EC = Buffer->commit())
-    reportError(OutputFilename, EC);
+  WriteObjectFile(*Obj, OutputFilename.getValue());
 }
 
 int main(int argc, char **argv) {
   // Print a stack trace if we signal out.
   sys::PrintStackTraceOnErrorSignal(argv[0]);
   PrettyStackTraceProgram X(argc, argv);
   llvm_shutdown_obj Y; // Call llvm_shutdown() on exit.
   cl::ParseCommandLineOptions(argc, argv, "llvm objcopy utility\n");