This is an archive of the discontinued LLVM Phabricator instance.

[llvm-pdbutil] Move InputFile/FormatUtil/LinePrinter to the PDB library.
ClosedPublic

Authored by CarlosAlbertoEnciso on Mar 22 2022, 6:27 AM.

Details

Summary

[llvm-pdbutil] Move InputFile/FormatUtil/LinePrinter to the PDB library.

At Sony we are developing llvm-dva

https://lists.llvm.org/pipermail/llvm-dev/2020-August/144174.html

For its PDB support, it requires functionality already present in llvm-pdbutil.

We intend to move that functionaly into the PDB library to be shared by both tools. That change will be done in 2 steps, that
will be submitted as 2 patches:

(1) Replace 'ExitOnError' with explicit error handling.
(2) Move the intended shared code to the PDB library.

Patch for step (1): https://reviews.llvm.org/D121801

This patch is for step (2).

Move InputFile.cpp[h], FormatUtil.cpp[h] and LinePrinter.cpp[h] files to the debug PDB library.

It exposes the following functionality that can be used by tools:

  • Open a PDB file.
  • Get module debug stream.
  • Traverse module sections.
  • Traverse module subsections.

Most of the needed functionality is in InputFile, but there are dependencies from LinePrinter and FormatUtil.

Some other functionality is in the following functions in DumpOutputStyle.cpp file:

  • iterateModuleSubsections
  • getModuleDebugStream
  • iterateOneModule
  • iterateSymbolGroups
  • iterateModuleSubsections

Only these specific functions from DumpOutputStyle are moved to the PDB library.

Diff Detail

Event Timeline

Herald added a project: Restricted Project. · View Herald TranscriptMar 22 2022, 6:27 AM
CarlosAlbertoEnciso requested review of this revision.Mar 22 2022, 6:27 AM

I am preparing some numbers about sizes before and after this patch.

Updated patch fixing some clang-format issues.

CarlosAlbertoEnciso added a comment.EditedMar 23 2022, 4:51 AM

The following modules are directly affected by this patch:

llvm\lib\DebugInfo\DebugInfoPDB
llvm\lib\DebugInfo\Symbolize
llvm\tools\llvm-objdump
llvm\tools\llvm-pdbutil
llvm\tools\llvm-readobj
llvm\tools\llvm-symbolizer
llvm\tools\sancov

The following tables show the sizes before and after the patch, for static and shared libraries.

Static libraries

ModuleBeforeAfter
libLLVMDebugInfoPDB.a2,646,3542,931,710
libLLVMSymbolize.a324,626324,626
llvm-objdump32,456,60032,456,600
llvm-pdbutil8,084,8168,090,784
llvm-readobj8,732,6088,732,608
llvm-symbolizer7,196,4327,196,432
sancov32,610,39232,610,392

Shared libraries

ModuleBeforeAfter
libLLVMDebugInfoPDB.so1,180,7441,314,320
libLLVMSymbolize.so232,464232,464
llvm-objdump912,696912,696
llvm-pdbutil1,293,7921,215,952
llvm-readobj2,168,0162,168,016
llvm-symbolizer76,92076,920
sancov158,432158,432

The following modules are directly affected by this patch:

Which build configuration is that in & could you include % growth in the chart? (it'd be interesting to see if there's any growth in a build that uses -ffunction-sections -fdata-sections -Wl,-gc-sections)

rnk accepted this revision.Mar 23 2022, 3:53 PM

lgtm

llvm/include/llvm/DebugInfo/PDB/Native/InputFile.h
163

These callbacks look like they could actually be function_refs. Can you put a // TODO comment here to de-templatify them? I don't want to block on it.

This revision is now accepted and ready to land.Mar 23 2022, 3:53 PM

There's a typo in the title:

[llmv-pdbutil] Move InputFile/FormatUtil/LinePrinter to PDB library.

llvm-pdbutil

llvm/lib/DebugInfo/PDB/Native/LinePrinter.cpp
33

Not a blocker, but I'm a bit allergic to global state, especially in a lib. I wonder how much work it would be to keep this Filters state inside the LinePrinter class and pass it by reference to the iterate* functions. That could be done in a later patch.

CarlosAlbertoEnciso retitled this revision from [llmv-pdbutil] Move InputFile/FormatUtil/LinePrinter to the PDB library. to [llvm-pdbutil] Move InputFile/FormatUtil/LinePrinter to the PDB library..Mar 24 2022, 4:05 AM
CarlosAlbertoEnciso edited the summary of this revision. (Show Details)

Which build configuration is that in & could you include % growth in the chart? (it'd be interesting to see if there's any growth in a build that uses -ffunction-sections -fdata-sections -Wl,-gc-sections)

It is a Release configuration. I have included new data for the requested options ( -ffunction-sections -fdata-sections -Wl,-gc-sections)

Static libraries

ModuleBeforeAfter% growth
libLLVMDebugInfoPDB.a2,646,3542,931,71010.78
libLLVMSymbolize.a324,626324,6260.00
llvm-objdump32,456,93632,456,9360.00
llvm-pdbutil8,084,8008,090,7680.07
llvm-readobj8,733,7848,733,7840.00
llvm-symbolizer7,201,7047,201,7040.00
sancov32,618,92032,618,9200.00

Shared libraries

ModuleBeforeAfter% growth
libLLVMDebugInfoPDB.so1,180,7441,314,32011.31
libLLVMSymbolize.so232,464232,4640.00
llvm-objdump912,696912,6960.00
llvm-pdbutil1,293,7921,215,952-6.02
llvm-readobj2,168,0162,168,0160.00
llvm-symbolizer76,92076,9200.00
sancov158,432158,4320.00
llvm/include/llvm/DebugInfo/PDB/Native/InputFile.h
163

I have added a // TODO comment.

llvm/lib/DebugInfo/PDB/Native/LinePrinter.cpp
33

I have added a // TODO comment.
After pushing this patch I will look at creating another patch to move Filters.

Add couple of // TODO comments for:

  • Move Filters inside the LinePrinter class.
  • Change callbacks used by the symbol group traversal to function-refs (de-templatify them).

This causes llvm-pdbutil crashes when modi is neither 0 nor 1, e.g. llvm-pdbutil dump --symbols -modi=2 a.pdb

llvm-pdbutil: ../../llvm/include/llvm/DebugInfo/PDB/Native/InputFile.h:185: llvm::Error llvm::pdb::iterateSymbolGroups(llvm::pdb::InputFile &, const Optional<llvm::pdb::PrintScope> &, CallbackT) [CallbackT = (lambda at ../../llvm/tools/llvm-pdbutil/DumpOutputStyle.cpp:1472:7)]: Assertion `llvm::pdb::Filters.DumpModi == 1' failed.

This causes llvm-pdbutil crashes when modi is neither 0 nor 1, e.g. llvm-pdbutil dump --symbols -modi=2 a.pdb

llvm-pdbutil: ../../llvm/include/llvm/DebugInfo/PDB/Native/InputFile.h:185: llvm::Error llvm::pdb::iterateSymbolGroups(llvm::pdb::InputFile &, const Optional<llvm::pdb::PrintScope> &, CallbackT) [CallbackT = (lambda at ../../llvm/tools/llvm-pdbutil/DumpOutputStyle.cpp:1472:7)]: Assertion `llvm::pdb::Filters.DumpModi == 1' failed.

I am sorry for my late reply.
When InputFile.cpp[h] were moved to the PDB library, no logic related with modi was changed.
I will have a look at it and raise a bugzilla for that issue.

CarlosAlbertoEnciso added a comment.EditedApr 8 2022, 12:23 PM

This causes llvm-pdbutil crashes when modi is neither 0 nor 1, e.g. llvm-pdbutil dump --symbols -modi=2 a.pdb

@zequanwu, these are my findings

Before this patch:

llvm-pdbutil dump --symbols -modi=2 a.pdb         --> OK
llvm-pdbutil dump --symbols -modi=2 -modi=1 a.pdb --> Assertion
llvm-pdbutil dump --symbols -modi=1 -modi=2 a.pdb --> Assertion

Assertion failed: opts::dump::DumpModi.getNumOccurrences() == 1, file X:\llvm-root\llvm-project\llvm\tools\llvm-pdbutil\DumpOutputStyle.cpp, line 456

It means that the -modi argument can't be specified more than once.

With this patch:

llvm-pdbutil dump --symbols -modi=2 a.pdb         --> Assertion
llvm-pdbutil dump --symbols -modi=2 -modi=1 a.pdb --> Assertion
llvm-pdbutil dump --symbols -modi=1 -modi=2 a.pdb --> Assertion

Assertion failed: Modi == 1, file X:\diva-root\llvm-project\llvm\include\llvm/DebugInfo/PDB/Native/InputFile.h, line 186

It means that the -modi argument can be '0' or '1', which is wrong.

I will upload a patch to fix the issue and move the assertion out of the library and put it in the driver.

Created https://reviews.llvm.org/D123483 with appropriated patch.