This is an archive of the discontinued LLVM Phabricator instance.

[Dsymutil][Debuginfo][NFC] Refactor dsymutil to separate DWARF optimizing part.
AbandonedPublic

Authored by avl on Nov 26 2019, 1:54 AM.

Details

Summary

The primary goal of this refactoring is to separate DWARF optimizing part. So that it could be reused by linker or by any other client. There was a thread on llvm-dev discussing the necessity of such a refactoring:

http://lists.llvm.org/pipermail/llvm-dev/2019-September/135068.html.

The current implementation(DwarfLinker) was separated into four main pieces:
DwarfOptimizer, AddressesMap, DwarfEmitter, DwarfLinker.

DwarfOptimizer - Implements DWARF optimization: starting from root DIEs, scans debug info, and mark dies, which refer to live sections. Creates clones for DIEs, which should be kept. Patches offsets.
AddressesMap - Evaluates relocations and represents live addresses.
DwarfEmitter - Generates .debug_info sections according to specified DIEs or tables.
DwarfLinker - Implements dsymutil functionality.

Testing: it passes "check-all" lit testing. It also passes manual testing, which checks .dSYM bundles for llvm binaries created by dsymutil(by comparing dwarf dump). If there is a better test suite for dsymutil, I would be happy to use it.

Diff Detail

Event Timeline

avl created this revision.Nov 26 2019, 1:54 AM

Given that this should NFC, can you please run dsymutil with and without your change on clang and ensure the MD5 hash is identical?

avl added a comment.Dec 3 2019, 2:03 PM

Given that this should NFC, can you please run dsymutil with and without your change on clang and ensure the MD5 hash is identical?

ls -la golden-dsymutil/dsymutil
120050844 3 dec 15:16 golden-dsymutil/dsymutil

ls -la new-dsymutil/dsymutil
120308552 3 dec 21:01 new-dsymutil/dsymutil

golden-dsymutil/dsymutil build-Debug/bin/clang

md5 -r build-Debug/bin/clang.dSYM/Contents/Resources/DWARF/clang
10173920ad05e0014607717e009a319f build-Debug/bin/clang.dSYM/Contents/Resources/DWARF/clang

new-dsymutil/dsymutil build-Debug/bin/clang

md5 -r build-Debug/bin/clang.dSYM/Contents/Resources/DWARF/clang
10173920ad05e0014607717e009a319f build-Debug/bin/clang.dSYM/Contents/Resources/DWARF/clang

The clang was build with -DLLVM_ENABLE_MODULES=ON and -DLLVM_USE_SPLIT_DWARF=YES,
Please make me known, If it would be usefull to show results for other building options and other dsymutil
options.

In D70709#1767884, @avl wrote:

The clang was build with -DLLVM_ENABLE_MODULES=ON and -DLLVM_USE_SPLIT_DWARF=YES,
Please make me known, If it would be usefull to show results for other building options and other dsymutil
options.

Thanks for checking, Alexey. The modules build is good because it's kind of a special case, but I'd like to check a clang without modules as well. I'm a little confused by LLVM_USE_SPLIT_DWARF though, as that's not something that makes sense on macOS, unless it's a NO-OP?

Regarding the patch itself, I haven't had a look yet as there's a lot going on. Would it be possible to split this patch into the four main pieces you describe in the summary? I think that would make reviewing a lot easier.

avl added a comment.Dec 4 2019, 1:17 PM

The modules build is good because it's kind of a special case, but I'd like to check a clang without modules as well. I'm a little confused by LLVM_USE_SPLIT_DWARF though, as that's not something that makes sense on macOS unless it's a NO-OP?

right. LLVM_USE_SPLIT_DWARF would be NO-OP on macOS, I used it mistakenly. Following is the result for clang build without modules:

golden-dsymutil/dsymutil build-Debug/bin/clang
md5 -r build-Debug/bin/clang.dSYM/Contents/Resources/DWARF/clang
a31430a6a2fbbd81fbed88888c769213 build-Debug/bin/clang.dSYM/Contents/Resources/DWARF/clang

new-dsymutil/dsymutil build-Debug/bin/clang
md5 -r build-Debug/bin/clang.dSYM/Contents/Resources/DWARF/clang
a31430a6a2fbbd81fbed88888c769213 build-Debug/bin/clang.dSYM/Contents/Resources/DWARF/clang

Regarding the patch itself, I haven't had a look yet as there's a lot going on. Would it be possible to split this patch into the four main pieces you describe in the summary? I think that would make reviewing a lot easier.

Actually main changes are in DwarfLinker and DwarfOptimizer. These are closely coupled and could not be done separately. But DwarfEmitter and AddressMap could be done as separate patches. It would not actually decrease the main portion too much. But yes it would be possible and I would start from those split patches then. Thank you.

In D70709#1769569, @avl wrote:

The modules build is good because it's kind of a special case, but I'd like to check a clang without modules as well. I'm a little confused by LLVM_USE_SPLIT_DWARF though, as that's not something that makes sense on macOS unless it's a NO-OP?

right. LLVM_USE_SPLIT_DWARF would be NO-OP on macOS, I used it mistakenly. Following is the result for clang build without modules:

golden-dsymutil/dsymutil build-Debug/bin/clang
md5 -r build-Debug/bin/clang.dSYM/Contents/Resources/DWARF/clang
a31430a6a2fbbd81fbed88888c769213 build-Debug/bin/clang.dSYM/Contents/Resources/DWARF/clang

new-dsymutil/dsymutil build-Debug/bin/clang
md5 -r build-Debug/bin/clang.dSYM/Contents/Resources/DWARF/clang
a31430a6a2fbbd81fbed88888c769213 build-Debug/bin/clang.dSYM/Contents/Resources/DWARF/clang

Great, thank you for checking!

Regarding the patch itself, I haven't had a look yet as there's a lot going on. Would it be possible to split this patch into the four main pieces you describe in the summary? I think that would make reviewing a lot easier.

Actually main changes are in DwarfLinker and DwarfOptimizer. These are closely coupled and could not be done separately. But DwarfEmitter and AddressMap could be done as separate patches. It would not actually decrease the main portion too much. But yes it would be possible and I would start from those split patches then. Thank you.

Cool, much appreciated!

avl added a comment.Jan 17 2020, 4:56 AM

abandon this revision, since it was split into several revisions : D71068, D71839, D72476, D72915.

avl abandoned this revision.Jan 17 2020, 4:56 AM
llvm/tools/dsymutil/DwarfLinker.h