This is an archive of the discontinued LLVM Phabricator instance.

[lldb][ELF] Read symbols from .gnu_debugdata sect.
ClosedPublic

Authored by kwk on Aug 27 2019, 2:25 AM.

Details

Summary

If the .symtab section is stripped from the binary it might be that
there's a .gnu_debugdata section which contains a smaller .symtab in
order to provide enough information to create a backtrace with function
names or to set and hit a breakpoint on a function name.

This change looks for a .gnu_debugdata section in the ELF object file.
The .gnu_debugdata section contains a xz-compressed ELF file with a
.symtab section inside. Symbols from that compressed .symtab section
are merged with the main object file's .dynsym symbols (if any).
In addition we always load the .dynsym even if there's a .symtab
section.

For example, the Fedora and RHEL operating systems strip their binaries
but keep a .gnu_debugdata section. While gdb already can read this
section, LLDB until this patch couldn't. To test this patch on a
Fedora or RHEL operating system, try to set a breakpoint on the "help"
symbol in the "zip" binary. Before this patch, only GDB can set this
breakpoint; now LLDB also can do so without installing extra debug
symbols:

lldb /usr/bin/zip -b -o "b help" -o "r" -o "bt" -- -h

The above line runs LLDB in batch mode and on the "/usr/bin/zip -h"
target:

(lldb) target create "/usr/bin/zip"
Current executable set to '/usr/bin/zip' (x86_64).
(lldb) settings set -- target.run-args  "-h"

Before the program starts, we set a breakpoint on the "help" symbol:

(lldb) b help
Breakpoint 1: where = zip`help, address = 0x00000000004093b0

Once the program is run and has hit the breakpoint we ask for a
backtrace:

(lldb) r
Process 10073 stopped
* thread #1, name = 'zip', stop reason = breakpoint 1.1
    frame #0: 0x00000000004093b0 zip`help
zip`help:
->  0x4093b0 <+0>:  pushq  %r12
    0x4093b2 <+2>:  movq   0x2af5f(%rip), %rsi       ;  + 4056
    0x4093b9 <+9>:  movl   $0x1, %edi
    0x4093be <+14>: xorl   %eax, %eax

Process 10073 launched: '/usr/bin/zip' (x86_64)
(lldb) bt
* thread #1, name = 'zip', stop reason = breakpoint 1.1
  * frame #0: 0x00000000004093b0 zip`help
    frame #1: 0x0000000000403970 zip`main + 3248
    frame #2: 0x00007ffff7d8bf33 libc.so.6`__libc_start_main + 243
    frame #3: 0x0000000000408cee zip`_start + 46

In order to support the .gnu_debugdata section, one has to have LZMA
development headers installed. The CMake section, that controls this
part looks for the LZMA headers and enables .gnu_debugdata support by
default if they are found; otherwise or if explicitly requested, the
minidebuginfo support is disabled.

GDB supports the "mini debuginfo" section .gnu_debugdata since v7.6
(2013).

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
kwk added inline comments.Sep 2 2019, 12:40 AM
lldb/source/Plugins/SymbolVendor/ELF/SymbolVendorELF.cpp
79

Fixed in f509e547ab793068e8b822317b93060077623b74

llvm/CMakeLists.txt
53–361 ↗(On Diff #217933)

Fixed in d4236447e75a882ab61bd369cb8253f2a908368f

kwk updated this revision to Diff 218300.Sep 2 2019, 12:45 AM
kwk marked an inline comment as done.
  • Moved and renamed m_gnu_debug_data_object_file
kwk marked 2 inline comments as done.Sep 2 2019, 12:47 AM
kwk added inline comments.
lldb/source/Plugins/ObjectFile/ELF/ObjectFileELF.h
394

Fixed in 0427898acf4a292f9737a33059e985239cc0e640

labath added inline comments.Sep 2 2019, 1:11 AM
lldb/cmake/modules/LLDBConfig.cmake
337 ↗(On Diff #218300)

This looks like a really useful macro. I'll be sure to remember it.

338 ↗(On Diff #218300)

It doesn't look like this is used anywhere.

341 ↗(On Diff #218300)

Please put this into Config.h.cmake. I know a lot of code in this file does not do that, but that's because we didn't have a Config.h back then.

344 ↗(On Diff #218300)

Is this code actually reachable? My understanding is that LLDB_ENABLE_LZMA should be automatically false if LIBLZMA_FOUND is not true...

lldb/source/Host/common/LZMA.cpp
81–82 ↗(On Diff #218300)

printf format string here.

91 ↗(On Diff #218300)

This is a valid printf string, but I am not sure if you intended it to come out like {lzma error: LZMA_MEMLIMIT_ERROR}

92 ↗(On Diff #218300)

Calling .data() on a StringRef and expecting to get a c string is ill-advised. convertLZMACodeToString is a local static function, you could just make it return a const char *...

lldb/source/Plugins/ObjectFile/ELF/ObjectFileELF.cpp
1904

No, I think you should just delete this bit of code completely (as the containing object file will call SetModulesArchitecture on its own).

lldb/source/Plugins/ObjectFile/ELF/ObjectFileELF.h
157–164

This doesn't need to be public now, right?

lldb/source/Plugins/SymbolVendor/ELF/SymbolVendorELF.cpp
72

Using a unique_ptr is not a bad idea, but as right now this patch does not need to touch this file, it shouldn't be done as a part of this patch. Feel free to submit the unique_ptr thingy as a separate patch. No need to go through the review process...

lldb/test/CMakeLists.txt
119–121 ↗(On Diff #218300)

I'd just make this unconditional, and add the tools to the big lldb-test-deps block.

labath added inline comments.Sep 2 2019, 1:20 AM
lldb/cmake/modules/LLDBConfig.cmake
341 ↗(On Diff #218300)

(Among other things, this helps with incremental rebuilds, as only the files which include Config.h need to be recompiled)

kwk updated this revision to Diff 218374.Sep 2 2019, 8:28 AM
kwk marked 24 inline comments as done.
  • Remove left-over variable in CMake
  • Move LLDB_ENABLE_LZMA into Config.h.cmake
  • Remove unused code
  • Change signature of lldb_private::lzma::uncompress
  • Fixup
  • Fixup
  • When using the ObjectFile's own GetArchitecture(), no debug symbols are found
  • Make GetGnuDebugDataObjectFile private
  • Cleanup
  • moved llvm-nm llvm-strip as test dependencies
kwk added a comment.Sep 2 2019, 8:29 AM

@labath I did go over a lot of your comments. Thank you for helping me in becoming in a better contributor. I'm still not done with the python/make -> lit test transition but the rest is mostly covered.

lldb/cmake/modules/LLDBConfig.cmake
337 ↗(On Diff #218300)

Make sure to include include(CMakeDependentOption)

338 ↗(On Diff #218300)

Right, that was a left-over. Removed in 7dc045e67bb.

341 ↗(On Diff #218300)

Addressed in a6277c07451

344 ↗(On Diff #218300)

The logic is as follows:

LLDB_ENABLE_LZMA is ON if LIBLZMA_FOUND is ON.

Until now I though that can always go ahead and manually toggle on the option LLDB_ENABLE_LZMA by hand. And if you do this when LZMA is not available, then you get the fatal error. But as I understand [1] after reading it more than once, I think you're right: "This macro presents an option to the user only if a set of other conditions are true."

Let me verify this with a small example... yes, you're right. Even if you specify cmake . -DLLVM_ENABLE_LZMA it will not enable the option. Awesome! Fixed in 156f0af8c29980139ff1a2a8e51a768050564898

[1]: https://cmake.org/cmake/help/v3.0/module/CMakeDependentOption.html

lldb/source/Plugins/ObjectFile/ELF/ObjectFileELF.cpp
1904

Let me try if it works how you suggested it. No, it doesn't work as no breakpoint from the embedded .gnu_debugdata section will be hit anymore. This section was inspired by ObjectFileELF::CreateInstance as you suggested to look at once.

lldb/source/Plugins/ObjectFile/ELF/ObjectFileELF.h
157–164

Right, now that it is only called from CreateSections, it can be made private again. Fixed in bb1f3f0b5bd.

lldb/source/Plugins/SymbolVendor/ELF/SymbolVendorELF.cpp
72

It wasn't an idea but a necessity back when the whole minidebuginfo stuff was here. It still is a left-over that I have remove this change.

lldb/test/CMakeLists.txt
119–121 ↗(On Diff #218300)

Done 1d731e225a243bb2afb66eb1f06cd010ee094208

labath added inline comments.Sep 2 2019, 10:09 AM
lldb/source/Plugins/ObjectFile/ELF/ObjectFileELF.cpp
1904

I'd be very surprised if this makes a difference as calling SetModulesArchitecture just invokes Module::SetArchitecture, which is actually a no-op if the module architecture has already been set.

However, I see now that SetArchitecture also acts as a check, which ensures that the architecture of the (nested) object file is compatible with the module architecture. This bit sounds like a useful check against someone accidentally (however unlikely) embedding a different object file into the gnu_debugdata section, so I guess we can keep that.

kwk updated this revision to Diff 218618.Sep 4 2019, 2:19 AM
kwk marked an inline comment as done.
  • add lit test for minidebuginfo
  • Fixup
  • test
  • Fiddle with the lit test
  • Fix binary in lit test
  • Working lit test - still need to only run it when minidebuginfo is available (LZMA)
  • Make FileCheck less verbose
  • Only run minidebuginfo lit test when lzma is available
  • fix mylib.so not found problem
  • Removed old python based minidebuginfo test
  • Remove changes that were only necessary for the old python based test for minidebuginfo
kwk updated this revision to Diff 218621.Sep 4 2019, 2:20 AM
  • remove not needed <vector> include
kwk updated this revision to Diff 218623.Sep 4 2019, 2:29 AM

squashed and rebased on top of master

labath added a comment.Sep 4 2019, 3:54 AM

I think we're getting pretty close. I have a couple of extra comments inline, but they're all pretty minor, I hope. I'd just like to add some more tests, per our discussion on the IRC, and I have some doubts about what exactly is the test you've written actually testing, which I'd like to clarify...

lldb/CMakeLists.txt
101 ↗(On Diff #218618)

It looks like you're already adding these in lldb/lit/CMakeLists.txt. Does it need to be in both?

lldb/include/lldb/Host/LZMA.h
24–25 ↗(On Diff #218623)

Now that the vector is auto-resized, do you think it's still useful to expose the getUncompressedSize function as public api? If yes, then please change it to return Expected<uint64_t>. (returning Expected would be nice even for a private api, but there it's less important...)

lldb/lit/Breakpoint/Inputs/minidebuginfo-lib.h
1 ↗(On Diff #218623)

Hmm... so, what is the exact scenario you want to test here? This function will end up in the dynsym section of the main executable, but it will be an *undefined* symbol there. Undefined symbols are ignored in the gnu_debugdata logic. The symbol will be *defined* in the dynsym section of the shared library, but you're not debugdata-ing that one.

While this scenario might be interesting to test, I am not sure if that's the one you actually intended...

PS: If you want to create a file with both dynsym, and non-dynsym defined symbols, you don't have to resort to using shared libraries to achieve that. A suitable combination of -rdynamic and visibility("hidden") will achieve that too. For example the following line:

clang -x c - <<<'__attribute__((visibility("hidden"))) int in_symtab() { return 42; } int in_dynsym() { return 47; } int main() { return in_symtab()+in_dynsym(); }' -rdynamic

will generate a file where the in_symtab function will only be present in the symtab, and in_dynsym will be in the dynsym section (until you apply the debug-data magic, it will be in the symtab too)...

lldb/lit/Breakpoint/minidebuginfo.test
32 ↗(On Diff #218623)

Shouldn't this be %t.keep_symbols? Typo ?

lldb/lit/lit.cfg.py
102 ↗(On Diff #218623)

you should apply llvm_canonicalize_cmake_booleans to the value of LLDB_ENABLE_LZMA at the cmake level. Otherwise, this will blow up if someone types -DLLDB_ENABLE_LZMA=On

lldb/source/Host/common/LZMA.cpp
72–82 ↗(On Diff #218618)

too much auto

lldb/source/Plugins/ObjectFile/ELF/ObjectFileELF.cpp
1889

You don't need the temporary ArrayRef. You can just pass data.GetData() directly..

kwk updated this revision to Diff 218929.Sep 5 2019, 8:34 AM
  • Fixup
  • Test for minidebuginfo image dump symtab
  • Fixed test for corrupted xz blob
  • Fix test
  • Added test that runs when no LZMA is configured
Herald added a reviewer: shafik. · View Herald Transcript
Herald added projects: Restricted Project, Restricted Project, Restricted Project, Restricted Project. · View Herald Transcript
Herald added subscribers: openmp-commits, libcxx-commits, Restricted Project and 23 others. · View Herald Transcript
kwk updated this revision to Diff 218930.Sep 5 2019, 8:36 AM
  • Fixup
  • Test for minidebuginfo image dump symtab
  • Fixed test for corrupted xz blob
  • Fix test
  • Added test that runs when no LZMA is configured
Harbormaster completed remote builds in B37791: Diff 218930.
kwk updated this revision to Diff 218931.Sep 5 2019, 8:39 AM
  • Better names for minidebuginfo files
kwk updated this revision to Diff 218932.Sep 5 2019, 8:40 AM
  • Better names for minidebuginfo files
Harbormaster completed remote builds in B37793: Diff 218932.
kwk updated this revision to Diff 219038.Sep 6 2019, 12:43 AM
  • Fix spelling error
labath removed projects: Restricted Project, Restricted Project, Restricted Project, Restricted Project.
labath removed subscribers: arsenm, nemanjai, jvesely and 24 others.
labath added a comment.Sep 6 2019, 2:48 AM

Thank you for adding the tests, and for adding the xz detection to lit in particular. We're getting _reaally_ close now. I have a bunch more comments, including highlighting some you seems to have missed from earlier, but they are all just about polishing. The only somewhat larger comment is about the minidebuginfo-set-and-hit-breakpoint.test test. I'd like to understand what exactly is the scenario you are intending to test there, because I have a feeling it's not testing what you want it to test...

lldb/CMakeLists.txt
101 ↗(On Diff #218618)

What about this?

lldb/include/lldb/Host/LZMA.h
24–25 ↗(On Diff #218623)

What about this?

lldb/lit/Breakpoint/minidebuginfo-corrupt-xz.yaml
1 ↗(On Diff #219038)

These tests should go to lit/Modules/ELF, as they have nothing to do with breakpoints, and everything to do with ELF files. The minidebuginfo-set-and-hit-breakpoint.test might stay here as it actually does deal with breakpoints, but it would also make sense to put it next to the ELF tests. I'll leave it up to you to decide where to place that one.

13 ↗(On Diff #219038)

You shouldn't need an explicit -x in these tests as %lldb adds --no-use-colors already.

lldb/lit/Breakpoint/minidebuginfo-find-symbols.yaml
15 ↗(On Diff #219038)

Could you add a symbol or two to the dynsym section of the outer file? I'd like to test that we get symbols from .dynsym *and* .symtab sections.

lldb/lit/Breakpoint/minidebuginfo-no-lzma.yaml
22 ↗(On Diff #219038)

Nit: You most likely don't need such a gigantic blob to test this bit of functionality, as you can't read the data anyway...

lldb/lit/Breakpoint/minidebuginfo-set-and-hit-breakpoint.test
50 ↗(On Diff #219038)

Instead of messing with the environment, it might be better to just set the rpath (-Wl,-rpath) of the executable appropriately (but I would still like to know what you're exactly trying to test with that extra shared library).

lldb/lit/lit.cfg.py
102 ↗(On Diff #218623)

What about this?

lldb/source/Host/common/LZMA.cpp
72–82 ↗(On Diff #218618)

What about this? (I think both of the autos here are unnecessary).

lldb/source/Plugins/ObjectFile/ELF/ObjectFileELF.cpp
1853

I didn't notice this before, but you shouln't be calling module->GetSectionList() here. The unified_section_list you got as an argument *is* the module's section list.

1889

What about this?

labath marked an inline comment as done.Sep 6 2019, 4:55 AM
labath added inline comments.
lldb/lit/Breakpoint/minidebuginfo-corrupt-xz.yaml
13 ↗(On Diff #219038)

Actually, I take this back. It seems %lldb does not pass this automatically. I am not sure why, but since colors aren't a problem you here, you probably don't have to explicitly disable them anyway...

kwk updated this revision to Diff 219064.Sep 6 2019, 5:05 AM
kwk marked 16 inline comments as done.
  • Removed not needed test dependencies
  • Changed return type of lzma::getUncompressedSize
  • Moved minidebuginfo to subtree that deals with ELF
  • Apply review comments
kwk added a comment.Sep 6 2019, 5:05 AM

I've tackled the easy comments now and will come back to the more detailed ones as well. I'm not ignoring them.

lldb/CMakeLists.txt
101 ↗(On Diff #218618)

You're right. one place is enough.

kwk added inline comments.Sep 6 2019, 5:05 AM
lldb/include/lldb/Host/LZMA.h
24–25 ↗(On Diff #218623)

I've changed the syntax as you suggested. I kept the function there because I find it easier to maintain this way.

lldb/lit/Breakpoint/minidebuginfo-corrupt-xz.yaml
1 ↗(On Diff #219038)

I've move all to lit/Modules/ELF.

13 ↗(On Diff #219038)

I use -x and not -X. See lldb --help:

-x Alias for --no-lldbinit
-X Alias for --no-use-color

lldb/lit/Breakpoint/minidebuginfo-no-lzma.yaml
22 ↗(On Diff #219038)

I kept it the same for the like the other files so you can text-diff it to find out that only one byte was modified. I thought this is a neat idea in order to actually error out upon decompression and not on file header for example. That being said what do you think about adding tests for the different errors that can occur during the whole inspection (e.g. in getUncompressedSize).

lldb/source/Host/common/LZMA.cpp
81–82 ↗(On Diff #218300)

All done now.

lldb/source/Plugins/ObjectFile/ELF/ObjectFileELF.cpp
1853

Done.

1889

Sorry for missing this. Done now.

labath marked an inline comment as done.Sep 6 2019, 5:20 AM
labath added inline comments.
lldb/include/lldb/Host/LZMA.h
24–25 ↗(On Diff #218623)

Ok. I am fine with that.

lldb/lit/Breakpoint/minidebuginfo-corrupt-xz.yaml
13 ↗(On Diff #219038)

Ah, my bad. In that case you definitely *can* remove it because %lldb does add --no-lldbinit.

lldb/lit/Breakpoint/minidebuginfo-no-lzma.yaml
22 ↗(On Diff #219038)

Ok. That's fine with me.
I am definitely not *against* adding more tests. :) I am not going to ask you to put more in because this patch already has more tests than most lldb patches, but I you want to do that, then by all means, go ahead. Things like these would be probably better off as a (c++) unit test, as they're really testing the details of the lzma handling code. The elf code doesn't really care about these details -- it just wants to know whether the decompression was successful or not.

kwk marked an inline comment as done.Sep 9 2019, 3:47 AM

labath, please see my answer to your question and maybe help me out why you think I don't test what I want to test.

lldb/lit/Breakpoint/minidebuginfo-set-and-hit-breakpoint.test
50 ↗(On Diff #219038)

labath, I'd like to test the following scenario: Set a breakpoint on a symbol (multiplyByThree) in the .dynsym section and one on a symbol (multiplyByFour) from the .symtab that's embedded within the .gnu_debugdata section. I have no clue how to get those symbols without using a shared lib. Do you?

kwk marked an inline comment as done.Sep 9 2019, 7:56 AM
kwk added inline comments.
lldb/lit/Breakpoint/minidebuginfo-set-and-hit-breakpoint.test
50 ↗(On Diff #219038)

I found a better and simpler way thanks to the help from @jankratochvil . Will implement it soon.

MaskRay added inline comments.Sep 9 2019, 8:12 AM
lldb/source/Host/common/LZMA.cpp
124 ↗(On Diff #219038)

empty

129 ↗(On Diff #219038)

if (auto err = ...)

return err;
136 ↗(On Diff #219038)

is more common

138 ↗(On Diff #219038)

stray inpos = 0;

kwk updated this revision to Diff 219369.Sep 9 2019, 9:24 AM
  • Remove -x from %lldb because that expands to the command which already includes it
  • Improve and fix the minidebuginfo-set-and-hit-breakpoint.test
kwk added a comment.Sep 9 2019, 9:27 AM

@labath I have greatly improved the set and hit breakpoint test. Thank you again @jankratochvil for bringing my attention to -Wl,--dynamic-list. That helped to put one symbol in .dynsym explicitly. I will see if I can utilize that knowledge to separate out the patch this comment of yours @labath :

Let's put this bit into a separate patch. As I said over IRC, I view this bit as functionally independent of the debugdata stuff (it's definitely independent in it's current form, as it will kick in for non-debugdata files too, which I think is fine).

kwk updated this revision to Diff 219680.Sep 11 2019, 3:18 AM

Rebased on master to include changes from D67390

kwk marked 2 inline comments as done.Sep 11 2019, 3:18 AM
kwk added inline comments.
lldb/source/Plugins/ObjectFile/ELF/ObjectFileELF.cpp
2815–2836

Done in D67390.

labath accepted this revision.Sep 11 2019, 6:13 AM

This looks great now. Thank you for your patience. Just make sure to deal with the breakages from the other patch first. If the fix is non trivial, or if it's going to take a while, then please revert to unbreak the bots.

lldb/lit/Modules/ELF/minidebuginfo-corrupt-xz.yaml
1 ↗(On Diff #219680)

system-linux shouldn't be required here. One of the reasons I wanted to have these tests is that they can run on any system, not just on linux.

lldb/lit/Modules/ELF/minidebuginfo-find-symbols.yaml
1 ↗(On Diff #219680)

No system-linux

lldb/lit/Modules/ELF/minidebuginfo-no-lzma.yaml
1 ↗(On Diff #219680)

no system-linux

lldb/source/Host/common/LZMA.cpp
129 ↗(On Diff #219038)

I have a feeling this pattern is actually more common in lldb. I tend to use this one, but I am fine with either.

This revision is now accepted and ready to land.Sep 11 2019, 6:13 AM
kwk updated this revision to Diff 219848.Sep 11 2019, 10:32 PM
kwk marked 5 inline comments as done.
  • Addressed review comments
kwk added a comment.Sep 11 2019, 10:33 PM
This comment was removed by kwk.
lldb/source/Host/common/LZMA.cpp
129 ↗(On Diff #219038)

So what now? I had getCompressedSize return an error before and changed it upon request to expected. I can live with both but like the expected thing a bit better in terms of: this goes in and this comes out mentality.

kwk marked an inline comment as done.Sep 12 2019, 1:59 AM
kwk added inline comments.
lldb/lit/Modules/ELF/minidebuginfo-corrupt-xz.yaml
1 ↗(On Diff #219680)

@labath what about the other file minidebuginfo-set-and-hit-breakpoint.test. Remove system-linux from there as well?

labath added inline comments.Sep 15 2019, 11:48 PM
lldb/lit/Modules/ELF/minidebuginfo-corrupt-xz.yaml
1 ↗(On Diff #219680)

You need to restrict that test somehow, because it only has a chance for working on elf platforms (as you're compiling binaries for the host and running them). It probably *should* work on all elf platforms, but we currently don't have a good way of expressing that and we're just restricting to system-linux currently.

If you feel adventurous you can try introducing a system-elf feature (or something), and then changing this test (and possibly others) to use that instead, but please do that as a separate patch.

lldb/source/Host/common/LZMA.cpp
129 ↗(On Diff #219038)

My interpretation of @MaskRay's comment was that he wanted you do write something like

if (auto err = uncompressedSize.takeError())
  return err;

That is orthogonal to what type is returned by getCompressedSize, and is indeed a more common pattern in at least some parts of llvm. I am personally fine with both.

kwk updated this revision to Diff 222845.Oct 2 2019, 8:58 AM
kwk marked 12 inline comments as done.
  • Change logic and comment for when .dynsym is parsed
  • Use different pattern for error checking
  • use less auto
  • Remove required system-linux test feature from LZMA tests
kwk planned changes to this revision.Oct 2 2019, 8:58 AM
kwk added inline comments.
lldb/source/Plugins/ObjectFile/ELF/ObjectFileELF.cpp
2815–2836

We agreed to not have this other patch. See D67390#1691424.

kwk requested review of this revision.Oct 2 2019, 9:05 AM

Tests did pass so this change is ready for review @labath @jankratochvil .

This revision is now accepted and ready to land.Oct 2 2019, 9:05 AM
kwk edited the summary of this revision. (Show Details)Oct 2 2019, 9:39 AM
jankratochvil added inline comments.Oct 2 2019, 1:07 PM
lldb/source/Plugins/ObjectFile/ELF/ObjectFileELF.cpp
1857

I miss here comparing that the sections inside .gnu_debugdata file match sections in the outer file:

jankratochvil: "hm.... I was under the impression that section number should remain the same in regular and split debug files" - maybe you are right but then we could just also compare the two section tables and if they differ then reject whole .gnu_debugdata.  And then we do not have to compare section names but it is enough to compare section indexes.
labath: "right but then we could just also compare the two section tables and if they differ then reject whole .gnu_debugdata."
labath: I would like that ^

Also I do not see here handling .strtab, why does it work without moving also .strtab?

labath accepted this revision.Oct 3 2019, 5:25 AM

Ok, let's try this once more. I took another look at the patch, and I have some more tiny comments, but nothing too major.

I'm sorry this took so long. When the topic of parsing both symbol sections came up, I was happy that it was implemented in a way that applies outside of minidebuginfo too, because it seemed like it could be a useful source of additional information. However, that turned into a huge rabbit hole, which isn't really relevant for the thing you're trying to accomplish. If we've managed to come this far without needing to consult both symbol tables, I think we'll manage to wait a couple years longer.

lldb/lit/Modules/ELF/minidebuginfo-no-lzma.yaml
4 ↗(On Diff #222845)

s/a. /a ./

lldb/source/Host/common/LZMA.cpp
1 ↗(On Diff #222845)

Fix header.

69–72 ↗(On Diff #222845)

This is redundant, as you're comparing the size against LZMA_STREAM_HEADER_SIZE anyway.

84 ↗(On Diff #222845)

Maybe InputBuffer.take_back(LZMA_STREAM_HEADER_SIZE).data() ?

104–105 ↗(On Diff #222845)

same here.

125–128 ↗(On Diff #222845)

This is redundant, as getUncompressedSize checks that anyway.

137 ↗(On Diff #222845)

= UINT64_MAX;. This looks too much like a function declaration, and is not consistent with the code below.

lldb/source/Plugins/ObjectFile/ELF/ObjectFileELF.cpp
1857

I'd leave that out for now. This is similar stuff to what is already being done in SymbolVendorELF, which should also probably check for section matches, and would be best handled together.

The reason that this works without moving .strtab is because the parsing is still done within the context of the owning object file. This whole section merging business is a pretty big mess, and in dire need of some TLC, but @kwk is not making that worse, so I wouldn't want to deal with that here...

2825

You can add that this is because minidebuginfo normally removes the symtab symbols which have their matching dynsym counterparts.

kwk updated this revision to Diff 223381.Oct 5 2019, 12:30 PM
kwk marked 10 inline comments as done.
  • Fix typo: s/a. /a ./
  • Fix comment header
  • Remove redundant input buffer check
  • Remove another redundant input buffer check
  • Change variable initialization to assign
  • added comment about minidebuginfo
kwk added a comment.Oct 5 2019, 12:31 PM

@labath I've addressed all you comments. Thank you for answering @jankratochvil's question as well. If this patch is good to go now, please give a thumbs up.

lldb/source/Host/common/LZMA.cpp
84 ↗(On Diff #222845)

take_back:

Return a copy of *this with only the last \p N elements.

I mean LZMA_STREAM_HEADER_SIZE is only 12 but do we really need it?

104–105 ↗(On Diff #222845)

Same answer here.

kwk added a comment.Oct 5 2019, 12:32 PM

@labath ah, I've just seen that you did accept the revision already. Will merge it then after running tests once more.

kwk updated this revision to Diff 223382.Oct 5 2019, 12:38 PM

Rebased onto current master

kwk added a comment.Oct 5 2019, 2:03 PM

@labath as much as I would love to merge this patch I get errors from these tests that I didn't see before. They only appear after rebasing onto master:

Failing Tests (4):
    lldb-Suite :: commands/process/attach/TestProcessAttach.py
    lldb-Suite :: python_api/hello_world/TestHelloWorld.py
    lldb-Suite :: tools/lldb-server/libraries-svr4/TestGdbRemoteLibrariesSvr4Support.py
    lldb-Suite :: tools/lldb-vscode/attach/TestVSCode_attach.py

I also get the same errors on current master .(revision 8815be04ec1f333564591d9593735f22efa9bee5) without my patch applied. But before I merrge my patch I'd like to see master in a better shape, no?

In D66791#1696340, @kwk wrote:
lldb-Suite :: commands/process/attach/TestProcessAttach.py
lldb-Suite :: tools/lldb-vscode/attach/TestVSCode_attach.py

These are by: D68289

lldb-Suite :: python_api/hello_world/TestHelloWorld.py
lldb-Suite :: tools/lldb-server/libraries-svr4/TestGdbRemoteLibrariesSvr4Support.py

The Fedora buildbot shows different two ones:

lldb-Suite :: commands/gui/basic/TestGuiBasic.py
lldb-Suite :: python_api/hello_world/TestHelloWorld.py

Although I do not have those reproducible on my local Fedora 30 host.

kwk updated this revision to Diff 223482.Oct 7 2019, 3:46 AM

rebased

kwk closed this revision.Oct 7 2019, 3:49 AM

Closed by 2c082b48274fcba62bf9b3acb63075aedcc7a976.

labath added inline comments.Oct 7 2019, 8:19 AM
lldb/source/Host/common/LZMA.cpp
84 ↗(On Diff #222845)

Yeah, that documentation is somewhat misleading. If you look at the implementation you'll see that there's no copying involved (ArrayRefs never copy anything -- that's their whole point of existence). So, I still think that this is better/more readable with take_back.