[lldb][ELF] Read symbols from .gnu_debugdata sect.

[kwk]

Summary

If the .symtab section is stripped from the binary it might be that
there's a .gnu_debugdata section which contains a smaller .symtab in
order to provide enough information to create a backtrace with function
names or to set and hit a breakpoint on a function name.

This change looks for a .gnu_debugdata section in the ELF object file.
The .gnu_debugdata section contains a xz-compressed ELF file with a
.symtab section inside. Symbols from that compressed .symtab section
are merged with the main object file's .dynsym symbols (if any).
In addition we always load the .dynsym even if there's a .symtab
section.

For example, the Fedora and RHEL operating systems strip their binaries
but keep a .gnu_debugdata section. While gdb already can read this
section, LLDB until this patch couldn't. To test this patch on a
Fedora or RHEL operating system, try to set a breakpoint on the "help"
symbol in the "zip" binary. Before this patch, only GDB can set this
breakpoint; now LLDB also can do so without installing extra debug
symbols:

lldb /usr/bin/zip -b -o "b help" -o "r" -o "bt" -- -h

The above line runs LLDB in batch mode and on the "/usr/bin/zip -h"
target:

(lldb) target create "/usr/bin/zip"
Current executable set to '/usr/bin/zip' (x86_64).
(lldb) settings set -- target.run-args  "-h"

Before the program starts, we set a breakpoint on the "help" symbol:

(lldb) b help
Breakpoint 1: where = zip`help, address = 0x00000000004093b0

Once the program is run and has hit the breakpoint we ask for a
backtrace:

(lldb) r
Process 10073 stopped
* thread #1, name = 'zip', stop reason = breakpoint 1.1
    frame #0: 0x00000000004093b0 zip`help
zip`help:
->  0x4093b0 <+0>:  pushq  %r12
    0x4093b2 <+2>:  movq   0x2af5f(%rip), %rsi       ;  + 4056
    0x4093b9 <+9>:  movl   $0x1, %edi
    0x4093be <+14>: xorl   %eax, %eax

Process 10073 launched: '/usr/bin/zip' (x86_64)
(lldb) bt
* thread #1, name = 'zip', stop reason = breakpoint 1.1
  * frame #0: 0x00000000004093b0 zip`help
    frame #1: 0x0000000000403970 zip`main + 3248
    frame #2: 0x00007ffff7d8bf33 libc.so.6`__libc_start_main + 243
    frame #3: 0x0000000000408cee zip`_start + 46

In order to support the .gnu_debugdata section, one has to have LZMA
development headers installed. The CMake section, that controls this
part looks for the LZMA headers and enables .gnu_debugdata support by
default if they are found; otherwise or if explicitly requested, the
minidebuginfo support is disabled.

GDB supports the "mini debuginfo" section .gnu_debugdata since v7.6
(2013).

[kwk]

lldb/source/Plugins/SymbolVendor/ELF/SymbolVendorELF.cpp

79

Fixed in f509e547ab793068e8b822317b93060077623b74

llvm/CMakeLists.txt

53–361 ↗

(On Diff #217933)

Fixed in d4236447e75a882ab61bd369cb8253f2a908368f

[kwk]

• Moved and renamed m_gnu_debug_data_object_file

[kwk]

lldb/source/Plugins/ObjectFile/ELF/ObjectFileELF.h

394

Fixed in 0427898acf4a292f9737a33059e985239cc0e640

[labath]

lldb/cmake/modules/LLDBConfig.cmake

337 ↗

(On Diff #218300)

This looks like a really useful macro. I'll be sure to remember it.

338 ↗

(On Diff #218300)

It doesn't look like this is used anywhere.

341 ↗

(On Diff #218300)

Please put this into Config.h.cmake. I know a lot of code in this file does not do that, but that's because we didn't have a Config.h back then.

344 ↗

(On Diff #218300)

Is this code actually reachable? My understanding is that LLDB_ENABLE_LZMA should be automatically false if LIBLZMA_FOUND is not true...

lldb/source/Host/common/LZMA.cpp

81–82 ↗

(On Diff #218300)

printf format string here.

91 ↗

(On Diff #218300)

This is a valid printf string, but I am not sure if you intended it to come out like {lzma error: LZMA_MEMLIMIT_ERROR}

92 ↗

(On Diff #218300)

Calling .data() on a StringRef and expecting to get a c string is ill-advised. convertLZMACodeToString is a local static function, you could just make it return a const char *...

lldb/source/Plugins/ObjectFile/ELF/ObjectFileELF.cpp

1904

No, I think you should just delete this bit of code completely (as the containing object file will call SetModulesArchitecture on its own).

lldb/source/Plugins/ObjectFile/ELF/ObjectFileELF.h

157–164

This doesn't need to be public now, right?

lldb/source/Plugins/SymbolVendor/ELF/SymbolVendorELF.cpp

72

Using a unique_ptr is not a bad idea, but as right now this patch does not need to touch this file, it shouldn't be done as a part of this patch. Feel free to submit the unique_ptr thingy as a separate patch. No need to go through the review process...

lldb/test/CMakeLists.txt

119–121 ↗

(On Diff #218300)

I'd just make this unconditional, and add the tools to the big lldb-test-deps block.

[labath]

lldb/cmake/modules/LLDBConfig.cmake

341 ↗

(On Diff #218300)

(Among other things, this helps with incremental rebuilds, as only the files which include Config.h need to be recompiled)

[kwk]

• Remove left-over variable in CMake
• Move LLDB_ENABLE_LZMA into Config.h.cmake
• Remove unused code
• Change signature of lldb_private::lzma::uncompress
• Fixup
• Fixup
• When using the ObjectFile's own GetArchitecture(), no debug symbols are found
• Make GetGnuDebugDataObjectFile private
• Cleanup
• moved llvm-nm llvm-strip as test dependencies

[kwk]

@labath I did go over a lot of your comments. Thank you for helping me in becoming in a better contributor. I'm still not done with the python/make -> lit test transition but the rest is mostly covered.

lldb/cmake/modules/LLDBConfig.cmake

337 ↗

(On Diff #218300)

Make sure to include include(CMakeDependentOption)

338 ↗

(On Diff #218300)

Right, that was a left-over. Removed in 7dc045e67bb.

341 ↗

(On Diff #218300)

Addressed in a6277c07451

344 ↗

(On Diff #218300)

The logic is as follows:

LLDB_ENABLE_LZMA is ON if LIBLZMA_FOUND is ON.

Until now I though that can always go ahead and manually toggle on the option LLDB_ENABLE_LZMA by hand. And if you do this when LZMA is not available, then you get the fatal error. But as I understand [1] after reading it more than once, I think you're right: "This macro presents an option to the user only if a set of other conditions are true."

Let me verify this with a small example... yes, you're right. Even if you specify cmake . -DLLVM_ENABLE_LZMA it will not enable the option. Awesome! Fixed in 156f0af8c29980139ff1a2a8e51a768050564898

[1]: https://cmake.org/cmake/help/v3.0/module/CMakeDependentOption.html

lldb/source/Plugins/ObjectFile/ELF/ObjectFileELF.cpp

1904

Let me try if it works how you suggested it. No, it doesn't work as no breakpoint from the embedded .gnu_debugdata section will be hit anymore. This section was inspired by ObjectFileELF::CreateInstance as you suggested to look at once.

lldb/source/Plugins/ObjectFile/ELF/ObjectFileELF.h

157–164

Right, now that it is only called from CreateSections, it can be made private again. Fixed in bb1f3f0b5bd.

lldb/source/Plugins/SymbolVendor/ELF/SymbolVendorELF.cpp

72

It wasn't an idea but a necessity back when the whole minidebuginfo stuff was here. It still is a left-over that I have remove this change.

lldb/test/CMakeLists.txt

119–121 ↗

(On Diff #218300)

Done 1d731e225a243bb2afb66eb1f06cd010ee094208

[labath]

lldb/source/Plugins/ObjectFile/ELF/ObjectFileELF.cpp

1904

I'd be very surprised if this makes a difference as calling SetModulesArchitecture just invokes Module::SetArchitecture, which is actually a no-op if the module architecture has already been set.

However, I see now that SetArchitecture also acts as a check, which ensures that the architecture of the (nested) object file is compatible with the module architecture. This bit sounds like a useful check against someone accidentally (however unlikely) embedding a different object file into the gnu_debugdata section, so I guess we can keep that.

[kwk]

• add lit test for minidebuginfo
• Fixup
• test
• Fiddle with the lit test
• Fix binary in lit test
• Working lit test - still need to only run it when minidebuginfo is available (LZMA)
• Make FileCheck less verbose
• Only run minidebuginfo lit test when lzma is available
• fix mylib.so not found problem
• Removed old python based minidebuginfo test
• Remove changes that were only necessary for the old python based test for minidebuginfo

[kwk]

• remove not needed <vector> include

[kwk]

squashed and rebased on top of master

[labath]

I think we're getting pretty close. I have a couple of extra comments inline, but they're all pretty minor, I hope. I'd just like to add some more tests, per our discussion on the IRC, and I have some doubts about what exactly is the test you've written actually testing, which I'd like to clarify...

lldb/CMakeLists.txt

101 ↗

(On Diff #218618)

It looks like you're already adding these in lldb/lit/CMakeLists.txt. Does it need to be in both?

lldb/include/lldb/Host/LZMA.h

24–25 ↗

(On Diff #218623)

Now that the vector is auto-resized, do you think it's still useful to expose the getUncompressedSize function as public api? If yes, then please change it to return Expected<uint64_t>. (returning Expected would be nice even for a private api, but there it's less important...)

lldb/lit/Breakpoint/Inputs/minidebuginfo-lib.h

1 ↗

(On Diff #218623)

Hmm... so, what is the exact scenario you want to test here? This function will end up in the dynsym section of the main executable, but it will be an *undefined* symbol there. Undefined symbols are ignored in the gnu_debugdata logic. The symbol will be *defined* in the dynsym section of the shared library, but you're not debugdata-ing that one.

While this scenario might be interesting to test, I am not sure if that's the one you actually intended...

PS: If you want to create a file with both dynsym, and non-dynsym defined symbols, you don't have to resort to using shared libraries to achieve that. A suitable combination of -rdynamic and visibility("hidden") will achieve that too. For example the following line:

clang -x c - <<<'__attribute__((visibility("hidden"))) int in_symtab() { return 42; } int in_dynsym() { return 47; } int main() { return in_symtab()+in_dynsym(); }' -rdynamic

will generate a file where the in_symtab function will only be present in the symtab, and in_dynsym will be in the dynsym section (until you apply the debug-data magic, it will be in the symtab too)...

lldb/lit/Breakpoint/minidebuginfo.test

32 ↗

(On Diff #218623)

Shouldn't this be %t.keep_symbols? Typo ?

lldb/lit/lit.cfg.py

102 ↗

(On Diff #218623)

you should apply llvm_canonicalize_cmake_booleans to the value of LLDB_ENABLE_LZMA at the cmake level. Otherwise, this will blow up if someone types -DLLDB_ENABLE_LZMA=On

lldb/source/Host/common/LZMA.cpp

72–82 ↗

(On Diff #218618)

too much auto

lldb/source/Plugins/ObjectFile/ELF/ObjectFileELF.cpp

1889

You don't need the temporary ArrayRef. You can just pass data.GetData() directly..

[kwk]

• Fixup
• Test for minidebuginfo image dump symtab
• Fixed test for corrupted xz blob
• Fix test
• Added test that runs when no LZMA is configured

[kwk]

• Fixup
• Test for minidebuginfo image dump symtab
• Fixed test for corrupted xz blob
• Fix test
• Added test that runs when no LZMA is configured

[kwk]

• Better names for minidebuginfo files

[kwk]

• Better names for minidebuginfo files

[kwk]

• Fix spelling error

[labath]

Thank you for adding the tests, and for adding the xz detection to lit in particular. We're getting _reaally_ close now. I have a bunch more comments, including highlighting some you seems to have missed from earlier, but they are all just about polishing. The only somewhat larger comment is about the minidebuginfo-set-and-hit-breakpoint.test test. I'd like to understand what exactly is the scenario you are intending to test there, because I have a feeling it's not testing what you want it to test...

lldb/CMakeLists.txt

101 ↗

(On Diff #218618)

What about this?

lldb/include/lldb/Host/LZMA.h

24–25 ↗

(On Diff #218623)

What about this?

lldb/lit/Breakpoint/minidebuginfo-corrupt-xz.yaml

1 ↗

(On Diff #219038)

These tests should go to lit/Modules/ELF, as they have nothing to do with breakpoints, and everything to do with ELF files. The minidebuginfo-set-and-hit-breakpoint.test might stay here as it actually does deal with breakpoints, but it would also make sense to put it next to the ELF tests. I'll leave it up to you to decide where to place that one.

13 ↗

(On Diff #219038)

You shouldn't need an explicit -x in these tests as %lldb adds --no-use-colors already.

lldb/lit/Breakpoint/minidebuginfo-find-symbols.yaml

15 ↗

(On Diff #219038)

Could you add a symbol or two to the dynsym section of the outer file? I'd like to test that we get symbols from .dynsym *and* .symtab sections.

lldb/lit/Breakpoint/minidebuginfo-no-lzma.yaml

22 ↗

(On Diff #219038)

Nit: You most likely don't need such a gigantic blob to test this bit of functionality, as you can't read the data anyway...

lldb/lit/Breakpoint/minidebuginfo-set-and-hit-breakpoint.test

50 ↗

(On Diff #219038)

Instead of messing with the environment, it might be better to just set the rpath (-Wl,-rpath) of the executable appropriately (but I would still like to know what you're exactly trying to test with that extra shared library).

lldb/lit/lit.cfg.py

102 ↗

(On Diff #218623)

What about this?

lldb/source/Host/common/LZMA.cpp

72–82 ↗

(On Diff #218618)

What about this? (I think both of the autos here are unnecessary).

lldb/source/Plugins/ObjectFile/ELF/ObjectFileELF.cpp

1853

I didn't notice this before, but you shouln't be calling module->GetSectionList() here. The unified_section_list you got as an argument *is* the module's section list.

1889

What about this?

[labath]

lldb/lit/Breakpoint/minidebuginfo-corrupt-xz.yaml

13 ↗

(On Diff #219038)

Actually, I take this back. It seems %lldb does not pass this automatically. I am not sure why, but since colors aren't a problem you here, you probably don't have to explicitly disable them anyway...

[kwk]

• Removed not needed test dependencies
• Changed return type of lzma::getUncompressedSize
• Moved minidebuginfo to subtree that deals with ELF
• Apply review comments

[kwk]

I've tackled the easy comments now and will come back to the more detailed ones as well. I'm not ignoring them.

lldb/CMakeLists.txt

101 ↗

(On Diff #218618)

You're right. one place is enough.

[kwk]

lldb/include/lldb/Host/LZMA.h

24–25 ↗

(On Diff #218623)

I've changed the syntax as you suggested. I kept the function there because I find it easier to maintain this way.

lldb/lit/Breakpoint/minidebuginfo-corrupt-xz.yaml

1 ↗

(On Diff #219038)

I've move all to lit/Modules/ELF.

13 ↗

(On Diff #219038)

I use -x and not -X. See lldb --help:

-x Alias for --no-lldbinit
-X Alias for --no-use-color

lldb/lit/Breakpoint/minidebuginfo-no-lzma.yaml

22 ↗

(On Diff #219038)

I kept it the same for the like the other files so you can text-diff it to find out that only one byte was modified. I thought this is a neat idea in order to actually error out upon decompression and not on file header for example. That being said what do you think about adding tests for the different errors that can occur during the whole inspection (e.g. in getUncompressedSize).

lldb/source/Host/common/LZMA.cpp

81–82 ↗

(On Diff #218300)

All done now.

lldb/source/Plugins/ObjectFile/ELF/ObjectFileELF.cpp

1853

Done.

1889

Sorry for missing this. Done now.

[labath]

lldb/include/lldb/Host/LZMA.h

24–25 ↗

(On Diff #218623)

Ok. I am fine with that.

lldb/lit/Breakpoint/minidebuginfo-corrupt-xz.yaml

13 ↗

(On Diff #219038)

Ah, my bad. In that case you definitely *can* remove it because %lldb does add --no-lldbinit.

lldb/lit/Breakpoint/minidebuginfo-no-lzma.yaml

22 ↗

(On Diff #219038)

Ok. That's fine with me.
I am definitely not *against* adding more tests. :) I am not going to ask you to put more in because this patch already has more tests than most lldb patches, but I you want to do that, then by all means, go ahead. Things like these would be probably better off as a (c++) unit test, as they're really testing the details of the lzma handling code. The elf code doesn't really care about these details -- it just wants to know whether the decompression was successful or not.

[kwk]

labath, please see my answer to your question and maybe help me out why you think I don't test what I want to test.

lldb/lit/Breakpoint/minidebuginfo-set-and-hit-breakpoint.test

50 ↗

(On Diff #219038)

labath, I'd like to test the following scenario: Set a breakpoint on a symbol (multiplyByThree) in the .dynsym section and one on a symbol (multiplyByFour) from the .symtab that's embedded within the .gnu_debugdata section. I have no clue how to get those symbols without using a shared lib. Do you?

[kwk]

lldb/lit/Breakpoint/minidebuginfo-set-and-hit-breakpoint.test

50 ↗

(On Diff #219038)

I found a better and simpler way thanks to the help from @jankratochvil . Will implement it soon.

[MaskRay]

lldb/source/Host/common/LZMA.cpp

124 ↗

(On Diff #219038)

empty

129 ↗

(On Diff #219038)

if (auto err = ...)

return err;

136 ↗

(On Diff #219038)

is more common

138 ↗

(On Diff #219038)

stray inpos = 0;

[kwk]

• Remove -x from %lldb because that expands to the command which already includes it
• Improve and fix the minidebuginfo-set-and-hit-breakpoint.test

[kwk]

@labath I have greatly improved the set and hit breakpoint test. Thank you again @jankratochvil for bringing my attention to -Wl,--dynamic-list. That helped to put one symbol in .dynsym explicitly. I will see if I can utilize that knowledge to separate out the patch this comment of yours @labath :

Let's put this bit into a separate patch. As I said over IRC, I view this bit as functionally independent of the debugdata stuff (it's definitely independent in it's current form, as it will kick in for non-debugdata files too, which I think is fine).

[kwk]

Rebased on master to include changes from D67390

[kwk]

lldb/source/Plugins/ObjectFile/ELF/ObjectFileELF.cpp

2815–2836

Done in D67390.

[labath]

This looks great now. Thank you for your patience. Just make sure to deal with the breakages from the other patch first. If the fix is non trivial, or if it's going to take a while, then please revert to unbreak the bots.

lldb/lit/Modules/ELF/minidebuginfo-corrupt-xz.yaml

1 ↗

(On Diff #219680)

system-linux shouldn't be required here. One of the reasons I wanted to have these tests is that they can run on any system, not just on linux.

lldb/lit/Modules/ELF/minidebuginfo-find-symbols.yaml

1 ↗

(On Diff #219680)

No system-linux

lldb/lit/Modules/ELF/minidebuginfo-no-lzma.yaml

1 ↗

(On Diff #219680)

no system-linux

lldb/source/Host/common/LZMA.cpp

129 ↗

(On Diff #219038)

I have a feeling this pattern is actually more common in lldb. I tend to use this one, but I am fine with either.

[kwk]

• Addressed review comments

[kwk]

This comment was removed by kwk.

lldb/source/Host/common/LZMA.cpp

129 ↗

(On Diff #219038)

So what now? I had getCompressedSize return an error before and changed it upon request to expected. I can live with both but like the expected thing a bit better in terms of: this goes in and this comes out mentality.

[kwk]

lldb/lit/Modules/ELF/minidebuginfo-corrupt-xz.yaml

1 ↗

(On Diff #219680)

@labath what about the other file minidebuginfo-set-and-hit-breakpoint.test. Remove system-linux from there as well?

[labath]

lldb/lit/Modules/ELF/minidebuginfo-corrupt-xz.yaml

1 ↗

(On Diff #219680)

You need to restrict that test somehow, because it only has a chance for working on elf platforms (as you're compiling binaries for the host and running them). It probably *should* work on all elf platforms, but we currently don't have a good way of expressing that and we're just restricting to system-linux currently.

If you feel adventurous you can try introducing a system-elf feature (or something), and then changing this test (and possibly others) to use that instead, but please do that as a separate patch.

lldb/source/Host/common/LZMA.cpp

129 ↗

(On Diff #219038)

My interpretation of @MaskRay's comment was that he wanted you do write something like

if (auto err = uncompressedSize.takeError())
  return err;

That is orthogonal to what type is returned by getCompressedSize, and is indeed a more common pattern in at least some parts of llvm. I am personally fine with both.

[kwk]

• Change logic and comment for when .dynsym is parsed
• Use different pattern for error checking
• use less auto
• Remove required system-linux test feature from LZMA tests

[kwk]

lldb/source/Plugins/ObjectFile/ELF/ObjectFileELF.cpp

2815–2836

We agreed to not have this other patch. See D67390#1691424.

[kwk]

Tests did pass so this change is ready for review @labath @jankratochvil .

[jankratochvil]

lldb/source/Plugins/ObjectFile/ELF/ObjectFileELF.cpp

1857

I miss here comparing that the sections inside .gnu_debugdata file match sections in the outer file:

jankratochvil: "hm.... I was under the impression that section number should remain the same in regular and split debug files" - maybe you are right but then we could just also compare the two section tables and if they differ then reject whole .gnu_debugdata.  And then we do not have to compare section names but it is enough to compare section indexes.
labath: "right but then we could just also compare the two section tables and if they differ then reject whole .gnu_debugdata."
labath: I would like that ^

Also I do not see here handling .strtab, why does it work without moving also .strtab?

[labath]

Ok, let's try this once more. I took another look at the patch, and I have some more tiny comments, but nothing too major.

I'm sorry this took so long. When the topic of parsing both symbol sections came up, I was happy that it was implemented in a way that applies outside of minidebuginfo too, because it seemed like it could be a useful source of additional information. However, that turned into a huge rabbit hole, which isn't really relevant for the thing you're trying to accomplish. If we've managed to come this far without needing to consult both symbol tables, I think we'll manage to wait a couple years longer.

lldb/lit/Modules/ELF/minidebuginfo-no-lzma.yaml

4 ↗

(On Diff #222845)

s/a. /a ./

lldb/source/Host/common/LZMA.cpp

1 ↗

(On Diff #222845)

Fix header.

69–72 ↗

(On Diff #222845)

This is redundant, as you're comparing the size against LZMA_STREAM_HEADER_SIZE anyway.

84 ↗

(On Diff #222845)

Maybe InputBuffer.take_back(LZMA_STREAM_HEADER_SIZE).data() ?

104–105 ↗

(On Diff #222845)

same here.

125–128 ↗

(On Diff #222845)

This is redundant, as getUncompressedSize checks that anyway.

137 ↗

(On Diff #222845)

= UINT64_MAX;. This looks too much like a function declaration, and is not consistent with the code below.

lldb/source/Plugins/ObjectFile/ELF/ObjectFileELF.cpp

1857

I'd leave that out for now. This is similar stuff to what is already being done in SymbolVendorELF, which should also probably check for section matches, and would be best handled together.

The reason that this works without moving .strtab is because the parsing is still done within the context of the owning object file. This whole section merging business is a pretty big mess, and in dire need of some TLC, but @kwk is not making that worse, so I wouldn't want to deal with that here...

2825

You can add that this is because minidebuginfo normally removes the symtab symbols which have their matching dynsym counterparts.

[kwk]

• Fix typo: s/a. /a ./
• Fix comment header
• Remove redundant input buffer check
• Remove another redundant input buffer check
• Change variable initialization to assign
• added comment about minidebuginfo

[kwk]

@labath I've addressed all you comments. Thank you for answering @jankratochvil's question as well. If this patch is good to go now, please give a thumbs up.

lldb/source/Host/common/LZMA.cpp

84 ↗

(On Diff #222845)

take_back:

Return a copy of *this with only the last \p N elements.

I mean LZMA_STREAM_HEADER_SIZE is only 12 but do we really need it?

104–105 ↗

(On Diff #222845)

Same answer here.

[kwk]

@labath ah, I've just seen that you did accept the revision already. Will merge it then after running tests once more.

[kwk]

Rebased onto current master

[kwk]

@labath as much as I would love to merge this patch I get errors from these tests that I didn't see before. They only appear after rebasing onto master:

Failing Tests (4):
    lldb-Suite :: commands/process/attach/TestProcessAttach.py
    lldb-Suite :: python_api/hello_world/TestHelloWorld.py
    lldb-Suite :: tools/lldb-server/libraries-svr4/TestGdbRemoteLibrariesSvr4Support.py
    lldb-Suite :: tools/lldb-vscode/attach/TestVSCode_attach.py

I also get the same errors on current master .(revision 8815be04ec1f333564591d9593735f22efa9bee5) without my patch applied. But before I merrge my patch I'd like to see master in a better shape, no?

[jankratochvil]

In D66791#1696340, @kwk wrote:

lldb-Suite :: commands/process/attach/TestProcessAttach.py
lldb-Suite :: tools/lldb-vscode/attach/TestVSCode_attach.py

These are by: D68289

lldb-Suite :: python_api/hello_world/TestHelloWorld.py
lldb-Suite :: tools/lldb-server/libraries-svr4/TestGdbRemoteLibrariesSvr4Support.py

The Fedora buildbot shows different two ones:

lldb-Suite :: commands/gui/basic/TestGuiBasic.py
lldb-Suite :: python_api/hello_world/TestHelloWorld.py

Although I do not have those reproducible on my local Fedora 30 host.

[kwk]

rebased

[kwk]

Closed by 2c082b48274fcba62bf9b3acb63075aedcc7a976.

[labath]

lldb/source/Host/common/LZMA.cpp

84 ↗

(On Diff #222845)

Yeah, that documentation is somewhat misleading. If you look at the implementation you'll see that there's no copying involved (ArrayRefs never copy anything -- that's their whole point of existence). So, I still think that this is better/more readable with take_back.

lldb/packages/Python/lldbsuite/test/linux/minidebuginfo/Makefile

+LEVEL = ../../make
+C_SOURCES := main.c
+
+all: clean binary
+
+.PHONY: binary
+binary: a.out
+  $(RM) -r $@
+  cp a.out binary
+  
+  # Extract the dynamic symbols from the main binary, there is no need
+  # to also have these in the normal symbol table.
+  llvm-nm -D binary --format=posix --defined-only \
+    | awk '{ print $$1 }' | sort > dynsyms
+  
+  # Extract all the text (i.e. function) symbols from the debuginfo.
+  # (Note that we actually also accept "D" symbols, for the benefit
+  # of platforms like PowerPC64 that use function descriptors.)
+  llvm-nm binary --format=posix --defined-only \
+    | awk '{ if ($$2 == "T" || $$2 == "t" || $$2 == "D") print $$1 }' \
+    | sort > funcsyms
+  
+  # Keep all the function symbols not already in the dynamic symbol
+  # table.
+  comm -13 dynsyms funcsyms > keep_symbols
+  
+  # Separate full debug info into debug binary.
+  $(OBJCOPY) --only-keep-debug binary debug
+  
+  # Copy the full debuginfo, keeping only a minimal set of symbols and
+  # removing some unnecessary sections.
+  $(OBJCOPY) -S --remove-section .gdb_index --remove-section .comment \
+    --keep-symbols=keep_symbols debug mini_debuginfo
+  
+  # Drop the full debug info from the original binary.
+  llvm-strip --strip-all -R .comment binary
+  
+  # Inject the compressed data into the .gnu_debugdata section of the
+  # original binary.
+  xz --keep mini_debuginfo
+  $(OBJCOPY) --add-section .gnu_debugdata=mini_debuginfo.xz binary
+  
+clean::
+  $(RM) -r mini_debuginfo \
+    mini_debuginfo.xz \
+    binary \
+    debug \
+    keep_symbols \
+    funcsyms \
+    dynsyms
+
+include $(LEVEL)/Makefile.rules

lldb/packages/Python/lldbsuite/test/linux/minidebuginfo/TestMiniDebugInfo.py

+""" Testing debugging of a binary with "mini debuginfo" in .gnu_debugdata section. """
+import lldb
+from lldbsuite.test.decorators import *
+from lldbsuite.test.lldbtest import *
+from lldbsuite.test import lldbutil
+
+
+class TestMinidebugInfo(TestBase):
+    mydir = TestBase.compute_mydir(__file__)
+
+    def setUp(self):
+        TestBase.setUp(self)
+
+    @no_debug_info_test  # Prevent the genaration of the dwarf version of this test
+    @add_test_categories(["dwo"])
+    @skipUnlessPlatform(["linux"])
+    def test_mini_debug_info(self):
+        """Test that we can set and hit a breakpoint on a symbol from .gnu_debugdata."""
+
+        self.build()
+        exe = self.getBuildArtifact("binary")
+    
+        self.target = self.dbg.CreateTarget(exe)
+        self.assertTrue(self.target, VALID_TARGET)
+
+        main_bp = self.target.BreakpointCreateByName("multiplyByThree", "binary")
+        self.assertTrue(main_bp, VALID_BREAKPOINT)
+
+        self.process = self.target.LaunchSimple(
+            None, None, self.get_process_working_directory())
+        self.assertTrue(self.process, PROCESS_IS_VALID)
+
+        # The stop reason of the thread should be breakpoint.
+        self.assertTrue(self.process.GetState() == lldb.eStateStopped,
+                        STOPPED_DUE_TO_BREAKPOINT)
+

lldb/packages/Python/lldbsuite/test/linux/minidebuginfo/main.c

+int multiplyByThree(int num) { return num * 3; }
+
+int main(int argc, char *argv[]) {
+  (void)argv;
+  return multiplyByThree(argc);
+}

lldb/source/Plugins/ObjectFile/ELF/CMakeLists.txt

+include(CMakeDependentOption)
+set(minidebuginfo_libs)
+find_package(LibLZMA)
+cmake_dependent_option(LLDB_ENABLE_ELF_LZMA "Compile LLDB with LZMA" ON "LIBLZMA_FOUND" OFF)
+if (LLDB_ENABLE_ELF_LZMA)
+    if (LIBLZMA_FOUND)
+        add_definitions(-DLLDB_ENABLE_ELF_MINIDEBUGINFO)
+        set(minidebuginfo_libs ${LIBLZMA_LIBRARIES})
+        include_directories(${LIBLZMA_INCLUDE_DIRS})
+    else()
+        message(FATAL_ERROR "LZMA is required when LLDB_ENABLE_ELF_LZMA is On.")
+    endif()
+endif()
+
 add_lldb_library(lldbPluginObjectFileELF PLUGIN
   ELFHeader.cpp
   ObjectFileELF.cpp
 
   LINK_LIBS
     lldbCore
     lldbHost
     lldbSymbol
     lldbTarget
+    ${minidebuginfo_libs}
   LINK_COMPONENTS
     BinaryFormat
     Object
     Support
   )

lldb/source/Plugins/ObjectFile/ELF/ObjectFileELF.h

public:
   llvm::ArrayRef<elf::ELFProgramHeader> ProgramHeaders();
   lldb_private::DataExtractor GetSegmentData(const elf::ELFProgramHeader &H);
 
   llvm::StringRef
   StripLinkerSymbolAnnotations(llvm::StringRef symbol_name) const override;
 
   void RelocateSection(lldb_private::Section *section) override;
 
+  /// Takes the .gnu_debugdata and returns the decompressed object file that is
+  /// stored within that section.
+  ///
+  /// \returns either the decompressed object file stored within the
+  /// .gnu_debugdata section or \c nullptr if an error occured.
+  std::shared_ptr<ObjectFileELF> GetGnuDebugDataObjectFile();
+
 protected:
 
   std::vector<LoadableData>
   GetLoadableData(lldb_private::Target &target) override;
 
 private:
   ObjectFileELF(const lldb::ModuleSP &module_sp, lldb::DataBufferSP &data_sp,
                 lldb::offset_t data_offset, const lldb_private::FileSpec *file,
                 lldb::offset_t offset, lldb::offset_t length);
private:
   unsigned PLTRelocationType();
 
   static lldb_private::Status
   RefineModuleDetailsFromNote(lldb_private::DataExtractor &data,
                               lldb_private::ArchSpec &arch_spec,
                               lldb_private::UUID &uuid);
 
   bool AnySegmentHasPhysicalAddress();
+
+  std::shared_ptr<ObjectFileELF> m_gnuDebugDataObjectFile;
 };
 
 #endif // liblldb_ObjectFileELF_h_

lldb/source/Plugins/ObjectFile/ELF/ObjectFileELF.cpp

 //===-- ObjectFileELF.cpp ------------------------------------- -*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "ObjectFileELF.h"
 
 #include <algorithm>
 #include <cassert>
 #include <unordered_map>
 
+#ifdef LLDB_ENABLE_ELF_MINIDEBUGINFO
+#include <lzma.h>
+#endif // LLDB_ENABLE_ELF_MINIDEBUGINFO
+
 #include "lldb/Core/FileSpecList.h"
 #include "lldb/Core/Module.h"
 #include "lldb/Core/ModuleSpec.h"
 #include "lldb/Core/PluginManager.h"
 #include "lldb/Core/Section.h"
 #include "lldb/Host/FileSystem.h"
 #include "lldb/Symbol/DWARFCallFrameInfo.h"
 #include "lldb/Symbol/SymbolContext.h"
void ObjectFileELF::CreateSections(SectionList &unified_section_list) {
   }
 
   // For eTypeDebugInfo files, the Symbol Vendor will take care of updating the
   // unified section list.
   if (GetType() != eTypeDebugInfo)
     unified_section_list = *m_sections_up;
 }
 
+#ifdef LLDB_ENABLE_ELF_MINIDEBUGINFO
+static std::string lzma_ret_to_string(lzma_ret code) {
+  switch (code) {
+#define X(n)                                                                   \
+  case n:                                                                      \
+    return #n;
+    X(LZMA_OK)
+    X(LZMA_STREAM_END)
+    X(LZMA_NO_CHECK)
+    X(LZMA_UNSUPPORTED_CHECK)
+    X(LZMA_GET_CHECK)
+    X(LZMA_MEM_ERROR)
+    X(LZMA_MEMLIMIT_ERROR)
+    X(LZMA_FORMAT_ERROR)
+    X(LZMA_OPTIONS_ERROR)
+    X(LZMA_DATA_ERROR)
+    X(LZMA_BUF_ERROR)
+    X(LZMA_PROG_ERROR)
+#undef X
+  }
+  return "Unknown LZMA return code: " + std::to_string(code);
+}
+
+/// Decompresses the xz-compressed content at \a data_start of size \a
+/// data_size.
+///
+/// \param[in] data_start
+///     Pointer to the start of the data.
+///
+/// \param[in] data_size
+///     Size of the compressed data.
+///
+/// \returns a vector with the decompressed content or an error.
+llvm::Expected<std::vector<uint8_t>> decompressXZ(const uint8_t *data_start,
+                                                  uint64_t data_size) {
+  if (!data_start || data_size == 0) {
+    return llvm::createStringError(
+        llvm::inconvertibleErrorCode(),
+        "address (%p) or size (%lu) of xz-compressed blob cannot be 0",
+        data_start, data_size);
+  }
+  auto opts = lzma_stream_flags{};
+  if (data_size < LZMA_STREAM_HEADER_SIZE) {
+    return llvm::createStringError(
+        llvm::inconvertibleErrorCode(),
+        "size of xz-compressed blob ({0} bytes) is smaller than the "
+        "LZMA_STREAM_HEADER_SIZE ({1} bytes)",
+        data_size, LZMA_STREAM_HEADER_SIZE);
+  }
+
+  // Decode xz footer.
+  auto xzerr = lzma_stream_footer_decode(&opts, data_start + data_size -
+                                                    LZMA_STREAM_HEADER_SIZE);
+  if (xzerr != LZMA_OK) {
+    return llvm::createStringError(llvm::inconvertibleErrorCode(),
+                                   "lzma_stream_footer_decode()={%s}",
+                                   lzma_ret_to_string(xzerr).c_str());
+  }
+  if (data_size < (opts.backward_size + LZMA_STREAM_HEADER_SIZE)) {
+    return llvm::createStringError(
+        llvm::inconvertibleErrorCode(),
+        "xz-compressed buffer size ({%lu} bytes) too small (required at "
+        "least {%lu} bytes) ",
+        data_size, (opts.backward_size + LZMA_STREAM_HEADER_SIZE));
+  }
+
+  // Decode xz index.
+  lzma_index *xzindex;
+  uint64_t memlimit(UINT64_MAX);
+  size_t inpos = 0;
+  xzerr = lzma_index_buffer_decode(
+      &xzindex, &memlimit, nullptr,
+      data_start + data_size - LZMA_STREAM_HEADER_SIZE - opts.backward_size,
+      &inpos, data_size);
+  if (xzerr != LZMA_OK) {
+    return llvm::createStringError(llvm::inconvertibleErrorCode(),
+                                   "lzma_index_buffer_decode()={%s}",
+                                   lzma_ret_to_string(xzerr).c_str());
+  }
+
+  // Get size of uncompressed file to construct an in-memory buffer of the
+  // same size.
+  const auto uncompressed_file_size = lzma_index_uncompressed_size(xzindex);
+  std::vector<uint8_t> result(uncompressed_file_size);
+  // std::vector<uint8_t> decompressed_buf(uncompressed_file_size);
+  if (result.size() != uncompressed_file_size) {
+    if (xzindex != nullptr) {
+      lzma_index_end(xzindex, nullptr);
+      xzindex = nullptr;
+    }
+    return llvm::createStringError(
+        llvm::inconvertibleErrorCode(),
+        "failed to allocate decompression buffer of size {%lu} bytes",
+        uncompressed_file_size);
+  }
+
+  // Deallocate xz index as it is no longer needed.
+  lzma_index_end(xzindex, nullptr);
+
+  // Decompress xz buffer to buffer.
+  inpos = 0;
+  size_t outpos = 0;
+  xzerr = lzma_stream_buffer_decode(&memlimit, 0, nullptr, data_start, &inpos,
+                                    data_size, result.data(), &outpos,
+                                    result.size());
+  if (xzerr != LZMA_OK) {
+    return llvm::createStringError(llvm::inconvertibleErrorCode(),
+                                   "lzma_stream_buffer_decode()={%s}",
+                                   lzma_ret_to_string(xzerr).c_str());
+  }
+
+  return std::move(result);
+}
+#endif // LLDB_ENABLE_ELF_MINIDEBUGINFO
+
+std::shared_ptr<ObjectFileELF> ObjectFileELF::GetGnuDebugDataObjectFile() {
+  if (m_gnuDebugDataObjectFile != nullptr) {
+    return m_gnuDebugDataObjectFile;
+  }
+
+  SectionSP section =
+      GetSectionList()->FindSectionByName(ConstString(".gnu_debugdata"));
+  if (!section) {
+    return nullptr;
+  }
+
+#ifndef LLDB_ENABLE_ELF_MINIDEBUGINFO
+  GetModule()->ReportWarning(
+      "No LZMA support found for reading .gnu_debugdata section");
+  return nullptr;
+#else
+
+  auto data = DataExtractor();
+  section->GetSectionData(data);
+  auto res = decompressXZ(data.GetDataStart(), data.GetByteSize());
+  if (!res) {
+    GetModule()->ReportWarning(
+        "An error occured while decompression the section %s: %s",
+        section->GetName().AsCString(), Status(res.takeError()).AsCString());
+    return nullptr;
+  }
+
+  // Construct ObjectFileELF object from decompressed buffer
+  DataBufferSP gdd_data_buf(new DataBufferHeap(res->data(), res->size()));
+  auto fspec = GetFileSpec().CopyByAppendingPathComponent(
+      llvm::StringRef("gnu_debugdata"));
+  m_gnuDebugDataObjectFile.reset(new ObjectFileELF(
+      GetModule(), gdd_data_buf, 0, &fspec, 0, gdd_data_buf->GetByteSize()));
+
+  // This line is essential; otherwise a breakpoint can be set but not hit.
+  m_gnuDebugDataObjectFile->SetType(ObjectFile::eTypeDebugInfo);
+
+  ArchSpec spec = m_gnuDebugDataObjectFile->GetArchitecture();
+  if (spec && m_gnuDebugDataObjectFile->SetModulesArchitecture(spec)) {
+       return m_gnuDebugDataObjectFile;
+  }
+
+  return nullptr;
+#endif // LLDB_ENABLE_ELF_MINIDEBUGINFO
+}
+
 // Find the arm/aarch64 mapping symbol character in the given symbol name.
 // Mapping symbols have the form of "$<char>[.<any>]*". Additionally we
 // recognize cases when the mapping symbol prefixed by an arbitrary string
 // because if a symbol prefix added to each symbol in the object file with
 // objcopy then the mapping symbols are also prefixed.
 static char FindArmAarch64MappingSymbol(const char *symbol_name) {
   if (!symbol_name)
     return '\0';
if (m_symtab_up == nullptr) {
     uint64_t symbol_id = 0;
     std::lock_guard<std::recursive_mutex> guard(module_sp->GetMutex());
 
     // Sharable objects and dynamic executables usually have 2 distinct symbol
     // tables, one named ".symtab", and the other ".dynsym". The dynsym is a
     // smaller version of the symtab that only contains global symbols. The
     // information found in the dynsym is therefore also found in the symtab,
     // while the reverse is not necessarily true.
     Section *symtab =
         section_list->FindSectionByType(eSectionTypeELFSymbolTable, true).get();
-    if (!symtab) {
-      // The symtab section is non-allocable and can be stripped, so if it
-      // doesn't exist then use the dynsym section which should always be
-      // there.
-      symtab =
-          section_list->FindSectionByType(eSectionTypeELFDynamicSymbols, true)
-              .get();
+    if (symtab) {
+      m_symtab_up.reset(new Symtab(symtab->GetObjectFile()));
+      symbol_id += ParseSymbolTable(m_symtab_up.get(), symbol_id, symtab);
+    }
+
+    // there. If both, .symtab and .dynsym exist, we load both.
+    auto dynsym =
+    if (dynsym) {
+      if (!m_symtab_up) {
+        auto sec = symtab ? symtab : dynsym;
+        m_symtab_up.reset(new Symtab(sec->GetObjectFile()));
       }
-    if (symtab) {
-      m_symtab_up.reset(new Symtab(symtab->GetObjectFile()));
-      symbol_id += ParseSymbolTable(m_symtab_up.get(), symbol_id, symtab);
+      symbol_id += ParseSymbolTable(m_symtab_up.get(), symbol_id, dynsym);
     }
 
     // DT_JMPREL
     //      If present, this entry's d_ptr member holds the address of
     //      relocation
     //      entries associated solely with the procedure linkage table.
     //      Separating
     //      these relocation entries lets the dynamic linker ignore them during
     //      process initialization, if lazy binding is enabled. If this entry is
     //      present, the related entries of types DT_PLTRELSZ and DT_PLTREL must
     //      also be present.

lldb/source/Plugins/SymbolVendor/ELF/SymbolVendorELF.cpp

 // CreateInstance
 //
 // Platforms can register a callback to use when creating symbol vendors to
 // allow for complex debug information file setups, and to also allow for
 // finding separate debug information files.
 SymbolVendor *
 SymbolVendorELF::CreateInstance(const lldb::ModuleSP &module_sp,
                                 lldb_private::Stream *feedback_strm) {
+  (void)feedback_strm;
+
   if (!module_sp)
     return nullptr;
 
   ObjectFileELF *obj_file =
       llvm::dyn_cast_or_null<ObjectFileELF>(module_sp->GetObjectFile());
   if (!obj_file)
     return nullptr;
 
+  std::unique_ptr<SymbolVendorELF> symbol_vendor(
+      new SymbolVendorELF(module_sp));
+  SectionList *module_section_list = module_sp->GetSectionList();
+
+  // If there's a .gnu_debugdata section, we'll try to read the .symtab that's
+  // embedded in there and replace the one in the original object file (if any).
+  // If there's none in the orignal object file, we add it to it.
+  if (auto gdd_obj_file =
+          obj_file->GetGnuDebugDataObjectFile()) {
+    if (auto gdd_objfile_section_list = gdd_obj_file->GetSectionList()) {
+      if (SectionSP symtab_section_sp =
+              gdd_objfile_section_list->FindSectionByType(
+                  eSectionTypeELFSymbolTable, true)) {
+        SectionSP module_section_sp = module_section_list->FindSectionByType(
+            eSectionTypeELFSymbolTable, true);
+        if (module_section_sp)
+          module_section_list->ReplaceSection(module_section_sp->GetID(),
+                                              symtab_section_sp);
+        else
+          module_section_list->AddSection(symtab_section_sp);
+      }
+    }
+  }
+
   lldb_private::UUID uuid = obj_file->GetUUID();
   if (!uuid)
     return nullptr;
 
   // If the main object file already contains debug info, then we are done.
   if (obj_file->GetSectionList()->FindSectionByType(
           lldb::eSectionTypeDWARFDebugInfo, true))
     return nullptr;
SymbolVendorELF::CreateInstance(const lldb::ModuleSP &module_sp,
   if (!dsym_objfile_sp)
     return nullptr;
 
   // This objfile is for debugging purposes. Sadly, ObjectFileELF won't
   // be able to figure this out consistently as the symbol file may not
   // have stripped the code sections, etc.
   dsym_objfile_sp->SetType(ObjectFile::eTypeDebugInfo);
 
-  SymbolVendorELF *symbol_vendor = new SymbolVendorELF(module_sp);
-
   // Get the module unified section list and add our debug sections to
   // that.
-  SectionList *module_section_list = module_sp->GetSectionList();
+  module_section_list = module_sp->GetSectionList();
   SectionList *objfile_section_list = dsym_objfile_sp->GetSectionList();
 
   static const SectionType g_sections[] = {
       eSectionTypeDWARFDebugAbbrev,   eSectionTypeDWARFDebugAddr,
       eSectionTypeDWARFDebugAranges,  eSectionTypeDWARFDebugCuIndex,
       eSectionTypeDWARFDebugFrame,    eSectionTypeDWARFDebugInfo,
       eSectionTypeDWARFDebugLine,     eSectionTypeDWARFDebugLoc,
       eSectionTypeDWARFDebugMacInfo,  eSectionTypeDWARFDebugPubNames,
if (SectionSP section_sp =
         module_section_list->ReplaceSection(module_section_sp->GetID(),
                                             section_sp);
       else
         module_section_list->AddSection(section_sp);
     }
   }
 
   symbol_vendor->AddSymbolFileRepresentation(dsym_objfile_sp);
-  return symbol_vendor;
+
+  return symbol_vendor.release();
 }
 
 // PluginInterface protocol
 ConstString SymbolVendorELF::GetPluginName() { return GetPluginNameStatic(); }
 
 uint32_t SymbolVendorELF::GetPluginVersion() { return 1; }

llvm/include/llvm/ObjectYAML/ELFYAML.h

 struct Object {
   FileHeader Header;
   std::vector<ProgramHeader> ProgramHeaders;
   std::vector<std::unique_ptr<Section>> Sections;
   // Although in reality the symbols reside in a section, it is a lot
   // cleaner and nicer if we read them from the YAML as a separate
   // top-level key, which automatically ensures that invariants like there
   // being a single SHT_SYMTAB section are upheld.
-  std::vector<Symbol> Symbols;
+  Optional<std::vector<Symbol>> Symbols;
   std::vector<Symbol> DynamicSymbols;
+
+  bool HasSymbolsEntryInYAML() const { return Symbols.hasValue(); }
 };
 
 } // end namespace ELFYAML
 } // end namespace llvm
 
 LLVM_YAML_IS_SEQUENCE_VECTOR(llvm::ELFYAML::DynamicEntry)
 LLVM_YAML_IS_SEQUENCE_VECTOR(llvm::ELFYAML::ProgramHeader)
 LLVM_YAML_IS_SEQUENCE_VECTOR(std::unique_ptr<llvm::ELFYAML::Section>)

llvm/include/llvm/Support/MathExtras.h

 #undef R6
 };
 
 /// Reverse the bits in \p Val.
 template <typename T>
 T reverseBits(T Val) {
   unsigned char in[sizeof(Val)];
   unsigned char out[sizeof(Val)];
-  std::memcpy(in, &Val, sizeof(Val));
+  memcpy(in, &Val, sizeof(Val));
   for (unsigned i = 0; i < sizeof(Val); ++i)
     out[(sizeof(Val) - i) - 1] = BitReverseTable256[in[i]];
   std::memcpy(&Val, out, sizeof(Val));
   return Val;
 }
 
 // NOTE: The following support functions use the _32/_64 extensions instead of
 // type overloading so that signed and unsigned integers can be used without

llvm/lib/ObjectYAML/ELFEmitter.cpp

template <class ELFT> ELFState<ELFT>::ELFState(ELFYAML::Object &D) : Doc(D) {
 
   // Insert SHT_NULL section implicitly when it is not defined in YAML.
   if (Doc.Sections.empty() || Doc.Sections.front()->Type != ELF::SHT_NULL)
     Doc.Sections.insert(
         Doc.Sections.begin(),
         std::make_unique<ELFYAML::Section>(
             ELFYAML::Section::SectionKind::RawContent, /*IsImplicit=*/true));
 
-  std::vector<StringRef> ImplicitSections = {".symtab", ".strtab", ".shstrtab"};
+  std::vector<StringRef> ImplicitSections = {".strtab", ".shstrtab"};
+  // We only generate .symtab ELF section if there was a "Symbols" entry in the
+  // YAML file.
+  if (Doc.HasSymbolsEntryInYAML()) {
+    ImplicitSections.insert(ImplicitSections.begin(), {".symtab"});
+  }
   if (!Doc.DynamicSymbols.empty())
     ImplicitSections.insert(ImplicitSections.end(), {".dynsym", ".dynstr"});
 
   // Insert placeholders for implicit sections that are not
   // defined explicitly in YAML.
   for (StringRef SecName : ImplicitSections) {
     if (DocSections.count(SecName))
       continue;
 

 
 template <class ELFT>
 void ELFState<ELFT>::initSymtabSectionHeader(Elf_Shdr &SHeader,
                                              SymtabType STType,
                                              ContiguousBlobAccumulator &CBA,
                                              ELFYAML::Section *YAMLSec) {
 
   bool IsStatic = STType == SymtabType::Static;
-  const auto &Symbols = IsStatic ? Doc.Symbols : Doc.DynamicSymbols;
+  const auto &Symbols =
+      IsStatic && Doc.Symbols ? *Doc.Symbols : Doc.DynamicSymbols;
 
   ELFYAML::RawContentSection *RawSec =
       dyn_cast_or_null<ELFYAML::RawContentSection>(YAMLSec);
   if (RawSec && !Symbols.empty() && (RawSec->Content || RawSec->Size)) {
     if (RawSec->Content)
       WithColor::error() << "Cannot specify both `Content` and " +
                                 (IsStatic ? Twine("`Symbols`")
                                           : Twine("`DynamicSymbols`")) +
if (!Name.empty() && !Map.addName(Name, I)) {
       WithColor::error() << "Repeated symbol name: '" << Name << "'.\n";
       return false;
     }
   }
   return true;
 }
 
 template <class ELFT> bool ELFState<ELFT>::buildSymbolIndexes() {
-  return buildSymbolsMap(Doc.Symbols, SymN2I) &&
+  return Doc.Symbols && buildSymbolsMap(*Doc.Symbols, SymN2I) &&
          buildSymbolsMap(Doc.DynamicSymbols, DynSymN2I);
 }
 
 template <class ELFT> void ELFState<ELFT>::finalizeStrings() {
   // Add the regular symbol names to .strtab section.
-  for (const ELFYAML::Symbol &Sym : Doc.Symbols)
+  if (Doc.Symbols)
+    for (const ELFYAML::Symbol &Sym : *Doc.Symbols)
       DotStrtab.add(dropUniqueSuffix(Sym.Name));
   DotStrtab.finalize();
 
   // Add the dynamic symbol names to .dynstr section.
   for (const ELFYAML::Symbol &Sym : Doc.DynamicSymbols)
     DotDynstr.add(dropUniqueSuffix(Sym.Name));
 
   // SHT_GNU_verdef and SHT_GNU_verneed sections might also
   // add strings to .dynstr section.

llvm/lib/ObjectYAML/ELFYAML.cpp

if (Other) {
     MappingNormalization<NormalizedOther, uint8_t> Keys(IO, Symbol.Other);
     IO.mapOptional("Visibility", Keys->Visibility, ELFYAML::ELF_STV(0));
     IO.mapOptional("Other", Keys->Other, ELFYAML::ELF_STO(0));
   }
 }
 
 StringRef MappingTraits<ELFYAML::Symbol>::validate(IO &IO,
                                                    ELFYAML::Symbol &Symbol) {
+  (void)IO;
   if (Symbol.Index && Symbol.Section.data())
     return "Index and Section cannot both be specified for Symbol";
   if (Symbol.NameIndex && !Symbol.Name.empty())
     return "Name and NameIndex cannot both be specified for Symbol";
   return StringRef();
 }
 
 static void commonSectionMapping(IO &IO, ELFYAML::Section &Section) {

llvm/tools/obj2yaml/elf2yaml.cpp

if (!SymTab || SymTabShndx->sh_link != SymTab - Sections.begin())
           obj2yaml_error::not_implemented,
           "only SHT_SYMTAB_SHNDX associated with SHT_SYMTAB are supported");
 
     auto TableOrErr = Obj.getSHNDXTable(*SymTabShndx);
     if (!TableOrErr)
       return TableOrErr.takeError();
     ShndxTable = *TableOrErr;
   }
-  if (SymTab)
-    if (Error E = dumpSymbols(SymTab, Y->Symbols))
+  if (SymTab && Y->Symbols)
+    if (Error E = dumpSymbols(SymTab, *Y->Symbols))
       return std::move(E);
   if (DynSymTab)
     if (Error E = dumpSymbols(DynSymTab, Y->DynamicSymbols))
       return std::move(E);
 
   for (const Elf_Shdr &Sec : Sections) {
     switch (Sec.sh_type) {
     case ELF::SHT_DYNAMIC: {