This is an archive of the discontinued LLVM Phabricator instance.

[lldb] [Process] Add proper support for NetBSD core files with threads
ClosedPublic

Authored by mgorny on Apr 17 2017, 6:43 PM.

Details

Summary

Improve the support for processing NetBSD cores. Fix reading process
identifier, thread information and associating the terminating signal
with the correct thread.

Includes test cases for single-threaded program receiving SIGSEGV,
and two dual-threaded programs: one where thread receives the signal,
and the other one when the whole process is signalled.

Diff Detail

Event Timeline

krytarowski created this revision.Apr 17 2017, 6:43 PM

Demo:

http://netbsd.org/~kamil/lldb/firefox-core.typescript

Replay:

script -p ./firefox-core.typescript

BSD script(1) is incompatible with the GNU one, so I prepared a quick port to Linux:

http://netbsd.org/~kamil/lldb/nbscript.c

gcc nbscript.c -lutil -o nbscript

labath edited edge metadata.Apr 20 2017, 4:07 AM

A test would infinitely times more valuable then a demo script. What is the tiniest core file you can produce on NetBSD? (on linux we've gotten them down to about 20K) Then we could check that in and write a test for it...

A test would infinitely times more valuable then a demo script. What is the tiniest core file you can produce on NetBSD? (on linux we've gotten them down to about 20K) Then we could check that in and write a test for it...

This is something I wanted to bring to the dev mailing list.

I wanted to prepare at least three tests, if possible four:

  • one thread (if possible two variations: signal to one particular thread + signal to all threads)
  • multiple threads (signal to one particular thread + signal to all threads)

And this in combination of all supported targets (x86_64, i386, etc).

Emitting SIGABRT for such program gives core of size 97kilobytes:

int main(){for(;;);}

I will write assembly programs for the above cases, without libc.

kettenis edited edge metadata.Apr 20 2017, 4:35 AM

Generally looks reasonable to me. A few comments inline.

source/Plugins/ObjectFile/ELF/ObjectFileELF.cpp
1374 ↗(On Diff #95522)

By making this change you risk losing recognition of "normal" NetBSD ELF files, i.e. executables and shared libraries.

source/Plugins/Process/elf-core/ProcessElfCore.cpp
563 ↗(On Diff #95522)

A bit strange to call something both generic and specific...

I think some of these note types originated on SVR4/Solaris, but I'm not sure how much Linux deviated from that design through the years.

815 ↗(On Diff #95522)

Wouldn't it make sense to move the parsing of the LWP ID before the switch. Otherwise you'll have to duplicate the code for every NetBSD architecture you'll add.

A test would infinitely times more valuable then a demo script. What is the tiniest core file you can produce on NetBSD? (on linux we've gotten them down to about 20K) Then we could check that in and write a test for it...

The smalles core dumps I managed to create for OpenBSD so far are a bit over 4M.

krytarowski added inline comments.Apr 20 2017, 4:48 AM
source/Plugins/ObjectFile/ELF/ObjectFileELF.cpp
1374 ↗(On Diff #95522)

Hmm, do you mean these notes:

$ readelf -n /bin/cat                                                                                                                                 

Displaying notes found at file offset 0x00000214 with length 0x00000018:
  Owner                 Data size	Description
  NetBSD		0x00000004	IDENT 799005900 (7.99.59)

Displaying notes found at file offset 0x0000022c with length 0x00000014:
  Owner                 Data size	Description
  NetBSD		0x00000004	PaX <>
source/Plugins/Process/elf-core/ProcessElfCore.cpp
563 ↗(On Diff #95522)

Can we call it: Type1 core (as of now: NetBSD) and Type2 core (Linux, Android, FreeBSD, OpenBSD)?

I don't care which would be 1 and which 2.

815 ↗(On Diff #95522)

Sounds good.

kettenis added inline comments.Apr 20 2017, 5:25 AM
source/Plugins/ObjectFile/ELF/ObjectFileELF.cpp
1374 ↗(On Diff #95522)

In particular the first one of these. On OpenBSD I just have a generic check on the name which catches both these and the core file notes. But on NetBSD those use different names.

source/Plugins/Process/elf-core/ProcessElfCore.cpp
563 ↗(On Diff #95522)

I'd just drop word "specific" from the comment. And OpenBSD will obviously be handled in a similar way as NetBSD in the near future.

krytarowski added inline comments.Apr 20 2017, 5:47 AM
source/Plugins/ObjectFile/ELF/ObjectFileELF.cpp
1374 ↗(On Diff #95522)

I will test it and add switch to support both types of ELF files: core(5) and executables.

source/Plugins/Process/elf-core/ProcessElfCore.cpp
563 ↗(On Diff #95522)

Assuming that NetBSD and OpenBSD will share the same function - we can go pickup appropriate name now.

As far as I can tell FreeBSD shares similar concept to Linux.

A thread specific data is defined by multiplication of the following notes:

FreeBSD       0x000000e0      NT_PRSTATUS (prstatus structure)
FreeBSD       0x00000200      NT_FPREGSET (floating point registers)
FreeBSD       0x00000018      NT_THRMISC (thrmisc structure)

A test would infinitely times more valuable then a demo script. What is the tiniest core file you can produce on NetBSD? (on linux we've gotten them down to about 20K) Then we could check that in and write a test for it...

This is something I wanted to bring to the dev mailing list.

I wanted to prepare at least three tests, if possible four:

  • one thread (if possible two variations: signal to one particular thread + signal to all threads)
  • multiple threads (signal to one particular thread + signal to all threads)

And this in combination of all supported targets (x86_64, i386, etc).

Emitting SIGABRT for such program gives core of size 97kilobytes:

int main(){for(;;);}

I will write assembly programs for the above cases, without libc.

Cool, I am looking forward to the results.

source/Plugins/Process/elf-core/ProcessElfCore.cpp
795 ↗(On Diff #95522)

How about StringRef name = note.n_name;

803 ↗(On Diff #95522)

Then this can be

else if (name.consume_front("NetBSD-CORE@")) {
  ...
  if (name.getAsInteger(0, tid))
    error...
848 ↗(On Diff #95522)

for (auto &data: m_thread_data) data.signo = signo seems shorter, more understandable, and consistent with other usages in this file.

856 ↗(On Diff #95522)

This could also be a range-based for.

krytarowski edited the summary of this revision. (Show Details)

Fix handling executable and shared library triple detection.

Apply changes from review.

Update revision summary.

joerg added inline comments.Apr 27 2017, 7:02 AM
source/Plugins/ObjectFile/ELF/ObjectFileELF.cpp
55 ↗(On Diff #96272)

Not your fault, but is there a reason why this are all pointers to strings and not just const char[] ?

302 ↗(On Diff #96272)

Unrelated cosmetic change.

412 ↗(On Diff #96272)

Unrelated cosmetic change.

775 ↗(On Diff #96272)

Dito.

1488 ↗(On Diff #96272)

Unrelated.

3042 ↗(On Diff #96272)

Unrelated.

3066 ↗(On Diff #96272)

Unrelated.

source/Plugins/Process/elf-core/ProcessElfCore.cpp
378 ↗(On Diff #96272)

Unrelated.

463 ↗(On Diff #96272)

Either sort them by value or by name, but not randomly

523 ↗(On Diff #96272)

Can you define a constant for the offset here below instead of a magic number?

685 ↗(On Diff #96272)

Unrelated.

844 ↗(On Diff #96272)

Typo

854 ↗(On Diff #96272)

Move the else to the } and the comment after the {

krytarowski added inline comments.Apr 27 2017, 8:56 AM
source/Plugins/ObjectFile/ELF/ObjectFileELF.cpp
302 ↗(On Diff #96272)

I let clang-format to go and alter minor things. I can run clang-format over original files - commit, and add my diff again.

source/Plugins/Process/elf-core/ProcessElfCore.cpp
463 ↗(On Diff #96272)

I will split this enum{} into two enums.

523 ↗(On Diff #96272)

I will try to get something to define aliases for these magic numbers.

844 ↗(On Diff #96272)

OK

854 ↗(On Diff #96272)

OK

ping?

Sorry, I wasn't responding partially because I was waiting to see how the discussion on https://reviews.llvm.org/D32434 settles, as I think it may have effect on the test strategy. I'll write more tomorrow. :)

krytarowski added a comment.EditedApr 27 2017, 11:18 AM

ping?

Sorry, I wasn't responding partially because I was waiting to see how the discussion on https://reviews.llvm.org/D32434 settles, as I think it may have effect on the test strategy. I'll write more tomorrow. :)

Thanks for working on it. I will upload soon changes to address noted by Joerg.

Strip non-functional style improvements in unrelated code-parts.

Apply changes to address comments from Joerg.

Remove another unrelated style improvement.

Rebase to HEAD.
Apply "Error" -> "Status" rename.

Can we go with this change?

What was your decision on the core files? I was under the impression you were gonna add the zip files as well. If so, then they should go in at the same time.

What was your decision on the core files? I was under the impression you were gonna add the zip files as well. If so, then they should go in at the same time.

In the same commit? If so I will try to amend the patch.

Affirmative. Tests should go in together with the feature they are testing.

krytarowski planned changes to this revision.Sep 27 2017, 1:38 AM

Suspended, I need to resurrect tracing of 1 thread and fix kernel bugs for multiple threads first.

mgorny commandeered this revision.Feb 2 2019, 8:49 AM
mgorny added a reviewer: krytarowski.
mgorny updated this revision to Diff 189503.Mar 6 2019, 6:44 AM

I've attempted to rebase Kamil's patch against the current sources. Please note that this is not the final version; I'll add a test case and PID reading later.

mgorny updated this revision to Diff 189536.Mar 6 2019, 10:42 AM
mgorny retitled this revision from Correct handling NetBSD core(5) files with threads to [lldb] [Process] Add proper support for NetBSD core files with threads.
mgorny edited the summary of this revision. (Show Details)

Updated to read PID and make unknown notes non-fatal.

mgorny updated this revision to Diff 189545.Mar 6 2019, 12:01 PM

Added two tests, more pending. Sadly, I wasn't able to reuse even the test cases since -nostdlib doesn't really work on NetBSD (resulting executables are not recognized as valid executables).

mgorny updated this revision to Diff 189658.Mar 7 2019, 12:31 AM
mgorny edited the summary of this revision. (Show Details)

I've added one more test case, and added stop reason/signal checking for both threads. I think it's ready for review now.

Looks good to me.

labath accepted this revision.Mar 8 2019, 12:35 PM

The code looks good, though I think we should really start figuring out how to write elf core tests without checking in binaries.

This revision is now accepted and ready to land.Mar 8 2019, 12:35 PM
This revision was automatically updated to reflect the committed changes.