Page MenuHomePhabricator

[Support] Enable file + line info in LLVM stack traces on Darwin.
Needs ReviewPublic

Authored by lhames on Nov 22 2019, 5:08 PM.

Details

Summary

This patch provides an implementation of findModulesAndOffsets for Darwin.
This function maps stack frames to (image name, vm-address) pairs which the
generic function printSymbolizedStackTrace (in Signals.cpp) can feed to
llvm-symbolize. Where a dSYM is present, llvm-symbolize will use this to find
source file and line info and add it to the trace.

Event Timeline

lhames created this revision.Nov 22 2019, 5:08 PM
Herald added a project: Restricted Project. · View Herald Transcript

Looking for feedback on the approach before I go too much further with this. I was motivated to enable file + line info by Don Hinton's work in https://reviews.llvm.org/D70259 and https://reviews.llvm.org/D70263, where he attaches source location information certain calls to aid in debugging of error handling failures. I have a review out for an alternative scheme that produces full backtraces for error handling failures (see https://reviews.llvm.org/D70600), but the lack of file and line info in the traces is a drawback compared to Don's approach.

As for why we don't have this on Darwin already: I'm not sure whether we had support for it and lost it at some point, or just never had it. It looks like we experimented with other approaches a while back (e.g. _Unwind_Backtrace in https://reviews.llvm.org/D28265) but ultimately gave up.

lhames updated this revision to Diff 230761.Nov 23 2019, 9:13 AM
  • Remove byte-swapping code. This will run in process, so endianness shouldn't
lhames updated this revision to Diff 230762.Nov 23 2019, 9:16 AM
  • Remove byte-swapping code. This will run in process, so endianness shouldn't
  • Fix variable naming.
Harbormaster completed remote builds in B41411: Diff 230762.

This is awesome, thanks for working on it!

I pulled down your patch and built it, but am not getting expected results. Did I miss something?

I just munged an existing test to force a failure and emit a stack trace, and got the following
(I have a recent llvm-symbolizer in my path):

/Users/dhinton/projects/llvm/llvm-project/llvm/unittests/Support/ErrorTest.cpp:394: Failure
Death test: FailToHandle()
    Result: died but not with expected error.
  Expected: Failure value returned from cantFail wrapped call
CustomError \{7\}x
Actual msg:
[  DEATH   ] Failure value returned from cantFail wrapped call
[  DEATH   ] CustomError {7}
[  DEATH   ] UNREACHABLE executed at /Users/dhinton/projects/llvm/llvm-project/llvm/include/llvm/Support/Error.h:713!
[  DEATH   ]  #0 0x0000000109e65c9c (SupportTests+0x100759c9c)
[  DEATH   ]  #1 0x0000000109e66229 (SupportTests+0x10075a229)
[  DEATH   ]  #2 0x0000000109e63fb6 (SupportTests+0x100757fb6)
[  DEATH   ]  #3 0x0000000109e687ac (SupportTests+0x10075c7ac)
<snip>
lhames updated this revision to Diff 230774.Nov 23 2019, 12:59 PM
  • Remove check for an impossible magic value.

This is awesome, thanks for working on it!

I pulled down your patch and built it, but am not getting expected results. Did I miss something?

Did you generate a dSYM for the binary with dsymutil? Unfortunately llvm-symbolize doesn't seem to know how to symbolize directly from objects containing debug info, but I'm hoping we can eventually teach it to do that.

This is awesome, thanks for working on it!

I pulled down your patch and built it, but am not getting expected results. Did I miss something?

Did you generate a dSYM for the binary with dsymutil? Unfortunately llvm-symbolize doesn't seem to know how to symbolize directly from objects containing debug info, but I'm hoping we can eventually teach it to do that.

Just a standard llvm Debug build. Is there an cmake option I'm missing?

Just a standard llvm Debug build. Is there an cmake option I'm missing?

I'm not sure whether there is a cmake option to generate dSYMs yet -- you'll have to run dsymutil on whatever tool you're testing with.

I think the best long term solution to this problem is to teach llvm-symbolize how to cope with debug info in objects, rather than requiring a dSYM. An intermediate step (possibly generically useful) would be to add an option to llvm-symbolize to run llvm-dsymutil if there's no dSYM available.

Just a standard llvm Debug build. Is there an cmake option I'm missing?

I'm not sure whether there is a cmake option to generate dSYMs yet -- you'll have to run dsymutil on whatever tool you're testing with.

I think the best long term solution to this problem is to teach llvm-symbolize how to cope with debug info in objects, rather than requiring a dSYM. An intermediate step (possibly generically useful) would be to add an option to llvm-symbolize to run llvm-dsymutil if there's no dSYM available.

I suppose that might be useful for individuals, but I don't think it would be a good idea for the bots, which is my primary target.

Just a standard llvm Debug build. Is there an cmake option I'm missing?

I'm not sure whether there is a cmake option to generate dSYMs yet -- you'll have to run dsymutil on whatever tool you're testing with.

I think the best long term solution to this problem is to teach llvm-symbolize how to cope with debug info in objects, rather than requiring a dSYM. An intermediate step (possibly generically useful) would be to add an option to llvm-symbolize to run llvm-dsymutil if there's no dSYM available.

I suppose that might be useful for individuals, but I don't think it would be a good idea for the bots, which is my primary target.

My last comment concerned the intermediate step. I think your long-term solution would be great.

lhames added a comment.Jan 3 2020, 1:51 PM

...
I think the best long term solution to this problem is to teach llvm-symbolize how to cope with debug info in objects, rather than requiring a dSYM. An intermediate step (possibly generically useful) would be to add an option to llvm-symbolize to run llvm-dsymutil if there's no dSYM available.

I suppose that might be useful for individuals, but I don't think it would be a good idea for the bots, which is my primary target.

My last comment concerned the intermediate step. I think your long-term solution would be great.

If the option is added (and LLVM's stack symbolication call-out knows to use it) then I think it should work on the bots too, right? The only issue would be that crashing test cases would be slightly slower (since we'd have to produce the dSYMs).

...
I think the best long term solution to this problem is to teach llvm-symbolize how to cope with debug info in objects, rather than requiring a dSYM. An intermediate step (possibly generically useful) would be to add an option to llvm-symbolize to run llvm-dsymutil if there's no dSYM available.

I suppose that might be useful for individuals, but I don't think it would be a good idea for the bots, which is my primary target.

My last comment concerned the intermediate step. I think your long-term solution would be great.

If the option is added (and LLVM's stack symbolication call-out knows to use it) then I think it should work on the bots too, right? The only issue would be that crashing test cases would be slightly slower (since we'd have to produce the dSYMs).

Bit of an aside: the speed of crashing tests in the presence of debug info is actually sort of important/currently a bit of a bottleneck - all gunit death tests (for instance, llvm::Error has a few of those) crash, and run the symbolizer as the normal part of the crash process. That symbolization is pretty slow currently in a debug/unoptimized build of the symbolizer itself. It'd be great to find a way to disable crash symbolizing for death tests. (I guess maybe this performance issue doesn't come up on the MachO platform because of the missing functionality that's causing it not to symbolize)

dexonsmith resigned from this revision.Jun 24 2020, 3:18 PM

Yep. I'd actually settled on the idea of teaching llvm-symbolizer how to chase down the debug info in the individual objects, but I'm not sure when I'll have time to actually do the work.