[lldb/crashlog] Add support for Application Specific Backtraces & Information

[mib]

Summary

For an exception crashlog, the thread backtraces aren't usually very helpful
and instead, developpers look at the "Application Specific Backtrace" that
was generated by objc_exception_throw.

LLDB could already parse and symbolicate these Application Specific Backtraces
for regular textual-based crashlog, so this patch adds support to parse them
in JSON crashlogs, and materialize them a HistoryThread extending the
crashed ScriptedThread.

This patch also includes the Application Specific Information messages
as part of the process extended crash information log. To do so, the
ScriptedProcess Python interface has a new GetMetadata method that
returns an arbitrary dictionary with data related to the process.

rdar://93207586

Signed-off-by: Med Ismail Bennani <[e-mail]>

[JDevlieghere]

Test? :-)

lldb/examples/python/crashlog.py

567–570

Do we have an existing regex for this from the textual parser?

576

For consistency with the other errors

[mib]

• Add test (wip)
• Add Application Specific Information to process extended crash info
• Add {,Scripted}Process::GetMetadata method

[JDevlieghere]

I'm not sure I understand how the test is exercising the application specific backtrace. I don't see any of the frames or the crash reason in there. What am I missing?

lldb/examples/python/crashlog.py

566–582

Can this be a top level function? It's hard to tell if this is capturing anything. We have other place where we have helper functions that are called from a loop.

lldb/examples/python/scripted_process/crashlog_scripted_process.py

142–145

nit

lldb/source/Plugins/Platform/MacOSX/PlatformDarwin.cpp

1000 ↗

(On Diff #465232)

Should this check if the size == 1?

lldb/source/Plugins/Process/scripted/ScriptedProcess.cpp

491 ↗

(On Diff #465232)

Is this ever populated?

[mib]

lldb/examples/python/crashlog.py

566–582

This parses a single backtrace. I'm not sure what you mean by "We have other place where we have helper functions that are called from a loop."

lldb/source/Plugins/Platform/MacOSX/PlatformDarwin.cpp

1000 ↗

(On Diff #465232)

I don't think this should only work if the dictionary has only 1 item.

lldb/source/Plugins/Process/scripted/ScriptedProcess.cpp

491 ↗

(On Diff #465232)

It is used below in ErrorWithMessage.

[mib]

• Address @JDevlieghere comments.
• Change implementation to have the ASI in the process extended crash info and ASB as an extended HistoryThread of the actually crashed thread.
• Fix test.

[mib]

Update tests following changes in D135547

[JDevlieghere]

A few nits but overall this LGTM.

lldb/examples/python/crashlog.py

566–582

ping

584–588

I don't think it's correct to treat the ASI as just another thread. IIUC it's the same as the crashing thread, just at another point in time (when the exception was thrown). To me that sounds like it should be its own thing.

lldb/examples/python/scripted_process/crashlog_scripted_process.py

44

lldb/include/lldb/Interpreter/ScriptedProcessInterface.h

105 ↗

(On Diff #472756)

lldb/include/lldb/Target/Process.h

2431 ↗

(On Diff #472756)

lldb/source/Plugins/Platform/MacOSX/PlatformDarwin.cpp

985 ↗

(On Diff #472756)

[mib]

lldb/examples/python/crashlog.py

584–588

This allows to show the Application Specific Backtrace with other threads when using the non-interactive crashlog command.

For interactive crashlogs, the Application Specific Backtrace is attached to the crashed thread as its extended backtrace and would only be show when doing thread backtrace --extended true

lldb/include/lldb/Interpreter/ScriptedProcessInterface.h

105 ↗

(On Diff #472756)

ditto

lldb/include/lldb/Target/Process.h

2431 ↗

(On Diff #472756)

I'll make a follow-up patch to update this. (D137359)

[mib]

Addressed @JDevlieghere feedbacks.

lldb/examples/python/crashlog.py

def parse(self):
         except:
             raise CrashLogFormatException()
 
         try:
             self.parse_process_info(self.data)
             self.parse_images(self.data['usedImages'])
             self.parse_main_image(self.data)
             self.parse_threads(self.data['threads'])
+            if 'asiBacktraces' in self.data:
+                self.parse_app_specific_backtraces(self.data['asiBacktraces'])
             self.parse_errors(self.data)
             thread = self.crashlog.threads[self.crashlog.crashed_thread_idx]
             reason = self.parse_crash_reason(self.data['exception'])
             if thread.reason:
                 thread.reason = '{} {}'.format(thread.reason, reason)
             else:
                 thread.reason = reason
         except (KeyError, ValueError, TypeError) as e:
def parse_threads(self, json_threads):
                     thread.registers = self.parse_thread_registers(
                         json_thread['threadState'])
             if 'queue' in json_thread:
                 thread.queue = json_thread.get('queue')
             self.parse_frames(thread, json_thread.get('frames', []))
             self.crashlog.threads.append(thread)
             idx += 1
 
+    def parse_app_specific_backtraces(self, json_app_specific_bts):
+        def parse_asi_backtrace(self, thread, bt):
+            frame_regex = re.compile(r'^([0-9]+)' r'\s'                # id
+                            r'+(.+?)'    r'\s+'               # img_name
+                            r'(0x[0-9a-fA-F]{7}[0-9a-fA-F]+)' # addr
+                            r' +(.*)'                         # offs
+                        )
+
+            for line in bt.split('\n'):
+                frame_match = frame_regex.search(line)
+                if not frame_match:
+                    print("error: Couldn't parse Application Specific Backtrace.")
+                    return False
+
+                (frame_id, frame_img_name,
+                    frame_addr, frame_ofs) = frame_match.groups()
+
+                thread.add_ident(frame_img_name)
+                if frame_img_name not in self.crashlog.idents:
+                    self.crashlog.idents.append(frame_img_name)
+                thread.frames.append(self.crashlog.Frame(int(frame_id), int(
+                    frame_addr, 0), frame_ofs))
+
+            return True
+
+        for idx, backtrace in enumerate(json_app_specific_bts):
+            thread = self.crashlog.Thread(idx, True)
+            thread.queue = "Application Specific Backtraces"
+            if parse_asi_backtrace(self, thread, backtrace):
+                self.crashlog.threads.append(thread)
+
     def parse_thread_registers(self, json_thread_state, prefix=None):
         registers = dict()
         for key, state in json_thread_state.items():
             if key == "rosetta":
                 registers.update(self.parse_thread_registers(state))
                 continue
             if key == "x":
                 gpr_dict = { str(idx) : reg for idx,reg in enumerate(state) }

lldb/examples/python/scripted_process/crashlog_scripted_process.py

def parse_crashlog(self):
                 load_images(self, crash_log.images)
             elif thread.did_crash():
                 for ident in thread.idents:
                     load_images(self, crash_log.find_images_with_identifier(ident))
 
             self.threads[thread.index] = CrashLogScriptedThread(self, None, thread)
 
     def __init__(self, target: lldb.SBTarget, args : lldb.SBStructuredData):
         super().__init__(target, args)
 
         if not self.target or not self.target.IsValid():
             return
 
         self.crashlog_path = None
 
         crashlog_path = args.GetValueForKey("crashlog_path")
         if crashlog_path and crashlog_path.IsValid():
def create_stackframes(self):
         return self.frames
 
     def __init__(self, process, args, crashlog_thread):
         super().__init__(process, args)
 
         self.backing_thread = crashlog_thread
         self.idx = self.backing_thread.index
         self.tid = self.backing_thread.id
         self.name = self.backing_thread.name
+        if self.backing_thread.app_specific_backtrace:
+            self.name = "Application Specific Backtrace - " + str(self.idx)
         self.queue = self.backing_thread.queue
         self.has_crashed = (self.scripted_process.crashed_thread_idx == self.idx)
         self.create_stackframes()
 
     def get_state(self):
         if not self.has_crashed:
             return lldb.eStateStopped
         return lldb.eStateCrashed