[MachCore] Report arm64 thread exception state

[vsk]

Summary

A MachO userspace corefile may contain LC_THREAD commands which specify
thread exception state.

For arm64* only (for now), report a human-readable version of this state
as the thread stop reason, instead of 'SIGSTOP'.

As a follow-up, similar functionality can be implemented for x86 cores
by translating the trapno/err exception registers.

rdar://82898146

[vsk]

• Add license header to the .def file.
• (I'm not sure whether/how this change can be tested, any pointers appreciated.)

[shafik]

lldb/include/lldb/Target/AppleArm64ExceptionClass.h

15

We should use fixed sized integer types whenever possible, I suppose in this case uint32_t.

lldb/source/Plugins/Process/mach-core/ThreadMachCore.cpp

124

const

125

Does 26 have a meaning? I am guessing we are shifting to get the EC bits?

[jasonmolenda]

This is much more comprehensive than what I was thinking of, nicely done. The mapping of the Exception Class in the esr reg to a name is AArch64 specific, not Apple specific, isn't it? I haven't looked in the ARM ARM, but I suspect it's just the specific names you're using here which might be what we use at apple, but the exception codes are the same for any AArch64 target.

What do you think about printing the FAR register value (in particular) in the stop message? For many exception types, it contains the address that was relevant to the fault iirc, and even if we don't explicitly spell out what that relationship is (it's prob going to be different for each exception code), just having it printed may be helpful for a human reading the message.

I think the patch is fine if you prefer it this way, just giving some other ideas.

[jasonmolenda]

lldb/source/Plugins/Process/mach-core/ThreadMachCore.cpp

125

Yeah I think the comment table in AppleArm64ExceptionClass.def would be better placed here, showing the bit positions of the three fields in the esr register probably. AppleArm64ExceptionClass.def should make it clear that it is operating on the exception class field of the esr register, but the bit positions are more relevant here.

[vsk]

Address review feedback:

• Include far register in output, which now looks like e.g.:

(lldb) bt all
* thread #1, stop reason = ESR_EC_DABORT_EL0 (fault address: 0x6261747563657860)
  * frame #0: 0x00000001aa5c2864 libsystem_platform.dylib`_platform_strlen + 4

• Include note that the exception class list can/does contain some Apple-specific classes, but that it's largely 1:1 with the ARM spec.

lldb/include/lldb/Target/AppleArm64ExceptionClass.h

15

I don't think we should use fixed size integer types in cases where the size doesn't matter (e.g. when uint8_t would do, as it would in this case), since it's distracting.

lldb/source/Plugins/Process/mach-core/ThreadMachCore.cpp

125

Good point - actually, maybe it's cleaner to keep knowledge of the EC bit position in the header. Lmk what you think.

[jasonmolenda]

lldb/source/Plugins/Process/mach-core/ThreadMachCore.cpp

125

I was thinking the same thing, that getAppleArm64ExceptionClass might just take the $esr register value and get the necessary bits from it. I guess every caller of this is probably starting with the ESR register value, so having every caller know how to grab the EC value from ESR doesn't seem good. Some callers might want to use EC/IL/ISS to describe the nature of the error so they may need to know where these bits are in the register, but if they just want the EC name, it's unnecessary.

[vsk]

FTR, I was able to write a test using process save-core, but even with -s dirty this is generating gigabytes of data and would presumably be super-flaky on the bots.
I also tried -s stack, but for the test program I passed in (just some 2-line .c file that dereferences NULL), process save-core -s stack ... generated an invalid .core with no sections, and lldb couldn't parse it.

Edit: This turned out to be user error, the thing I was testing was not supported with an out-of-tree (system) debugserver.

[vsk]

• Add a test (thanks @jasonmolenda!).

[jasonmolenda]

very nice.

lldb/test/API/macosx/corefile-exception-reason/TestCorefileExceptionReason.py

20

also not supported on armv7k.

[vsk]

lldb/test/API/macosx/corefile-exception-reason/TestCorefileExceptionReason.py

20

Thanks. I'll fix this before landing the patch (eta tomorrow morning).

[vsk]

• Limit testing to arm64/arm64e only.
• Set a stop reason for crashing threads only (and not for threads waiting on a syscall to finish).
• Use @jasonmolenda's multi-threaded test case; make sure we only select the crashing thread.

[vsk]

• Add missing skipIf(arch=no_match(['arm64', 'arm64e'])) change.

lldb/include/lldb/Target/AppleArm64ExceptionClass.h

+//===-- AppleArm64ExceptionClass.h ------------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLDB_TARGET_APPLEARM64EXCEPTIONCLASS_H
+#define LLDB_TARGET_APPLEARM64EXCEPTIONCLASS_H
+
+#include <cstdint>
+
+namespace lldb_private {
+
+enum class AppleArm64ExceptionClass : unsigned {
+#define APPLE_ARM64_EXCEPTION_CLASS(Name, Code) Name = Code,
+#include "AppleArm64ExceptionClass.def"
+};
+
+/// Get the Apple ARM64 exception class encoded within \p esr.
+inline AppleArm64ExceptionClass getAppleArm64ExceptionClass(uint32_t esr) {
+  /*
+   * Exception Syndrome Register
+   *
+   *  31  26 25 24               0
+   * +------+--+------------------+
+   * |  EC  |IL|       ISS        |
+   * +------+--+------------------+
+   *
+   * EC  - Exception Class
+   * IL  - Instruction Length
+   * ISS - Instruction Specific Syndrome
+   */
+  return static_cast<AppleArm64ExceptionClass>(esr >> 26);
+}
+
+inline const char *toString(AppleArm64ExceptionClass EC) {
+  switch (EC) {
+#define APPLE_ARM64_EXCEPTION_CLASS(Name, Code)                                \
+  case AppleArm64ExceptionClass::Name:                                         \
+    return #Name;
+#include "AppleArm64ExceptionClass.def"
+  }
+  return "Unknown Exception Class";
+}
+
+} // namespace lldb_private
+
+#endif // LLDB_TARGET_APPLEARM64EXCEPTIONCLASS_H

lldb/include/lldb/Target/AppleArm64ExceptionClass.def

+/*===-- AppleArm64ExceptionClass.def ---------------------------*- C++ -*-=== *\
+|*
+|* Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+|* See https://llvm.org/LICENSE.txt for license information.
+|* SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+|*
+\*===----------------------------------------------------------------------===*/
+
+// Defines ESR exception classes for Apple arm64* targets.
+// These largely map 1:1 to the exception classes defined in ARM's architecture
+// reference manual, but there are some Apple-specific additions.
+
+#ifndef APPLE_ARM64_EXCEPTION_CLASS
+#error "APPLE_ARM64_EXCEPTION_CLASS(Name, Code) not defined."
+#endif
+
+APPLE_ARM64_EXCEPTION_CLASS(ESR_EC_UNCATEGORIZED, 0x00)
+APPLE_ARM64_EXCEPTION_CLASS(ESR_EC_WFI_WFE, 0x01)
+APPLE_ARM64_EXCEPTION_CLASS(ESR_EC_MCR_MRC_CP15_TRAP, 0x03)
+APPLE_ARM64_EXCEPTION_CLASS(ESR_EC_MCRR_MRRC_CP15_TRAP, 0x04)
+APPLE_ARM64_EXCEPTION_CLASS(ESR_EC_MCR_MRC_CP14_TRAP, 0x05)
+APPLE_ARM64_EXCEPTION_CLASS(ESR_EC_LDC_STC_CP14_TRAP, 0x06)
+APPLE_ARM64_EXCEPTION_CLASS(ESR_EC_TRAP_SIMD_FP, 0x07)
+APPLE_ARM64_EXCEPTION_CLASS(ESR_EC_PTRAUTH_INSTR_TRAP, 0x09)
+APPLE_ARM64_EXCEPTION_CLASS(ESR_EC_MCRR_MRRC_CP14_TRAP, 0x0c)
+APPLE_ARM64_EXCEPTION_CLASS(ESR_EC_ILLEGAL_INSTR_SET, 0x0e)
+APPLE_ARM64_EXCEPTION_CLASS(ESR_EC_SVC_32, 0x11)
+APPLE_ARM64_EXCEPTION_CLASS(ESR_EC_SVC_64, 0x15)
+APPLE_ARM64_EXCEPTION_CLASS(ESR_EC_MSR_TRAP, 0x18)
+APPLE_ARM64_EXCEPTION_CLASS(ESR_EC_PAC_FAIL, 0x1C)
+APPLE_ARM64_EXCEPTION_CLASS(ESR_EC_IABORT_EL0, 0x20)
+APPLE_ARM64_EXCEPTION_CLASS(ESR_EC_IABORT_EL1, 0x21)
+APPLE_ARM64_EXCEPTION_CLASS(ESR_EC_PC_ALIGN, 0x22)
+APPLE_ARM64_EXCEPTION_CLASS(ESR_EC_DABORT_EL0, 0x24)
+APPLE_ARM64_EXCEPTION_CLASS(ESR_EC_DABORT_EL1, 0x25)
+APPLE_ARM64_EXCEPTION_CLASS(ESR_EC_SP_ALIGN, 0x26)
+APPLE_ARM64_EXCEPTION_CLASS(ESR_EC_FLOATING_POINT_32, 0x28)
+APPLE_ARM64_EXCEPTION_CLASS(ESR_EC_FLOATING_POINT_64, 0x2C)
+APPLE_ARM64_EXCEPTION_CLASS(ESR_EC_SERROR_INTERRUPT, 0x2F)
+APPLE_ARM64_EXCEPTION_CLASS(ESR_EC_BKPT_REG_MATCH_EL0, 0x30)
+APPLE_ARM64_EXCEPTION_CLASS(ESR_EC_BKPT_REG_MATCH_EL1, 0x31)
+APPLE_ARM64_EXCEPTION_CLASS(ESR_EC_SW_STEP_DEBUG_EL0, 0x32)
+APPLE_ARM64_EXCEPTION_CLASS(ESR_EC_SW_STEP_DEBUG_EL1, 0x33)
+APPLE_ARM64_EXCEPTION_CLASS(ESR_EC_WATCHPT_MATCH_EL0, 0x34)
+APPLE_ARM64_EXCEPTION_CLASS(ESR_EC_WATCHPT_MATCH_EL1, 0x35)
+APPLE_ARM64_EXCEPTION_CLASS(ESR_EC_BKPT_AARCH32, 0x38)
+APPLE_ARM64_EXCEPTION_CLASS(ESR_EC_BRK_AARCH64, 0x3C)
+APPLE_ARM64_EXCEPTION_CLASS(ESR_EC_PRIV, 0x3F)
+
+#undef APPLE_ARM64_EXCEPTION_CLASS

lldb/include/lldb/module.modulemap

module lldb_Symbol {
 
     umbrella "Symbol"
     module * { export * }
   }
   module lldb_Target {
     requires cplusplus
 
     umbrella "Target"
+    textual header "Target/AppleArm64ExceptionClass.def"
     module * { export * }
   }
 }
 
 
 module lldb_Utility {
   requires cplusplus
 

lldb/source/Plugins/ObjectFile/Mach-O/ObjectFileMachO.cpp

if (reg_ctx_sp) {
       PrintRegisterValue(reg_ctx, "x26", nullptr, 8, data);
       PrintRegisterValue(reg_ctx, "x27", nullptr, 8, data);
       PrintRegisterValue(reg_ctx, "x28", nullptr, 8, data);
       PrintRegisterValue(reg_ctx, "fp", nullptr, 8, data);
       PrintRegisterValue(reg_ctx, "lr", nullptr, 8, data);
       PrintRegisterValue(reg_ctx, "sp", nullptr, 8, data);
       PrintRegisterValue(reg_ctx, "pc", nullptr, 8, data);
       PrintRegisterValue(reg_ctx, "cpsr", nullptr, 4, data);
+      data.PutHex32(0); // uint32_t pad at the end
 
       // Write out the EXC registers
-      //            data.PutHex32 (EXCRegSet);
-      //            data.PutHex32 (EXCWordCount);
-      //            WriteRegister (reg_ctx, "far", NULL, 8, data);
-      //            WriteRegister (reg_ctx, "esr", NULL, 4, data);
-      //            WriteRegister (reg_ctx, "exception", NULL, 4, data);
+      data.PutHex32(EXCRegSet);
+      data.PutHex32(EXCWordCount);
+      PrintRegisterValue(reg_ctx, "far", NULL, 8, data);
+      PrintRegisterValue(reg_ctx, "esr", NULL, 4, data);
+      PrintRegisterValue(reg_ctx, "exception", NULL, 4, data);
       return true;
     }
     return false;
   }
 
 protected:
   int DoReadGPR(lldb::tid_t tid, int flavor, GPR &gpr) override { return -1; }
 

lldb/source/Plugins/Process/mach-core/ThreadMachCore.cpp

 //===-- ThreadMachCore.cpp ------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "ThreadMachCore.h"
 
 #include "lldb/Breakpoint/Watchpoint.h"
+#include "lldb/Host/SafeMachO.h"
 #include "lldb/Symbol/ObjectFile.h"
+#include "lldb/Target/AppleArm64ExceptionClass.h"
 #include "lldb/Target/Process.h"
 #include "lldb/Target/RegisterContext.h"
 #include "lldb/Target/StopInfo.h"
 #include "lldb/Target/Target.h"
 #include "lldb/Target/Unwind.h"
 #include "lldb/Utility/ArchSpec.h"
 #include "lldb/Utility/DataExtractor.h"
+#include "lldb/Utility/RegisterValue.h"
 #include "lldb/Utility/State.h"
 #include "lldb/Utility/StreamString.h"
 
 #include "ProcessMachCore.h"
 //#include "RegisterContextKDP_arm.h"
 //#include "RegisterContextKDP_i386.h"
 //#include "RegisterContextKDP_x86_64.h"
 
if (concrete_frame_idx == 0) {
     }
     reg_ctx_sp = m_thread_reg_ctx_sp;
   } else {
     reg_ctx_sp = GetUnwinder().CreateRegisterContextForFrame(frame);
   }
   return reg_ctx_sp;
 }
 
+static bool IsCrashExceptionClass(AppleArm64ExceptionClass EC) {
+  switch (EC) {
+  case AppleArm64ExceptionClass::ESR_EC_UNCATEGORIZED:
+  case AppleArm64ExceptionClass::ESR_EC_SVC_32:
+  case AppleArm64ExceptionClass::ESR_EC_SVC_64:
+    // In the ARM exception model, a process takes an exception when asking the
+    // kernel to service a system call. Don't treat this like a crash.
+    return false;
+  default:
+    return true;
+  }
+}
+
 bool ThreadMachCore::CalculateStopInfo() {
   ProcessSP process_sp(GetProcess());
   if (process_sp) {
-    SetStopInfo(StopInfo::CreateStopReasonWithSignal(*this, SIGSTOP));
+    StopInfoSP stop_info;
+    RegisterContextSP reg_ctx_sp = GetRegisterContext();
+
+    if (reg_ctx_sp) {
+      Target &target = process_sp->GetTarget();
+      const ArchSpec arch_spec = target.GetArchitecture();
+      const uint32_t cputype = arch_spec.GetMachOCPUType();
+
+      if (cputype == llvm::MachO::CPU_TYPE_ARM64 ||
+          cputype == llvm::MachO::CPU_TYPE_ARM64_32) {
+        const RegisterInfo *esr_info = reg_ctx_sp->GetRegisterInfoByName("esr");
+        const RegisterInfo *far_info = reg_ctx_sp->GetRegisterInfoByName("far");
+        RegisterValue esr, far;
+        if (reg_ctx_sp->ReadRegister(esr_info, esr) &&
+            reg_ctx_sp->ReadRegister(far_info, far)) {
+          const uint32_t esr_val = esr.GetAsUInt32();
+          const AppleArm64ExceptionClass exception_class =
+              getAppleArm64ExceptionClass(esr_val);
+          if (IsCrashExceptionClass(exception_class)) {
+            StreamString S;
+            S.Printf("%s (fault address: 0x%" PRIx64 ")",
+                     toString(exception_class), far.GetAsUInt64());
+            stop_info =
+                StopInfo::CreateStopReasonWithException(*this, S.GetData());
+          }
+        }
+      }
+    }
+
+    // Set a stop reason for crashing threads only so that they get selected
+    // preferentially.
+    if (stop_info)
+      SetStopInfo(stop_info);
     return true;
   }
   return false;
 }

lldb/test/API/macosx/corefile-exception-reason/Makefile

+CXX_SOURCES = main.cpp
+
+include Makefile.rules

lldb/test/API/macosx/corefile-exception-reason/TestCorefileExceptionReason.py

+"""Test that lldb can report the exception reason for threads in a corefile."""
+
+import os
+import re
+import subprocess
+
+import lldb
+from lldbsuite.test.decorators import *
+from lldbsuite.test.lldbtest import *
+from lldbsuite.test import lldbutil
+
+class TestCorefileExceptionReason(TestBase):
+
+    mydir = TestBase.compute_mydir(__file__)
+
+    @skipIfOutOfTreeDebugserver  # newer debugserver required for these qMemoryRegionInfo types
+    @no_debug_info_test
+    @skipUnlessDarwin
+    @skipIf(archs=no_match(['arm64','arm64e']))
+    def test(self):
+
+        corefile = self.getBuildArtifact("process.core")
+        self.build()
+        (target, process, thread, bkpt) = lldbutil.run_to_source_breakpoint(
+            self, "// break here", lldb.SBFileSpec("main.cpp"))
+
+        self.runCmd("continue")
+
+        self.runCmd("process save-core -s stack " + corefile)
+        process.Kill()
+        self.dbg.DeleteTarget(target)
+
+        # Now load the corefile
+        target = self.dbg.CreateTarget('')
+        process = target.LoadCore(corefile)
+        thread = process.GetSelectedThread()
+        self.assertTrue(process.GetSelectedThread().IsValid())
+        if self.TraceOn():
+            self.runCmd("image list")
+            self.runCmd("bt")
+            self.runCmd("fr v")
+
+        self.assertTrue(thread.GetStopDescription(256) == "ESR_EC_DABORT_EL0 (fault address: 0x0)")

lldb/test/API/macosx/corefile-exception-reason/main.cpp

+#include <stdlib.h>
+#include <thread>
+#include <unistd.h>
+#include <vector>
+
+void *sleep_worker(void *in) {
+  sleep(30);
+  sleep(30);
+  return nullptr;
+}
+
+void *crash_worker(void *in) {
+  sleep(1);
+  volatile int *p = nullptr; // break here
+  return (void *)*p;
+}
+
+int main() {
+  std::vector<std::thread> threads;
+  threads.push_back(std::move(std::thread(crash_worker, nullptr)));
+  for (int i = 0; i < 15; i++)
+    threads.push_back(std::move(std::thread(sleep_worker, nullptr)));
+  sleep(10);
+}