Check return address stored in normal stack and CET shadow stack in unwind process phase2

[xiongji90]

Summary

If CET shadow stack is enabled, we will count the number of stack frames skipped and adjust CET shadow stack based on the number in libunwind unwind_phase2. When unwinding stack, we can compare the return address in normal stack against counterpart in CET shadow stack, if they don't match, it means the return address stored in normal stack has been corrupted, we will return _URC_FATAL_PHASE2_ERROR in that case. We don't check for non-catchable exception since such exceptions will lead to process termination.

[xiongji90]

Hi, @hjl.tools
Could you take a look at this patch, we aim to align with libgcc's behavior in CET shadow stack unwinding process in order to enhance security.

Thanks very much.

[MaskRay]

Is there any way to validate correctness without a CET-capable machine?
Is there any emulator

[xiongji90]

In D136667#3883963, @MaskRay wrote:

Is there any way to validate correctness without a CET-capable machine?
Is there any emulator

Hi, @MaskRay
I am afraid we don't have anyway to validate correctness without a CET-enable machine since the intrinisc "_get_ssp" will return 0 in machine without CET and all the check based on comparison between normal stack ret addr and CET shadow stack ret addr will be skipped.
Currently, I set up a CET enabled machine and ran all libunwind, libcxx, libcxxabi tests in CET mode to validate this patch. In testing, we need to also build libcxx, libcxxabi and corresponding test with CET options on.

Thanks very much.

[manojgupta]

thanks, I assume @MaskRay is a good reviewer for it.

[MaskRay]

Is this check a libgcc behavior? Is it intended for just C++ exceptions (regular phase2) or also for _Unwind_ForcedUnwind?

Can you craft an example to demonstrate a failure caught by the new check? It is going to be brittle so I can understand that a test is likely inappropriate.

For C++ exceptions exception_object->exception_class != 0 is always true while for _Unwind_ForcedUnwind exception_class is typically zero, so I am not sure having the code in unwind_phase2_forced is useful...

libunwind/src/UnwindLevel1.c

317

#ifdef

[hjl.tools]

In libgcc, the same macro is used for both _Unwind_RaiseException_Phase2 and _Unwind_ForcedUnwind_Phase2.
_Unwind_ForcedUnwind usages in glibc always terminate and don't unwind stack. We may not need to check
return address on shadow stack in this case.

libunwind/src/UnwindLevel1.c

317

#ifdef

[xiongji90]

In D136667#3897494, @MaskRay wrote:

Is this check a libgcc behavior? Is it intended for just C++ exceptions (regular phase2) or also for _Unwind_ForcedUnwind?

Can you craft an example to demonstrate a failure caught by the new check? It is going to be brittle so I can understand that a test is likely inappropriate.

For C++ exceptions exception_object->exception_class != 0 is always true while for _Unwind_ForcedUnwind exception_class is typically zero, so I am not sure having the code in unwind_phase2_forced is useful...

Hi, @MaskRay
If so, I will remove unnecessary check in unwind_phase2_forced and where can I find any document or description about
"For C++ exceptions exception_object->exception_class != 0 is always true while for _Unwind_ForcedUnwind exception_class is typically zero"?
Thanks very much.

Thanks very much.

[xiongji90]

libunwind/src/UnwindLevel1.c

317

Done.

[xiongji90]

Hi, @MaskRay
Unnecessary check in forced unwind is removed. This patch is to align with libgcc's behavior in stack unwinding process on CET enabled platform. libgcc added check in stacking unwinding phase2, we need to go through the stacks to count how many stack frames to skip when CET is enabled and we can enhance security by comparing return addr in normal stack against CET shadow stack at the same time. If return addr in the 2 stacks don't match, it means the normal stack has been corrupted by someone and we can report fatal error in advance.
We also ran libunwind, libcxx, libcxxabi tests with CET enabled with latest patch, all tests passed.

Thanks very much.

[MaskRay]

LGTM.

libunwind/src/UnwindLevel1.c

 // a regular function call to avoid pushing to CET shadow stack again.
 #if !defined(_LIBUNWIND_USE_CET)
 #define __unw_phase2_resume(cursor, fn)                                        \
   do {                                                                         \
     (void)fn;                                                                  \
     __unw_resume((cursor));                                                    \
   } while (0)
 #elif defined(_LIBUNWIND_TARGET_I386)
+#define __cet_ss_step_size 4
 #define __unw_phase2_resume(cursor, fn)                                        \
   do {                                                                         \
     _LIBUNWIND_POP_CET_SSP((fn));                                              \
     void *cetRegContext = __libunwind_cet_get_registers((cursor));             \
     void *cetJumpAddress = __libunwind_cet_get_jump_target();                  \
     __asm__ volatile("push %%edi\n\t"                                          \
                      "sub $4, %%esp\n\t"                                       \
                      "jmp *%%edx\n\t" :: "D"(cetRegContext),                   \
                      "d"(cetJumpAddress));                                     \
   } while (0)
 #elif defined(_LIBUNWIND_TARGET_X86_64)
+#define __cet_ss_step_size 8
 #define __unw_phase2_resume(cursor, fn)                                        \
   do {                                                                         \
     _LIBUNWIND_POP_CET_SSP((fn));                                              \
     void *cetRegContext = __libunwind_cet_get_registers((cursor));             \
     void *cetJumpAddress = __libunwind_cet_get_jump_target();                  \
     __asm__ volatile("jmpq *%%rdx\n\t" :: "D"(cetRegContext),                  \
                      "d"(cetJumpAddress));                                     \
   } while (0)
unwind_phase2(unw_context_t *uc, unw_cursor_t *cursor, _Unwind_Exception *exception_object) {
   __unw_init_local(cursor, uc);
 
   _LIBUNWIND_TRACE_UNWINDING("unwind_phase2(ex_ojb=%p)",
                              (void *)exception_object);
 
   // uc is initialized by __unw_getcontext in the parent frame. The first stack
   // frame walked is unwind_phase2.
   unsigned framesWalked = 1;
+#ifdef _LIBUNWIND_USE_CET
+  unsigned long shadowStackTop = _get_ssp();
+#endif
   // Walk each frame until we reach where search phase said to stop.
   while (true) {
 
     // Ask libunwind to get next frame (skip over first which is
     // _Unwind_RaiseException).
     int stepResult = __unw_step_stage2(cursor);
     if (stepResult == 0) {
       _LIBUNWIND_TRACE_UNWINDING(
if (_LIBUNWIND_TRACING_UNWINDING) {
                                  ", func=%s, sp=0x%" PRIxPTR ", lsda=0x%" PRIxPTR
                                  ", personality=0x%" PRIxPTR,
                                  (void *)exception_object, frameInfo.start_ip,
                                  functionName, sp, frameInfo.lsda,
                                  frameInfo.handler);
     }
 #endif
 
+// In CET enabled environment, we check return address stored in normal stack
+// against return address stored in CET shadow stack, if the 2 addresses don't
+// match, it means return address in normal stack has been corrupted, we return
+// _URC_FATAL_PHASE2_ERROR.
+#ifdef _LIBUNWIND_USE_CET
+    if (shadowStackTop != 0) {
+      unw_word_t retInNormalStack;
+      __unw_get_reg(cursor, UNW_REG_IP, &retInNormalStack);
+      unsigned long retInShadowStack = *(
+          unsigned long *)(shadowStackTop + __cet_ss_step_size * framesWalked);
+      if (retInNormalStack != retInShadowStack)
+        return _URC_FATAL_PHASE2_ERROR;
+    }
+#endif
     ++framesWalked;
     // If there is a personality routine, tell it we are unwinding.
     if (frameInfo.handler != 0) {
       _Unwind_Personality_Fn p =
           (_Unwind_Personality_Fn)(uintptr_t)(frameInfo.handler);
       _Unwind_Action action = _UA_CLEANUP_PHASE;
       if (sp == exception_object->private_2) {
         // Tell personality this was the frame it marked in phase 1.

 unwind_phase2_forced(unw_context_t *uc, unw_cursor_t *cursor,
                      _Unwind_Exception *exception_object,
                      _Unwind_Stop_Fn stop, void *stop_parameter) {
   __unw_init_local(cursor, uc);
 
   // uc is initialized by __unw_getcontext in the parent frame. The first stack
   // frame walked is unwind_phase2_forced.
   unsigned framesWalked = 1;
   // Walk each frame until we reach where search phase said to stop
   while (__unw_step_stage2(cursor) > 0) {
 
     // Update info about this frame.
     unw_proc_info_t frameInfo;
     if (__unw_get_proc_info(cursor, &frameInfo) != UNW_ESUCCESS) {
       _LIBUNWIND_TRACE_UNWINDING(
           "unwind_phase2_forced(ex_ojb=%p): __unw_step_stage2 "
           "failed => _URC_END_OF_STACK",