[libunwind][CET] Support exception handling stack unwind in CET environment

[xiongji90]

Summary

Control-flow Enforcement Technology (CET), published by Intel, introduces shadow stack feature aiming to ensure a return from a function is directed to where the function was called.
In a CET enabled system, each function call will push return address into normal stack and shadow stack, when the function returns, the address stored in shadow stack will be popped and compared with the return address, program will fail if the 2 addresses don't match.
In exception handling, the control flow may skip some stack frames and we must adjust shadow stack to avoid violating CET restriction.
In order to achieve this, we count the number of stack frames skipped and adjust shadow stack by this number before jumping to landing pad.
We have run libcxx, libcxxabi, libunwind tests with this patch in a CET enabled Fedora34 system, no failed tests found.
To build an application with CET enabled. we need to ensure:

1. build the source code with "-fcf-protection=full"
2. all the libraries linked with .o files must be CET enabled too

To run libcxx and libcxxabi tests with libunwind in a CET enabled system, we built libc++ and libc++abi.so with "-fcf-protection". Another patch has been uploaded for review to enable building llvm toolchian with CET enabled: https://reviews.llvm.org/D105603 This patch is not enough to enable all components but it does build libcxx, libcxxabi with CET enabled.

[hjl.tools]

libunwind/src/UnwindRegistersRestore.S

51

I don't think it is safe to access memory below stack pointer in 32-bit mode since there is no red zone. Please check what
the GCC unwinder does.

126

It is OK to access memory below stack pointer in 64-bit mode since there is a red zone.

libunwind/src/cet_unwind.h

29

Why are / and % used here? You pop 255 frames a time if numbers of frame > 255.

[xiongji90]

libunwind/src/UnwindRegistersRestore.S

51

Hi, @hjl.tools
Before jumping to -4(%esp), we have added 4 to %esp value, so we should access a valid stack frame here.
Before adding 4 to esp, it should point to memory location where jump target address resides.
Thanks very much.

[hjl.tools]

libunwind/src/UnwindRegistersRestore.S

51

When an interrupt comes between "add $4, %esp" and "jmp -4(%esp)", the content at -4(%esp) can be changed by
the interrupt.

[xiongji90]

This update fixes some format issue and updates the way to jump to landing pad in 32bit node.
Previously, we used following code to jump to landing pad:
"
add $4, %esp
jmp *-4(%esp)
"
If interrupt comes in between these 2 instructions, interrupt handler may change the contents of "-4(%esp)" since there is no red zone in 32bit mode. To fix this issue, we align with gcc unwind to use %ecx to store landing pad address and jmp to it:
"
pop %ecx
jmp *%ecx
"

According to i386 ABI:
"Prior to executing code in the landing pad, the unwind library restores registers not altered by the personality routine, using the context record, to their state in that frame before the call that threw the exception, as follows. All registers specified as callee-saved by the base ABI are restored, as well as scratch registers %eax and %edx (see below). Except for those exceptions, scratch (or caller-saved) registers are not preserved, and their contents are undefined on transfer."
It is OK to use %ecx for jumping to landing pad.

[xiongji90]

libunwind/src/UnwindRegistersRestore.S

51

Hi, @hjl.tools
For 32bit mode, I have aligned with gcc unwind to use %ecx to store landing pad address then jump to it.
Thanks very much.

libunwind/src/cet_unwind.h

29

Removed unnecessary % and /.

[hjl.tools]

libunwind/src/cet_unwind.h

29

Can you do

while (tmp > 255)
{
...
}

and remove if/else?

[hjl.tools]

libunwind/src/UnwindLevel1.c

42

Why is this needed? Can you unwind the shadow stack and jump to the landing pad as usual?

[xiongji90]

Use scratch register %rcx for indirect jumping to landing pad, there are 2 reasons for this:

1. it is safer since red zone may be unavailable
2. align with 32bit mode

[xiongji90]

Hi, @manojgupta
This patch aims to fix https://bugs.llvm.org/show_bug.cgi?id=45946
Could you help me review it?
Thanks very much!

[manojgupta]

@pcc @cferris Would you be able to review this libunwind patch?

[xiongji90]

Add a build option "LIBUNWIND_BUILD_CET_ENABLE" to enable CET for libunwind.

[cferris]

This isn't code I'm really familiar with. You probably want @rprichard since he's worked on this recently.

[xiongji90]

libunwind/src/UnwindLevel1.c

42

Hi, @hjl.tools
I have updated the patch to remove some unnecessary code, could you help review again?
Thanks very much.

[compnerd]

libunwind/CMakeLists.txt

55

Please name this LIBUNWIND_ENABLE_CET

146

I don't think that this is needed. You should be able to directly use LIBUNWIND_BUILD_CET_ENABLE at the sites using this local variable.

188

I think that we want to test and set:

• /Gy
• /guard:ehcont
• /guard:cf
• /cetcompat

Or explicitly error with MSVC.

libunwind/src/UnwindLevel1.c

47

Why does this have to be a macro? Can we just define the function instead?

48

Can we avoid the value here and use #if defined(__x86_64__) || defined(_M_ARM64) instead to select between the inline assembly?

libunwind/src/assembly.h

18

Do you need to consider MinGW or cywin?

libunwind/src/cet_unwind.h

13

Please add a newline above this line.

libunwind/src/libunwind_ext.h

37 ↗

(On Diff #361944)

Why is this new interface needed? Who is the expected user?

[hjl.tools]

libunwind/src/UnwindLevel1.c

42

__unw_phase2_resume looks quite fragile. Can you call the function pointer, cetJumpAddress, in C directly?

[xiongji90]

libunwind/CMakeLists.txt

188

Hi, @compnerd
I suggest to report error with MSVC currently since CET enabling for Windows is still ongoing, once MS claims the enabling work is done, we can support MSVC.
Thanks very much.

libunwind/src/UnwindLevel1.c

42

Hi, @hjl.tools
I am afraid calling a function pointer in C is risky. The cetJumpAddress is address of libunwind_Registers_x86/x86_64_jumpto and these functions will never return, if we call this function or call the function pointer, compiler codegen may generate following code:
"
mov libunwind_Registers_x86_64_jumpto, %rax
call *%rax
"
Under such circumstance, we must adjust CET SSP by 1 in __libunwind_Registers_x86_64_jumpto since the call instruction will push CET shadow stack.

However, if compiler codegen use "jmp" instead of "call", for example:
"
jmp xxxxx (the addr of __libunwind_Registers_x86_64_jumpto)
"
we shouldn't adjust CET shadow stack. It is difficult for us to decide whether or not should we adjust CET shadow stack since we can't know what code compiler will generate for the function/function pointer call in C source level.

Thanks very much.

47

Hi, @compnerd
In previous version, I defined a function the code looks like following:
void unw_phase2_resume(cursor, fn) {
LIBUNWIND_POP_CET_SSP(...);
....
....
asm volatile(....)
}
The stack layout looks like following if compiler doesn't do any optimization to __unw_phase2_resume:

---

.....user functions

---

__cxx_throw

---

_Unwind_RaiseException

---

unwind_phase2

---

__unw_phase2_resume

---

Under such circumstances, the stack frame for unw_phase2_resume will be skipped too and we must adjust CET shadow stack for it.
However, as we can see, the unw_phase2_resume will never return. I am afraid compiler may directly jump to this function instead of "call" it. If so, there will be no stack frame for unw_phase2_resume and if compiler uses "jmp" to jump to it instead of calling it, we mustn't adjust CET shadow stack for it.
In our source code, we can't know whether compiler will do such optimization and we can't decide whether we should adjust CET shadow stack for unw_phase2_resume. So I used macro here to avoid this problem.
I used inline asm to jump to __libunwind_Registers_x86_64_jumpto for the same purpose.
Thanks very much.

libunwind/src/assembly.h

18

Hi, @compnerd
I suggest to ignore MinGW and cywin currently.
Hi, @hjl.tools
Do you think we can ignore MinGW and cywin?

Thanks very much.

[hjl.tools]

libunwind/src/UnwindLevel1.c

42

GCC uses an undocumented intrinsic, __builtin_eh_return, to avoid such issue.

libunwind/src/assembly.h

18

If MinGW and cywin support CET, libunwind should support MinGW and cywin.

[xiongji90]

1. Rename LIBUNWIND_BUILD_CET_ENABLE to LIBUNWIND_ENABLE_CET in libunwind CMakeFile and remove unnecessary LIBUNWIND_CET_ON local variable
2. Get rid of using value of _LIBUNWIND_CET_ENABLED and using "_LIBUNWIND_TARGET_I386" and "_LIBUNWIND_TARGET_X86_64" to select inline asm.

[xiongji90]

libunwind/CMakeLists.txt

55

Done.

Thanks very much!

146

Removed the unnecessary local variable and use LIBUNWIND_ENABLE_CET directly.

Thanks very much!

libunwind/src/UnwindLevel1.c

48

Hi, @compnerd
I have abandoned usage of _LIBUNWIND_CET_ENABLED value and used "_LIBUNWIND_TARGET_I386" and "_LIBUNWIND_TARGET_X86_64" to select inline assembly. Currently CET is implemented in GNU Linux + Intel platform, so we don't need to check for ARM64.

Thanks very much.

[xiongji90]

libunwind/src/UnwindLevel1.c

42

Hi, @hjl.tools
The undocumented GCC intrinsic builtin_eh_return are used by GCC unwinder and its semantic are strongly related to GCC unwind's implementation.
For example, GCC's unwinder will install target context and then get target landing pad address, after that GCC unwinder will use builtin_eh_return to adjust the SP value and jump to landing pad, builtin_eh_return assumes that target context has been prepared.
In our implementation, we need to adjust CET SSP and then jump to libunwind_Registers_X86/_64_jumpto(asm code), this asm function will prepare target context (restore registers), adjust SP and jump to final landing pad. Our requirement here is just directly jump to libunwind_Registers_X86/_64_jumpto function to avoid pushing CET shadow stack again. So, I think builtin_eh_return can't help much in libunwind.

Thanks very much.

libunwind/src/assembly.h

18

Hi, @hjl.tools and @compnerd
I didn't find any info of CET support for MinGW and cywin after some google search, so I suggest we can ignore MinGW and cywin currently.

Thanks very much.

libunwind/src/libunwind_ext.h

37 ↗

(On Diff #361944)

Hi, @compnerd
Currently, Registers are encapsulated in UnwindCursor class and libunwind_Registers_x86_jumpto/libunwind_Registers_x86_64_jumpto need Registers as input parameter. We need to prepare the parameter and then jump to libunwind_Registers_x86/_64_jumpto function, so we need to get internal Registers from a cursor. This function is added for this purpose and we don't expect other users to use it.
Do you think we should define it as an internal function instead of a function with "unw" prefix?
Thanks very much.

[hjl.tools]

libunwind/src/UnwindLevel1.c

42

One feature of __builtin_eh_return is to perform the special tail call. Can clang provide a way to do that?

[compnerd]

libunwind/CMakeLists.txt

188

I'm not sure about the statement (I believe that the CET support is available today), but I would be okay with the error stating that libunwind doesn't support CET on Windows yet. I'm more concerned that someone who builds the library may be under the false impression that CET is enabled when it is not.

libunwind/src/Registers.hpp

46

A newline before this line like in the x86_64 case would be nice.

libunwind/src/UnwindLevel1.c

47

I wonder if we can get away with some combination of noinline, and pragmas for no-optimize-sibling-calls.

libunwind/src/libunwind_ext.h

37 ↗

(On Diff #361944)

Yes please move this to an internal function. This header is exported to users, and this is why I was asking specifically about this function.

[xiongji90]

libunwind/CMakeLists.txt

188

Hi, @compnerd
I will update the patch to report error for MSVC + CET. All components including OS, toolchain, system libraries need to enable CET so that CET feature can be fully enabled in a platform. I know MS is working on CET but I am not sure whether they have finished the work.
Hi, @hjl.tools
Could you please correct me if anything is wrong about CET enabling status for Windows and MSVC.
Thanks very much.

libunwind/src/UnwindLevel1.c

42

Hi, @hjl.tools
I think clang is compatible with GCC for __builtin_eh_return and could you share some sample code or info about this special feature?
If it can work, we can get rid of the inline asm.
Thanks very much.

[xiongji90]

1. Report error for LIBUNWIND_ENABLE_CET + MSVC
2. Turn unw_cet_get_registers to an internal function libunwind_cet_get_registers
3. Some tiny code format change.

[xiongji90]

libunwind/src/Registers.hpp

46

Done.
Thanks very much.

[xiongji90]

libunwind/src/UnwindLevel1.c

42

Hi, @hjl.tools
I tried to use the undocumented intrinsic "builtin_eh_return" to replace inline asm. The code for x86_64 is like following:
builtin_eh_return(0, libunwind_Registers_x86_64_jumpto);
With "-fcf-protection=full", both gcc and clang will use a "jmpq *%rcx" to jump to libunwind_Registers_x86_64 which meets our requirement.
However, this intrinsic is used to jump to landing pad and expects to pass parameters via rax and rdx while libunwind_Registers_x86_64_jumpto is not a landing pad but a "normal" function which doesn't return. libunwind_Registers_x86_64_jumpto follows x86_64 abi to receive input parameters which means it expects input parameters from rdi, rsi.... And libunwind_Registers_x86_jumpto follows x86 abi to receive input parameters which means it expects input parameters stored in stack by caller. So I am afraid builtin_eh_return can't fully meet our requirement.
Thanks very much.

[xiongji90]

Chang default value of LIBUNWIND_ENABLE_CET to OFF in libunwind CMakeFile.
Since the LLVM_BUILD_CET_ENABLE has not been added to LLVM building system, we can set LIBUNWIND_ENABLE_CET default value to OFF currently.
When LLVM_BUILD_CET_ENABLE is supported in LLVM building, we can set the default value to LLVM_BUILD_CET_ENABLE.

[xiongji90]

libunwind/src/UnwindLevel1.c

47

Hi, @compnerd
I think using combination of following:

1. using attribute((noinline)) to decorate target functions
2. building source file containing target functions with -fno-optimize-sibling-calls

can help us to get away with the issue mentioned above. I only found gcc/clang compiling options for "-fno-optimize-sibling-calls" and didn't find corresponding pragma or attribute, so if we choose to use "-fno-optimize-sibling-calls", tail call optimization for other functions will be disabled too which seems to be overshooting.
And if choosing the combination of "noinline" and "-fno-optimize-sibling-calls", we are relying on gcc/clang's implementation which is a implicit dependency, using macro here should be safe to all compilers.

Thanks very much.

[xiongji90]

libunwind/src/UnwindLevel1.c

47

Hi, @compnerd
If you have concern about the macro "unw_phase_resume", I can try to update the patch to use combination of "noinline" and "-fno-optimize-sibling-calls". And do we have any way to use "no-optimize-sibling-calls" to decorate a function? I only found compiling option "-fno-optimize-sibling-calls".
Thanks very much.

[xiongji90]

Hi, @compnerd
I also uploaded a patch to enable building libc++ and libc++abi with CET enabled https://reviews.llvm.org/D107560
libc++ and libc++abi source code doesn't include any assembly, we just need to build the source with "-fcf-protection=full". Could you help review the libc++,libc++abi patch too?
BTW, if using combination of "noinline" and "-fno-optimize-sibling-calls" is the right approach, I will update the patch.
Thanks very much.

[MaskRay]

Have you checked check-cxxabi test/forced_unwind[12].pass.cpp? Do they pass with CET?

libunwind/src/UnwindLevel1.c

47

I think using a macro is a right call here, more robust than dancing with -noinline, no-optimize-sibling-calls and using functions and computing how many additional frames are skipped.

51

Question: why is this 1 + (fn)?

55

Is the calling convention here wrong?

173

223

https://llvm.org/docs/CodingStandards.html#prefer-preincrement

287

libunwind/src/cet_unwind.h

21

Since CET is x86 only, (defined(_LIBUNWIND_TARGET_I386) || defined(_LIBUNWIND_TARGET_X86_64)) can be omitted.

[xiongji90]

In D105968#2932821, @MaskRay wrote:

Have you checked check-cxxabi test/forced_unwind[12].pass.cpp? Do they pass with CET?

Hi, @MaskRay
Yes, all libcxx and libcxxabi case including forced_unwind[12].pass.cpp have been tested using libunwind. Those "forced_unwind" test cases depend on _Unwind_ForcedUnwind and this patch also supports _Unwind_ForcedUnwind in CET system.
Thanks very much.

[xiongji90]

libunwind/src/UnwindLevel1.c

51

Hi, @MaskRay
This is related to libunwind implementation for _Unwind_RaiseException/Resume and _Unwind_ForcedUnwind.
For example, following is _Unwind_RaiseException code:

/// Called by __cxa_throw. Only returns if there is a fatal error.
_LIBUNWIND_EXPORT _Unwind_Reason_Code
_Unwind_RaiseException(_Unwind_Exception *exception_object) {

_LIBUNWIND_TRACE_API("_Unwind_RaiseException(ex_obj=%p)",
                     (void *)exception_object);
unw_context_t uc;
unw_cursor_t cursor;
__unw_getcontext(&uc);

// Mark that this is a non-forced unwind, so _Unwind_Resume()
// can do the right thing.
exception_object->private_1 = 0;
exception_object->private_2 = 0;

// phase 1: the search phase
_Unwind_Reason_Code phase1 = unwind_phase1(&uc, &cursor, exception_object);
if (phase1 != _URC_NO_REASON)
  return phase1;

// phase 2: the clean up phase
return unwind_phase2(&uc, &cursor, exception_object);

}

In phase2, _Unwind_RaiseException will call unwind_phase2 and will never return if landing pad is found. So, we need to adjust CET shadow stack for "unwind_phase2" too.
In unwind_phase2, we will record how many stack frames we need to skip(starting from _Unwind_RaiseException and unwind_phase2 is not included), store the value into framesWalked. At last, we use _LIBUNWIND_POP_CET_SSP to adjust CET shadow stack by 1 + framesWalked frames.

It seems we can have a clear way to achieve this, initializing framesWalked to 1 instead 0 and add some comments to demonstrate the initialization is for unwind_phase2.
Thanks very much.

55

Hi, @MaskRay
The code here is to emulate x86 calling convention. As we know, x86 uses stack to pass input parameter and store return address.
We will jump to "libunwind_Registers_x86_jumpto" instead of calling it to avoid any change to CET shadow stack.
In libunwind, "libunwind_Registers_x86_jumpto" is a pure x86 assembly function, the implementation expects the stack frame to be a standard x86 function call stack frame like following:

+______________+
+ input param +
+______________+
+ return addr +
+___________ +
.....
libunwind_Registers_x86_jumpto will never return, so the return address stored in stack will not be used. However, the implementation does rely on the stack layout, following code is from __libunwind_Registers_x86_jumpto:

DEFINE_LIBUNWIND_FUNCTION(__libunwind_Registers_x86_jumpto)

2. extern "C" void __libunwind_Registers_x86_jumpto(Registers_x86 *); #
3. On entry:
4. + +
5. +-----------------------+
6. + thread_state pointer +
7. +-----------------------+
8. + return address +
9. +-----------------------+ <-- SP
10. + + _LIBUNWIND_CET_ENDBR movl 4(%esp), %eax
    1. set up eax and ret on new stack location movl 28(%eax), %edx # edx holds new stack pointer subl $8,%edx movl %edx, 28(%eax) movl 0(%eax), %ebx

......
As we can see, we must ensure the input parameter is placed in 4(%esp), 0(%esp) stores dummy return address which will never be used.
So, after pushing input parameter for libunwind_Registers_x86_jumpto into stack, we have to "sub $4, %%esp" in order to emulate pushing return address into stack, the pushing is aimed to make stack layout to meet libunwind_Registers_x86_jumpto's requirement.
Thanks very much.

[xiongji90]

libunwind/src/UnwindLevel1.c

173

Fixed.
Thanks very much.

223

Fixed.
Thanks very much.

287

Fixed.
Thanks very much.

[MaskRay]

Mostly looks good.

libunwind/CMakeLists.txt

185

add_compile_flags.

When LIBUNWIND_ENABLE_CET=on, it is an error if the compiler does not support the options.

libunwind/src/UnwindLevel1.c

55

sub $4, %%esp is for the dummay return address. Haven't used x86-32 assembly for a while and forgot this. Sorry!

You may just write asm instead of __asm__. Other places use asm directly as well.

173

git diff -U0 --no-color 'HEAD^' | clang/tools/clang-format/clang-format-diff.py -i -p1 reformat as needed.

libunwind/src/cet_unwind.h

18

on Linux x86 platforms.

libunwind/test/lit.site.cfg.in

30

perhaps config.x86_cet

[xiongji90]

1. Check if compiler used to build libunwind supports CET compiling options.
2. rename "cet_on" to "x86_cet"

[xiongji90]

libunwind/CMakeLists.txt

185

Hi, @MaskRay
I update the patch to check "LIBUNWIND_SUPPORTS_FCF_PROTECTION_EQ_FULL_FLAG" and "LIBUNWIND_SUPPORTS_MSHSTK_FLAG", if either flag is not set, error will be reported.
Thanks very much.

libunwind/src/UnwindLevel1.c

55

Hi, @MaskRay
I also think using C++ keyword "asm" is better but libunwind source has already used "asm": https://github.com/llvm/llvm-project/blob/main/libunwind/src/config.h#L79 So I aligned with this style.
Thanks very much.

173

Fixed.
Thanks very much.

libunwind/src/cet_unwind.h

18

Fixed.
Thanks very much.

libunwind/test/lit.site.cfg.in

30

Fixed.
Thanks very much.

[MaskRay]

libunwind/src/cet_unwind.h

2

This file probably should just be moved into config.h, then other files don't need to add a new #include.

20

Perhaps rename to _LIBUNWIND_USE_CET

[xiongji90]

Hi, @compnerd
Do you have any other concerns or comments?
Thanks very much.

[MaskRay]

It'd be good to get compnerd's opinion as well.

[compnerd]

Minor comments about the implementation, but I think that this is reasonable.

libunwind/CMakeLists.txt

188

I would recommend SEND_ERROR here instead of FATAL_ERROR. This will allow getting subsequent errors as well.

191

I would recommend SEND_ERROR here instead of FATAL_ERROR.

libunwind/src/UnwindCursor.hpp

456 ↗

(On Diff #365663)

Most of the warnings indicate the name. I would suggest we follow the same convention to help quickly identify the function which is unimplemented.

libunwind/src/UnwindLevel1.c

56

57–58

66

67–68

libunwind/src/cet_unwind.h

11–12

18

[xiongji90]

Hi, @compnerd and @MaskRay
I just updated the patch to fix some code format/comments issue, do I need your approval again to commit this patch?
Thanks very much.

libunwind/CMakeLists.txt

188

Done.

Thanks very much.

191

Done.

Thanks very much.

libunwind/src/cet_unwind.h

11–12

Done.

Thanks very much.

18

Done.

Thanks very much.

[MaskRay]

There are still many unresolved comments. Please mark resolved ones as done and then click "Submit".

[xiongji90]

libunwind/src/UnwindCursor.hpp

456 ↗

(On Diff #365663)

Done.

Thanks very much.

libunwind/src/UnwindLevel1.c

57–58

Done.

Thanks very much.

67–68

Done.

Thanks very much.

[xiongji90]

Rebase the patch to latest llvm and address some tiny issues.

[xiongji90]

In D105968#2961508, @MaskRay wrote:

There are still many unresolved comments. Please mark resolved ones as done and then click "Submit".

Hi, @MaskRay
Left unresolved comments are resolved and I also rebased the patch to latest llvm libunwind code.

Thanks very much.

[MaskRay]

In D105968#2962009, @xiongji90 wrote:

In D105968#2961508, @MaskRay wrote:

There are still many unresolved comments. Please mark resolved ones as done and then click "Submit".

Hi, @MaskRay
Left unresolved comments are resolved and I also rebased the patch to latest llvm libunwind code.

Thanks very much.

You marked your own comments as "Done" but not reviewers' comments as "Done".

So I currently see 35 among 79 comments are done, while the rest are not.

Please mark resolved ones as done.

If you need a reviewer to commit this patch on your behalf, please provide git commit --amend --author="name <email>".

[xiongji90]

In D105968#2963188, @MaskRay wrote:

In D105968#2962009, @xiongji90 wrote:

In D105968#2961508, @MaskRay wrote:

There are still many unresolved comments. Please mark resolved ones as done and then click "Submit".

Hi, @MaskRay
Left unresolved comments are resolved and I also rebased the patch to latest llvm libunwind code.

Thanks very much.

You marked your own comments as "Done" but not reviewers' comments as "Done".

So I currently see 35 among 79 comments are done, while the rest are not.

Please mark resolved ones as done.

If you need a reviewer to commit this patch on your behalf, please provide git commit --amend --author="name <email>".

Hi, @MaskRay

Sorry for the mistake, I just marked all reviewers' comments as "Done". I just got the commit access and will commit the code soon.

Thanks very much for your kind help.

libunwind/src/cet_unwind.h

2

Hi, @MaskRay
I suggest to keep cet_unwind.h currently since CET is still a narrow used hardware feature right now and all macro definitions in config.h seems to have general usage. In the long term, if CET is enabled by default in x86/x86_64 platforms, we can move the contents in cet_unwind.h into config.h.

Thanks very much.

20

Done.
Thanks very much.

libunwind/CMakeLists.txt

   set(LLVM_LIT "${CMAKE_SOURCE_DIR}/utils/lit/lit.py")
 endif()
 
+if (LLVM_BUILD_CET_ENABLE)
+set(LIBUNWIND_CET_ENABLE True)
+else()
+set(LIBUNWIND_CET_ENABLE False)
+endif()
 #===============================================================================
 # Setup CMake Options
 #===============================================================================

 endif()
 
 add_compile_flags_if_supported(-Werror=return-type)
+if (LIBUNWIND_CET_ENABLE)
+  add_compile_flags_if_supported(-fcf-protection=full)
+  add_compile_flags_if_supported(-mshstk)
+endif()
 
 # Get warning flags
 add_compile_flags_if_supported(-W)

libunwind/src/CMakeLists.txt

     AddressSpace.hpp
     assembly.h
     CompactUnwinder.hpp
+    cet_unwind.h
     config.h
     dwarf2.h
     DwarfInstructions.hpp

libunwind/src/Registers.hpp

 #include <stdint.h>
 #include <string.h>
 
-#include "libunwind.h"
+#include "cet_unwind.h"
 #include "config.h"
+#include "libunwind.h"
 
 namespace libunwind {
 

 inline void Registers_x86::setVectorRegister(int, v128) {
   _LIBUNWIND_ABORT("no x86 vector registers");
 }
+
+#if defined(_LIBUNWIND_CET_ENABLED)
+void _LIBUNWIND_HIDDEN cetGetRegs_X86(unw_cursor_t *cursor,
+                                      Registers_x86 *regs) {
+  if (!cursor || !regs) {
+    _LIBUNWIND_ABORT("Invalid cursor or regs when trying to get x86_64 regs in "
+                     "CET environment");
+  }
+
+  int regNum;
+  unw_word_t regValue = 0;
+  for (regNum = UNW_X86_EAX; regNum <= UNW_X86_EDI; ++regNum) {
+    unw_get_reg(cursor, regNum, &regValue);
+    regs->setRegister(regNum, regValue);
+  }
+
+  unw_get_reg(cursor, UNW_REG_IP, &regValue);
+  regs->setRegister(UNW_REG_IP, regValue);
+}
+
+extern "C" void __libunwind_cet_jumpto(unw_cursor_t *cursor,
+                                       unsigned int framesSkipped) {
+  Registers_x86 x86_Regs;
+  cetGetRegs_X86(cursor, &x86_Regs);
+  void *cetRegContext = &x86_Regs;
+  void *cetJumpAddress = (void *)&__libunwind_Registers_x86_jumpto;
+  // In CET enabled environment, each function call will also push return
+  // address into shadow stack for further check. We must adjust the shadow
+  // stack when unwinding stack frames.
+  _LIBUNWIND_POP_CET_SSP(framesSkipped + 1);
+  __asm__ volatile("push %%edi\n\t"
+                   "sub $4, %%esp\n\t"
+                   "jmp *%%edx\n\t" ::"D"(cetRegContext),
+                   "d"(cetJumpAddress)
+                   :);
+}
+#endif // _LIBUNWIND_CET_ENABLED
 #endif // _LIBUNWIND_TARGET_I386
 
 

   _LIBUNWIND_ABORT("no x86_64 vector registers");
 #endif
 }
+
+#if defined(_LIBUNWIND_CET_ENABLED)
+void _LIBUNWIND_HIDDEN cetGetRegs_X86_64(unw_cursor_t *cursor,
+                                         Registers_x86_64 *regs) {
+  if (!cursor || !regs) {
+    _LIBUNWIND_ABORT("Invalid cursor or regs when trying to get x86_64 regs in "
+                     "CET environment");
+  }
+
+  int regNum;
+  unw_word_t regValue = 0;
+  for (regNum = UNW_X86_64_RAX; regNum < UNW_X86_64_RIP; ++regNum) {
+    unw_get_reg(cursor, regNum, &regValue);
+    regs->setRegister(regNum, regValue);
+  }
+
+  unw_get_reg(cursor, UNW_REG_IP, &regValue);
+  regs->setRegister(UNW_REG_IP, regValue);
+}
+
+extern "C" void __libunwind_cet_jumpto(unw_cursor_t *cursor,
+                                       unsigned int framesSkipped) {
+  Registers_x86_64 x86_64_Regs;
+  cetGetRegs_X86_64(cursor, &x86_64_Regs);
+  void *cetRegContext = &x86_64_Regs;
+  void *cetJumpAddress = (void *)&__libunwind_Registers_x86_64_jumpto;
+  _LIBUNWIND_POP_CET_SSP(framesSkipped + 1);
+  __asm__ volatile("jmpq *%%rdx\n\t" ::"D"(cetRegContext), "d"(cetJumpAddress)
+                   :);
+}
+
+#endif // _LIBUNWIND_CET_ENABLED
 #endif // _LIBUNWIND_TARGET_X86_64
 
 

libunwind/src/UnwindLevel1.c

 #include <stdio.h>
 #include <string.h>
 
+#include "cet_unwind.h"
 #include "config.h"
 #include "libunwind.h"
 #include "libunwind_ext.h"

 
 #ifndef _LIBUNWIND_SUPPORT_SEH_UNWIND
 
+#if defined(_LIBUNWIND_CET_ENABLED)
+#define __unw_phase2_resume(cursor, frames_skipped)                            \
+  do {                                                                         \
+    _LIBUNWIND_POP_CET_SSP(1);                                                 \
+    __libunwind_cet_jumpto((cursor), (frames_skipped));                        \
+  } while (0)
+#else
+#define __unw_phase2_resume(cursor, frames_skipped) __unw_resume((cursor))
+#endif
+
 static _Unwind_Reason_Code
 unwind_phase1(unw_context_t *uc, unw_cursor_t *cursor, _Unwind_Exception *exception_object) {
   __unw_init_local(cursor, uc);

   _LIBUNWIND_TRACE_UNWINDING("unwind_phase2(ex_ojb=%p)",
                              (void *)exception_object);
 
+  unsigned int framesWalked = 0;
   // Walk each frame until we reach where search phase said to stop.
   while (true) {
 

                                  frameInfo.handler);
     }
 
+    framesWalked++;
     // If there is a personality routine, tell it we are unwinding.
     if (frameInfo.handler != 0) {
       _Unwind_Personality_Fn p =

                                      ", sp=0x%" PRIxPTR,
                                      (void *)exception_object, pc, sp);
         }
-        __unw_resume(cursor);
-        // __unw_resume() only returns if there was an error.
+
+        __unw_phase2_resume(cursor, framesWalked);
+        // __unw_phase2_resume() only returns if there was an error.
         return _URC_FATAL_PHASE2_ERROR;
       default:
         // Personality routine returned an unknown result code.

                      _Unwind_Stop_Fn stop, void *stop_parameter) {
   __unw_init_local(cursor, uc);
 
+  unsigned int framesWalked = 0;
   // Walk each frame until we reach where search phase said to stop
   while (__unw_step(cursor) > 0) {
 

       return _URC_FATAL_PHASE2_ERROR;
     }
 
+    framesWalked++;
     // If there is a personality routine, tell it we are unwinding.
     if (frameInfo.handler != 0) {
       _Unwind_Personality_Fn p =

                                    "personality returned "
                                    "_URC_INSTALL_CONTEXT",
                                    (void *)exception_object);
+
         // We may get control back if landing pad calls _Unwind_Resume().
-        __unw_resume(cursor);
+        __unw_phase2_resume(cursor, framesWalked);
         break;
       default:
         // Personality routine returned an unknown result code.

libunwind/src/UnwindRegistersRestore.S

 #  + return address        +
 #  +-----------------------+   <-- SP
 #  +                       +
+  _LIBUNWIND_CET_ENDBR
   movl   4(%esp), %eax
   # set up eax and ret on new stack location
   movl  28(%eax), %edx # edx holds new stack pointer

   # skip ss
   # skip eflags
   pop    %eax  # eax was already pushed on new stack
-  ret        # eip was already pushed on new stack
+  pop    %ecx
+  jmp    *%ecx
   # skip cs
   # skip ds
   # skip es

 # On entry, thread_state pointer is in rdi
 #endif
 
+  _LIBUNWIND_CET_ENDBR
   movq  56(%rdi), %rax # rax holds new stack pointer
   subq  $16, %rax
   movq  %rax, 56(%rdi)

 #endif
   movq  56(%rdi), %rsp  # cut back rsp to new location
   pop    %rdi      # rdi was saved here earlier
-  ret            # rip was saved here
-
+  addq   $8, %rsp
+  jmpq  *-8(%rsp)
 
 #elif defined(__powerpc64__)
 

libunwind/src/UnwindRegistersSave.S

 #   +                       +
 #
 DEFINE_LIBUNWIND_FUNCTION(__unw_getcontext)
+  _LIBUNWIND_CET_ENDBR
   push  %eax
   movl  8(%esp), %eax
   movl  %ebx,  4(%eax)

 #define TMP %rsi
 #endif
 
+  _LIBUNWIND_CET_ENDBR
   movq  %rax,   (PTR)
   movq  %rbx,  8(PTR)
   movq  %rcx, 16(PTR)

libunwind/src/assembly.h

 #ifndef UNWIND_ASSEMBLY_H
 #define UNWIND_ASSEMBLY_H
 
+#if (defined(__i386__) || defined(__x86_64__)) && defined(__linux__)
+#include <cet.h>
+#define _LIBUNWIND_CET_ENDBR _CET_ENDBR
+#else
+#define _LIBUNWIND_CET_ENDBR
+#endif
+
 #if defined(__powerpc64__)
 #define SEPARATOR ;
 #define PPC64_OFFS_SRR0   0

libunwind/src/cet_unwind.h

+//===--------------------------- cet_unwind.h -----------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef _CET_UNWIND_H_
+#define _CET_UNWIND_H_
+#include "libunwind.h"
+#if defined(_LIBUNWIND_TARGET_LINUX) &&                                        \
+    (defined(_LIBUNWIND_TARGET_I386) || defined(_LIBUNWIND_TARGET_X86_64)) &&  \
+    defined(__CET__) && defined(__SHSTK__)
+#define _LIBUNWIND_CET_ENABLED 1
+#include <cet.h>
+#include <immintrin.h>
+
+#define _LIBUNWIND_POP_CET_SSP(x)                                              \
+  do {                                                                         \
+    unsigned long ssp = _get_ssp();                                            \
+    if (ssp != 0) {                                                            \
+      unsigned int tmp = (x);                                                  \
+      if (tmp < 256)                                                           \
+        _inc_ssp(tmp);                                                         \
+      else {                                                                   \
+        do {                                                                   \
+          _inc_ssp(255);                                                       \
+          tmp -= 255;                                                          \
+        } while (tmp > 255);                                                   \
+        _inc_ssp(tmp);                                                         \
+      }                                                                        \
+    }                                                                          \
+  } while (0)
+#endif
+
+extern void __libunwind_cet_jumpto(unw_cursor_t *, unsigned int);
+#endif

libunwind/test/libunwind/test/config.py

         if not self.get_lit_bool('enable_threads', True):
             self.cxx.compile_flags += ['-D_LIBUNWIND_HAS_NO_THREADS']
             self.config.available_features.add('libunwind-no-threads')
+        if self.get_lit_bool('cet_build'):
+            self.cxx.compile_flags += ['-fcf-protection=full']
         super(Configuration, self).configure_compile_flags()
 
     def configure_compile_flags_header_includes(self):

libunwind/test/lit.site.cfg.in

 config.sysroot                  = "@LIBUNWIND_SYSROOT@"
 config.gcc_toolchain            = "@LIBUNWIND_GCC_TOOLCHAIN@"
 config.cxx_ext_threads          = @LIBUNWIND_BUILD_EXTERNAL_THREAD_LIBRARY@
+config.cet_build                = @LIBUNWIND_CET_ENABLE@
 
 site.addsitedir(os.path.join(config.libunwind_src_root, 'test'))
 site.addsitedir(os.path.join(config.libcxx_src_root, 'utils'))