[libunwind] Unwind through Linux riscv sigreturn trampoline

[wf520gg]

Summary

Similar to D90898 (Linux AArch64) and D124765 (SystemZ).

On an Arch Linux RISC-V (riscv64gc), the following code

#define _GNU_SOURCE
#include <dlfcn.h>
#include <libunwind.h>
#include <signal.h>
#include <stdio.h>
#include <stdlib.h>

static void handler(int signo) {
  unw_context_t context;
  unw_cursor_t cursor;
  unw_getcontext(&context);
  unw_init_local(&cursor, &context);
  unw_word_t pc, sp;
  do {
    unw_get_reg(&cursor, UNW_REG_IP, &pc);
    unw_get_reg(&cursor, UNW_REG_SP, &sp);
    printf("pc=0x%016zx sp=0x%016zx", (size_t)pc, (size_t)sp);
    Dl_info info = {};
    if (dladdr((void *)pc, &info))
      printf(" %s:%s", info.dli_fname, info.dli_sname ? info.dli_sname : "");
    puts("");
  } while (unw_step(&cursor) > 0);
  exit(0);
}

int main() {
  signal(SIGUSR1, handler);
  raise(SIGUSR1);
  return 1;
}

linked with -Wl,--export-dynamic gives an output like

pc=0x0000000000010a82 sp=0x00007fffd8a0b910 ./b:
pc=0x00007fffa7e77800 sp=0x00007fffd8a0c520 linux-vdso.so.1:__vdso_rt_sigreturn
pc=0x00007fffa7d73bee sp=0x00007fffd8a0c960 /usr/lib/libc.so.6:
pc=0x00007fffa7d3ed66 sp=0x00007fffd8a0c9b0 /usr/lib/libc.so.6:gsignal
pc=0x0000000000010a3c sp=0x00007fffd8a0c9c0 ./b:main
pc=0x00007fffa7d2f1d4 sp=0x00007fffd8a0c9e0 /usr/lib/libc.so.6:
pc=0x00007fffa7d2f27c sp=0x00007fffd8a0cb10 /usr/lib/libc.so.6:__libc_start_main
pc=0x00000000000109a0 sp=0x00007fffd8a0cb60 ./b:_start

Co-Authored-By: Fangrui Song <[e-mail]>

[wf520gg]

Fix clang-format issue

[wf520gg]

Remove redundant assignments

[wf520gg]

Set PC reg

[MaskRay]

Do you know how to test this? The code looks reasonable, but I'd like to reproduce the issue... I may need a RISC-V image (with gcc,gdb,etc) which can run in qemu-system. On the non-riscv host I'll need some LLVM_ENABLE_RUNTIMES way to cross compile for RISC-V.

libunwind/src/UnwindCursor.hpp

2916

delete this blank line

[wf520gg]

In D148499#4279283, @MaskRay wrote:

Do you know how to test this? The code looks reasonable, but I'd like to reproduce the issue... I may need a RISC-V image (with gcc,gdb,etc) which can run in qemu-system. On the non-riscv host I'll need some LLVM_ENABLE_RUNTIMES way to cross compile for RISC-V.

Hi MaskRay,
Thanks for your review.This issue happend when I tested the bionic unwind test case in the RISC-V Android environment. The test case file is stack_unwinding_test.cpp based on gtest(path is bionic/test/), I upload some part of this file{F27186091}.Please check it. If you want to reporduce it, I can show the command.Thanks!

[wf520gg]

Delete the blank line

[wf520gg]

libunwind/src/UnwindCursor.hpp

2916

done

[MaskRay]

Please adjust libunwind/test/signal_unwind.pass.cpp and ensure it passes.

libunwind/src/UnwindCursor.hpp

2899

Where is __default_sa_restorer defined? Isn't it only used by Linux arm?

2922

This causes
error: use of undeclared identifier 'siginfo_t'
on glibc riscv64.

[wf520gg]

libunwind/src/UnwindCursor.hpp

2899

Yes,you are right,just arm has this defination.There is some ambiguity about this comment, that should be "A signal frame will have a return address pointing to sigreturn syscall just like arm __default_sa_restorer."。

2922

I will add riscv arch for signal_unwind.pass.cpp. Could you tell me how to test it? or is there any doc about how to run the libunwind test?I haven't found it.Thanks.

[wf520gg]

I'm building and testing the code. After add test case will upload new patch. Thanks.

[MaskRay]

libunwind/src/UnwindCursor.hpp

2899

FYI: I tested the first a.c example on https://maskray.me/blog/2022-04-10-unwinding-through-signal-handler on a very new Arch Linux RISC-V qemu image and I don't find that this patch fixes the unwinding.

For Linux, IIUC newer kernel ports place the signal trampoline in vdso instead of libc. __vdso_rt_sigreturn is more accurate.

[wf520gg]

I change some part of a.c code accroding to my test case, and then the problem can be reproduced, please check the file{F27260437}.
Without this change, the back trace only get one layer.

[wf520gg]

Modify some comment and

[MaskRay]

Fix link (prefer Linux kernel's to glibc's)
Don't set UNW_RISCV_X0
Use _registers.getSP()

[MaskRay]

Move riscv code before s390x for an alphabetical order

[MaskRay]

LGTM.

libunwind/src/UnwindCursor.hpp

2899

This is __vdso_rt_sigreturn, not __vdso_rt_sigteturn. I'll fix it.

The canonical term is sigreturn trampoline (used the most in the Linux kernel), not signal frame.

[MaskRay]

Fix comment. Add an example to the description

libunwind/src/UnwindCursor.hpp

 #endif
 #ifdef _AIX
 #include <dlfcn.h>
 #include <sys/debug.h>
 #include <sys/pseg.h>
 #endif
 
 #if defined(_LIBUNWIND_TARGET_LINUX) &&                                        \
-    (defined(_LIBUNWIND_TARGET_AARCH64) || defined(_LIBUNWIND_TARGET_S390X))
+    (defined(_LIBUNWIND_TARGET_AARCH64) || defined(_LIBUNWIND_TARGET_RISCV) || \
+     defined(_LIBUNWIND_TARGET_S390X))
 #include <sys/syscall.h>
 #include <sys/uio.h>
 #include <unistd.h>
 #define _LIBUNWIND_CHECK_LINUX_SIGRETURN 1
 #endif
 
 #include "AddressSpace.hpp"
 #include "CompactUnwinder.hpp"
#if defined(_LIBUNWIND_CHECK_LINUX_SIGRETURN)
   int stepThroughSigReturn() {
     R dummy;
     return stepThroughSigReturn(dummy);
   }
 #if defined(_LIBUNWIND_TARGET_AARCH64)
   bool setInfoForSigReturn(Registers_arm64 &);
   int stepThroughSigReturn(Registers_arm64 &);
 #endif
+#if defined(_LIBUNWIND_TARGET_RISCV)
+  bool setInfoForSigReturn(Registers_riscv &);
+  int stepThroughSigReturn(Registers_riscv &);
+#endif
 #if defined(_LIBUNWIND_TARGET_S390X)
   bool setInfoForSigReturn(Registers_s390x &);
   int stepThroughSigReturn(Registers_s390x &);
 #endif
   template <typename Registers> bool setInfoForSigReturn(Registers &) {
     return false;
   }
   template <typename Registers> int stepThroughSigReturn(Registers &) {
int UnwindCursor<A, R>::stepThroughSigReturn(Registers_arm64 &) {
   _registers.setIP(_addressSpace.get64(sigctx + kOffsetPc));
   _isSignalFrame = true;
   return UNW_STEP_SUCCESS;
 }
 #endif // defined(_LIBUNWIND_CHECK_LINUX_SIGRETURN) &&
        // defined(_LIBUNWIND_TARGET_AARCH64)
 
 #if defined(_LIBUNWIND_CHECK_LINUX_SIGRETURN) &&                               \
+    defined(_LIBUNWIND_TARGET_RISCV)
+template <typename A, typename R>
+bool UnwindCursor<A, R>::setInfoForSigReturn(Registers_riscv &) {
+  const pint_t pc = static_cast<pint_t>(getReg(UNW_REG_IP));
+  uint32_t instructions[2];
+  struct iovec local_iov = {&instructions, sizeof instructions};
+  struct iovec remote_iov = {reinterpret_cast<void *>(pc), sizeof instructions};
+  long bytesRead =
+      syscall(SYS_process_vm_readv, getpid(), &local_iov, 1, &remote_iov, 1, 0);
+  // Look for the two instructions used in the sigreturn trampoline
+  // __vdso_rt_sigreturn:
+  //
+  // 0x08b00893 li a7,0x8b
+  // 0x00000073 ecall
+  if (bytesRead != sizeof instructions || instructions[0] != 0x08b00893 ||
+      instructions[1] != 0x00000073)
+    return false;
+
+  _info = {};
+  _info.start_ip = pc;
+  _info.end_ip = pc + 4;
+  _isSigReturn = true;
+  return true;
+}
+
+template <typename A, typename R>
+int UnwindCursor<A, R>::stepThroughSigReturn(Registers_riscv &) {
+  // In the signal trampoline frame, sp points to an rt_sigframe[1], which is:
+  //  - 128-byte siginfo struct
+  //  - ucontext_t struct:
+  //     - 8-byte long (__uc_flags)
+  //     - 8-byte pointer (*uc_link)
+  //     - 24-byte uc_stack
+  //     - 8-byte uc_sigmask
+  //     - 120-byte of padding to allow sigset_t to be expanded in the future
+  //     - 8 bytes of padding because sigcontext has 16-byte alignment
+  //     - struct sigcontext uc_mcontext
+  // [1]
+  // https://github.com/torvalds/linux/blob/master/arch/riscv/kernel/signal.c
+  const pint_t kOffsetSpToSigcontext = 128 + 8 + 8 + 24 + 8 + 128;
+
+  const pint_t sigctx = _registers.getSP() + kOffsetSpToSigcontext;
+  _registers.setIP(_addressSpace.get64(sigctx));
+  for (int i = UNW_RISCV_X1; i <= UNW_RISCV_X31; ++i) {
+    uint64_t value = _addressSpace.get64(sigctx + static_cast<pint_t>(i * 8));
+    _registers.setRegister(i, value);
+  }
+  _isSignalFrame = true;
+  return UNW_STEP_SUCCESS;
+}
+#endif // defined(_LIBUNWIND_CHECK_LINUX_SIGRETURN) &&
+       // defined(_LIBUNWIND_TARGET_RISCV)
+
+#if defined(_LIBUNWIND_CHECK_LINUX_SIGRETURN) &&                               \
     defined(_LIBUNWIND_TARGET_S390X)
 template <typename A, typename R>
 bool UnwindCursor<A, R>::setInfoForSigReturn(Registers_s390x &) {
   // Look for the sigreturn trampoline. The trampoline's body is a
   // specific instruction (see below). Typically the trampoline comes from the
   // vDSO (i.e. the __kernel_[rt_]sigreturn function). A libc might provide its
   // own restorer function, though, or user-mode QEMU might write a trampoline
   // onto the stack.
template <typename A, typename R> int UnwindCursor<A, R>::step(bool stage2) {
   if (_unwindInfoMissing)
     return UNW_STEP_END;
 
   // Use unwinding info to modify register set as if function returned.
   int result;
 #if defined(_LIBUNWIND_CHECK_LINUX_SIGRETURN)
   if (_isSigReturn) {
     result = this->stepThroughSigReturn();
   } else
 #endif
   {
 #if defined(_LIBUNWIND_SUPPORT_COMPACT_UNWIND)
     result = this->stepWithCompactEncoding(stage2);
 #elif defined(_LIBUNWIND_SUPPORT_SEH_UNWIND)
     result = this->stepWithSEHData();
 #elif defined(_LIBUNWIND_SUPPORT_TBTAB_UNWIND)
     result = this->stepWithTBTableData();
 #elif defined(_LIBUNWIND_SUPPORT_DWARF_UNWIND)
     result = this->stepWithDwarfFDE(stage2);
 #elif defined(_LIBUNWIND_ARM_EHABI)
     result = this->stepWithEHABI();
 #else
   #error Need _LIBUNWIND_SUPPORT_COMPACT_UNWIND or \
               _LIBUNWIND_SUPPORT_SEH_UNWIND or \
               _LIBUNWIND_SUPPORT_DWARF_UNWIND or \
               _LIBUNWIND_ARM_EHABI
 #endif
   }
 
   // update info based on new PC
   if (result == UNW_STEP_SUCCESS) {
     this->setInfoBasedOnIPRegister(true);
     if (_unwindInfoMissing)
       return UNW_STEP_END;
   }
 
   return result;
 }
 
 template <typename A, typename R>

libunwind/test/signal_unwind.pass.cpp

 // -*- C++ -*-
 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // Ensure that the unwinder can cope with the signal handler.
-// REQUIRES: target={{(aarch64|s390x|x86_64)-.+linux.*}}
+// REQUIRES: target={{(aarch64|riscv64|s390x|x86_64)-.+linux.*}}
 
 // TODO: Figure out why this fails with Memory Sanitizer.
 // XFAIL: msan
 
 #undef NDEBUG
 #include <assert.h>
 #include <dlfcn.h>
 #include <signal.h>

libunwind/test/unwind_leaffunction.pass.cpp

 // -*- C++ -*-
 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // Ensure that leaf function can be unwund.
-// REQUIRES: target={{(aarch64|s390x|x86_64)-.+linux.*}}
+// REQUIRES: target={{(aarch64|riscv64|s390x|x86_64)-.+linux.*}}
 
 // TODO: Figure out why this fails with Memory Sanitizer.
 // XFAIL: msan
 
 #undef NDEBUG
 #include <assert.h>
 #include <dlfcn.h>
 #include <signal.h>