[libunwind][ARM] Improve unwinder stack usage on baremetal targets - part 1

[rmaprath]

Summary

Currently, unw_context_t and unw_cursor_t are sized to allow the virtual register set of any target architecuture supported by libunwind. This does not work well for baremetal ARM targets (where memory is at a premium).

This patch makes unw_context_t and unw_cursor_t use just enough space to hold the ARM virtual register set on baremetal arm targets.

The various Registers_xxx definitions (and their dependencies) had to be conditionally compiled out in order to allow each of the targets to perform size checks independently.

This change saves about 1500 bytes of stack on ARM v7 (similar amount on AArch32).

[t.p.northover]

I think it would be better to use a generic method so the size is minimal everywhere rather than making ARM a special case. Possibly move the #define bits from libunwind.cpp to __libunwind_config.h and use REGISTER_KIND to declare unw_context_t?

src/CompactUnwinder.hpp

490

__aarch64__ is defined for iOS too, so no need for both.

[jroelofs]

Wouldn't this break cross unwinding?

[rengolin]

I agree with Tim, I think this should be more generic.

[rmaprath]

In D19920#421131, @t.p.northover wrote:

I think it would be better to use a generic method so the size is minimal everywhere rather than making ARM a special case. Possibly move the #define bits from libunwind.cpp to __libunwind_config.h and use REGISTER_KIND to declare unw_context_t?

Wouldn't this make libunwind.h depend on Registers_xxx types? I thought the whole idea of statically allocated buffers was to keep these two separate. I may be wrong though, pretty new to unwinder.

Thanks.

/ Asiri

[rmaprath]

In D19920#421145, @jroelofs wrote:

Wouldn't this break cross unwinding?

I wasn't aware of cross unwinding, I think you are referring to [1]. Thanks for the pointer.

Would it make sense to support a libunwind build that only supports native unwinding? For baremetal (embedded) applications, I don't think cross unwinding (if I understand it correctly) makes a lot of sense.

Thanks!

/ Asiri

[1] http://www.nongnu.org/libunwind/man/libunwind(3).html#section_4

[jroelofs]

In D19920#421173, @rmaprath wrote:

In D19920#421145, @jroelofs wrote:

Wouldn't this break cross unwinding?

I wasn't aware of cross unwinding, I think you are referring to [1]. Thanks for the pointer.

Would it make sense to support a libunwind build that only supports native unwinding? For baremetal (embedded) applications, I don't think cross unwinding (if I understand it correctly) makes a lot of sense.

Thanks!

/ Asiri

[1] http://www.nongnu.org/libunwind/man/libunwind(3).html#section_4

Yeah, that's what I'm referring to. I have no idea if it's currently working or not, but *adding* a build mode that only supports native unwinding sounds like a good idea.

[rmaprath]

In D19920#421189, @jroelofs wrote:

In D19920#421173, @rmaprath wrote:

In D19920#421145, @jroelofs wrote:

Wouldn't this break cross unwinding?

I wasn't aware of cross unwinding, I think you are referring to [1]. Thanks for the pointer.

Would it make sense to support a libunwind build that only supports native unwinding? For baremetal (embedded) applications, I don't think cross unwinding (if I understand it correctly) makes a lot of sense.

Thanks!

/ Asiri

[1] http://www.nongnu.org/libunwind/man/libunwind(3).html#section_4

Yeah, that's what I'm referring to. I have no idea if it's currently working or not, but *adding* a build mode that only supports native unwinding sounds like a good idea.

Makes sense. I will update the patch to hide behind a build option.

@rengolin, @t.p.northover: From what I gathered so far, libunwind.h is supposed to be platform independent, so we cannot make the size computation generic as suggested (without tying up libunwind.h into library sources).

We could, on the other hand, do this tightening for all the supported architectures (for the new, native-only libunwind build suggested by @jroelofs) with appropriate asserts in place so that we maintain these tight bounds as we move forward. Not sure if that effort would be worth though, given that for most non-ARM targets, unwinder stack usage is not a huge concern.

Cheers,

/ Asiri

[bcraig]

We could, on the other hand, do this tightening for all the supported architectures (for the new, native-only libunwind build suggested by @jroelofs) with appropriate asserts in place so that we maintain these tight bounds as we move forward. Not sure if that effort would be worth though, given that for most non-ARM targets, unwinder stack usage is not a huge concern.

libunwind doesn't currently support Hexagon, but it has been investigated some there. Hexagon would care about stack usage as well. I wouldn't be surprised if some embedded MIPS and Power targets also cared. I don't think any of those targets come close to using 1K for their unwind context.

[rmaprath]

In D19920#421230, @bcraig wrote:

We could, on the other hand, do this tightening for all the supported architectures (for the new, native-only libunwind build suggested by @jroelofs) with appropriate asserts in place so that we maintain these tight bounds as we move forward. Not sure if that effort would be worth though, given that for most non-ARM targets, unwinder stack usage is not a huge concern.

libunwind doesn't currently support Hexagon, but it has been investigated some there. Hexagon would care about stack usage as well. I wouldn't be surprised if some embedded MIPS and Power targets also cared. I don't think any of those targets come close to using 1K for their unwind context.

Right, in that case, I will do the tightening for all the existing architectures. Will add a comment explaining the expected behaviour for future changes as well.

Thanks.

/ Asiri

[rmaprath]

Superseded by D20119.

include/libunwind.h

   UNW_EINVALIDIP    = -6545, /* invalid IP */
   UNW_EBADFRAME     = -6546, /* bad frame */
   UNW_EINVAL        = -6547, /* unsupported operation or bad value */
   UNW_EBADVERSION   = -6548, /* unwind info has unsupported version */
   UNW_ENOINFO       = -6549  /* no unwind info found */
 };
 
 struct unw_context_t {
+#if defined(__arm__) && defined(_LIBUNWIND_IS_BAREMETAL)
+  // For bare-metal arm targets, use just enough space to hold the
+  // virtual register set (Registers_arm). This helps limit unwinder's
+  // stack usage.
+  uint64_t data[60];
+#else
   uint64_t data[128];
+#endif
 };
 typedef struct unw_context_t unw_context_t;
 
 struct unw_cursor_t {
+#if defined(__arm__) && defined(_LIBUNWIND_IS_BAREMETAL)
+  // As before (unw_context_t), use just enough space to hold the
+  // unwind cursor.
+  uint64_t data[68];
+#else
   uint64_t data[140];
+#endif
 };
 typedef struct unw_cursor_t unw_cursor_t;
 
 typedef struct unw_addr_space *unw_addr_space_t;
 
 typedef int unw_regnum_t;
 #if _LIBUNWIND_ARM_EHABI
 typedef uint32_t unw_word_t;

src/CompactUnwinder.hpp

 #include "AddressSpace.hpp"
 #include "Registers.hpp"
 
 #define EXTRACT_BITS(value, mask)                                              \
   ((value >> __builtin_ctz(mask)) & (((1 << __builtin_popcount(mask))) - 1))
 
 namespace libunwind {
 
+#if defined(__i386__)
 /// CompactUnwinder_x86 uses a compact unwind info to virtually "step" (aka
 /// unwind) by modifying a Registers_x86 register set
 template <typename A>
 class CompactUnwinder_x86 {
 public:
 
   static int stepWithCompactEncoding(compact_unwind_encoding_t info,
                                      uint32_t functionStart, A &addressSpace,

     A &addressSpace, typename A::pint_t returnAddressLocation,
     Registers_x86 &registers) {
   // return address is on stack after last saved register
   registers.setIP(addressSpace.get32(returnAddressLocation));
   // old esp is before return address
   registers.setSP((uint32_t)returnAddressLocation + 4);
 }
 
-
+#elif defined(__x86_64__)
 /// CompactUnwinder_x86_64 uses a compact unwind info to virtually "step" (aka
 /// unwind) by modifying a Registers_x86_64 register set
 template <typename A>
 class CompactUnwinder_x86_64 {
 public:
 
   static int stepWithCompactEncoding(compact_unwind_encoding_t compactEncoding,
                                      uint64_t functionStart, A &addressSpace,

                                                 Registers_x86_64 &registers) {
   // return address is on stack after last saved register
   registers.setIP(addressSpace.get64(returnAddressLocation));
   // old esp is before return address
   registers.setSP(returnAddressLocation + 8);
 }
 
 
-
+#elif defined(__arm64__) || defined(__aarch64__)
 /// CompactUnwinder_arm64 uses a compact unwind info to virtually "step" (aka
 /// unwind) by modifying a Registers_arm64 register set
 template <typename A>
 class CompactUnwinder_arm64 {
 public:
 
   static int stepWithCompactEncoding(compact_unwind_encoding_t compactEncoding,
                                      uint64_t functionStart, A &addressSpace,

   registers.setFP(addressSpace.get64(fp));
   // old sp is fp less saved fp and lr
   registers.setSP(fp + 16);
   // pop return address into pc
   registers.setIP(addressSpace.get64(fp + 8));
 
   return UNW_STEP_SUCCESS;
 }
+#endif
 
 
 } // namespace libunwind
 
 #endif // __COMPACT_UNWINDER_HPP__

src/Registers.hpp

 #include "libunwind.h"
 #include "config.h"
 
 namespace libunwind {
 
 // For emulating 128-bit registers
 struct v128 { uint32_t vec[4]; };
 
-
+#if defined(__i386__)
 /// Registers_x86 holds the register state of a thread in a 32-bit intel
 /// process.
 class _LIBUNWIND_HIDDEN Registers_x86 {
 public:
   Registers_x86();
   Registers_x86(const void *registers);
 
   bool        validRegister(int num) const;

 inline v128 Registers_x86::getVectorRegister(int) const {
   _LIBUNWIND_ABORT("no x86 vector registers");
 }
 
 inline void Registers_x86::setVectorRegister(int, v128) {
   _LIBUNWIND_ABORT("no x86 vector registers");
 }
 
-
+#elif defined(__x86_64__)
 /// Registers_x86_64  holds the register state of a thread in a 64-bit intel
 /// process.
 class _LIBUNWIND_HIDDEN Registers_x86_64 {
 public:
   Registers_x86_64();
   Registers_x86_64(const void *registers);
 
   bool        validRegister(int num) const;

 inline v128 Registers_x86_64::getVectorRegister(int) const {
   _LIBUNWIND_ABORT("no x86_64 vector registers");
 }
 
 inline void Registers_x86_64::setVectorRegister(int, v128) {
   _LIBUNWIND_ABORT("no x86_64 vector registers");
 }
 
-
+#elif defined(__ppc__)
 /// Registers_ppc holds the register state of a thread in a 32-bit PowerPC
 /// process.
 class _LIBUNWIND_HIDDEN Registers_ppc {
 public:
   Registers_ppc();
   Registers_ppc(const void *registers);
 
   bool        validRegister(int num) const;

   case UNW_PPC_LR:
     return "lr";
   default:
     return "unknown register";
   }
 
 }
 
-
+#elif defined(__arm64__) || defined(__aarch64__)
 /// Registers_arm64  holds the register state of a thread in a 64-bit arm
 /// process.
 class _LIBUNWIND_HIDDEN Registers_arm64 {
 public:
   Registers_arm64();
   Registers_arm64(const void *registers);
 
   bool        validRegister(int num) const;

 inline v128 Registers_arm64::getVectorRegister(int) const {
   _LIBUNWIND_ABORT("no arm64 vector register support yet");
 }
 
 inline void Registers_arm64::setVectorRegister(int, v128) {
   _LIBUNWIND_ABORT("no arm64 vector register support yet");
 }
 
+#elif _LIBUNWIND_ARM_EHABI
 /// Registers_arm holds the register state of a thread in a 32-bit arm
 /// process.
 ///
 /// NOTE: Assumes VFPv3. On ARM processors without a floating point unit,
 /// this uses more memory than required.
 class _LIBUNWIND_HIDDEN Registers_arm {
 public:
   Registers_arm();

 };
 
 inline Registers_arm::Registers_arm(const void *registers)
   : _use_X_for_vfp_save(false),
     _saved_vfp_d0_d15(false),
     _saved_vfp_d16_d31(false),
     _saved_iwmmx(false),
     _saved_iwmmx_control(false) {
-  static_assert(sizeof(Registers_arm) < sizeof(unw_context_t),
+  static_assert(sizeof(Registers_arm) <= sizeof(unw_context_t),
                     "arm registers do not fit into unw_context_t");
   // See unw_getcontext() note about data.
   memcpy(&_registers, registers, sizeof(_registers));
   memset(&_vfp_d0_d15_pad, 0, sizeof(_vfp_d0_d15_pad));
   memset(&_vfp_d16_d31, 0, sizeof(_vfp_d16_d31));
   memset(&_iwmmx, 0, sizeof(_iwmmx));
   memset(&_iwmmx_control, 0, sizeof(_iwmmx_control));
 }

 
 inline v128 Registers_arm::getVectorRegister(int) const {
   _LIBUNWIND_ABORT("ARM vector support not implemented");
 }
 
 inline void Registers_arm::setVectorRegister(int, v128) {
   _LIBUNWIND_ABORT("ARM vector support not implemented");
 }
+
+#elif defined(__or1k__)
 /// Registers_or1k holds the register state of a thread in an OpenRISC1000
 /// process.
 class _LIBUNWIND_HIDDEN Registers_or1k {
 public:
   Registers_or1k();
   Registers_or1k(const void *registers);
 
   bool        validRegister(int num) const;

     return "r30";
   case UNW_OR1K_R31:
     return "r31";
   default:
     return "unknown register";
   }
 
 }
+#else
+#error "Architecture not supported."
+#endif
 } // namespace libunwind
 
 #endif // __REGISTERS_HPP__

src/UnwindCursor.hpp

   #if _LIBUNWIND_SUPPORT_DWARF_UNWIND
     if ( compactSaysUseDwarf() )
       return stepWithDwarfFDE();
   #endif
     R dummy;
     return stepWithCompactEncoding(dummy);
   }
 
+#if defined(__x86_64__)
   int stepWithCompactEncoding(Registers_x86_64 &) {
     return CompactUnwinder_x86_64<A>::stepWithCompactEncoding(
         _info.format, _info.start_ip, _addressSpace, _registers);
   }
 
+#elif defined(__i386__)
   int stepWithCompactEncoding(Registers_x86 &) {
     return CompactUnwinder_x86<A>::stepWithCompactEncoding(
         _info.format, (uint32_t)_info.start_ip, _addressSpace, _registers);
   }
 
+#elif defined(__ppc__)
   int stepWithCompactEncoding(Registers_ppc &) {
     return UNW_EINVAL;
   }
 
+#elif defined(__arm64__) || defined(__aarch64__)
   int stepWithCompactEncoding(Registers_arm64 &) {
     return CompactUnwinder_arm64<A>::stepWithCompactEncoding(
         _info.format, _info.start_ip, _addressSpace, _registers);
   }
+#endif
 
   bool compactSaysUseDwarf(uint32_t *offset=NULL) const {
     R dummy;
     return compactSaysUseDwarf(dummy, offset);
   }
 
+#if defined(__x86_64__)
   bool compactSaysUseDwarf(Registers_x86_64 &, uint32_t *offset) const {
     if ((_info.format & UNWIND_X86_64_MODE_MASK) == UNWIND_X86_64_MODE_DWARF) {
       if (offset)
         *offset = (_info.format & UNWIND_X86_64_DWARF_SECTION_OFFSET);
       return true;
     }
     return false;
   }
 
+#elif defined(__i386__)
   bool compactSaysUseDwarf(Registers_x86 &, uint32_t *offset) const {
     if ((_info.format & UNWIND_X86_MODE_MASK) == UNWIND_X86_MODE_DWARF) {
       if (offset)
         *offset = (_info.format & UNWIND_X86_DWARF_SECTION_OFFSET);
       return true;
     }
     return false;
   }
 
+#elif defined(__ppc__)
   bool compactSaysUseDwarf(Registers_ppc &, uint32_t *) const {
     return true;
   }
 
+#elif defined(__arm64__) || defined(__aarch64__)
   bool compactSaysUseDwarf(Registers_arm64 &, uint32_t *offset) const {
     if ((_info.format & UNWIND_ARM64_MODE_MASK) == UNWIND_ARM64_MODE_DWARF) {
       if (offset)
         *offset = (_info.format & UNWIND_ARM64_DWARF_SECTION_OFFSET);
       return true;
     }
     return false;
   }
+#endif
 #endif // _LIBUNWIND_SUPPORT_COMPACT_UNWIND
 
 #if _LIBUNWIND_SUPPORT_DWARF_UNWIND
   compact_unwind_encoding_t dwarfEncoding() const {
     R dummy;
     return dwarfEncoding(dummy);
   }
 
+#if defined(__x86_64__)
   compact_unwind_encoding_t dwarfEncoding(Registers_x86_64 &) const {
     return UNWIND_X86_64_MODE_DWARF;
   }
 
+#elif defined(__i386__)
   compact_unwind_encoding_t dwarfEncoding(Registers_x86 &) const {
     return UNWIND_X86_MODE_DWARF;
   }
 
+#elif defined(__ppc__)
   compact_unwind_encoding_t dwarfEncoding(Registers_ppc &) const {
     return 0;
   }
 
+#elif defined(__arm64__) || defined(__aarch64__)
   compact_unwind_encoding_t dwarfEncoding(Registers_arm64 &) const {
     return UNWIND_ARM64_MODE_DWARF;
   }
 
+#elif defined (__or1k__)
   compact_unwind_encoding_t dwarfEncoding(Registers_or1k &) const {
     return 0;
   }
+#endif
 #endif // _LIBUNWIND_SUPPORT_DWARF_UNWIND
 
 
   A               &_addressSpace;
   R                _registers;
   unw_proc_info_t  _info;
   bool             _unwindInfoMissing;
   bool             _isSignalFrame;

src/libunwind.cpp

 
 /// Create a cursor of a thread in this process given 'context' recorded by
 /// unw_getcontext().
 _LIBUNWIND_EXPORT int unw_init_local(unw_cursor_t *cursor,
                                      unw_context_t *context) {
   _LIBUNWIND_TRACE_API("unw_init_local(cursor=%p, context=%p)\n",
                        static_cast<void *>(cursor),
                        static_cast<void *>(context));
-  // Use "placement new" to allocate UnwindCursor in the cursor buffer.
 #if defined(__i386__)
-  new ((void *)cursor) UnwindCursor<LocalAddressSpace, Registers_x86>(
-                                 context, LocalAddressSpace::sThisAddressSpace);
+# define REGISTER_KIND Registers_x86
 #elif defined(__x86_64__)
-  new ((void *)cursor) UnwindCursor<LocalAddressSpace, Registers_x86_64>(
-                                 context, LocalAddressSpace::sThisAddressSpace);
+# define REGISTER_KIND Registers_x86_64
 #elif defined(__ppc__)
-  new ((void *)cursor) UnwindCursor<LocalAddressSpace, Registers_ppc>(
-                                 context, LocalAddressSpace::sThisAddressSpace);
+# define REGISTER_KIND Registers_ppc
 #elif defined(__arm64__) || defined(__aarch64__)
-  new ((void *)cursor) UnwindCursor<LocalAddressSpace, Registers_arm64>(
-                                 context, LocalAddressSpace::sThisAddressSpace);
+# define REGISTER_KIND Registers_arm64
 #elif _LIBUNWIND_ARM_EHABI
-  new ((void *)cursor) UnwindCursor<LocalAddressSpace, Registers_arm>(
-                                 context, LocalAddressSpace::sThisAddressSpace);
+# define REGISTER_KIND Registers_arm
 #elif defined(__or1k__)
-  new ((void *)cursor) UnwindCursor<LocalAddressSpace, Registers_or1k>(
-                                 context, LocalAddressSpace::sThisAddressSpace);
+# define REGISTER_KIND Registers_or1k
 #elif defined(__mips__)
-#warning The MIPS architecture is not supported.
+# warning The MIPS architecture is not supported.
 #else
-#error Architecture not supported
+# error Architecture not supported
 #endif
+  static_assert(sizeof(UnwindCursor<LocalAddressSpace, REGISTER_KIND>) <=
+    sizeof(unw_cursor_t), "Unwind cursor does not fit in unw_cursor_t");
+  // Use "placement new" to allocate UnwindCursor in the cursor buffer.
+  new ((void *)cursor) UnwindCursor<LocalAddressSpace, REGISTER_KIND>(
+                                 context, LocalAddressSpace::sThisAddressSpace);
+#undef REGISTER_KIND
   AbstractUnwindCursor *co = (AbstractUnwindCursor *)cursor;
   co->setInfoBasedOnIPRegister();
 
   return UNW_ESUCCESS;
 }
 
 #ifdef UNW_REMOTE
 /// Create a cursor into a thread in another process.