[sanitizers] Add init function to set alignment of low level allocator

[waltl]

Summary

ASan requires that the min alignment be at least the shadow
granularity, so add an init function to do that.

[vitalybuka]

compiler-rt/lib/sanitizer_common/sanitizer_allocator.cc

185 ↗

(On Diff #121052)

Why can't this be just a shadow granularity?

[waltl]

compiler-rt/lib/sanitizer_common/sanitizer_allocator.cc

185 ↗

(On Diff #121052)

It can, but I was not clear on how to get that value --
SHADOW_GRANULARITY is defined in an asan header and I don't see any sanitizer_common sources including any asan headers. And I'm not sure we wanted to move that to a sanitizer header.

[vitalybuka]

maybe we need to set this to low level allocator during initialization?
@alekseyshl WDYT?

[alekseyshl]

Why do we need this at all for internal allocator? It's internal to sanitizer and sanitizer code is not instrumented, that means no shadow.

[waltl]

In D39473#928001, @alekseyshl wrote:

Why do we need this at all for internal allocator? It's internal to sanitizer and sanitizer code is not instrumented, that means no shadow.

Here is a typical backtrace. It seems that asan installs a callback to the internal allocator that poisons those memory.

#0 0xf7fd7c89 in kernel_vsyscall ()
#1 0xf7e3c627 in syscall () from /lib/i386-linux-gnu/libc.so.6
#2 0x0811c3cc in sanitizer::internal_sched_yield() () at /usr/local/google/home/waltl/Work/llvm-scale5/scale5/compiler-rt/lib/sanitizer_common/sanitizer_linux.cc:401
#3 0x08062c75 in LockSlow () at /usr/local/google/home/waltl/Work/llvm-scale5/scale5/compiler-rt/lib/asan/../sanitizer_common/sanitizer_mutex.h:55
#4 0x08129696 in Lock () at /usr/local/google/home/waltl/Work/llvm-scale5/scale5/compiler-rt/lib/sanitizer_common/sanitizer_mutex.h:32
#5 GenericScopedLock () at /usr/local/google/home/waltl/Work/llvm-scale5/scale5/compiler-rt/lib/sanitizer_common/sanitizer_mutex.h:187
#6 GetOrInit () at /usr/local/google/home/waltl/Work/llvm-scale5/scale5/compiler-rt/lib/sanitizer_common/sanitizer_symbolizer_libcdep.cc:21
#7 0x08128147 in Print () at /usr/local/google/home/waltl/Work/llvm-scale5/scale5/compiler-rt/lib/sanitizer_common/sanitizer_stacktrace_libcdep.cc:35
#8 0x0810f31f in AsanCheckFailed () at /usr/local/google/home/waltl/Work/llvm-scale5/scale5/compiler-rt/lib/asan/asan_rtl.cc:69
#9 0x08124884 in CheckFailed () at /usr/local/google/home/waltl/Work/llvm-scale5/scale5/compiler-rt/lib/sanitizer_common/sanitizer_termination.cc:79
#10 0x08108b26 in PoisonShadow () at /usr/local/google/home/waltl/Work/llvm-scale5/scale5/compiler-rt/lib/asan/asan_poisoning.cc:38
#11 0x0810f355 in __asan::OnLowLevelAllocate(unsigned long, unsigned long) () at /usr/local/google/home/waltl/Work/llvm-scale5/scale5/compiler-rt/lib/asan/asan_rtl.cc:89
#12 0x0811328b in Allocate () at /usr/local/google/home/waltl/Work/llvm-scale5/scale5/compiler-rt/lib/sanitizer_common/sanitizer_allocator.cc:193
#13 0x0812b49a in operator new () at /usr/local/google/home/waltl/Work/llvm-scale5/scale5/compiler-rt/lib/sanitizer_common/sanitizer_common.h:937
#14 ChooseExternalSymbolizer () at /usr/local/google/home/waltl/Work/llvm-scale5/scale5/compiler-rt/lib/sanitizer_common/sanitizer_symbolizer_posix_libcdep.cc:486
#15 ChooseSymbolizerTools () at /usr/local/google/home/waltl/Work/llvm-scale5/scale5/compiler-rt/lib/sanitizer_common/sanitizer_symbolizer_posix_libcdep.cc:516
#16 PlatformInit () at /usr/local/google/home/waltl/Work/llvm-scale5/scale5/compiler-rt/lib/sanitizer_common/sanitizer_symbolizer_posix_libcdep.cc:529
#17 0x081296aa in GetOrInit () at /usr/local/google/home/waltl/Work/llvm-scale5/scale5/compiler-rt/lib/sanitizer_common/sanitizer_symbolizer_libcdep.cc:24
#18 0x0812b635 in LateInitialize () at /usr/local/google/home/waltl/Work/llvm-scale5/scale5/compiler-rt/lib/sanitizer_common/sanitizer_symbolizer_posix_libcdep.cc:534
#19 0x0810ee62 in AsanInitInternal () at /usr/local/google/home/waltl/Work/llvm-scale5/scale5/compiler-rt/lib/asan/asan_rtl.cc:479
#20 0xf7fe8de4 in ?? () from /lib/ld-linux.so.2
#21 0xf7fd9a5f in ?? () from /lib/ld-linux.so.2

[vitalybuka]

In D39473#928001, @alekseyshl wrote:

Why do we need this at all for internal allocator? It's internal to sanitizer and sanitizer code is not instrumented, that means no shadow.

So if it's just internal allocator memory usage increase should not be significant?

[alekseyshl]

In D39473#928171, @waltl wrote:

In D39473#928001, @alekseyshl wrote:

Why do we need this at all for internal allocator? It's internal to sanitizer and sanitizer code is not instrumented, that means no shadow.

Here is a typical backtrace. It seems that asan installs a callback to the internal allocator that poisons those memory.

#0 0xf7fd7c89 in kernel_vsyscall ()
#1 0xf7e3c627 in syscall () from /lib/i386-linux-gnu/libc.so.6
#2 0x0811c3cc in sanitizer::internal_sched_yield() () at /usr/local/google/home/waltl/Work/llvm-scale5/scale5/compiler-rt/lib/sanitizer_common/sanitizer_linux.cc:401
#3 0x08062c75 in LockSlow () at /usr/local/google/home/waltl/Work/llvm-scale5/scale5/compiler-rt/lib/asan/../sanitizer_common/sanitizer_mutex.h:55
#4 0x08129696 in Lock () at /usr/local/google/home/waltl/Work/llvm-scale5/scale5/compiler-rt/lib/sanitizer_common/sanitizer_mutex.h:32
#5 GenericScopedLock () at /usr/local/google/home/waltl/Work/llvm-scale5/scale5/compiler-rt/lib/sanitizer_common/sanitizer_mutex.h:187
#6 GetOrInit () at /usr/local/google/home/waltl/Work/llvm-scale5/scale5/compiler-rt/lib/sanitizer_common/sanitizer_symbolizer_libcdep.cc:21
#7 0x08128147 in Print () at /usr/local/google/home/waltl/Work/llvm-scale5/scale5/compiler-rt/lib/sanitizer_common/sanitizer_stacktrace_libcdep.cc:35
#8 0x0810f31f in AsanCheckFailed () at /usr/local/google/home/waltl/Work/llvm-scale5/scale5/compiler-rt/lib/asan/asan_rtl.cc:69
#9 0x08124884 in CheckFailed () at /usr/local/google/home/waltl/Work/llvm-scale5/scale5/compiler-rt/lib/sanitizer_common/sanitizer_termination.cc:79
#10 0x08108b26 in PoisonShadow () at /usr/local/google/home/waltl/Work/llvm-scale5/scale5/compiler-rt/lib/asan/asan_poisoning.cc:38
#11 0x0810f355 in __asan::OnLowLevelAllocate(unsigned long, unsigned long) () at /usr/local/google/home/waltl/Work/llvm-scale5/scale5/compiler-rt/lib/asan/asan_rtl.cc:89
#12 0x0811328b in Allocate () at /usr/local/google/home/waltl/Work/llvm-scale5/scale5/compiler-rt/lib/sanitizer_common/sanitizer_allocator.cc:193
#13 0x0812b49a in operator new () at /usr/local/google/home/waltl/Work/llvm-scale5/scale5/compiler-rt/lib/sanitizer_common/sanitizer_common.h:937
#14 ChooseExternalSymbolizer () at /usr/local/google/home/waltl/Work/llvm-scale5/scale5/compiler-rt/lib/sanitizer_common/sanitizer_symbolizer_posix_libcdep.cc:486
#15 ChooseSymbolizerTools () at /usr/local/google/home/waltl/Work/llvm-scale5/scale5/compiler-rt/lib/sanitizer_common/sanitizer_symbolizer_posix_libcdep.cc:516
#16 PlatformInit () at /usr/local/google/home/waltl/Work/llvm-scale5/scale5/compiler-rt/lib/sanitizer_common/sanitizer_symbolizer_posix_libcdep.cc:529
#17 0x081296aa in GetOrInit () at /usr/local/google/home/waltl/Work/llvm-scale5/scale5/compiler-rt/lib/sanitizer_common/sanitizer_symbolizer_libcdep.cc:24
#18 0x0812b635 in LateInitialize () at /usr/local/google/home/waltl/Work/llvm-scale5/scale5/compiler-rt/lib/sanitizer_common/sanitizer_symbolizer_posix_libcdep.cc:534
#19 0x0810ee62 in AsanInitInternal () at /usr/local/google/home/waltl/Work/llvm-scale5/scale5/compiler-rt/lib/asan/asan_rtl.cc:479
#20 0xf7fe8de4 in ?? () from /lib/ld-linux.so.2
#21 0xf7fd9a5f in ?? () from /lib/ld-linux.so.2

Ah, right, thanks!

Ok then, we need SetLowLevelAllocatorMinAlignment to complement SetLowLevelAllocateCallback, default it to 8 and call it from ASan to set it to shadow granularity.

[waltl]

Add SetLowLevelAllocateMinAlignment

[waltl]

In D39473#929041, @alekseyshl wrote:

Ok then, we need SetLowLevelAllocatorMinAlignment to complement SetLowLevelAllocateCallback, default it to 8 and call it from ASan to set it to shadow granularity.

Done.

[alekseyshl]

compiler-rt/lib/asan/asan_rtl.cc

411 ↗

(On Diff #123586)

Swap the Set.. calls, set the min alignment first.

compiler-rt/lib/sanitizer_common/sanitizer_allocator.cc

210 ↗

(On Diff #123586)

Did you really mean that alignment can only go up? How about this:

constexpr uptr kLowLevelAllocatorDefaultAlignment = 8;
static uptr low_level_alloc_min_alignment = kLowLevelAllocatorDefaultAlignment;
...
low_level_alloc_min_alignment = Max(alignment, kLowLevelAllocatorDefaultAlignment);

[waltl]

Address CR comments

[waltl]

compiler-rt/lib/sanitizer_common/sanitizer_allocator.cc

210 ↗

(On Diff #123586)

Done. Not sure if the final kLowLevelAllocatorDefaultAlignment is a typo, but I changed it to low_level_alloc_min_alignment so that if SetLowLevelAllocateMinAlignment is called multiple times, we get a max of all the input alignments.

compiler-rt/trunk/lib/asan/asan_rtl.cc

static void AsanInitInternal() {
 
   __asan_option_detect_stack_use_after_return =
       flags()->detect_stack_use_after_return;
 
   // Re-exec ourselves if we need to set additional env or command line args.
   MaybeReexec();
 
   // Setup internal allocator callback.
+  SetLowLevelAllocateMinAlignment(SHADOW_GRANULARITY);
   SetLowLevelAllocateCallback(OnLowLevelAllocate);
 
   InitializeAsanInterceptors();
 
   // Enable system log ("adb logcat") on Android.
   // Doing this before interceptors are initialized crashes in:
   // AsanInitInternal -> android_log_write -> __interceptor_strcmp
   AndroidLogInit();

compiler-rt/trunk/lib/sanitizer_common/sanitizer_allocator.cc

if (!addr)
     return;
   addr = (char*)addr - sizeof(u64);
   CHECK_EQ(kBlockMagic, ((u64*)addr)[0]);
   ((u64*)addr)[0] = 0;
   RawInternalFree(addr, cache);
 }
 
 // LowLevelAllocator
+constexpr uptr kLowLevelAllocatorDefaultAlignment = 8;
+static uptr low_level_alloc_min_alignment = kLowLevelAllocatorDefaultAlignment;
 static LowLevelAllocateCallback low_level_alloc_callback;
 
 void *LowLevelAllocator::Allocate(uptr size) {
   // Align allocation size.
-  size = RoundUpTo(size, 8);
+  size = RoundUpTo(size, low_level_alloc_min_alignment);
   if (allocated_end_ - allocated_current_ < (sptr)size) {
     uptr size_to_allocate = Max(size, GetPageSizeCached());
     allocated_current_ =
         (char*)MmapOrDie(size_to_allocate, __func__);
     allocated_end_ = allocated_current_ + size_to_allocate;
     if (low_level_alloc_callback) {
       low_level_alloc_callback((uptr)allocated_current_,
                                size_to_allocate);
     }
   }
   CHECK(allocated_end_ - allocated_current_ >= (sptr)size);
   void *res = allocated_current_;
   allocated_current_ += size;
   return res;
 }
 
+void SetLowLevelAllocateMinAlignment(uptr alignment) {
+  CHECK(IsPowerOfTwo(alignment));
+  low_level_alloc_min_alignment = Max(alignment, low_level_alloc_min_alignment);
+}
+
 void SetLowLevelAllocateCallback(LowLevelAllocateCallback callback) {
   low_level_alloc_callback = callback;
 }
 
 static atomic_uint8_t allocator_out_of_memory = {0};
 static atomic_uint8_t allocator_may_return_null = {0};
 
 bool IsAllocatorOutOfMemory() {

compiler-rt/trunk/lib/sanitizer_common/sanitizer_common.h

 class LowLevelAllocator {
  public:
   // Requires an external lock.
   void *Allocate(uptr size);
  private:
   char *allocated_end_;
   char *allocated_current_;
 };
+// Set the min alignment of LowLevelAllocator to at least alignment.
+void SetLowLevelAllocateMinAlignment(uptr alignment);
 typedef void (*LowLevelAllocateCallback)(uptr ptr, uptr size);
 // Allows to register tool-specific callbacks for LowLevelAllocator.
 // Passing NULL removes the callback.
 void SetLowLevelAllocateCallback(LowLevelAllocateCallback callback);
 
 // IO
 void CatastrophicErrorWrite(const char *buffer, uptr length);
 void RawWrite(const char *buffer);