[compiler-rt] Fix Asan build on Android

[etienneb]

Summary

The dynamic shadow code is not detected correctly on Android.
The android shadow seems to start at address zero.

The bug is introduced here:

https://reviews.llvm.org/D23363

Started here: https://build.chromium.org/p/chromium.fyi/builders/ClangToTAndroidASan/builds/4029
Likely due to an asan runtime change, filed https://llvm.org/bugs/show_bug.cgi?id=30462

From asan_mapping.h:

#if SANITIZER_WORDSIZE == 32
#  if SANITIZER_ANDROID
#    define SHADOW_OFFSET (0)   <<---- HERE
#  elif defined(__mips__)

Shadow address on android is 0.

From asan_rtl.c:

if (shadow_start == 0) {
  [...]
  shadow_start = FindAvailableMemoryRange(space_size, alignment, granularity);
}

We assumed that 0 is dynamic address.

On windows, the address was determined with:

#  elif SANITIZER_WINDOWS64
#   define SHADOW_OFFSET __asan_shadow_memory_dynamic_address
#  else

and __asan_shadow_memory_dynamic_address is initially zero.

[rnk]

lib/asan/asan_rtl.cc

483–484

I think kcc likes regular if over #if, when possible. Also, isn't this macro undefined usually?

[etienneb]

Vitaly, could you double check this.

[vitalybuka]

lib/asan/asan_rtl.cc

483–484

Maybe ~0 instead of 0?

[kcc]

lib/asan/asan_rtl.cc

483–484

Correct, kcc has very strong opinion on #ifs.

1. When you can avoid an #if -- avoid it.
2. When you can't avoid it, check #1 once more
3. If desperate, split the files and put a top-level #if on the entire file.

[etienneb]

lib/asan/asan_rtl.cc

483–484

I do not like #if and worse "nested" #if :)
I'm do not have a strong opinion on the way to solve this.
I just want to fix Android bots that we (ok, I broke).

Maybe ~0 instead of 0?

We can't use ~0 here because we rely on the way macro are expanded.

Initially, the variable is assigned to zero (this is needed):

__asan_shadow_memory_dynamic_address = 0;

Because the following statement is assigned from a macro:

uptr shadow_start = kLowShadowBeg;

That macro is expanded from other macros, ...

#define kLowShadowBeg   SHADOW_OFFSET

And ends up to be "based" on __asan_shadow_memory_dynamic_address which must be zero.

I'm gonna think a something not to ugly.

[etienneb]

fix without #if

[etienneb]

simplify

[etienneb]

what about this solution?

[etienneb]

fix assert failing

[vitalybuka]

LGTM

[vitalybuka]

lib/asan/asan_rtl.cc

472

How kLowShadowBeg can be 0 if it's kDefaultShadowSentinel

[vitalybuka]

lib/asan/asan_rtl.cc

472

I see

[etienneb]

add comment

[vitalybuka]

Please land it if it fixes the bot.

[etienneb]

Tests are fine on linux:

~/llvm/build$ ninja check-asan
[36/37] Running the AddressSanitizer tests
Testing Time: 89.23s

Expected Passes    : 1398
Expected Failures  : 3
Unsupported Tests  : 425

[etienneb]

Tests on windows win7 64-bits are fine:

-- Testing: 543 tests, 16 threads --
Testing: 0 .. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90..
Testing Time: 309.71s
  Expected Passes    : 373
  Passes With Retry  : 3
  Expected Failures  : 16
  Unsupported Tests  : 151

Landing...

[thakis]

this went in at r282085

lib/asan/asan_mapping.h

 //  on WoW64, /LARGEADDRESSAWARE, etc).
 // || `[0x50000000, 0xffffffff]` || HighMem    ||
 // || `[0x3a000000, 0x4fffffff]` || HighShadow ||
 // || `[0x36000000, 0x39ffffff]` || ShadowGap  ||
 // || `[0x30000000, 0x35ffffff]` || LowShadow  ||
 // || `[0x00000000, 0x2fffffff]` || LowMem     ||
 
 static const u64 kDefaultShadowScale = 3;
+static const u64 kDefaultShadowSentinel = ~(uptr)0;
 static const u64 kDefaultShadowOffset32 = 1ULL << 29;  // 0x20000000
 static const u64 kDefaultShadowOffset64 = 1ULL << 44;
 static const u64 kDefaultShort64bitShadowOffset = 0x7FFF8000;  // < 2G.
 static const u64 kIosShadowOffset32 = 1ULL << 30;  // 0x40000000
 static const u64 kIosShadowOffset64 = 0x120200000;
 static const u64 kIosSimShadowOffset32 = 1ULL << 30;
 static const u64 kIosSimShadowOffset64 = kDefaultShadowOffset64;
 static const u64 kAArch64_ShadowOffset64 = 1ULL << 36;

lib/asan/asan_rtl.cc

 }
 
 static void AsanInitInternal() {
   if (LIKELY(asan_inited)) return;
   SanitizerToolName = "AddressSanitizer";
   CHECK(!asan_init_is_running && "ASan init calls itself!");
   asan_init_is_running = true;
 
+  // Set the shadow memory address to uninitialized.
+  __asan_shadow_memory_dynamic_address = kDefaultShadowSentinel;
+
   CacheBinaryName();
 
   // Initialize flags. This must be done early, because most of the
   // initialization steps look at flags().
   InitializeFlags();
 
   AsanCheckIncompatibleRT();
   AsanCheckDynamicRTPrereqs();
static void AsanInitInternal() {
 
   // Enable system log ("adb logcat") on Android.
   // Doing this before interceptors are initialized crashes in:
   // AsanInitInternal -> android_log_write -> __interceptor_strcmp
   AndroidLogInit();
 
   ReplaceSystemMalloc();
 
-  __asan_shadow_memory_dynamic_address = 0;
   uptr shadow_start = kLowShadowBeg;
-  if (shadow_start == 0) {
+  // Detect if a dynamic shadow address must used and find a available location
+  // when necessary.
+  if (shadow_start == kDefaultShadowSentinel) {
+    __asan_shadow_memory_dynamic_address = 0;
+    shadow_start = kLowShadowBeg;
+    CHECK_EQ(0, shadow_start);
+
     uptr granularity = GetMmapGranularity();
     uptr alignment = 8 * granularity;
     uptr left_padding = granularity;
     uptr space_size = kHighShadowEnd + left_padding;
 
     shadow_start = FindAvailableMemoryRange(space_size, alignment, granularity);
     CHECK_NE((uptr)0, shadow_start);
     CHECK(IsAligned(shadow_start, alignment));
   }
+  // Update the shadow memory address (potentially) used by instrumentation.
   __asan_shadow_memory_dynamic_address = shadow_start;
 
   if (kLowShadowBeg)
     shadow_start -= GetMmapGranularity();
   bool full_shadow_is_available =
       MemoryRangeIsAvailable(shadow_start, kHighShadowEnd);
 
 #if SANITIZER_LINUX && defined(__x86_64__) && defined(_LP64) &&                \