[ASAN] Add check for not-in-mapping address for mmap

[yingcong-wu]

Summary

Add check for not-in-mapping addr for mmap. This will fix the
failure of SanitizerCommon-asan-x86_64-Linux::Linux/mmap_56bit_test.c
in machine that support 5-level paging. Without this check if user
try to mmap a address outside ASAN mapping, then ASAN will use that
address to calculate and get the invalid shadow memory address, which
then lead to SEGV.

[vitalybuka]

compiler-rt/lib/asan/asan_interceptors.cpp

156

Please check https://reviews.llvm.org/D154659#inline-1495335

Asan is suppose to work with prebuild non-instrumented code, so we are not as aggressive here.
Maybe it's fine if we update the check for protect_shadow_gap=0

compiler-rt/lib/asan/asan_mapping.h

389

we should not allow mapping in AddrIsInShadow

[yingcong-wu]

Asan is suppose to work with prebuild non-instrumented code, so we are not as aggressive here.

Yeah, I forgot about that. I guess if prebuild non-instrumented code are using mmap to operate memory regions outside ASAN mapping, as long as they keep those addresses to themselves(not passing those not-in-mapping addresses to instrumented code), then it will be fine.

However, there is no guarantee for that. But that looks like a dilemma to me.

[yingcong-wu]

Abandon this because we want to keep the ability to work with not-instrumented code.

compiler-rt/lib/asan/asan_interceptors.cpp

#  define COMMON_INTERCEPTOR_GET_TLS_RANGE(begin, end) \
     if (AsanThread *t = GetCurrentThread()) {          \
       *begin = t->tls_begin();                         \
       *end = t->tls_end();                             \
     } else {                                           \
       *begin = *end = 0;                               \
     }
 
 template <class Mmap>
-static void* mmap_interceptor(Mmap real_mmap, void *addr, SIZE_T length,
+static void *mmap_interceptor(Mmap real_mmap, void *addr, SIZE_T length,
                               int prot, int flags, int fd, OFF64_T offset) {
+  SIZE_T rounded_length = RoundUpTo(length, GetPageSize());
+  void *end_addr = (char *)addr + (rounded_length - 1);
+  // Do not mmap outside mapping, or it will have invalid shadow memory address
+  if (addr && length &&
+      (!AddrInMapping(reinterpret_cast<uptr>(addr)) ||
+       !AddrInMapping(reinterpret_cast<uptr>(end_addr)))) {
+    if (flags & map_fixed) {
+      errno = errno_EINVAL;
+      return (void *)-1;
+    } else {
+      addr = nullptr;
+    }
+  }
   void *res = real_mmap(addr, length, prot, flags, fd, offset);
   if (length && res != (void *)-1) {
     const uptr beg = reinterpret_cast<uptr>(res);
     DCHECK(IsAligned(beg, GetPageSize()));
-    SIZE_T rounded_length = RoundUpTo(length, GetPageSize());
     // Only unpoison shadow if it's an ASAN managed address.
     if (AddrIsInMem(beg) && AddrIsInMem(beg + rounded_length - 1))
       PoisonShadow(beg, RoundUpTo(length, GetPageSize()), 0);
   }
   return res;
 }
 
 template <class Munmap>

compiler-rt/lib/asan/asan_mapping.h

 }
 
 static inline uptr ShadowToMem(uptr p) {
   PROFILE_ASAN_MAPPING();
   CHECK(AddrIsInShadow(p));
   return SHADOW_TO_MEM(p);
 }
 
+static inline bool AddrInMapping(uptr a) {
+  PROFILE_ASAN_MAPPING();
+  return AddrIsInMem(a) || AddrIsInShadowGap(a) || AddrIsInShadow(a);
+}
+
 static inline bool AddrIsAlignedByGranularity(uptr a) {
   PROFILE_ASAN_MAPPING();
   return (a & (ASAN_SHADOW_GRANULARITY - 1)) == 0;
 }
 
 static inline bool AddressIsPoisoned(uptr a) {
   PROFILE_ASAN_MAPPING();
   const uptr kAccessSize = 1;