hwasan: enable mmap interception (no tagging used)

[thurston]

Summary

This enables HWASan interception for mmap, to prevent users from allocating in the shadow memory regions. For compatibility, it does not use pointer tagging, nor does it allow MAP_FIXED with a tagged address.

This patch initializes the common interceptors, but that should be a no-op (except for the mmap interceptor), due to the disable-by-default nature of hwasan_platform_interceptors.h (from D150708). As the first patch to utilize this common interceptor machinery for HWASan, it also defines some macros (e.g., COMMON_INTERCEPT_FUNCTION) that will be useful as future interceptors are enabled.

TestCases/Posix/mmap_write_exec.cpp now passes for HWASan.

[thurston]

Allow using tagged pointer (from mmap) as parameter to another mmap call

Relax assumption about mmap alignment when tagging

[thurston]

I need to add support for 'hwasan_allocator_tagging_enabled'.

[thurston]

Added mmap support for hwasan_allocator_tagging_enabled

Added test case that checks whether mmap pointers are tagged

[thurston]

Removed unused macro

[thurston]

Changed mmap interceptor to use tagging only if given a tagged address plus MAP_FIXED.

Also add munmap interceptor.

[vitalybuka]

compiler-rt/lib/hwasan/hwasan_allocator.cpp

31 ↗

(On Diff #525271)

Instead of making them extern, please add getter function

but with the recent discussion, we don't need to change this at all?

compiler-rt/lib/hwasan/hwasan_interceptors.h

12 ↗

(On Diff #525271)

Move this into hwasan_interceptors.cpp

we don't need this outsize of the cpp

[thurston]

Disallow MAP_FIXED with a tagged pointer, which is an odd usecase. This simplifies the code greatly, since an invariant
is that mmap will no longer return any tagged pointers, hence there is no need to intercept munmap.

Also moved hwasan_interceptors.h into hwasan_interceptors.cpp.

[thurston]

compiler-rt/lib/hwasan/hwasan_allocator.cpp

31 ↗

(On Diff #525271)

Yes, this is no longer needed.

compiler-rt/lib/hwasan/hwasan_interceptors.h

12 ↗

(On Diff #525271)

Done

[kstoimenov]

compiler-rt/lib/hwasan/hwasan_interceptors.cpp

89

Space after 'if'.

90

Should this be upper case?

92

Should we return an error instead?

99

The line is too long, please split.

112

Probably split here also.

compiler-rt/lib/hwasan/hwasan_platform_interceptors.h

731–733

Remove dead code.

[vitalybuka]

compiler-rt/lib/hwasan/hwasan_interceptors.cpp

89

just clangformat the thing

90

hm, how this compiles?

92

CHECK_EQ(tag, 0);

Also maybe CHECK_EQ(addr, addr_untagger); is nicer to read.

We agreed on CHECK to get from users if this is a valid usecase.

[thurston]

compiler-rt/lib/hwasan/hwasan_interceptors.cpp

90

map_fixed is defined in compiler-rt/lib/sanitizer_common/sanitizer_platform_limits_posix.cpp

'int map_fixed = MAP_FIXED;'

[thurston]

compiler-rt/lib/hwasan/hwasan_platform_interceptors.h

731–733

I deliberately leave in the dead code to show that this is intentionally re-enabling the interceptor, not an accidental omission.

For example, suppose we notice that sanitizer_common has SANITIZER_INTERCEPT_FOO, but SANITIZER_INTERCEPT_FOO doesn't appear in hwasan_platform_interceptors.h. Did someone add SANITIZER_INTERCEPT_FOO recently to sanitizer_common and forget to add it to hwasan_platform_interceptors, or was it deliberately removed from hwasan?

[thurston]

clang-format'ed

[thurston]

For readability, change CHECK(tag == 0) to CHECK_EQ(addr, UntagPtr(addr));

[thurston]

compiler-rt/lib/hwasan/hwasan_interceptors.cpp

92

I changed it to

CHECK_EQ(addr, UntagPtr(addr));

to avoid needing to change the rest of the code to use 'addr_untagged'.

[kstoimenov]

compiler-rt/lib/hwasan/hwasan_interceptors.cpp

90

Makes sense. Unrelated to this patch, but shouldn't those be constexpr?

[vitalybuka]

compiler-rt/lib/hwasan/hwasan_interceptors.cpp

181

we don't use {} around oneliners

[thurston]

Remove unnecessary braces

[fmayer]

I think this might have caused https://github.com/llvm/llvm-project/issues/64730

compiler-rt/lib/hwasan/hwasan_interceptors.cpp

 //
 // Interceptors for standard library functions.
 //
 // FIXME: move as many interceptors as possible into
 // sanitizer_common/sanitizer_common_interceptors.h
 //===----------------------------------------------------------------------===//
 
 #include "hwasan.h"
+#include "hwasan_allocator.h"
 #include "hwasan_checks.h"
 #include "hwasan_platform_interceptors.h"
 #include "hwasan_thread.h"
 #include "hwasan_thread_list.h"
 #include "interception/interception.h"
+#include "sanitizer_common/sanitizer_errno.h"
 #include "sanitizer_common/sanitizer_linux.h"
 #include "sanitizer_common/sanitizer_stackdepot.h"
 
 #if !SANITIZER_FUCHSIA
 
 using namespace __hwasan;
 
+#  if !SANITIZER_APPLE
+#    define HWASAN_INTERCEPT_FUNC(name)                                        \
+      do {                                                                     \
+        if (!INTERCEPT_FUNCTION(name))                                         \
+          VReport(1, "HWAddressSanitizer: failed to intercept '%s'\n", #name); \
+      } while (0)
+#    define HWASAN_INTERCEPT_FUNC_VER(name, ver)                           \
+      do {                                                                 \
+        if (!INTERCEPT_FUNCTION_VER(name, ver))                            \
+          VReport(1, "HWAddressSanitizer: failed to intercept '%s@@%s'\n", \
+                  #name, ver);                                             \
+      } while (0)
+#    define HWASAN_INTERCEPT_FUNC_VER_UNVERSIONED_FALLBACK(name, ver)          \
+      do {                                                                     \
+        if (!INTERCEPT_FUNCTION_VER(name, ver) && !INTERCEPT_FUNCTION(name))   \
+          VReport(                                                             \
+              1, "HWAddressSanitizer: failed to intercept '%s@@%s' or '%s'\n", \
+              #name, ver, #name);                                              \
+      } while (0)
+
+#  else
+// OS X interceptors don't need to be initialized with INTERCEPT_FUNCTION.
+#    define HWASAN_INTERCEPT_FUNC(name)
+#  endif  // SANITIZER_APPLE
+
 #  if HWASAN_WITH_INTERCEPTORS
 
 #    define COMMON_SYSCALL_PRE_READ_RANGE(p, s) __hwasan_loadN((uptr)p, (uptr)s)
 #    define COMMON_SYSCALL_PRE_WRITE_RANGE(p, s) \
       __hwasan_storeN((uptr)p, (uptr)s)
 #    define COMMON_SYSCALL_POST_READ_RANGE(p, s) \
       do {                                       \
         (void)(p);                               \
         (void)(s);                               \
       } while (false)
 #    define COMMON_SYSCALL_POST_WRITE_RANGE(p, s) \
       do {                                        \
         (void)(p);                                \
         (void)(s);                                \
       } while (false)
 #    include "sanitizer_common/sanitizer_common_syscalls.inc"
 #    include "sanitizer_common/sanitizer_syscalls_netbsd.inc"
 
-#define COMMON_INTERCEPTOR_WRITE_RANGE(ctx, ptr, size) \
-  do {                                                \
+#    define COMMON_INTERCEPTOR_WRITE_RANGE(ctx, ptr, size) \
+      do {                                                 \
       } while (false)
 
-#define COMMON_INTERCEPTOR_READ_RANGE(ctx, ptr, size) \
+#    define COMMON_INTERCEPTOR_READ_RANGE(ctx, ptr, size) \
       do {                                                \
         (void)(ctx);                                      \
         (void)(ptr);                                      \
         (void)(size);                                     \
       } while (false)
 
-#define COMMON_INTERCEPTOR_ENTER(ctx, func, ...) \
+#    define COMMON_INTERCEPTOR_ENTER(ctx, func, ...) \
       do {                                           \
         (void)(ctx);                                 \
         (void)(func);                                \
       } while (false)
 
-#define COMMON_INTERCEPTOR_DIR_ACQUIRE(ctx, path) \
+#    define COMMON_INTERCEPTOR_DIR_ACQUIRE(ctx, path) \
       do {                                            \
         (void)(ctx);                                  \
         (void)(path);                                 \
       } while (false)
 
-#define COMMON_INTERCEPTOR_FD_ACQUIRE(ctx, fd) \
+#    define COMMON_INTERCEPTOR_FD_ACQUIRE(ctx, fd) \
       do {                                         \
         (void)(ctx);                               \
         (void)(fd);                                \
       } while (false)
 
-#define COMMON_INTERCEPTOR_FD_RELEASE(ctx, fd) \
+#    define COMMON_INTERCEPTOR_FD_RELEASE(ctx, fd) \
       do {                                         \
         (void)(ctx);                               \
         (void)(fd);                                \
       } while (false)
 
-#define COMMON_INTERCEPTOR_FD_SOCKET_ACCEPT(ctx, fd, newfd) \
+#    define COMMON_INTERCEPTOR_FD_SOCKET_ACCEPT(ctx, fd, newfd) \
       do {                                                      \
         (void)(ctx);                                            \
         (void)(fd);                                             \
         (void)(newfd);                                          \
       } while (false)
 
-#define COMMON_INTERCEPTOR_SET_THREAD_NAME(ctx, name) \
+#    define COMMON_INTERCEPTOR_SET_THREAD_NAME(ctx, name) \
       do {                                                \
         (void)(ctx);                                      \
         (void)(name);                                     \
       } while (false)
 
-#define COMMON_INTERCEPTOR_SET_PTHREAD_NAME(ctx, thread, name) \
+#    define COMMON_INTERCEPTOR_SET_PTHREAD_NAME(ctx, thread, name) \
       do {                                                         \
         (void)(ctx);                                               \
         (void)(thread);                                            \
         (void)(name);                                              \
       } while (false)
 
-#define COMMON_INTERCEPTOR_BLOCK_REAL(name) \
+#    define COMMON_INTERCEPTOR_BLOCK_REAL(name) \
       do {                                      \
         (void)(name);                           \
       } while (false)
 
-#define COMMON_INTERCEPTOR_MEMMOVE_IMPL(ctx, to, from, size) \
+#    define COMMON_INTERCEPTOR_MEMMOVE_IMPL(ctx, to, from, size) \
       do {                                                       \
         (void)(ctx);                                             \
         (void)(to);                                              \
         (void)(from);                                            \
         (void)(size);                                            \
       } while (false)
 
-#define COMMON_INTERCEPTOR_MEMCPY_IMPL(ctx, to, from, size) \
+#    define COMMON_INTERCEPTOR_MEMCPY_IMPL(ctx, to, from, size) \
       do {                                                      \
         (void)(ctx);                                            \
         (void)(to);                                             \
         (void)(from);                                           \
         (void)(size);                                           \
       } while (false)
 
-#define COMMON_INTERCEPTOR_MEMSET_IMPL(ctx, block, c, size) \
+#    define COMMON_INTERCEPTOR_MEMSET_IMPL(ctx, block, c, size) \
       do {                                                      \
         (void)(ctx);                                            \
         (void)(block);                                          \
         (void)(c);                                              \
         (void)(size);                                           \
       } while (false)
 
-#define COMMON_INTERCEPTOR_STRERROR() \
+#    define COMMON_INTERCEPTOR_STRERROR() \
       do {                                \
       } while (false)
 
-#define COMMON_INTERCEPT_FUNCTION(name) \
-  do {                                  \
-    (void)(name);                       \
+#    define COMMON_INTERCEPT_FUNCTION(name) HWASAN_INTERCEPT_FUNC(name)
+
+#    define COMMON_INTERCEPTOR_NOTHING_IS_INITIALIZED (!hwasan_inited)
+
+// The main purpose of the mmap interceptor is to prevent the user from
+// allocating on top of shadow pages.
+//
+// For compatibility, it does not tag pointers, nor does it allow
+// MAP_FIXED in combination with a tagged pointer. (Since mmap itself
+// will not return a tagged pointer, the tagged pointer must have come
+// from elsewhere, such as the secondary allocator, which makes it a
+// very odd usecase.)
+template <class Mmap>
+static void *mmap_interceptor(Mmap real_mmap, void *addr, SIZE_T length,
+                              int prot, int flags, int fd, OFF64_T offset) {
+  if (addr) {
+    if (flags & map_fixed) CHECK_EQ(addr, UntagPtr(addr));
+
+    addr = UntagPtr(addr);
+  }
+  SIZE_T rounded_length = RoundUpTo(length, GetPageSize());
+  void *end_addr = (char *)addr + (rounded_length - 1);
+  if (addr && (!MemIsApp(reinterpret_cast<uptr>(addr)) ||
+               !MemIsApp(reinterpret_cast<uptr>(end_addr)))) {
+    // User requested an address that is incompatible with HWASan's
+    // memory layout. Use a different address if allowed, else fail.
+    if (flags & map_fixed) {
+      errno = errno_EINVAL;
+      return (void *)-1;
+    } else {
+      addr = nullptr;
+    }
+  }
+  void *res = real_mmap(addr, length, prot, flags, fd, offset);
+  if (res != (void *)-1) {
+    void *end_res = (char *)res + (rounded_length - 1);
+    if (!MemIsApp(reinterpret_cast<uptr>(res)) ||
+        !MemIsApp(reinterpret_cast<uptr>(end_res))) {
+      // Application has attempted to map more memory than is supported by
+      // HWASan. Act as if we ran out of memory.
+      internal_munmap(res, length);
+      errno = errno_ENOMEM;
+      return (void *)-1;
+    }
+  }
+
+  return res;
+}
+
+#    define COMMON_INTERCEPTOR_MMAP_IMPL(ctx, mmap, addr, length, prot, flags, \
+                                         fd, offset)                           \
+      do {                                                                     \
+        (void)(ctx);                                                           \
+        return mmap_interceptor(REAL(mmap), addr, sz, prot, flags, fd, off);   \
       } while (false)
 
-#include "sanitizer_common/sanitizer_common_interceptors.inc"
+#    include "sanitizer_common/sanitizer_common_interceptors.inc"
 
 struct ThreadStartArg {
   __sanitizer_sigset_t starting_sigset_;
 };
 
 static void *HwasanThreadStartFunc(void *arg) {
   __hwasan_thread_enter();
   ThreadStartArg A = *reinterpret_cast<ThreadStartArg *>(arg);

 }  // namespace __hwasan
 
 namespace __hwasan {
 
 void InitializeInterceptors() {
   static int inited = 0;
   CHECK_EQ(inited, 0);
 
-  (void)(InitializeCommonInterceptors);
+  InitializeCommonInterceptors();
+
   (void)(read_iovec);
   (void)(write_iovec);
 
 #  if HWASAN_WITH_INTERCEPTORS
 #    if defined(__linux__)
   INTERCEPT_FUNCTION(__libc_longjmp);
   INTERCEPT_FUNCTION(longjmp);
   INTERCEPT_FUNCTION(siglongjmp);

compiler-rt/lib/hwasan/hwasan_platform_interceptors.h

 #define SANITIZER_INTERCEPT_UTMP 0
 
 #undef SANITIZER_INTERCEPT_UTMPX
 #define SANITIZER_INTERCEPT_UTMPX 0
 
 #undef SANITIZER_INTERCEPT_GETLOADAVG
 #define SANITIZER_INTERCEPT_GETLOADAVG 0
 
-#undef SANITIZER_INTERCEPT_MMAP
-#define SANITIZER_INTERCEPT_MMAP 0
+// #undef SANITIZER_INTERCEPT_MMAP
+// #define SANITIZER_INTERCEPT_MMAP 0
 
 #undef SANITIZER_INTERCEPT_MMAP64
 #define SANITIZER_INTERCEPT_MMAP64 0
 
 #undef SANITIZER_INTERCEPT_MALLOPT_AND_MALLINFO
 #define SANITIZER_INTERCEPT_MALLOPT_AND_MALLINFO 0
 
 #undef SANITIZER_INTERCEPT_MEMALIGN
 #define SANITIZER_INTERCEPT_MEMALIGN 0

compiler-rt/test/sanitizer_common/TestCases/Posix/mmap_write_exec.cpp

 // RUN: %clangxx %s -o %t
 // RUN: %env_tool_opts=detect_write_exec=1 %run %t 2>&1 | FileCheck %s
 // RUN: %env_tool_opts=detect_write_exec=0 %run %t 2>&1 | FileCheck %s \
 // RUN:   --check-prefix=CHECK-DISABLED
 // ubsan and lsan do not install mmap interceptors UNSUPPORTED: ubsan, lsan
 
 // TODO: Fix option on Android, it hangs there for unknown reasons.
 // XFAIL: android
 
-// FIXME: Implement.
-// XFAIL: hwasan
-
 #if defined(__APPLE__)
 #include <Availability.h>
 #endif
 #include <pthread.h>
 #include <stdio.h>
 #include <sys/mman.h>
 
 int main(int argc, char **argv) {