[lsan][darwin] Unmask camouflaged class_rw_t pointers

[lgrey]

Summary

Detailed motivation here: https://docs.google.com/document/d/1xUNo5ovPKJMYxitiHUQVRxGI3iUmspI51Jm4w8puMwo

check-asan (with LSAN enabled) and check-lsan are currently broken on recent macOS versions, due to pervasive false positives. Whenever the Objective-C runtime realizes a class, it allocates data for it, then stores that data with flags in the low bits. This means LSAN can not recognize it as a pointer while scanning.

This change checks every potential pointer on Apple platforms, and if the high bit is set, attempts to extract a pointer by masking out the high bit and flags. This is ugly, but it's also the best approach I could think of (see doc above); very open to other suggestions.

[yln]

Thanks!

I've applied the patch and can confirm that this fixes the LSan-related test failures for check-asan on a recent version of macOS.

compiler-rt/lib/lsan/lsan_common.cpp

 #include "sanitizer_common/sanitizer_report_decorator.h"
 #include "sanitizer_common/sanitizer_stackdepot.h"
 #include "sanitizer_common/sanitizer_stacktrace.h"
 #include "sanitizer_common/sanitizer_suppressions.h"
 #include "sanitizer_common/sanitizer_thread_registry.h"
 #include "sanitizer_common/sanitizer_tls_get_addr.h"
 
 #if CAN_SANITIZE_LEAKS
+
+#  if SANITIZER_APPLE
+// https://github.com/apple-oss-distributions/objc4/blob/8701d5672d3fd3cd817aeb84db1077aafe1a1604/runtime/objc-runtime-new.h#L127
+#    if SANITIZER_IOS && !SANITIZER_IOSSIM
+#      define OBJC_DATA_MASK 0x0000007ffffffff8UL
+#    else
+#      define OBJC_DATA_MASK 0x00007ffffffffff8UL
+#    endif
+// https://github.com/apple-oss-distributions/objc4/blob/8701d5672d3fd3cd817aeb84db1077aafe1a1604/runtime/objc-runtime-new.h#L139
+#    define OBJC_FAST_IS_RW 0x8000000000000000UL
+#  endif
+
 namespace __lsan {
 
 // This mutex is used to prevent races between DoLeakCheck and IgnoreObject, and
 // also to protect the global list of root regions.
 Mutex global_mutex;
 
 Flags lsan_flags;
 

 
 static uptr GetCallerPC(const StackTrace &stack) {
   // The top frame is our malloc/calloc/etc. The next frame is the caller.
   if (stack.size >= 2)
     return stack.trace[1];
   return 0;
 }
 
+#  if SANITIZER_APPLE
+// Objective-C class data pointers are stored with flags in the low bits, so
+// they need to be transformed back into something that looks like a pointer.
+static inline void *MaybeTransformPointer(void *p) {
+  uptr ptr = reinterpret_cast<uptr>(p);
+  if ((ptr & OBJC_FAST_IS_RW) == OBJC_FAST_IS_RW)
+    ptr &= OBJC_DATA_MASK;
+  return reinterpret_cast<void *>(ptr);
+}
+#  endif
+
 // On Linux, treats all chunks allocated from ld-linux.so as reachable, which
 // covers dynamically allocated TLS blocks, internal dynamic loader's loaded
 // modules accounting etc.
 // Dynamic TLS blocks contain the TLS variables of dynamically loaded modules.
 // They are allocated with a __libc_memalign() call in allocate_and_init()
 // (elf/dl-tls.c). Glibc won't tell us the address ranges occupied by those
 // blocks, but we can make sure they come from our own allocator by intercepting
 // __libc_memalign(). On top of that, there is no easy way to reach them. Their
void ScanRangeForPointers(uptr begin, uptr end, Frontier *frontier,
   const uptr alignment = flags()->pointer_alignment();
   LOG_POINTERS("Scanning %s range %p-%p.\n", region_type, (void *)begin,
                (void *)end);
   uptr pp = begin;
   if (pp % alignment)
     pp = pp + alignment - pp % alignment;
   for (; pp + sizeof(void *) <= end; pp += alignment) {
     void *p = *reinterpret_cast<void **>(pp);
+#  if SANITIZER_APPLE
+    p = MaybeTransformPointer(p);
+#  endif
     if (!MaybeUserPointer(reinterpret_cast<uptr>(p)))
       continue;
     uptr chunk = PointsIntoChunk(p);
     if (!chunk)
       continue;
     // Pointers to self don't count. This matters when tag == kIndirectlyLeaked.
     if (chunk == begin)
       continue;