[asan] Complete the Fuchsia port

[vitalybuka]

lib/asan/asan_interceptors.h

22 ↗

(On Diff #108182)

I assume you disable entire things instead of particular interceptors, like other platforms, because Fuchsia does not need interceptors?

Could you please note that in the code.

lib/asan/asan_rtl.cc

89 ↗

(On Diff #108182)

Why not just SANITIZER_FUCHSIA

342 ↗

(On Diff #108182)

Maybe better to move some stuff into
asan_rtl_no_fuchsia.cc (as separate CL)
and put new stuff into asan_rtl_fuchsia.cc?

Same for other files

lib/asan/asan_thread.cc

234 ↗

(On Diff #108182)

Could you please move this into fuchsia version of AsanThread::SetThreadStackAndTls()

[filcab]

lib/asan/asan_interceptors.h

118 ↗

(On Diff #108182)

It seems you can disable most of the file anyway. Do you really need InitializeAsanInterceptors if you don't have interceptors?
Can you just hoist the macro (and maybe the functions) to either the top of the header or another header?

[mcgrathr]

lib/asan/asan_rtl.cc

342 ↗

(On Diff #108182)

Done in D36037

[mcgrathr]

Rebased after D36037 split out

[mcgrathr]

I'll split this into smaller pieces.

lib/asan/asan_interceptors.h

118 ↗

(On Diff #108182)

InitializeAsanInterceptors is called unconditionally from asan_rtl.cc and the norm in this code seems to be to write stub functions instead of conditionalizing the calls. The main thing I need is the ENSURE_ASAN_INITED macro, which is used by asan_interceptors_memintrinsics.h and by asan_malloc_linux.cc (both of which are used on Fuchsia).

I'll move the preserved code to the top of the file if that's what you prefer. If it's to go in a different header, I'm not sure which one you'd want it in. I'll do it however I'm told.

[mcgrathr]

lib/asan/asan_thread.cc

234 ↗

(On Diff #108182)

Currently I don't have such a function. It would have to have a different signature than the generic one so it can take the arguments passed to Init.
Or I could make the generic one take optional arguments as I did for Init and always pass them down here.
Is that what you meant?

[mcgrathr]

Rebased after splitting out D36189 and D36190.
Moved some work from AsanThread::Init to AsanThread::SetThreadStackAndTls.

[mcgrathr]

Rebased.

[mcgrathr]

Rebased.

[vitalybuka]

lib/asan/asan_errors.cc

74 ↗

(On Diff #109586)

Why you can't leave the current implementation?

lib/asan/asan_rtl.cc

370 ↗

(On Diff #109586)

Please move into CL with thread API refactoring as well

lib/asan/asan_shadow_setup.cc

17 ↗

(On Diff #109586)

Can I ask you to try later to move such full file "#if !SANITIZER_FUCHSIA ..." into cmake rules?

lib/asan/asan_thread.cc

273 ↗

(On Diff #109586)

Please move API change and other thread code movements, without fucshia, into separate patch

281 ↗

(On Diff #109586)

Could you please put fuchsia version
void AsanThread::SetThreadStackAndTls() into asan_fuchsia

[mcgrathr]

lib/asan/asan_errors.cc

74 ↗

(On Diff #109586)

It's dead code already because Fuchsia doesn't have anything like signal handling (it would take a bunch of changes in generic code to get rid of the references to it in other dead code paths). Defining it would require me to define more functions of dead code just to satisfy its references (sanitizer::DescribeSignalOrException, sanitizer::SignalContext::DumpAllRegisters). But there is plenty of dead code cruft already and plenty of stub functions I've defined just because of all the dead code already. And reducing #if complexity is good. So I'll do that.

lib/asan/asan_rtl.cc

370 ↗

(On Diff #109586)

Done in D36385.

lib/asan/asan_shadow_setup.cc

17 ↗

(On Diff #109586)

That would be inconsistent with the rest of the code base.
I don't have an opinion on how this is done, but it should be consistent.
If you also want to change all the *_win, *_mac, *_posix etc. files to a new style where they don't have #if around their whole contents are are excluded at the cmake level instead, that's fine with me and I'm happy to help make the Fuchsia-specific code consistent with the rest of the code base.

lib/asan/asan_thread.cc

273 ↗

(On Diff #109586)

Done in D36385.

[mcgrathr]

Split out D36385. Reduced number of #if's. Moved Fuchsia-specific implementations into asan_fuchsia.cc.

[mcgrathr]

Rebased

[mcgrathr]

Rebased

[vitalybuka]

BTW. You can try to apply for committer access.

[mcgrathr]

Rebased and ready to land once D36385 has landed.
Please land it for me!

Thanks,
Roland

compiler-rt/trunk/lib/asan/CMakeLists.txt

 # Build for the AddressSanitizer runtime support library.
 
 set(ASAN_SOURCES
   asan_allocator.cc
   asan_activation.cc
   asan_debugging.cc
   asan_descriptions.cc
   asan_errors.cc
   asan_fake_stack.cc
   asan_flags.cc
+  asan_fuchsia.cc
   asan_globals.cc
   asan_globals_win.cc
   asan_interceptors.cc
   asan_interceptors_memintrinsics.cc
   asan_linux.cc
   asan_mac.cc
   asan_malloc_linux.cc
   asan_malloc_mac.cc

compiler-rt/trunk/lib/asan/asan_fake_stack.cc

for (uptr i = 0, n = NumberOfFrames(stack_size_log(), class_id); i < n;
       FakeFrame *ff = reinterpret_cast<FakeFrame *>(
           GetFrame(stack_size_log(), class_id, i));
       uptr begin = reinterpret_cast<uptr>(ff);
       callback(begin, begin + FakeStack::BytesInSizeClass(class_id), arg);
     }
   }
 }
 
-#if SANITIZER_LINUX && !SANITIZER_ANDROID
+#if (SANITIZER_LINUX && !SANITIZER_ANDROID) || SANITIZER_FUCHSIA
 static THREADLOCAL FakeStack *fake_stack_tls;
 
 FakeStack *GetTLSFakeStack() {
   return fake_stack_tls;
 }
 void SetTLSFakeStack(FakeStack *fs) {
   fake_stack_tls = fs;
 }
 #else
 FakeStack *GetTLSFakeStack() { return 0; }
 void SetTLSFakeStack(FakeStack *fs) { }
-#endif  // SANITIZER_LINUX && !SANITIZER_ANDROID
+#endif  // (SANITIZER_LINUX && !SANITIZER_ANDROID) || SANITIZER_FUCHSIA
 
 static FakeStack *GetFakeStack() {
   AsanThread *t = GetCurrentThread();
   if (!t) return nullptr;
   return t->fake_stack();
 }
 
 static FakeStack *GetFakeStackFast() {

compiler-rt/trunk/lib/asan/asan_fuchsia.cc

+//===-- asan_fuchsia.cc --------------------------------------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===---------------------------------------------------------------------===//
+//
+// This file is a part of AddressSanitizer, an address sanity checker.
+//
+// Fuchsia-specific details.
+//===---------------------------------------------------------------------===//
+
+#include "sanitizer_common/sanitizer_fuchsia.h"
+#if SANITIZER_FUCHSIA
+
+#include "asan_interceptors.h"
+#include "asan_internal.h"
+#include "asan_stack.h"
+#include "asan_thread.h"
+
+#include <limits.h>
+#include <magenta/sanitizer.h>
+#include <magenta/syscalls.h>
+#include <magenta/threads.h>
+
+namespace __asan {
+
+// The system already set up the shadow memory for us.
+// __sanitizer::GetMaxVirtualAddress has already been called by
+// AsanInitInternal->InitializeHighMemEnd (asan_rtl.cc).
+// Just do some additional sanity checks here.
+void InitializeShadowMemory() {
+  if (Verbosity()) PrintAddressSpaceLayout();
+
+  // Make sure SHADOW_OFFSET doesn't use __asan_shadow_memory_dynamic_address.
+  __asan_shadow_memory_dynamic_address = kDefaultShadowSentinel;
+  DCHECK(kLowShadowBeg != kDefaultShadowSentinel);
+  __asan_shadow_memory_dynamic_address = kLowShadowBeg;
+
+  CHECK_EQ(kShadowGapEnd, kHighShadowBeg - 1);
+  CHECK_EQ(kHighMemEnd, __sanitizer::ShadowBounds.memory_limit - 1);
+  CHECK_EQ(kHighMemBeg, __sanitizer::ShadowBounds.shadow_limit);
+  CHECK_EQ(kHighShadowBeg, __sanitizer::ShadowBounds.shadow_base);
+  CHECK_EQ(kShadowGapEnd, __sanitizer::ShadowBounds.shadow_base - 1);
+  CHECK_EQ(kLowShadowEnd, 0);
+  CHECK_EQ(kLowShadowBeg, 0);
+}
+
+void AsanApplyToGlobals(globals_op_fptr op, const void *needle) {
+  UNIMPLEMENTED();
+}
+
+void AsanCheckDynamicRTPrereqs() {}
+void AsanCheckIncompatibleRT() {}
+void InitializeAsanInterceptors() {}
+
+void *AsanDoesNotSupportStaticLinkage() { return nullptr; }
+
+void InitializePlatformExceptionHandlers() {}
+void AsanOnDeadlySignal(int signo, void *siginfo, void *context) {
+  UNIMPLEMENTED();
+}
+
+// We can use a plain thread_local variable for TSD.
+static thread_local void *per_thread;
+
+void *AsanTSDGet() { return per_thread; }
+
+void AsanTSDSet(void *tsd) { per_thread = tsd; }
+
+// There's no initialization needed, and the passed-in destructor
+// will never be called.  Instead, our own thread destruction hook
+// (below) will call AsanThread::TSDDtor directly.
+void AsanTSDInit(void (*destructor)(void *tsd)) {
+  DCHECK(destructor == &PlatformTSDDtor);
+}
+
+void PlatformTSDDtor(void *tsd) { UNREACHABLE(__func__); }
+
+static inline size_t AsanThreadMmapSize() {
+  return RoundUpTo(sizeof(AsanThread), PAGE_SIZE);
+}
+
+struct AsanThread::InitOptions {
+  uptr stack_bottom, stack_size;
+};
+
+// Shared setup between thread creation and startup for the initial thread.
+static AsanThread *CreateAsanThread(StackTrace *stack, u32 parent_tid,
+                                    uptr user_id, bool detached,
+                                    const char *name, uptr stack_bottom,
+                                    uptr stack_size) {
+  // In lieu of AsanThread::Create.
+  AsanThread *thread = (AsanThread *)MmapOrDie(AsanThreadMmapSize(), __func__);
+
+  AsanThreadContext::CreateThreadContextArgs args = {thread, stack};
+  u32 tid =
+      asanThreadRegistry().CreateThread(user_id, detached, parent_tid, &args);
+  asanThreadRegistry().SetThreadName(tid, name);
+
+  // On other systems, AsanThread::Init() is called from the new
+  // thread itself.  But on Fuchsia we already know the stack address
+  // range beforehand, so we can do most of the setup right now.
+  const AsanThread::InitOptions options = {stack_bottom, stack_size};
+  thread->Init(&options);
+
+  return thread;
+}
+
+// This gets the same arguments passed to Init by CreateAsanThread, above.
+// We're in the creator thread before the new thread is actually started,
+// but its stack address range is already known.  We don't bother tracking
+// the static TLS address range because the system itself already uses an
+// ASan-aware allocator for that.
+void AsanThread::SetThreadStackAndTls(const AsanThread::InitOptions *options) {
+  DCHECK_NE(GetCurrentThread(), this);
+  DCHECK_NE(GetCurrentThread(), nullptr);
+  CHECK_NE(options->stack_bottom, 0);
+  CHECK_NE(options->stack_size, 0);
+  stack_bottom_ = options->stack_bottom;
+  stack_top_ = options->stack_bottom + options->stack_size;
+}
+
+// Called by __asan::AsanInitInternal (asan_rtl.c).
+AsanThread *CreateMainThread() {
+  thrd_t self = thrd_current();
+  char name[MX_MAX_NAME_LEN];
+  CHECK_NE(__sanitizer::MainThreadStackBase, 0);
+  CHECK_GT(__sanitizer::MainThreadStackSize, 0);
+  AsanThread *t = CreateAsanThread(
+      nullptr, 0, reinterpret_cast<uptr>(self), true,
+      _mx_object_get_property(thrd_get_mx_handle(self), MX_PROP_NAME, name,
+                              sizeof(name)) == MX_OK
+          ? name
+          : nullptr,
+      __sanitizer::MainThreadStackBase, __sanitizer::MainThreadStackSize);
+  SetCurrentThread(t);
+  return t;
+}
+
+// This is called before each thread creation is attempted.  So, in
+// its first call, the calling thread is the initial and sole thread.
+static void *BeforeThreadCreateHook(uptr user_id, bool detached,
+                                    const char *name, uptr stack_bottom,
+                                    uptr stack_size) {
+  EnsureMainThreadIDIsCorrect();
+  // Strict init-order checking is thread-hostile.
+  if (flags()->strict_init_order) StopInitOrderChecking();
+
+  GET_STACK_TRACE_THREAD;
+  u32 parent_tid = GetCurrentTidOrInvalid();
+
+  return CreateAsanThread(&stack, parent_tid, user_id, detached, name,
+                          stack_bottom, stack_size);
+}
+
+// This is called after creating a new thread (in the creating thread),
+// with the pointer returned by BeforeThreadCreateHook (above).
+static void ThreadCreateHook(void *hook, bool aborted) {
+  AsanThread *thread = static_cast<AsanThread *>(hook);
+  if (!aborted) {
+    // The thread was created successfully.
+    // ThreadStartHook is already running in the new thread.
+  } else {
+    // The thread wasn't created after all.
+    // Clean up everything we set up in BeforeThreadCreateHook.
+    asanThreadRegistry().FinishThread(thread->tid());
+    UnmapOrDie(thread, AsanThreadMmapSize());
+  }
+}
+
+// This is called in the newly-created thread before it runs anything else,
+// with the pointer returned by BeforeThreadCreateHook (above).
+// cf. asan_interceptors.cc:asan_thread_start
+static void ThreadStartHook(void *hook, uptr os_id) {
+  AsanThread *thread = static_cast<AsanThread *>(hook);
+  SetCurrentThread(thread);
+
+  // In lieu of AsanThread::ThreadStart.
+  asanThreadRegistry().StartThread(thread->tid(), os_id, /*workerthread*/ false,
+                                   nullptr);
+}
+
+// Each thread runs this just before it exits,
+// with the pointer returned by BeforeThreadCreateHook (above).
+// All per-thread destructors have already been called.
+static void ThreadExitHook(void *hook, uptr os_id) {
+  AsanThread::TSDDtor(per_thread);
+}
+
+}  // namespace __asan
+
+// These are declared (in extern "C") by <magenta/sanitizer.h>.
+// The system runtime will call our definitions directly.
+
+void *__sanitizer_before_thread_create_hook(thrd_t thread, bool detached,
+                                            const char *name, void *stack_base,
+                                            size_t stack_size) {
+  return __asan::BeforeThreadCreateHook(
+      reinterpret_cast<uptr>(thread), detached, name,
+      reinterpret_cast<uptr>(stack_base), stack_size);
+}
+
+void __sanitizer_thread_create_hook(void *hook, thrd_t thread, int error) {
+  __asan::ThreadCreateHook(hook, error != thrd_success);
+}
+
+void __sanitizer_thread_start_hook(void *hook, thrd_t self) {
+  __asan::ThreadStartHook(hook, reinterpret_cast<uptr>(self));
+}
+
+void __sanitizer_thread_exit_hook(void *hook, thrd_t self) {
+  __asan::ThreadExitHook(hook, reinterpret_cast<uptr>(self));
+}
+
+#endif  // SANITIZER_FUCHSIA

compiler-rt/trunk/lib/asan/asan_mapping.h

 static const u64 kPPC64_ShadowOffset64 = 1ULL << 41;
 static const u64 kSystemZ_ShadowOffset64 = 1ULL << 52;
 static const u64 kFreeBSD_ShadowOffset32 = 1ULL << 30;  // 0x40000000
 static const u64 kFreeBSD_ShadowOffset64 = 1ULL << 46;  // 0x400000000000
 static const u64 kWindowsShadowOffset32 = 3ULL << 28;  // 0x30000000
 
 #define SHADOW_SCALE kDefaultShadowScale
 
-
-#if SANITIZER_WORDSIZE == 32
+#if SANITIZER_FUCHSIA
+#  define SHADOW_OFFSET (0)
+#elif SANITIZER_WORDSIZE == 32
 #  if SANITIZER_ANDROID
 #    define SHADOW_OFFSET (0)
 #  elif defined(__mips__)
 #    define SHADOW_OFFSET kMIPS32_ShadowOffset32
 #  elif SANITIZER_FREEBSD
 #    define SHADOW_OFFSET kFreeBSD_ShadowOffset32
 #  elif SANITIZER_WINDOWS
 #    define SHADOW_OFFSET kWindowsShadowOffset32

compiler-rt/trunk/lib/asan/asan_poisoning.h

ALWAYS_INLINE void FastPoisonShadow(uptr aligned_beg, uptr aligned_size,
   DCHECK(CanPoisonMemory());
   uptr shadow_beg = MEM_TO_SHADOW(aligned_beg);
   uptr shadow_end = MEM_TO_SHADOW(
       aligned_beg + aligned_size - SHADOW_GRANULARITY) + 1;
   // FIXME: Page states are different on Windows, so using the same interface
   // for mapping shadow and zeroing out pages doesn't "just work", so we should
   // probably provide higher-level interface for these operations.
   // For now, just memset on Windows.
-  if (value ||
-      SANITIZER_WINDOWS == 1 ||
+  if (value || SANITIZER_WINDOWS == 1 ||
+      // TODO(mcgrathr): Fuchsia doesn't allow the shadow mapping to be
+      // changed at all.  It doesn't currently have an efficient means
+      // to zero a bunch of pages, but maybe we should add one.
+      SANITIZER_FUCHSIA == 1 ||
       shadow_end - shadow_beg < common_flags()->clear_shadow_mmap_threshold) {
     REAL(memset)((void*)shadow_beg, value, shadow_end - shadow_beg);
   } else {
     uptr page_size = GetPageSizeCached();
     uptr page_beg = RoundUpTo(shadow_beg, page_size);
     uptr page_end = RoundDownTo(shadow_end, page_size);
 
     if (page_beg >= page_end) {

compiler-rt/trunk/lib/asan/asan_rtl.cc

void NOINLINE __asan_handle_no_return() {
   int local_stack;
   AsanThread *curr_thread = GetCurrentThread();
   uptr PageSize = GetPageSizeCached();
   uptr top, bottom;
   if (curr_thread) {
     top = curr_thread->stack_top();
     bottom = ((uptr)&local_stack - PageSize) & ~(PageSize - 1);
   } else {
+    CHECK(!SANITIZER_FUCHSIA);
     // If we haven't seen this thread, try asking the OS for stack bounds.
     uptr tls_addr, tls_size, stack_size;
     GetThreadStackAndTls(/*main=*/false, &bottom, &stack_size, &tls_addr,
                          &tls_size);
     top = bottom + stack_size;
   }
   static const uptr kMaxExpectedCleanupSize = 64 << 20;  // 64M
   if (top - bottom > kMaxExpectedCleanupSize) {

compiler-rt/trunk/lib/asan/asan_shadow_setup.cc

 //===-- asan_shadow_setup.cc ----------------------------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // This file is a part of AddressSanitizer, an address sanity checker.
 //
 // Set up the shadow memory.
 //===----------------------------------------------------------------------===//
 
+#include "sanitizer_common/sanitizer_platform.h"
+
+// asan_fuchsia.cc has its own InitializeShadowMemory implementation.
+#if !SANITIZER_FUCHSIA
+
 #include "asan_internal.h"
 #include "asan_mapping.h"
 
 namespace __asan {
 
 // ---------------------- mmap -------------------- {{{1
 // Reserve memory range [beg, end].
 // We need to use inclusive range because end+1 may not be representable.
if (full_shadow_is_available) {
     Report("ASan shadow was supposed to be located in the [%p-%p] range.\n",
            shadow_start, kHighShadowEnd);
     DumpProcessMap();
     Die();
   }
 }
 
 }  // namespace __asan
+
+#endif  // !SANITIZER_FUCHSIA

compiler-rt/trunk/lib/asan/asan_thread.cc

void AsanThread::Init() {
   CHECK(AddrIsInMem(stack_top_ - 1));
   ClearShadowForThreadStackAndTLS();
   int local = 0;
   VReport(1, "T%d: stack [%p,%p) size 0x%zx; local=%p\n", tid(),
           (void *)stack_bottom_, (void *)stack_top_, stack_top_ - stack_bottom_,
           &local);
 }
 
+// Fuchsia doesn't use ThreadStart.
+// asan_fuchsia.c defines CreateMainThread and SetThreadStackAndTls.
+#if !SANITIZER_FUCHSIA
+
 thread_return_t AsanThread::ThreadStart(
     tid_t os_id, atomic_uintptr_t *signal_thread_is_registered) {
   Init();
   asanThreadRegistry().StartThread(tid(), os_id, /*workerthread*/ false,
                                    nullptr);
   if (signal_thread_is_registered)
     atomic_store(signal_thread_is_registered, 1, memory_order_release);
 
void AsanThread::SetThreadStackAndTls() {
   stack_top_ = stack_bottom_ + stack_size;
   tls_end_ = tls_begin_ + tls_size;
   dtls_ = DTLS_Get();
 
   int local;
   CHECK(AddrIsInStack((uptr)&local));
 }
 
+#endif  // !SANITIZER_FUCHSIA
+
 void AsanThread::ClearShadowForThreadStackAndTLS() {
   PoisonShadow(stack_bottom_, stack_top_ - stack_bottom_, 0);
   if (tls_begin_ != tls_end_)
     PoisonShadow(tls_begin_, tls_end_ - tls_begin_, 0);
 }
 
 bool AsanThread::GetStackFrameAccessByAddr(uptr addr,
                                            StackFrameAccess *access) {

compiler-rt/trunk/lib/sanitizer_common/sanitizer_fuchsia.cc

 void DisableCoreDumperIfNecessary() {}
 void InstallDeadlySignalHandlers(SignalHandlerType handler) {}
 void SetAlternateSignalStack() {}
 void UnsetAlternateSignalStack() {}
 void InitTlsSize() {}
 
 void PrintModuleMap() {}
 
+void SignalContext::DumpAllRegisters(void *context) { UNIMPLEMENTED(); }
+const char *DescribeSignalOrException(int signo) { UNIMPLEMENTED(); }
+
 struct UnwindTraceArg {
   BufferedStackTrace *stack;
   u32 max_depth;
 };
 
 _Unwind_Reason_Code Unwind_Trace(struct _Unwind_Context *ctx, void *param) {
   UnwindTraceArg *arg = static_cast<UnwindTraceArg *>(param);
   CHECK_LT(arg->stack->size, arg->max_depth);

compiler-rt/trunk/lib/sanitizer_common/sanitizer_thread_registry.cc

return FindThreadContextLocked(FindThreadContextByOsIdCallback,
                                  (void *)os_id);
 }
 
 void ThreadRegistry::SetThreadName(u32 tid, const char *name) {
   BlockingMutexLock l(&mtx_);
   CHECK_LT(tid, n_contexts_);
   ThreadContextBase *tctx = threads_[tid];
   CHECK_NE(tctx, 0);
-  CHECK_EQ(ThreadStatusRunning, tctx->status);
+  CHECK_EQ(SANITIZER_FUCHSIA ? ThreadStatusCreated : ThreadStatusRunning,
+           tctx->status);
   tctx->SetName(name);
 }
 
 void ThreadRegistry::SetThreadNameByUserId(uptr user_id, const char *name) {
   BlockingMutexLock l(&mtx_);
   for (u32 tid = 0; tid < n_contexts_; tid++) {
     ThreadContextBase *tctx = threads_[tid];
     if (tctx != 0 && tctx->user_id == user_id &&