[WIP][clang][Fuchsia] Support HWASan for Fuchsia

[leonardchan]

Summary

This tracks all the compiler/llvm changes needed for building Fuchsia with the HWASan variant.

[pcc]

How does Zircon handle tagged addresses in syscalls? Are they handled equivalently to Linux's tagged address ABI?

[leonardchan]

Locally I'm able to build and run the bringup.arm64 config (that is launch the shell and run some commands).

[leonardchan]

In D91466#2395297, @pcc wrote:

How does Zircon handle tagged addresses in syscalls? Are they handled equivalently to Linux's tagged address ABI?

Woops, accidentally let this slide. I'm guessing you saw this in the Fuchsia patch, but I believe we haven't made a final decision yet on how syscalls will handle tagged pointers.

[phosek]

clang/lib/Driver/ToolChains/Fuchsia.cpp

100–101

Nit: can you move this above TSan so it's alphabetically sorted?

215

215–222

Can you move this above relative-vtables so we keep all sanitizers together?

218

compiler-rt/lib/hwasan/hwasan_fuchsia.cpp

123–125

I don't think this applies to Fuchsia.

compiler-rt/lib/hwasan/hwasan_poisoning.cpp

22

It might be better to move this function to a per-platform file rather than conditionally compiling the entire body.

[phosek]

@leonardchan would it be possible to break up this patch into multiple changes:

• Clang driver
• Cache file
• hwasan_new_delete.cpp
• HWAddressSanitizer.cpp
• sanitizer_platform_limits_fuchsia.h
• sanitizer_symbolizer_markup.cpp
• the rest

[leonardchan]

Rebase against submitted changes.

[leonardchan]

Rebased.

[leonardchan]

compiler-rt/lib/hwasan/hwasan_dynamic_shadow.cpp

120

Add comment. Fuchsia never uses dynamic shadow, which is what this sets up in the Linux implementation.

From linux implemenation:

// Call the ifunc resolver for __hwasan_shadow and fill in its GOT entry. This
// needs to be done before other ifunc resolvers (which are handled by libc)
// because a resolver might read __hwasan_shadow.

compiler-rt/lib/hwasan/hwasan_fuchsia.cpp

47–52

Compare hwasan thread tracking implementation with asan + fuchsia implementation. HWASan uses different sanitizer machinery.

54

This can be replaced with a zircon-equivalent check that checks if TBI is enabled. This function is Linux-specific.

TODO: File a bug that tracks this once the TBI stuff is addressed.

61

See equivalent for other asan+lsan implementations. We use fuchsia sanitizer hooks instead of atexit. __sanitizer_exit_hook is what's used.

65–75

Don't define these. Move this logic into the hooks.

Ask why these are in the public header when we already have the thread hooks.

106–112

This is compiler-defined and should be abstracted out. Shouldn't have duplicate definitions.

106–131

This could probably be shared code with the Linux implementation. Move out of hwasan.

147–148

Ideally, the checks on these uninitialized globals should not be called.

[leonardchan]

Update to some feedback from first round of reviews. Minor refactoring of code that can be shared between the linux and fuchsia implementations. Still need to discuss refactoring the hwasan threads implementation.

clang/lib/Driver/ToolChains/Fuchsia.cpp

void fuchsia::Linker::ConstructJob(Compilation &C, const JobAction &JA,
   const SanitizerArgs &SanArgs = ToolChain.getSanitizerArgs();
 
   if (!Args.hasArg(options::OPT_shared)) {
     std::string Dyld = D.DyldPrefix;
     if (SanArgs.needsAsanRt() && SanArgs.needsSharedRt())
       Dyld += "asan/";
     if (SanArgs.needsTsanRt() && SanArgs.needsSharedRt())
       Dyld += "tsan/";
+    if (SanArgs.needsHwasanRt() && SanArgs.needsSharedRt())
+      Dyld += "hwasan/";
     Dyld += "ld.so.1";
     CmdArgs.push_back("-dynamic-linker");
     CmdArgs.push_back(Args.MakeArgString(Dyld));
   }
 
   CmdArgs.push_back("-o");
   CmdArgs.push_back(Output.getFilename());
 
Fuchsia::Fuchsia(const Driver &D, const llvm::Triple &Triple,
   // ASan has higher priority because we always want the instrumentated version.
   Multilibs.push_back(Multilib("asan", {}, {}, 2)
                           .flag("+fsanitize=address"));
   // Use the asan+noexcept variant with ASan and -fno-exceptions.
   Multilibs.push_back(Multilib("asan+noexcept", {}, {}, 3)
                           .flag("+fsanitize=address")
                           .flag("-fexceptions")
                           .flag("+fno-exceptions"));
   Multilibs.FilterOut([&](const Multilib &M) {
     std::vector<std::string> RD = FilePaths(M);
     return std::all_of(RD.begin(), RD.end(), [&](std::string P) {
       return !getVFS().exists(P);
     });
   });
 
   Multilib::flags_list Flags;
   addMultilibFlag(
       Args.hasFlag(options::OPT_fexceptions, options::OPT_fno_exceptions, true),
       "fexceptions", Flags);
   addMultilibFlag(getSanitizerArgs().needsAsanRt(), "fsanitize=address", Flags);
   Multilibs.setFilePathsCallback(FilePaths);
 
   if (Multilibs.select(Flags, SelectedMultilib))
     if (!SelectedMultilib.isDefault())
void Fuchsia::AddCXXStdlibLibArgs(const ArgList &Args,
   case ToolChain::CST_Libstdcxx:
     llvm_unreachable("invalid stdlib name");
   }
 }
 
 SanitizerMask Fuchsia::getSupportedSanitizers() const {
   SanitizerMask Res = ToolChain::getSupportedSanitizers();
   Res |= SanitizerKind::Address;
+  Res |= SanitizerKind::HWAddress;
   Res |= SanitizerKind::PointerCompare;
   Res |= SanitizerKind::PointerSubtract;
   Res |= SanitizerKind::Fuzzer;
   Res |= SanitizerKind::FuzzerNoLink;
   Res |= SanitizerKind::Leak;
   Res |= SanitizerKind::SafeStack;
   Res |= SanitizerKind::Scudo;
   Res |= SanitizerKind::Thread;

compiler-rt/cmake/config-ix.cmake

 if (COMPILER_RT_HAS_SANITIZER_COMMON AND MSAN_SUPPORTED_ARCH AND
     OS_NAME MATCHES "Linux|FreeBSD|NetBSD")
   set(COMPILER_RT_HAS_MSAN TRUE)
 else()
   set(COMPILER_RT_HAS_MSAN FALSE)
 endif()
 
 if (COMPILER_RT_HAS_SANITIZER_COMMON AND HWASAN_SUPPORTED_ARCH AND
-    OS_NAME MATCHES "Linux|Android")
+    OS_NAME MATCHES "Linux|Android|Fuchsia")
   set(COMPILER_RT_HAS_HWASAN TRUE)
 else()
   set(COMPILER_RT_HAS_HWASAN FALSE)
 endif()
 
 if (COMPILER_RT_HAS_SANITIZER_COMMON AND MEMPROF_SUPPORTED_ARCH AND
     OS_NAME MATCHES "Linux")
   set(COMPILER_RT_HAS_MEMPROF TRUE)

compiler-rt/lib/hwasan/CMakeLists.txt

 include_directories(..)
 
 # Runtime library sources and build flags.
 set(HWASAN_RTL_SOURCES
   hwasan.cpp
   hwasan_allocator.cpp
   hwasan_dynamic_shadow.cpp
   hwasan_exceptions.cpp
+  hwasan_fuchsia.cpp
   hwasan_globals.cpp
   hwasan_interceptors.cpp
   hwasan_interceptors_vfork.S
   hwasan_linux.cpp
   hwasan_memintrinsics.cpp
   hwasan_poisoning.cpp
   hwasan_report.cpp
   hwasan_setjmp.S
set(HWASAN_RTL_HEADERS
   hwasan_mapping.h
   hwasan_poisoning.h
   hwasan_report.h
   hwasan_thread.h
   hwasan_thread_list.h
   )
 
 set(HWASAN_DEFINITIONS)
+if (NOT FUCHSIA)
   append_list_if(COMPILER_RT_HWASAN_WITH_INTERCEPTORS HWASAN_WITH_INTERCEPTORS=1 HWASAN_DEFINITIONS)
+endif()
 
 set(HWASAN_RTL_CFLAGS ${SANITIZER_COMMON_CFLAGS})
 append_rtti_flag(OFF HWASAN_RTL_CFLAGS)
 append_list_if(COMPILER_RT_HAS_FPIC_FLAG -fPIC HWASAN_RTL_CFLAGS)
 # Prevent clang from generating libc calls.
 append_list_if(COMPILER_RT_HAS_FFREESTANDING_FLAG -ffreestanding HWASAN_RTL_CFLAGS)
 
 set(HWASAN_DYNAMIC_LINK_FLAGS ${SANITIZER_COMMON_LINK_FLAGS})

compiler-rt/lib/hwasan/hwasan_dynamic_shadow.cpp

 /// region and handles ifunc resolver case, when necessary.
 ///
 //===----------------------------------------------------------------------===//
 
 #include "hwasan.h"
 #include "hwasan_dynamic_shadow.h"
 #include "hwasan_mapping.h"
 #include "sanitizer_common/sanitizer_common.h"
+#if SANITIZER_POSIX
 #include "sanitizer_common/sanitizer_posix.h"
+#endif
 
 #include <elf.h>
 #include <link.h>
 
 // The code in this file needs to run in an unrelocated binary. It should not
 // access any external symbol, including its own non-hidden globals.
 
 #if SANITIZER_ANDROID

 uptr FindDynamicShadowStart(uptr shadow_size_bytes) {
   if (IsPremapShadowAvailable())
     return FindPremappedShadowStart(shadow_size_bytes);
   return MapDynamicShadow(shadow_size_bytes, kShadowScale, kShadowBaseAlignment,
                           kHighMemEnd);
 }
 
 }  // namespace __hwasan
+#elif SANITIZER_FUCHSIA
+
+namespace __hwasan {
+
+void InitShadowGOT() {}
+
+}  // namespace __hwasan
+
 #else
 namespace __hwasan {
 
 void InitShadowGOT() {}
 
 uptr FindDynamicShadowStart(uptr shadow_size_bytes) {
   return MapDynamicShadow(shadow_size_bytes, kShadowScale, kShadowBaseAlignment,
                           kHighMemEnd);
 }
 
 }  // namespace __hwasan
 
 #endif  // SANITIZER_ANDROID

compiler-rt/lib/hwasan/hwasan_fuchsia.cpp

+//===-- hwasan_fuchsia.cpp --------------------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+///
+/// \file
+/// This file is a part of HWAddressSanitizer and contains Fuchsia-specific
+/// code.
+///
+//===----------------------------------------------------------------------===//
+
+#include "sanitizer_common/sanitizer_fuchsia.h"
+#if SANITIZER_FUCHSIA
+
+#include "hwasan.h"
+#include "hwasan_interface_internal.h"
+#include "hwasan_thread.h"
+
+SANITIZER_INTERFACE_ATTRIBUTE
+THREADLOCAL uptr __hwasan_tls;
+
+namespace __hwasan {
+
+bool InitShadow() { return true; }
+
+void InitThreads() {}
+
+void InitPrctl() {}
+
+void MadviseShadow() {}
+
+bool MemIsApp(uptr p) { return false; }
+
+void InstallAtExitHandler() {}
+
+// ---------------------- TSD ---------------- {{{
+
+extern "C" void __hwasan_thread_enter() {}
+
+extern "C" void __hwasan_thread_exit() {}
+
+void HwasanTSDInit() {}
+void HwasanTSDThreadInit() {}
+
+uptr *GetCurrentThreadLongPtr() { return nullptr; }
+
+void AndroidTestTlsSlot() {}
+
+Thread *GetCurrentThread() { return nullptr; }
+
+// Entry point stub for interoperability between __hwasan_tag_mismatch (ASM) and
+// the rest of the mismatch handling code (C++).
+extern "C" void __hwasan_tag_mismatch4(uptr addr, uptr access_info,
+                                       uptr *registers_frame, size_t outsize) {}
+
+void HwasanOnDeadlySignal(int signo, void *info, void *context) {}
+
+}  // namespace __hwasan
+
+#endif  // SANITIZER_FUCHSIA

compiler-rt/lib/hwasan/hwasan_interceptors.cpp

 
 #include "interception/interception.h"
 #include "hwasan.h"
 #include "hwasan_allocator.h"
 #include "hwasan_mapping.h"
 #include "hwasan_thread.h"
 #include "hwasan_poisoning.h"
 #include "hwasan_report.h"
+#if SANITIZER_FUCHSIA
+#include "sanitizer_common/sanitizer_platform_limits_fuchsia.h"
+#endif
+#if SANITIZER_POSIX
 #include "sanitizer_common/sanitizer_platform_limits_posix.h"
+#endif
 #include "sanitizer_common/sanitizer_allocator.h"
 #include "sanitizer_common/sanitizer_allocator_interface.h"
 #include "sanitizer_common/sanitizer_allocator_internal.h"
 #include "sanitizer_common/sanitizer_atomic.h"
 #include "sanitizer_common/sanitizer_common.h"
 #include "sanitizer_common/sanitizer_errno.h"
 #include "sanitizer_common/sanitizer_stackdepot.h"
 #include "sanitizer_common/sanitizer_libc.h"

 INTERCEPTOR(void, longjmp, __hw_jmp_buf env, int val) {
   InternalLongjmp(env[0].__jmpbuf, val);
 }
 #undef SIG_BLOCK
 #undef SIG_SETMASK
 
 #endif // HWASAN_WITH_INTERCEPTORS && __aarch64__
 
+#if HWASAN_WITH_INTERCEPTORS
 static void BeforeFork() {
   StackDepotLockAll();
 }
 
 static void AfterFork() {
   StackDepotUnlockAll();
 }
 
 INTERCEPTOR(int, fork, void) {
   ENSURE_HWASAN_INITED();
   BeforeFork();
   int pid = REAL(fork)();
   AfterFork();
   return pid;
 }
+#endif  // HWASAN_WITH_INTERCEPTORS
 
 namespace __hwasan {
 
 int OnExit() {
   // FIXME: ask frontend whether we need to return failure.
   return 0;
 }
 
 } // namespace __hwasan
 
 namespace __hwasan {
 
 void InitializeInterceptors() {
   static int inited = 0;
   CHECK_EQ(inited, 0);
 
+#if HWASAN_WITH_INTERCEPTORS
   INTERCEPT_FUNCTION(fork);
 
-#if HWASAN_WITH_INTERCEPTORS
 #if defined(__linux__)
   INTERCEPT_FUNCTION(vfork);
 #endif  // __linux__
   INTERCEPT_FUNCTION(pthread_create);
 #endif
 
   inited = 1;
 }
 } // namespace __hwasan

compiler-rt/lib/hwasan/hwasan_interface_internal.h

 //
 // Private Hwasan interface header.
 //===----------------------------------------------------------------------===//
 
 #ifndef HWASAN_INTERFACE_INTERNAL_H
 #define HWASAN_INTERFACE_INTERNAL_H
 
 #include "sanitizer_common/sanitizer_internal_defs.h"
+#if SANITIZER_FUCHSIA
+#include "sanitizer_common/sanitizer_platform_limits_fuchsia.h"
+#endif
+#if SANITIZER_POSIX
 #include "sanitizer_common/sanitizer_platform_limits_posix.h"
+#endif
 #include <link.h>
 
 extern "C" {
 
 SANITIZER_INTERFACE_ATTRIBUTE
 void __hwasan_init_static();
 
 SANITIZER_INTERFACE_ATTRIBUTE

compiler-rt/lib/hwasan/hwasan_poisoning.cpp

 
 #include "hwasan_mapping.h"
 #include "interception/interception.h"
 #include "sanitizer_common/sanitizer_common.h"
 #include "sanitizer_common/sanitizer_linux.h"
 
 namespace __hwasan {
 
 uptr TagMemoryAligned(uptr p, uptr size, tag_t tag) {
   CHECK(IsAligned(p, kShadowAlignment));
   CHECK(IsAligned(size, kShadowAlignment));
+#if SANITIZER_FUCHSIA
+  __sanitizer_fill_shadow(p, size, tag,
+                          common_flags()->clear_shadow_mmap_threshold);
+#else
   uptr shadow_start = MemToShadow(p);
   uptr shadow_size = MemToShadowSize(size);
 
   uptr page_size = GetPageSizeCached();
   uptr page_start = RoundUpTo(shadow_start, page_size);
   uptr page_end = RoundDownTo(shadow_start + shadow_size, page_size);
   uptr threshold = common_flags()->clear_shadow_mmap_threshold;
   if (SANITIZER_LINUX &&
       UNLIKELY(page_end >= page_start + threshold && tag == 0)) {
     internal_memset((void *)shadow_start, tag, page_start - shadow_start);
     internal_memset((void *)page_end, tag,
                     shadow_start + shadow_size - page_end);
     // For an anonymous private mapping MADV_DONTNEED will return a zero page on
     // Linux.
     ReleaseMemoryPagesToOSAndZeroFill(page_start, page_end);
   } else {
     internal_memset((void *)shadow_start, tag, shadow_size);
   }
+#endif
   return AddTagToPointer(p, tag);
 }
 
 uptr TagMemory(uptr p, uptr size, tag_t tag) {
   uptr start = RoundDownTo(p, kShadowAlignment);
   uptr end = RoundUpTo(p + size, kShadowAlignment);
   return TagMemoryAligned(start, end - start, tag);
 }
 
 }  // namespace __hwasan

compiler-rt/lib/hwasan/hwasan_thread.cpp

void Thread::Init(uptr stack_buffer_start, uptr stack_buffer_size) {
   // Check that it worked.
   CHECK_EQ(GetCurrentThread(), this);
 
   // ScopedTaggingDisable needs GetCurrentThread to be set up.
   ScopedTaggingDisabler disabler;
 
   uptr tls_size;
   uptr stack_size;
+// FIXME: Just ignore this so we can build.
+#if !SANITIZER_FUCHSIA
   GetThreadStackAndTls(IsMainThread(), &stack_bottom_, &stack_size, &tls_begin_,
                        &tls_size);
+  tls_size = 0;
+  stack_size = 0;
+#endif
   stack_top_ = stack_bottom_ + stack_size;
   tls_end_ = tls_begin_ + tls_size;
 
   if (stack_bottom_) {
     int local;
     CHECK(AddrIsInStack((uptr)&local));
     CHECK(MemIsApp(stack_bottom_));
     CHECK(MemIsApp(stack_top_ - 1));

compiler-rt/lib/sanitizer_common/sanitizer_platform_limits_fuchsia.h

+//===-- sanitizer_platform_limits_fuchsia.h -------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file is a part of Sanitizer common code.
+//
+// Sizes and layouts of platform-specific Fuchsia data structures.
+//===----------------------------------------------------------------------===//
+
+#ifndef SANITIZER_PLATFORM_LIMITS_FUCHSIA_H
+#define SANITIZER_PLATFORM_LIMITS_FUCHSIA_H
+
+#if SANITIZER_FUCHSIA
+
+namespace __sanitizer {
+struct __sanitizer_struct_mallinfo {};
+}  // namespace __sanitizer
+
+#endif  // SANITIZER_FUCHSIA
+
+#endif

compiler-rt/lib/sanitizer_common/sanitizer_symbolizer_markup.cpp

 // once in UBSan to say "in module ..." in a message that also
 // includes an address in the module, so post-processing can already
 // pretty-print that so as to indicate the module.
 bool Symbolizer::GetModuleNameAndOffsetForPC(uptr pc, const char **module_name,
                                              uptr *module_address) {
   return false;
 }
 
+// TODO: We should also have an offline implementation. This function was
+// initially undefined when building hwasan. It's probably just because no one
+// used this until now that we didn't see this before.
+bool Symbolizer::SymbolizeFrame(uptr addr, FrameInfo *info) { return false; }
+
 // This is used in some places for suppression checking, which we
 // don't really support for Fuchsia.  It's also used in UBSan to
 // identify a PC location to a function name, so we always fill in
 // the function member with a string containing markup around the PC
 // value.
 // TODO(mcgrathr): Under SANITIZER_GO, it's currently used by TSan
 // to render stack frames, but that should be changed to use
 // RenderStackFrame.