[compiler-rt][hwasan] Refactor Thread::Init

[leonardchan]

Summary

This allows for other implementations to define their own version of Thread::Init. This will be the case for Fuchsia where much of the thread initialization can be broken up between different thread hooks (__sanitizer_before_thread_create_hook, __sanitizer_thread_create_hook, __sanitizer_thread_start_hook). Namely, setting up the heap ring buffer and stack info and can be setup before thread creation. The stack ring buffer can also be setup before thread creation, but storing it into __hwasan_tls can only be done on the thread start hook since it's only then we can access __hwasan_tls for that thread correctly.

[mcgrathr]

compiler-rt/lib/hwasan/hwasan_linux.cpp

430

Most of this code can actually be reused for Fuchsia (just not necessarily in Thread::Init).
It's probably better to split it up for reuse rather than just moving the whole thing to linux-specific.

[leonardchan]

compiler-rt/lib/hwasan/hwasan_linux.cpp

430

So maybe something like the current patch? Then on Fuchsia we could (1) have a custom implementation of InitStackAndTls, (2) wrap the call to Thread::InitStackRingBuffer in Thread::Init with a #if SANITIZER_FUCHSIA so it's not called before thread creation, then (2) call Thread::InitStackRingBuffer in the __sanitizer_thread_start_hook hook.

[leonardchan]

compiler-rt/lib/hwasan/hwasan_linux.cpp

430

Sorry. Meant wrapping with a #if !SANITIZER_FUCHSIA.

[vitalybuka]

compiler-rt/lib/hwasan/hwasan_linux.cpp

430

in other sanitizers only GetThreadStackAndTls is platform specific

compiler-rt/lib/hwasan/hwasan_thread.cpp

41

clang-format: please reformat the code

please fix

[mcgrathr]

compiler-rt/lib/hwasan/hwasan_linux.cpp

430

in other sanitizers only GetThreadStackAndTls is platform specific

That's not entirely accurate. AsanThread::SetThreadStackAndTls might be a model here. But asan on Fuchsia also splits up the initialization into pre-creation and on-created-thread portions (AsanThread::Init is called before thread creation, and AsanThreadRegistry::StartThread is called on the created thread).

[leonardchan]

Rebased and formatted.

[vitalybuka]

compiler-rt/lib/hwasan/hwasan_linux.cpp

430

Patch itself is LGTM.
But I am not sure if @mcgrathr is asking for particular changes.

[leonardchan]

Will submit around end of day if there's no more comments.

compiler-rt/lib/hwasan/hwasan_linux.cpp

void HwasanOnDeadlySignal(int signo, void *info, void *context) {
   // Probably a tag mismatch.
   if (signo == SIGTRAP)
     if (HwasanOnSIGTRAP(signo, (siginfo_t *)info, (ucontext_t*)context))
       return;
 
   HandleDeadlySignal(info, context, GetTid(), &OnStackUnwind, nullptr);
 }
 
+void Thread::Init(uptr stack_buffer_start, uptr stack_buffer_size) {
+  CHECK_EQ(0, unique_id_);  // try to catch bad stack reuse
+  CHECK_EQ(0, stack_top_);
+  CHECK_EQ(0, stack_bottom_);
+
+  static u64 unique_id;
+  unique_id_ = unique_id++;
+  if (auto sz = flags()->heap_history_size)
+    heap_allocations_ = HeapAllocationsRingBuffer::New(sz);
+
+  HwasanTSDThreadInit();  // Only needed with interceptors.
+  uptr *ThreadLong = GetCurrentThreadLongPtr();
+  // The following implicitly sets (this) as the current thread.
+  stack_allocations_ = new (ThreadLong)
+      StackAllocationsRingBuffer((void *)stack_buffer_start, stack_buffer_size);
+  // Check that it worked.
+  CHECK_EQ(GetCurrentThread(), this);
+
+  // ScopedTaggingDisable needs GetCurrentThread to be set up.
+  ScopedTaggingDisabler disabler;
+
+  uptr tls_size;
+  uptr stack_size;
+  GetThreadStackAndTls(IsMainThread(), &stack_bottom_, &stack_size, &tls_begin_,
+                       &tls_size);
+  stack_top_ = stack_bottom_ + stack_size;
+  tls_end_ = tls_begin_ + tls_size;
+
+  if (stack_bottom_) {
+    int local;
+    CHECK(AddrIsInStack((uptr)&local));
+    CHECK(MemIsApp(stack_bottom_));
+    CHECK(MemIsApp(stack_top_ - 1));
+  }
+
+  if (flags()->verbose_threads) {
+    if (IsMainThread()) {
+      Printf("sizeof(Thread): %zd sizeof(HeapRB): %zd sizeof(StackRB): %zd\n",
+             sizeof(Thread), heap_allocations_->SizeInBytes(),
+             stack_allocations_->size() * sizeof(uptr));
+    }
+    Print("Creating  : ");
+  }
+}
 
 } // namespace __hwasan
 
 // Entry point for interoperability between __hwasan_tag_mismatch (ASM) and the
 // rest of the mismatch handling code (C++).
 void __hwasan_tag_mismatch4(uptr addr, uptr access_info, uptr *registers_frame,
                             size_t outsize) {
   __hwasan::HwasanTagMismatch(addr, access_info, registers_frame, outsize);
 }
 
 #endif // SANITIZER_FREEBSD || SANITIZER_LINUX || SANITIZER_NETBSD

compiler-rt/lib/hwasan/hwasan_thread.cpp

void Thread::InitRandomState() {
   random_state_ = flags()->random_tags ? RandomSeed() : unique_id_;
 
   // Push a random number of zeros onto the ring buffer so that the first stack
   // tag base will be random.
   for (tag_t i = 0, e = GenerateRandomTag(); i != e; ++i)
     stack_allocations_->push(0);
 }
 
-void Thread::Init(uptr stack_buffer_start, uptr stack_buffer_size) {
-  CHECK_EQ(0, unique_id_);  // try to catch bad stack reuse
-  CHECK_EQ(0, stack_top_);
-  CHECK_EQ(0, stack_bottom_);
-
-  static u64 unique_id;
-  unique_id_ = unique_id++;
-  if (auto sz = flags()->heap_history_size)
-    heap_allocations_ = HeapAllocationsRingBuffer::New(sz);
-
-  HwasanTSDThreadInit();  // Only needed with interceptors.
-  uptr *ThreadLong = GetCurrentThreadLongPtr();
-  // The following implicitly sets (this) as the current thread.
-  stack_allocations_ = new (ThreadLong)
-      StackAllocationsRingBuffer((void *)stack_buffer_start, stack_buffer_size);
-  // Check that it worked.
-  CHECK_EQ(GetCurrentThread(), this);
-
-  // ScopedTaggingDisable needs GetCurrentThread to be set up.
-  ScopedTaggingDisabler disabler;
-
-  uptr tls_size;
-  uptr stack_size;
-  GetThreadStackAndTls(IsMainThread(), &stack_bottom_, &stack_size, &tls_begin_,
-                       &tls_size);
-  stack_top_ = stack_bottom_ + stack_size;
-  tls_end_ = tls_begin_ + tls_size;
-
-  if (stack_bottom_) {
-    int local;
-    CHECK(AddrIsInStack((uptr)&local));
-    CHECK(MemIsApp(stack_bottom_));
-    CHECK(MemIsApp(stack_top_ - 1));
-  }
-
-  if (flags()->verbose_threads) {
-    if (IsMainThread()) {
-      Printf("sizeof(Thread): %zd sizeof(HeapRB): %zd sizeof(StackRB): %zd\n",
-             sizeof(Thread), heap_allocations_->SizeInBytes(),
-             stack_allocations_->size() * sizeof(uptr));
-    }
-    Print("Creating  : ");
-  }
-}
-
 void Thread::ClearShadowForThreadStackAndTLS() {
   if (stack_top_ != stack_bottom_)
     TagMemory(stack_bottom_, stack_top_ - stack_bottom_, 0);
   if (tls_begin_ != tls_end_)
     TagMemory(tls_begin_, tls_end_ - tls_begin_, 0);
 }
 
 void Thread::Destroy() {
   if (flags()->verbose_threads)
     Print("Destroying: ");
   AllocatorSwallowThreadLocalCache(allocator_cache());
   ClearShadowForThreadStackAndTLS();
   if (heap_allocations_)