[HWASAN] Implemented LSAN SetLsanTag and IgnoreObjectLocked

[kstoimenov]

reinterpret_cast.

[kstoimenov]

After pull.

[kstoimenov]

compiler-rt/lib/hwasan/hwasan_allocator.cpp

240

This is coped from the ASAN implementation. I think the idea is that if DisabledInThisThread in this thread is true the chunks are marked as ignored.

[kstoimenov]

Moved makefile.

[vitalybuka]

compiler-rt/lib/hwasan/hwasan_allocator.cpp

81–82

400

you need Metadata, not a view to edit tags

compiler-rt/lib/hwasan/hwasan_allocator.h

99 ↗

(On Diff #489046)

this is View, read only

109 ↗

(On Diff #489046)

we don't need it header, make it static in cpp file

[kstoimenov]

Removed FindHeapChunkByAddressFastLocked from header.

[kstoimenov]

compiler-rt/lib/hwasan/hwasan_allocator.cpp

400

But the Metadata class doesn't have IsAllocated and Beg methods which are needed in PointsIntoChunk for example.

[vitalybuka]

update

[vitalybuka]

update

[vitalybuka]

compiler-rt/lib/asan/asan_allocator.cpp

1097

this needs to be fixed

compiler-rt/lib/hwasan/hwasan_allocator.cpp

400

It's implemented as

bool HwasanChunkView::IsAllocated() const {
  return metadata_ && metadata_->IsAllocated();
}

531

this needs to be fixed

537

Same as above

[mysterymath]

Hi, it looks like this change broke the Fuchsia toolchain build with an undefined symbol reference. Would you mind fixing this forward if it's easy enough, and reverting otherwise?

https://bugs.fuchsia.dev/p/fuchsia/issues/detail?id=120168

FAILED: /opt/s/w/ir/x/w/staging/llvm_build/lib/clang/16/lib/x86_64-unknown-fuchsia/libclang_rt.hwasan.so 
ld.lld: error: undefined symbol: __lsan::UseExitcodeOnLeak()
>>> referenced by lsan_common_fuchsia.cpp:61 (../staging/llvm_build/runtimes/runtimes-x86_64-unknown-fuchsia-bins/compiler-rt/lib/lsan/lsan_common_fuchsia.cpp:61)
>>>               compiler-rt/lib/lsan/CMakeFiles/RTLSanCommon.x86_64.dir/lsan_common_fuchsia.cpp.obj:(__lsan::ExitHook(int))
>>> referenced by lsan_common_fuchsia.cpp:61 (../staging/llvm_build/runtimes/runtimes-x86_64-unknown-fuchsia-bins/compiler-rt/lib/lsan/lsan_common_fuchsia.cpp:61)
>>>               compiler-rt/lib/lsan/CMakeFiles/RTLSanCommon.x86_64.dir/lsan_common_fuchsia.cpp.obj:(__sanitizer_process_exit_hook)
clang++: error: ld.lld command failed with exit code 1 (use -v to see invocation)

[vitalybuka]

In D141642#4063380, @mysterymath wrote:

Hi, it looks like this change broke the Fuchsia toolchain build with an undefined symbol reference. Would you mind fixing this forward if it's easy enough, and reverting otherwise?

https://bugs.fuchsia.dev/p/fuchsia/issues/detail?id=120168

FAILED: /opt/s/w/ir/x/w/staging/llvm_build/lib/clang/16/lib/x86_64-unknown-fuchsia/libclang_rt.hwasan.so 
ld.lld: error: undefined symbol: __lsan::UseExitcodeOnLeak()
>>> referenced by lsan_common_fuchsia.cpp:61 (../staging/llvm_build/runtimes/runtimes-x86_64-unknown-fuchsia-bins/compiler-rt/lib/lsan/lsan_common_fuchsia.cpp:61)
>>>               compiler-rt/lib/lsan/CMakeFiles/RTLSanCommon.x86_64.dir/lsan_common_fuchsia.cpp.obj:(__lsan::ExitHook(int))
>>> referenced by lsan_common_fuchsia.cpp:61 (../staging/llvm_build/runtimes/runtimes-x86_64-unknown-fuchsia-bins/compiler-rt/lib/lsan/lsan_common_fuchsia.cpp:61)
>>>               compiler-rt/lib/lsan/CMakeFiles/RTLSanCommon.x86_64.dir/lsan_common_fuchsia.cpp.obj:(__sanitizer_process_exit_hook)
clang++: error: ld.lld command failed with exit code 1 (use -v to see invocation)

@kstoimenov
hwasan needs similar to asans bool UseExitcodeOnLeak() { return __asan::flags()->halt_on_error; }

[kstoimenov]

In D141642#4063380, @mysterymath wrote:

Hi, it looks like this change broke the Fuchsia toolchain build with an undefined symbol reference. Would you mind fixing this forward if it's easy enough, and reverting otherwise?

https://bugs.fuchsia.dev/p/fuchsia/issues/detail?id=120168

FAILED: /opt/s/w/ir/x/w/staging/llvm_build/lib/clang/16/lib/x86_64-unknown-fuchsia/libclang_rt.hwasan.so 
ld.lld: error: undefined symbol: __lsan::UseExitcodeOnLeak()
>>> referenced by lsan_common_fuchsia.cpp:61 (../staging/llvm_build/runtimes/runtimes-x86_64-unknown-fuchsia-bins/compiler-rt/lib/lsan/lsan_common_fuchsia.cpp:61)
>>>               compiler-rt/lib/lsan/CMakeFiles/RTLSanCommon.x86_64.dir/lsan_common_fuchsia.cpp.obj:(__lsan::ExitHook(int))
>>> referenced by lsan_common_fuchsia.cpp:61 (../staging/llvm_build/runtimes/runtimes-x86_64-unknown-fuchsia-bins/compiler-rt/lib/lsan/lsan_common_fuchsia.cpp:61)
>>>               compiler-rt/lib/lsan/CMakeFiles/RTLSanCommon.x86_64.dir/lsan_common_fuchsia.cpp.obj:(__sanitizer_process_exit_hook)
clang++: error: ld.lld command failed with exit code 1 (use -v to see invocation)

Correction: D142057 should fix that.

compiler-rt/lib/asan/asan_allocator.cpp

uptr PointsIntoChunk(void *p) {
   if (m->AddrIsInside(addr))
     return chunk;
   if (IsSpecialCaseOfOperatorNew0(chunk, m->UsedSize(), addr))
     return chunk;
   return 0;
 }
 
 uptr GetUserBegin(uptr chunk) {
+  // FIXME: All usecases provide chunk address, GetAsanChunkByAddrFastLocked is
+  // not needed.
   __asan::AsanChunk *m = __asan::instance.GetAsanChunkByAddrFastLocked(chunk);
   return m ? m->Beg() : 0;
 }
 
 LsanMetadata::LsanMetadata(uptr chunk) {
   metadata_ = chunk ? reinterpret_cast<void *>(chunk - __asan::kChunkHeaderSize)
                     : nullptr;
 }

compiler-rt/lib/hwasan/CMakeLists.txt

add_compiler_rt_runtime(${hwasan_runtime}
     ARCHS ${arch}
     OBJECT_LIBS ${hwasan_object_lib}
                 RTHwasan_preinit
                 RTInterception
                 RTSanitizerCommon
                 RTSanitizerCommonLibc
                 RTSanitizerCommonCoverage
                 RTSanitizerCommonSymbolizer
+                RTLSanCommon
                 RTUbsan
     CFLAGS ${hwasan_rtl_flags}
     PARENT_TARGET hwasan)
   add_compiler_rt_runtime(${hwasan_runtime}_cxx
     STATIC
     ARCHS ${arch}
     OBJECT_LIBS RTHwasan_cxx
                 RTUbsan_cxx
add_compiler_rt_runtime(${hwasan_runtime}
     ARCHS ${arch}
     OBJECT_LIBS
             ${hwasan_object_dyn_lib}
             RTInterception
             RTSanitizerCommon
             RTSanitizerCommonLibc
             RTSanitizerCommonCoverage
             RTSanitizerCommonSymbolizer
+            RTLSanCommon
             RTUbsan
             RTUbsan_cxx
             # The only purpose of RTHWAsan_dynamic_version_script_dummy is to
             # carry a dependency of the shared runtime on the version script.
             # Replacing it with a straightforward
             # add_dependencies(clang_rt.asan-dynamic-${arch} clang_rt.asan-dynamic-${arch}-version-list)
             # generates an order-only dependency in ninja.
             RTHwasan_dynamic_version_script_dummy

compiler-rt/lib/hwasan/hwasan_allocator.cpp

 
 bool HwasanChunkView::AddrIsInside(uptr addr) const {
   return (addr >= Beg()) && (addr < Beg() + UsedSize());
 }
 
 inline void Metadata::SetAllocated(u32 stack, u64 size) {
   Thread *t = GetCurrentThread();
   u64 context = t ? t->unique_id() : kMainTid;
   context <<= 32;
   context += stack;
   requested_size_low = size & ((1ul << 32) - 1);
   requested_size_high = size >> 32;
   atomic_store(&alloc_context_id, context, memory_order_relaxed);
   atomic_store(&chunk_state, CHUNK_ALLOCATED, memory_order_release);
 }
 
 inline void Metadata::SetUnallocated() {
   atomic_store(&chunk_state, CHUNK_INVALID, memory_order_release);
if (flags()->tag_in_malloc && malloc_bisect(stack, orig_size)) {
       }
     } else {
       user_ptr = (void *)TagMemoryAligned((uptr)user_ptr, size, 0);
     }
   }
 
   Metadata *meta =
       reinterpret_cast<Metadata *>(allocator.GetMetaData(allocated));
+#if CAN_SANITIZE_LEAKS
+  meta->SetLsanTag(__lsan::DisabledInThisThread() ? __lsan::kIgnored
+                                                  : __lsan::kDirectlyLeaked);
+#endif
   meta->SetAllocated(StackDepotPut(*stack), orig_size);
   RunMallocHooks(user_ptr, size);
   return user_ptr;
 }
 
 static bool PointerAndMemoryTagsMatch(void *tagged_ptr) {
   CHECK(tagged_ptr);
   uptr tagged_uptr = reinterpret_cast<uptr>(tagged_ptr);
HwasanChunkView FindHeapChunkByAddress(uptr address) {
   void *block = allocator.GetBlockBegin(reinterpret_cast<void*>(address));
   if (!block)
     return HwasanChunkView();
   Metadata *metadata =
       reinterpret_cast<Metadata*>(allocator.GetMetaData(block));
   return HwasanChunkView(reinterpret_cast<uptr>(block), metadata);
 }
 
+static inline HwasanChunkView FindHeapChunkByAddressFastLocked(uptr address) {
+  void *block =
+      allocator.GetBlockBeginFastLocked(reinterpret_cast<void *>(address));
+  if (!block)
+    return HwasanChunkView();
+  Metadata *metadata =
+      reinterpret_cast<Metadata *>(allocator.GetMetaData(block));
+  return HwasanChunkView(reinterpret_cast<uptr>(block), metadata);
+}
+
 static uptr AllocationSize(const void *tagged_ptr) {
   const void *untagged_ptr = UntagPtr(tagged_ptr);
   if (!untagged_ptr) return 0;
   const void *beg = allocator.GetBlockBegin(untagged_ptr);
   Metadata *b = (Metadata *)allocator.GetMetaData(untagged_ptr);
   if (beg != untagged_ptr) return 0;
   return b->GetRequestedSize();
 }

 
 void GetAllocatorGlobalRange(uptr *begin, uptr *end) {
   *begin = (uptr)&__hwasan::allocator;
   *end = *begin + sizeof(__hwasan::allocator);
 }
 
 uptr PointsIntoChunk(void *p) {
   uptr addr = reinterpret_cast<uptr>(p);
-  __hwasan::HwasanChunkView view = __hwasan::FindHeapChunkByAddress(addr);
-  if (!view.IsAllocated()) 
+  __hwasan::HwasanChunkView view =
+      __hwasan::FindHeapChunkByAddressFastLocked(addr);
+  if (!view.IsAllocated())
     return 0;
   uptr chunk = view.Beg();
   if (view.AddrIsInside(addr))
     return chunk;
   if (IsSpecialCaseOfOperatorNew0(chunk, view.UsedSize(), addr))
     return chunk;
   return 0;
 }
 
 uptr GetUserBegin(uptr chunk) {
-  return __hwasan::FindHeapChunkByAddress(chunk).Beg();
+  // FIXME: All usecases provide chunk address, FindHeapChunkByAddressFastLocked
+  // is not needed.
+  return __hwasan::FindHeapChunkByAddressFastLocked(chunk).Beg();
 }
 
 LsanMetadata::LsanMetadata(uptr chunk) {
   metadata_ = chunk ? reinterpret_cast<__hwasan::Metadata *>(
                           chunk - __hwasan::kChunkHeaderSize)
                     : nullptr;
 }
 
 bool LsanMetadata::allocated() const {
   if (!metadata_)
     return false;
   __hwasan::Metadata *m = reinterpret_cast<__hwasan::Metadata *>(metadata_);
u32 LsanMetadata::stack_trace_id() const {
   __hwasan::Metadata *m = reinterpret_cast<__hwasan::Metadata *>(metadata_);
   return m->GetAllocStackId();
 }
 
 void ForEachChunk(ForEachChunkCallback callback, void *arg) {
   __hwasan::allocator.ForEachChunk(callback, arg);
 }
 
+IgnoreObjectResult IgnoreObjectLocked(const void *p) {
+  void *block =
+      __hwasan::allocator.GetBlockBeginFastLocked(const_cast<void *>(p));
+  if (!block)
+    return kIgnoreObjectInvalid;
+  __hwasan::Metadata *metadata = reinterpret_cast<__hwasan::Metadata *>(
+      __hwasan::allocator.GetMetaData(block));
+  uptr addr = reinterpret_cast<uptr>(p);
+  __hwasan::HwasanChunkView view(reinterpret_cast<uptr>(block), metadata);
+  if (!view.IsAllocated() || !view.AddrIsInside(addr)) {
+    return kIgnoreObjectInvalid;
+  }
+  if (metadata->GetLsanTag() == kIgnored)
+    return kIgnoreObjectAlreadyIgnored;
+  metadata->SetLsanTag(kIgnored);
+  return kIgnoreObjectSuccess;
+}
+
 }  // namespace __lsan
 
 using namespace __hwasan;
 
 void __hwasan_enable_allocator_tagging() {
   atomic_store_relaxed(&hwasan_allocator_tagging_enabled, 1);
 }