[HWASAN] Implemented LSAN SetLsanTag and IgnoreObjectLocked

[kstoimenov]

reinterpret_cast.

[kstoimenov]

After pull.

[kstoimenov]

compiler-rt/lib/hwasan/hwasan_allocator.cpp

247

This is coped from the ASAN implementation. I think the idea is that if DisabledInThisThread in this thread is true the chunks are marked as ignored.

[kstoimenov]

Moved makefile.

[vitalybuka]

compiler-rt/lib/hwasan/hwasan_allocator.cpp

81–82

407

you need Metadata, not a view to edit tags

compiler-rt/lib/hwasan/hwasan_allocator.h

99

this is View, read only

109

we don't need it header, make it static in cpp file

[kstoimenov]

Removed FindHeapChunkByAddressFastLocked from header.

[kstoimenov]

compiler-rt/lib/hwasan/hwasan_allocator.cpp

407

But the Metadata class doesn't have IsAllocated and Beg methods which are needed in PointsIntoChunk for example.

[vitalybuka]

update

[vitalybuka]

update

[vitalybuka]

compiler-rt/lib/asan/asan_allocator.cpp

1097 ↗

(On Diff #490005)

this needs to be fixed

compiler-rt/lib/hwasan/hwasan_allocator.cpp

407

It's implemented as

bool HwasanChunkView::IsAllocated() const {
  return metadata_ && metadata_->IsAllocated();
}

538

this needs to be fixed

541–542

Same as above

[mysterymath]

Hi, it looks like this change broke the Fuchsia toolchain build with an undefined symbol reference. Would you mind fixing this forward if it's easy enough, and reverting otherwise?

https://bugs.fuchsia.dev/p/fuchsia/issues/detail?id=120168

FAILED: /opt/s/w/ir/x/w/staging/llvm_build/lib/clang/16/lib/x86_64-unknown-fuchsia/libclang_rt.hwasan.so 
ld.lld: error: undefined symbol: __lsan::UseExitcodeOnLeak()
>>> referenced by lsan_common_fuchsia.cpp:61 (../staging/llvm_build/runtimes/runtimes-x86_64-unknown-fuchsia-bins/compiler-rt/lib/lsan/lsan_common_fuchsia.cpp:61)
>>>               compiler-rt/lib/lsan/CMakeFiles/RTLSanCommon.x86_64.dir/lsan_common_fuchsia.cpp.obj:(__lsan::ExitHook(int))
>>> referenced by lsan_common_fuchsia.cpp:61 (../staging/llvm_build/runtimes/runtimes-x86_64-unknown-fuchsia-bins/compiler-rt/lib/lsan/lsan_common_fuchsia.cpp:61)
>>>               compiler-rt/lib/lsan/CMakeFiles/RTLSanCommon.x86_64.dir/lsan_common_fuchsia.cpp.obj:(__sanitizer_process_exit_hook)
clang++: error: ld.lld command failed with exit code 1 (use -v to see invocation)

[vitalybuka]

In D141642#4063380, @mysterymath wrote:

Hi, it looks like this change broke the Fuchsia toolchain build with an undefined symbol reference. Would you mind fixing this forward if it's easy enough, and reverting otherwise?

https://bugs.fuchsia.dev/p/fuchsia/issues/detail?id=120168

FAILED: /opt/s/w/ir/x/w/staging/llvm_build/lib/clang/16/lib/x86_64-unknown-fuchsia/libclang_rt.hwasan.so 
ld.lld: error: undefined symbol: __lsan::UseExitcodeOnLeak()
>>> referenced by lsan_common_fuchsia.cpp:61 (../staging/llvm_build/runtimes/runtimes-x86_64-unknown-fuchsia-bins/compiler-rt/lib/lsan/lsan_common_fuchsia.cpp:61)
>>>               compiler-rt/lib/lsan/CMakeFiles/RTLSanCommon.x86_64.dir/lsan_common_fuchsia.cpp.obj:(__lsan::ExitHook(int))
>>> referenced by lsan_common_fuchsia.cpp:61 (../staging/llvm_build/runtimes/runtimes-x86_64-unknown-fuchsia-bins/compiler-rt/lib/lsan/lsan_common_fuchsia.cpp:61)
>>>               compiler-rt/lib/lsan/CMakeFiles/RTLSanCommon.x86_64.dir/lsan_common_fuchsia.cpp.obj:(__sanitizer_process_exit_hook)
clang++: error: ld.lld command failed with exit code 1 (use -v to see invocation)

@kstoimenov
hwasan needs similar to asans bool UseExitcodeOnLeak() { return __asan::flags()->halt_on_error; }

[kstoimenov]

In D141642#4063380, @mysterymath wrote:

Hi, it looks like this change broke the Fuchsia toolchain build with an undefined symbol reference. Would you mind fixing this forward if it's easy enough, and reverting otherwise?

https://bugs.fuchsia.dev/p/fuchsia/issues/detail?id=120168

FAILED: /opt/s/w/ir/x/w/staging/llvm_build/lib/clang/16/lib/x86_64-unknown-fuchsia/libclang_rt.hwasan.so 
ld.lld: error: undefined symbol: __lsan::UseExitcodeOnLeak()
>>> referenced by lsan_common_fuchsia.cpp:61 (../staging/llvm_build/runtimes/runtimes-x86_64-unknown-fuchsia-bins/compiler-rt/lib/lsan/lsan_common_fuchsia.cpp:61)
>>>               compiler-rt/lib/lsan/CMakeFiles/RTLSanCommon.x86_64.dir/lsan_common_fuchsia.cpp.obj:(__lsan::ExitHook(int))
>>> referenced by lsan_common_fuchsia.cpp:61 (../staging/llvm_build/runtimes/runtimes-x86_64-unknown-fuchsia-bins/compiler-rt/lib/lsan/lsan_common_fuchsia.cpp:61)
>>>               compiler-rt/lib/lsan/CMakeFiles/RTLSanCommon.x86_64.dir/lsan_common_fuchsia.cpp.obj:(__sanitizer_process_exit_hook)
clang++: error: ld.lld command failed with exit code 1 (use -v to see invocation)

Correction: D142057 should fix that.

compiler-rt/lib/hwasan/CMakeLists.txt

add_compiler_rt_runtime(${hwasan_runtime}
     ARCHS ${arch}
     OBJECT_LIBS ${hwasan_object_lib}
                 RTHwasan_preinit
                 RTInterception
                 RTSanitizerCommon
                 RTSanitizerCommonLibc
                 RTSanitizerCommonCoverage
                 RTSanitizerCommonSymbolizer
+                RTLSanCommon
                 RTUbsan
     CFLAGS ${hwasan_rtl_flags}
     PARENT_TARGET hwasan)
   add_compiler_rt_runtime(${hwasan_runtime}_cxx
     STATIC
     ARCHS ${arch}
     OBJECT_LIBS RTHwasan_cxx
                 RTUbsan_cxx
add_compiler_rt_runtime(${hwasan_runtime}
     ARCHS ${arch}
     OBJECT_LIBS
             ${hwasan_object_dyn_lib}
             RTInterception
             RTSanitizerCommon
             RTSanitizerCommonLibc
             RTSanitizerCommonCoverage
             RTSanitizerCommonSymbolizer
+            RTLSanCommon
             RTUbsan
             RTUbsan_cxx
             # The only purpose of RTHWAsan_dynamic_version_script_dummy is to
             # carry a dependency of the shared runtime on the version script.
             # Replacing it with a straightforward
             # add_dependencies(clang_rt.asan-dynamic-${arch} clang_rt.asan-dynamic-${arch}-version-list)
             # generates an order-only dependency in ninja.
             RTHwasan_dynamic_version_script_dummy

compiler-rt/lib/hwasan/hwasan_allocator.h

public:
   bool IsAllocated() const;    // Checks if the memory is currently allocated
   uptr Beg() const;            // First byte of user memory
   uptr End() const;            // Last byte of user memory
   uptr UsedSize() const;       // Size requested by the user
   uptr ActualSize() const;     // Size allocated by the allocator.
   u32 GetAllocStackId() const;
   bool FromSmallHeap() const;
   bool AddrIsInside(uptr addr) const;
+  inline void SetLsanTag(__lsan::ChunkTag tag);
+  inline __lsan::ChunkTag GetLsanTag() const;
 
  private:
   friend class __lsan::LsanMetadata;
   uptr block_;
   Metadata *const metadata_;
 };
 
 HwasanChunkView FindHeapChunkByAddress(uptr address);
 
 // Information about one (de)allocation that happened in the past.
 // These are recorded in a thread-local ring buffer.
 // TODO: this is currently 24 bytes (20 bytes + alignment).
 // Compress it to 16 bytes or extend it to be more useful.
 struct HeapAllocationRecord {
   uptr tagged_addr;
   u32  alloc_context_id;
   u32  free_context_id;

compiler-rt/lib/hwasan/hwasan_allocator.cpp

 bool HwasanChunkView::FromSmallHeap() const {
   return allocator.FromPrimary(reinterpret_cast<void *>(block_));
 }
 
 bool HwasanChunkView::AddrIsInside(uptr addr) const {
   return (addr >= Beg()) && (addr < Beg() + UsedSize());
 }
 
+inline void HwasanChunkView::SetLsanTag(__lsan::ChunkTag tag) {
+  CHECK(metadata_);
+  metadata_->SetLsanTag(tag);
+}
+inline __lsan::ChunkTag HwasanChunkView::GetLsanTag() const {
+  CHECK(metadata_);
+  return metadata_->GetLsanTag();
+}
+
 inline void Metadata::SetAllocated(u32 stack, u64 size) {
   Thread *t = GetCurrentThread();
   u64 context = t ? t->unique_id() : kMainTid;
   context <<= 32;
   context += stack;
   requested_size_low = size & ((1ul << 32) - 1);
   requested_size_high = size >> 32;
   atomic_store(&alloc_context_id, context, memory_order_relaxed);

 inline u64 Metadata::GetRequestedSize() const {
   return (static_cast<u64>(requested_size_high) << 32) + requested_size_low;
 }
 
 inline u32 Metadata::GetAllocStackId() const {
   return atomic_load(&alloc_context_id, memory_order_relaxed);
 }
 
-static const uptr kChunkHeaderSize = sizeof(HwasanChunkView);
-
 void GetAllocatorStats(AllocatorStatCounters s) {
   allocator.GetStats(s);
 }
 
 inline void Metadata::SetLsanTag(__lsan::ChunkTag tag) {
   lsan_tag = tag;
 }
 
if (flags()->tag_in_malloc && malloc_bisect(stack, orig_size)) {
       }
     } else {
       user_ptr = (void *)TagMemoryAligned((uptr)user_ptr, size, 0);
     }
   }
 
   Metadata *meta =
       reinterpret_cast<Metadata *>(allocator.GetMetaData(allocated));
+#if CAN_SANITIZE_LEAKS
+  meta->SetLsanTag(__lsan::DisabledInThisThread() ? __lsan::kIgnored
+                                                  : __lsan::kDirectlyLeaked);
+#endif
   meta->SetAllocated(StackDepotPut(*stack), orig_size);
   RunMallocHooks(user_ptr, size);
   return user_ptr;
 }
 
 static bool PointerAndMemoryTagsMatch(void *tagged_ptr) {
   CHECK(tagged_ptr);
   uptr tagged_uptr = reinterpret_cast<uptr>(tagged_ptr);
HwasanChunkView FindHeapChunkByAddress(uptr address) {
   void *block = allocator.GetBlockBegin(reinterpret_cast<void*>(address));
   if (!block)
     return HwasanChunkView();
   Metadata *metadata =
       reinterpret_cast<Metadata*>(allocator.GetMetaData(block));
   return HwasanChunkView(reinterpret_cast<uptr>(block), metadata);
 }
 
+static inline HwasanChunkView FindHeapChunkByAddressFastLocked(uptr address) {
+  void *block =
+      allocator.GetBlockBeginFastLocked(reinterpret_cast<void *>(address));
+  if (!block)
+    return HwasanChunkView();
+  Metadata *metadata =
+      reinterpret_cast<Metadata *>(allocator.GetMetaData(block));
+  return HwasanChunkView(reinterpret_cast<uptr>(block), metadata);
+}
+
 static uptr AllocationSize(const void *tagged_ptr) {
   const void *untagged_ptr = UntagPtr(tagged_ptr);
   if (!untagged_ptr) return 0;
   const void *beg = allocator.GetBlockBegin(untagged_ptr);
   Metadata *b = (Metadata *)allocator.GetMetaData(untagged_ptr);
   if (beg != untagged_ptr) return 0;
   return b->GetRequestedSize();
 }

 
 void GetAllocatorGlobalRange(uptr *begin, uptr *end) {
   *begin = (uptr)&__hwasan::allocator;
   *end = *begin + sizeof(__hwasan::allocator);
 }
 
 uptr PointsIntoChunk(void *p) {
   uptr addr = reinterpret_cast<uptr>(p);
-  __hwasan::HwasanChunkView view = __hwasan::FindHeapChunkByAddress(addr);
-  if (!view.IsAllocated()) 
+  __hwasan::HwasanChunkView view =
+      __hwasan::FindHeapChunkByAddressFastLocked(addr);
+  if (!view.IsAllocated())
     return 0;
   uptr chunk = view.Beg();
   if (view.AddrIsInside(addr))
     return chunk;
   if (IsSpecialCaseOfOperatorNew0(chunk, view.UsedSize(), addr))
     return chunk;
   return 0;
 }
 
 uptr GetUserBegin(uptr chunk) {
-  return __hwasan::FindHeapChunkByAddress(chunk).Beg();
+  return __hwasan::FindHeapChunkByAddressFastLocked(chunk).Beg();
 }
 
 LsanMetadata::LsanMetadata(uptr chunk) {
-  metadata_ = chunk ? reinterpret_cast<__hwasan::Metadata *>(
-                          chunk - __hwasan::kChunkHeaderSize)
-                    : nullptr;
+  metadata_ =
+      chunk ? __hwasan::allocator.GetMetaData(reinterpret_cast<void *>(chunk))
 }
 
 bool LsanMetadata::allocated() const {
   if (!metadata_)
     return false;
   __hwasan::Metadata *m = reinterpret_cast<__hwasan::Metadata *>(metadata_);
   return m->IsAllocated();
 }
u32 LsanMetadata::stack_trace_id() const {
   __hwasan::Metadata *m = reinterpret_cast<__hwasan::Metadata *>(metadata_);
   return m->GetAllocStackId();
 }
 
 void ForEachChunk(ForEachChunkCallback callback, void *arg) {
   __hwasan::allocator.ForEachChunk(callback, arg);
 }
 
+IgnoreObjectResult IgnoreObjectLocked(const void *p) {
+  uptr addr = reinterpret_cast<uptr>(p);
+  __hwasan::HwasanChunkView view = __hwasan::FindHeapChunkByAddressFastLocked(addr);
+  if (!view.IsAllocated() || !view.AddrIsInside(addr)) {
+    return kIgnoreObjectInvalid;
+  }
+  if (view.GetLsanTag() == kIgnored)
+    return kIgnoreObjectAlreadyIgnored;
+  view.SetLsanTag(kIgnored);
+  return kIgnoreObjectSuccess;
+}
+
 }  // namespace __lsan
 
 using namespace __hwasan;
 
 void __hwasan_enable_allocator_tagging() {
   atomic_store_relaxed(&hwasan_allocator_tagging_enabled, 1);
 }