scudo: Add support for tracking stack traces of secondary allocations.

[pcc]

Summary

There is no centralized store of information related to secondary
allocations. Moreover the allocations themselves become inaccessible
when the allocation is freed in order to implement UAF detection,
so we can't store information there to be used in case of UAF
anyway.

Therefore our storage location for tracking stack traces of secondary
allocations is a secondary ring buffer. The ring buffer is copied to
the process creating the crash dump when a fault occurs.

In order to support the scenario where an access to the ring buffer
is interrupted by a concurrently occurring crash, the secondary
ring buffer is accessed in a lock-free manner. This requires the
use of 128-bit atomics, which are generally only supported on 64-bit
platforms. Therefore the secondary ring buffer is disabled on 32-bit
platforms. The ring buffer would be unused on such platforms anyway
due to the lack of support for memory tagging on platforms other
than arm64.

This change enables 16-byte compare-exchange instructions on x86 in
order to avoid a warning where 128-bit atomics are used. Since the code
wouldn't actually be called in production on x86 as mentioned, this is
mostly in order to provide compile-time coverage and support fuzzing.

Depends on D93731

[eugenis]

Can we have a setting where the secondary ring buffer is used for primary allocations, too? The primary record could be pushed when the chunk is about to be reused.

compiler-rt/lib/scudo/standalone/combined.h

1339

This loop seems expensive. Why not store alloc and dealloc as two independent events? It will use a bit more memory, but given the same memory budget, dealloc events should survive longer because they will be generally closer to the end of the ring (and they are the most interesting ones).

[pcc]

In D94212#2488097, @eugenis wrote:

Can we have a setting where the secondary ring buffer is used for primary allocations, too? The primary record could be pushed when the chunk is about to be reused.

Let's do that in a separate change.

compiler-rt/lib/scudo/standalone/combined.h

1339

Hmm, I wouldn't expect it to be that expensive compared to the cost of the allocation itself.

I thought about storing these as two separate events but I wasn't sure that just a deallocation stack trace on its own would be useful. But having given it more thought I suppose that:

1. We don't need to concern ourselves too much with memory usage here because secondary allocations are rare enough that we can choose a relatively small buffer size, so doubling it shouldn't matter that much
2. Just the deallocation stack trace would be better than nothing. That being said, we could store the allocation trace and tid inline with the allocation and copy them into the ring buffer entry on deallocation. That would cost more memory, but as mentioned in 1) that should be inconsequential.

So let's go with the separate events.

[pcc]

• Instead of updating an existing ring buffer entry, create a new entry

[pcc]

• Use ring buffer for primary deallocations as well

[pcc]

• Update comment

[eugenis]

compiler-rt/lib/scudo/standalone/combined.h

1115

Does this actually do anything if all stores are relaxed?

1138

Why create this ring buffer entry while the allocation is still alive?
This would add some complexity to error reporting, but we could iterate over the live secondary allocations to find the right one. That should reduce ring buffer traffic by 2x (not counting primary of course).

[pcc]

compiler-rt/lib/scudo/standalone/combined.h

1115

The goal is to guard against the thread executing this access being interrupted by a crash, not against concurrent access. By the time the ring buffer entry is accessed the thread will have been stopped so the effects of any stores will have been committed to memory.

That being said, I suspect that the compiler would be allowed to eliminate this store, so we may need something stronger here.

1138

It would add quite a bit of complexity. malloc_iterate is implemented via pointer chasing, whereas __scudo_get_error_info expects a few fixed blocks of data to be copied. Implementing the pointer chasing on the __scudo_get_error_info side would probably require callbacks to read memory, or something like that. It doesn't seem worth it to reduce ring buffer traffic for secondary allocations, which are quite uncommon anyway.

[pcc]

• Use acquire/release for accesses to Ptr

[eugenis]

LGTM with a nit

compiler-rt/lib/scudo/standalone/combined.h

1138

Would __atomic_signal_fence(__ATOMIC_SEQ_CST) be more appropriate?

[pcc]

compiler-rt/lib/scudo/standalone/combined.h

1138

(assuming this was meant as a reply to the other comment)

Yes, that's a good idea, I'll do that.

compiler-rt/lib/scudo/standalone/combined.h

if (LIKELY(ClassId)) {
             // the chunk data.
             memset(TaggedPtr, 0, archMemoryTagGranuleSize());
           }
         } else {
           const uptr OddEvenMask =
               computeOddEvenMaskForPointerMaybe(Options, BlockUptr, BlockSize);
           TaggedPtr = prepareTaggedChunk(Ptr, Size, OddEvenMask, BlockEnd);
         }
-        storeAllocationStackMaybe(Options, Ptr);
+        storePrimaryAllocationStackMaybe(Options, Ptr);
       } else {
         Block = addHeaderTag(Block);
         Ptr = addHeaderTag(Ptr);
         if (UNLIKELY(FillContents != NoFill)) {
           // This condition is not necessarily unlikely, but since memset is
           // costly, we might as well mark it as such.
           memset(Block, FillContents == ZeroFill ? 0 : PatternFillByte,
                  PrimaryT::getSizeByClassId(ClassId));
         }
       }
     } else {
       Block = addHeaderTag(Block);
       Ptr = addHeaderTag(Ptr);
-      if (UNLIKELY(useMemoryTagging<Params>(Options)))
+      if (UNLIKELY(useMemoryTagging<Params>(Options))) {
         storeTags(reinterpret_cast<uptr>(Block), reinterpret_cast<uptr>(Ptr));
+        storeSecondaryAllocationStackMaybe(Options, Ptr, Size);
+      }
     }
 
     Chunk::UnpackedHeader Header = {};
     if (UNLIKELY(UnalignedUserPtr != UserPtr)) {
       const uptr Offset = UserPtr - UnalignedUserPtr;
       DCHECK_GE(Offset, 2 * sizeof(u32));
       // The BlockMarker has no security purpose, but is specifically meant for
       // the chunk iteration function that can be used in debugging situations.
if (reinterpret_cast<uptr>(OldTaggedPtr) + NewSize <= BlockEnd) {
       if (NewSize > OldSize || (OldSize - NewSize) < getPageSizeCached()) {
         Chunk::UnpackedHeader NewHeader = OldHeader;
         NewHeader.SizeOrUnusedBytes =
             (ClassId ? NewSize
                      : BlockEnd -
                            (reinterpret_cast<uptr>(OldTaggedPtr) + NewSize)) &
             Chunk::SizeOrUnusedBytesMask;
         Chunk::compareExchangeHeader(Cookie, OldPtr, &NewHeader, &OldHeader);
-        if (UNLIKELY(ClassId && useMemoryTagging<Params>(Options))) {
+        if (UNLIKELY(useMemoryTagging<Params>(Options))) {
+          if (ClassId) {
             resizeTaggedChunk(reinterpret_cast<uptr>(OldTaggedPtr) + OldSize,
                               reinterpret_cast<uptr>(OldTaggedPtr) + NewSize,
                               BlockEnd);
-          storeAllocationStackMaybe(Options, OldPtr);
+            storePrimaryAllocationStackMaybe(Options, OldPtr);
+          } else {
+            storeSecondaryAllocationStackMaybe(Options, OldPtr, NewSize);
+          }
         }
         return OldTaggedPtr;
       }
     }
 
     // Otherwise we allocate a new one, and deallocate the old one. Some
     // allocators will allocate an even larger chunk (by a fixed factor) to
     // allow for potential further in-place realloc. The gains of such a trick
#endif // GWP_ASAN_HOOKS
   const char *getRegionInfoArrayAddress() const {
     return Primary.getRegionInfoArrayAddress();
   }
 
   static uptr getRegionInfoArraySize() {
     return PrimaryT::getRegionInfoArraySize();
   }
 
-  static void getErrorInfo(struct scudo_error_info *ErrorInfo,
-                           uintptr_t FaultAddr, const char *DepotPtr,
-                           const char *RegionInfoPtr, const char *Memory,
-                           const char *MemoryTags, uintptr_t MemoryAddr,
-                           size_t MemorySize) {
-    *ErrorInfo = {};
-    if (!allocatorSupportsMemoryTagging<Params>() ||
-        MemoryAddr + MemorySize < MemoryAddr)
-      return;
-
-    uptr UntaggedFaultAddr = untagPointer(FaultAddr);
-    u8 FaultAddrTag = extractTag(FaultAddr);
-    BlockInfo Info =
-        PrimaryT::findNearestBlock(RegionInfoPtr, UntaggedFaultAddr);
-
-    auto GetGranule = [&](uptr Addr, const char **Data, uint8_t *Tag) -> bool {
-      if (Addr < MemoryAddr || Addr + archMemoryTagGranuleSize() < Addr ||
-          Addr + archMemoryTagGranuleSize() > MemoryAddr + MemorySize)
-        return false;
-      *Data = &Memory[Addr - MemoryAddr];
-      *Tag = static_cast<u8>(
-          MemoryTags[(Addr - MemoryAddr) / archMemoryTagGranuleSize()]);
-      return true;
-    };
-
-    auto ReadBlock = [&](uptr Addr, uptr *ChunkAddr,
-                         Chunk::UnpackedHeader *Header, const u32 **Data,
-                         u8 *Tag) {
-      const char *BlockBegin;
-      u8 BlockBeginTag;
-      if (!GetGranule(Addr, &BlockBegin, &BlockBeginTag))
-        return false;
-      uptr ChunkOffset = getChunkOffsetFromBlock(BlockBegin);
-      *ChunkAddr = Addr + ChunkOffset;
+  const char *getRingBufferAddress() const {
+    return reinterpret_cast<const char *>(&RingBuffer);
+  }
 
-      const char *ChunkBegin;
-      if (!GetGranule(*ChunkAddr, &ChunkBegin, Tag))
-        return false;
-      *Header = *reinterpret_cast<const Chunk::UnpackedHeader *>(
-          ChunkBegin - Chunk::getHeaderSize());
-      *Data = reinterpret_cast<const u32 *>(ChunkBegin);
-      return true;
-    };
+  static uptr getRingBufferSize() { return sizeof(RingBuffer); }
 
-    auto *Depot = reinterpret_cast<const StackDepot *>(DepotPtr);
+  static const uptr MaxTraceSize = 64;
 
-    auto MaybeCollectTrace = [&](uintptr_t(&Trace)[MaxTraceSize], u32 Hash) {
-      uptr RingPos, Size;
-      if (!Depot->find(Hash, &RingPos, &Size))
-        return;
-      for (unsigned I = 0; I != Size && I != MaxTraceSize; ++I)
-        Trace[I] = (*Depot)[RingPos + I];
-    };
-
-    size_t NextErrorReport = 0;
-
-    // First, check for UAF.
-    {
-      uptr ChunkAddr;
-      Chunk::UnpackedHeader Header;
-      const u32 *Data;
-      uint8_t Tag;
-      if (ReadBlock(Info.BlockBegin, &ChunkAddr, &Header, &Data, &Tag) &&
-          Header.State != Chunk::State::Allocated &&
-          Data[MemTagPrevTagIndex] == FaultAddrTag) {
-        auto *R = &ErrorInfo->reports[NextErrorReport++];
-        R->error_type = USE_AFTER_FREE;
-        R->allocation_address = ChunkAddr;
-        R->allocation_size = Header.SizeOrUnusedBytes;
-        MaybeCollectTrace(R->allocation_trace,
-                          Data[MemTagAllocationTraceIndex]);
-        R->allocation_tid = Data[MemTagAllocationTidIndex];
-        MaybeCollectTrace(R->deallocation_trace,
-                          Data[MemTagDeallocationTraceIndex]);
-        R->deallocation_tid = Data[MemTagDeallocationTidIndex];
-      }
-    }
-
-    auto CheckOOB = [&](uptr BlockAddr) {
-      if (BlockAddr < Info.RegionBegin || BlockAddr >= Info.RegionEnd)
-        return false;
-
-      uptr ChunkAddr;
-      Chunk::UnpackedHeader Header;
-      const u32 *Data;
-      uint8_t Tag;
-      if (!ReadBlock(BlockAddr, &ChunkAddr, &Header, &Data, &Tag) ||
-          Header.State != Chunk::State::Allocated || Tag != FaultAddrTag)
-        return false;
+  static void collectTraceMaybe(const StackDepot *Depot,
+                                uintptr_t (&Trace)[MaxTraceSize], u32 Hash) {
 
-      auto *R = &ErrorInfo->reports[NextErrorReport++];
-      R->error_type =
-          UntaggedFaultAddr < ChunkAddr ? BUFFER_UNDERFLOW : BUFFER_OVERFLOW;
-      R->allocation_address = ChunkAddr;
-      R->allocation_size = Header.SizeOrUnusedBytes;
-      MaybeCollectTrace(R->allocation_trace, Data[MemTagAllocationTraceIndex]);
-      R->allocation_tid = Data[MemTagAllocationTidIndex];
-      return NextErrorReport ==
-             sizeof(ErrorInfo->reports) / sizeof(ErrorInfo->reports[0]);
-    };
-
-    if (CheckOOB(Info.BlockBegin))
+  static void getErrorInfo(struct scudo_error_info *ErrorInfo,
+                           uintptr_t FaultAddr, const char *DepotPtr,
+                           const char *RegionInfoPtr, const char *RingBufferPtr,
+                           const char *Memory, const char *MemoryTags,
+                           uintptr_t MemoryAddr, size_t MemorySize) {
+    *ErrorInfo = {};
+    if (!allocatorSupportsMemoryTagging<Params>() ||
+        MemoryAddr + MemorySize < MemoryAddr)
       return;
 
-    // Check for OOB in the 30 surrounding blocks. Beyond that we are likely to
-    // hit false positives.
-    for (int I = 1; I != 16; ++I)
-      if (CheckOOB(Info.BlockBegin + I * Info.BlockSize) ||
-          CheckOOB(Info.BlockBegin - I * Info.BlockSize))
-        return;
+    auto *Depot = reinterpret_cast<const StackDepot *>(DepotPtr);
+    size_t NextErrorReport = 0;
+    if (extractTag(FaultAddr) != 0)
+      getInlineErrorInfo(ErrorInfo, NextErrorReport, FaultAddr, Depot,
+                         RegionInfoPtr, Memory, MemoryTags, MemoryAddr,
+                         MemorySize);
+    getRingBufferErrorInfo(ErrorInfo, NextErrorReport, FaultAddr, Depot,
+                           RingBufferPtr);
   }
 
 private:
   using SecondaryT = MapAllocator<Params>;
   typedef typename PrimaryT::SizeClassMap SizeClassMap;
 
   static const uptr MinAlignmentLog = SCUDO_MIN_ALIGNMENT_LOG;
   static const uptr MaxAlignmentLog = 24U; // 16 MB seems reasonable.
   static const uptr MinAlignment = 1UL << MinAlignmentLog;
   static const uptr MaxAlignment = 1UL << MaxAlignmentLog;
   static const uptr MaxAllowedMallocSize =
       FIRST_32_SECOND_64(1UL << 31, 1ULL << 40);
 
   static_assert(MinAlignment >= sizeof(Chunk::PackedHeader),
                 "Minimal alignment must at least cover a chunk header.");
   static_assert(!allocatorSupportsMemoryTagging<Params>() ||
                     MinAlignment >= archMemoryTagGranuleSize(),
                 "");
 
   static const u32 BlockMarker = 0x44554353U;
 
   // These are indexes into an "array" of 32-bit values that store information
   // inline with a chunk that is relevant to diagnosing memory tag faults, where
-  // 0 corresponds to the address of the user memory. This means that negative
-  // indexes may be used to store information about allocations, while positive
-  // indexes may only be used to store information about deallocations, because
-  // the user memory is in use until it has been deallocated. The smallest index
-  // that may be used is -2, which corresponds to 8 bytes before the user
-  // memory, because the chunk header size is 8 bytes and in allocators that
-  // support memory tagging the minimum alignment is at least the tag granule
-  // size (16 on aarch64), and the largest index that may be used is 3 because
-  // we are only guaranteed to have at least a granule's worth of space in the
-  // user memory.
+  // 0 corresponds to the address of the user memory. This means that only
+  // negative indexes may be used. The smallest index that may be used is -2,
+  // which corresponds to 8 bytes before the user memory, because the chunk
+  // header size is 8 bytes and in allocators that support memory tagging the
+  // minimum alignment is at least the tag granule size (16 on aarch64).
   static const sptr MemTagAllocationTraceIndex = -2;
   static const sptr MemTagAllocationTidIndex = -1;
-  static const sptr MemTagDeallocationTraceIndex = 0;
-  static const sptr MemTagDeallocationTidIndex = 1;
-  static const sptr MemTagPrevTagIndex = 2;
-
-  static const uptr MaxTraceSize = 64;
 
   u32 Cookie;
   u32 QuarantineMaxChunkSize;
 
   GlobalStats Stats;
   PrimaryT Primary;
   SecondaryT Secondary;
   QuarantineT Quarantine;
   TSDRegistryT TSDRegistry;
 
 #ifdef GWP_ASAN_HOOKS
   gwp_asan::GuardedPoolAllocator GuardedAlloc;
 #endif // GWP_ASAN_HOOKS
 
   StackDepot Depot;
 
+  struct AllocationRingBuffer {
+    struct Entry {
+      atomic_uptr Ptr;
+      atomic_uptr AllocationSize;
+      atomic_u32 AllocationTrace;
+      atomic_u32 AllocationTid;
+      atomic_u32 DeallocationTrace;
+      atomic_u32 DeallocationTid;
+    };
+
+    atomic_uptr Pos;
+#ifdef SCUDO_FUZZ
+    static const uptr NumEntries = 2;
+#else
+    static const uptr NumEntries = 32768;
+#endif
+    Entry Entries[NumEntries];
+  };
+  AllocationRingBuffer RingBuffer;
+
   // The following might get optimized out by the compiler.
   NOINLINE void performSanityChecks() {
     // Verify that the header offset field can hold the maximum offset. In the
     // case of the Secondary allocator, it takes care of alignment and the
     // offset will always be small. In the case of the Primary, the worst case
     // scenario happens in the last size class, when the backend allocation
     // would already be aligned on the requested alignment, which would happen
     // to be the maximum alignment that would fit in that size class. As a
if (allocatorSupportsMemoryTagging<Params>())
       Ptr = untagPointer(const_cast<void *>(Ptr));
     return SecondaryT::getBlockEnd(getBlockBegin(Ptr, Header)) -
            reinterpret_cast<uptr>(Ptr) - SizeOrUnusedBytes;
   }
 
   void quarantineOrDeallocateChunk(Options Options, void *Ptr,
                                    Chunk::UnpackedHeader *Header, uptr Size) {
     Chunk::UnpackedHeader NewHeader = *Header;
-    if (UNLIKELY(NewHeader.ClassId && useMemoryTagging<Params>(Options))) {
-      u8 PrevTag = extractTag(loadTag(reinterpret_cast<uptr>(Ptr)));
+    if (UNLIKELY(useMemoryTagging<Params>(Options))) {
+      u8 PrevTag = 0;
+      if (NewHeader.ClassId) {
+        PrevTag = extractTag(loadTag(reinterpret_cast<uptr>(Ptr)));
         if (!TSDRegistry.getDisableMemInit()) {
           uptr TaggedBegin, TaggedEnd;
           const uptr OddEvenMask = computeOddEvenMaskForPointerMaybe(
               Options, reinterpret_cast<uptr>(getBlockBegin(Ptr, &NewHeader)),
               SizeClassMap::getSizeByClassId(NewHeader.ClassId));
-        // Exclude the previous tag so that immediate use after free is detected
-        // 100% of the time.
+          // Exclude the previous tag so that immediate use after free is
+          // detected 100% of the time.
           setRandomTag(Ptr, Size, OddEvenMask | (1UL << PrevTag), &TaggedBegin,
                        &TaggedEnd);
         }
         NewHeader.OriginOrWasZeroed = !TSDRegistry.getDisableMemInit();
-      storeDeallocationStackMaybe(Options, Ptr, PrevTag);
+      }
+      storeDeallocationStackMaybe(Options, Ptr, PrevTag, Size);
     }
     // If the quarantine is disabled, the actual size of a chunk is 0 or larger
     // than the maximum allowed, we return a chunk directly to the backend.
     // This purposefully underflows for Size == 0.
     const bool BypassQuarantine = !Quarantine.getCacheSize() ||
                                   ((Size - 1) >= QuarantineMaxChunkSize) ||
                                   !NewHeader.ClassId;
     if (BypassQuarantine) {
#endif
 
   static uptr getChunkOffsetFromBlock(const char *Block) {
     u32 Offset = 0;
     if (reinterpret_cast<const u32 *>(Block)[0] == BlockMarker)
       Offset = reinterpret_cast<const u32 *>(Block)[1];
     return Offset + Chunk::getHeaderSize();
   }
 
-  void storeAllocationStackMaybe(Options Options, void *Ptr) {
+  void storePrimaryAllocationStackMaybe(Options Options, void *Ptr) {
     if (!UNLIKELY(Options.get(OptionBit::TrackAllocationStacks)))
       return;
     auto *Ptr32 = reinterpret_cast<u32 *>(Ptr);
     Ptr32[MemTagAllocationTraceIndex] = collectStackTrace();
     Ptr32[MemTagAllocationTidIndex] = getThreadID();
   }
 
-  void storeDeallocationStackMaybe(Options Options, void *Ptr,
-                                   uint8_t PrevTag) {
+  void storeRingBufferEntry(void *Ptr, u32 AllocationTrace, u32 AllocationTid,
+                            uptr AllocationSize, u32 DeallocationTrace,
+                            u32 DeallocationTid) {
+    uptr Pos = atomic_fetch_add(&RingBuffer.Pos, 1, memory_order_relaxed);
+    typename AllocationRingBuffer::Entry *Entry =
+        &RingBuffer.Entries[Pos % AllocationRingBuffer::NumEntries];
+
+    // First invalidate our entry so that we don't attempt to interpret a
+    // partially written state in getSecondaryErrorInfo().
+    atomic_store(&Entry->Ptr, 0, memory_order_acquire);
+    atomic_store_relaxed(&Entry->AllocationTrace, AllocationTrace);
+    atomic_store_relaxed(&Entry->AllocationTid, AllocationTid);
+    atomic_store_relaxed(&Entry->AllocationSize, AllocationSize);
+    atomic_store_relaxed(&Entry->DeallocationTrace, DeallocationTrace);
+    atomic_store_relaxed(&Entry->DeallocationTid, DeallocationTid);
+
+    atomic_store(&Entry->Ptr, reinterpret_cast<uptr>(Ptr),
+                 memory_order_release);
+  }
+
+  void storeSecondaryAllocationStackMaybe(Options Options, void *Ptr,
+                                          uptr Size) {
+    if (!UNLIKELY(Options.get(OptionBit::TrackAllocationStacks)))
+      return;
+
+    u32 Trace = collectStackTrace();
+    u32 Tid = getThreadID();
+
+    auto *Ptr32 = reinterpret_cast<u32 *>(Ptr);
+    Ptr32[MemTagAllocationTraceIndex] = Trace;
+    Ptr32[MemTagAllocationTidIndex] = Tid;
+
+    storeRingBufferEntry(untagPointer(Ptr), Trace, Tid, Size, 0, 0);
+  }
+
+  void storeDeallocationStackMaybe(Options Options, void *Ptr, u8 PrevTag,
+                                   uptr Size) {
     if (!UNLIKELY(Options.get(OptionBit::TrackAllocationStacks)))
       return;
 
-    // Disable tag checks here so that we don't need to worry about zero sized
-    // allocations.
-    ScopedDisableMemoryTagChecks x;
     auto *Ptr32 = reinterpret_cast<u32 *>(Ptr);
-    Ptr32[MemTagDeallocationTraceIndex] = collectStackTrace();
-    Ptr32[MemTagDeallocationTidIndex] = getThreadID();
-    Ptr32[MemTagPrevTagIndex] = PrevTag;
+    u32 AllocationTrace = Ptr32[MemTagAllocationTraceIndex];
+    u32 AllocationTid = Ptr32[MemTagAllocationTidIndex];
+
+    u32 DeallocationTrace = collectStackTrace();
+    u32 DeallocationTid = getThreadID();
+
+    storeRingBufferEntry(addFixedTag(untagPointer(Ptr), PrevTag),
+                         AllocationTrace, AllocationTid, Size,
+                         DeallocationTrace, DeallocationTid);
+  }
+
+  static const size_t NumErrorReports =
+      sizeof(((scudo_error_info *)0)->reports) /
+      sizeof(((scudo_error_info *)0)->reports[0]);
+
+  static void getInlineErrorInfo(struct scudo_error_info *ErrorInfo,
+                                 size_t &NextErrorReport, uintptr_t FaultAddr,
+                                 const StackDepot *Depot,
+                                 const char *RegionInfoPtr, const char *Memory,
+                                 const char *MemoryTags, uintptr_t MemoryAddr,
+                                 size_t MemorySize) {
+    uptr UntaggedFaultAddr = untagPointer(FaultAddr);
+    u8 FaultAddrTag = extractTag(FaultAddr);
+    BlockInfo Info =
+        PrimaryT::findNearestBlock(RegionInfoPtr, UntaggedFaultAddr);
+
+    auto GetGranule = [&](uptr Addr, const char **Data, uint8_t *Tag) -> bool {
+      if (Addr < MemoryAddr || Addr + archMemoryTagGranuleSize() < Addr ||
+          Addr + archMemoryTagGranuleSize() > MemoryAddr + MemorySize)
+        return false;
+      *Data = &Memory[Addr - MemoryAddr];
+      *Tag = static_cast<u8>(
+          MemoryTags[(Addr - MemoryAddr) / archMemoryTagGranuleSize()]);
+      return true;
+    };
+
+    auto ReadBlock = [&](uptr Addr, uptr *ChunkAddr,
+                         Chunk::UnpackedHeader *Header, const u32 **Data,
+                         u8 *Tag) {
+      const char *BlockBegin;
+      u8 BlockBeginTag;
+      if (!GetGranule(Addr, &BlockBegin, &BlockBeginTag))
+        return false;
+      uptr ChunkOffset = getChunkOffsetFromBlock(BlockBegin);
+      *ChunkAddr = Addr + ChunkOffset;
+
+      const char *ChunkBegin;
+      if (!GetGranule(*ChunkAddr, &ChunkBegin, Tag))
+        return false;
+      *Header = *reinterpret_cast<const Chunk::UnpackedHeader *>(
+          ChunkBegin - Chunk::getHeaderSize());
+      *Data = reinterpret_cast<const u32 *>(ChunkBegin);
+      return true;
+    };
+
+    if (NextErrorReport == NumErrorReports)
+      return;
+
+    auto CheckOOB = [&](uptr BlockAddr) {
+      if (BlockAddr < Info.RegionBegin || BlockAddr >= Info.RegionEnd)
+        return false;
+
+      uptr ChunkAddr;
+      Chunk::UnpackedHeader Header;
+      const u32 *Data;
+      uint8_t Tag;
+      if (!ReadBlock(BlockAddr, &ChunkAddr, &Header, &Data, &Tag) ||
+          Header.State != Chunk::State::Allocated || Tag != FaultAddrTag)
+        return false;
+
+      auto *R = &ErrorInfo->reports[NextErrorReport++];
+      R->error_type =
+          UntaggedFaultAddr < ChunkAddr ? BUFFER_UNDERFLOW : BUFFER_OVERFLOW;
+      R->allocation_address = ChunkAddr;
+      R->allocation_size = Header.SizeOrUnusedBytes;
+      collectTraceMaybe(Depot, R->allocation_trace,
+                        Data[MemTagAllocationTraceIndex]);
+      R->allocation_tid = Data[MemTagAllocationTidIndex];
+      return NextErrorReport == NumErrorReports;
+    };
+
+    if (CheckOOB(Info.BlockBegin))
+      return;
+
+    // Check for OOB in the 30 surrounding blocks. Beyond that we are likely to
+    // hit false positives.
+    for (int I = 1; I != 16; ++I)
+      if (CheckOOB(Info.BlockBegin + I * Info.BlockSize) ||
+          CheckOOB(Info.BlockBegin - I * Info.BlockSize))
+        return;
+  }
+
+  static void getRingBufferErrorInfo(struct scudo_error_info *ErrorInfo,
+                                     size_t &NextErrorReport,
+                                     uintptr_t FaultAddr,
+                                     const StackDepot *Depot,
+                                     const char *RingBufferPtr) {
+    auto *RingBuffer =
+        reinterpret_cast<const AllocationRingBuffer *>(RingBufferPtr);
+    uptr Pos = atomic_load_relaxed(&RingBuffer->Pos);
+
+    for (uptr I = Pos - 1; I != Pos - 1 - AllocationRingBuffer::NumEntries &&
+                           NextErrorReport != NumErrorReports;
+         --I) {
+      auto *Entry = &RingBuffer->Entries[I % AllocationRingBuffer::NumEntries];
+      uptr EntryPtr = atomic_load_relaxed(&Entry->Ptr);
+      uptr UntaggedEntryPtr = untagPointer(EntryPtr);
+      uptr EntrySize = atomic_load_relaxed(&Entry->AllocationSize);
+      if (!EntryPtr || FaultAddr < EntryPtr - getPageSizeCached() ||
+          FaultAddr >= EntryPtr + EntrySize + getPageSizeCached())
+        continue;
+
+      u32 AllocationTrace = atomic_load_relaxed(&Entry->AllocationTrace);
+      u32 AllocationTid = atomic_load_relaxed(&Entry->AllocationTid);
+      u32 DeallocationTrace = atomic_load_relaxed(&Entry->DeallocationTrace);
+      u32 DeallocationTid = atomic_load_relaxed(&Entry->DeallocationTid);
+
+      // For UAF the ring buffer will contain two entries, one for the
+      // allocation and another for the deallocation. Don't report buffer
+      // overflow/underflow using the allocation entry if we have already
+      // collected a report from the deallocation entry.
+      if (!DeallocationTrace) {
+        bool Found = false;
+        for (uptr J = 0; J != NextErrorReport; ++J) {
+          if (ErrorInfo->reports[J].allocation_address == UntaggedEntryPtr) {
+            Found = true;
+            break;
+          }
+        }
+        if (Found)
+          continue;
+      }
+
+      auto *R = &ErrorInfo->reports[NextErrorReport++];
+      if (DeallocationTid)
+        R->error_type = USE_AFTER_FREE;
+      else if (FaultAddr < EntryPtr)
+        R->error_type = BUFFER_UNDERFLOW;
+      else
+        R->error_type = BUFFER_OVERFLOW;
+
+      R->allocation_address = UntaggedEntryPtr;
+      R->allocation_size = EntrySize;
+      collectTraceMaybe(Depot, R->allocation_trace, AllocationTrace);
+      R->allocation_tid = AllocationTid;
+      collectTraceMaybe(Depot, R->deallocation_trace, DeallocationTrace);
+      R->deallocation_tid = DeallocationTid;
+    }
   }
 
   uptr getStats(ScopedString *Str) {
     Primary.getStats(Str);
     Secondary.getStats(Str);
     Quarantine.getStats(Str);
     return Str->length();
   }
 };
 
 } // namespace scudo
 
 #endif // SCUDO_COMBINED_H_

compiler-rt/lib/scudo/standalone/fuzz/get_error_info_fuzzer.cpp

extern "C" int LLVMFuzzerTestOneInput(uint8_t *Data, size_t Size) {
   std::string StackDepotBytes =
       FDP.ConsumeRandomLengthString(FDP.remaining_bytes());
   std::vector<char> StackDepot(sizeof(scudo::StackDepot), 0);
   for (size_t i = 0; i < StackDepotBytes.length() && i < StackDepot.size();
        ++i) {
     StackDepot[i] = StackDepotBytes[i];
   }
 
-  std::string RegionInfoBytes = FDP.ConsumeRemainingBytesAsString();
+  std::string RegionInfoBytes =
+      FDP.ConsumeRandomLengthString(FDP.remaining_bytes());
   std::vector<char> RegionInfo(AllocatorT::getRegionInfoArraySize(), 0);
   for (size_t i = 0; i < RegionInfoBytes.length() && i < RegionInfo.size();
        ++i) {
     RegionInfo[i] = RegionInfoBytes[i];
   }
 
+  std::string RingBufferBytes = FDP.ConsumeRemainingBytesAsString();
+  std::vector<char> RingBuffer(AllocatorT::getRingBufferSize(), 0);
+  for (size_t i = 0; i < RingBufferBytes.length() && i < RingBuffer.size();
+       ++i) {
+    RingBuffer[i] = RingBufferBytes[i];
+  }
+
   scudo_error_info ErrorInfo;
   AllocatorT::getErrorInfo(&ErrorInfo, FaultAddr, StackDepot.data(),
-                           RegionInfo.data(), Memory, MemoryTags, MemoryAddr,
-                           MemorySize);
+                           RegionInfo.data(), RingBuffer.data(), Memory,
+                           MemoryTags, MemoryAddr, MemorySize);
   return 0;
 }

compiler-rt/lib/scudo/standalone/include/scudo/interface.h

 //
 // memory_addr is the start address of memory in the crashing process's address
 // space.
 //
 // memory_size is the size of the memory region referred to by the memory
 // pointer.
 void __scudo_get_error_info(struct scudo_error_info *error_info,
                             uintptr_t fault_addr, const char *stack_depot,
-                            const char *region_info, const char *memory,
-                            const char *memory_tags, uintptr_t memory_addr,
-                            size_t memory_size);
+                            const char *region_info, const char *ring_buffer,
+                            const char *memory, const char *memory_tags,
+                            uintptr_t memory_addr, size_t memory_size);
 
 enum scudo_error_type {
   UNKNOWN,
   USE_AFTER_FREE,
   BUFFER_OVERFLOW,
   BUFFER_UNDERFLOW,
 };
 

 };
 
 const char *__scudo_get_stack_depot_addr();
 size_t __scudo_get_stack_depot_size();
 
 const char *__scudo_get_region_info_addr();
 size_t __scudo_get_region_info_size();
 
+const char *__scudo_get_ring_buffer_addr();
+size_t __scudo_get_ring_buffer_size();
+
 #ifndef M_DECAY_TIME
 #define M_DECAY_TIME -100
 #endif
 
 #ifndef M_PURGE
 #define M_PURGE -101
 #endif
 

compiler-rt/lib/scudo/standalone/memtag.h

inline void setRandomTag(void *Ptr, uptr Size, uptr ExcludeMask,
   *TaggedBegin = selectRandomTag(reinterpret_cast<uptr>(Ptr), ExcludeMask);
   *TaggedEnd = storeTags(*TaggedBegin, *TaggedBegin + Size);
 }
 
 inline void *untagPointer(void *Ptr) {
   return reinterpret_cast<void *>(untagPointer(reinterpret_cast<uptr>(Ptr)));
 }
 
+inline void *loadTag(void *Ptr) {
+  return reinterpret_cast<void *>(loadTag(reinterpret_cast<uptr>(Ptr)));
+}
+
+inline void *addFixedTag(void *Ptr, uptr Tag) {
+  return reinterpret_cast<void *>(
+      addFixedTag(reinterpret_cast<uptr>(Ptr), Tag));
+}
+
 template <typename Config>
 inline constexpr bool allocatorSupportsMemoryTagging() {
   return archSupportsMemoryTagging() && Config::MaySupportMemoryTagging;
 }
 
 } // namespace scudo
 
 #endif

compiler-rt/lib/scudo/standalone/wrappers_c_bionic.cpp

 #include "wrappers_c.inc"
 
 #undef SCUDO_ALLOCATOR
 #undef SCUDO_PREFIX
 
 // TODO(kostyak): support both allocators.
 INTERFACE void __scudo_print_stats(void) { Allocator.printStats(); }
 
-INTERFACE void __scudo_get_error_info(
-    struct scudo_error_info *error_info, uintptr_t fault_addr,
-    const char *stack_depot, const char *region_info, const char *memory,
-    const char *memory_tags, uintptr_t memory_addr, size_t memory_size) {
+INTERFACE void
+__scudo_get_error_info(struct scudo_error_info *error_info,
+                       uintptr_t fault_addr, const char *stack_depot,
+                       const char *region_info, const char *ring_buffer,
+                       const char *memory, const char *memory_tags,
+                       uintptr_t memory_addr, size_t memory_size) {
   Allocator.getErrorInfo(error_info, fault_addr, stack_depot, region_info,
-                         memory, memory_tags, memory_addr, memory_size);
+                         ring_buffer, memory, memory_tags, memory_addr,
+                         memory_size);
 }
 
 INTERFACE const char *__scudo_get_stack_depot_addr() {
   return Allocator.getStackDepotAddress();
 }
 
 INTERFACE size_t __scudo_get_stack_depot_size() {
   return sizeof(scudo::StackDepot);
 }
 
 INTERFACE const char *__scudo_get_region_info_addr() {
   return Allocator.getRegionInfoArrayAddress();
 }
 
 INTERFACE size_t __scudo_get_region_info_size() {
   return Allocator.getRegionInfoArraySize();
 }
 
+INTERFACE const char *__scudo_get_ring_buffer_addr() {
+  return Allocator.getRingBufferAddress();
+}
+
+INTERFACE size_t __scudo_get_ring_buffer_size() {
+  return Allocator.getRingBufferSize();
+}
+
 #endif // SCUDO_ANDROID && _BIONIC