[GWP-ASan] Refactor memory mapping functions

[cryptoad]

Summary

In preparation for Fuchsia support, this CL refactors the memory
mapping functions.

The new functions are as follows:

• for Freeslots and Metadata: void *map(size_t Size, const char *Name) const; void unmap(void *Ptr, size_t Size) const;
• for the Pool: void *reservePool(size_t Size); void commitPool(void *Ptr, size_t Size) const; void decommitPool(void *Ptr, size_t Size) const; void unreservePool(); Note that those don't need a Name parameter as those are fixed per function. {reserve,unreserve}Pool are not const because they will modify platform specific class member on Fuchsia.

I added a plethora of assert() as the initial code was not enforcing
page alignment for sizes and addresses, which caused problem in the
initial Fuchsia draft. All sizes should now be properly rounded up to
a page.

[cryptoad]

Correcting a potential conversion warning.

[hctim]

compiler-rt/lib/gwp_asan/guarded_pool_allocator.h

128

nits just to be really clear/descriptive with the function names.

mapWithRW()

reserveGuardedPool()
commitGuardedPoolRegionWithRW()
decommitGuardedPoolRegion()
unreserveGuardedPool()

Also can you add a comment above why the pool is special? Something like:

The pool is managed separately, as some platforms (particularly Fuchsia) manage virtual memory regions as a chunk where individual pages can still have separate permissions. These platforms maintain metadata about the region in order to perform operations. The pool is unique as it's the only thing in GWP-ASan that treats pages in a single VM region on an individual basis for page protection.

[mcgrathr]

LGTM. Different names and more comments would be preferable.

compiler-rt/lib/gwp_asan/guarded_pool_allocator.h

128

I'm not in favor of the "decommit" name because the semantics on Fuchsia are to unmap the region so that later accesses will fault. "decommit" usually means "restore to lazy zero-fill state", not "make potentially inaccessible".

I'm certainly in favor of any clarifying comments that anyone thinks help.

135

I'd like to see some explicit comments here about the API contract: the returned ptr is the reserved address range of (at least) Size bytes; a Ptr/Size in commitPool must be a subrange of that range; a Ptr/Size in decommitPool must be an exact pair previously passed to commitPool.

136

How about allocateInPool and deallocateInPool?

compiler-rt/lib/gwp_asan/platform_specific/guarded_pool_allocator_posix.cpp

29

Google style is to prefer anonymous namespace, I don't know about LLVM style.

49

could use the name in the error

[cryptoad]

Addressing review comments:

• tried to strike a middle ground between name suggestions, settled on: `void *reserveGuardedPool(size_t Size);

void allocateInGuardedPool(void *Ptr, size_t Size) const;
void deallocateInGuardedPool(void *Ptr, size_t Size) const;
void unreserveGuardedPool();`

• added comments and clarifications about API contracts
• used anonymous namespace instead of static

[cryptoad]

clang-format a file that was missed.

[hctim]

LGTM

[mcgrathr]

Thanks!

compiler-rt/lib/gwp_asan/guarded_pool_allocator.h

130–131

Fuchsia supports it too and doesn't have this constraint.

compiler-rt/lib/gwp_asan/guarded_pool_allocator.h

private:
   static constexpr const char *kGwpAsanFreeSlotsName = "GWP-ASan Metadata";
 
   static constexpr size_t kInvalidSlotID = SIZE_MAX;
 
   // These functions anonymously map memory or change the permissions of mapped
   // memory into this process in a platform-specific way. Pointer and size
   // arguments are expected to be page-aligned. These functions will never
   // return on error, instead electing to kill the calling process on failure.
-  // Note that memory is initially mapped inaccessible. In order for RW
-  // mappings, call mapMemory() followed by markReadWrite() on the returned
-  // pointer. Each mapping is named on platforms that support it, primarily
-  // Android. This name must be a statically allocated string, as the Android
-  // kernel uses the string pointer directly.
-  void *mapMemory(size_t Size, const char *Name) const;
-  void unmapMemory(void *Ptr, size_t Size, const char *Name) const;
-  void markReadWrite(void *Ptr, size_t Size, const char *Name) const;
-  void markInaccessible(void *Ptr, size_t Size, const char *Name) const;
+  // The pool memory is initially reserved and inaccessible, and RW mappings are
+  // subsequently created and destroyed via allocateInGuardedPool() and
+  // deallocateInGuardedPool(). Each mapping is named on platforms that support
+  // it, primarily Android. This name must be a statically allocated string, as
+  // the Android kernel uses the string pointer directly.
+  void *map(size_t Size, const char *Name) const;
+  void unmap(void *Ptr, size_t Size) const;
+
+  // The pool is managed separately, as some platforms (particularly Fuchsia)
+  // manage virtual memory regions as a chunk where individual pages can still
+  // have separate permissions. These platforms maintain metadata about the
+  // region in order to perform operations. The pool is unique as it's the only
+  // thing in GWP-ASan that treats pages in a single VM region on an individual
+  // basis for page protection.
+  // The pointer returned by reserveGuardedPool() is the reserved address range
+  // of (at least) Size bytes.
+  void *reserveGuardedPool(size_t Size);
+  // allocateInGuardedPool() Ptr and Size must be a subrange of the previously
+  // reserved pool range.
+  void allocateInGuardedPool(void *Ptr, size_t Size) const;
+  // deallocateInGuardedPool() Ptr and Size must be an exact pair previously
+  // passed to allocateInGuardedPool().
+  void deallocateInGuardedPool(void *Ptr, size_t Size) const;
+  void unreserveGuardedPool();
 
   // Get the page size from the platform-specific implementation. Only needs to
   // be called once, and the result should be cached in PageSize in this class.
   static size_t getPlatformPageSize();
 
   // Returns a pointer to the metadata for the owned pointer. If the pointer is
   // not owned by this pool, the result is undefined.
   AllocationMetadata *addrToMetadata(uintptr_t Ptr) const;

compiler-rt/lib/gwp_asan/guarded_pool_allocator.cpp

 } // anonymous namespace
 
 // Gets the singleton implementation of this class. Thread-compatible until
 // init() is called, thread-safe afterwards.
 GuardedPoolAllocator *GuardedPoolAllocator::getSingleton() {
   return SingletonPtr;
 }
 
+static size_t roundUpTo(size_t Size, size_t Boundary) {
+  return (Size + Boundary - 1) & ~(Boundary - 1);
+}
+
 void GuardedPoolAllocator::init(const options::Options &Opts) {
   // Note: We return from the constructor here if GWP-ASan is not available.
   // This will stop heap-allocation of class members, as well as mmap() of the
   // guarded slots.
   if (!Opts.Enabled || Opts.SampleRate == 0 ||
       Opts.MaxSimultaneousAllocations == 0)
     return;
 
   Check(Opts.SampleRate >= 0, "GWP-ASan Error: SampleRate is < 0.");
   Check(Opts.SampleRate < (1 << 30), "GWP-ASan Error: SampleRate is >= 2^30.");
   Check(Opts.MaxSimultaneousAllocations >= 0,
         "GWP-ASan Error: MaxSimultaneousAllocations is < 0.");
 
   SingletonPtr = this;
   Backtrace = Opts.Backtrace;
 
   State.MaxSimultaneousAllocations = Opts.MaxSimultaneousAllocations;
 
-  State.PageSize = getPlatformPageSize();
+  const size_t PageSize = getPlatformPageSize();
+  // getPageAddr() and roundUpTo() assume the page size to be a power of 2.
+  assert((PageSize & (PageSize - 1)) == 0);
+  State.PageSize = PageSize;
 
   PerfectlyRightAlign = Opts.PerfectlyRightAlign;
 
   size_t PoolBytesRequired =
-      State.PageSize * (1 + State.MaxSimultaneousAllocations) +
+      PageSize * (1 + State.MaxSimultaneousAllocations) +
       State.MaxSimultaneousAllocations * State.maximumAllocationSize();
-  void *GuardedPoolMemory = mapMemory(PoolBytesRequired, kGwpAsanGuardPageName);
+  assert(PoolBytesRequired % PageSize == 0);
+  void *GuardedPoolMemory = reserveGuardedPool(PoolBytesRequired);
 
-  size_t BytesRequired = State.MaxSimultaneousAllocations * sizeof(*Metadata);
+  size_t BytesRequired =
+      roundUpTo(State.MaxSimultaneousAllocations * sizeof(*Metadata), PageSize);
   Metadata = reinterpret_cast<AllocationMetadata *>(
-      mapMemory(BytesRequired, kGwpAsanMetadataName));
-  markReadWrite(Metadata, BytesRequired, kGwpAsanMetadataName);
+      map(BytesRequired, kGwpAsanMetadataName));
 
   // Allocate memory and set up the free pages queue.
-  BytesRequired = State.MaxSimultaneousAllocations * sizeof(*FreeSlots);
-  FreeSlots = reinterpret_cast<size_t *>(
-      mapMemory(BytesRequired, kGwpAsanFreeSlotsName));
-  markReadWrite(FreeSlots, BytesRequired, kGwpAsanFreeSlotsName);
+  BytesRequired = roundUpTo(
+      State.MaxSimultaneousAllocations * sizeof(*FreeSlots), PageSize);
+  FreeSlots =
+      reinterpret_cast<size_t *>(map(BytesRequired, kGwpAsanFreeSlotsName));
 
   // Multiply the sample rate by 2 to give a good, fast approximation for (1 /
   // SampleRate) chance of sampling.
   if (Opts.SampleRate != 1)
     AdjustedSampleRatePlusOne = static_cast<uint32_t>(Opts.SampleRate) * 2 + 1;
   else
     AdjustedSampleRatePlusOne = 2;
 
for (size_t i = 0; i < State.MaxSimultaneousAllocations; ++i) {
     if (Meta.Addr && !Meta.IsDeallocated && Meta.Addr >= Start &&
         Meta.Addr < Start + Size)
       Cb(Meta.Addr, Meta.Size, Arg);
   }
 }
 
 void GuardedPoolAllocator::uninitTestOnly() {
   if (State.GuardedPagePool) {
-    unmapMemory(reinterpret_cast<void *>(State.GuardedPagePool),
-                State.GuardedPagePoolEnd - State.GuardedPagePool,
-                kGwpAsanGuardPageName);
+    unreserveGuardedPool();
     State.GuardedPagePool = 0;
     State.GuardedPagePoolEnd = 0;
   }
   if (Metadata) {
-    unmapMemory(Metadata, State.MaxSimultaneousAllocations * sizeof(*Metadata),
-                kGwpAsanMetadataName);
+    unmap(Metadata,
+          roundUpTo(State.MaxSimultaneousAllocations * sizeof(*Metadata),
+                    State.PageSize));
     Metadata = nullptr;
   }
   if (FreeSlots) {
-    unmapMemory(FreeSlots,
-                State.MaxSimultaneousAllocations * sizeof(*FreeSlots),
-                kGwpAsanFreeSlotsName);
+    unmap(FreeSlots,
+          roundUpTo(State.MaxSimultaneousAllocations * sizeof(*FreeSlots),
+                    State.PageSize));
     FreeSlots = nullptr;
   }
 }
 
 static uintptr_t getPageAddr(uintptr_t Ptr, uintptr_t PageSize) {
   return Ptr & ~(PageSize - 1);
 }
 
if (getRandomUnsigned32() % 2 == 0) {
     Ptr +=
         State.maximumAllocationSize() - rightAlignedAllocationSize(Size, Align);
   }
   AllocationMetadata *Meta = addrToMetadata(Ptr);
 
   // If a slot is multiple pages in size, and the allocation takes up a single
   // page, we can improve overflow detection by leaving the unused pages as
   // unmapped.
-  markReadWrite(reinterpret_cast<void *>(getPageAddr(Ptr, State.PageSize)),
-                Size, kGwpAsanAliveSlotName);
+  const size_t PageSize = State.PageSize;
+  allocateInGuardedPool(reinterpret_cast<void *>(getPageAddr(Ptr, PageSize)),
+                        roundUpTo(Size, PageSize));
 
   Meta->RecordAllocation(Ptr, Size);
   Meta->AllocationTrace.RecordBacktrace(Backtrace);
 
   return reinterpret_cast<void *>(Ptr);
 }
 
 void GuardedPoolAllocator::trapOnAddress(uintptr_t Address, Error E) {
// pool during the expensive markInaccessible() call.
     // Ensure that the unwinder is not called if the recursive flag is set,
     // otherwise non-reentrant unwinders may deadlock.
     if (!ThreadLocals.RecursiveGuard) {
       ScopedRecursiveGuard SRG;
       Meta->DeallocationTrace.RecordBacktrace(Backtrace);
     }
   }
 
-  markInaccessible(reinterpret_cast<void *>(SlotStart),
-                   State.maximumAllocationSize(), kGwpAsanGuardPageName);
+  deallocateInGuardedPool(reinterpret_cast<void *>(SlotStart),
+                          State.maximumAllocationSize());
 
   // And finally, lock again to release the slot back into the pool.
   ScopedLock L(PoolMutex);
   freeSlot(Slot);
 }
 
 size_t GuardedPoolAllocator::getSize(const void *Ptr) {
   assert(pointerIsMine(Ptr));

compiler-rt/lib/gwp_asan/platform_specific/guarded_pool_allocator_posix.cpp

 #include <unistd.h>
 
 #ifdef ANDROID
 #include <sys/prctl.h>
 #define PR_SET_VMA 0x53564d41
 #define PR_SET_VMA_ANON_NAME 0
 #endif // ANDROID
 
+namespace {
 void MaybeSetMappingName(void *Mapping, size_t Size, const char *Name) {
 #ifdef ANDROID
   prctl(PR_SET_VMA, PR_SET_VMA_ANON_NAME, Mapping, Size, Name);
 #endif // ANDROID
   // Anonymous mapping names are only supported on Android.
   return;
 }
+} // anonymous namespace
 
 namespace gwp_asan {
 
 void GuardedPoolAllocator::initPRNG() {
-  ThreadLocals.RandomState = time(nullptr) + getThreadID();
+  ThreadLocals.RandomState =
+      static_cast<uint32_t>(time(nullptr) + getThreadID());
 }
 
-void *GuardedPoolAllocator::mapMemory(size_t Size, const char *Name) const {
-  void *Ptr =
-      mmap(nullptr, Size, PROT_NONE, MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
+void *GuardedPoolAllocator::map(size_t Size, const char *Name) const {
+  assert((Size % State.PageSize) == 0);
+  void *Ptr = mmap(nullptr, Size, PROT_READ | PROT_WRITE,
+                   MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
   Check(Ptr != MAP_FAILED, "Failed to map guarded pool allocator memory");
   MaybeSetMappingName(Ptr, Size, Name);
   return Ptr;
 }
 
-void GuardedPoolAllocator::unmapMemory(void *Ptr, size_t Size,
-                                       const char *Name) const {
+void GuardedPoolAllocator::unmap(void *Ptr, size_t Size) const {
+  assert((reinterpret_cast<uintptr_t>(Ptr) % State.PageSize) == 0);
+  assert((Size % State.PageSize) == 0);
   Check(munmap(Ptr, Size) == 0,
         "Failed to unmap guarded pool allocator memory.");
-  MaybeSetMappingName(Ptr, Size, Name);
 }
 
-void GuardedPoolAllocator::markReadWrite(void *Ptr, size_t Size,
-                                         const char *Name) const {
+void *GuardedPoolAllocator::reserveGuardedPool(size_t Size) {
+  assert((Size % State.PageSize) == 0);
+  void *Ptr =
+      mmap(nullptr, Size, PROT_NONE, MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
+  Check(Ptr != MAP_FAILED, "Failed to reserve guarded pool allocator memory");
+  MaybeSetMappingName(Ptr, Size, kGwpAsanGuardPageName);
+  return Ptr;
+}
+
+void GuardedPoolAllocator::unreserveGuardedPool() {
+  unmap(reinterpret_cast<void *>(State.GuardedPagePool),
+        State.GuardedPagePoolEnd - State.GuardedPagePool);
+}
+
+void GuardedPoolAllocator::allocateInGuardedPool(void *Ptr, size_t Size) const {
+  assert((reinterpret_cast<uintptr_t>(Ptr) % State.PageSize) == 0);
+  assert((Size % State.PageSize) == 0);
   Check(mprotect(Ptr, Size, PROT_READ | PROT_WRITE) == 0,
-        "Failed to set guarded pool allocator memory at as RW.");
-  MaybeSetMappingName(Ptr, Size, Name);
+        "Failed to allocate in guarded pool allocator memory");
+  MaybeSetMappingName(Ptr, Size, kGwpAsanAliveSlotName);
 }
 
-void GuardedPoolAllocator::markInaccessible(void *Ptr, size_t Size,
-                                            const char *Name) const {
+void GuardedPoolAllocator::deallocateInGuardedPool(void *Ptr,
+                                                   size_t Size) const {
+  assert((reinterpret_cast<uintptr_t>(Ptr) % State.PageSize) == 0);
+  assert((Size % State.PageSize) == 0);
   // mmap() a PROT_NONE page over the address to release it to the system, if
   // we used mprotect() here the system would count pages in the quarantine
   // against the RSS.
   Check(mmap(Ptr, Size, PROT_NONE, MAP_FIXED | MAP_ANONYMOUS | MAP_PRIVATE, -1,
              0) != MAP_FAILED,
-        "Failed to set guarded pool allocator memory as inaccessible.");
-  MaybeSetMappingName(Ptr, Size, Name);
+        "Failed to deallocate in guarded pool allocator memory");
+  MaybeSetMappingName(Ptr, Size, kGwpAsanGuardPageName);
 }
 
 size_t GuardedPoolAllocator::getPlatformPageSize() {
   return sysconf(_SC_PAGESIZE);
 }
 
 void GuardedPoolAllocator::installAtFork() {
   auto Disable = []() {
     if (auto *S = getSingleton())
       S->disable();
   };
   auto Enable = []() {
     if (auto *S = getSingleton())
       S->enable();
   };
   pthread_atfork(Disable, Enable, Enable);
 }
-
 } // namespace gwp_asan