[scudo] Manage pages with MemMap in Secondary Allocator

[Chia-hungDuan]

Summary

Replace the uses of raw map()/unmap(), .etc calls with MemMap. Also
remove the direct use of MapPlatformData in the secondary allocator.

Also add setMemoryPermission() in MemMap.

[Chia-hungDuan]

Rebase only

[cferris]

compiler-rt/lib/scudo/standalone/secondary.h

345

Do you expect that this struct will ever have other members in the near future?

If not, it might be better to dump the struct and create an array like MemMaps[] instead.

[Chia-hungDuan]

Address review comment

compiler-rt/lib/scudo/standalone/secondary.h

345

Nice catch!

[cferris]

LGTM

[fabio-d]

compiler-rt/lib/scudo/standalone/secondary.h

114

Like in the other CL, this breaks releasePagesToOS on Fuchsia. I think it's the usual bad offset due to the fact that MemMap was created like this

MemMapT MemMap = ReservedMemory.dispatch(ReservedMemory.getBase(),
                                         ReservedMemory.getCapacity());

but ReservedMemory.getBase() != CommitBase

[Chia-hungDuan]

compiler-rt/lib/scudo/standalone/secondary.h

114

Sorry, I'm not following. The logic here is supposed to be the same,

Before:

const uptr MapSize = RoundedSize + 2 * PageSize;
uptr MapBase = reinterpret_cast<uptr>(
      map(nullptr, MapSize, nullptr, MAP_NOACCESS | MAP_ALLOWNOMEM, &Data));
...
mapSecondary<Config>(Options, CommitBase, CommitSize, AllocPos, 0, &Data);

After

ReservedMemoryT ReservedMemory;
const uptr MapSize = RoundedSize + 2 * PageSize;
ReservedMemory.create(/*Addr=*/0U, MapSize, nullptr, MAP_ALLOWNOMEM);

  // Take the entire ownership of reserved region.
MemMapT MemMap = ReservedMemory.dispatch(ReservedMemory.getBase(), ReservedMemory.getCapacity());
...
mapSecondary<Config>(Options, CommitBase, CommitSize, AllocPos, 0, MemMap);

The arguments of mapSecondary<Config> are the same as before (except the MapPlatformData) and the call to map() is still the same. I.e., it's CommitBase, not MapBase(which is ReservedMemory.getBase()) since before.

Besides, I think Fuchsia doesn't enable cache in secondary allocator so it's supposed not to do releasePagesToOS in secondary allocator. Can you share more details about why it didn't cause problem before?

[fabio-d]

compiler-rt/lib/scudo/standalone/secondary.h

114

Sure! What used to happen was that a VMAR was created which was slightly larger than the actual VMO, to leave room for guard pages (at the beginning and at then end) and skipping the misaligned portion.

Now it's still the case. However, since the MemMap is now initialized initialized to cover the whole VMAR range, it thinks that the VMO is mapped at at ReservedMemory.getBase() while in fact it starts at CommitBase:

MemMapT MemMap = ReservedMemory.dispatch(ReservedMemory.getBase(), // this ends up being stored in its Base field
                                         ReservedMemory.getCapacity());
...
mapSecondary<Config>(Options, CommitBase, CommitSize, AllocPos, 0, MemMap);

but in mapSecondary:

MemMap.remap(CommitBase, CommitSize, "scudo:secondary", RemapFlags); // the VMO is created and mapped now at CommitBase

This leaves a hole at the beginning of the MemMap which is causing the same issue as the other CL. In particular, when the Secondary allocator decides it's time to decommit the whole [CommitBase, CommitBase + CommitSize] range, it will call:

// Base = ReservedMemory.getBase()
// From = CommitBase
// Size = CommitSize
void MemMapDefault::releaseAndZeroPagesToOSImpl(uptr From, uptr Size) {
  return ::scudo::releasePagesToOS(Base, From - Base, Size, &Data); // From - Base should have been zero, but it isn't
}

As for the question of why this matters on Fuchsia at all, given that it doesn't enable the secondary allocator cache in its config, the answer is that some tests exercise other configs too. In this case, for instance, I'm seeing this issue manifesting as a ScudoCombinedDeathTestBasicCombined1_AndroidSvelteConfig failure.

[Chia-hungDuan]

compiler-rt/lib/scudo/standalone/secondary.h

114

So the problem is not the mapping, it's the releasePagesToOS instead. The offset should be the offset from MappedBase Given that MemMapDefault is a temporary bridge before the transition complete. I would suggest two ways for this,

1. Apply D148141 (and/or revert D147792 because the problem is not in mapping)
2. Temporarily disable the AndroidSvelteConfig in combined_test.cpp. (Given that AndroidSvelte is being deprecated)

Sorry for the breakage.

[Chia-hungDuan]

compiler-rt/lib/scudo/standalone/secondary.h

114

After thinking this a little bit, let me take the solution 2 back. It'll be better to keep everything pass with MmeMapDefault at this moment.

So let's go for 1. and 2. can be an option after it.

compiler-rt/lib/scudo/standalone/mem_map.h

 public:
   constexpr MemMapDefault() = default;
   MemMapDefault(uptr Base, uptr Capacity) : Base(Base), Capacity(Capacity) {}
 
   // Impls for base functions.
   bool mapImpl(uptr Addr, uptr Size, const char *Name, uptr Flags);
   void unmapImpl(uptr Addr, uptr Size);
   bool remapImpl(uptr Addr, uptr Size, const char *Name, uptr Flags);
+  void setMemoryPermissionImpl(uptr Addr, uptr Size, uptr Flags);
   void releasePagesToOSImpl(uptr From, uptr Size) {
     return releaseAndZeroPagesToOSImpl(From, Size);
   }
   void releaseAndZeroPagesToOSImpl(uptr From, uptr Size);
   uptr getBaseImpl() { return Base; }
   uptr getCapacityImpl() { return Capacity; }
 
   void setMapPlatformData(MapPlatformData &NewData) { Data = NewData; }

compiler-rt/lib/scudo/standalone/mem_map.cpp

void *RemappedAddr =
       ::scudo::map(reinterpret_cast<void *>(Addr), Size, Name, Flags, &Data);
   return reinterpret_cast<uptr>(RemappedAddr) == Addr;
 }
 
 void MemMapDefault::releaseAndZeroPagesToOSImpl(uptr From, uptr Size) {
   return ::scudo::releasePagesToOS(Base, From - Base, Size, &Data);
 }
 
+void MemMapDefault::setMemoryPermissionImpl(uptr Addr, uptr Size, uptr Flags) {
+  return ::scudo::setMemoryPermission(Addr, Size, Flags);
+}
+
 void ReservedMemoryDefault::releaseImpl() {
   ::scudo::unmap(reinterpret_cast<void *>(Base), Capacity, UNMAP_ALL, &Data);
 }
 
 bool ReservedMemoryDefault::createImpl(uptr Addr, uptr Size, const char *Name,
                                        uptr Flags) {
   void *Reserved = ::scudo::map(reinterpret_cast<void *>(Addr), Size, Name,
                                 Flags | MAP_NOACCESS, &Data);

compiler-rt/lib/scudo/standalone/mem_map_base.h

public:
   // ReservedMemory). For example, we have reserved several pages and then we
   // want to remap them with different accessibility.
   bool remap(uptr Addr, uptr Size, const char *Name, uptr Flags = 0) {
     DCHECK(isAllocated());
     DCHECK((Addr >= getBase()) || (Addr + Size <= getBase() + getCapacity()));
     return invokeImpl(&Derived::remapImpl, Addr, Size, Name, Flags);
   }
 
+  // This is used to update the pages' access permission. For example, mark
+  // pages as no read/write permission.
+  void setMemoryPermission(uptr Addr, uptr Size, uptr Flags) {
+    DCHECK(isAllocated());
+    DCHECK((Addr >= getBase()) || (Addr + Size <= getBase() + getCapacity()));
+    return static_cast<Derived *>(this)->setMemoryPermissionImpl(Addr, Size,
+                                                                 Flags);
+  }
+
   // Suggest releasing a set of contiguous physical pages back to the OS. Note
   // that only physical pages are supposed to be released. Any release of
   // virtual pages may lead to undefined behavior.
   void releasePagesToOS(uptr From, uptr Size) {
     DCHECK(isAllocated());
     DCHECK((From >= getBase()) || (From + Size <= getBase() + getCapacity()));
     invokeImpl(&Derived::releasePagesToOSImpl, From, Size);
   }

compiler-rt/lib/scudo/standalone/secondary.h

 //===-- secondary.h ---------------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef SCUDO_SECONDARY_H_
 #define SCUDO_SECONDARY_H_
 
 #include "chunk.h"
 #include "common.h"
 #include "list.h"
+#include "mem_map.h"
 #include "memtag.h"
 #include "mutex.h"
 #include "options.h"
 #include "stats.h"
 #include "string_utils.h"
 #include "thread_annotations.h"
 
 namespace scudo {

 struct alignas(Max<uptr>(archSupportsMemoryTagging()
                              ? archMemoryTagGranuleSize()
                              : 1,
                          1U << SCUDO_MIN_ALIGNMENT_LOG)) Header {
   LargeBlock::Header *Prev;
   LargeBlock::Header *Next;
   uptr CommitBase;
   uptr CommitSize;
-  uptr MapBase;
-  uptr MapSize;
-  [[no_unique_address]] MapPlatformData Data;
+  MemMapT MemMap;
 };
 
 static_assert(sizeof(Header) % (1U << SCUDO_MIN_ALIGNMENT_LOG) == 0, "");
 static_assert(!archSupportsMemoryTagging() ||
                   sizeof(Header) % archMemoryTagGranuleSize() == 0,
               "");
 
 constexpr uptr getHeaderSize() { return sizeof(Header); }

 
 template <typename Config> static Header *getHeader(const void *Ptr) {
   return getHeader<Config>(reinterpret_cast<uptr>(Ptr));
 }
 
 } // namespace LargeBlock
 
 static void unmap(LargeBlock::Header *H) {
-  MapPlatformData Data = H->Data;
-  unmap(reinterpret_cast<void *>(H->MapBase), H->MapSize, UNMAP_ALL, &Data);
+  // Note that the `H->MapMap` is stored on the pages managed by itself. Take
+  // over the ownership before unmap() so that any operation along with unmap()
+  // won't touch inaccessible pages.
+  MemMapT MemMap = H->MemMap;
+  MemMap.unmap(MemMap.getBase(), MemMap.getCapacity());
 }
 
 class MapAllocatorNoCache {
 public:
   void init(UNUSED s32 ReleaseToOsInterval) {}
   bool retrieve(UNUSED Options Options, UNUSED uptr Size, UNUSED uptr Alignment,
                 UNUSED LargeBlock::Header **H, UNUSED bool *Zeroed) {
     return false;
bool setOption(Option O, UNUSED sptr Value) {
     return true;
   }
 };
 
 static const uptr MaxUnusedCachePages = 4U;
 
 template <typename Config>
 void mapSecondary(Options Options, uptr CommitBase, uptr CommitSize,
-                  uptr AllocPos, uptr Flags, MapPlatformData *Data) {
+                  uptr AllocPos, uptr Flags, MemMapT &MemMap) {
   const uptr MaxUnusedCacheBytes = MaxUnusedCachePages * getPageSizeCached();
   if (useMemoryTagging<Config>(Options) && CommitSize > MaxUnusedCacheBytes) {
     const uptr UntaggedPos = Max(AllocPos, CommitBase + MaxUnusedCacheBytes);
-    map(reinterpret_cast<void *>(CommitBase), UntaggedPos - CommitBase,
-        "scudo:secondary", MAP_RESIZABLE | MAP_MEMTAG | Flags, Data);
-    map(reinterpret_cast<void *>(UntaggedPos),
-        CommitBase + CommitSize - UntaggedPos, "scudo:secondary",
-        MAP_RESIZABLE | Flags, Data);
+    MemMap.remap(CommitBase, UntaggedPos - CommitBase, "scudo:secondary",
+                 MAP_RESIZABLE | MAP_MEMTAG | Flags);
+    MemMap.remap(UntaggedPos, CommitBase + CommitSize - UntaggedPos,
+                 "scudo:secondary", MAP_RESIZABLE | Flags);
   } else {
-    map(reinterpret_cast<void *>(CommitBase), CommitSize, "scudo:secondary",
+    const uptr RemapFlags =
         MAP_RESIZABLE | (useMemoryTagging<Config>(Options) ? MAP_MEMTAG : 0) |
-            Flags,
-        Data);
+        Flags;
+    MemMap.remap(CommitBase, CommitSize, "scudo:secondary", RemapFlags);
   }
 }
 
 // Template specialization to avoid producing zero-length array
 template <typename T, size_t Size> class NonZeroLengthArray {
 public:
   T &operator[](uptr Idx) { return values[Idx]; }
 
void store(Options Options, LargeBlock::Header *H) EXCLUDES(Mutex) {
     bool EntryCached = false;
     bool EmptyCache = false;
     const s32 Interval = atomic_load_relaxed(&ReleaseToOsIntervalMs);
     const u64 Time = getMonotonicTime();
     const u32 MaxCount = atomic_load_relaxed(&MaxEntriesCount);
     CachedBlock Entry;
     Entry.CommitBase = H->CommitBase;
     Entry.CommitSize = H->CommitSize;
-    Entry.MapBase = H->MapBase;
-    Entry.MapSize = H->MapSize;
     Entry.BlockBegin = reinterpret_cast<uptr>(H + 1);
-    Entry.Data = H->Data;
+    Entry.MemMap = H->MemMap;
     Entry.Time = Time;
     if (useMemoryTagging<Config>(Options)) {
       if (Interval == 0 && !SCUDO_FUCHSIA) {
         // Release the memory and make it inaccessible at the same time by
         // creating a new MAP_NOACCESS mapping on top of the existing mapping.
         // Fuchsia does not support replacing mappings by creating a new mapping
         // on top so we just do the two syscalls there.
         Entry.Time = 0;
         mapSecondary<Config>(Options, Entry.CommitBase, Entry.CommitSize,
-                             Entry.CommitBase, MAP_NOACCESS, &Entry.Data);
+                             Entry.CommitBase, MAP_NOACCESS, Entry.MemMap);
       } else {
-        setMemoryPermission(Entry.CommitBase, Entry.CommitSize, MAP_NOACCESS,
-                            &Entry.Data);
+        Entry.MemMap.setMemoryPermission(Entry.CommitBase, Entry.CommitSize,
+                                         MAP_NOACCESS);
       }
     } else if (Interval == 0) {
-      releasePagesToOS(Entry.CommitBase, 0, Entry.CommitSize, &Entry.Data);
+      Entry.MemMap.releasePagesToOS(Entry.CommitBase, Entry.CommitSize);
       Entry.Time = 0;
     }
     do {
       ScopedLock L(Mutex);
       if (useMemoryTagging<Config>(Options) && QuarantinePos == -1U) {
         // If we get here then memory tagging was disabled in between when we
         // read Options and when we locked Mutex. We can't insert our entry into
         // the quarantine or the cache because the permissions would be wrong so
do {
         }
       }
     } while (0);
     if (EmptyCache)
       empty();
     else if (Interval >= 0)
       releaseOlderThan(Time - static_cast<u64>(Interval) * 1000000);
     if (!EntryCached)
-      unmap(reinterpret_cast<void *>(Entry.MapBase), Entry.MapSize, UNMAP_ALL,
-            &Entry.Data);
+      Entry.MemMap.unmap(Entry.MemMap.getBase(), Entry.MemMap.getCapacity());
   }
 
   bool retrieve(Options Options, uptr Size, uptr Alignment,
                 LargeBlock::Header **H, bool *Zeroed) EXCLUDES(Mutex) {
     const uptr PageSize = getPageSizeCached();
     const u32 MaxCount = atomic_load_relaxed(&MaxEntriesCount);
     bool Found = false;
     CachedBlock Entry;
bool retrieve(Options Options, uptr Size, uptr Alignment,
     }
     if (!Found)
       return false;
 
     *H = reinterpret_cast<LargeBlock::Header *>(
         LargeBlock::addHeaderTag<Config>(HeaderPos));
     *Zeroed = Entry.Time == 0;
     if (useMemoryTagging<Config>(Options))
-      setMemoryPermission(Entry.CommitBase, Entry.CommitSize, 0, &Entry.Data);
+      Entry.MemMap.setMemoryPermission(Entry.CommitBase, Entry.CommitSize, 0);
     uptr NewBlockBegin = reinterpret_cast<uptr>(*H + 1);
     if (useMemoryTagging<Config>(Options)) {
       if (*Zeroed) {
         storeTags(LargeBlock::addHeaderTag<Config>(Entry.CommitBase),
                   NewBlockBegin);
       } else if (Entry.BlockBegin < NewBlockBegin) {
         storeTags(Entry.BlockBegin, NewBlockBegin);
       } else {
         storeTags(untagPointer(NewBlockBegin),
                   untagPointer(Entry.BlockBegin));
       }
     }
     (*H)->CommitBase = Entry.CommitBase;
     (*H)->CommitSize = Entry.CommitSize;
-    (*H)->MapBase = Entry.MapBase;
-    (*H)->MapSize = Entry.MapSize;
-    (*H)->Data = Entry.Data;
+    (*H)->MemMap = Entry.MemMap;
     return true;
   }
 
   bool canCache(uptr Size) {
     return atomic_load_relaxed(&MaxEntriesCount) != 0U &&
            Size <= atomic_load_relaxed(&MaxEntrySize);
   }
 
public:
   }
 
   void releaseToOS() { releaseOlderThan(UINT64_MAX); }
 
   void disableMemoryTagging() EXCLUDES(Mutex) {
     ScopedLock L(Mutex);
     for (u32 I = 0; I != Config::SecondaryCacheQuarantineSize; ++I) {
       if (Quarantine[I].CommitBase) {
-        unmap(reinterpret_cast<void *>(Quarantine[I].MapBase),
-              Quarantine[I].MapSize, UNMAP_ALL, &Quarantine[I].Data);
+        MemMapT &MemMap = Quarantine[I].MemMap;
+        MemMap.unmap(MemMap.getBase(), MemMap.getCapacity());
         Quarantine[I].CommitBase = 0;
       }
     }
     const u32 MaxCount = atomic_load_relaxed(&MaxEntriesCount);
-    for (u32 I = 0; I < MaxCount; I++)
-      if (Entries[I].CommitBase)
-        setMemoryPermission(Entries[I].CommitBase, Entries[I].CommitSize, 0,
-                            &Entries[I].Data);
+    for (u32 I = 0; I < MaxCount; I++) {
+      if (Entries[I].CommitBase) {
+        Entries[I].MemMap.setMemoryPermission(Entries[I].CommitBase,
+                                              Entries[I].CommitSize, 0);
+      }
+    }
     QuarantinePos = -1U;
   }
 
   void disable() NO_THREAD_SAFETY_ANALYSIS { Mutex.lock(); }
 
   void enable() NO_THREAD_SAFETY_ANALYSIS { Mutex.unlock(); }
 
   void unmapTestOnly() { empty(); }
 
 private:
   void empty() {
-    struct {
-      void *MapBase;
-      uptr MapSize;
-      MapPlatformData Data;
-    } MapInfo[Config::SecondaryCacheEntriesArraySize];
+    MemMapT MapInfo[Config::SecondaryCacheEntriesArraySize];
     uptr N = 0;
     {
       ScopedLock L(Mutex);
       for (uptr I = 0; I < Config::SecondaryCacheEntriesArraySize; I++) {
         if (!Entries[I].CommitBase)
           continue;
-        MapInfo[N].MapBase = reinterpret_cast<void *>(Entries[I].MapBase);
-        MapInfo[N].MapSize = Entries[I].MapSize;
-        MapInfo[N].Data = Entries[I].Data;
+        MapInfo[N] = Entries[I].MemMap;
         Entries[I].CommitBase = 0;
         N++;
       }
       EntriesCount = 0;
       IsFullEvents = 0;
     }
-    for (uptr I = 0; I < N; I++)
-      unmap(MapInfo[I].MapBase, MapInfo[I].MapSize, UNMAP_ALL,
-            &MapInfo[I].Data);
+    for (uptr I = 0; I < N; I++) {
+      MemMapT &MemMap = MapInfo[I];
+      MemMap.unmap(MemMap.getBase(), MemMap.getCapacity());
+    }
   }
 
   struct CachedBlock {
     uptr CommitBase;
     uptr CommitSize;
-    uptr MapBase;
-    uptr MapSize;
     uptr BlockBegin;
-    [[no_unique_address]] MapPlatformData Data;
+    MemMapT MemMap;
     u64 Time;
   };
 
   void releaseIfOlderThan(CachedBlock &Entry, u64 Time) REQUIRES(Mutex) {
     if (!Entry.CommitBase || !Entry.Time)
       return;
     if (Entry.Time > Time) {
       if (OldestTime == 0 || Entry.Time < OldestTime)
         OldestTime = Entry.Time;
       return;
     }
-    releasePagesToOS(Entry.CommitBase, 0, Entry.CommitSize, &Entry.Data);
+    Entry.MemMap.releasePagesToOS(Entry.CommitBase, Entry.CommitSize);
     Entry.Time = 0;
   }
 
   void releaseOlderThan(u64 Time) EXCLUDES(Mutex) {
     ScopedLock L(Mutex);
     if (!EntriesCount || OldestTime == 0 || OldestTime > Time)
       return;
     OldestTime = 0;
if (Cache.retrieve(Options, Size, Alignment, &H, &Zeroed)) {
                BlockEnd - PtrInt);
       const uptr BlockSize = BlockEnd - HInt;
       {
         ScopedLock L(Mutex);
         InUseBlocks.push_back(H);
         AllocatedBytes += BlockSize;
         NumberOfAllocs++;
         Stats.add(StatAllocated, BlockSize);
-        Stats.add(StatMapped, H->MapSize);
+        Stats.add(StatMapped, H->MemMap.getCapacity());
       }
       return Ptr;
     }
   }
 
-  MapPlatformData Data = {};
+  ReservedMemoryT ReservedMemory;
   const uptr MapSize = RoundedSize + 2 * PageSize;
-  uptr MapBase = reinterpret_cast<uptr>(
-      map(nullptr, MapSize, nullptr, MAP_NOACCESS | MAP_ALLOWNOMEM, &Data));
+  ReservedMemory.create(/*Addr=*/0U, MapSize, nullptr, MAP_ALLOWNOMEM);
+
+  // Take the entire ownership of reserved region.
+  MemMapT MemMap = ReservedMemory.dispatch(ReservedMemory.getBase(),
+                                           ReservedMemory.getCapacity());
+  uptr MapBase = MemMap.getBase();
   if (UNLIKELY(!MapBase))
     return nullptr;
   uptr CommitBase = MapBase + PageSize;
   uptr MapEnd = MapBase + MapSize;
 
   // In the unlikely event of alignments larger than a page, adjust the amount
   // of memory we want to commit, and trim the extra memory.
   if (UNLIKELY(Alignment >= PageSize)) {
     // For alignments greater than or equal to a page, the user pointer (eg: the
     // pointer that is returned by the C or C++ allocation APIs) ends up on a
     // page boundary , and our headers will live in the preceding page.
     CommitBase = roundUp(MapBase + PageSize + 1, Alignment) - PageSize;
     const uptr NewMapBase = CommitBase - PageSize;
     DCHECK_GE(NewMapBase, MapBase);
     // We only trim the extra memory on 32-bit platforms: 64-bit platforms
     // are less constrained memory wise, and that saves us two syscalls.
     if (SCUDO_WORDSIZE == 32U && NewMapBase != MapBase) {
-      unmap(reinterpret_cast<void *>(MapBase), NewMapBase - MapBase, 0, &Data);
+      MemMap.unmap(MapBase, NewMapBase - MapBase);
       MapBase = NewMapBase;
     }
     const uptr NewMapEnd =
         CommitBase + PageSize + roundUp(Size, PageSize) + PageSize;
     DCHECK_LE(NewMapEnd, MapEnd);
     if (SCUDO_WORDSIZE == 32U && NewMapEnd != MapEnd) {
-      unmap(reinterpret_cast<void *>(NewMapEnd), MapEnd - NewMapEnd, 0, &Data);
+      MemMap.unmap(NewMapEnd, MapEnd - NewMapEnd);
       MapEnd = NewMapEnd;
     }
   }
 
   const uptr CommitSize = MapEnd - PageSize - CommitBase;
   const uptr AllocPos = roundDown(CommitBase + CommitSize - Size, Alignment);
-  mapSecondary<Config>(Options, CommitBase, CommitSize, AllocPos, 0, &Data);
+  mapSecondary<Config>(Options, CommitBase, CommitSize, AllocPos, 0, MemMap);
   const uptr HeaderPos =
       AllocPos - Chunk::getHeaderSize() - LargeBlock::getHeaderSize();
   LargeBlock::Header *H = reinterpret_cast<LargeBlock::Header *>(
       LargeBlock::addHeaderTag<Config>(HeaderPos));
   if (useMemoryTagging<Config>(Options))
     storeTags(LargeBlock::addHeaderTag<Config>(CommitBase),
               reinterpret_cast<uptr>(H + 1));
-  H->MapBase = MapBase;
-  H->MapSize = MapEnd - MapBase;
   H->CommitBase = CommitBase;
   H->CommitSize = CommitSize;
-  H->Data = Data;
+  H->MemMap = MemMap;
   if (BlockEndPtr)
     *BlockEndPtr = CommitBase + CommitSize;
   {
     ScopedLock L(Mutex);
     InUseBlocks.push_back(H);
     AllocatedBytes += CommitSize;
     if (LargestSize < CommitSize)
       LargestSize = CommitSize;
     NumberOfAllocs++;
     Stats.add(StatAllocated, CommitSize);
-    Stats.add(StatMapped, H->MapSize);
+    Stats.add(StatMapped, H->MemMap.getCapacity());
   }
   return reinterpret_cast<void *>(HeaderPos + LargeBlock::getHeaderSize());
 }
 
 template <typename Config>
 void MapAllocator<Config>::deallocate(Options Options, void *Ptr)
     EXCLUDES(Mutex) {
   LargeBlock::Header *H = LargeBlock::getHeader<Config>(Ptr);
   const uptr CommitSize = H->CommitSize;
   {
     ScopedLock L(Mutex);
     InUseBlocks.remove(H);
     FreedBytes += CommitSize;
     NumberOfFrees++;
     Stats.sub(StatAllocated, CommitSize);
-    Stats.sub(StatMapped, H->MapSize);
+    Stats.sub(StatMapped, H->MemMap.getCapacity());
   }
   Cache.store(Options, H);
 }
 
 template <typename Config>
 void MapAllocator<Config>::getStats(ScopedString *Str) EXCLUDES(Mutex) {
   ScopedLock L(Mutex);
   Str->append("Stats: MapAllocator: allocated %u times (%zuK), freed %u times "