[scudo] Manage pages with MemMap in SizeClassAllocator64

[Chia-hungDuan]

Summary

Introduce a new data structure to manage the allocated pages from the
system. This is meant to deprecate certain memory system call wrappers
in Scudo, e.g., map()/unmap(). Besides, we would like to make
MapPlatformData to be appeared in platform specific data structure only.
Given that there are several allocators in Scudo and each of them has
different way of page management. The deprecation will be done in
several CLs. In this commit, we start from SizeClassAllocator64.

[Chia-hungDuan]

So far I call it RFC but it's a formal CL as well. There are still many places we can simplify. In order to make the review process easier, I'll do the simplification in coming CLs.

[Chia-hungDuan]

compiler-rt/lib/scudo/standalone/allocator_config.h

104 ↗

(On Diff #504942)

As @fabio-d suggested, I'll move this out from the config. So that we can still easily mix different configs on different platforms

[Chia-hungDuan]

Don't define MapReserver in allocator config

[Chia-hungDuan]

compiler-rt/lib/scudo/standalone/release.h

50

Note:This will be deprecated as well.

[Chia-hungDuan]

Make getCapacity as required and add few more comments

[Chia-hungDuan]

Add unmap() and remove the use of MapPlatformData in primary64

[Chia-hungDuan]

Remove the defining of MapReserver in the tests

[Chia-hungDuan]

compiler-rt/lib/scudo/standalone/mem_map.h

76

Per discussion, this may not be a good name. It may lead some confusion. I'm considering to make it either ReservedMemory or change it to has-a relation.

[Chia-hungDuan]

1. Rename MapReserver to ReservedMap.
2. Add more comments on why ReservedMap still derives from MemMapBase
3. Leave the unused functions in ReservedMap unimplemented (which were having default implementations which does nothing)

[Chia-hungDuan]

compiler-rt/lib/scudo/standalone/mem_map.h

81

Will be rephrased as

`ReservedMap` is a special MemMap which still manages a contiguous pages

[Chia-hungDuan]

compiler-rt/lib/scudo/standalone/mem_map.h

68

We may consider having two versions of this operation,

1. releaseAndZeroPagesToOS will ensure the later accessing the released pages will get it zero-filled
2. releasePagesToOS is optional to zero-fill

The reason for this is these two operations have performance difference on some platforms and in Scudo, not all releasePagesToOS calls reply on the zero-fill property.

Will update the CL with this change

[Chia-hungDuan]

Add releaseAndZeroPagesToOS and add more DCHECK in the base classes

[Chia-hungDuan]

Move some stuff to mem_map_base.h so that it will make the use of platform MemMap easier.

Also add more comments and checks

[Chia-hungDuan]

Slightly improve the semantic of DefaultMemMap::remap()

[cferris]

compiler-rt/lib/scudo/standalone/mem_map.cpp

42

Should you verify that Addr is within the actual map?

47

Similar to above, should you verify that this is within the actual map?

compiler-rt/lib/scudo/standalone/mem_map.h

58

This is slightly confusing. Perhaps reworded something like:

The following functions correspond to the Impl functions from MemMapBase.

compiler-rt/lib/scudo/standalone/mem_map_base.h

19

this

20–21

Do you mean before the operation is executed? The wording is a bit confusing.

23

A class derived from

23

handle contiguous pages

24

derived class uses.

29–30

it has to implement all of the 'Impl' named functions.

35

set of pages

36

any

59

set of physical

86

a set of contiguous pages

87

a set of contiguous pages

98

functions are not used and should not be defined in

99

following

100

that a derived class has to implement.

[Chia-hungDuan]

Address review comment

compiler-rt/lib/scudo/standalone/mem_map.cpp

42

The basic invariant of the API is verified in the base class (mem_map_base.h), For example,

bool remap(uptr Addr, uptr Size, const char *Name, uptr Flags = 0) {              
  DCHECK(isAllocated());                                                          
  DCHECK((Addr >= getMapBase()) ||                                                
         (Addr + Size <= getMapBase() + getMapCapacity()));                       
  return static_cast<Derived *>(this)->remapImpl(Addr, Size, Name, Flags);        
}

47

Same as above.

compiler-rt/lib/scudo/standalone/mem_map_base.h

20–21

Yes, reworded a little bit.

87

The suggestion is a little bit different from above. I follow the suggestion here (suppose the above didn't mark the right range to update)

[Chia-hungDuan]

compiler-rt/lib/scudo/standalone/mem_map_base.h

46

I'm wrapping this ugly cast as Fabio suggested

[Chia-hungDuan]

wrap static_cast<Derived *>(this)->...Impl(...)

[Chia-hungDuan]

Rename ReservedMap to ReservedMemory and untangle it with MemMap

Change the names of some APIs

[fabio-d]

Hi @Chia-hungDuan, thank you for bringing this new design forward!

I've left some notes on specific APIs, and I also have some other general suggestions:

• It would be useful to document what flags are acceptable in a comment next to each method.
• As you know, Fuchsia doesn't accept addresses and sizes that are not page-aligned, so I suggest to add page-alignment checks to all the arguments.
• I am slightly confused by the fact that some arguments are called Size but the corresponding property is called Capacity. Is there any specific reason why you didn't use Size everywhere?

compiler-rt/lib/scudo/standalone/mem_map.h

35–37

Since these methods are only called by the wrappers in MemMapBase and they always pass a Flags value, I suggest to remove the default values here

60–61

Same suggestion as above on the Flags.

compiler-rt/lib/scudo/standalone/mem_map_base.h

23

I assume that the contract is that the ctor creates an object such that !isAllocated(). Should we also have a dtor that automatically unmaps if still isAllocated()?

25–26

If Addr is only a suggestion, I suggest to remove it completely from this API. For carefully-placed memory ranges, clients will use ReservedMemory and dispatch

27–28

Are you referring to Trusty? I would move this comment to unmap

30

Should this also check that !isAllocated() as a precondition?

35

I think we do not need Flags for unmap in this new design. Is there any specific flag that you have in mind? From a quick look, it seems to me that we can safely get rid of the UNMAP_ALL flag if we add explicit alignment control to cover primary32.h's special case.

Also, Fuchsia does not have a syscall to shrink a VMO from the beginning. Would it be a problem if we only allowed to shrink from the end?

41–43

I would separate resizing and changing permissions into different methods (even on Linux, mremap and mprotect are separate syscalls).
Also, I'm not sure we actually need Addr at all:

• If this is being called with the intent of setting new permissions, I would assume that the upper layers would always want to apply them to the entire range covered by the MemMap. Unless we already have use case for this level flexibility, I would suggest to leave it out of the new API and we cann add it back later as needed.
• If this is being called with the intent of resizing, I also would assume that it applies to the entire range too (in fact, I cannot think of a good formal definition of "resizing a subrange", especially if the resized range ends before the end of the MemMap).

50–52

What do "physical" and "virtual" mean in this context? (maybe this a stale comment from the previous revision of this CL)

60

The distinction between releasePagesToOS and releaseAndZeroPagesToOS does not currently exist in Scudo, right? Can we add it as a new feature in a later CL?

87

Similar to above, I assume that the ctor creates an object such that !isCreated(). Should we also have a dtor that automatically releases if still isCreated()?

92

Should this also check that !isCreated() as a precondition?

96

Similarly, I think we do not need Flags for release either.

[Chia-hungDuan]

Address review comment

compiler-rt/lib/scudo/standalone/mem_map_base.h

23

These two are supposed to be lazy initialized so the creation needs to be done explicitly.

The order of destructing allocator fields is tricky. Especially we have the thread local cache. Given that tearing down the allocator only happens at program exit, we just make them being released naturally. Therefore, the creation/deletion are done explicitly.

We have been thinking to make it more elegantly but that depends on too many stuff (e.g., libc). Besides, this CL is supposed to do the abstraction transition only, I prefer to do further logic changes in the later CLs.

25–26

I prefer to keep this suggestion because it can still benefit some contexts (like some cache management stuff we will have in the future). It's a different case than ReservedMemory supposed to cover.

35

Like the release one, the only case I was thinking is the UNMAP_ALL for Fuchsia, if you think it's fine to remove it. I'm happy to do that :)

is there any alternative way to achieve this? Like create a new VMO ?

41–43

The MemMap is a handle of range of virtual address and it's supposed to do the supported operations in the range. So it can remap part of the virtual address. For example, the RegionBeg is unlikely to be equal to Region base address. That's why we have this Addr argument here (For future function it'll be useful as well). If a platform doesn't allow this, then we just do the valid operation in that platform to fulfill their constraint.

BTW, we have setMemoryPermission (which is added in the follow up CL)

50–52

It means virtual memory and physical memory.

60

In general, yes, we may consider adding this in a separate CL.

The reason I do it here is that we are revising API contract and try to reduce ambiguity. For example, the arguments of releasePagesToOS was confusing. And then we found that the zero-page stuff is ambiguous as well. That's why I bring it in this CL.

87

Same as above.

92

This is intended to make it implementation defined behavior. In some case we may want few reserved spaces.

96

I was hesitating this before and the only case is UNMAP_ALL used by Fuchsia. If you agree that we can get rid of it. I'm happy with that :) Done.

[Chia-hungDuan]

compiler-rt/lib/scudo/standalone/mem_map_base.h

92

Oops, please ignore this wrong reply.

[cferris]

LGTM.

[Chia-hungDuan]

Make ReservedMemory::create return boolean to indicate if the creation succeeds.
This makes the page allocation have the consistent style of API

[cferris]

Still looks good.

[fabio-d]

compiler-rt/lib/scudo/standalone/primary64.h

748–760

This used to be a grow operation for a memory region that always started at RegionBeg. Now we are dispatching at getRegionBaseByClassId(ClassId) and then resizing starting from RegionBeg. Can we directly dispatch at RegionBeg instead?
This results in MemMapDefault::releaseAndZeroPagesToOSImpl passing an offset relative to getRegionBaseByClassId(ClassId) instead of RegionBeg to the ::releasePagesToOS function, which makes Scudo release in-use pages on Fuchsia.

[Chia-hungDuan]

compiler-rt/lib/scudo/standalone/primary64.h

748–760

Because it's a region's base, I tended to think it's still part of a region. To make it work before transition to each platform specific implementation. I agree that we can dispatch the reserved memory from RegionBeg and fix it back later

[fabio-d]

Closing because the issue on Fuchsia was fixed in D147792

compiler-rt/lib/scudo/standalone/CMakeLists.txt

set(SCUDO_HEADERS
   flags_parser.h
   flags.h
   fuchsia.h
   internal_defs.h
   linux.h
   list.h
   local_cache.h
   memtag.h
+  mem_map.h
+  mem_map_base.h
   mutex.h
   options.h
   platform.h
   primary32.h
   primary64.h
   quarantine.h
   release.h
   report.h

 set(SCUDO_SOURCES
   checksum.cpp
   common.cpp
   crc32_hw.cpp
   flags_parser.cpp
   flags.cpp
   fuchsia.cpp
   linux.cpp
+  mem_map.cpp
   release.cpp
   report.cpp
   rss_limit_checker.cpp
   string_utils.cpp
   )
 
 # Enable the necessary instruction set for scudo_crc32.cpp, if available.
 # Newer compiler versions use -mcrc32 rather than -msse4.2.

compiler-rt/lib/scudo/standalone/mem_map.h

+//===-- mem_map.h -----------------------------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef SCUDO_MEM_MAP_H_
+#define SCUDO_MEM_MAP_H_
+
+#include "mem_map_base.h"
+
+#include "common.h"
+#include "internal_defs.h"
+
+// TODO: This is only used for `MapPlatformData`. Remove these includes when we
+// have all three platform specific `MemMap` and `ReservedMemory`
+// implementations.
+#include "fuchsia.h"
+#include "linux.h"
+#include "trusty.h"
+
+namespace scudo {
+
+// This will be deprecated when every allocator has been supported by each
+// platform's `MemMap` implementation.
+class MemMapDefault final : public MemMapBase<MemMapDefault> {
+public:
+  constexpr MemMapDefault() = default;
+  MemMapDefault(uptr Base, uptr Capacity) : Base(Base), Capacity(Capacity) {}
+
+  // Impls for base functions.
+  bool mapImpl(uptr Addr, uptr Size, const char *Name, uptr Flags);
+  void unmapImpl(uptr Addr, uptr Size);
+  bool remapImpl(uptr Addr, uptr Size, const char *Name, uptr Flags);
+  void releasePagesToOSImpl(uptr From, uptr Size) {
+    return releaseAndZeroPagesToOSImpl(From, Size);
+  }
+  void releaseAndZeroPagesToOSImpl(uptr From, uptr Size);
+  uptr getBaseImpl() { return Base; }
+  uptr getCapacityImpl() { return Capacity; }
+
+  void setMapPlatformData(MapPlatformData &NewData) { Data = NewData; }
+
+private:
+  uptr Base = 0;
+  uptr Capacity = 0;
+  MapPlatformData Data = {};
+};
+
+// This will be deprecated when every allocator has been supported by each
+// platform's `MemMap` implementation.
+class ReservedMemoryDefault final
+    : public ReservedMemory<ReservedMemoryDefault, MemMapDefault> {
+public:
+  constexpr ReservedMemoryDefault() = default;
+
+  void createImpl(uptr Addr, uptr Size, const char *Name, uptr Flags);
+  void releaseImpl();
+  MemMapT dispatchImpl(uptr Addr, uptr Size);
+  uptr getBaseImpl() { return Base; }
+  uptr getCapacityImpl() { return Capacity; }
+
+private:
+  uptr Base = 0;
+  uptr Capacity = 0;
+  MapPlatformData Data = {};
+};
+
+#if SCUDO_LINUX
+using ReservedMemoryT = ReservedMemoryDefault;
+using MemMapT = ReservedMemoryT::MemMapT;
+#elif SCUDO_FUCHSIA
+using ReservedMemoryT = ReservedMemoryDefault;
+using MemMapT = ReservedMemoryT::MemMapT;
+#elif SCUDO_TRUSTY
+using ReservedMemoryT = ReservedMemoryDefault;
+using MemMapT = ReservedMemoryT::MemMapT;
+#else
+#error                                                                         \
+    "Unsupported platform, please implement the ReservedMemory for your platform!"
+#endif
+
+} // namespace scudo
+
+#endif // SCUDO_MEM_MAP_H_

compiler-rt/lib/scudo/standalone/mem_map.cpp

+//===-- mem_map.cpp ---------------------------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "mem_map.h"
+
+#include "common.h"
+
+namespace scudo {
+
+bool MemMapDefault::mapImpl(uptr Addr, uptr Size, const char *Name,
+                            uptr Flags) {
+  void *MappedAddr =
+      ::scudo::map(reinterpret_cast<void *>(Addr), Size, Name, Flags, &Data);
+  if (MappedAddr == nullptr)
+    return false;
+  Base = reinterpret_cast<uptr>(MappedAddr);
+  Capacity = Size;
+  return true;
+}
+
+void MemMapDefault::unmapImpl(uptr Addr, uptr Size) {
+  if (Size == Capacity) {
+    Base = Capacity = 0;
+  } else {
+    if (Base == Addr)
+      Base = Addr + Size;
+    Capacity -= Size;
+  }
+
+  ::scudo::unmap(reinterpret_cast<void *>(Addr), Size, UNMAP_ALL, &Data);
+}
+
+bool MemMapDefault::remapImpl(uptr Addr, uptr Size, const char *Name,
+                              uptr Flags) {
+  void *RemappedAddr =
+      ::scudo::map(reinterpret_cast<void *>(Addr), Size, Name, Flags, &Data);
+  return reinterpret_cast<uptr>(RemappedAddr) == Addr;
+}
+
+void MemMapDefault::releaseAndZeroPagesToOSImpl(uptr From, uptr Size) {
+  return ::scudo::releasePagesToOS(Base, From - Base, Size, &Data);
+}
+
+void ReservedMemoryDefault::releaseImpl() {
+  ::scudo::unmap(reinterpret_cast<void *>(Base), Capacity, UNMAP_ALL, &Data);
+}
+
+void ReservedMemoryDefault::createImpl(uptr Addr, uptr Size, const char *Name,
+                                       uptr Flags) {
+  void *Reserved = ::scudo::map(reinterpret_cast<void *>(Addr), Size, Name,
+                                Flags | MAP_NOACCESS, &Data);
+  if (Reserved == nullptr)
+    return;
+
+  Base = reinterpret_cast<uptr>(Reserved);
+  Capacity = Size;
+}
+
+ReservedMemoryDefault::MemMapT ReservedMemoryDefault::dispatchImpl(uptr Addr,
+                                                                   uptr Size) {
+  ReservedMemoryDefault::MemMapT NewMap(Addr, Size);
+  NewMap.setMapPlatformData(Data);
+  return NewMap;
+}
+
+} // namespace scudo

compiler-rt/lib/scudo/standalone/mem_map_base.h

+//===-- mem_map_base.h ------------------------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef SCUDO_MEM_MAP_BASE_H_
+#define SCUDO_MEM_MAP_BASE_H_
+
+#include "common.h"
+
+namespace scudo {
+
+// In Scudo, every memory operation will be fulfilled through a
+// platform-specific `MemMap` instance. The essential APIs are listed in the
+// `MemMapBase` below. This is implemented in CRTP, so for each implementation,
+// it has to implement all of the 'Impl' named functions.
+template <class Derived> class MemMapBase {
+public:
+  constexpr MemMapBase() = default;
+
+  // This is used to map a new set of contiguous pages. Note that the `Addr` is
+  // only a suggestion to the system.
+  bool map(uptr Addr, uptr Size, const char *Name, uptr Flags = 0) {
+    DCHECK(!isAllocated());
+    return invokeImpl(&Derived::mapImpl, Addr, Size, Name, Flags);
+  }
+
+  // This is used to unmap partial/full pages from the beginning or the end.
+  // I.e., the result pages are expected to be still contiguous.
+  void unmap(uptr Addr, uptr Size) {
+    DCHECK(isAllocated());
+    DCHECK((Addr == getBase()) || (Addr + Size == getBase() + getCapacity()));
+    invokeImpl(&Derived::unmapImpl, Addr, Size);
+  }
+
+  // This is used to remap a mapped range (either from map() or dispatched from
+  // ReservedMemory). For example, we have reserved several pages and then we
+  // want to remap them with different accessibility.
+  bool remap(uptr Addr, uptr Size, const char *Name, uptr Flags = 0) {
+    DCHECK(isAllocated());
+    DCHECK((Addr >= getBase()) || (Addr + Size <= getBase() + getCapacity()));
+    return invokeImpl(&Derived::remapImpl, Addr, Size, Name, Flags);
+  }
+
+  // Suggest releasing a set of contiguous physical pages back to the OS. Note
+  // that only physical pages are supposed to be released. Any release of
+  // virtual pages may lead to undefined behavior.
+  void releasePagesToOS(uptr From, uptr Size) {
+    DCHECK(isAllocated());
+    DCHECK((From >= getBase()) || (From + Size <= getBase() + getCapacity()));
+    invokeImpl(&Derived::releasePagesToOSImpl, From, Size);
+  }
+  // This is similar to the above one except that any subsequent access to the
+  // released pages will return with zero-filled pages.
+  void releaseAndZeroPagesToOS(uptr From, uptr Size) {
+    DCHECK(isAllocated());
+    DCHECK((From >= getBase()) || (From + Size <= getBase() + getCapacity()));
+    invokeImpl(&Derived::releaseAndZeroPagesToOSImpl, From, Size);
+  }
+
+  uptr getBase() { return invokeImpl(&Derived::getBaseImpl); }
+  uptr getCapacity() { return invokeImpl(&Derived::getCapacityImpl); }
+
+  bool isAllocated() { return getBase() != 0U; }
+
+protected:
+  template <typename R, typename... Args>
+  R invokeImpl(R (Derived::*MemFn)(Args...), Args... args) {
+    return (static_cast<Derived *>(this)->*MemFn)(args...);
+  }
+};
+
+// `ReservedMemory` is a special memory handle which can be viewed as a page
+// allocator. `ReservedMemory` will reserve a contiguous pages and the later
+// page request can be fulfilled at the designated address. This is used when
+// we want to ensure the virtual address of the MemMap will be in a known range.
+// This is implemented in CRTP, so for each
+// implementation, it has to implement all of the 'Impl' named functions.
+template <class Derived, typename MemMapTy> class ReservedMemory {
+public:
+  using MemMapT = MemMapTy;
+  constexpr ReservedMemory() = default;
+
+  // Reserve a chunk of memory.
+  void create(uptr Addr, uptr Size, const char *Name, uptr Flags = 0) {
+    DCHECK(!isCreated());
+    invokeImpl(&Derived::createImpl, Addr, Size, Name, Flags);
+  }
+
+  // Release the entire reserved memory.
+  void release() {
+    DCHECK(isCreated());
+    invokeImpl(&Derived::releaseImpl);
+  }
+
+  // Dispatch a sub-range of reserved memory. Note that any fragmentation of
+  // the reserved pages is managed by each implementation.
+  MemMapT dispatch(uptr Addr, uptr Size) {
+    DCHECK(isCreated());
+    DCHECK((Addr >= getBase()) || (Addr + Size <= getBase() + getCapacity()));
+    return invokeImpl(&Derived::dispatchImpl, Addr, Size);
+  }
+
+  uptr getBase() { return invokeImpl(&Derived::getBaseImpl); }
+  uptr getCapacity() { return invokeImpl(&Derived::getCapacityImpl); }
+
+  bool isCreated() { return getBase() != 0U; }
+
+protected:
+  template <typename R, typename... Args>
+  R invokeImpl(R (Derived::*MemFn)(Args...), Args... args) {
+    return (static_cast<Derived *>(this)->*MemFn)(args...);
+  }
+};
+
+} // namespace scudo
+
+#endif // SCUDO_MEM_MAP_BASE_H_

compiler-rt/lib/scudo/standalone/primary64.h

 //===-- primary64.h ---------------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef SCUDO_PRIMARY64_H_
 #define SCUDO_PRIMARY64_H_
 
 #include "bytemap.h"
 #include "common.h"
 #include "list.h"
 #include "local_cache.h"
+#include "mem_map.h"
 #include "memtag.h"
 #include "options.h"
 #include "release.h"
 #include "stats.h"
 #include "string_utils.h"
 #include "thread_annotations.h"
 
 namespace scudo {
void init(s32 ReleaseToOsInterval) NO_THREAD_SAFETY_ANALYSIS {
     // releaseToOSMaybe(), we require the difference between `PushedBlocks`
     // should be greater than 7 pages. This mitigates the page releasing
     // thrashing which is caused by memory usage bouncing around the threshold.
     // The smallest size class takes longest time to do the page release so we
     // use its size of in-use blocks as a heuristic.
     SmallerBlockReleasePageDelta =
         PagesInGroup * (1 + MinSizeClass / 16U) / 100;
 
-    DCHECK_EQ(PrimaryBase, 0U);
     // Reserve the space required for the Primary.
-    PrimaryBase = reinterpret_cast<uptr>(map(
-        nullptr, PrimarySize, "scudo:primary_reserve", MAP_NOACCESS, &Data));
+    ReservedMemory.create(/*Addr=*/0U, PrimarySize, "scudo:primary_reserve");
+    PrimaryBase = ReservedMemory.getBase();
+    DCHECK_NE(PrimaryBase, 0U);
 
     u32 Seed;
     const u64 Time = getMonotonicTimeFast();
     if (!getRandom(reinterpret_cast<void *>(&Seed), sizeof(Seed)))
       Seed = static_cast<u32>(Time ^ (PrimaryBase >> 12));
 
     for (uptr I = 0; I < NumClasses; I++) {
       RegionInfo *Region = getRegionInfo(I);
public:
   }
 
   void unmapTestOnly() NO_THREAD_SAFETY_ANALYSIS {
     for (uptr I = 0; I < NumClasses; I++) {
       RegionInfo *Region = getRegionInfo(I);
       *Region = {};
     }
     if (PrimaryBase)
-      unmap(reinterpret_cast<void *>(PrimaryBase), PrimarySize, UNMAP_ALL,
-            &Data);
+      ReservedMemory.release();
     PrimaryBase = 0U;
   }
 
   TransferBatch *popBatch(CacheT *C, uptr ClassId) {
     DCHECK_LT(ClassId, NumClasses);
     RegionInfo *Region = getRegionInfo(ClassId);
     bool PrintStats = false;
     {
struct UnpaddedRegionInfo {
     RegionStats Stats GUARDED_BY(Mutex) = {};
     u32 RandState GUARDED_BY(Mutex) = 0;
     // Bytes mapped for user memory.
     uptr MappedUser GUARDED_BY(Mutex) = 0;
     // Bytes allocated for user memory.
     uptr AllocatedUser GUARDED_BY(Mutex) = 0;
     // The minimum size of pushed blocks to trigger page release.
     uptr TryReleaseThreshold GUARDED_BY(Mutex) = 0;
-    MapPlatformData Data GUARDED_BY(Mutex) = {};
+    MemMapT MemMap = {};
     ReleaseToOsInfo ReleaseInfo GUARDED_BY(Mutex) = {};
     bool Exhausted GUARDED_BY(Mutex) = false;
   };
   struct RegionInfo : UnpaddedRegionInfo {
     char Padding[SCUDO_CACHE_LINE_SIZE -
                  (sizeof(UnpaddedRegionInfo) % SCUDO_CACHE_LINE_SIZE)] = {};
   };
   static_assert(sizeof(RegionInfo) % SCUDO_CACHE_LINE_SIZE == 0, "");
 
+  // TODO: `PrimaryBase` can be obtained from ReservedMemory. This needs to be
+  // deprecated.
   uptr PrimaryBase = 0;
+  ReservedMemoryT ReservedMemory = {};
   // The minimum size of pushed blocks that we will try to release the pages in
   // that size class.
   uptr SmallerBlockReleasePageDelta = 0;
-  MapPlatformData Data = {};
   atomic_s32 ReleaseToOsIntervalMs = {};
   alignas(SCUDO_CACHE_LINE_SIZE) RegionInfo RegionInfoArray[NumClasses];
 
   RegionInfo *getRegionInfo(uptr ClassId) {
     DCHECK_LT(ClassId, NumClasses);
     return &RegionInfoArray[ClassId];
   }
 
if (TotalUserBytes > MappedUser) {
       // Do the mmap for the user memory.
       const uptr MapSize =
           roundUp(TotalUserBytes - MappedUser, MapSizeIncrement);
       const uptr RegionBase = RegionBeg - getRegionBaseByClassId(ClassId);
       if (UNLIKELY(RegionBase + MappedUser + MapSize > RegionSize)) {
         Region->Exhausted = true;
         return false;
       }
-      if (MappedUser == 0)
-        Region->Data = Data;
-      if (UNLIKELY(!map(
-              reinterpret_cast<void *>(RegionBeg + MappedUser), MapSize,
-              "scudo:primary",
+      // TODO: Consider allocating MemMap in init().
+      if (!Region->MemMap.isAllocated()) {
+        Region->MemMap = ReservedMemory.dispatch(
+            getRegionBaseByClassId(ClassId), RegionSize);
+      }
+      DCHECK(Region->MemMap.isAllocated());
+
+      if (UNLIKELY(!Region->MemMap.remap(
+              RegionBeg + MappedUser, MapSize, "scudo:primary",
               MAP_ALLOWNOMEM | MAP_RESIZABLE |
-                  (useMemoryTagging<Config>(Options.load()) ? MAP_MEMTAG : 0),
-              &Region->Data))) {
+                  (useMemoryTagging<Config>(Options.load()) ? MAP_MEMTAG
+                                                            : 0)))) {
         return false;
       }
       Region->MappedUser += MapSize;
       C->getStats().add(StatMapped, MapSize);
     }
 
     const u32 NumberOfBlocks = Min(
         MaxNumBatches * MaxCount,
         static_cast<u32>((Region->MappedUser - Region->AllocatedUser) / Size));
     DCHECK_GT(NumberOfBlocks, 0);
NOINLINE uptr releaseToOSMaybe(RegionInfo *Region, uptr ClassId,
     // The last block may straddle the group boundary. Rounding up to BlockSize
     // to get the exact range.
     const uptr ReleaseEnd =
         roundUpSlow(LastGroupEnd - Region->RegionBeg, BlockSize) +
         Region->RegionBeg;
     const uptr ReleaseRangeSize = ReleaseEnd - ReleaseBase;
     const uptr ReleaseOffset = ReleaseBase - Region->RegionBeg;
 
-    ReleaseRecorder Recorder(Region->RegionBeg, ReleaseOffset, &Region->Data);
+    RegionReleaseRecorder<MemMapT> Recorder(&Region->MemMap, Region->RegionBeg,
+                                            ReleaseOffset);
     PageReleaseContext Context(BlockSize, /*NumberOfRegions=*/1U,
                                ReleaseRangeSize, ReleaseOffset);
 
     for (BatchGroup &BG : GroupToRelease) {
       const uptr BatchGroupBase =
           decompactGroupBase(CompactPtrBase, BG.CompactPtrGroupBase);
       const uptr BatchGroupEnd = BatchGroupBase + GroupSize;
       const uptr AllocatedGroupSize = AllocatedUserEnd >= BatchGroupEnd

compiler-rt/lib/scudo/standalone/release.h

 //===-- release.h -----------------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef SCUDO_RELEASE_H_
 #define SCUDO_RELEASE_H_
 
 #include "common.h"
 #include "list.h"
+#include "mem_map.h"
 #include "mutex.h"
 #include "thread_annotations.h"
 
 namespace scudo {
 
+template <typename MemMapT> class RegionReleaseRecorder {
+public:
+  RegionReleaseRecorder(MemMapT *RegionMemMap, uptr Base, uptr Offset = 0)
+      : RegionMemMap(RegionMemMap), Base(Base), Offset(Offset) {}
+
+  uptr getReleasedRangesCount() const { return ReleasedRangesCount; }
+
+  uptr getReleasedBytes() const { return ReleasedBytes; }
+
+  uptr getBase() const { return Base; }
+
+  // Releases [From, To) range of pages back to OS. Note that `From` and `To`
+  // are offseted from `Base` + Offset.
+  void releasePageRangeToOS(uptr From, uptr To) {
+    const uptr Size = To - From;
+    RegionMemMap->releasePagesToOS(getBase() + Offset + From, Size);
+    ReleasedRangesCount++;
+    ReleasedBytes += Size;
+  }
+
+private:
+  uptr ReleasedRangesCount = 0;
+  uptr ReleasedBytes = 0;
+  MemMapT *RegionMemMap = nullptr;
+  uptr Base = 0;
+  // The release offset from Base. This is used when we know a given range after
+  // Base will not be released.
+  uptr Offset = 0;
+};
+
 class ReleaseRecorder {
 public:
   ReleaseRecorder(uptr Base, uptr Offset = 0, MapPlatformData *Data = nullptr)
       : Base(Base), Offset(Offset), Data(Data) {}
 
   uptr getReleasedRangesCount() const { return ReleasedRangesCount; }
 
   uptr getReleasedBytes() const { return ReleasedBytes; }