[scudo] Manage pages with MemMap in SizeClassAllocator64

[Chia-hungDuan]

Summary

Introduce a new data structure to manage the allocated pages from the
system. This is meant to deprecate certain memory system call wrappers
in Scudo, e.g., map()/unmap(). Besides, we would like to make
MapPlatformData to be appeared in platform specific data structure only.
Given that there are several allocators in Scudo and each of them has
different way of page management. The deprecation will be done in
several CLs. In this commit, we start from SizeClassAllocator64.

[Chia-hungDuan]

So far I call it RFC but it's a formal CL as well. There are still many places we can simplify. In order to make the review process easier, I'll do the simplification in coming CLs.

[Chia-hungDuan]

compiler-rt/lib/scudo/standalone/allocator_config.h

104

As @fabio-d suggested, I'll move this out from the config. So that we can still easily mix different configs on different platforms

[Chia-hungDuan]

Don't define MapReserver in allocator config

[Chia-hungDuan]

compiler-rt/lib/scudo/standalone/release.h

50

Note:This will be deprecated as well.

[Chia-hungDuan]

Make getCapacity as required and add few more comments

[Chia-hungDuan]

Add unmap() and remove the use of MapPlatformData in primary64

[Chia-hungDuan]

Remove the defining of MapReserver in the tests

[Chia-hungDuan]

compiler-rt/lib/scudo/standalone/mem_map.h

76

Per discussion, this may not be a good name. It may lead some confusion. I'm considering to make it either ReservedMemory or change it to has-a relation.

[Chia-hungDuan]

1. Rename MapReserver to ReservedMap.
2. Add more comments on why ReservedMap still derives from MemMapBase
3. Leave the unused functions in ReservedMap unimplemented (which were having default implementations which does nothing)

[Chia-hungDuan]

compiler-rt/lib/scudo/standalone/mem_map.h

81

Will be rephrased as

`ReservedMap` is a special MemMap which still manages a contiguous pages

[Chia-hungDuan]

compiler-rt/lib/scudo/standalone/mem_map.h

68

We may consider having two versions of this operation,

1. releaseAndZeroPagesToOS will ensure the later accessing the released pages will get it zero-filled
2. releasePagesToOS is optional to zero-fill

The reason for this is these two operations have performance difference on some platforms and in Scudo, not all releasePagesToOS calls reply on the zero-fill property.

Will update the CL with this change

[Chia-hungDuan]

Add releaseAndZeroPagesToOS and add more DCHECK in the base classes

[Chia-hungDuan]

Move some stuff to mem_map_base.h so that it will make the use of platform MemMap easier.

Also add more comments and checks

[Chia-hungDuan]

Slightly improve the semantic of DefaultMemMap::remap()

[cferris]

compiler-rt/lib/scudo/standalone/mem_map.cpp

42

Should you verify that Addr is within the actual map?

47

Similar to above, should you verify that this is within the actual map?

compiler-rt/lib/scudo/standalone/mem_map.h

58

This is slightly confusing. Perhaps reworded something like:

The following functions correspond to the Impl functions from MemMapBase.

compiler-rt/lib/scudo/standalone/mem_map_base.h

18 ↗

(On Diff #507135)

this

19–20 ↗

(On Diff #507135)

Do you mean before the operation is executed? The wording is a bit confusing.

22 ↗

(On Diff #507135)

A class derived from

22 ↗

(On Diff #507135)

handle contiguous pages

23 ↗

(On Diff #507135)

derived class uses.

28–29 ↗

(On Diff #507135)

it has to implement all of the 'Impl' named functions.

34 ↗

(On Diff #507135)

set of pages

35 ↗

(On Diff #507135)

any

58 ↗

(On Diff #507135)

set of physical

85 ↗

(On Diff #507135)

a set of contiguous pages

86 ↗

(On Diff #507135)

a set of contiguous pages

97 ↗

(On Diff #507135)

functions are not used and should not be defined in

98 ↗

(On Diff #507135)

following

99 ↗

(On Diff #507135)

that a derived class has to implement.

[Chia-hungDuan]

Address review comment

compiler-rt/lib/scudo/standalone/mem_map.cpp

42

The basic invariant of the API is verified in the base class (mem_map_base.h), For example,

bool remap(uptr Addr, uptr Size, const char *Name, uptr Flags = 0) {              
  DCHECK(isAllocated());                                                          
  DCHECK((Addr >= getMapBase()) ||                                                
         (Addr + Size <= getMapBase() + getMapCapacity()));                       
  return static_cast<Derived *>(this)->remapImpl(Addr, Size, Name, Flags);        
}

47

Same as above.

compiler-rt/lib/scudo/standalone/mem_map_base.h

19–20 ↗

(On Diff #507135)

Yes, reworded a little bit.

86 ↗

(On Diff #507135)

The suggestion is a little bit different from above. I follow the suggestion here (suppose the above didn't mark the right range to update)

[Chia-hungDuan]

compiler-rt/lib/scudo/standalone/mem_map_base.h

45 ↗

(On Diff #507215)

I'm wrapping this ugly cast as Fabio suggested

[Chia-hungDuan]

wrap static_cast<Derived *>(this)->...Impl(...)

[Chia-hungDuan]

Rename ReservedMap to ReservedMemory and untangle it with MemMap

Change the names of some APIs

[fabio-d]

Hi @Chia-hungDuan, thank you for bringing this new design forward!

I've left some notes on specific APIs, and I also have some other general suggestions:

• It would be useful to document what flags are acceptable in a comment next to each method.
• As you know, Fuchsia doesn't accept addresses and sizes that are not page-aligned, so I suggest to add page-alignment checks to all the arguments.
• I am slightly confused by the fact that some arguments are called Size but the corresponding property is called Capacity. Is there any specific reason why you didn't use Size everywhere?

compiler-rt/lib/scudo/standalone/mem_map.h

35–37

Since these methods are only called by the wrappers in MemMapBase and they always pass a Flags value, I suggest to remove the default values here

60–61

Same suggestion as above on the Flags.

compiler-rt/lib/scudo/standalone/mem_map_base.h

22 ↗

(On Diff #507936)

I assume that the contract is that the ctor creates an object such that !isAllocated(). Should we also have a dtor that automatically unmaps if still isAllocated()?

24–25 ↗

(On Diff #507936)

If Addr is only a suggestion, I suggest to remove it completely from this API. For carefully-placed memory ranges, clients will use ReservedMemory and dispatch

26–27 ↗

(On Diff #507936)

Are you referring to Trusty? I would move this comment to unmap

29 ↗

(On Diff #507936)

Should this also check that !isAllocated() as a precondition?

34 ↗

(On Diff #507936)

I think we do not need Flags for unmap in this new design. Is there any specific flag that you have in mind? From a quick look, it seems to me that we can safely get rid of the UNMAP_ALL flag if we add explicit alignment control to cover primary32.h's special case.

Also, Fuchsia does not have a syscall to shrink a VMO from the beginning. Would it be a problem if we only allowed to shrink from the end?

40–42 ↗

(On Diff #507936)

I would separate resizing and changing permissions into different methods (even on Linux, mremap and mprotect are separate syscalls).
Also, I'm not sure we actually need Addr at all:

• If this is being called with the intent of setting new permissions, I would assume that the upper layers would always want to apply them to the entire range covered by the MemMap. Unless we already have use case for this level flexibility, I would suggest to leave it out of the new API and we cann add it back later as needed.
• If this is being called with the intent of resizing, I also would assume that it applies to the entire range too (in fact, I cannot think of a good formal definition of "resizing a subrange", especially if the resized range ends before the end of the MemMap).

49–51 ↗

(On Diff #507936)

What do "physical" and "virtual" mean in this context? (maybe this a stale comment from the previous revision of this CL)

59 ↗

(On Diff #507936)

The distinction between releasePagesToOS and releaseAndZeroPagesToOS does not currently exist in Scudo, right? Can we add it as a new feature in a later CL?

86 ↗

(On Diff #507936)

Similar to above, I assume that the ctor creates an object such that !isCreated(). Should we also have a dtor that automatically releases if still isCreated()?

91 ↗

(On Diff #507936)

Should this also check that !isCreated() as a precondition?

95 ↗

(On Diff #507936)

Similarly, I think we do not need Flags for release either.

[Chia-hungDuan]

Address review comment

compiler-rt/lib/scudo/standalone/mem_map_base.h

22 ↗

(On Diff #507936)

These two are supposed to be lazy initialized so the creation needs to be done explicitly.

The order of destructing allocator fields is tricky. Especially we have the thread local cache. Given that tearing down the allocator only happens at program exit, we just make them being released naturally. Therefore, the creation/deletion are done explicitly.

We have been thinking to make it more elegantly but that depends on too many stuff (e.g., libc). Besides, this CL is supposed to do the abstraction transition only, I prefer to do further logic changes in the later CLs.

24–25 ↗

(On Diff #507936)

I prefer to keep this suggestion because it can still benefit some contexts (like some cache management stuff we will have in the future). It's a different case than ReservedMemory supposed to cover.

34 ↗

(On Diff #507936)

Like the release one, the only case I was thinking is the UNMAP_ALL for Fuchsia, if you think it's fine to remove it. I'm happy to do that :)

is there any alternative way to achieve this? Like create a new VMO ?

40–42 ↗

(On Diff #507936)

The MemMap is a handle of range of virtual address and it's supposed to do the supported operations in the range. So it can remap part of the virtual address. For example, the RegionBeg is unlikely to be equal to Region base address. That's why we have this Addr argument here (For future function it'll be useful as well). If a platform doesn't allow this, then we just do the valid operation in that platform to fulfill their constraint.

BTW, we have setMemoryPermission (which is added in the follow up CL)

49–51 ↗

(On Diff #507936)

It means virtual memory and physical memory.

59 ↗

(On Diff #507936)

In general, yes, we may consider adding this in a separate CL.

The reason I do it here is that we are revising API contract and try to reduce ambiguity. For example, the arguments of releasePagesToOS was confusing. And then we found that the zero-page stuff is ambiguous as well. That's why I bring it in this CL.

86 ↗

(On Diff #507936)

Same as above.

91 ↗

(On Diff #507936)

This is intended to make it implementation defined behavior. In some case we may want few reserved spaces.

95 ↗

(On Diff #507936)

I was hesitating this before and the only case is UNMAP_ALL used by Fuchsia. If you agree that we can get rid of it. I'm happy with that :) Done.

[Chia-hungDuan]

compiler-rt/lib/scudo/standalone/mem_map_base.h

91 ↗

(On Diff #507936)

Oops, please ignore this wrong reply.

[cferris]

LGTM.

[Chia-hungDuan]

Make ReservedMemory::create return boolean to indicate if the creation succeeds.
This makes the page allocation have the consistent style of API

[cferris]

Still looks good.

[fabio-d]

compiler-rt/lib/scudo/standalone/primary64.h

741–753

This used to be a grow operation for a memory region that always started at RegionBeg. Now we are dispatching at getRegionBaseByClassId(ClassId) and then resizing starting from RegionBeg. Can we directly dispatch at RegionBeg instead?
This results in MemMapDefault::releaseAndZeroPagesToOSImpl passing an offset relative to getRegionBaseByClassId(ClassId) instead of RegionBeg to the ::releasePagesToOS function, which makes Scudo release in-use pages on Fuchsia.

[Chia-hungDuan]

compiler-rt/lib/scudo/standalone/primary64.h

741–753

Because it's a region's base, I tended to think it's still part of a region. To make it work before transition to each platform specific implementation. I agree that we can dispatch the reserved memory from RegionBeg and fix it back later

[fabio-d]

Closing because the issue on Fuchsia was fixed in D147792

compiler-rt/lib/scudo/standalone/CMakeLists.txt

set(SCUDO_HEADERS
   flags_parser.h
   flags.h
   fuchsia.h
   internal_defs.h
   linux.h
   list.h
   local_cache.h
   memtag.h
+  mem_map.h
   mutex.h
   options.h
   platform.h
   primary32.h
   primary64.h
   quarantine.h
   release.h
   report.h

 set(SCUDO_SOURCES
   checksum.cpp
   common.cpp
   crc32_hw.cpp
   flags_parser.cpp
   flags.cpp
   fuchsia.cpp
   linux.cpp
+  mem_map.cpp
   release.cpp
   report.cpp
   rss_limit_checker.cpp
   string_utils.cpp
   )
 
 # Enable the necessary instruction set for scudo_crc32.cpp, if available.
 # Newer compiler versions use -mcrc32 rather than -msse4.2.

compiler-rt/lib/scudo/standalone/allocator_config.h

 //===-- allocator_config.h --------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef SCUDO_ALLOCATOR_CONFIG_H_
 #define SCUDO_ALLOCATOR_CONFIG_H_
 
 #include "combined.h"
 #include "common.h"
 #include "flags.h"
+#include "mem_map.h"
 #include "primary32.h"
 #include "primary64.h"
 #include "secondary.h"
 #include "size_class_map.h"
 #include "tsd_exclusive.h"
 #include "tsd_shared.h"
 
 namespace scudo {

 //   // Thread-Specific Data Registry used, shared or exclusive.
 //   template <class A> using TSDRegistryT = TSDRegistrySharedT<A, 8U, 4U>;
 // };
 
 // Default configurations for various platforms.
 
 struct DefaultConfig {
   using SizeClassMap = DefaultSizeClassMap;
+  using MapReserver = DefaultMapReserver;
   static const bool MaySupportMemoryTagging = true;
 
 #if SCUDO_CAN_USE_PRIMARY64
   typedef SizeClassAllocator64<DefaultConfig> Primary;
   static const uptr PrimaryRegionSizeLog = 32U;
   static const uptr PrimaryGroupSizeLog = 21U;
   typedef uptr PrimaryCompactPtrT;
   static const uptr PrimaryCompactPtrScale = 0;
#endif
   static const uptr SecondaryCacheDefaultMaxEntrySize = 1UL << 19;
   static const s32 SecondaryCacheMinReleaseToOsIntervalMs = INT32_MIN;
   static const s32 SecondaryCacheMaxReleaseToOsIntervalMs = INT32_MAX;
 
   template <class A> using TSDRegistryT = TSDRegistryExT<A>; // Exclusive
 };
 struct AndroidConfig {
   using SizeClassMap = AndroidSizeClassMap;
+  using MapReserver = DefaultMapReserver;
   static const bool MaySupportMemoryTagging = true;
 
 #if SCUDO_CAN_USE_PRIMARY64
   typedef SizeClassAllocator64<AndroidConfig> Primary;
   static const uptr PrimaryRegionSizeLog = 28U;
   typedef u32 PrimaryCompactPtrT;
   static const uptr PrimaryCompactPtrScale = SCUDO_MIN_ALIGNMENT_LOG;
   static const uptr PrimaryGroupSizeLog = 20U;
#endif
   static const s32 SecondaryCacheMaxReleaseToOsIntervalMs = 1000;
 
   template <class A>
   using TSDRegistryT = TSDRegistrySharedT<A, 8U, 2U>; // Shared, max 8 TSDs.
 };
 
 struct AndroidSvelteConfig {
   using SizeClassMap = SvelteSizeClassMap;
+  using MapReserver = DefaultMapReserver;
   static const bool MaySupportMemoryTagging = false;
 
 #if SCUDO_CAN_USE_PRIMARY64
   typedef SizeClassAllocator64<AndroidSvelteConfig> Primary;
   static const uptr PrimaryRegionSizeLog = 27U;
   typedef u32 PrimaryCompactPtrT;
   static const uptr PrimaryCompactPtrScale = SCUDO_MIN_ALIGNMENT_LOG;
   static const uptr PrimaryGroupSizeLog = 18U;
#endif
 
   template <class A>
   using TSDRegistryT = TSDRegistrySharedT<A, 2U, 1U>; // Shared, max 2 TSDs.
 };
 
 #if SCUDO_CAN_USE_PRIMARY64
 struct FuchsiaConfig {
   using SizeClassMap = FuchsiaSizeClassMap;
+  using MapReserver = DefaultMapReserver;
   static const bool MaySupportMemoryTagging = false;
 
   typedef SizeClassAllocator64<FuchsiaConfig> Primary;
   static const uptr PrimaryRegionSizeLog = 30U;
   static const uptr PrimaryGroupSizeLog = 21U;
   typedef u32 PrimaryCompactPtrT;
   static const bool PrimaryEnableRandomOffset = true;
   static const uptr PrimaryMapSizeIncrement = 1UL << 18;
   static const uptr PrimaryCompactPtrScale = SCUDO_MIN_ALIGNMENT_LOG;
   static const s32 PrimaryMinReleaseToOsIntervalMs = INT32_MIN;
   static const s32 PrimaryMaxReleaseToOsIntervalMs = INT32_MAX;
 
   typedef MapAllocatorNoCache SecondaryCache;
   template <class A>
   using TSDRegistryT = TSDRegistrySharedT<A, 8U, 4U>; // Shared, max 8 TSDs.
 };
 
 struct TrustyConfig {
   using SizeClassMap = TrustySizeClassMap;
+  using MapReserver = DefaultMapReserver;
   static const bool MaySupportMemoryTagging = false;
 
   typedef SizeClassAllocator64<TrustyConfig> Primary;
   // Some apps have 1 page of heap total so small regions are necessary.
   static const uptr PrimaryRegionSizeLog = 10U;
   static const uptr PrimaryGroupSizeLog = 10U;
   typedef u32 PrimaryCompactPtrT;
   static const bool PrimaryEnableRandomOffset = false;

compiler-rt/lib/scudo/standalone/mem_map.h

+//===-- mem_map.h -----------------------------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef SCUDO_MEM_MAP_H_
+#define SCUDO_MEM_MAP_H_
+
+#include "common.h"
+#include "internal_defs.h"
+
+// TODO: This is only used for `MapPlatformData`. Remove these includes when we
+// have all three platform specific `MemMap` and `MapReserver` implementations.
+#include "fuchsia.h"
+#include "linux.h"
+#include "trusty.h"
+
+namespace scudo {
+
+// In Scudo, each page request from the system will be tracked by a
+// platform-specific `MemMap` implementation. For example, the `DefaultMemMap`
+// below records the base of the contiguous pages and its size. With these
+// information, we are able to validate various page operations before it goes
+// beyond the first place.
+
+// This class contains the required interfaces for a MemMap implementation.
+template <class Derived> class MemMapBase {
+public:
+  constexpr MemMapBase() {}
+
+  // This is used to map a new contiguous of pages. Whether the holding pages
+  // (if nay) will be released is implementation defined.
+  bool map(uptr Addr, uptr Size, const char *Name, uptr Flags = 0) {
+    return static_cast<Derived *>(this)->mapImpl(Addr, Size, Name, Flags);
+  }
+
+  // This is used to remap a mapped range. For example, we have reserved several
+  // pages and then we want to remap them with different flags. The validation
+  // of the remapping range is done by each implementation.
+  bool remap(uptr Addr, uptr Size, const char *Name, uptr Flags = 0) {
+    return static_cast<Derived *>(this)->remapImpl(Addr, Size, Name, Flags);
+  }
+
+  // Release a contiguous physical pages back to the OS. Note that only physical
+  // pages can be released. Any releasing of virtual pages may lead to undefined
+  // behavior.
+  void releasePagesToOS(uptr From, uptr Size) {
+    return static_cast<Derived *>(this)->releasePagesToOSImpl(From, Size);
+  }
+
+  uptr getMapBase() { return static_cast<Derived *>(this)->getMapBaseImpl(); }
+
+  bool isAllocated() { return static_cast<Derived *>(this)->isAllocatedImpl(); }
+};
+
+// `MapReserver` reserves a contiguous pages for later dispatch. This ensures
+// the memory map address is in a known range.
+template <class Derived, typename GeneralMapT>
+class MapReserver : public MemMapBase<Derived> {
+public:
+  using Base = MemMapBase<Derived>;
+  using MemMapT = GeneralMapT;
+
+  using Base::getMapBase;
+  using Base::isAllocated;
+
+  // Reserve a chunk of memory. Note that if there's already a reserved range,
+  // the behavior is undefined.
+  void reserveMemory(uptr Addr, uptr Size, const char *Name) {
+    static_cast<Derived *>(this)->reserveMemoryImpl(Addr, Size, Name);
+  }
+
+  // Prepare a MemMap from the reserved range. Note that any fragmentation of
+  // the reserved pages is managed by each implementation.
+  MemMapT extractMemory(uptr Addr, uptr Size) {
+    return static_cast<Derived *>(this)->extractMemoryImpl(Addr, Size);
+  }
+};
+
+// This will be deprecated when every allocator has been supported by each
+// platform's `MemMap` implementation.
+class DefaultMemMap : public MemMapBase<DefaultMemMap> {
+public:
+  DefaultMemMap() = default;
+  DefaultMemMap(uptr Base, uptr Capacity)
+      : MapBase(Base), MapCapacity(Capacity) {}
+
+  bool mapImpl(uptr Addr, uptr Size, const char *Name, uptr Flags = 0);
+  bool remapImpl(uptr Addr, uptr Size, const char *Name, uptr Flags = 0);
+  void releasePagesToOSImpl(uptr From, uptr Size);
+
+  void setMapPlatformData(MapPlatformData &NewData) { Data = NewData; }
+
+  uptr getMapBaseImpl() { return MapBase; }
+  uptr getCapacity() { return MapCapacity; }
+
+  bool isAllocatedImpl() { return MapBase != 0; }
+
+private:
+  uptr MapBase = 0;
+  uptr MapCapacity = 0;
+  MapPlatformData Data = {};
+};
+
+// This will be deprecated when every allocator has been supported by each
+// platform's `MemMap` implementation.
+class DefaultMapReserver
+    : public MapReserver<DefaultMapReserver, DefaultMemMap> {
+public:
+  void reserveMemoryImpl(uptr Addr, uptr Size, const char *Name);
+
+  MemMapT extractMemoryImpl(uptr Addr, uptr Size);
+
+  uptr getMapBaseImpl() { return MapBase; }
+
+  bool isAllocatedImpl() { return MapBase != 0; }
+
+private:
+  uptr MapBase = 0;
+  uptr MapCapacity = 0;
+  MapPlatformData Data = {};
+};
+
+} // namespace scudo
+
+#endif // SCUDO_MEM_MAP_H_

compiler-rt/lib/scudo/standalone/mem_map.cpp

+//===-- mem_map.cpp ---------------------------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "mem_map.h"
+
+#include "common.h"
+
+namespace scudo {
+
+bool DefaultMemMap::mapImpl(uptr Addr, uptr Size, const char *Name,
+                            uptr Flags) {
+  return ::scudo::map(reinterpret_cast<void *>(Addr), Size, Name, Flags, &Data);
+}
+
+bool DefaultMemMap::remapImpl(uptr Addr, uptr Size, const char *Name,
+                              uptr Flags) {
+  DCHECK_GE(Addr, MapBase);
+  DCHECK_LE(Addr + Size, MapBase + MapCapacity);
+  return ::scudo::map(reinterpret_cast<void *>(Addr), Size, Name, Flags, &Data);
+}
+
+void DefaultMemMap::releasePagesToOSImpl(uptr From, uptr Size) {
+  DCHECK_GE(From, MapBase);
+  DCHECK_LE(From + Size, MapBase + MapCapacity);
+  return ::scudo::releasePagesToOS(MapBase, From - MapBase, Size, &Data);
+}
+
+void DefaultMapReserver::reserveMemoryImpl(uptr Addr, uptr Size,
+                                           const char *Name) {
+  void *Reserved = ::scudo::map(reinterpret_cast<void *>(Addr), Size, Name,
+                                MAP_NOACCESS, &Data);
+  if (Reserved == nullptr)
+    return;
+
+  MapBase = reinterpret_cast<uptr>(Reserved);
+  MapCapacity = Size;
+}
+
+DefaultMapReserver::MemMapT DefaultMapReserver::extractMemoryImpl(uptr Addr,
+                                                                  uptr Size) {
+  DCHECK(isAllocated());
+  DCHECK_GE(Addr, MapBase);
+  DCHECK_LE(Addr + Size, MapBase + MapCapacity);
+  DefaultMapReserver::MemMapT NewMap(Addr, Size);
+  NewMap.setMapPlatformData(Data);
+  return NewMap;
+}
+
+} // namespace scudo

compiler-rt/lib/scudo/standalone/primary64.h

 //===-- primary64.h ---------------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef SCUDO_PRIMARY64_H_
 #define SCUDO_PRIMARY64_H_
 
 #include "bytemap.h"
 #include "common.h"
 #include "list.h"
 #include "local_cache.h"
+#include "mem_map.h"
 #include "memtag.h"
 #include "options.h"
 #include "release.h"
 #include "stats.h"
 #include "string_utils.h"
 #include "thread_annotations.h"
 
 namespace scudo {

 // freelist to the thread specific freelist, and back.
 //
 // The memory used by this allocator is never unmapped, but can be partially
 // released if the platform allows for it.
 
 template <typename Config> class SizeClassAllocator64 {
 public:
   typedef typename Config::PrimaryCompactPtrT CompactPtrT;
+  typedef typename Config::MapReserver MapReserverT;
+  typedef typename MapReserverT::MemMapT MemMapT;
   static const uptr CompactPtrScale = Config::PrimaryCompactPtrScale;
   static const uptr GroupSizeLog = Config::PrimaryGroupSizeLog;
   static const uptr GroupScale = GroupSizeLog - CompactPtrScale;
   typedef typename Config::SizeClassMap SizeClassMap;
   typedef SizeClassAllocator64<Config> ThisT;
   typedef SizeClassAllocatorLocalCache<ThisT> CacheT;
   typedef typename CacheT::TransferBatch TransferBatch;
   typedef typename CacheT::BatchGroup BatchGroup;
 
   static uptr getSizeByClassId(uptr ClassId) {
     return (ClassId == SizeClassMap::BatchClassId)
                ? roundUp(sizeof(TransferBatch), 1U << CompactPtrScale)
                : SizeClassMap::getSizeByClassId(ClassId);
   }
 
   static bool canAllocate(uptr Size) { return Size <= SizeClassMap::MaxSize; }
 
   void init(s32 ReleaseToOsInterval) NO_THREAD_SAFETY_ANALYSIS {
     DCHECK(isAligned(reinterpret_cast<uptr>(this), alignof(ThisT)));
     DCHECK_EQ(PrimaryBase, 0U);
 
     // Reserve the space required for the Primary.
-    PrimaryBase = reinterpret_cast<uptr>(map(
-        nullptr, PrimarySize, "scudo:primary_reserve", MAP_NOACCESS, &Data));
+    PrimaryMaps.reserveMemory(/*Addr=*/0U, PrimarySize,
+                              "scudo:primary_reserve");
+    PrimaryBase = PrimaryMaps.getMapBase();
+    DCHECK_NE(PrimaryBase, 0U);
 
     u32 Seed;
     const u64 Time = getMonotonicTimeFast();
     if (!getRandom(reinterpret_cast<void *>(&Seed), sizeof(Seed)))
       Seed = static_cast<u32>(Time ^ (PrimaryBase >> 12));
     const uptr PageSize = getPageSizeCached();
     for (uptr I = 0; I < NumClasses; I++) {
       RegionInfo *Region = getRegionInfo(I);
struct UnpaddedRegionInfo {
     uptr RegionBeg = 0;
     RegionStats Stats GUARDED_BY(Mutex) = {};
     u32 RandState GUARDED_BY(Mutex) = 0;
     // Bytes mapped for user memory.
     uptr MappedUser GUARDED_BY(Mutex) = 0;
     // Bytes allocated for user memory.
     uptr AllocatedUser GUARDED_BY(Mutex) = 0;
     MapPlatformData Data GUARDED_BY(Mutex) = {};
+    MemMapT MemMap = {};
     ReleaseToOsInfo ReleaseInfo GUARDED_BY(Mutex) = {};
     bool Exhausted GUARDED_BY(Mutex) = false;
   };
   struct RegionInfo : UnpaddedRegionInfo {
     char Padding[SCUDO_CACHE_LINE_SIZE -
                  (sizeof(UnpaddedRegionInfo) % SCUDO_CACHE_LINE_SIZE)] = {};
   };
   static_assert(sizeof(RegionInfo) % SCUDO_CACHE_LINE_SIZE == 0, "");
 
+  // TODO: `PrimaryBase` can be obtained from PrimaryMaps. This needs to be
+  // deprecated.
   uptr PrimaryBase = 0;
+  MapReserverT PrimaryMaps = {};
   // The minimum size of pushed blocks that we will try to release the pages in
   // that size class.
   uptr SmallerBlockReleasePageDelta = 0;
   MapPlatformData Data = {};
   atomic_s32 ReleaseToOsIntervalMs = {};
   alignas(SCUDO_CACHE_LINE_SIZE) RegionInfo RegionInfoArray[NumClasses];
 
   RegionInfo *getRegionInfo(uptr ClassId) {
if (TotalUserBytes > MappedUser) {
       // Do the mmap for the user memory.
       const uptr MapSize =
           roundUp(TotalUserBytes - MappedUser, MapSizeIncrement);
       const uptr RegionBase = RegionBeg - getRegionBaseByClassId(ClassId);
       if (UNLIKELY(RegionBase + MappedUser + MapSize > RegionSize)) {
         Region->Exhausted = true;
         return false;
       }
-      if (MappedUser == 0)
-        Region->Data = Data;
-      if (UNLIKELY(!map(
-              reinterpret_cast<void *>(RegionBeg + MappedUser), MapSize,
-              "scudo:primary",
+      // TODO: Consider allocating MemMap in init().
+      if (!Region->MemMap.isAllocated()) {
+        Region->MemMap = PrimaryMaps.extractMemory(
+            getRegionBaseByClassId(ClassId), RegionSize);
+      }
+      DCHECK(Region->MemMap.isAllocated());
+
+      if (UNLIKELY(!Region->MemMap.remap(
+              RegionBeg + MappedUser, MapSize, "scudo:primary",
               MAP_ALLOWNOMEM | MAP_RESIZABLE |
-                  (useMemoryTagging<Config>(Options.load()) ? MAP_MEMTAG : 0),
-              &Region->Data))) {
+                  (useMemoryTagging<Config>(Options.load()) ? MAP_MEMTAG
+                                                            : 0)))) {
         return false;
       }
       Region->MappedUser += MapSize;
       C->getStats().add(StatMapped, MapSize);
     }
 
     const u32 NumberOfBlocks = Min(
         MaxNumBatches * MaxCount,
         static_cast<u32>((Region->MappedUser - Region->AllocatedUser) / Size));
     DCHECK_GT(NumberOfBlocks, 0);
NOINLINE uptr releaseToOSMaybe(RegionInfo *Region, uptr ClassId,
     // The last block may straddle the group boundary. Rounding up to BlockSize
     // to get the exact range.
     const uptr ReleaseEnd =
         roundUpSlow(LastGroupEnd - Region->RegionBeg, BlockSize) +
         Region->RegionBeg;
     const uptr ReleaseRangeSize = ReleaseEnd - ReleaseBase;
     const uptr ReleaseOffset = ReleaseBase - Region->RegionBeg;
 
-    ReleaseRecorder Recorder(Region->RegionBeg, ReleaseOffset, &Region->Data);
+    RegionReleaseRecorder<MemMapT> Recorder(&Region->MemMap, Region->RegionBeg,
+                                            ReleaseOffset);
     PageReleaseContext Context(BlockSize, /*NumberOfRegions=*/1U,
                                ReleaseRangeSize, ReleaseOffset);
 
     for (BatchGroup &BG : GroupToRelease) {
       BG.PushedBlocksAtLastCheckpoint = BG.PushedBlocks;
 
       const uptr BatchGroupBase =
           decompactGroupBase(CompactPtrBase, BG.CompactPtrGroupBase);

compiler-rt/lib/scudo/standalone/release.h

 //===-- release.h -----------------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef SCUDO_RELEASE_H_
 #define SCUDO_RELEASE_H_
 
 #include "common.h"
 #include "list.h"
+#include "mem_map.h"
 #include "mutex.h"
 #include "thread_annotations.h"
 
 namespace scudo {
 
+template <typename MemMapT> class RegionReleaseRecorder {
+public:
+  RegionReleaseRecorder(MemMapT *RegionMemMap, uptr Base, uptr Offset = 0)
+      : RegionMemMap(RegionMemMap), Base(Base), Offset(Offset) {}
+
+  uptr getReleasedRangesCount() const { return ReleasedRangesCount; }
+
+  uptr getReleasedBytes() const { return ReleasedBytes; }
+
+  uptr getBase() const { return Base; }
+
+  // Releases [From, To) range of pages back to OS. Note that `From` and `To`
+  // are offseted from `Base` + Offset.
+  void releasePageRangeToOS(uptr From, uptr To) {
+    const uptr Size = To - From;
+    RegionMemMap->releasePagesToOS(getBase() + Offset + From, Size);
+    ReleasedRangesCount++;
+    ReleasedBytes += Size;
+  }
+
+private:
+  uptr ReleasedRangesCount = 0;
+  uptr ReleasedBytes = 0;
+  MemMapT *RegionMemMap = nullptr;
+  uptr Base = 0;
+  // The release offset from Base. This is used when we know a given range after
+  // Base will not be released.
+  uptr Offset = 0;
+};
+
 class ReleaseRecorder {
 public:
   ReleaseRecorder(uptr Base, uptr Offset = 0, MapPlatformData *Data = nullptr)
       : Base(Base), Offset(Offset), Data(Data) {}
 
   uptr getReleasedRangesCount() const { return ReleasedRangesCount; }
 
   uptr getReleasedBytes() const { return ReleasedBytes; }
 

compiler-rt/lib/scudo/standalone/tests/combined_test.cpp

 };
 
 static const scudo::uptr DeathRegionSizeLog = 21U;
 struct DeathConfig {
   static const bool MaySupportMemoryTagging = false;
 
   // Tiny allocator, its Primary only serves chunks of four sizes.
   using SizeClassMap = scudo::FixedSizeClassMap<DeathSizeClassConfig>;
+  using MapReserver = scudo::DefaultMapReserver;
   typedef scudo::SizeClassAllocator64<DeathConfig> Primary;
   static const scudo::uptr PrimaryRegionSizeLog = DeathRegionSizeLog;
   static const scudo::s32 PrimaryMinReleaseToOsIntervalMs = INT32_MIN;
   static const scudo::s32 PrimaryMaxReleaseToOsIntervalMs = INT32_MAX;
   typedef scudo::uptr PrimaryCompactPtrT;
   static const scudo::uptr PrimaryCompactPtrScale = 0;
   static const bool PrimaryEnableRandomOffset = true;
   static const scudo::uptr PrimaryMapSizeIncrement = 1UL << 18;

compiler-rt/lib/scudo/standalone/tests/primary_test.cpp

 //===-- primary_test.cpp ----------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "tests/scudo_unit_test.h"
 
+#include "mem_map.h"
 #include "primary32.h"
 #include "primary64.h"
 #include "size_class_map.h"
 
 #include <algorithm>
 #include <chrono>
 #include <condition_variable>
 #include <mutex>
#endif
   typedef scudo::u32 PrimaryCompactPtrT;
   static const bool PrimaryEnableRandomOffset = true;
   static const scudo::uptr PrimaryMapSizeIncrement = 1UL << 18;
 };
 
 template <typename BaseConfig, typename SizeClassMapT>
 struct Config : public BaseConfig {
   using SizeClassMap = SizeClassMapT;
+  using MapReserver = scudo::DefaultMapReserver;
 };
 
 template <typename BaseConfig, typename SizeClassMapT>
 struct SizeClassAllocator
     : public scudo::SizeClassAllocator64<Config<BaseConfig, SizeClassMapT>> {};
 template <typename SizeClassMapT>
 struct SizeClassAllocator<TestConfig1, SizeClassMapT>
     : public scudo::SizeClassAllocator32<Config<TestConfig1, SizeClassMapT>> {};
SCUDO_TYPED_TEST(ScudoPrimaryTest, BasicPrimary) {
   Allocator->releaseToOS();
   scudo::ScopedString Str;
   Allocator->getStats(&Str);
   Str.output();
 }
 
 struct SmallRegionsConfig {
   using SizeClassMap = scudo::DefaultSizeClassMap;
+  using MapReserver = scudo::DefaultMapReserver;
   static const scudo::uptr PrimaryRegionSizeLog = 21U;
   static const scudo::s32 PrimaryMinReleaseToOsIntervalMs = INT32_MIN;
   static const scudo::s32 PrimaryMaxReleaseToOsIntervalMs = INT32_MAX;
   static const bool MaySupportMemoryTagging = false;
   typedef scudo::uptr PrimaryCompactPtrT;
   static const scudo::uptr PrimaryCompactPtrScale = 0;
   static const bool PrimaryEnableRandomOffset = true;
   static const scudo::uptr PrimaryMapSizeIncrement = 1UL << 18;