Page MenuHomePhabricator

[scudo] Manage pages with MemMap in SizeClassAllocator64

Authored by Chia-hungDuan on Mar 13 2023, 10:30 PM.



Introduce a new data structure to manage the allocated pages from the
system. This is meant to deprecate certain memory system call wrappers
in Scudo, e.g., map()/unmap(). Besides, we would like to make
MapPlatformData to be appeared in platform specific data structure only.
Given that there are several allocators in Scudo and each of them has
different way of page management. The deprecation will be done in
several CLs. In this commit, we start from SizeClassAllocator64.

Diff Detail

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Chia-hungDuan added inline comments.Mar 15 2023, 11:34 AM

Per discussion, this may not be a good name. It may lead some confusion. I'm considering to make it either ReservedMemory or change it to has-a relation.

  1. Rename MapReserver to ReservedMap.
  2. Add more comments on why ReservedMap still derives from MemMapBase
  3. Leave the unused functions in ReservedMap unimplemented (which were having default implementations which does nothing)
Chia-hungDuan added inline comments.Mar 15 2023, 5:09 PM

Will be rephrased as

`ReservedMap` is a special MemMap which still manages a contiguous pages
Chia-hungDuan added inline comments.Mar 16 2023, 1:17 PM

We may consider having two versions of this operation,

  1. releaseAndZeroPagesToOS will ensure the later accessing the released pages will get it zero-filled
  2. releasePagesToOS is optional to zero-fill

The reason for this is these two operations have performance difference on some platforms and in Scudo, not all releasePagesToOS calls reply on the zero-fill property.

Will update the CL with this change

Add releaseAndZeroPagesToOS and add more DCHECK in the base classes

Move some stuff to mem_map_base.h so that it will make the use of platform MemMap easier.

Also add more comments and checks

Slightly improve the semantic of DefaultMemMap::remap()

cferris added inline comments.Mar 21 2023, 8:01 PM

Should you verify that Addr is within the actual map?


Similar to above, should you verify that this is within the actual map?


This is slightly confusing. Perhaps reworded something like:

The following functions correspond to the Impl functions from MemMapBase.




Do you mean before the operation is executed? The wording is a bit confusing.


A class derived from


handle contiguous pages


derived class uses.


it has to implement all of the 'Impl' named functions.


set of pages




set of physical


a set of contiguous pages


a set of contiguous pages


functions are not used and should not be defined in




that a derived class has to implement.

Chia-hungDuan marked 16 inline comments as done.

Address review comment


The basic invariant of the API is verified in the base class (mem_map_base.h), For example,

bool remap(uptr Addr, uptr Size, const char *Name, uptr Flags = 0) {              
  DCHECK((Addr >= getMapBase()) ||                                                
         (Addr + Size <= getMapBase() + getMapCapacity()));                       
  return static_cast<Derived *>(this)->remapImpl(Addr, Size, Name, Flags);        

Same as above.


Yes, reworded a little bit.


The suggestion is a little bit different from above. I follow the suggestion here (suppose the above didn't mark the right range to update)

Chia-hungDuan added inline comments.Mar 22 2023, 3:17 PM

I'm wrapping this ugly cast as Fabio suggested

wrap static_cast<Derived *>(this)->...Impl(...)

Rename ReservedMap to ReservedMemory and untangle it with MemMap

Change the names of some APIs

Hi @Chia-hungDuan, thank you for bringing this new design forward!

I've left some notes on specific APIs, and I also have some other general suggestions:

  • It would be useful to document what flags are acceptable in a comment next to each method.
  • As you know, Fuchsia doesn't accept addresses and sizes that are not page-aligned, so I suggest to add page-alignment checks to all the arguments.
  • I am slightly confused by the fact that some arguments are called Size but the corresponding property is called Capacity. Is there any specific reason why you didn't use Size everywhere?

Since these methods are only called by the wrappers in MemMapBase and they always pass a Flags value, I suggest to remove the default values here


Same suggestion as above on the Flags.


I assume that the contract is that the ctor creates an object such that !isAllocated(). Should we also have a dtor that automatically unmaps if still isAllocated()?


If Addr is only a suggestion, I suggest to remove it completely from this API. For carefully-placed memory ranges, clients will use ReservedMemory and dispatch


Are you referring to Trusty? I would move this comment to unmap


Should this also check that !isAllocated() as a precondition?


I think we do not need Flags for unmap in this new design. Is there any specific flag that you have in mind? From a quick look, it seems to me that we can safely get rid of the UNMAP_ALL flag if we add explicit alignment control to cover primary32.h's special case.

Also, Fuchsia does not have a syscall to shrink a VMO from the beginning. Would it be a problem if we only allowed to shrink from the end?


I would separate resizing and changing permissions into different methods (even on Linux, mremap and mprotect are separate syscalls).
Also, I'm not sure we actually need Addr at all:

  • If this is being called with the intent of setting new permissions, I would assume that the upper layers would always want to apply them to the entire range covered by the MemMap. Unless we already have use case for this level flexibility, I would suggest to leave it out of the new API and we cann add it back later as needed.
  • If this is being called with the intent of resizing, I also would assume that it applies to the entire range too (in fact, I cannot think of a good formal definition of "resizing a subrange", especially if the resized range ends before the end of the MemMap).

What do "physical" and "virtual" mean in this context? (maybe this a stale comment from the previous revision of this CL)


The distinction between releasePagesToOS and releaseAndZeroPagesToOS does not currently exist in Scudo, right? Can we add it as a new feature in a later CL?


Similar to above, I assume that the ctor creates an object such that !isCreated(). Should we also have a dtor that automatically releases if still isCreated()?


Should this also check that !isCreated() as a precondition?


Similarly, I think we do not need Flags for release either.

Chia-hungDuan marked 5 inline comments as done.

Address review comment


These two are supposed to be lazy initialized so the creation needs to be done explicitly.

The order of destructing allocator fields is tricky. Especially we have the thread local cache. Given that tearing down the allocator only happens at program exit, we just make them being released naturally. Therefore, the creation/deletion are done explicitly.

We have been thinking to make it more elegantly but that depends on too many stuff (e.g., libc). Besides, this CL is supposed to do the abstraction transition only, I prefer to do further logic changes in the later CLs.


I prefer to keep this suggestion because it can still benefit some contexts (like some cache management stuff we will have in the future). It's a different case than ReservedMemory supposed to cover.


Like the release one, the only case I was thinking is the UNMAP_ALL for Fuchsia, if you think it's fine to remove it. I'm happy to do that :)

is there any alternative way to achieve this? Like create a new VMO ?


The MemMap is a handle of range of virtual address and it's supposed to do the supported operations in the range. So it can remap part of the virtual address. For example, the RegionBeg is unlikely to be equal to Region base address. That's why we have this Addr argument here (For future function it'll be useful as well). If a platform doesn't allow this, then we just do the valid operation in that platform to fulfill their constraint.

BTW, we have setMemoryPermission (which is added in the follow up CL)


It means virtual memory and physical memory.


In general, yes, we may consider adding this in a separate CL.

The reason I do it here is that we are revising API contract and try to reduce ambiguity. For example, the arguments of releasePagesToOS was confusing. And then we found that the zero-page stuff is ambiguous as well. That's why I bring it in this CL.


Same as above.


This is intended to make it implementation defined behavior. In some case we may want few reserved spaces.


I was hesitating this before and the only case is UNMAP_ALL used by Fuchsia. If you agree that we can get rid of it. I'm happy with that :) Done.

Chia-hungDuan added inline comments.Mar 24 2023, 12:13 PM

Oops, please ignore this wrong reply.

cferris accepted this revision.Apr 3 2023, 3:29 PM


This revision is now accepted and ready to land.Apr 3 2023, 3:29 PM

Make ReservedMemory::create return boolean to indicate if the creation succeeds.
This makes the page allocation have the consistent style of API

cferris accepted this revision.Apr 4 2023, 2:52 PM

Still looks good.

fabio-d reopened this revision.Apr 7 2023, 6:49 AM
fabio-d added inline comments.

This used to be a grow operation for a memory region that always started at RegionBeg. Now we are dispatching at getRegionBaseByClassId(ClassId) and then resizing starting from RegionBeg. Can we directly dispatch at RegionBeg instead?
This results in MemMapDefault::releaseAndZeroPagesToOSImpl passing an offset relative to getRegionBaseByClassId(ClassId) instead of RegionBeg to the ::releasePagesToOS function, which makes Scudo release in-use pages on Fuchsia.

This revision is now accepted and ready to land.Apr 7 2023, 6:49 AM
Chia-hungDuan added inline comments.Apr 7 2023, 9:24 AM

Because it's a region's base, I tended to think it's still part of a region. To make it work before transition to each platform specific implementation. I agree that we can dispatch the reserved memory from RegionBeg and fix it back later

fabio-d closed this revision.Apr 20 2023, 5:12 AM

Closing because the issue on Fuchsia was fixed in D147792