[sanitizer] Support dynamic premapped R/W range in primary allocator.

[morehouse]

Summary

The main use case for this change is HWASan aliasing mode, which premaps
the alias space adjacent to the dynamic shadow. With this change, the
primary allocator can allocate from the alias space instead of a
separate region.

[eugenis]

LGTM

compiler-rt/lib/sanitizer_common/sanitizer_allocator_primary64.h

108–109

I wonder if MADVISE_DONTNEED works correctly on aliased memory? You probably need to run it on each alias for it to have an effect on RSS.

[morehouse]

compiler-rt/lib/sanitizer_common/sanitizer_allocator_primary64.h

108–109

Yes, just ran an experiment. Each alias occupies RSS independently. So we can fault in one alias and the others won't take up RSS until we touch them. If we touch multiple aliases, they each take up a chunk of RSS.

Probably we need a callback that hwasan_allocator.cpp can register in order to release alias pages as well.

[pcc]

compiler-rt/lib/sanitizer_common/sanitizer_allocator_primary64.h

108–109

I don't think it would. Since the original mapping was created with MAP_SHARED, MADV_DONTNEED will not affect the contents of the pages. I confirmed this by running this program:

#define _GNU_SOURCE
#include <sys/mman.h>
#include <stdio.h>

int main() {
  void *p = mmap(0, 4096, PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANON, -1, 0); 
  void *p2 = mremap(p, 0, 4096, MREMAP_MAYMOVE);
  *(char *)p = 1;
  printf("%d\n", *(volatile char *)p2);
  madvise(p, 4096, MADV_DONTNEED);
  printf("%d\n", *(volatile char *)p2);
  madvise(p2, 4096, MADV_DONTNEED);
  printf("%d\n", *(volatile char *)p2);
}

The output is

1
1
1

[morehouse]

compiler-rt/lib/sanitizer_common/sanitizer_allocator_primary64.h

108–109

Right, the memory is not cleared, but RSS does go down by the expected amount after the MADV_DONTNEED.

So it seems worth adding a callback?

[pcc]

compiler-rt/lib/sanitizer_common/sanitizer_allocator_primary64.h

108–109

The goal isn't just to make RSS go down though, right? Presumably if the data is still accessible then it's still in memory. I'm not sure if there's a syscall sequence that we can use to release this memory.

This is also a bit more concerning:

#define _GNU_SOURCE
#include <sys/mman.h>
#include <stdio.h>

int main() {
  void *p = mmap(0, 4096, PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANON, -1, 0);
  void *p2 = mremap(p, 0, 4096, MREMAP_MAYMOVE);
  *(char *)p = 1;
  printf("%d\n", *(volatile char *)p2);
  madvise(p, 4096, MADV_DONTNEED);
  printf("%d\n", *(volatile char *)p2);
  madvise(p2, 4096, MADV_DONTNEED);
  printf("%d\n", *(volatile char *)p2);
  if (fork() == 0) {
    *(char *)p = 2;
    exit(0);
  }
  sleep(1);
  printf("%d\n", *(volatile char *)p);
}

The output is

1
1
1
2

[vitalybuka]

LGTM if "release" will be addressed in another patch

compiler-rt/lib/sanitizer_common/sanitizer_allocator_primary64.h

678

do we need to check that beg/size are in mapped range?

[morehouse]

• Check bounds on mmaps.

[morehouse]

In D98293#2618636, @vitalybuka wrote:

LGTM if "release" will be addressed in another patch

Yes, I will address it in a different patch.

compiler-rt/lib/sanitizer_common/sanitizer_allocator_primary64.h

108–109

I think memory sharing across fork is the price we pay for MAP_SHARED. At least within Google I expect it to not be much problem since uses of fork are 99%+ fork+exec. Maybe we can duplicate the alias region with a fork interceptor, but this is complexity I'd like to avoid unless necessary.

For actually releasing the physical memory, the only thing I think *might* work is tmpfs + fallocate(FALLOCATE_FL_PUNCH_HOLE). I'd need to experiment to be sure. But that also involves more complexity that I'd like to avoid (at least initially). Let's only deal with it if it becomes an issue?

[morehouse]

compiler-rt/lib/sanitizer_common/sanitizer_allocator_primary64.h

108–109

Unfortunately some of the build infra (python, flex) within Google calls fork. I tried changing TCMalloc to use MAP_SHARED, and got crashes when building things:

Fatal Python error: GC object already tracked

(Segmentation fault) flex failed: error executing command...

So I am less confident we can deploy this now...

With a fork interceptor we might be able to duplicate and recreate aliases in the child, but copying TBs of memory will be very slow.

[eugenis]

compiler-rt/lib/sanitizer_common/sanitizer_allocator_primary64.h

108–109

Ouch. Even in fork + exec code there may be a heap memory write between fork and exec that would affect the parent process.

We do not need to copy TBs of memory - only the actual physical pages. We could do this by scanning /proc/$PID/pagemap. Or maybe mlock + mincore? Even such limited copy might be prohibitively expensive.

[morehouse]

• Add TODO for release callback support.

[eugenis]

LGTM

[vitalybuka]

Reverted, it fails on Windows https://lab.llvm.org/buildbot/#/builders/127/builds/7999 and Android https://lab.llvm.org/buildbot/#/builders/77/builds/4839

[morehouse]

• Disable tests on Android and Windows due to OOM.

compiler-rt/lib/sanitizer_common/sanitizer_allocator_combined.h

using SecondaryAllocator =
                          typename PrimaryAllocator::AddressSpaceView>;
 
   void InitLinkerInitialized(s32 release_to_os_interval_ms) {
     stats_.InitLinkerInitialized();
     primary_.Init(release_to_os_interval_ms);
     secondary_.InitLinkerInitialized();
   }
 
-  void Init(s32 release_to_os_interval_ms) {
+  void Init(s32 release_to_os_interval_ms, uptr heap_start = 0) {
     stats_.Init();
-    primary_.Init(release_to_os_interval_ms);
+    primary_.Init(release_to_os_interval_ms, heap_start);
     secondary_.Init();
   }
 
   void *Allocate(AllocatorCache *cache, uptr size, uptr alignment) {
     // Returning 0 on malloc(0) may break a lot of code.
     if (size == 0)
       size = 1;
     if (size + alignment < size) {

compiler-rt/lib/sanitizer_common/sanitizer_allocator_primary32.h

public:
   static uptr ClassIdToSize(uptr class_id) {
     return (class_id == SizeClassMap::kBatchClassID) ?
         kBatchSize : SizeClassMap::Size(class_id);
   }
 
   typedef SizeClassAllocator32<Params> ThisT;
   typedef SizeClassAllocator32LocalCache<ThisT> AllocatorCache;
 
-  void Init(s32 release_to_os_interval_ms) {
+  void Init(s32 release_to_os_interval_ms, uptr heap_start = 0) {
+    CHECK(!heap_start);
     possible_regions.Init();
     internal_memset(size_class_info_array, 0, sizeof(size_class_info_array));
   }
 
   s32 ReleaseToOSIntervalMs() const {
     return kReleaseToOSIntervalNever;
   }
 

compiler-rt/lib/sanitizer_common/sanitizer_allocator_primary64.h

public:
   static const uptr kCompactPtrScale = 4;
   CompactPtrT PointerToCompactPtr(uptr base, uptr ptr) const {
     return static_cast<CompactPtrT>((ptr - base) >> kCompactPtrScale);
   }
   uptr CompactPtrToPointer(uptr base, CompactPtrT ptr32) const {
     return base + (static_cast<uptr>(ptr32) << kCompactPtrScale);
   }
 
-  void Init(s32 release_to_os_interval_ms) {
+  // If heap_start is nonzero, assumes kSpaceSize bytes are already mapped R/W
+  // at heap_start and places the heap there.  This mode requires kSpaceBeg ==
+  // ~(uptr)0.
+  void Init(s32 release_to_os_interval_ms, uptr heap_start = 0) {
     uptr TotalSpaceSize = kSpaceSize + AdditionalSize();
+    PremappedHeap = heap_start != 0;
+    if (PremappedHeap) {
+      CHECK(!kUsingConstantSpaceBeg);
+      NonConstSpaceBeg = heap_start;
+      uptr RegionInfoSize = AdditionalSize();
+      RegionInfoSpace =
+          address_range.Init(RegionInfoSize, PrimaryAllocatorName);
+      CHECK_NE(RegionInfoSpace, ~(uptr)0);
+      CHECK_EQ(RegionInfoSpace,
+               address_range.MapOrDie(RegionInfoSpace, RegionInfoSize,
+                                      "SizeClassAllocator: region info"));
+      MapUnmapCallback().OnMap(RegionInfoSpace, RegionInfoSize);
+    } else {
       if (kUsingConstantSpaceBeg) {
         CHECK(IsAligned(kSpaceBeg, SizeClassMap::kMaxSize));
-      CHECK_EQ(kSpaceBeg, address_range.Init(TotalSpaceSize,
-                                             PrimaryAllocatorName, kSpaceBeg));
+        CHECK_EQ(kSpaceBeg,
+                 address_range.Init(TotalSpaceSize, PrimaryAllocatorName,
+                                    kSpaceBeg));
       } else {
-      // Combined allocator expects that an 2^N allocation is always aligned to
-      // 2^N. For this to work, the start of the space needs to be aligned as
-      // high as the largest size class (which also needs to be a power of 2).
+        // Combined allocator expects that an 2^N allocation is always aligned
+        // to 2^N. For this to work, the start of the space needs to be aligned
+        // as high as the largest size class (which also needs to be a power of
+        // 2).
         NonConstSpaceBeg = address_range.InitAligned(
             TotalSpaceSize, SizeClassMap::kMaxSize, PrimaryAllocatorName);
         CHECK_NE(NonConstSpaceBeg, ~(uptr)0);
       }
-    SetReleaseToOSIntervalMs(release_to_os_interval_ms);
-    MapWithCallbackOrDie(SpaceEnd(), AdditionalSize(),
+      RegionInfoSpace = SpaceEnd();
+      MapWithCallbackOrDie(RegionInfoSpace, AdditionalSize(),
                            "SizeClassAllocator: region info");
+    }
+    SetReleaseToOSIntervalMs(release_to_os_interval_ms);
     // Check that the RegionInfo array is aligned on the CacheLine size.
-    DCHECK_EQ(SpaceEnd() % kCacheLineSize, 0);
+    DCHECK_EQ(RegionInfoSpace % kCacheLineSize, 0);
   }
 
   s32 ReleaseToOSIntervalMs() const {
     return atomic_load(&release_to_os_interval_ms_, memory_order_relaxed);
   }
 
   void SetReleaseToOSIntervalMs(s32 release_to_os_interval_ms) {
     atomic_store(&release_to_os_interval_ms_, release_to_os_interval_ms,
private:
   static const uptr kUserMapSize = 1 << 16;
   // Call mmap for metadata memory with at least this size.
   static const uptr kMetaMapSize = 1 << 16;
   // Call mmap for free array memory with at least this size.
   static const uptr kFreeArrayMapSize = 1 << 16;
 
   atomic_sint32_t release_to_os_interval_ms_;
 
+  uptr RegionInfoSpace;
+
+  // True if the user has already mapped the entire heap R/W.
+  bool PremappedHeap;
+
   struct Stats {
     uptr n_allocated;
     uptr n_freed;
   };
 
   struct ReleaseToOsInfo {
     uptr n_freed_at_last_release;
     uptr num_releases;
struct ALIGNED(SANITIZER_CACHE_LINE_SIZE) RegionInfo {
     bool exhausted;  // Whether region is out of space for new chunks.
     Stats stats;
     ReleaseToOsInfo rtoi;
   };
   COMPILER_CHECK(sizeof(RegionInfo) % kCacheLineSize == 0);
 
   RegionInfo *GetRegionInfo(uptr class_id) const {
     DCHECK_LT(class_id, kNumClasses);
-    RegionInfo *regions = reinterpret_cast<RegionInfo *>(SpaceEnd());
+    RegionInfo *regions = reinterpret_cast<RegionInfo *>(RegionInfoSpace);
     return &regions[class_id];
   }
 
   uptr GetMetadataEnd(uptr region_beg) const {
     return region_beg + kRegionSize - kFreeArraySize;
   }
 
   uptr GetChunkIdx(uptr chunk, uptr size) const {
     if (!kUsingConstantSpaceBeg)
       chunk -= SpaceBeg();
 
     uptr offset = chunk % kRegionSize;
     // Here we divide by a non-constant. This is costly.
     // size always fits into 32-bits. If the offset fits too, use 32-bit div.
     if (offset >> (SANITIZER_WORDSIZE / 2))
       return offset / size;
     return (u32)offset / (u32)size;
   }
 
   CompactPtrT *GetFreeArray(uptr region_beg) const {
     return reinterpret_cast<CompactPtrT *>(GetMetadataEnd(region_beg));
   }
 
   bool MapWithCallback(uptr beg, uptr size, const char *name) {
+    if (PremappedHeap)
+      return beg >= NonConstSpaceBeg &&
+             beg + size <= NonConstSpaceBeg + kSpaceSize;
     uptr mapped = address_range.Map(beg, size, name);
     if (UNLIKELY(!mapped))
       return false;
     CHECK_EQ(beg, mapped);
     MapUnmapCallback().OnMap(beg, size);
     return true;
   }
 
   void MapWithCallbackOrDie(uptr beg, uptr size, const char *name) {
+    if (PremappedHeap) {
+      CHECK_GE(beg, NonConstSpaceBeg);
+      CHECK_LE(beg + size, NonConstSpaceBeg + kSpaceSize);
+      return;
+    }
     CHECK_EQ(beg, address_range.MapOrDie(beg, size, name));
     MapUnmapCallback().OnMap(beg, size);
   }
 
   void UnmapWithCallbackOrDie(uptr beg, uptr size) {
+    if (PremappedHeap)
+      return;
     MapUnmapCallback().OnUnmap(beg, size);
     address_range.Unmap(beg, size);
   }
 
   bool EnsureFreeArraySpace(RegionInfo *region, uptr region_beg,
                             uptr num_freed_chunks) {
     uptr needed_space = num_freed_chunks * sizeof(CompactPtrT);
     if (region->mapped_free_array < needed_space) {
private:
     const ThisT& allocator;
     const uptr region_base;
     uptr released_ranges_count;
     uptr released_bytes;
   };
 
   // Attempts to release RAM occupied by freed chunks back to OS. The region is
   // expected to be locked.
+  //
+  // TODO(morehouse): Support a callback on memory release so HWASan can release
+  // aliases as well.
   void MaybeReleaseToOS(uptr class_id, bool force) {
     RegionInfo *region = GetRegionInfo(class_id);
     const uptr chunk_size = ClassIdToSize(class_id);
     const uptr page_size = GetPageSizeCached();
 
     uptr n = region->num_freed_chunks;
     if (n * chunk_size < page_size)
       return;  // No chance to release anything.

compiler-rt/lib/sanitizer_common/tests/sanitizer_allocator_test.cpp

TEST(SanitizerCommon, InternalSizeClassMap) {
   TestSizeClassMap<InternalSizeClassMap>();
 }
 
 TEST(SanitizerCommon, DenseSizeClassMap) {
   TestSizeClassMap<VeryCompactSizeClassMap>();
 }
 
 template <class Allocator>
-void TestSizeClassAllocator() {
+void TestSizeClassAllocator(uptr premapped_heap = 0) {
   Allocator *a = new Allocator;
-  a->Init(kReleaseToOSIntervalNever);
+  a->Init(kReleaseToOSIntervalNever, premapped_heap);
   typename Allocator::AllocatorCache cache;
   memset(&cache, 0, sizeof(cache));
   cache.Init(0);
 
   static const uptr sizes[] = {
     1, 16,  30, 40, 100, 1000, 10000,
     50000, 60000, 100000, 120000, 300000, 500000, 1000000, 2000000
   };
for (uptr x = 0, step = kAddressSpaceSize / 100000;
     if (a->PointerIsMine(reinterpret_cast<void *>(x)))
       Ident(a->GetBlockBegin(reinterpret_cast<void *>(x)));
 
   a->TestOnlyUnmap();
   delete a;
 }
 
 #if SANITIZER_CAN_USE_ALLOCATOR64
+
+// Allocates kAllocatorSize aligned bytes on construction and frees it on
+// destruction.
+class ScopedPremappedHeap {
+ public:
+  ScopedPremappedHeap() {
+    BasePtr = MmapNoReserveOrDie(2 * kAllocatorSize, "preallocated heap");
+    AlignedAddr = RoundUpTo(reinterpret_cast<uptr>(BasePtr), kAllocatorSize);
+  }
+
+  ~ScopedPremappedHeap() { UnmapOrDie(BasePtr, kAllocatorSize); }
+
+  uptr Addr() { return AlignedAddr; }
+
+ private:
+  void *BasePtr;
+  uptr AlignedAddr;
+};
+
 // These tests can fail on Windows if memory is somewhat full and lit happens
 // to run them all at the same time. FIXME: Make them not flaky and reenable.
 #if !SANITIZER_WINDOWS
 TEST(SanitizerCommon, SizeClassAllocator64) {
   TestSizeClassAllocator<Allocator64>();
 }
 
 TEST(SanitizerCommon, SizeClassAllocator64Dynamic) {
   TestSizeClassAllocator<Allocator64Dynamic>();
 }
 
 #if !SANITIZER_ANDROID
+// Android only has 39-bit address space, so mapping 2 * kAllocatorSize
+// sometimes fails.
+TEST(SanitizerCommon, SizeClassAllocator64DynamicPremapped) {
+  ScopedPremappedHeap h;
+  TestSizeClassAllocator<Allocator64Dynamic>(h.Addr());
+}
+
 //FIXME(kostyak): find values so that those work on Android as well.
 TEST(SanitizerCommon, SizeClassAllocator64Compact) {
   TestSizeClassAllocator<Allocator64Compact>();
 }
 
 TEST(SanitizerCommon, SizeClassAllocator64Dense) {
   TestSizeClassAllocator<Allocator64Dense>();
 }

 using Allocator32SeparateBatches =
     Allocator32SeparateBatchesASVT<LocalAddressSpaceView>;
 
 TEST(SanitizerCommon, SizeClassAllocator32SeparateBatches) {
   TestSizeClassAllocator<Allocator32SeparateBatches>();
 }
 
 template <class Allocator>
-void SizeClassAllocatorMetadataStress() {
+void SizeClassAllocatorMetadataStress(uptr premapped_heap = 0) {
   Allocator *a = new Allocator;
-  a->Init(kReleaseToOSIntervalNever);
+  a->Init(kReleaseToOSIntervalNever, premapped_heap);
   typename Allocator::AllocatorCache cache;
   memset(&cache, 0, sizeof(cache));
   cache.Init(0);
 
   const uptr kNumAllocs = 1 << 13;
   void *allocated[kNumAllocs];
   void *meta[kNumAllocs];
   for (uptr i = 0; i < kNumAllocs; i++) {
TEST(SanitizerCommon, SizeClassAllocator64MetadataStress) {
   SizeClassAllocatorMetadataStress<Allocator64>();
 }
 
 TEST(SanitizerCommon, SizeClassAllocator64DynamicMetadataStress) {
   SizeClassAllocatorMetadataStress<Allocator64Dynamic>();
 }
 
 #if !SANITIZER_ANDROID
+TEST(SanitizerCommon, SizeClassAllocator64DynamicPremappedMetadataStress) {
+  ScopedPremappedHeap h;
+  SizeClassAllocatorMetadataStress<Allocator64Dynamic>(h.Addr());
+}
+
 TEST(SanitizerCommon, SizeClassAllocator64CompactMetadataStress) {
   SizeClassAllocatorMetadataStress<Allocator64Compact>();
 }
 #endif
 
 #endif
 #endif  // SANITIZER_CAN_USE_ALLOCATOR64
 TEST(SanitizerCommon, SizeClassAllocator32CompactMetadataStress) {
   SizeClassAllocatorMetadataStress<Allocator32Compact>();
 }
 
 template <class Allocator>
-void SizeClassAllocatorGetBlockBeginStress(u64 TotalSize) {
+void SizeClassAllocatorGetBlockBeginStress(u64 TotalSize,
+                                           uptr premapped_heap = 0) {
   Allocator *a = new Allocator;
-  a->Init(kReleaseToOSIntervalNever);
+  a->Init(kReleaseToOSIntervalNever, premapped_heap);
   typename Allocator::AllocatorCache cache;
   memset(&cache, 0, sizeof(cache));
   cache.Init(0);
 
   uptr max_size_class = Allocator::SizeClassMapT::kLargestClassID;
   uptr size = Allocator::SizeClassMapT::Size(max_size_class);
   // Make sure we correctly compute GetBlockBegin() w/o overflow.
   for (size_t i = 0; i <= TotalSize / size; i++) {
TEST(SanitizerCommon, SizeClassAllocator64GetBlockBegin) {
   SizeClassAllocatorGetBlockBeginStress<Allocator64>(
       1ULL << (SANITIZER_ANDROID ? 31 : 33));
 }
 TEST(SanitizerCommon, SizeClassAllocator64DynamicGetBlockBegin) {
   SizeClassAllocatorGetBlockBeginStress<Allocator64Dynamic>(
       1ULL << (SANITIZER_ANDROID ? 31 : 33));
 }
 #if !SANITIZER_ANDROID
+TEST(SanitizerCommon, SizeClassAllocator64DynamicPremappedGetBlockBegin) {
+  ScopedPremappedHeap h;
+  SizeClassAllocatorGetBlockBeginStress<Allocator64Dynamic>(
+      1ULL << (SANITIZER_ANDROID ? 31 : 33), h.Addr());
+}
 TEST(SanitizerCommon, SizeClassAllocator64CompactGetBlockBegin) {
   SizeClassAllocatorGetBlockBeginStress<Allocator64Compact>(1ULL << 33);
 }
 #endif
 TEST(SanitizerCommon, SizeClassAllocator64VeryCompactGetBlockBegin) {
   // Does not have > 4Gb for each class.
   SizeClassAllocatorGetBlockBeginStress<Allocator64VeryCompact>(1ULL << 31);
 }
TEST(SanitizerCommon, LargeMmapAllocator) {
   char *p = (char *)a.Allocate(&stats, page_size, 1);
   CHECK_EQ(p, a.GetBlockBegin(p));
   CHECK_EQ(p, (char *)a.GetBlockBegin(p + page_size - 1));
   CHECK_NE(p, (char *)a.GetBlockBegin(p + page_size));
   a.Deallocate(&stats, p);
 }
 
 template <class PrimaryAllocator>
-void TestCombinedAllocator() {
+void TestCombinedAllocator(uptr premapped_heap = 0) {
   typedef CombinedAllocator<PrimaryAllocator> Allocator;
   Allocator *a = new Allocator;
-  a->Init(kReleaseToOSIntervalNever);
+  a->Init(kReleaseToOSIntervalNever, premapped_heap);
   std::mt19937 r;
 
   typename Allocator::AllocatorCache cache;
   memset(&cache, 0, sizeof(cache));
   a->InitCache(&cache);
 
   EXPECT_EQ(a->Allocate(&cache, -1, 1), (void*)0);
   EXPECT_EQ(a->Allocate(&cache, -1, 1024), (void*)0);
TEST(SanitizerCommon, CombinedAllocator64) {
   TestCombinedAllocator<Allocator64>();
 }
 
 TEST(SanitizerCommon, CombinedAllocator64Dynamic) {
   TestCombinedAllocator<Allocator64Dynamic>();
 }
 
 #if !SANITIZER_ANDROID
+#if !SANITIZER_WINDOWS
+// Windows fails to map 1TB, so disable this test.
+TEST(SanitizerCommon, CombinedAllocator64DynamicPremapped) {
+  ScopedPremappedHeap h;
+  TestCombinedAllocator<Allocator64Dynamic>(h.Addr());
+}
+#endif
+
 TEST(SanitizerCommon, CombinedAllocator64Compact) {
   TestCombinedAllocator<Allocator64Compact>();
 }
 #endif
 
 TEST(SanitizerCommon, CombinedAllocator64VeryCompact) {
   TestCombinedAllocator<Allocator64VeryCompact>();
 }
 #endif
 
 TEST(SanitizerCommon, SKIP_ON_SOLARIS_SPARCV9(CombinedAllocator32Compact)) {
   TestCombinedAllocator<Allocator32Compact>();
 }
 
 template <class Allocator>
-void TestSizeClassAllocatorLocalCache() {
+void TestSizeClassAllocatorLocalCache(uptr premapped_heap = 0) {
   using AllocatorCache = typename Allocator::AllocatorCache;
   AllocatorCache cache;
   Allocator *a = new Allocator();
 
-  a->Init(kReleaseToOSIntervalNever);
+  a->Init(kReleaseToOSIntervalNever, premapped_heap);
   memset(&cache, 0, sizeof(cache));
   cache.Init(0);
 
   const uptr kNumAllocs = 10000;
   const int kNumIter = 100;
   uptr saved_total = 0;
   for (int class_id = 1; class_id <= 5; class_id++) {
     for (int it = 0; it < kNumIter; it++) {
TEST(SanitizerCommon, SizeClassAllocator64LocalCache) {
   TestSizeClassAllocatorLocalCache<Allocator64>();
 }
 
 TEST(SanitizerCommon, SizeClassAllocator64DynamicLocalCache) {
   TestSizeClassAllocatorLocalCache<Allocator64Dynamic>();
 }
 
 #if !SANITIZER_ANDROID
+TEST(SanitizerCommon, SizeClassAllocator64DynamicPremappedLocalCache) {
+  ScopedPremappedHeap h;
+  TestSizeClassAllocatorLocalCache<Allocator64Dynamic>(h.Addr());
+}
+
 TEST(SanitizerCommon, SizeClassAllocator64CompactLocalCache) {
   TestSizeClassAllocatorLocalCache<Allocator64Compact>();
 }
 #endif
 TEST(SanitizerCommon, SizeClassAllocator64VeryCompactLocalCache) {
   TestSizeClassAllocatorLocalCache<Allocator64VeryCompact>();
 }
 #endif
TEST(Allocator, ScopedBuffer) {
   }
 }
 
 void IterationTestCallback(uptr chunk, void *arg) {
   reinterpret_cast<std::set<uptr> *>(arg)->insert(chunk);
 }
 
 template <class Allocator>
-void TestSizeClassAllocatorIteration() {
+void TestSizeClassAllocatorIteration(uptr premapped_heap = 0) {
   Allocator *a = new Allocator;
-  a->Init(kReleaseToOSIntervalNever);
+  a->Init(kReleaseToOSIntervalNever, premapped_heap);
   typename Allocator::AllocatorCache cache;
   memset(&cache, 0, sizeof(cache));
   cache.Init(0);
 
   static const uptr sizes[] = {1, 16, 30, 40, 100, 1000, 10000,
     50000, 60000, 100000, 120000, 300000, 500000, 1000000, 2000000};
 
   std::vector<void *> allocated;

 // to run them all at the same time. FIXME: Make them not flaky and reenable.
 #if !SANITIZER_WINDOWS
 TEST(SanitizerCommon, SizeClassAllocator64Iteration) {
   TestSizeClassAllocatorIteration<Allocator64>();
 }
 TEST(SanitizerCommon, SizeClassAllocator64DynamicIteration) {
   TestSizeClassAllocatorIteration<Allocator64Dynamic>();
 }
+#if !SANITIZER_ANDROID
+TEST(SanitizerCommon, SizeClassAllocator64DynamicPremappedIteration) {
+  ScopedPremappedHeap h;
+  TestSizeClassAllocatorIteration<Allocator64Dynamic>(h.Addr());
+}
+#endif
 #endif
 #endif
 
 TEST(SanitizerCommon, SKIP_ON_SOLARIS_SPARCV9(SizeClassAllocator32Iteration)) {
   TestSizeClassAllocatorIteration<Allocator32Compact>();
 }
 
 TEST(SanitizerCommon, LargeMmapAllocatorIteration) {