[scudo] Add Scudo support for Trusty OS

[danieljm]

Summary

trusty.cpp and trusty.h define Trusty implementations of map and other
platform-specific functions. In addition to adding Trusty configurations
in allocator_config.h and size_class_map.h, MapSizeIncrement and
PrimaryEnableRandomOffset are added as configurable options in
allocator_config.h.
Background on Trusty: https://source.android.com/security/trusty

[vitalybuka]

compiler-rt/lib/scudo/standalone/memtag.h

58–60

Maybe

compiler-rt/lib/scudo/standalone/trusty.cpp

63–67

some tests should fails without reasonable implementation
what is the plan for testing?

[cryptoad]

Thank you Daniel, this looks like a good start!
I'll work on my side on reducing some of the memory footprint of some structures and whatnot (scoped strings come to mind).
As discussed, we can re-introduce the use-separate-class-for-batches distinction for Trusty, which can be a large win to get rid of a class size.
I still think the Primary32 might be better suited here, but we'll follow up.

compiler-rt/lib/scudo/standalone/allocator_config.h

48

I think this should be PrimaryMapSizeIncrement to be homogeneous with the naming of all the Primary settings.

compiler-rt/lib/scudo/standalone/primary64.h

79

Maybe use a ternary here?
Region->RegionBeg = getRegionBaseByClassId(I) + (PrimaryEnableRandomOffset ? getRandomModN(&Seed, 16) + 1) * PageSize : 0);

274

If it's only used once, you might just want to use Config::PrimaryEnableRandomOffset and not add the extra definition.

compiler-rt/lib/scudo/standalone/trusty.cpp

34

Addr is UNUSED

37

The variables in this function don't have the "proper" casing. Should be ProgramBreak and so on.

[danieljm]

Updated scudo/standalone/tests with primary64 configurable
options PrimaryEnableRandomOffset and PrimaryMapSizeIncrement.

[danieljm]

compiler-rt/lib/scudo/standalone/memtag.h

58–60

Even if I make this suggested change, there is non-Linux-specific AArch64 assembly which cannot be compiled with Trusty's build system (see lines 119-265).

compiler-rt/lib/scudo/standalone/trusty.cpp

63–67

Trusty has its own build system, so the existing Scudo tests would need to be ported to Trusty as test apps. We have a Trusty test app for Scudo running downstream on the Trusty side, and we can consider porting other Scudo tests to Trusty downstream.

[danieljm]

Passed primary64.h through clang-format.

[pcc]

compiler-rt/lib/scudo/standalone/memtag.h

58–60

Isn't it a compiler issue rather than a build system issue? Can you resolve it by upgrading the compiler? (I think we technically should be making this conditional on the compiler version since our minimum supported compiler version is older than the version that added MTE support, but that's a separate issue.)

[danieljm]

Reversion of changes to memtag.h and mutex.h

[danieljm]

compiler-rt/lib/scudo/standalone/memtag.h

58–60

You're right, I meant the Trusty compiler was out of date. As it turns out, the compiler was just updated so now this change is unnecessary and memtag.h can be left as it was before.

[cryptoad]

compiler-rt/lib/scudo/standalone/trusty.cpp

94

Isn't printf using malloc (or others)? If so this will recurse.

[cryptoad]

compiler-rt/lib/scudo/standalone/trusty.cpp

53

Can _trusty_brk set the errno to ENOMEM?

[danieljm]

set errno manually in map since _trusty_brk does not set it

[danieljm]

compiler-rt/lib/scudo/standalone/trusty.cpp

53

It turns out that _trusty_brk does not set errno, so I modify map to set it manually. This is consistent with how Trusty implements sbrk using _trusty_brk as well. Also, since we know errno equals ENOMEM in this if branch, we just directly call dieOnMapUnmapError(Size) instead of dieOnMapUnmapError(errno == ENOMEM ? Size : 0);

94

Trusty uses Musl's implementation of printf which uses a fixed size buffer allocated on the stack.

[danieljm]

Rebased patch.

[cryptoad]

I patched it locally and everything passes for me.
Given the current timeline, I am OK with landing it now as is, and improve gradually on it.
I will pick up some of the action items, notably to try and reduce the memory footprint.

[vitalybuka]

compiler-rt/lib/scudo/standalone/allocator_config.h

172

Please add corresponding SCUDO_TYPED_TEST_TYPE(FIXTURE, NAME, TrustyConfig) into combined test

compiler-rt/lib/scudo/standalone/tests/combined_test.cpp

112

SCUDO_TYPED_TEST_TYPE(FIXTURE, NAME, TrustyConfig)

[cryptoad]

compiler-rt/lib/scudo/standalone/tests/combined_test.cpp

112

Enabling the Trusty config on non-trusty config is wonky:

#4  0x0000555555438f23 in scudo::map (Addr=0x7fffcfaab400, Size=1024, 
    Name=0x7ffff7a68237 "scudo:primary", Flags=5, Data=0x172aff200210)
    at third_party/llvm/llvm-project/compiler-rt/lib/scudo/standalone/linux.cpp:69

It ends up trying to map at a non-page boundary a non-multiple-of-page size, ending up in an abort.

[vitalybuka]

LGTM but upstream testing could be very helpful for those who change code without TrustyOS setup

compiler-rt/lib/scudo/standalone/allocator_config.h

 //   // Log2 of the size of a size class region, as used by the Primary.
 //   static const uptr PrimaryRegionSizeLog = 30U;
 //   // Defines the type and scale of a compact pointer. A compact pointer can
 //   // be understood as the offset of a pointer within the region it belongs
 //   // to, in increments of a power-of-2 scale.
 //   // eg: Ptr = Base + (CompactPtr << Scale).
 //   typedef u32 PrimaryCompactPtrT;
 //   static const uptr PrimaryCompactPtrScale = SCUDO_MIN_ALIGNMENT_LOG;
+//   // Indicates support for offsetting the start of a region by
+//   // a random number of pages. Only used with primary64.
+//   static const bool PrimaryEnableRandomOffset = true;
+//   // Call map for user memory with at least this size. Only used with
+//   // primary64.
+//   static const uptr PrimaryMapSizeIncrement = 1UL << 18;
 //   // Defines the minimal & maximal release interval that can be set.
 //   static const s32 PrimaryMinReleaseToOsIntervalMs = INT32_MIN;
 //   static const s32 PrimaryMaxReleaseToOsIntervalMs = INT32_MAX;
 //   // Defines the type of cache used by the Secondary. Some additional
 //   // configuration entries can be necessary depending on the Cache.
 //   typedef MapAllocatorNoCache SecondaryCache;
 //   // Thread-Specific Data Registry used, shared or exclusive.
 //   template <class A> using TSDRegistryT = TSDRegistrySharedT<A, 8U, 4U>;
 // };
 
 // Default configurations for various platforms.
 
 struct DefaultConfig {
   using SizeClassMap = DefaultSizeClassMap;
   static const bool MaySupportMemoryTagging = false;
 
 #if SCUDO_CAN_USE_PRIMARY64
   typedef SizeClassAllocator64<DefaultConfig> Primary;
   static const uptr PrimaryRegionSizeLog = 32U;
   typedef uptr PrimaryCompactPtrT;
   static const uptr PrimaryCompactPtrScale = 0;
+  static const bool PrimaryEnableRandomOffset = true;
+  static const uptr PrimaryMapSizeIncrement = 1UL << 18;
 #else
   typedef SizeClassAllocator32<DefaultConfig> Primary;
   static const uptr PrimaryRegionSizeLog = 19U;
   typedef uptr PrimaryCompactPtrT;
 #endif
   static const s32 PrimaryMinReleaseToOsIntervalMs = INT32_MIN;
   static const s32 PrimaryMaxReleaseToOsIntervalMs = INT32_MAX;
 
struct AndroidConfig {
   using SizeClassMap = AndroidSizeClassMap;
   static const bool MaySupportMemoryTagging = true;
 
 #if SCUDO_CAN_USE_PRIMARY64
   typedef SizeClassAllocator64<AndroidConfig> Primary;
   static const uptr PrimaryRegionSizeLog = 28U;
   typedef u32 PrimaryCompactPtrT;
   static const uptr PrimaryCompactPtrScale = SCUDO_MIN_ALIGNMENT_LOG;
+  static const bool PrimaryEnableRandomOffset = true;
+  static const uptr PrimaryMapSizeIncrement = 1UL << 18;
 #else
   typedef SizeClassAllocator32<AndroidConfig> Primary;
   static const uptr PrimaryRegionSizeLog = 18U;
   typedef uptr PrimaryCompactPtrT;
 #endif
   static const s32 PrimaryMinReleaseToOsIntervalMs = 1000;
   static const s32 PrimaryMaxReleaseToOsIntervalMs = 1000;
 
struct AndroidSvelteConfig {
   using SizeClassMap = SvelteSizeClassMap;
   static const bool MaySupportMemoryTagging = false;
 
 #if SCUDO_CAN_USE_PRIMARY64
   typedef SizeClassAllocator64<AndroidSvelteConfig> Primary;
   static const uptr PrimaryRegionSizeLog = 27U;
   typedef u32 PrimaryCompactPtrT;
   static const uptr PrimaryCompactPtrScale = SCUDO_MIN_ALIGNMENT_LOG;
+  static const bool PrimaryEnableRandomOffset = true;
+  static const uptr PrimaryMapSizeIncrement = 1UL << 18;
 #else
   typedef SizeClassAllocator32<AndroidSvelteConfig> Primary;
   static const uptr PrimaryRegionSizeLog = 16U;
   typedef uptr PrimaryCompactPtrT;
 #endif
   static const s32 PrimaryMinReleaseToOsIntervalMs = 1000;
   static const s32 PrimaryMaxReleaseToOsIntervalMs = 1000;
 

 #if SCUDO_CAN_USE_PRIMARY64
 struct FuchsiaConfig {
   using SizeClassMap = FuchsiaSizeClassMap;
   static const bool MaySupportMemoryTagging = false;
 
   typedef SizeClassAllocator64<FuchsiaConfig> Primary;
   static const uptr PrimaryRegionSizeLog = 30U;
   typedef u32 PrimaryCompactPtrT;
+  static const bool PrimaryEnableRandomOffset = true;
+  static const uptr PrimaryMapSizeIncrement = 1UL << 18;
   static const uptr PrimaryCompactPtrScale = SCUDO_MIN_ALIGNMENT_LOG;
   static const s32 PrimaryMinReleaseToOsIntervalMs = INT32_MIN;
   static const s32 PrimaryMaxReleaseToOsIntervalMs = INT32_MAX;
 
   typedef MapAllocatorNoCache SecondaryCache;
   template <class A>
   using TSDRegistryT = TSDRegistrySharedT<A, 8U, 4U>; // Shared, max 8 TSDs.
 };
+
+struct TrustyConfig {
+  using SizeClassMap = TrustySizeClassMap;
+  static const bool MaySupportMemoryTagging = false;
+
+  typedef SizeClassAllocator64<TrustyConfig> Primary;
+  // Some apps have 1 page of heap total so small regions are necessary.
+  static const uptr PrimaryRegionSizeLog = 10U;
+  typedef u32 PrimaryCompactPtrT;
+  static const bool PrimaryEnableRandomOffset = false;
+  // Trusty is extremely memory-constrained so minimally round up map calls.
+  static const uptr PrimaryMapSizeIncrement = 1UL << 4;
+  static const uptr PrimaryCompactPtrScale = SCUDO_MIN_ALIGNMENT_LOG;
+  static const s32 PrimaryMinReleaseToOsIntervalMs = INT32_MIN;
+  static const s32 PrimaryMaxReleaseToOsIntervalMs = INT32_MAX;
+
+  typedef MapAllocatorNoCache SecondaryCache;
+  template <class A>
+  using TSDRegistryT = TSDRegistrySharedT<A, 1U, 1U>; // Shared, max 1 TSD.
+};
 #endif
 
 #if SCUDO_ANDROID
 typedef AndroidConfig Config;
 #elif SCUDO_FUCHSIA
 typedef FuchsiaConfig Config;
+#elif SCUDO_TRUSTY
+typedef TrustyConfig Config;
 #else
 typedef DefaultConfig Config;
 #endif
 
 } // namespace scudo
 
 #endif // SCUDO_ALLOCATOR_CONFIG_H_

compiler-rt/lib/scudo/standalone/common.h

 //===-- common.h ------------------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef SCUDO_COMMON_H_
 #define SCUDO_COMMON_H_
 
 #include "internal_defs.h"
 
 #include "fuchsia.h"
 #include "linux.h"
+#include "trusty.h"
 
 #include <stddef.h>
 #include <string.h>
 
 namespace scudo {
 
 template <class Dest, class Source> inline Dest bit_cast(const Source &S) {
   static_assert(sizeof(Dest) == sizeof(Source), "");

compiler-rt/lib/scudo/standalone/memtag.h

 
 inline uint8_t extractTag(uptr Ptr) {
   (void)Ptr;
   UNREACHABLE("memory tagging not supported");
 }
 
 #endif
 
-#if defined(__aarch64__)
+#if defined(__aarch64__) && !SCUDO_TRUSTY
 
 #if SCUDO_LINUX
 
 inline bool systemSupportsMemoryTagging() {
 #ifndef HWCAP2_MTE
 #define HWCAP2_MTE (1 << 18)
 #endif
   return getauxval(AT_HWCAP2) & HWCAP2_MTE;
 }
 

compiler-rt/lib/scudo/standalone/mutex.h

#endif
     lockSlow();
   }
   void unlock();
 
 private:
   static constexpr u8 NumberOfTries = 8U;
   static constexpr u8 NumberOfYields = 8U;
 
-#if SCUDO_LINUX
+#if SCUDO_LINUX || SCUDO_TRUSTY
   atomic_u32 M = {};
 #elif SCUDO_FUCHSIA
   sync_mutex_t M = {};
 #endif
 
   void lockSlow();
 };
 

compiler-rt/lib/scudo/standalone/platform.h

 //===-- platform.h ----------------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef SCUDO_PLATFORM_H_
 #define SCUDO_PLATFORM_H_
 
 // Transitive includes of stdint.h specify some of the defines checked below.
 #include <stdint.h>
 
-#if defined(__linux__)
+#if defined(__linux__) && !defined(__TRUSTY__)
 #define SCUDO_LINUX 1
 #else
 #define SCUDO_LINUX 0
 #endif
 
 // See https://android.googlesource.com/platform/bionic/+/master/docs/defines.md
 #if defined(__BIONIC__)
 #define SCUDO_ANDROID 1
 #else
 #define SCUDO_ANDROID 0
 #endif
 
 #if defined(__Fuchsia__)
 #define SCUDO_FUCHSIA 1
 #else
 #define SCUDO_FUCHSIA 0
 #endif
 
+#if defined(__TRUSTY__)
+#define SCUDO_TRUSTY 1
+#else
+#define SCUDO_TRUSTY 0
+#endif
+
 #if __LP64__
 #define SCUDO_WORDSIZE 64U
 #else
 #define SCUDO_WORDSIZE 32U
 #endif
 
 #if SCUDO_WORDSIZE == 64U
 #define FIRST_32_SECOND_64(a, b) (b)

compiler-rt/lib/scudo/standalone/primary32.h

public:
   }
 
   static bool canAllocate(uptr Size) { return Size <= SizeClassMap::MaxSize; }
 
   void init(s32 ReleaseToOsInterval) {
     if (SCUDO_FUCHSIA)
       reportError("SizeClassAllocator32 is not supported on Fuchsia");
 
+    if (SCUDO_TRUSTY)
+      reportError("SizeClassAllocator32 is not supported on Trusty");
+
     PossibleRegions.init();
 
     u32 Seed;
     const u64 Time = getMonotonicTime();
     if (!getRandom(reinterpret_cast<void *>(&Seed), sizeof(Seed)))
       Seed = static_cast<u32>(
           Time ^ (reinterpret_cast<uptr>(SizeClassInfoArray) >> 6));
     for (uptr I = 0; I < NumClasses; I++) {

compiler-rt/lib/scudo/standalone/primary64.h

 #include "string_utils.h"
 
 namespace scudo {
 
 // SizeClassAllocator64 is an allocator tuned for 64-bit address space.
 //
 // It starts by reserving NumClasses * 2^RegionSizeLog bytes, equally divided in
 // Regions, specific to each size class. Note that the base of that mapping is
-// random (based to the platform specific map() capabilities), and that each
-// Region actually starts at a random offset from its base.
+// random (based to the platform specific map() capabilities). If
+// PrimaryEnableRandomOffset is set, each Region actually starts at a random
+// offset from its base.
 //
 // Regions are mapped incrementally on demand to fulfill allocation requests,
 // those mappings being split into equally sized Blocks based on the size class
 // they belong to. The Blocks created are shuffled to prevent predictable
 // address patterns (the predictability increases with the size of the Blocks).
 //
 // The 1st Region (for size class 0) holds the TransferBatches. This is a
 // structure used to transfer arrays of available pointers from the class size
void init(s32 ReleaseToOsInterval) {
 
     u32 Seed;
     const u64 Time = getMonotonicTime();
     if (!getRandom(reinterpret_cast<void *>(&Seed), sizeof(Seed)))
       Seed = static_cast<u32>(Time ^ (PrimaryBase >> 12));
     const uptr PageSize = getPageSizeCached();
     for (uptr I = 0; I < NumClasses; I++) {
       RegionInfo *Region = getRegionInfo(I);
-      // The actual start of a region is offseted by a random number of pages.
-      Region->RegionBeg =
-          getRegionBaseByClassId(I) + (getRandomModN(&Seed, 16) + 1) * PageSize;
+      // The actual start of a region is offset by a random number of pages
+      // when PrimaryEnableRandomOffset is set.
+      Region->RegionBeg = getRegionBaseByClassId(I) +
+                          (Config::PrimaryEnableRandomOffset
+                           ? ((getRandomModN(&Seed, 16) + 1) * PageSize) : 0);
       Region->RandState = getRandomU32(&Seed);
       Region->ReleaseInfo.LastReleaseAtNs = Time;
     }
     setOption(Option::ReleaseInterval, static_cast<sptr>(ReleaseToOsInterval));
   }
 
   void unmapTestOnly() {
     for (uptr I = 0; I < NumClasses; I++) {
       RegionInfo *Region = getRegionInfo(I);
public:
 
   AtomicOptions Options;
 
 private:
   static const uptr RegionSize = 1UL << Config::PrimaryRegionSizeLog;
   static const uptr NumClasses = SizeClassMap::NumClasses;
   static const uptr PrimarySize = RegionSize * NumClasses;
 
-  // Call map for user memory with at least this size.
-  static const uptr MapSizeIncrement = 1UL << 18;
+  static const uptr MapSizeIncrement = Config::PrimaryMapSizeIncrement;
   // Fill at most this number of batches from the newly map'd memory.
   static const u32 MaxNumBatches = SCUDO_ANDROID ? 4U : 8U;
 
   struct RegionStats {
     uptr PoppedBlocks;
     uptr PushedBlocks;
   };
 
   struct ReleaseToOsInfo {

compiler-rt/lib/scudo/standalone/size_class_map.h

#else
   static const u32 MaxNumCachedHint = 14;
   static const uptr MaxBytesCachedLog = 10;
   static const uptr SizeDelta = Chunk::getHeaderSize();
 #endif
 };
 
 typedef FixedSizeClassMap<SvelteSizeClassConfig> SvelteSizeClassMap;
 
+// Trusty is configured to only have one region containing blocks of size
+// 2^7 bytes.
+struct TrustySizeClassConfig {
+  static const uptr NumBits = 1;
+  static const uptr MinSizeLog = 7;
+  static const uptr MidSizeLog = 7;
+  static const uptr MaxSizeLog = 7;
+  static const u32 MaxNumCachedHint = 8;
+  static const uptr MaxBytesCachedLog = 10;
+  static const uptr SizeDelta = 0;
+};
+
+typedef FixedSizeClassMap<TrustySizeClassConfig> TrustySizeClassMap;
+
 template <typename SCMap> inline void printMap() {
   ScopedString Buffer(1024);
   uptr PrevS = 0;
   uptr TotalCached = 0;
   for (uptr I = 0; I < SCMap::NumClasses; I++) {
     if (I == SCMap::BatchClassId)
       continue;
     const uptr S = SCMap::getSizeByClassId(I);

compiler-rt/lib/scudo/standalone/tests/combined_test.cpp

 
 #if SCUDO_FUCHSIA
 #define SCUDO_TYPED_TEST_ALL_TYPES(FIXTURE, NAME)                              \
   SCUDO_TYPED_TEST_TYPE(FIXTURE, NAME, AndroidSvelteConfig)                    \
   SCUDO_TYPED_TEST_TYPE(FIXTURE, NAME, FuchsiaConfig)
 #else
 #define SCUDO_TYPED_TEST_ALL_TYPES(FIXTURE, NAME)                              \
   SCUDO_TYPED_TEST_TYPE(FIXTURE, NAME, AndroidSvelteConfig)                    \
   SCUDO_TYPED_TEST_TYPE(FIXTURE, NAME, DefaultConfig)                          \
   SCUDO_TYPED_TEST_TYPE(FIXTURE, NAME, AndroidConfig)
 #endif
 
 #define SCUDO_TYPED_TEST_TYPE(FIXTURE, NAME, TYPE)                             \
   using FIXTURE##NAME##_##TYPE = FIXTURE##NAME<scudo::TYPE>;                   \
   TEST_F(FIXTURE##NAME##_##TYPE, NAME) { Run(); }
 
 #define SCUDO_TYPED_TEST(FIXTURE, NAME)                                        \
struct DeathConfig {
   // Tiny allocator, its Primary only serves chunks of four sizes.
   using SizeClassMap = scudo::FixedSizeClassMap<DeathSizeClassConfig>;
   typedef scudo::SizeClassAllocator64<DeathConfig> Primary;
   static const scudo::uptr PrimaryRegionSizeLog = DeathRegionSizeLog;
   static const scudo::s32 PrimaryMinReleaseToOsIntervalMs = INT32_MIN;
   static const scudo::s32 PrimaryMaxReleaseToOsIntervalMs = INT32_MAX;
   typedef scudo::uptr PrimaryCompactPtrT;
   static const scudo::uptr PrimaryCompactPtrScale = 0;
+  static const bool PrimaryEnableRandomOffset = true;
+  static const scudo::uptr PrimaryMapSizeIncrement = 1UL << 18;
 
   typedef scudo::MapAllocatorNoCache SecondaryCache;
   template <class A> using TSDRegistryT = scudo::TSDRegistrySharedT<A, 1U, 1U>;
 };
 
 TEST(ScudoCombinedTest, DeathCombined) {
   using AllocatorT = TestAllocator<DeathConfig>;
   auto Allocator = std::unique_ptr<AllocatorT>(new AllocatorT());

compiler-rt/lib/scudo/standalone/tests/primary_test.cpp

 
 struct TestConfig1 {
   static const scudo::uptr PrimaryRegionSizeLog = 18U;
   static const scudo::s32 PrimaryMinReleaseToOsIntervalMs = INT32_MIN;
   static const scudo::s32 PrimaryMaxReleaseToOsIntervalMs = INT32_MAX;
   static const bool MaySupportMemoryTagging = false;
   typedef scudo::uptr PrimaryCompactPtrT;
   static const scudo::uptr PrimaryCompactPtrScale = 0;
+  static const bool PrimaryEnableRandomOffset = true;
+  static const scudo::uptr PrimaryMapSizeIncrement = 1UL << 18;
 };
 
 struct TestConfig2 {
 #if defined(__mips__)
   // Unable to allocate greater size on QEMU-user.
   static const scudo::uptr PrimaryRegionSizeLog = 23U;
 #else
   static const scudo::uptr PrimaryRegionSizeLog = 24U;
 #endif
   static const scudo::s32 PrimaryMinReleaseToOsIntervalMs = INT32_MIN;
   static const scudo::s32 PrimaryMaxReleaseToOsIntervalMs = INT32_MAX;
   static const bool MaySupportMemoryTagging = false;
   typedef scudo::uptr PrimaryCompactPtrT;
   static const scudo::uptr PrimaryCompactPtrScale = 0;
+  static const bool PrimaryEnableRandomOffset = true;
+  static const scudo::uptr PrimaryMapSizeIncrement = 1UL << 18;
 };
 
 struct TestConfig3 {
 #if defined(__mips__)
   // Unable to allocate greater size on QEMU-user.
   static const scudo::uptr PrimaryRegionSizeLog = 23U;
 #else
   static const scudo::uptr PrimaryRegionSizeLog = 24U;
 #endif
   static const scudo::s32 PrimaryMinReleaseToOsIntervalMs = INT32_MIN;
   static const scudo::s32 PrimaryMaxReleaseToOsIntervalMs = INT32_MAX;
   static const bool MaySupportMemoryTagging = true;
   typedef scudo::uptr PrimaryCompactPtrT;
   static const scudo::uptr PrimaryCompactPtrScale = 0;
+  static const bool PrimaryEnableRandomOffset = true;
+  static const scudo::uptr PrimaryMapSizeIncrement = 1UL << 18;
 };
 
 template <typename BaseConfig, typename SizeClassMapT>
 struct Config : public BaseConfig {
   using SizeClassMap = SizeClassMapT;
 };
 
 template <typename BaseConfig, typename SizeClassMapT>

 struct SmallRegionsConfig {
   using SizeClassMap = scudo::DefaultSizeClassMap;
   static const scudo::uptr PrimaryRegionSizeLog = 20U;
   static const scudo::s32 PrimaryMinReleaseToOsIntervalMs = INT32_MIN;
   static const scudo::s32 PrimaryMaxReleaseToOsIntervalMs = INT32_MAX;
   static const bool MaySupportMemoryTagging = false;
   typedef scudo::uptr PrimaryCompactPtrT;
   static const scudo::uptr PrimaryCompactPtrScale = 0;
+  static const bool PrimaryEnableRandomOffset = true;
+  static const scudo::uptr PrimaryMapSizeIncrement = 1UL << 18;
 };
 
 // The 64-bit SizeClassAllocator can be easily OOM'd with small region sizes.
 // For the 32-bit one, it requires actually exhausting memory, so we skip it.
 TEST(ScudoPrimaryTest, Primary64OOM) {
   using Primary = scudo::SizeClassAllocator64<SmallRegionsConfig>;
   using TransferBatch = Primary::CacheT::TransferBatch;
   Primary Allocator;

compiler-rt/lib/scudo/standalone/trusty.h

+//===-- trusty.h -----------------------------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef SCUDO_TRUSTY_H_
+#define SCUDO_TRUSTY_H_
+
+#include "platform.h"
+
+#if SCUDO_TRUSTY
+
+namespace scudo {
+// MapPlatformData is unused on Trusty, define it as a minimially sized
+// structure.
+struct MapPlatformData {};
+} // namespace scudo
+
+#endif // SCUDO_TRUSTY
+
+#endif // SCUDO_TRUSTY_H_

compiler-rt/lib/scudo/standalone/trusty.cpp

+//===-- trusty.cpp ---------------------------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "platform.h"
+
+#if SCUDO_TRUSTY
+
+#include "common.h"
+#include "mutex.h"
+#include "string_utils.h"
+#include "trusty.h"
+
+#include <errno.h>           // for errno
+#include <stdio.h>           // for printf()
+#include <stdlib.h>          // for getenv()
+#include <sys/auxv.h>        // for getauxval()
+#include <time.h>            // for clock_gettime()
+#include <trusty_syscalls.h> // for _trusty_brk()
+
+#define SBRK_ALIGN 32
+
+namespace scudo {
+
+uptr getPageSize() { return getauxval(AT_PAGESZ); }
+
+void NORETURN die() { abort(); }
+
+void *map(UNUSED void *Addr, uptr Size, UNUSED const char *Name, uptr Flags,
+          UNUSED MapPlatformData *Data) {
+  // Calling _trusty_brk(0) returns the current program break.
+  uptr ProgramBreak = reinterpret_cast<uptr>(_trusty_brk(0));
+  uptr Start;
+  uptr End;
+
+  Start = roundUpTo(ProgramBreak, SBRK_ALIGN);
+  // Don't actually extend the heap if MAP_NOACCESS flag is set since this is
+  // the case where Scudo tries to reserve a memory region without mapping
+  // physical pages.
+  if (Flags & MAP_NOACCESS)
+    return reinterpret_cast<void *>(Start);
+
+  // Attempt to extend the heap by Size bytes using _trusty_brk.
+  End = roundUpTo(Start + Size, SBRK_ALIGN);
+  ProgramBreak =
+      reinterpret_cast<uptr>(_trusty_brk(reinterpret_cast<void *>(End)));
+  if (ProgramBreak < End) {
+    dieOnMapUnmapError(errno == ENOMEM);
+    return nullptr;
+  }
+  return reinterpret_cast<void *>(Start); // Base of new reserved region.
+}
+
+// Unmap is a no-op since Trusty uses sbrk instead of memory mapping.
+void unmap(UNUSED void *Addr, UNUSED uptr Size, UNUSED uptr Flags,
+           UNUSED MapPlatformData *Data) {}
+
+void setMemoryPermission(UNUSED uptr Addr, UNUSED uptr Size, UNUSED uptr Flags,
+                         UNUSED MapPlatformData *Data) {}
+
+void releasePagesToOS(UNUSED uptr BaseAddress, UNUSED uptr Offset,
+                      UNUSED uptr Size, UNUSED MapPlatformData *Data) {}
+
+const char *getEnv(const char *Name) { return getenv(Name); }
+
+// All mutex operations are a no-op since Trusty doesn't currently support
+// threads.
+bool HybridMutex::tryLock() { return true; }
+
+void HybridMutex::lockSlow() {}
+
+void HybridMutex::unlock() {}
+
+u64 getMonotonicTime() {
+  timespec TS;
+  clock_gettime(CLOCK_MONOTONIC, &TS);
+  return static_cast<u64>(TS.tv_sec) * (1000ULL * 1000 * 1000) +
+         static_cast<u64>(TS.tv_nsec);
+}
+
+u32 getNumberOfCPUs() { return 0; }
+
+u32 getThreadID() { return 0; }
+
+bool getRandom(UNUSED void *Buffer, UNUSED uptr Length, UNUSED bool Blocking) {
+  return false;
+}
+
+void outputRaw(const char *Buffer) { printf("%s", Buffer); }
+
+void setAbortMessage(UNUSED const char *Message) {}
+
+} // namespace scudo
+
+#endif // SCUDO_TRUSTY

compiler-rt/lib/scudo/standalone/vector.h

public:
   T *begin() { return data(); }
   const T *end() const { return data() + size(); }
   T *end() { return data() + size(); }
 
 private:
   void reallocate(uptr NewCapacity) {
     DCHECK_GT(NewCapacity, 0);
     DCHECK_LE(Size, NewCapacity);
-    const uptr NewCapacityBytes =
-        roundUpTo(NewCapacity * sizeof(T), getPageSizeCached());
+    // By default, NewCapacityBytes is rounded up to the nearest page.
+    // Because Trusty is extremely memory-constrained, NewCapacityBytes is
+    // only minimally rounded up.
+    const uptr NewCapacityBytes = roundUpTo(
+        NewCapacity * sizeof(T),
+        SCUDO_TRUSTY ? 1U << SCUDO_MIN_ALIGNMENT_LOG : getPageSizeCached());
     T *NewData =
         reinterpret_cast<T *>(map(nullptr, NewCapacityBytes, "scudo:vector"));
     if (Data) {
       memcpy(NewData, Data, Size * sizeof(T));
       unmap(Data, CapacityBytes);
     }
     Data = NewData;
     CapacityBytes = NewCapacityBytes;