Page MenuHomePhabricator

[scudo][standalone] Compact pointers for Caches/Batches

Authored by cryptoad on Feb 10 2021, 10:17 AM.



This CL introduces configuration options to allow pointers to be
compacted in the thread-specific caches and transfer batches. This
offers the possibility to have them use 32-bit of space instead of
64-bit for the 64-bit Primary, thus cutting the size of the caches
and batches by nearly half (and as such the memory used in size
class 0). The cost is an additional read from the region information
in the fast path.

This is not a new idea, as it's being used in the sanitizer_common
64-bit primary. The difference here is that it is configurable via
the allocator config, with the possibility of not compacting at all.

This CL enables compacting pointers in the Android and Fuchsia default

Diff Detail

Unit TestsFailed

420 msx64 debian > AddressSanitizer-x86_64-linux-dynamic.TestCases/Linux::odr-violation.cpp
Script: -- : 'RUN: at line 10'; /mnt/disks/ssd0/agent/llvm-project/build/./bin/clang --driver-mode=g++ -fsanitize=address -mno-omit-leaf-frame-pointer -fno-omit-frame-pointer -fno-optimize-sibling-calls -gline-tables-only -m64 -shared-libasan -DBUILD_SO=1 -fPIC -shared /mnt/disks/ssd0/agent/llvm-project/compiler-rt/test/asan/TestCases/Linux/odr-violation.cpp -o /mnt/disks/ssd0/agent/llvm-project/build/projects/compiler-rt/test/asan/X86_64LinuxDynamicConfig/TestCases/Linux/Output/
700 msx64 debian > AddressSanitizer-x86_64-linux.TestCases/Linux::odr-violation.cpp
Script: -- : 'RUN: at line 10'; /mnt/disks/ssd0/agent/llvm-project/build/./bin/clang --driver-mode=g++ -fsanitize=address -mno-omit-leaf-frame-pointer -fno-omit-frame-pointer -fno-optimize-sibling-calls -gline-tables-only -m64 -DBUILD_SO=1 -fPIC -shared /mnt/disks/ssd0/agent/llvm-project/compiler-rt/test/asan/TestCases/Linux/odr-violation.cpp -o /mnt/disks/ssd0/agent/llvm-project/build/projects/compiler-rt/test/asan/X86_64LinuxConfig/TestCases/Linux/Output/
280 msx64 debian > MemProfiler-x86_64-linux-dynamic.TestCases::test_malloc_load_store.c
Script: -- : 'RUN: at line 5'; /mnt/disks/ssd0/agent/llvm-project/build/./bin/clang --driver-mode=g++ -fmemory-profile -mno-omit-leaf-frame-pointer -fno-omit-frame-pointer -fno-optimize-sibling-calls -gline-tables-only -m64 -shared-libsan -O0 /mnt/disks/ssd0/agent/llvm-project/compiler-rt/test/memprof/TestCases/test_malloc_load_store.c -o /mnt/disks/ssd0/agent/llvm-project/build/projects/compiler-rt/test/memprof/X86_64LinuxDynamicConfig/TestCases/Output/test_malloc_load_store.c.tmp

Event Timeline

cryptoad requested review of this revision.Feb 10 2021, 10:17 AM
cryptoad created this revision.
Herald added a project: Restricted Project. · View Herald TranscriptFeb 10 2021, 10:17 AM
Herald added a subscriber: Restricted Project. · View Herald Transcript

Ping please!

cferris accepted this revision.Feb 22 2021, 1:13 PM

This looks good to me.

On Android, there is no detectable performance degradation, it has the same performance when running CTS tests that caused a problem before because these two tests triggered a lot of OS releases. Also, when running the traces, the total RSS goes down on most traces by between 0.5 MB and 1MB and with no performance degradation.

Finally, all known tests pass on Android.

This revision is now accepted and ready to land.Feb 22 2021, 1:13 PM
hctim added inline comments.Feb 22 2021, 1:27 PM

nit: cache


This seems like a common construct, maybe worth introducing

void* Allocator::decompactPointer(ClassId, MaybeCompactedPtr)


why not just compactPtr and decompactPtr?


Why have the logic in the 32-bit primary if it's not possible to compact here?

cryptoad added inline comments.Feb 22 2021, 2:07 PM





It does sound better, thank you!


I initially wanted the 32 & 64 to "look alike" but I think I can get rid of this here

cryptoad updated this revision to Diff 325810.Feb 23 2021, 9:05 AM
cryptoad marked an inline comment as done.

Address Mitch's comments:

  • rename the compacting/decompacting functions
  • add a "helper" version of those that uses the class ID
  • remove the compacting logic from the 32-bit primary
cryptoad updated this revision to Diff 325812.Feb 23 2021, 9:15 AM

And remove 32-bit scale from the allocator config.

cryptoad updated this revision to Diff 325820.Feb 23 2021, 9:27 AM

Rename function in test for uniformity

hctim accepted this revision.Feb 23 2021, 9:44 AM

LGTM w/ one nit.


Looks like every reference to this function immediately casts to void*. Can we just cast it here instead?

cryptoad added inline comments.Feb 23 2021, 9:53 AM

Good point. I missed that, sorry!

cryptoad updated this revision to Diff 325825.Feb 23 2021, 10:10 AM

Update decompactPtr to return a void *

cryptoad updated this revision to Diff 325838.Feb 23 2021, 10:47 AM

I mistakenly snuck in an experiment I Was doing in the local cache, so
revert that line. Apologies.

hctim accepted this revision.Feb 23 2021, 11:05 AM


Harbormaster completed remote builds in B90433: Diff 325838.
This revision was landed with ongoing or failed builds.Feb 25 2021, 12:15 PM
This revision was automatically updated to reflect the committed changes.