This is an archive of the discontinued LLVM Phabricator instance.

[HWASan] Use page aliasing on x86_64.
ClosedPublic

Authored by morehouse on Mar 18 2021, 9:54 AM.

Details

Summary

Userspace page aliasing allows us to use middle pointer bits for tags
without untagging them before syscalls or accesses. This should enable
easier experimentation with HWASan on x86_64 platforms.

Currently stack, global, and secondary heap tagging are unsupported.
Only primary heap allocations get tagged.

Note that aliasing mode will not work properly in the presence of
fork(), since heap memory will be shared between the parent and child
processes. This mode is non-ideal; we expect Intel LAM to enable full
HWASan support on x86_64 in the future.

Diff Detail

Event Timeline

morehouse created this revision.Mar 18 2021, 9:54 AM
morehouse requested review of this revision.Mar 18 2021, 9:54 AM
Herald added projects: Restricted Project, Restricted Project. · View Herald TranscriptMar 18 2021, 9:54 AM
Herald added subscribers: llvm-commits, Restricted Project. · View Herald Transcript
eugenis accepted this revision.Mar 19 2021, 6:12 PM

LGTM

compiler-rt/lib/hwasan/hwasan_dynamic_shadow.cpp
125

this seems kind of arbitrary, why not RingBufferSize()?

This revision is now accepted and ready to land.Mar 19 2021, 6:12 PM
morehouse updated this revision to Diff 332310.Mar 22 2021, 9:05 AM
morehouse marked an inline comment as done.
  • Fix includes; use RingBufferSize().
vitalybuka accepted this revision.Mar 22 2021, 2:43 PM
This revision was automatically updated to reflect the committed changes.

Removing all the UNTAGs from tests caused an aarch64 bot (presumably without TBI) to fail: https://lab.llvm.org/buildbot/#/builders/53/builds/1634

I'll put the UNTAGs back in but make them no-ops on x86.

AArch64 had TBI from the start, but the kernel support was added only recently. Add untags only in the failing tests, for memory that is passed to system calls.

morehouse reopened this revision.Mar 25 2021, 6:31 AM
This revision is now accepted and ready to land.Mar 25 2021, 6:31 AM
morehouse updated this revision to Diff 333286.Mar 25 2021, 6:31 AM
  • Add back UNTAGs for Linux syscalls.
This revision was landed with ongoing or failed builds.Mar 25 2021, 7:04 AM
This revision was automatically updated to reflect the committed changes.
EccoTheDolphin added inline comments.Apr 21 2021, 1:24 PM
compiler-rt/test/hwasan/TestCases/hwasan-print-shadow.cpp
11–14

@morehouse and @eugenis it seems that __hwasan_tag_memory expects untagged pointer. This particular piece looks a little bit suspicious. Could you please clarify?

morehouse added inline comments.Apr 21 2021, 4:55 PM
compiler-rt/test/hwasan/TestCases/hwasan-print-shadow.cpp
11–14

Nice catch. I think it currently works since allocator tagging is disabled by default for the HWASan tests.

But we probably shouldn't rely on that.

morehouse marked an inline comment as done.Apr 21 2021, 5:13 PM
morehouse added inline comments.
compiler-rt/test/hwasan/TestCases/hwasan-print-shadow.cpp
11–14

We do need the malloc instead of mmap on x86 since only the heap is aliased.

Submitted https://reviews.llvm.org/rG3511022f5f0a to explicitly untag the pointer.

xiangzhangllvm added inline comments.
compiler-rt/lib/hwasan/hwasan.h
41

Hello @morehouse, I have some questions here:

  1. Does the "page alias" in your patch mean "tagged address and untagged address map to same place" ?
  2. "The alias region is placed next to the shadow", what does the "alias region" here mean?
  3. Why kAddressTagShift = 39 , kTagBits = 3 ? Does it mean the "alias region" size is 2^39Bytes ? It seems to me that the ""alias region is placed inside the shadow?

Thank you !

morehouse added inline comments.Apr 22 2021, 8:57 AM
compiler-rt/lib/hwasan/hwasan.h
41

Hello @morehouse, I have some questions here:

  1. Does the "page alias" in your patch mean "tagged address and untagged address map to same place" ?

Yes, the virtual addresses map to the same physical addresses.

  1. "The alias region is placed next to the shadow", what does the "alias region" here mean?

We preallocate address space for the entire heap and create aliases for that region at startup. So "alias region" == heap and aliases.

  1. Why kAddressTagShift = 39 , kTagBits = 3 ?

kTagBits = 3 is due to some preliminary performance benchmarks. Anything higher than that seems to greatly increase overhead. kAddressTagShift = 39 is to allow a contiguous heap of 512GB, which should be plenty in practice.

Does it mean the "alias region" size is 2^39Bytes ?

Heap is 2^39 == 512GB. Then we have 7 other aliases of that memory, for a total alias region size of 2^42 == 4TB.

It seems to me that the ""alias region is placed inside the shadow?

The shadow region on x86_64 is 8TB starting at a dynamic offset. We dynamically place the alias region (heap and aliases) immediately following the shadow region.

Thank you !

compiler-rt/lib/hwasan/hwasan.h
41

Thank you!
I think I understand:
We "pre-allocate" 4TB for heap, but we just really used in 512GB, so the bit 39-42 can be use as tag for a heap point!

compiler-rt/test/hwasan/TestCases/use-after-free.c