This is an archive of the discontinued LLVM Phabricator instance.

[esan|wset] Iterate all memory to compute the total working set
ClosedPublic

Authored by bruening on May 24 2016, 10:20 AM.

Details

Summary

Adds iteration of all application memory in an efficient manner using
shadow faults. Shadow memory starts out inaccessible and we mark it
writable one page at a time on each fault when the instrumentation touches
it. This allows iteration over just the mapped shadow memory, saving
significant time.

Adds a process-end iteration and pretty-printing of the final result.

Adds a new test and updates the existing tests.

Diff Detail

Event Timeline

bruening updated this revision to Diff 58263.May 24 2016, 10:20 AM
bruening retitled this revision from to [esan|wset] Iterate all memory to compute the total working set.
bruening updated this object.
bruening added a reviewer: aizatsky.
bruening added subscribers: llvm-commits, eugenis, kcc and 2 others.
aizatsky added inline comments.May 24 2016, 2:36 PM
lib/esan/working_set.cpp
79

This function changes shadow value. Should be at least reflected in the name.

Plus "processShadowRegion" is not very telling.

105

Same.

E.g. "computeWorkingSizeAndClearRecent".

130

symbolic constant for 64 please.

132

static

lib/esan/working_set_posix.cpp
69

Should this be working_set_linux.cpp then?

filcab added a subscriber: filcab.May 25 2016, 8:21 AM
filcab added inline comments.
lib/esan/working_set.cpp
85

RoundDown? We'll be looking at some shadow bytes before ShadowStart. Won't that be a problem?

89

sizeof(u32)

90

ByteValue has only one bit set. No need to test for equality.

129

We don't really care, since we only do this at finalize time (and we're printing stuff anyway), no?

131

No sense, again. Just make a global kDefaultCacheLineSize, and document next to it that we only support that size.

lib/esan/working_set_posix.cpp
69

or at least the _posix variant.

test/esan/TestCases/workingset-simple.cpp
18

What do you mean?

bruening updated this revision to Diff 58506.May 25 2016, 2:08 PM
bruening marked 9 inline comments as done.

Address reviewer requests.

lib/esan/working_set.cpp
85

No, this routine may be used for handling smaller regions in the future (e.g., just the heap) where it is convenient to not require precision or alignment from the caller.

129

There are linker errors for missing symbols like __cxa_guard_acquire which would require adding libraries these sanitizer libs try to avoid.

lib/esan/working_set_posix.cpp
69

POSIX specifies that si_addr be filled in for SIGSEGV. It's older Linux that fails to do so (we're talking really old: 10+ years IIRC).

filcab accepted this revision.May 26 2016, 8:38 AM
filcab added a reviewer: filcab.

LGTM

lib/esan/working_set.cpp
35

Should this be in the header?

79

We should probably mention it might count and clear a few bytes more than [ShadowStart,ShadowEnd)

This revision is now accepted and ready to land.May 26 2016, 8:38 AM
bruening added inline comments.May 26 2016, 10:04 AM
lib/esan/working_set.cpp
35

We can move it to the header once another file needs it.

79

That is what the word-align comment implies.

vitalybuka added inline comments.May 26 2016, 10:46 AM
lib/esan/working_set.cpp
84

Is this the same as?
u32 WordValue = 0x01010101 << BitIdx;

eugenis requested changes to this revision.May 26 2016, 10:57 AM
eugenis added a reviewer: eugenis.

This revision depends on r270650, which broke the bots. Please fix that first.
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux/builds/24123/steps/test%20standalone%20compiler-rt/logs/stdio

This revision now requires changes to proceed.May 26 2016, 10:57 AM

The bot is now green.

eugenis accepted this revision.May 27 2016, 11:04 AM
eugenis edited edge metadata.
This revision is now accepted and ready to land.May 27 2016, 11:04 AM
This revision was automatically updated to reflect the committed changes.