[DFSan] Fix flakey release_shadow_space.c accounting for Origin chains.

[browneee]

Summary

Test sometimes fails on buildbot (after two non-Origins executions):

/usr/bin/ld: warning: Cannot export local symbol 'dfsan_flush'
RSS at start: 4620, after mmap: 107020, after mmap+set label: 209424, after fixed map: 4624, after another mmap+set label: 209424, after munmap: 4624
/usr/bin/ld: warning: Cannot export local symbol 'dfsan_flush'
RSS at start: 4620, after mmap: 107020, after mmap+set label: 209424, after fixed map: 4624, after another mmap+set label: 209424, after munmap: 4624
/usr/bin/ld: warning: Cannot export local symbol 'dfsan_flush'
RSS at start: 4620, after mmap: 107020, after mmap+set label: 317992, after fixed map: 10792, after another mmap+set label: 317992, after munmap: 10792
release_shadow_space.c.tmp: /b/sanitizer-x86_64-linux/build/llvm-project/compiler-rt/test/dfsan/release_shadow_space.c:91: int main(int, char **): Assertion `after_fixed_mmap <= before + delta' failed.

[browneee]

compiler-rt/test/dfsan/release_shadow_space.c

100–101

Note: This delta could probably be reduced (maybe removed?), but I won't change it in this CL.

[browneee]

compiler-rt/test/dfsan/release_shadow_space.c

100–101

Another more revealing test failure log:

/usr/bin/ld: warning: Cannot export local symbol 'dfsan_flush'
RSS at start: 2516, after mmap: 104916, after mmap+set label: 207320, after fixed map: 2520, after another mmap+set label: 207320, after munmap: 2520
/usr/bin/ld: warning: Cannot export local symbol 'dfsan_flush'
RSS at start: 4552, after mmap: 106952, after mmap+set label: 209356, after fixed map: 4556, after another mmap+set label: 209356, after munmap: 4556
/usr/bin/ld: warning: Cannot export local symbol 'dfsan_flush'
RSS at start: 4552, after mmap: 106952, after mmap+set label: 313836, after fixed map: 6636, after another mmap+set label: 313836, after munmap: 6636
/usr/bin/ld: warning: Cannot export local symbol 'dfsan_flush'
RSS at start: 4552, after mmap: 106952, after mmap+set label: 317924, after fixed map: 10724, after another mmap+set label: 317924, after munmap: 10724
release_shadow_space.c.tmp: /b/sanitizer-x86_64-linux/build/llvm-project/compiler-rt/test/dfsan/release_shadow_space.c:91: int main(int, char **): Assertion `after_fixed_mmap <= before + delta' failed.

• It looks like the origin chain data is more like 6636-4556 = 2080kb
• The delta should not be removed, there is some variation even in the first two executions (no origin chain involved).
• The delta can be fairly large 10724-6636=4088kb (possibly larger in some cases, so keeping at 5000 seems reasonable)

[browneee]

Reduce estimate for origin chain size.

[vitalybuka]

Sorry, just noticed this.

compiler-rt/test/dfsan/release_shadow_space.c

int main(int argc, char **argv) {
   fprintf(
       stderr,
       "RSS at start: %zu, after mmap: %zu, after mmap+set label: %zu, after "
       "fixed map: %zu, after another mmap+set label: %zu, after munmap: %zu\n",
       before, after_mmap, after_mmap_and_set_label, after_fixed_mmap,
       after_mmap_and_set_label2, after_munmap);
 
   const size_t mmap_cost_kb = map_size >> 10;
+  // Shadow space (1:1 with application memory)
   const size_t mmap_shadow_cost_kb = sizeof(dfsan_label) * mmap_cost_kb;
 #ifdef ORIGIN_TRACKING
+  // Origin space (1:1 with application memory)
   const size_t mmap_origin_cost_kb = mmap_cost_kb;
 #else
   const size_t mmap_origin_cost_kb = 0;
 #endif
   assert(after_mmap >= before + mmap_cost_kb);
   assert(after_mmap_and_set_label >=
          after_mmap + mmap_shadow_cost_kb + mmap_origin_cost_kb);
   assert(after_mmap_and_set_label2 >=
          before + mmap_cost_kb + mmap_shadow_cost_kb + mmap_origin_cost_kb);
 
+#ifdef ORIGIN_TRACKING
+  // Origin chain (sanitizer PersistentAllocator, never freed).
+  // This value is chosen based on observed difference.
+  const size_t mmap_origin_chain_kb = 4000;
+#else
+  const size_t mmap_origin_chain_kb = 0;
+#endif
+
   // RSS may not change memory amount after munmap to the same level as the
   // start of the program. The assert checks the memory up to a delta.
   const size_t delta = 5000;
-  assert(after_fixed_mmap <= before + delta);
-  assert(after_munmap <= before + delta);
+  // Origin chains are not freed, even when the origin space which refers to
+  // them is freed, so mmap_origin_chain_kb is added to account for this.
+  assert(after_fixed_mmap <= before + delta + mmap_origin_chain_kb);
+  assert(after_munmap <= before + delta + mmap_origin_chain_kb);
 
   return 0;
 }