[asan] Reduce flakiness in stack-overflow detection

[kubamracek]

Summary

IsStackOverflow only treats accesses within 512 bytes of SP as stack-overflow. This should really be the size of a page instead.

The scariness_score_test.cc triggers stack overflow with frames that are even larger than a page, which can also trigger a fault that will not be recognized as stack-overflow. Let's just use smaller frames.

[george.karpenkov]

LGTM, but probably someone else would need to check this as well.

[eugenis]

Out of curiosity, how does access that far below SP happen in that test? Redzone? But that's not a leaf function.

[kubamracek]

In D45509#1064637, @eugenis wrote:

Out of curiosity, how does access that far below SP happen in that test? Redzone? But that's not a leaf function.

This is related to stack protectors and stack probing, see https://reviews.llvm.org/D40856. When stack probing is enabled, we make an access to the end of the frame before SP is updated.

[eugenis]

LGTM, please add a comment though

[kubamracek]

Landed in r329980.

lib/sanitizer_common/sanitizer_posix_libcdep.cc

bool SignalContext::IsStackOverflow() const {
   // for x86_64 or PowerPC redzone, ARM push of multiple registers, etc) is
   // probably a stack overflow.
 #ifdef __s390__
   // On s390, the fault address in siginfo points to start of the page, not
   // to the precise word that was accessed.  Mask off the low bits of sp to
   // take it into account.
   bool IsStackAccess = addr >= (sp & ~0xFFF) && addr < sp + 0xFFFF;
 #else
-  bool IsStackAccess = addr + 512 > sp && addr < sp + 0xFFFF;
+  bool IsStackAccess = addr + GetPageSizeCached() > sp && addr < sp + 0xFFFF;
 #endif
 
 #if __powerpc__
   // Large stack frames can be allocated with e.g.
   //   lis r0,-10000
   //   stdux r1,r1,r0 # store sp to [sp-10000] and update sp by -10000
   // If the store faults then sp will not have been updated, so test above
   // will not work, because the fault address will be more than just "slightly"

test/asan/TestCases/scariness_score_test.cc

 void DoubleFree() {
   int *x = new int;
   static volatile int two = 2;
   for (int i = 0; i < two; i++)
     delete x;
 }
 
 void StackOverflow(int Idx) {
-  int some_stack[10000];
+  int some_stack[256];
   static volatile int *x;
   x = &some_stack[0];
   if (Idx > 0)
     StackOverflow(Idx - 1);
 }
 
 void UseAfterPoison() {
   int buf[100];