[libFuzzer] Fix Out of Memory tests to work on 32 bits.

[mpividori]

Summary

Hi,
I add 2 changes to make the tests work on 32 bits and on 64 bits. I change the size allocated to 0x20000000 and add the flag: -rss_limit_mb=300.
Otherwise the output for 32 bits and 64 bits is different.
For 64 bits the value 0xff000000 doesn't exceed kMaxAllowedMallocSize.
For 32 bits, kMaxAllowedMallocSize is set to 0xc0000000 , so the call to Allocate() will fail earlier printing WARNING: AddressSanitizer failed to allocate ... , and wont't call malloc hooks.
So, we need to consider a size smaller than 2GB (so malloc doesn't fail on 32bits) and greater that the value provided by -rss_limit_mb. Because of that I use: 0x20000000.

[kcc]

LGTM

llvm/trunk/lib/Fuzzer/test/OutOfMemorySingleLargeMallocTest.cpp

 #include <iostream>
 
 static volatile char *SinkPtr;
 
 extern "C" int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) {
   if (Size > 0 && Data[0] == 'H') {
     if (Size > 1 && Data[1] == 'i') {
       if (Size > 2 && Data[2] == '!') {
-          size_t kSize = 0xff000000U;
+          size_t kSize = 0x20000000U;
           char *p = new char[kSize];
           SinkPtr = p;
           delete [] p;
       }
     }
   }
   return 0;
 }
 

llvm/trunk/lib/Fuzzer/test/fuzzer-oom.test

 RUN: not LLVMFuzzer-OutOfMemoryTest -rss_limit_mb=300 2>&1 | FileCheck %s
 CHECK: ERROR: libFuzzer: out-of-memory (used: {{.*}}; limit: 300Mb)
 CHECK: Test unit written to ./oom-
 SUMMARY: libFuzzer: out-of-memory
 
-RUN: not LLVMFuzzer-OutOfMemorySingleLargeMallocTest 2>&1 | FileCheck %s --check-prefix=SINGLE_LARGE_MALLOC
-SINGLE_LARGE_MALLOC: libFuzzer: out-of-memory (malloc(42{{.*}}))
+RUN: not LLVMFuzzer-OutOfMemorySingleLargeMallocTest -rss_limit_mb=300 2>&1 | FileCheck %s --check-prefix=SINGLE_LARGE_MALLOC
+SINGLE_LARGE_MALLOC: libFuzzer: out-of-memory (malloc(53{{.*}}))
 SINGLE_LARGE_MALLOC: in LLVMFuzzerTestOneInput
 
 # Check that -rss_limit_mb=0 means no limit.
 RUN: LLVMFuzzer-AccumulateAllocationsTest -runs=1000 -rss_limit_mb=0