[libFuzzer] Increase timeouts on fork tests and skip one on aarch64

[peter.smith]

Summary

The tests fork.text, fork.sigusr.test and fork-ubsan.test intermittently fail on the aarch64 buildbots. Input gathered from the fork.sigusr.test implies that when the builder is under load the sleep and timeout values used in fork.sigusr.test and fork-ubsan.test are not sufficient. This change doubles the sleep in fork.sigusr.test and the timeout value in fork.test The fork-ubsan.test doesn't have a timeout and I think is not always finding the error after 10000 runs so I've marked as unsupported on aarch64 for now.

The only useful input I got from the --dump-input-on-failure was from fork.sigusr.test, it was all of the form:

/home/buildslave/buildslave/clang-cmake-aarch64-full/llvm/projects/compiler-rt/test/fuzzer/fork-sigusr.test:14:8: error: CHECK: expected string not found in input
CHECK: libFuzzer: {{.*}}exiting
       ^
<stdin>:1:1: note: scanning from here
INFO: Seed: 442182642
^
<stdin>:5:71: note: possible intended match here
INFO: -fork=3: 0 seed inputs, starting to fuzz in /tmp/lit_tmp_2nvgvn/libFuzzerTemp.31336.dir
                                                                      ^

Input file: <stdin>
Check file: /home/buildslave/buildslave/clang-cmake-aarch64-full/llvm/projects/compiler-rt/test/fuzzer/fork-sigusr.test

-dump-input=help describes the format of the following dump.

Full input was:
<<<<<<
            1: INFO: Seed: 442182642
check:14'0     X~~~~~~~~~~~~~~~~~~~~
            2: INFO: Loaded 1 modules (7 inline 8-bit counters): 7 [0x540ec0, 0x540ec7), 
check:14'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            3: INFO: Loaded 1 PC tables (7 PCs): 7 [0x50f098,0x50f108), 
check:14'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            4: INFO: -fork=3: fuzzing in separate process(s)
check:14'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            5: INFO: -fork=3: 0 seed inputs, starting to fuzz in /tmp/lit_tmp_2nvgvn/libFuzzerTemp.31336.dir
check:14'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
check:14'1                                                                           ?                       possible intended match
            6: INFO: signal received, trying to exit gracefully
check:14'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ error: no match found
>>>>>>

The next line should be

31336== INFO: libFuzzer: exiting as requested

The last output in the file is from Fuzzer::MaybeExitGracefully() which sets a flag that is later checked to see if the loop should exit and print the final exiting as requested, it looks like this could have at least some delay.

If increasing the timeouts isn't acceptable I suggest marking all the fork tests as unsupported on aarch64

[morehouse]

LGTM

test/fuzzer/fork-sigusr.test

 # Check that libFuzzer honors SIGUSR1/SIGUSR2
 # Disabled on Windows which does not have SIGUSR1/SIGUSR2.
 UNSUPPORTED: darwin, windows
 RUN: rm -rf %t
 RUN: mkdir -p %t
 RUN: %cpp_compiler %S/ShallowOOMDeepCrash.cpp -o %t/ForkSIGUSR
 
 RUN: %run %t/ForkSIGUSR -fork=3 -rss_limit_mb=128 -ignore_crashes=1 2>  %t/log & export PID=$!
 RUN: sleep 3
 RUN: pkill -SIGUSR2 -f %t/ForkSIGUSR
-RUN: sleep 3
-RUN: cat %t/log | FileCheck %s --dump-input-on-failure
+RUN: sleep 6
+RUN: cat %t/log | FileCheck %s
 
 CHECK: libFuzzer: {{.*}}exiting

test/fuzzer/fork-ubsan.test

-# UNSUPPORTED: darwin, freebsd
+# UNSUPPORTED: darwin, freebsd, aarch64
 # Tests how the fork mode works together with ubsan.
 RUN: %cpp_compiler %S/IntegerOverflowTest.cpp -o %t-IntegerOverflowTest -fsanitize=signed-integer-overflow -fno-sanitize-recover=signed-integer-overflow
 RUN: not %run %t-IntegerOverflowTest -fork=1 -ignore_crashes=1  -runs=10000 2>&1 | FileCheck %s --check-prefix=UBSAN_FORK
 UBSAN_FORK: runtime error: signed integer overflow: 1073741824 + 1073741824 cannot be represented in type 'int'
 UBSAN_FORK: INFO: fuzzed for {{.*}} iterations, wrapping up soon

test/fuzzer/fork.test

 # UNSUPPORTED: darwin, freebsd
 BINGO: BINGO
 RUN: %cpp_compiler %S/SimpleTest.cpp -o %t-SimpleTest
-RUN: not %run %t-SimpleTest -fork=1 2>&1 | FileCheck %s --dump-input-on-failure --check-prefix=BINGO
+RUN: not %run %t-SimpleTest -fork=1 2>&1 | FileCheck %s --check-prefix=BINGO
 
 TIMEOUT: ERROR: libFuzzer: timeout
 RUN: %cpp_compiler %S/TimeoutTest.cpp -o %t-TimeoutTest
-RUN: not %run %t-TimeoutTest -fork=1 -timeout=1 -ignore_timeouts=0 2>&1 | FileCheck %s --dump-input-on-failure --check-prefix=TIMEOUT
+RUN: not %run %t-TimeoutTest -fork=1 -timeout=1 -ignore_timeouts=0 2>&1 | FileCheck %s --check-prefix=TIMEOUT
 
 OOM: ERROR: libFuzzer: out-of-memory
 RUN: %cpp_compiler %S/OutOfMemoryTest.cpp -o %t-OutOfMemoryTest
-RUN: not %run %t-OutOfMemoryTest -fork=1 -ignore_ooms=0  -rss_limit_mb=128 2>&1 | FileCheck %s --dump-input-on-failure --check-prefix=OOM
+RUN: not %run %t-OutOfMemoryTest -fork=1 -ignore_ooms=0  -rss_limit_mb=128 2>&1 | FileCheck %s --check-prefix=OOM
 
 # access-violation is the error thrown on Windows. Address will be smaller on i386.
 CRASH: {{SEGV|access-violation}} on unknown address 0x00000000
 RUN: %cpp_compiler %S/ShallowOOMDeepCrash.cpp -o %t-ShallowOOMDeepCrash
-RUN: not %run %t-ShallowOOMDeepCrash -fork=1 -rss_limit_mb=128 2>&1 | FileCheck %s --dump-input-on-failure --check-prefix=CRASH
+RUN: not %run %t-ShallowOOMDeepCrash -fork=1 -rss_limit_mb=128 2>&1 | FileCheck %s --check-prefix=CRASH
 
 MAX_TOTAL_TIME: INFO: fuzzed for {{.*}} seconds, wrapping up soon
 MAX_TOTAL_TIME: INFO: exiting: {{.*}} time:
-RUN: not %run %t-ShallowOOMDeepCrash -fork=1 -rss_limit_mb=128 -ignore_crashes=1 -max_total_time=5 2>&1 | FileCheck %s --dump-input-on-failure --check-prefix=MAX_TOTAL_TIME
+RUN: not %run %t-ShallowOOMDeepCrash -fork=1 -rss_limit_mb=128 -ignore_crashes=1 -max_total_time=10 2>&1 | FileCheck %s --dump-input-on-failure --check-prefix=MAX_TOTAL_TIME