[libFuzzer] Fix value-profile-load test.

[dokyungs]

Summary

value-profile-load.test needs adjustment with a mutator change in
bb54bcf84970c04c9748004f3a4cf59b0c1832a7, which reverted as of now, but will be
recommitted after landing this patch.

This patch makes value-profile-load.test more friendly to (and aware of) the
current value profiling strategy, which is based on the hamming as well as the
absolute distance. To this end, this patch adjusts the set of input values that
trigger an expected crash. More specifically, this patch now uses a single value
0x01effffe as a crashing input, because this value is close to values like
{0x1ffffff, 0xffffff, ...}, which are very likely to be added to the corpus per
the current hamming- and absolute-distance-based value profiling strategy. Note
that previously the crashing input values were {1234568 * {1, 2, ...} - 1,
s.t. < INT_MAX}.

Every byte in the chosen value 0x01effeef is intentionally different; this was
to make it harder to find the value without the intermediate inputs added to the
corpus by the value profiling strategy.

Also note that LoadTest.cpp now uses a narrower condition (Size != 8) for
initial pruning of inputs, effectively preventing libFuzzer from generating
inputs longer than necessary and spending time on mutating such long inputs in
the corpus - a functionality not meant to be tested by this specific test.

[morehouse]

LGTM

[vitalybuka]

Fails on Windows

FAIL: libFuzzer :: value-profile-load.test (127 of 127)
******************** TEST 'libFuzzer :: value-profile-load.test' FAILED ********************
Script:
--
: 'RUN: at line 2';     C:/b/slave/sanitizer-windows/build/stage1/./bin/clang.exe  --driver-mode=g++ -O2 -gline-tables-only -fsanitize=address,fuzzer -IC:/b/slave/sanitizer-windows/llvm-project/compiler-rt\lib\fuzzer  C:\b\slave\sanitizer-windows\llvm-project\compiler-rt\test\fuzzer/LoadTest.cpp -fsanitize-coverage=trace-gep -o C:\b\slave\sanitizer-windows\build\stage1\projects\compiler-rt\test\fuzzer\X86_64DefaultWindowsConfig\Output\value-profile-load.test.tmp-LoadTest
: 'RUN: at line 3';   not  C:\b\slave\sanitizer-windows\build\stage1\projects\compiler-rt\test\fuzzer\X86_64DefaultWindowsConfig\Output\value-profile-load.test.tmp-LoadTest -seed=2 -use_cmp=0 -use_value_profile=1 -runs=20000000 2>&1 | FileCheck C:\b\slave\sanitizer-windows\llvm-project\compiler-rt\test\fuzzer\value-profile-load.test
--
Exit Code: 1

Command Output (stdout):
--
$ ":" "RUN: at line 2"
$ "C:/b/slave/sanitizer-windows/build/stage1/./bin/clang.exe" "--driver-mode=g++" "-O2" "-gline-tables-only" "-fsanitize=address,fuzzer" "-IC:/b/slave/sanitizer-windows/llvm-project/compiler-rt\lib\fuzzer" "C:\b\slave\sanitizer-windows\llvm-project\compiler-rt\test\fuzzer/LoadTest.cpp" "-fsanitize-coverage=trace-gep" "-o" "C:\b\slave\sanitizer-windows\build\stage1\projects\compiler-rt\test\fuzzer\X86_64DefaultWindowsConfig\Output\value-profile-load.test.tmp-LoadTest"
# command output:
   Creating library C:\b\slave\sanitizer-windows\build\stage1\projects\compiler-rt\test\fuzzer\X86_64DefaultWindowsConfig\Output\value-profile-load.test.lib and object C:\b\slave\sanitizer-windows\build\stage1\projects\compiler-rt\test\fuzzer\X86_64DefaultWindowsConfig\Output\value-profile-load.test.exp


$ ":" "RUN: at line 3"
$ "not" "C:\b\slave\sanitizer-windows\build\stage1\projects\compiler-rt\test\fuzzer\X86_64DefaultWindowsConfig\Output\value-profile-load.test.tmp-LoadTest" "-seed=2" "-use_cmp=0" "-use_value_profile=1" "-runs=20000000"
note: command had no output on stdout or stderr
error: command failed with exit status: 1
$ "FileCheck" "C:\b\slave\sanitizer-windows\llvm-project\compiler-rt\test\fuzzer\value-profile-load.test"
# command stderr:
C:\b\slave\sanitizer-windows\llvm-project\compiler-rt\test\fuzzer\value-profile-load.test:1:8: error: CHECK: expected string not found in input

CHECK: AddressSanitizer: global-buffer-overflow

       ^

<stdin>:1:1: note: scanning from here

INFO: Seed: 2

^

<stdin>:5:27: note: possible intended match here

INFO: A corpus is not provided, starting from an empty corpus

                          ^



Input file: <stdin>

Check file: C:\b\slave\sanitizer-windows\llvm-project\compiler-rt\test\fuzzer\value-profile-load.test



-dump-input=help explains the following input dump.



Input was:

<<<<<<

           1: INFO: Seed: 2

check:1'0     X~~~~~~~~~~~~ error: no match found

           2: INFO: Loaded 1 modules (3 inline 8-bit counters): 3 [00007FF790936108, 00007FF79093610B), 

check:1'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

           3: INFO: Loaded 1 PC tables (3 PCs): 3 [00007FF7908D6730,00007FF7908D6760), 

check:1'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

           4: INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes

check:1'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

           5: INFO: A corpus is not provided, starting from an empty corpus

check:1'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

check:1'1                               ?                                   possible intended match

           6: #2 INITED cov: 2 ft: 4 corp: 1/1b exec/s: 0 rss: 66Mb

check:1'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

           7: #11 NEW cov: 2 ft: 5 corp: 2/4b lim: 4 exec/s: 0 rss: 67Mb L: 3/3 MS: 4 CopyPart-ChangeByte-ChangeBit-CrossOver-

check:1'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

           8: #219 NEW cov: 2 ft: 6 corp: 3/10b lim: 6 exec/s: 0 rss: 67Mb L: 6/6 MS: 3 ChangeBit-InsertByte-CopyPart-

check:1'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

           9: #246 REDUCE cov: 2 ft: 6 corp: 3/9b lim: 6 exec/s: 0 rss: 67Mb L: 5/5 MS: 2 ChangeByte-EraseBytes-

check:1'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

          10: #455 REDUCE cov: 3 ft: 11 corp: 4/17b lim: 8 exec/s: 0 rss: 67Mb L: 8/8 MS: 4 ChangeByte-CopyPart-InsertByte-CopyPart-

check:1'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

           .

           .

           .

>>>>>>


error: command failed with exit status: 1

--

********************
Testing:  0.. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90.. 
********************
Failed Tests (1):
  libFuzzer :: value-profile-load.test


Testing Time: 355.72s
  Unsupported: 41
  Passed     : 85
  Failed     :  1
llvm-lit.py: C:/b/slave/sanitizer-windows/llvm-project/compiler-rt/test/fuzzer/lit.cfg.py:32: note: lsan feature unavailable
llvm-lit.py: C:/b/slave/sanitizer-windows/llvm-project/compiler-rt/test/fuzzer/lit.cfg.py:40: note: msan feature unavailable
llvm-lit.py: C:/b/slave/sanitizer-windows/llvm-project/compiler-rt/test/fuzzer/lit.cfg.py:58: note: linux feature unavailable
[98/105] Generating dynamic/Asan-x86_64-calls-Dynamic-Test.exe
FAILED: projects/compiler-rt/test/fuzzer/CMakeFiles/check-fuzzer-default-x86_64 
cmd.exe /C "cd /D C:\b\slave\sanitizer-windows\build\stage1\projects\compiler-rt\test\fuzzer && C:\Python27\python.exe C:/b/slave/sanitizer-windows/build/stage1/./bin/llvm-lit.py -sv -j 16 C:/b/slave/sanitizer-windows/build/stage1/projects/compiler-rt/test/fuzzer/X86_64DefaultWindowsConfig/"
ninja: build stopped: subcommand failed.
Command 'ninja' failed with return code 1


--------------------------------------------------------------------------------
started: Thu Aug 20 17:35:18 2020
ended: Thu Aug 20 17:44:24 2020
duration: 9 mins, 6 secs

[vitalybuka]

Reverted with 66c882e529a388465463e80de93b4f353a9391a8

[dokyungs]

Change of the crashing input.

[morehouse]

compiler-rt/test/fuzzer/LoadTest.cpp

21

Why is the mask necessary when we're already doing a mod?

[dokyungs]

compiler-rt/test/fuzzer/LoadTest.cpp

21

I added this mask for the following two reasons:

(i) to increase chances of failure when -use_value_profile=1 is not given: only a single value is a crashing input with the mask, and it is harder for the fuzzer to find that exact value without GEP value profiling.

and (ii) to increase chances of success (i.e., to lower the # of trials required) with GEP value profiling: the mask limits the range of values of GEP to 0 to 0x1ffff. This effectively prevents dead-end inputs (such as 0xffffffff which are added because it has a unique hamming- and absolute-distance) from being added to the corpus, reducing time spent to fuzz such dead-end inputs.

[morehouse]

compiler-rt/test/fuzzer/LoadTest.cpp

21

Re (i): I think this isn't quite true. In fact, there's probably more inputs that cause the crash with the mask, since the top 4 bytes can be anything now.

Re (ii): This makes sense. We're effectively halving the search space for value profiling, since the top 4 bytes no longer affect the GEP.

[dokyungs]

compiler-rt/test/fuzzer/LoadTest.cpp

21

Oh you are right, I was misled. As you said, we are reducing the search space for -use_value_profile=1. Some concrete numbers:

If -use_value_profile=1:
Without masking, it takes 26,881,975 execs. With masking, it's reduced to 1,014,478 execs. (This is the number of execs required for this test to pass.)

If -use_value_profile=0:
Without masking, it takes 76,275,283 execs. With masking, it takes 168,433,617 execs.

[dokyungs]

Use original -runs=20000000, to reduce possiblity of failure on other platforms.

compiler-rt/test/fuzzer/LoadTest.cpp

 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 
 // Simple test for a fuzzer: find interesting value of array index.
 #include <assert.h>
 #include <cstddef>
 #include <cstdint>
 #include <cstring>
 #include <iostream>
 
-static volatile int Sink;
-const int kArraySize = 1234567;
-int array[kArraySize];
+static volatile uint8_t Sink;
+const int kArraySize = 32505854; // 0x01effffe
+uint8_t array[kArraySize];
 
 extern "C" int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) {
-  if (Size < 8) return 0;
+  if (Size != 8)
+    return 0;
   uint64_t a = 0;
-  memcpy(&a, Data, 8);
+  memcpy(&a, Data, sizeof(a));
+  a &= 0x1fffffff;
   Sink = array[a % (kArraySize + 1)];
   return 0;
 }
-

compiler-rt/test/fuzzer/value-profile-load.test

 CHECK: AddressSanitizer: global-buffer-overflow
 RUN: %cpp_compiler %S/LoadTest.cpp -fsanitize-coverage=trace-gep -o %t-LoadTest
-RUN: not %run %t-LoadTest -seed=2 -use_cmp=0 -use_value_profile=1 -runs=20000000 2>&1 | FileCheck %s
+RUN: not %run %t-LoadTest -seed=1 -use_cmp=0 -use_value_profile=1 -runs=10000000 2>&1 | FileCheck %s