Page MenuHomePhabricator

[libFuzzer] Fix value-profile-load test.
ClosedPublic

Authored by dokyungs on Aug 19 2020, 2:14 PM.

Details

Summary

value-profile-load.test needs adjustment with a mutator change in
bb54bcf84970c04c9748004f3a4cf59b0c1832a7, which reverted as of now, but will be
recommitted after landing this patch.

This patch makes value-profile-load.test more friendly to (and aware of) the
current value profiling strategy, which is based on the hamming as well as the
absolute distance. To this end, this patch adjusts the set of input values that
trigger an expected crash. More specifically, this patch now uses a single value
0x01effffe as a crashing input, because this value is close to values like
{0x1ffffff, 0xffffff, ...}, which are very likely to be added to the corpus per
the current hamming- and absolute-distance-based value profiling strategy. Note
that previously the crashing input values were {1234568 * {1, 2, ...} - 1,
s.t. < INT_MAX}.

Every byte in the chosen value 0x01effeef is intentionally different; this was
to make it harder to find the value without the intermediate inputs added to the
corpus by the value profiling strategy.

Also note that LoadTest.cpp now uses a narrower condition (Size != 8) for
initial pruning of inputs, effectively preventing libFuzzer from generating
inputs longer than necessary and spending time on mutating such long inputs in
the corpus - a functionality not meant to be tested by this specific test.

Diff Detail

Event Timeline

dokyungs created this revision.Aug 19 2020, 2:14 PM
Herald added a project: Restricted Project. · View Herald TranscriptAug 19 2020, 2:14 PM
Herald added subscribers: Restricted Project, dexonsmith. · View Herald Transcript
dokyungs requested review of this revision.Aug 19 2020, 2:14 PM
This revision is now accepted and ready to land.Aug 19 2020, 2:35 PM
This revision was automatically updated to reflect the committed changes.
vitalybuka reopened this revision.Aug 20 2020, 5:47 PM
vitalybuka added a subscriber: vitalybuka.

Fails on Windows

FAIL: libFuzzer :: value-profile-load.test (127 of 127)
******************** TEST 'libFuzzer :: value-profile-load.test' FAILED ********************
Script:
--
: 'RUN: at line 2';     C:/b/slave/sanitizer-windows/build/stage1/./bin/clang.exe  --driver-mode=g++ -O2 -gline-tables-only -fsanitize=address,fuzzer -IC:/b/slave/sanitizer-windows/llvm-project/compiler-rt\lib\fuzzer  C:\b\slave\sanitizer-windows\llvm-project\compiler-rt\test\fuzzer/LoadTest.cpp -fsanitize-coverage=trace-gep -o C:\b\slave\sanitizer-windows\build\stage1\projects\compiler-rt\test\fuzzer\X86_64DefaultWindowsConfig\Output\value-profile-load.test.tmp-LoadTest
: 'RUN: at line 3';   not  C:\b\slave\sanitizer-windows\build\stage1\projects\compiler-rt\test\fuzzer\X86_64DefaultWindowsConfig\Output\value-profile-load.test.tmp-LoadTest -seed=2 -use_cmp=0 -use_value_profile=1 -runs=20000000 2>&1 | FileCheck C:\b\slave\sanitizer-windows\llvm-project\compiler-rt\test\fuzzer\value-profile-load.test
--
Exit Code: 1

Command Output (stdout):
--
$ ":" "RUN: at line 2"
$ "C:/b/slave/sanitizer-windows/build/stage1/./bin/clang.exe" "--driver-mode=g++" "-O2" "-gline-tables-only" "-fsanitize=address,fuzzer" "-IC:/b/slave/sanitizer-windows/llvm-project/compiler-rt\lib\fuzzer" "C:\b\slave\sanitizer-windows\llvm-project\compiler-rt\test\fuzzer/LoadTest.cpp" "-fsanitize-coverage=trace-gep" "-o" "C:\b\slave\sanitizer-windows\build\stage1\projects\compiler-rt\test\fuzzer\X86_64DefaultWindowsConfig\Output\value-profile-load.test.tmp-LoadTest"
# command output:
   Creating library C:\b\slave\sanitizer-windows\build\stage1\projects\compiler-rt\test\fuzzer\X86_64DefaultWindowsConfig\Output\value-profile-load.test.lib and object C:\b\slave\sanitizer-windows\build\stage1\projects\compiler-rt\test\fuzzer\X86_64DefaultWindowsConfig\Output\value-profile-load.test.exp


$ ":" "RUN: at line 3"
$ "not" "C:\b\slave\sanitizer-windows\build\stage1\projects\compiler-rt\test\fuzzer\X86_64DefaultWindowsConfig\Output\value-profile-load.test.tmp-LoadTest" "-seed=2" "-use_cmp=0" "-use_value_profile=1" "-runs=20000000"
note: command had no output on stdout or stderr
error: command failed with exit status: 1
$ "FileCheck" "C:\b\slave\sanitizer-windows\llvm-project\compiler-rt\test\fuzzer\value-profile-load.test"
# command stderr:
C:\b\slave\sanitizer-windows\llvm-project\compiler-rt\test\fuzzer\value-profile-load.test:1:8: error: CHECK: expected string not found in input

CHECK: AddressSanitizer: global-buffer-overflow

       ^

<stdin>:1:1: note: scanning from here

INFO: Seed: 2

^

<stdin>:5:27: note: possible intended match here

INFO: A corpus is not provided, starting from an empty corpus

                          ^



Input file: <stdin>

Check file: C:\b\slave\sanitizer-windows\llvm-project\compiler-rt\test\fuzzer\value-profile-load.test



-dump-input=help explains the following input dump.



Input was:

<<<<<<

           1: INFO: Seed: 2

check:1'0     X~~~~~~~~~~~~ error: no match found

           2: INFO: Loaded 1 modules (3 inline 8-bit counters): 3 [00007FF790936108, 00007FF79093610B), 

check:1'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

           3: INFO: Loaded 1 PC tables (3 PCs): 3 [00007FF7908D6730,00007FF7908D6760), 

check:1'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

           4: INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes

check:1'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

           5: INFO: A corpus is not provided, starting from an empty corpus

check:1'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

check:1'1                               ?                                   possible intended match

           6: #2 INITED cov: 2 ft: 4 corp: 1/1b exec/s: 0 rss: 66Mb

check:1'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

           7: #11 NEW cov: 2 ft: 5 corp: 2/4b lim: 4 exec/s: 0 rss: 67Mb L: 3/3 MS: 4 CopyPart-ChangeByte-ChangeBit-CrossOver-

check:1'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

           8: #219 NEW cov: 2 ft: 6 corp: 3/10b lim: 6 exec/s: 0 rss: 67Mb L: 6/6 MS: 3 ChangeBit-InsertByte-CopyPart-

check:1'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

           9: #246 REDUCE cov: 2 ft: 6 corp: 3/9b lim: 6 exec/s: 0 rss: 67Mb L: 5/5 MS: 2 ChangeByte-EraseBytes-

check:1'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

          10: #455 REDUCE cov: 3 ft: 11 corp: 4/17b lim: 8 exec/s: 0 rss: 67Mb L: 8/8 MS: 4 ChangeByte-CopyPart-InsertByte-CopyPart-

check:1'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

           .

           .

           .

>>>>>>


error: command failed with exit status: 1

--

********************
Testing:  0.. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90.. 
********************
Failed Tests (1):
  libFuzzer :: value-profile-load.test


Testing Time: 355.72s
  Unsupported: 41
  Passed     : 85
  Failed     :  1
llvm-lit.py: C:/b/slave/sanitizer-windows/llvm-project/compiler-rt/test/fuzzer/lit.cfg.py:32: note: lsan feature unavailable
llvm-lit.py: C:/b/slave/sanitizer-windows/llvm-project/compiler-rt/test/fuzzer/lit.cfg.py:40: note: msan feature unavailable
llvm-lit.py: C:/b/slave/sanitizer-windows/llvm-project/compiler-rt/test/fuzzer/lit.cfg.py:58: note: linux feature unavailable
[98/105] Generating dynamic/Asan-x86_64-calls-Dynamic-Test.exe
FAILED: projects/compiler-rt/test/fuzzer/CMakeFiles/check-fuzzer-default-x86_64 
cmd.exe /C "cd /D C:\b\slave\sanitizer-windows\build\stage1\projects\compiler-rt\test\fuzzer && C:\Python27\python.exe C:/b/slave/sanitizer-windows/build/stage1/./bin/llvm-lit.py -sv -j 16 C:/b/slave/sanitizer-windows/build/stage1/projects/compiler-rt/test/fuzzer/X86_64DefaultWindowsConfig/"
ninja: build stopped: subcommand failed.
Command 'ninja' failed with return code 1


--------------------------------------------------------------------------------
started: Thu Aug 20 17:35:18 2020
ended: Thu Aug 20 17:44:24 2020
duration: 9 mins, 6 secs
This revision is now accepted and ready to land.Aug 20 2020, 5:47 PM
dokyungs updated this revision to Diff 287121.Aug 21 2020, 3:07 PM

Change of the crashing input.

dokyungs edited the summary of this revision. (Show Details)Aug 21 2020, 3:08 PM
morehouse accepted this revision.Aug 21 2020, 4:58 PM
morehouse added inline comments.
compiler-rt/test/fuzzer/LoadTest.cpp
21

Why is the mask necessary when we're already doing a mod?

dokyungs added inline comments.Aug 24 2020, 9:03 AM
compiler-rt/test/fuzzer/LoadTest.cpp
21

I added this mask for the following two reasons:

(i) to increase chances of failure when -use_value_profile=1 is not given: only a single value is a crashing input with the mask, and it is harder for the fuzzer to find that exact value without GEP value profiling.

and (ii) to increase chances of success (i.e., to lower the # of trials required) with GEP value profiling: the mask limits the range of values of GEP to 0 to 0x1ffff. This effectively prevents dead-end inputs (such as 0xffffffff which are added because it has a unique hamming- and absolute-distance) from being added to the corpus, reducing time spent to fuzz such dead-end inputs.

morehouse added inline comments.Aug 24 2020, 9:50 AM
compiler-rt/test/fuzzer/LoadTest.cpp
21

Re (i): I think this isn't quite true. In fact, there's probably more inputs that cause the crash with the mask, since the top 4 bytes can be anything now.

Re (ii): This makes sense. We're effectively halving the search space for value profiling, since the top 4 bytes no longer affect the GEP.

dokyungs added inline comments.Aug 24 2020, 3:09 PM
compiler-rt/test/fuzzer/LoadTest.cpp
21

Oh you are right, I was misled. As you said, we are reducing the search space for -use_value_profile=1. Some concrete numbers:

If -use_value_profile=1:
Without masking, it takes 26,881,975 execs. With masking, it's reduced to 1,014,478 execs. (This is the number of execs required for this test to pass.)

If -use_value_profile=0:
Without masking, it takes 76,275,283 execs. With masking, it takes 168,433,617 execs.

dokyungs marked 2 inline comments as done.Aug 25 2020, 12:47 PM
dokyungs updated this revision to Diff 288411.Aug 27 2020, 11:54 AM

Use original -runs=20000000, to reduce possiblity of failure on other platforms.

This revision was landed with ongoing or failed builds.Aug 27 2020, 12:12 PM
This revision was automatically updated to reflect the committed changes.