This is an archive of the discontinued LLVM Phabricator instance.

Add FuzzedDataProvider helper class / single header library.
ClosedPublic

Authored by Dor1s on May 31 2019, 7:09 AM.

Details

Summary

This class is useful for writing fuzz target that have multiple inputs.

Current CL imports the existing FuzzedDataProvider from Chromium
without any modifications. Feel free to review it thoroughly, if you're
interested, but I'd prefer changing the class in a follow up CL.

The CL also introduces an exhaustive test for the library, as the behavior
of FuzzedDataProvider must not change over time.

In follow up CLs I'm planning on changing some implementation details
(I can share a doc with some comments to be addressed). After that, we
will document how FuzzedDataProvider should be used.

I have tested this on Linux, Windows and Mac platforms.

Event Timeline

Dor1s created this revision.May 31 2019, 7:09 AM
Herald added projects: Restricted Project, Restricted Project. · View Herald TranscriptMay 31 2019, 7:09 AM
Herald added subscribers: llvm-commits, Restricted Project, delcypher. · View Herald Transcript
Dor1s updated this revision to Diff 203584.Jun 7 2019, 11:11 AM

Add the tests.

Dor1s updated this revision to Diff 203585.Jun 7 2019, 11:12 AM

Remove commented line

Dor1s retitled this revision from [WORK IN PROGRESS] Add FuzzedDataProvider helper class / single header library. to Add FuzzedDataProvider helper class / single header library..Jun 7 2019, 11:21 AM
Dor1s edited the summary of this revision. (Show Details)
Dor1s added reviewers: morehouse, metzman, kcc.
Dor1s edited the summary of this revision. (Show Details)Jun 7 2019, 11:23 AM
Dor1s updated this revision to Diff 203673.Jun 7 2019, 11:23 PM

Workaround Windows-specific linking issue.

Dor1s edited the summary of this revision. (Show Details)Jun 7 2019, 11:27 PM
Dor1s added subscribers: rnk, thakis.
Dor1s added a subscriber: metzman.Jun 7 2019, 11:36 PM

Nico, Reid, Jonathan, I'd greatly appreciate your Windows expertise here. If I remove lines 11-16 from lib/fuzzer/tests/FuzzedDataProviderUnittest.cpp, I'm getting the following link errors on Windows:

C:\src\llvm\build>ninja -v check-fuzzer-unit
[1/3] cmd.exe /C "cd /D C:\src\llvm\build\projects\compiler-rt\lib\fuzzer\tests && C:\src\llvm\build\.\bin\clang.exe -DWIN32 -D_WINDOWS -Wno-unknown-warning-option -DGTEST_NO_LLVM_RAW_OSTREAM=1 -DGTEST_HAS_RTTI=0 -IC:/src/llvm/llvm/utils/unittest/googletest/include -IC:/src/llvm/llvm/utils/unittest/googletest -Wno-deprecated-declarations -IC:/src/llvm/llvm/projects/compiler-rt/lib/fuzzer -fno-rtti -O2 -c -o FuzzedDataProviderTestObjects.FuzzedDataProviderUnittest.cpp.x86_64.o C:/src/llvm/llvm/projects/compiler-rt/lib/fuzzer/tests/FuzzedDataProviderUnittest.cpp"
[2/3] cmd.exe /C "cd /D C:\src\llvm\build\projects\compiler-rt\lib\fuzzer\tests && C:\src\llvm\build\.\bin\clang.exe FuzzedDataProviderTestObjects.FuzzedDataProviderUnittest.cpp.x86_64.o FuzzedDataProviderTestObjects.gtest-all.cc.x86_64.o C:/src/llvm/build/projects/compiler-rt/lib/fuzzer/tests/RTFuzzerTest.x86_64.lib -o C:/src/llvm/build/projects/compiler-rt/lib/fuzzer/tests/./FuzzerUtils-x86_64-Test.exe --driver-mode=g++"
FAILED: projects/compiler-rt/lib/fuzzer/tests/FuzzerUtils-x86_64-Test.exe
cmd.exe /C "cd /D C:\src\llvm\build\projects\compiler-rt\lib\fuzzer\tests && C:\src\llvm\build\.\bin\clang.exe FuzzedDataProviderTestObjects.FuzzedDataProviderUnittest.cpp.x86_64.o FuzzedDataProviderTestObjects.gtest-all.cc.x86_64.o C:/src/llvm/build/projects/compiler-rt/lib/fuzzer/tests/RTFuzzerTest.x86_64.lib -o C:/src/llvm/build/projects/compiler-rt/lib/fuzzer/tests/./FuzzerUtils-x86_64-Test.exe --driver-mode=g++"
   Creating library C:/src/llvm/build/projects/compiler-rt/lib/fuzzer/tests/./FuzzerUtils-x86_64-Test.lib and object C:/src/llvm/build/projects/compiler-rt/lib/fuzzer/tests/./FuzzerUtils-x86_64-Test.exp
FuzzedDataProviderTestObjects.gtest-all.cc.x86_64.o : error LNK2019: unresolved external symbol fdopen referenced in function "void __cdecl testing::internal::DeathTestAbort(class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const &)" (?DeathTestAbort@internal@testing@@YAXAEBV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@Z)
FuzzedDataProviderTestObjects.gtest-all.cc.x86_64.o : error LNK2019: unresolved external symbol read referenced in function "protected: void __cdecl testing::internal::DeathTestImpl::ReadAndInterpretStatusByte(void)" (?ReadAndInterpretStatusByte@DeathTestImpl@internal@testing@@IEAAXXZ)
FuzzedDataProviderTestObjects.gtest-all.cc.x86_64.o : error LNK2019: unresolved external symbol close referenced in function "protected: void __cdecl testing::internal::DeathTestImpl::ReadAndInterpretStatusByte(void)" (?ReadAndInterpretStatusByte@DeathTestImpl@internal@testing@@IEAAXXZ)
FuzzedDataProviderTestObjects.gtest-all.cc.x86_64.o : error LNK2019: unresolved external symbol write referenced in function "protected: virtual void __cdecl testing::internal::DeathTestImpl::Abort(enum testing::internal::DeathTest::AbortReason)" (?Abort@DeathTestImpl@internal@testing@@MEAAXW4AbortReason@DeathTest@23@@Z)
FuzzedDataProviderTestObjects.gtest-all.cc.x86_64.o : error LNK2019: unresolved external symbol dup referenced in function "public: __cdecl testing::internal::CapturedStream::CapturedStream(int)" (??0CapturedStream@internal@testing@@QEAA@H@Z)
FuzzedDataProviderTestObjects.gtest-all.cc.x86_64.o : error LNK2019: unresolved external symbol creat referenced in function "public: __cdecl testing::internal::CapturedStream::CapturedStream(int)" (??0CapturedStream@internal@testing@@QEAA@H@Z)
FuzzedDataProviderTestObjects.gtest-all.cc.x86_64.o : error LNK2019: unresolved external symbol dup2 referenced in function "public: __cdecl testing::internal::CapturedStream::CapturedStream(int)" (??0CapturedStream@internal@testing@@QEAA@H@Z)
C:/src/llvm/build/projects/compiler-rt/lib/fuzzer/tests/./FuzzerUtils-x86_64-Test.exe : fatal error LNK1120: 7 unresolved externals
clang: error: linker command failed with exit code 1120 (use -v to see invocation)
ninja: build stopped: subcommand failed.

To me the issue looks like libc (or whatever equivalent it has on Windows) isn't specified as a gtest dependency, that's why symbols like read, write, or dup2 are unresolved. It also looks like libFuzzer has the correct dependencies specified on Windows, and that's why everything works fine if my FuzzedDataProviderUnittest depends on libFuzzer (and actually uses it). However, I was not able to track this down and find any magic dependency that I should specify in the CMakeLists.txt for FuzzedDataProviderUnittest.

Does this look familiar at all? My FuzzedDataProviderUnittest doesn't really need libFuzzer (aka RTFuzzer), it actually needs only lib/fuzzer/utils/FuzzedDataProvider.h and gtest.

rnk added a comment.Jun 10 2019, 11:04 AM

Nico, Reid, Jonathan, I'd greatly appreciate your Windows expertise here. If I remove lines 11-16 from lib/fuzzer/tests/FuzzedDataProviderUnittest.cpp, I'm getting the following link errors on Windows:

Typically the CRT gets pulled into the link by /defaultlib:libcmt flags in the .drective section of the object file. Objects compiled with clang-cl and cl will have these directives, but objects compiled by plain clang will not, because clang has no flag that the user can use to indicate whether the static or dynamic CRT should be used (/MT vs /MD). That spelling obviously conflicts with the .d file related options -MD, and we never invented a GCC-style spelling, so it doesn't exist, and we also didn't try guessing which CRT is in use. I think that's what's going on here: objects in RTFuzzer are compiled by MSVC, but the unittests build configuration uses clang because they were designed to test compiler instrumentation.

As a fix, try adding /defaultlib:libcmt to link flags.

In D62733#1536561, @rnk wrote:

As a fix, try adding /defaultlib:libcmt to link flags.

Reid, thanks so much for taking a look. Adding -Wl,-defaultlib:libcmt,-defaultlib:oldnames indeed does the trick! I'll update the patch to remove that hacky dependency on RTFuzzer.

Dor1s updated this revision to Diff 203878.Jun 10 2019, 12:47 PM

Fix linking on Windows, get rid of unnecessary RTFuzzer dependency.

Dor1s added a comment.Jun 10 2019, 1:09 PM

Yay, the current version works across Linux / Windows / Mac without any hacks. Thanks again, Reid!

Matt and Kostya, I need your approval on the directory structure and CMake changes. Reviewing the tests / library is not necessary, but would be helpful of course :)

morehouse accepted this revision.Jun 10 2019, 6:23 PM
morehouse added inline comments.
lib/fuzzer/tests/CMakeLists.txt
26 ↗(On Diff #203878)

Nit: Let's invert the if-condition:

if(WIN32) 
  ...
else()
 ...
This revision is now accepted and ready to land.Jun 10 2019, 6:23 PM
Dor1s updated this revision to Diff 204063.Jun 11 2019, 7:26 AM

Fix CMakeLists.txt as per Matt's comment

Dor1s marked an inline comment as done.Jun 11 2019, 7:26 AM
This revision was automatically updated to reflect the committed changes.