This is an archive of the discontinued LLVM Phabricator instance.

Add functions to encode/decode feature files
AbandonedPublic

Authored by aarongreen on Jan 12 2021, 9:23 AM.

Details

Summary

This change adds EncodeFeatures and DecodeFeatures to FuzzerFork. These are fairly vacuous in this change: features files are simply sorted lists of unsigned, 32 bit integers. The addition of these methods and their associated tests helps encapsulate the feature file format, allowing subsequent changes to it.

This is change 3 of (at least) 18 for cross-process fuzzing support

Diff Detail

Event Timeline

aarongreen created this revision.Jan 12 2021, 9:23 AM
aarongreen requested review of this revision.Jan 12 2021, 9:23 AM
Herald added a project: Restricted Project. · View Herald TranscriptJan 12 2021, 9:23 AM
Herald added a subscriber: Restricted Project. · View Herald Transcript
charco added inline comments.Jan 20 2021, 6:58 PM
compiler-rt/lib/fuzzer/FuzzerFork.cpp
197

What happens if the features here are not sorted?

compiler-rt/lib/fuzzer/FuzzerFork.h
19

empty line

compiler-rt/lib/fuzzer/tests/FuzzerUnittest.cpp
1318

Why did you have to add this?

aarongreen added inline comments.Feb 3 2021, 11:14 AM
compiler-rt/lib/fuzzer/FuzzerFork.cpp
197

DecodeFeatures will return false, and the FeatureFile will be skipped. This shouldn't happen if EncodeFeatures was used to write the FeatureFile in the first place.

compiler-rt/lib/fuzzer/FuzzerFork.h
19

Intentional. Most of the header files have a blank lines separating the 'namespace' lines from other code, and clang-format allows it.

compiler-rt/lib/fuzzer/tests/FuzzerUnittest.cpp
1318

In this change, the rationale is to ensure srand(0) is called and the tests using pseudorandom values are deterministic. Unit tests should *always* be deterministic!

Later, this is reused to instantiate the fuzzer::EF global in a way that prevents heap use-after-frees when running with --gtest_repeat=N for N > 1. The unit tests were broken when using this feature, but repeating tests was absolutely critical to hunting down flake in later changes adding the IPC layer.

morehouse accepted this revision.Feb 4 2021, 8:38 AM
morehouse added inline comments.
compiler-rt/lib/fuzzer/tests/FuzzerTestUtil.h
44
67

Nit: Let's use a single naming convention, either "Stop" or "End"

compiler-rt/lib/fuzzer/tests/FuzzerUnittest.cpp
702

What does EXPECT_NO_FATAL_FAILURE do here? How is this line different from just CheckEncodedFeatures(...)?

This revision is now accepted and ready to land.Feb 4 2021, 8:38 AM
aarongreen updated this revision to Diff 321575.Feb 4 2021, 3:02 PM
aarongreen marked an inline comment as done.

Address morehouse's comments.

compiler-rt/lib/fuzzer/tests/FuzzerUnittest.cpp
702

This is me juggling too many changes simultaneously. When the features files become DSO-relative, CheckEncodedFeatures gets a few ASSERT_...(...) statements. Until then, you are correct, these aren't needed.

aarongreen marked 2 inline comments as done.
aarongreen abandoned this revision.Sep 1 2021, 9:06 AM
aarongreen marked an inline comment as done.

Multiprocess fuzzing will not be supported by the libFuzzer maintainers. Fuchsia has implemented a new approach with their Component Fuzzing Framework (RFC-117).