[libFuzzer] Enable entropic by default.

[morehouse]

Summary

Entropic has performed at least on par with vanilla scheduling on
Clusterfuzz, and has shown a slight coverage improvement on FuzzBench:
https://www.fuzzbench.com/reports/2020-08-31/index.html

[Dor1s]

compiler-rt/lib/fuzzer/FuzzerDriver.cpp

771

maybe we should preserve an explicit error in case both focus function and entropic are used? Otherwise it's not obvious and might be even tricky to realize without checking libFuzzer's source code.

[morehouse]

compiler-rt/lib/fuzzer/FuzzerDriver.cpp

771

The reason I took the error out is to avoid -focus-function requiring two flags to use now (i.e. -entropic=0 -focus_function=auto).

I figured it doesn't make much sense to use focus_function with entropic anyway, since -focus_function is essentially its own scheduling algorithm.

[Dor1s]

compiler-rt/lib/fuzzer/FuzzerDriver.cpp

771

Ah, that's right, as now we'd need to disable it explicitly. Let's update the focus_function doc string in such case, pointing out that it disables entropic?

[morehouse]

@dokyungs Any idea why enabling entropic causes the KeepSeed tests to fail?

[dokyungs]

In D87476#2266496, @morehouse wrote:

@dokyungs Any idea why enabling entropic causes the KeepSeed tests to fail?

With -entropic=1, the two tests apparently require more runs.

For cross_over_uniform_dist.test, it requires 3,782,970 runs, so increasing runs to 4,000,000 should solve the issue. Note that without cross_over_uniform_dist=1, it requires 9,278,291 runs.

For keep-seed.test, it requires 2,038,980 runs, so increasing runs to 3,000,000 should solve the issue. Note that without keep_seed=1, it requires even more runs as expected: 9,621,535 runs.

Let me know how this sounds, and also if you want this change to be a separate patch.

[morehouse]

• Update focus_function doc string.
• Increase runs for keep-seed and uniform crossover tests.

[Dor1s]

LGTM

compiler-rt/lib/fuzzer/FuzzerDriver.cpp

int FuzzerDriver(int *argc, char ***argv, UserCallback Callback) {
   if (Flags.stop_file)
     Options.StopFile = Flags.stop_file;
   Options.Entropic = Flags.entropic;
   Options.EntropicFeatureFrequencyThreshold =
       (size_t)Flags.entropic_feature_frequency_threshold;
   Options.EntropicNumberOfRarestFeatures =
       (size_t)Flags.entropic_number_of_rarest_features;
   Options.EntropicScalePerExecTime = Flags.entropic_scale_per_exec_time;
-  if (Options.Entropic) {
-    if (!Options.FocusFunction.empty()) {
-      Printf("ERROR: The parameters `--entropic` and `--focus_function` cannot "
-             "be used together.\n");
-      exit(1);
-    }
+  if (!Options.FocusFunction.empty())
+    Options.Entropic = false; // FocusFunction overrides entropic scheduling.
+  if (Options.Entropic)
     Printf("INFO: Running with entropic power schedule (0x%X, %d).\n",
            Options.EntropicFeatureFrequencyThreshold,
            Options.EntropicNumberOfRarestFeatures);
-  }
   struct EntropicOptions Entropic;
   Entropic.Enabled = Options.Entropic;
   Entropic.FeatureFrequencyThreshold =
       Options.EntropicFeatureFrequencyThreshold;
   Entropic.NumberOfRarestFeatures = Options.EntropicNumberOfRarestFeatures;
   Entropic.ScalePerExecTime = Options.EntropicScalePerExecTime;
 
   unsigned Seed = Flags.seed;

compiler-rt/lib/fuzzer/FuzzerFlags.def

FUZZER_FLAG_STRING(exit_on_item, "Exit if an item with a given sha1 sum"
     " was added to the corpus. "
     "Used primarily for testing libFuzzer itself.")
 FUZZER_FLAG_INT(ignore_remaining_args, 0, "If 1, ignore all arguments passed "
                 "after this one. Useful for fuzzers that need to do their own "
                 "argument parsing.")
 FUZZER_FLAG_STRING(focus_function, "Experimental. "
      "Fuzzing will focus on inputs that trigger calls to this function. "
      "If -focus_function=auto and -data_flow_trace is used, libFuzzer "
-     "will choose the focus functions automatically.")
-FUZZER_FLAG_INT(entropic, 0, "Experimental. Enables entropic power schedule.")
+     "will choose the focus functions automatically. Disables -entropic when "
+     "specified.")
+FUZZER_FLAG_INT(entropic, 1, "Enables entropic power schedule.")
 FUZZER_FLAG_INT(entropic_feature_frequency_threshold, 0xFF, "Experimental. If "
      "entropic is enabled, all features which are observed less often than "
      "the specified value are considered as rare.")
 FUZZER_FLAG_INT(entropic_number_of_rarest_features, 100, "Experimental. If "
      "entropic is enabled, we keep track of the frequencies only for the "
      "Top-X least abundant features (union features that are considered as "
      "rare).")
 FUZZER_FLAG_INT(entropic_scale_per_exec_time, 0, "Experimental. If 1, "

compiler-rt/lib/fuzzer/FuzzerOptions.h

struct FuzzingOptions {
   bool Shrink = false;
   bool ReduceInputs = false;
   int ReloadIntervalSec = 1;
   bool ShuffleAtStartUp = true;
   bool PreferSmall = true;
   size_t MaxNumberOfRuns = -1L;
   int ReportSlowUnits = 10;
   bool OnlyASCII = false;
-  bool Entropic = false;
+  bool Entropic = true;
   size_t EntropicFeatureFrequencyThreshold = 0xFF;
   size_t EntropicNumberOfRarestFeatures = 100;
   bool EntropicScalePerExecTime = false;
   std::string OutputCorpus;
   std::string ArtifactPrefix = "./";
   std::string ExactArtifactPath;
   std::string ExitOnSrcPos;
   std::string ExitOnItem;

compiler-rt/test/fuzzer/cross_over_uniform_dist.test

 REQUIRES: linux, x86_64
 RUN: %cpp_compiler %S/KeepSeedTest.cpp -o %t-CrossOverUniformDistTest
 
 RUN: rm -rf %t-corpus
 RUN: mkdir %t-corpus
 RUN: echo -n "@SELECT" > %t-corpus/A
 RUN: echo -n "@FROM WHERE" > %t-corpus/B
 
-RUN: not %run %t-CrossOverUniformDistTest -keep_seed=1 -cross_over_uniform_dist=1 -seed=1 -runs=2000000 %t-corpus 2>&1 | FileCheck %s
+RUN: not %run %t-CrossOverUniformDistTest -keep_seed=1 -cross_over_uniform_dist=1 -seed=1 -runs=5000000 %t-corpus 2>&1 | FileCheck %s
 CHECK: BINGO
 
 RUN: rm -rf %t-corpus
 RUN: mkdir %t-corpus
 RUN: echo -n "@SELECT" > %t-corpus/A
 RUN: echo -n "@FROM WHERE" > %t-corpus/B
-RUN: %run %t-CrossOverUniformDistTest -keep_seed=1 -seed=1 -runs=2000000 %t-corpus 2>&1
+RUN: %run %t-CrossOverUniformDistTest -keep_seed=1 -seed=1 -runs=5000000 %t-corpus 2>&1

compiler-rt/test/fuzzer/keep-seed.test

 REQUIRES: linux, x86_64
 RUN: %cpp_compiler %S/KeepSeedTest.cpp -o %t-KeepSeedTest
 
 RUN: rm -rf %t-corpus
 RUN: mkdir %t-corpus
 RUN: echo -n SELECTxFROMxWHERE > %t-corpus/valid-fragments
 
-RUN: not %run %t-KeepSeedTest -keep_seed=1 -seed=1 -runs=2000000 %t-corpus 2>&1 | FileCheck %s
+RUN: not %run %t-KeepSeedTest -keep_seed=1 -seed=1 -runs=3000000 %t-corpus 2>&1 | FileCheck %s
 CHECK: BINGO
 
 RUN: rm -rf %t-corpus-baseline
 RUN: mkdir %t-corpus-baseline
 RUN: echo -n SELECTxFROMxWHERE > %t-corpus-baseline/valid-fragments
 
 # The following checks whether without -keep_seed=1 libFuzzer does not find the
-# crashing input "SELECT FROM WHERE" even with 2x more runs.
+# crashing input "SELECT FROM WHERE" even with more runs.
 RUN: %run %t-KeepSeedTest -seed=1 -runs=4000000 %t-corpus-baseline -print_final_stats=1