[libFuzzer] Replace -seed_corpus to better support fork mode on Win

[metzman]

Summary

Pass seed corpus list in a file to get around argument length limits on Windows.
This limit was preventing many uses of fork mode on Windows.

[metzman]

I'm going to add a test for this.

[metzman]

Add test

[metzman]

• improve comment

[metzman]

• undo accidental

compiler-rt/lib/fuzzer/FuzzerDriver.cpp

771 ↗

(On Diff #196154)

Removing the file is somewhat hostile to users but is the best way to prevent the accumulation of files in fork mode.
Since the primary purpose of -seed_inputs_file is for fork mode, I think this is acceptable. Thoughts?

compiler-rt/test/fuzzer/cross_over.test

18 ↗

(On Diff #196154)

The reason why I do this hacky python thing is because echo leaves a trailing newline and printf didn't work well with the percent formatting.
Is it OK to use python for this purpose (I think python is necessary to run lit commands anyway)?
If not, I can use echo so long as I end with a trailing ,, that way the last seed input is "\n" instead of a newline being added to a filename we actually care about. Does this sound better than the python hack?

[metzman]

PTAL.
This change allows fork mode to be used reasonably on Win.

[metzman]

More context is here: https://github.com/google/clusterfuzz/issues/267

[kcc]

compiler-rt/lib/fuzzer/FuzzerDriver.cpp

771 ↗

(On Diff #196154)

A bit too hostile indeed, and in this case the file is deleted by a process that didn't create it, making it more confusing.
I'd prefer to delete the file in the same place we delete other temporary files for a child.

compiler-rt/lib/fuzzer/FuzzerFlags.def

23 ↗

(On Diff #196165)

I found this flag to be useful by itself, outside the fork mode, so instead of replacing it with a new flag,
I'd suggest enhancing it's behavior:

• --seed_inputs=aaa,bbb keeps working as is
• --seed_inputs=@file_path reads the list from the file

compiler-rt/lib/fuzzer/FuzzerFork.cpp

126 ↗

(On Diff #196165)

for readability, I'd prefer to introduce another variant of WriteToFile:

void WriteToFile(const std::string &Str, const std::string &Path);

compiler-rt/test/fuzzer/cross_over.test

18 ↗

(On Diff #196154)

no need to change this test with the change I proposed.

compiler-rt/test/fuzzer/seed_inputs_file.test

4 ↗

(On Diff #196165)

will echo -n work in this case?
python is indeed a bit unwelcome where.

[metzman]

compiler-rt/lib/fuzzer/FuzzerFlags.def

23 ↗

(On Diff #196165)

So that "@" will be necessary to distinguish between a case where we want to use one seed and a case where we want to use the file as the seed list?

[kcc]

compiler-rt/lib/fuzzer/FuzzerFlags.def

23 ↗

(On Diff #196165)

Yes.
W/o @ the current behavior is preserved: the argument is a comma-separated list of seeds.
W/ @ the argument (with @ stripped) will be treated as a file name, which contains the comma-separated list of seeds

[metzman]

• change name of test
• Get list argument working again
• combine code
• rename
• Use old method

[metzman]

• improve comments

[metzman]

compiler-rt/lib/fuzzer/FuzzerDriver.cpp

771 ↗

(On Diff #196154)

Done.

compiler-rt/lib/fuzzer/FuzzerFlags.def

23 ↗

(On Diff #196165)

Done. Please let me know if you think the help message needs work.

compiler-rt/lib/fuzzer/FuzzerFork.cpp

126 ↗

(On Diff #196165)

Done.

compiler-rt/test/fuzzer/cross_over.test

18 ↗

(On Diff #196154)

Undid this change and the one in len_control.

compiler-rt/test/fuzzer/seed_inputs_file.test

4 ↗

(On Diff #196165)

Yeah good suggestion, that's much better. For some reason I thought it wouldn't work on Windows.

[kcc]

LGTM with several nits.

compiler-rt/lib/fuzzer/FuzzerDriver.cpp

771 ↗

(On Diff #197346)

plz make this if-else more compact (no {}, comments on the same line)

compiler-rt/lib/fuzzer/FuzzerFlags.def

24 ↗

(On Diff #197346)

"of input files" repeated twice?

compiler-rt/test/fuzzer/seed_inputs.test

3 ↗

(On Diff #197346)

replace with CHECK: then remove all --check-prefix

13 ↗

(On Diff #197346)

terminate the file with a newline

[metzman]

• fmt
• fix nits
• fmt

[metzman]

• ideal but test failing
• fix issue

[metzman]

• remove extra newline

[metzman]

compiler-rt/test/fuzzer/seed_inputs.test

3 ↗

(On Diff #197346)

I added a test to ensure we handle a single file correctly instead.

[metzman]

• use new format

[metzman]

• only use @ in argument

[metzman]

This comment was removed by metzman.

[metzman]

• fix bug

[metzman]

• Make LF fail if no seed list

[metzman]

• Add more tests to verify we catch empty lists

[metzman]

@kcc I've changed things so that libFuzzer will fail if the argument to -seed_inputs is a non existent file or is empty? What do you think of this change?

[metzman]

• add newline

[kcc]

LGTM

compiler-rt/lib/fuzzer/FuzzerDriver.cpp

776 ↗

(On Diff #197399)

use exit(1) instead

[metzman]

• improve error message and look for it in tests

[metzman]

• improve message

[metzman]

• use exit(1)

[metzman]

• add missing period

compiler-rt/trunk/lib/fuzzer/FuzzerDriver.cpp

if (Flags.analyze_dict) {
     if (AnalyzeDictionary(F, Dictionary, InitialCorpus)) {
       Printf("Dictionary analysis failed\n");
       exit(1);
     }
     Printf("Dictionary analysis succeeded\n");
     exit(0);
   }
 
-  // Parse -seed_inputs=file1,file2,...
+  // Parse -seed_inputs=file1,file2,... or -seed_inputs=@seed_inputs_file
   Vector<std::string> ExtraSeedFiles;
   if (Flags.seed_inputs) {
-    std::string s = Flags.seed_inputs;
-    size_t comma_pos;
-    while ((comma_pos = s.find_last_of(',')) != std::string::npos) {
-      ExtraSeedFiles.push_back(s.substr(comma_pos + 1));
-      s = s.substr(0, comma_pos);
+    std::string SeedInputs;
+    if (Flags.seed_inputs[0] == '@')
+      SeedInputs = FileToString(Flags.seed_inputs + 1); // File contains list.
+    else
+      SeedInputs = Flags.seed_inputs; // seed_inputs contains the list.
+    if (SeedInputs.empty()) {
+      Printf("seed_inputs is empty or @file does not exist.\n");
+      exit(1);
+    }
+    // Parse SeedInputs.
+    size_t comma_pos = 0;
+    while ((comma_pos = SeedInputs.find_last_of(',')) != std::string::npos) {
+      ExtraSeedFiles.push_back(SeedInputs.substr(comma_pos + 1));
+      SeedInputs = SeedInputs.substr(0, comma_pos);
     }
-    ExtraSeedFiles.push_back(s);
+    ExtraSeedFiles.push_back(SeedInputs);
   }
 
   F->Loop(*Inputs, ExtraSeedFiles);
 
   if (Flags.verbosity)
     Printf("Done %zd runs in %zd second(s)\n", F->getTotalNumberOfRuns(),
            F->secondsSinceProcessStartUp());
   F->PrintFinalStats();
 
   exit(0);  // Don't let F destroy itself.
 }
 
 // Storage for global ExternalFunctions object.
 ExternalFunctions *EF = nullptr;
 
 }  // namespace fuzzer

compiler-rt/trunk/lib/fuzzer/FuzzerFlags.def

 FUZZER_FLAG_INT(max_len, 0, "Maximum length of the test input. "
     "If 0, libFuzzer tries to guess a good value based on the corpus "
     "and reports it. ")
 FUZZER_FLAG_INT(len_control, 100, "Try generating small inputs first, "
   "then try larger inputs over time.  Specifies the rate at which the length "
   "limit is increased (smaller == faster).  If 0, immediately try inputs with "
   "size up to max_len.")
 FUZZER_FLAG_STRING(seed_inputs, "A comma-separated list of input files "
-  "to use as an additional seed corpus")
+  "to use as an additional seed corpus. Alternatively, an \"@\" followed by "
+  "the name of a file containing the comma-seperated list.")
 FUZZER_FLAG_INT(cross_over, 1, "If 1, cross over inputs.")
 FUZZER_FLAG_INT(mutate_depth, 5,
             "Apply this number of consecutive mutations to each input.")
 FUZZER_FLAG_INT(reduce_depth, 0, "Experimental/internal. "
                 "Reduce depth if mutations lose unique features")
 FUZZER_FLAG_INT(shuffle, 1, "Shuffle inputs at startup")
 FUZZER_FLAG_INT(prefer_small, 1,
     "If 1, always prefer smaller inputs during the corpus shuffle.")

compiler-rt/trunk/lib/fuzzer/FuzzerFork.cpp

 }
 
 struct FuzzJob {
   // Inputs.
   Command Cmd;
   std::string CorpusDir;
   std::string FeaturesDir;
   std::string LogPath;
+  std::string SeedListPath;
   std::string CFPath;
 
   // Fuzzing Outputs.
   int ExitCode;
 
   ~FuzzJob() {
     RemoveFile(CFPath);
     RemoveFile(LogPath);
+    RemoveFile(SeedListPath);
     RmDirRecursive(CorpusDir);
     RmDirRecursive(FeaturesDir);
   }
 };
 
 struct GlobalEnv {
   Vector<std::string> Args;
   Vector<std::string> CorpusDirs;
FuzzJob *CreateNewJob(size_t JobId) {
 
     auto Job = new FuzzJob;
     std::string Seeds;
     if (size_t CorpusSubsetSize =
             std::min(Files.size(), (size_t)sqrt(Files.size() + 2)))
       for (size_t i = 0; i < CorpusSubsetSize; i++)
         Seeds += (Seeds.empty() ? "" : ",") +
                  Files[Rand->SkewTowardsLast(Files.size())];
-    if (!Seeds.empty())
-      Cmd.addFlag("seed_inputs", Seeds);
+    if (!Seeds.empty()) {
+      Job->SeedListPath = std::to_string(JobId) + ".seeds";
+      WriteToFile(Seeds, Job->SeedListPath);
+      Cmd.addFlag("seed_inputs", "@" + Job->SeedListPath);
+    }
     Job->LogPath = DirPlusFile(TempDir, std::to_string(JobId) + ".log");
     Job->CorpusDir = DirPlusFile(TempDir, "C" + std::to_string(JobId));
     Job->FeaturesDir = DirPlusFile(TempDir, "F" + std::to_string(JobId));
     Job->CFPath = DirPlusFile(TempDir, std::to_string(JobId) + ".merge");
 
 
     Cmd.addArgument(Job->CorpusDir);
     Cmd.addFlag("features_dir", Job->FeaturesDir);

compiler-rt/trunk/lib/fuzzer/FuzzerIO.h

 Unit FileToVector(const std::string &Path, size_t MaxSize = 0,
                   bool ExitOnError = true);
 
 std::string FileToString(const std::string &Path);
 
 void CopyFileToErr(const std::string &Path);
 
 void WriteToFile(const uint8_t *Data, size_t Size, const std::string &Path);
+// Write Data.c_str() to the file without terminating null character.
+void WriteToFile(const std::string &Data, const std::string &Path);
 void WriteToFile(const Unit &U, const std::string &Path);
 
 void ReadDirToVectorOfUnits(const char *Path, Vector<Unit> *V,
                             long *Epoch, size_t MaxSize, bool ExitOnError);
 
 // Returns "Dir/FileName" or equivalent for the current OS.
 std::string DirPlusFile(const std::string &DirPath,
                         const std::string &FileName);

compiler-rt/trunk/lib/fuzzer/FuzzerIO.cpp

 void CopyFileToErr(const std::string &Path) {
   Printf("%s", FileToString(Path).c_str());
 }
 
 void WriteToFile(const Unit &U, const std::string &Path) {
   WriteToFile(U.data(), U.size(), Path);
 }
 
+void WriteToFile(const std::string &Data, const std::string &Path) {
+  WriteToFile(reinterpret_cast<const uint8_t *>(Data.c_str()), Data.size(),
+              Path);
+}
+
 void WriteToFile(const uint8_t *Data, size_t Size, const std::string &Path) {
   // Use raw C interface because this function may be called from a sig handler.
   FILE *Out = fopen(Path.c_str(), "wb");
   if (!Out) return;
   fwrite(Data, sizeof(Data[0]), Size, Out);
   fclose(Out);
 }
 

compiler-rt/trunk/test/fuzzer/seed_inputs.test

+RUN: %cpp_compiler %S/SimpleTest.cpp -o %t-SimpleTest
+
+USE-1: INFO: seed corpus: files: 1
+RUN: echo -n "%t-SimpleTest" > %t.seed-inputs
+# Test both formats of -seed_inputs argument.
+RUN: %run %t-SimpleTest -runs=1 -seed_inputs=@%t.seed-inputs 2>&1 | FileCheck %s --check-prefix=USE-1
+RUN: %run %t-SimpleTest -runs=1 -seed_inputs=%t-SimpleTest 2>&1 | FileCheck %s --check-prefix=USE-1
+
+USE-2: INFO: seed corpus: files: 2
+RUN: echo -n "%t-SimpleTest,%t-SimpleTest" > %t.seed-inputs
+RUN: %run %t-SimpleTest -runs=1 -seed_inputs=@%t.seed-inputs 2>&1 | FileCheck %s --check-prefix=USE-2
+RUN: %run %t-SimpleTest -runs=1 -seed_inputs=%t-SimpleTest,%t-SimpleTest 2>&1 | FileCheck %s --check-prefix=USE-2
+
+# Test that missing files and trailing commas are tolerated.
+RUN: echo -n "%t-SimpleTest,%t-SimpleTest,nonexistent-file," > %t.seed-inputs
+RUN: %run %t-SimpleTest -runs=1 -seed_inputs=@%t.seed-inputs 2>&1 | FileCheck %s --check-prefix=USE-2
+RUN: %run %t-SimpleTest -runs=1 -seed_inputs=%t-SimpleTest,%t-SimpleTest,nonexistent-file, 2>&1 | FileCheck %s --check-prefix=USE-2
+
+# Test that using a non existent file or an empty seed list fails.
+EMPTY: seed_inputs is empty or @file does not exist.
+RUN: not %run %t-SimpleTest -runs=1 -seed_inputs=@nonexistent-file 2>&1 | FileCheck %s --check-prefix=EMPTY
+RUN: echo -n "" > %t.seed-inputs
+RUN: not %run %t-SimpleTest -runs=1 -seed_inputs=@%t.seed-inputs 2>&1 | FileCheck %s --check-prefix=EMPTY
+RUN: not %run %t-SimpleTest -runs=1 -seed_inputs= 2>&1 | FileCheck %s --check-prefix=EMPTY