[fuzzer] Read files as binary

[metzman]

Summary

Read corpus files as binary to avoid automatic conversions

[metzman]

• Fix binary issue afl_driver

[metzman]

Please take a look

This should fix crbug.com/902460

I didn't change the uses of ifstream in FuzzerDataFlowTrace.cpp and FuzzerMerge.cpp since they seem to be used on nonbinary files written by libFuzzer.

compiler-rt/lib/fuzzer/FuzzerIO.cpp

54 ↗

(On Diff #172837)

I don't think it is 100% necessary to use binary for this line, but probably better safe than sorry.

[Dor1s]

I guess it might be worth adding the binary mode argument to FuzzerMerge.cpp and FuzzerDataFlowTrace.cpp as well, since apparently we may have some unexpected platform-dependent specifics when reading files.

[morehouse]

Is there a test for this that fails on Windows?

[metzman]

In D54180#1289406, @Dor1s wrote:

I guess it might be worth adding the binary mode argument to FuzzerMerge.cpp and FuzzerDataFlowTrace.cpp as well, since apparently we may have some unexpected platform-dependent specifics when reading files.

Do you think those files can contain binary? They look like they cannot to me.

Is there a test for this that fails on Windows?

Do you mean a test for this behavior that passes on Windows?

[metzman]

• Add test

[Dor1s]

LGTM!

[metzman]

• Use echo instead of printf

[metzman]

@morehouse I added a test for this. Please take another look.

[metzman]

• fix comment

[morehouse]

LGTM

lib/fuzzer/FuzzerIO.cpp

 long GetEpoch(const std::string &Path) {
   struct stat St;
   if (stat(Path.c_str(), &St))
     return 0;  // Can't stat, be conservative.
   return St.st_mtime;
 }
 
 Unit FileToVector(const std::string &Path, size_t MaxSize, bool ExitOnError) {
-  std::ifstream T(Path);
+  std::ifstream T(Path, std::ios::binary);
   if (ExitOnError && !T) {
     Printf("No such directory: %s; exiting\n", Path.c_str());
     exit(1);
   }
 
   T.seekg(0, T.end);
   auto EndPos = T.tellg();
   if (EndPos < 0) return {};
   size_t FileLen = EndPos;
   if (MaxSize)
     FileLen = std::min(FileLen, MaxSize);
 
   T.seekg(0, T.beg);
   Unit Res(FileLen);
   T.read(reinterpret_cast<char *>(Res.data()), FileLen);
   return Res;
 }
 
 std::string FileToString(const std::string &Path) {
-  std::ifstream T(Path);
+  std::ifstream T(Path, std::ios::binary);
   return std::string((std::istreambuf_iterator<char>(T)),
                      std::istreambuf_iterator<char>());
 }
 
 void CopyFileToErr(const std::string &Path) {
   Printf("%s", FileToString(Path).c_str());
 }
 

lib/fuzzer/afl/afl_driver.cpp

 extern "C" size_t LLVMFuzzerMutate(uint8_t *Data, size_t Size, size_t MaxSize) {
   assert(false && "LLVMFuzzerMutate should not be called from afl_driver");
   return 0;
 }
 
 // Execute any files provided as parameters.
 int ExecuteFilesOnyByOne(int argc, char **argv) {
   for (int i = 1; i < argc; i++) {
-    std::ifstream in(argv[i]);
+    std::ifstream in(argv[i], std::ios::binary);
     in.seekg(0, in.end);
     size_t length = in.tellg();
     in.seekg (0, in.beg);
     std::cout << "Reading " << length << " bytes from " << argv[i] << std::endl;
     // Allocate exactly length bytes so that we reliably catch buffer overflows.
     std::vector<char> bytes(length);
     in.read(bytes.data(), bytes.size());
     assert(in);

test/fuzzer/ReadBinaryTest.cpp

+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+
+// Simple test for a fuzzer. Tests that fuzzer can read a file containing
+// carriage returns.
+#include <cstddef>
+#include <cstdint>
+#include <iostream>
+#include <string>
+
+extern "C" int LLVMFuzzerTestOneInput(const uint8_t* Data, size_t Size) {
+  std::string InputStr(reinterpret_cast<const char*>(Data), Size);
+  std::string MagicStr("Hello\r\nWorld\r\n");
+  if (InputStr == MagicStr) {
+    std::cout << "BINGO!";
+  }
+  return 0;
+}

test/fuzzer/read-binary.test

+# Test that libFuzzer reads files properly.
+
+# Account for the fact that echo will add a trailing newline.
+RUN: echo -e "Hello\r\nWorld\r" > %t-testcase
+RUN: %cpp_compiler %S/ReadBinaryTest.cpp -o %t-fuzzer
+RUN: %run %t-fuzzer %t-testcase | FileCheck %s
+CHECK: BINGO!