[fuzzer] Create user provided fuzzer writeable directories when requested if they dont exist

[dgg5503]

Summary

Currently, libFuzzer will exit with an error message if a non-existent
directory is provided for any of the appropriate arguments. For cases
where libFuzzer is used in a specialized embedded environment, it would
be much easier to have libFuzzer create the directories for the user.

This patch accommodates for this scenario by allowing the user to provide
the argument -create_missing_dirs=1 which makes libFuzzer attempt to
create the artifact_prefix, exact_artifact_path,
features_dir and/or corpus directory if they don't already exist rather
than throw an error and exit.

Split off from D84808 as requested here.

[morehouse]

compiler-rt/lib/fuzzer/FuzzerDriver.cpp

254

s/const int/bool

255

Does this check break the case where we run the fuzzer without giving a corpus directory?

compiler-rt/lib/fuzzer/FuzzerIO.cpp

148

Nit: We really only need one parameter since we can compute DirName inside this helper function.

compiler-rt/test/fuzzer/fuzzer-dirs.test

28

s/emtpy/empty

36

Does chmod return an error on Windows? If so, this test will fail.

44

Do we expect dummy_dir to be created, or not? And should we test this?

[dgg5503]

Addressed @morehouse comments:

• const int to bool for argument in ValidateDirectoryExists
• Simplified MkDirRecursiveInner
• Spelling

compiler-rt/lib/fuzzer/FuzzerDriver.cpp

255

This case does not break since if no corpus directory is provided (i.e. Inputs is empty), ValidateDirectoryExists wont be called. This can be seen here.

compiler-rt/test/fuzzer/fuzzer-dirs.test

36

On Windows, I use chmod provided by Git bash (version 2.27.0.windows.1) which I believe is from MSYS2. Here's what I get when looking at the error code:

$ mkdir testchmod
$ chmod u-w testchmod/
$ echo $?
0

I've realized now that this approach may not guarantee a 0 exit code if some other set of Windows ported GNU tools were used which correctly reported a non-zero exit code. Do you happen to know if there's a more robust approach we can use in order to trigger directory creation failure on both Windows and Linux? If not, could we always ignore the exit code from chmod since we know what output to expect anyway?

44

My intention with this test was just to make sure ".." and "." in paths worked as expected when creating directories but I overlooked the fact that dummy_dir was also created, thanks for pointing this out. I'll see if there's an elegant way of preventing dummy_dir and the likes from being created without making too many additions.

[dgg5503]

Unchecked some mistakenly checked Dones.

[morehouse]

compiler-rt/test/fuzzer/fuzzer-dirs.test

36

I think ignoring the exit code from chmod is sufficient.

44

mkdir -p also creates dummy_dir, so I think the behavior is fine. But maybe we should test it.

[dgg5503]

Address @morehouse comments:

• Ignore chmod exit code for improved compatibility when running the test fuzzer-dirs.test on Windows
• Check for dummy_dir creation

[dgg5503]

[morehouse]

LGTM

[morehouse]

Had to revert because the Windows bot doesn't have chmod at all: http://lab.llvm.org:8011/builders/sanitizer-windows/builds/68987/steps/stage%201%20check/logs/stdio

[dgg5503]

Improved test compatibility for Windows.

compiler-rt/test/fuzzer/fuzzer-dirs.test

36

I took some time to update my Windows environment to debug this test. It looks like I need to split (chmod u-w %t.dir/access_restricted || true) into its own run line, otherwise, Windows tries to find and call (chmod.exe. After doing this, this portion of the test passes fine.

45

Unfortunately, this test doesn't pass because dummy_dir isn't created on Windows like it is on Linux using the path %t.dir/subdirb/../dummy_dir/././../subdirb/features/. Interestingly, Windows considers the path %t.dir/subdirb/../dummy_dir/././.. as existing so directory creation for dummy_dir is skipped completely since leaf directories are created as soon as the first parent directory exists in MkDirRecursiveInner. If we use %t.dir/subdirb/dummy_dir/././../subdirb/features/ instead, we'll get the same behavior on both Windows and Linux.

[morehouse]

LGTM

compiler-rt/lib/fuzzer/FuzzerDriver.cpp

if (ExitCode != 0)
       *HasErrors = true;
     std::lock_guard<std::mutex> Lock(Mu);
     Printf("================== Job %u exited with exit code %d ============\n",
            C, ExitCode);
     fuzzer::CopyFileToErr(Log);
   }
 }
 
-static void ValidateDirectoryExists(const std::string &Path) {
-  if (!Path.empty() && !IsDirectory(Path)) {
-    Printf("ERROR: The required directory \"%s\" does not exist\n", Path.c_str());
+static void ValidateDirectoryExists(const std::string &Path,
+                                    const int CreateDirectory) {
+  if (Path.empty()) {
+    Printf("ERROR: Provided directory path is an empty string\n");
+    exit(1);
+  }
+
+  if (IsDirectory(Path))
+    return;
+
+  if (CreateDirectory) {
+    if (!MkDirRecursive(Path)) {
+      Printf("ERROR: Failed to create directory \"%s\"\n", Path.c_str());
       exit(1);
     }
+    return;
+  }
+
+  Printf("ERROR: The required directory \"%s\" does not exist\n", Path.c_str());
+  exit(1);
 }
 
 std::string CloneArgsWithoutX(const Vector<std::string> &Args,
                               const char *X1, const char *X2) {
   std::string Cmd;
   for (auto &S : Args) {
     if (FlagValue(S.c_str(), X1) || FlagValue(S.c_str(), X2))
       continue;
int FuzzerDriver(int *argc, char ***argv, UserCallback Callback) {
   if (Flags.runs >= 0)
     Options.MaxNumberOfRuns = Flags.runs;
   if (!Inputs->empty() && !Flags.minimize_crash_internal_step) {
     // Ensure output corpus assumed to be the first arbitrary argument input
     // is not a path to an existing file.
     std::string OutputCorpusDir = (*Inputs)[0];
     if (!IsFile(OutputCorpusDir)) {
       Options.OutputCorpus = OutputCorpusDir;
-      ValidateDirectoryExists(Options.OutputCorpus);
+      ValidateDirectoryExists(Options.OutputCorpus, Flags.create_missing_dirs);
     }
   }
   Options.ReportSlowUnits = Flags.report_slow_units;
   if (Flags.artifact_prefix) {
     Options.ArtifactPrefix = Flags.artifact_prefix;
 
     // Since the prefix could be a full path to a file name prefix, assume
     // that if the path ends with the platform's separator that a directory
     // is desired
     std::string ArtifactPathDir = Options.ArtifactPrefix;
     if (!IsSeparator(ArtifactPathDir[ArtifactPathDir.length() - 1])) {
       ArtifactPathDir = DirName(ArtifactPathDir);
     }
-    ValidateDirectoryExists(ArtifactPathDir);
+    ValidateDirectoryExists(ArtifactPathDir, Flags.create_missing_dirs);
   }
   if (Flags.exact_artifact_path) {
     Options.ExactArtifactPath = Flags.exact_artifact_path;
-    ValidateDirectoryExists(DirName(Options.ExactArtifactPath));
+    ValidateDirectoryExists(DirName(Options.ExactArtifactPath),
+                            Flags.create_missing_dirs);
   }
   Vector<Unit> Dictionary;
   if (Flags.dict)
     if (!ParseDictionaryFile(FileToString(Flags.dict), &Dictionary))
       return 1;
   if (Flags.verbosity > 0 && !Dictionary.empty())
     Printf("Dictionary: %zd entries\n", Dictionary.size());
   bool RunIndividualFiles = AllInputsAreFiles();
int FuzzerDriver(int *argc, char ***argv, UserCallback Callback) {
   if (Flags.exit_on_item)
     Options.ExitOnItem = Flags.exit_on_item;
   if (Flags.focus_function)
     Options.FocusFunction = Flags.focus_function;
   if (Flags.data_flow_trace)
     Options.DataFlowTrace = Flags.data_flow_trace;
   if (Flags.features_dir) {
     Options.FeaturesDir = Flags.features_dir;
-    ValidateDirectoryExists(Options.FeaturesDir);
+    ValidateDirectoryExists(Options.FeaturesDir, Flags.create_missing_dirs);
   }
   if (Flags.collect_data_flow)
     Options.CollectDataFlow = Flags.collect_data_flow;
   if (Flags.stop_file)
     Options.StopFile = Flags.stop_file;
   Options.Entropic = Flags.entropic;
   Options.EntropicFeatureFrequencyThreshold =
       (size_t)Flags.entropic_feature_frequency_threshold;

compiler-rt/lib/fuzzer/FuzzerFlags.def

FUZZER_FLAG_INT(entropic_number_of_rarest_features, 100, "Experimental. If "
      "Top-X least abundant features (union features that are considered as "
      "rare).")
 
 FUZZER_FLAG_INT(analyze_dict, 0, "Experimental")
 FUZZER_DEPRECATED_FLAG(use_clang_coverage)
 FUZZER_FLAG_STRING(data_flow_trace, "Experimental: use the data flow trace")
 FUZZER_FLAG_STRING(collect_data_flow,
                    "Experimental: collect the data flow trace")
+
+FUZZER_FLAG_INT(create_missing_dirs, 0, "Automatically attempt to create "
+     "directories for arguments that would normally expect them to already "
+     "exist (i.e. artifact_prefix, exact_artifact_path, features_dir, corpus)")

compiler-rt/lib/fuzzer/FuzzerIO.h

 // Platform specific functions:
 bool IsFile(const std::string &Path);
 bool IsDirectory(const std::string &Path);
 size_t FileSize(const std::string &Path);
 
 void ListFilesInDirRecursive(const std::string &Dir, long *Epoch,
                              Vector<std::string> *V, bool TopDir);
 
+bool MkDirRecursive(const std::string &Dir);
 void RmDirRecursive(const std::string &Dir);
 
 // Iterate files and dirs inside Dir, recursively.
 // Call DirPreCallback/DirPostCallback on dirs before/after
 // calling FileCallback on files.
 void IterateDirRecursive(const std::string &Dir,
                          void (*DirPreCallback)(const std::string &Dir),
                          void (*DirPostCallback)(const std::string &Dir),

compiler-rt/lib/fuzzer/FuzzerIO.cpp

void VPrintf(bool Verbose, const char *Fmt, ...) {
   if (!Verbose) return;
   va_list ap;
   va_start(ap, Fmt);
   vfprintf(OutputFile, Fmt, ap);
   va_end(ap);
   fflush(OutputFile);
 }
 
+static bool MkDirRecursiveInner(const std::string &Dir,
+                                const std::string &Leaf) {
+  // Prevent chance of potential infinite recursion
+  if (Leaf == ".")
+    return true;
+
+  if (IsDirectory(Dir)) {
+    MkDir(Leaf);
+    return IsDirectory(Leaf);
+  }
+
+  bool ret = MkDirRecursiveInner(DirName(Dir), Dir);
+  if (!ret) {
+    // Give up early if a previous MkDir failed
+    return ret;
+  }
+
+  MkDir(Leaf);
+  return IsDirectory(Leaf);
+}
+
+bool MkDirRecursive(const std::string &Dir) {
+  if (Dir.empty())
+    return false;
+
+  if (IsDirectory(Dir))
+    return true;
+
+  return MkDirRecursiveInner(DirName(Dir), Dir);
+}
+
 void RmDirRecursive(const std::string &Dir) {
   IterateDirRecursive(
       Dir, [](const std::string &Path) {},
       [](const std::string &Path) { RmDir(Path); },
       [](const std::string &Path) { RemoveFile(Path); });
 }
 
 std::string TempPath(const char *Prefix, const char *Extension) {
   return DirPlusFile(TmpDir(), std::string("libFuzzerTemp.") + Prefix +
                                    std::to_string(GetPid()) + Extension);
 }
 
 }  // namespace fuzzer

compiler-rt/test/fuzzer/fuzzer-dirs.test

 RUN: cat %t/SUB1/f64 %t/SUB1/f64 %t/SUB1/f64 %t/SUB1/f64 > %t/SUB1/f256
 RUN: cat %t/SUB1/f256 %t/SUB1/f256 %t/SUB1/f256 %t/SUB1/f256 > %t/SUB1/f1024
 RUN: cat %t/SUB1/f1024 %t/SUB1/f1024 %t/SUB1/f1024 %t/SUB1/f1024 > %t/SUB1/f4096
 RUN: cat %t/SUB1/f4096 %t/SUB1/f4096 > %t/SUB1/f8192
 RUN: %run %t-SimpleTest %t/SUB1 -runs=0 2>&1 | FileCheck %s --check-prefix=LONG
 LONG: INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 8192 bytes
 RUN: rm -rf %t/SUB1
 
+# Verify error message prints to console when directory does not exist
 RUN: rm -rf %t.dir && mkdir -p %t.dir
 RUN: not %run %t-SimpleTest -artifact_prefix=%t.dir/NONEXISTENT_DIR/ 2>&1 | FileCheck %s --check-prefix=NONEXISTENT_DIR_RGX
 RUN: not %run %t-SimpleTest -artifact_prefix=%t.dir/NONEXISTENT_DIR/myprefix 2>&1 | FileCheck %s --check-prefix=NONEXISTENT_DIR_RGX
 RUN: not %run %t-SimpleTest -features_dir=%t.dir/NONEXISTENT_DIR/ 2>&1 | FileCheck %s --check-prefix=NONEXISTENT_DIR_RGX
 RUN: not %run %t-SimpleTest %t.dir/NONEXISTENT_DIR 2>&1 | FileCheck %s --check-prefix=NONEXISTENT_DIR_RGX
 RUN: not %run %t-SimpleTest -exact_artifact_path=%t.dir/NONEXISTENT_DIR/myprefix 2>&1 | FileCheck %s --check-prefix=NONEXISTENT_DIR_RGX
 NONEXISTENT_DIR_RGX: ERROR: The required directory "{{.*/NONEXISTENT_DIR/?}}" does not exist
+
+# Verify error message prints to console when given directory is an emtpy
+# string
+RUN: not %run %t-SimpleTest "" 2>&1 | FileCheck %s --check-prefix=INVALID_DIR_RGX
+INVALID_DIR_RGX: ERROR: Provided directory path is an empty string
+
+# Verify error message prints to console when directory creation fails
+# For platforms without functioning chmod (i.e. Windows), use a forbidden
+# character in the directory name.
+RUN: rm -rf %t.dir && mkdir -p %t.dir/access_restricted && chmod u-w %t.dir/access_restricted
+RUN: not %run %t-SimpleTest -create_missing_dirs=1 %t.dir/access_restricted/?corpus? 2>&1 | FileCheck %s --check-prefix=DIR_CREATION_FAILURE
+DIR_CREATION_FAILURE: ERROR: Failed to create directory "{{.*/access_restricted/\?corpus\?}}"
+
+# Verify directories and sub-directories are created when -create_missing_dirs=1
+RUN: not %run %t-SimpleTest -create_missing_dirs=1 -artifact_prefix=%t.dir/subdira/./././artifacts/ -features_dir=%t.dir/subdirb/../dummy_dir/././../subdirb/features/ %t.dir/subdirc/corpus
+RUN: test -e %t.dir/subdira/artifacts/
+RUN: test -e %t.dir/subdirb/features/
+RUN: test -e %t.dir/subdirc/corpus/
+
+# Verify directories and sub-directories are created for exact_artifact_path
+# when -create_missing_dirs=1
+RUN: not %run %t-SimpleTest -create_missing_dirs=1 -exact_artifact_path=%t.dir/subdird/exact_artifacts/abc
+RUN: test -e %t.dir/subdird/exact_artifacts/abc
+
+# Verify directories and sub-directories are created for artifact_prefix when
+# it's referring to a file name prefix and -create_missing_dirs=1
+RUN: not %run %t-SimpleTest -create_missing_dirs=1 -artifact_prefix=%t.dir/subdire/myprefix
+RUN: test -e %t.dir/subdire/ && not test -e %t.dir/subdire/myprefix
+
+# Verify directories are created when referring to relative paths and
+# -create_missing_dirs=1
+RUN: cd %t.dir && not %run %t-SimpleTest -create_missing_dirs=1 -artifact_prefix=cwd_artifacts/ -features_dir=cwd_features/subdirtest/ ./cwd_corpus
+RUN: test -e %t.dir/cwd_artifacts/
+RUN: test -e %t.dir/cwd_features/subdirtest/
+RUN: test -e %t.dir/cwd_corpus/