[lldb] Add reproducer verifier

[JDevlieghere]

Summary

Add a reproducer verifier that catches:

• Missing or invalid home directory
• Missing or invalid working directory
• Missing or invalid module/symbol paths
• Missing files from the VFS

[JDevlieghere]

This will require a test and maybe some code deduplication in CommandObjectReproducer but I already wanted to put the patch up in case I can't get to that today.

[kastiglione]

lldb/source/Commands/CommandObjectReproducer.cpp

670

Is this lambda missing an assignment to errors?

[JDevlieghere]

lldb/source/Commands/CommandObjectReproducer.cpp

670

Good catch, it is indeed.

[JDevlieghere]

• Add tests
• Extract common code from CommandObjectReproducerDump and CommandObjectReproducerVerify into GetLoaderFromPathOrCurrent
• Address code review feedback

[JDevlieghere]

• Make CHECK-line more specific

[teemperor]

As discussed offline, I think the warnings here seem also useful when just replaying. I guess we should make it clear that things like missing files are fine when replaying a reproducer (e.g., just pointing that out as a 'note' and not warning/error). Beside that this LGTM.

lldb/source/Commands/CommandObjectReproducer.cpp

146

form -> from

156

tot -> to

595

Is that dumping implemented? It's also kind of surprising that verify dumps the current reproducer instead of verifying it.

[JDevlieghere]

• Address code review feedback
• Run the verifier when replaying a reproducer
• Add SBReplayOptions instead of adding another overload
• Add --reproducer-skip-verify flag

[teemperor]

One nit about naming and another small thing, but otherwise this LGTM.

lldb/include/lldb/Utility/Reproducer.h

242 ↗

(On Diff #288051)

I kinda like verify = true a bit better than skip_verification = false. 'skip' feels like a negation, so you end up checks like if (!skip_verification) (= "if we should not not verify" instead of just if (verify). Same below.

But that's just personal preference so I won't mind the name if you feel strongly about it (and it kinda fits the command line argument names)

lldb/source/Utility/Reproducer.cpp

337 ↗

(On Diff #288051)

Maybe log the error to llvm::errs? Or is there no symbol-files.yaml when the replay doesn't involve symbol files, so this is expected?

[JDevlieghere]

lldb/source/Utility/Reproducer.cpp

337 ↗

(On Diff #288051)

Correct, the symbol files are optional.

[davezarzycki]

Hello. I have an auto-bisecting multi-stage bot that has identified this change as breaking release (without assertions) testing on Fedora 33 x86-64. Can we get a quick fix or revert this change for now?

FAIL: lldb-shell :: Reproducer/Functionalities/TestImageList.test (68782 of 69683)
******************** TEST 'lldb-shell :: Reproducer/Functionalities/TestImageList.test' FAILED ********************
Script:
--
: 'RUN: at line 6';   /tmp/_update_lc/r/bin/clang --target=specify-a-target-or-use-a-_host-substitution --target=x86_64-unknown-linux-gnu -pthread -fmodules-cache-path=/tmp/_update_lc/r/lldb-test-build.noindex/module-cache-clang/lldb-shell /home/dave/ro_s/lp/lldb/test/Shell/Reproducer/Functionalities/Inputs/stepping.c -g -o /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.out
: 'RUN: at line 8';   rm -rf /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.txt
: 'RUN: at line 10';   echo "CAPTURE" >> /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.txt
: 'RUN: at line 11';   /tmp/_update_lc/r/bin/lldb --no-lldbinit -S /tmp/_update_lc/r/tools/lldb/test/Shell/lit-lldb-init -x -b  --capture --capture-path /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.repro     -o 'run'     -o 'image list'     -o 'reproducer generate'     /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.out >> /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.txt 2>&1
: 'RUN: at line 17';   echo "REPLAY" >> /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.txt
: 'RUN: at line 18';   /tmp/_update_lc/r/bin/lldb --no-lldbinit -S /tmp/_update_lc/r/tools/lldb/test/Shell/lit-lldb-init -x -b --replay /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.repro >> /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.txt 2>&1
: 'RUN: at line 20';   cat /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.txt | /tmp/_update_lc/r/bin/FileCheck /home/dave/ro_s/lp/lldb/test/Shell/Reproducer/Functionalities/TestImageList.test
--
Exit Code: 1

Command Output (stderr):
--
clang-12: warning: argument unused during compilation: '-fmodules-cache-path=/tmp/_update_lc/r/lldb-test-build.noindex/module-cache-clang/lldb-shell' [-Wunused-command-line-argument]

--

********************
Testing:  0.. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90..
********************
Failed Tests (1):
  lldb-shell :: Reproducer/Functionalities/TestImageList.test


Testing Time: 125.54s
  Unsupported      : 10774
  Passed           : 58806
  Expectedly Failed:   102
  Failed           :     1
FAILED: CMakeFiles/check-all

[teemperor]

Don't see an obvious fix, so I reverted for Jonas in 7c80f2da812e45bbdfa3c8f9ab24440f8ef3362a

[JDevlieghere]

In D86497#2248934, @davezarzycki wrote:

Hello. I have an auto-bisecting multi-stage bot that has identified this change as breaking release (without assertions) testing on Fedora 33 x86-64. Can we get a quick fix or revert this change for now?

FAIL: lldb-shell :: Reproducer/Functionalities/TestImageList.test (68782 of 69683)
******************** TEST 'lldb-shell :: Reproducer/Functionalities/TestImageList.test' FAILED ********************
Script:
--
: 'RUN: at line 6';   /tmp/_update_lc/r/bin/clang --target=specify-a-target-or-use-a-_host-substitution --target=x86_64-unknown-linux-gnu -pthread -fmodules-cache-path=/tmp/_update_lc/r/lldb-test-build.noindex/module-cache-clang/lldb-shell /home/dave/ro_s/lp/lldb/test/Shell/Reproducer/Functionalities/Inputs/stepping.c -g -o /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.out
: 'RUN: at line 8';   rm -rf /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.txt
: 'RUN: at line 10';   echo "CAPTURE" >> /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.txt
: 'RUN: at line 11';   /tmp/_update_lc/r/bin/lldb --no-lldbinit -S /tmp/_update_lc/r/tools/lldb/test/Shell/lit-lldb-init -x -b  --capture --capture-path /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.repro     -o 'run'     -o 'image list'     -o 'reproducer generate'     /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.out >> /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.txt 2>&1
: 'RUN: at line 17';   echo "REPLAY" >> /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.txt
: 'RUN: at line 18';   /tmp/_update_lc/r/bin/lldb --no-lldbinit -S /tmp/_update_lc/r/tools/lldb/test/Shell/lit-lldb-init -x -b --replay /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.repro >> /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.txt 2>&1
: 'RUN: at line 20';   cat /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.txt | /tmp/_update_lc/r/bin/FileCheck /home/dave/ro_s/lp/lldb/test/Shell/Reproducer/Functionalities/TestImageList.test
--
Exit Code: 1

Command Output (stderr):
--
clang-12: warning: argument unused during compilation: '-fmodules-cache-path=/tmp/_update_lc/r/lldb-test-build.noindex/module-cache-clang/lldb-shell' [-Wunused-command-line-argument]

--

********************
Testing:  0.. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90..
********************
Failed Tests (1):
  lldb-shell :: Reproducer/Functionalities/TestImageList.test


Testing Time: 125.54s
  Unsupported      : 10774
  Passed           : 58806
  Expectedly Failed:   102
  Failed           :     1
FAILED: CMakeFiles/check-all

Hey Dave, I cannot repro this, neither on macOS nor Linux when building in release without assertions. Could you rerun the test manually and copy the output of the different commands? The lit output is really abysmal :(

[davezarzycki]

Line 18 fails: %lldb -x -b --replay %t.repro >> %t.txt 2>&1

error: reproducer replay failed:
warning: home directory '/home/dave' not in VFS

I tried rebuilding in my home directory as opposed to /tmp and the error went away. Did you try building outside of your home directory?

[davezarzycki]

And just to be clear, the source directory in my setup is in the home directory. My cron job / "bot" build just builds in /tmp.

[JDevlieghere]

Thanks, should be fixed by 426fa35b655ffb8647d9d69580a69627c0d19024

lldb/include/lldb/Utility/ReproducerProvider.h

 template <typename T>
 llvm::Expected<std::string> GetDirectoryFrom(repro::Loader *loader) {
   llvm::Expected<std::string> dir = loader->LoadBuffer<T>();
   if (!dir)
     return dir.takeError();
   return std::string(llvm::StringRef(*dir).rtrim());
 }
 
+class Verifier {
+public:
+  Verifier(Loader *loader) : m_loader(loader) {}
+  void Verify(std::function<void(llvm::StringRef)> error_callback,
+              std::function<void(llvm::StringRef)> warning_callback) const;
+
+private:
+  Loader *m_loader;
+};
+
 } // namespace repro
 } // namespace lldb_private
 
 LLVM_YAML_IS_SEQUENCE_VECTOR(lldb_private::repro::SymbolFileProvider::Entry)
 
 namespace llvm {
 namespace yaml {
 template <>

lldb/source/Commands/CommandObjectReproducer.cpp

 
 static constexpr OptionEnumValues ReproducerSignalType() {
   return OptionEnumValues(g_reproducer_signaltype);
 }
 
 #define LLDB_OPTIONS_reproducer_xcrash
 #include "CommandOptions.inc"
 
+#define LLDB_OPTIONS_reproducer_verify
+#include "CommandOptions.inc"
+
 template <typename T>
 llvm::Expected<T> static ReadFromYAML(StringRef filename) {
   auto error_or_file = MemoryBuffer::getFile(filename);
   if (auto err = error_or_file.getError()) {
     return errorCodeToError(err);
   }
 
   T t;
   yaml::Input yin((*error_or_file)->getBuffer());
   yin >> t;
 
   if (auto err = yin.error()) {
     return errorCodeToError(err);
   }
 
   return t;
 }
 
 class CommandObjectReproducerGenerate : public CommandObjectParsed {
 public:
   CommandObjectReproducerGenerate(CommandInterpreter &interpreter)
       : CommandObjectParsed(
             interpreter, "reproducer generate",
             "Generate reproducer on disk. When the debugger is in capture "
             "mode, this command will output the reproducer to a directory on "
             "disk and quit. In replay mode this command in a no-op.",
             nullptr) {}
 
   ~CommandObjectReproducerGenerate() override = default;
 
 protected:
   bool DoExecute(Args &command, CommandReturnObject &result) override {
     if (!command.empty()) {
       result.AppendErrorWithFormat("'%s' takes no arguments",
                                    m_cmd_name.c_str());
       return false;
     }
 
     auto &r = Reproducer::Instance();
     if (auto generator = r.GetGenerator()) {
       generator->Keep();
     } else if (r.IsReplaying()) {
       // Make this operation a NO-OP in replay mode.
bool DoExecute(Args &command, CommandReturnObject &result) override {
     result.SetStatus(eReturnStatusSuccessFinishNoResult);
     return result.Succeeded();
   }
 
 private:
   CommandOptions m_options;
 };
 
+class CommandObjectReproducerVerify : public CommandObjectParsed {
+public:
+  CommandObjectReproducerVerify(CommandInterpreter &interpreter)
+      : CommandObjectParsed(interpreter, "reproducer verify",
+                            "Verify the contents of a reproducer. "
+                            "If no reproducer is specified during replay, it "
+                            "dumps the content of the current reproducer.",
+                            nullptr) {}
+
+  ~CommandObjectReproducerVerify() override = default;
+
+  Options *GetOptions() override { return &m_options; }
+
+  class CommandOptions : public Options {
+  public:
+    CommandOptions() : Options(), file() {}
+
+    ~CommandOptions() override = default;
+
+    Status SetOptionValue(uint32_t option_idx, StringRef option_arg,
+                          ExecutionContext *execution_context) override {
+      Status error;
+      const int short_option = m_getopt_table[option_idx].val;
+
+      switch (short_option) {
+      case 'f':
+        file.SetFile(option_arg, FileSpec::Style::native);
+        FileSystem::Instance().Resolve(file);
+        break;
+      default:
+        llvm_unreachable("Unimplemented option");
+      }
+
+      return error;
+    }
+
+    void OptionParsingStarting(ExecutionContext *execution_context) override {
+      file.Clear();
+    }
+
+    ArrayRef<OptionDefinition> GetDefinitions() override {
+      return makeArrayRef(g_reproducer_verify_options);
+    }
+
+    FileSpec file;
+  };
+
+protected:
+  bool DoExecute(Args &command, CommandReturnObject &result) override {
+    if (!command.empty()) {
+      result.AppendErrorWithFormat("'%s' takes no arguments",
+                                   m_cmd_name.c_str());
+      return false;
+    }
+
+    // If no reproducer path is specified, use the loader currently used for
+    // replay. Otherwise create a new loader just for dumping.
+    llvm::Optional<Loader> loader_storage;
+    Loader *loader = nullptr;
+    if (!m_options.file) {
+      loader = Reproducer::Instance().GetLoader();
+      if (loader == nullptr) {
+        result.SetError(
+            "Not specifying a reproducer is only support during replay.");
+        result.SetStatus(eReturnStatusSuccessFinishNoResult);
+        return false;
+      }
+    } else {
+      loader_storage.emplace(m_options.file);
+      loader = &(*loader_storage);
+      if (Error err = loader->LoadIndex()) {
+        SetError(result, std::move(err));
+        return false;
+      }
+    }
+
+    // If we get here we should have a valid loader.
+    assert(loader);
+
+    bool errors = false;
+    auto error_callback = [&](llvm::StringRef error) {
+      result.AppendError(error);
+    };
+
+    bool warnings = false;
+    auto warning_callback = [&](llvm::StringRef warning) {
+      warnings = true;
+      result.AppendWarning(warning);
+    };
+
+    Verifier verifier(loader);
+    verifier.Verify(error_callback, warning_callback);
+
+    if (warnings || errors) {
+      result.AppendMessage("reproducer verification failed");
+      result.SetStatus(eReturnStatusFailed);
+    } else {
+      result.AppendMessage("reproducer verification succeeded");
+      result.SetStatus(eReturnStatusSuccessFinishResult);
+    }
+
+    return result.Succeeded();
+  }
+
+private:
+  CommandOptions m_options;
+};
+
 CommandObjectReproducer::CommandObjectReproducer(
     CommandInterpreter &interpreter)
     : CommandObjectMultiword(
           interpreter, "reproducer",
           "Commands for manipulating reproducers. Reproducers make it "
           "possible "
           "to capture full debug sessions with all its dependencies. The "
           "resulting reproducer is used to replay the debug session while "
           "debugging the debugger.\n"
           "Because reproducers need the whole the debug session from "
           "beginning to end, you need to launch the debugger in capture or "
           "replay mode, commonly though the command line driver.\n"
           "Reproducers are unrelated record-replay debugging, as you cannot "
           "interact with the debugger during replay.\n",
           "reproducer <subcommand> [<subcommand-options>]") {
   LoadSubCommand(
       "generate",
       CommandObjectSP(new CommandObjectReproducerGenerate(interpreter)));
   LoadSubCommand("status", CommandObjectSP(
                                new CommandObjectReproducerStatus(interpreter)));
   LoadSubCommand("dump",
                  CommandObjectSP(new CommandObjectReproducerDump(interpreter)));
+  LoadSubCommand("verify", CommandObjectSP(
+                               new CommandObjectReproducerVerify(interpreter)));
   LoadSubCommand("xcrash", CommandObjectSP(
                                new CommandObjectReproducerXCrash(interpreter)));
 }
 
 CommandObjectReproducer::~CommandObjectReproducer() = default;

lldb/source/Commands/Options.td

def breakpoint_set_address_slide : Option<"address-slide", "R">,
     Desc<"Add the specified offset to whatever address(es) the breakpoint "
     "resolves to. At present this applies the offset directly as given, and "
     "doesn't try to align it to instruction boundaries.">;
   def breakpoint_set_move_to_nearest_code : Option<"move-to-nearest-code", "m">,
     Groups<[1,9,12]>, Arg<"Boolean">,
     Desc<"Move breakpoints to nearest code. If not set the "
     "target.move-to-nearest-codesetting is used.">;
   def breakpoint_set_file_colon_line : Option<"joint-specifier", "y">, Group<12>, Arg<"FileLineColumn">,
-    Required, Completion<"SourceFile">, 
-    Desc<"A specifier in the form filename:line[:column] for setting file & line breakpoints.">;  
+    Required, Completion<"SourceFile">,
+    Desc<"A specifier in the form filename:line[:column] for setting file & line breakpoints.">;
   /* Don't add this option till it actually does something useful...
   def breakpoint_set_exception_typename : Option<"exception-typename", "O">,
     Arg<"TypeName">, Desc<"The breakpoint will only stop if an "
     "exception Object of this type is thrown.  Can be repeated multiple times "
     "to stop for multiple object types">;
    */
 }
 
let Command = "reproducer dump" in {
   def reproducer_provider : Option<"provider", "p">, Group<1>,
     EnumArg<"None", "ReproducerProviderType()">,
     Required, Desc<"The reproducer provider to dump.">;
   def reproducer_file : Option<"file", "f">, Group<1>, Arg<"Filename">,
     Desc<"The reproducer path. If a reproducer is replayed and no path is "
     "provided, that reproducer is dumped.">;
 }
 
+let Command = "reproducer verify" in {
+  def reproducer_verify_file : Option<"file", "f">, Group<1>, Arg<"Filename">,
+    Desc<"The reproducer path. If a reproducer is replayed and no path is "
+    "provided, that reproducer is dumped.">;
+}
+
 let Command = "reproducer xcrash" in {
   def reproducer_signal : Option<"signal", "s">, Group<1>,
     EnumArg<"None", "ReproducerSignalType()">,
     Required, Desc<"The signal to crash the debugger.">;
 }
 
 let Command = "memory read" in {
   def memory_read_num_per_line : Option<"num-per-line", "l">, Group<1>,
def source_list_name : Option<"name", "n">, Group<2>, Arg<"Symbol">,
     Completion<"Symbol">,
     Desc<"The name of a function whose source to display.">;
   def source_list_address : Option<"address", "a">, Group<3>,
     Arg<"AddressOrExpression">, Desc<"Lookup the address and display the source"
     " information for the corresponding file and line.">;
   def source_list_reverse : Option<"reverse", "r">, Group<4>, Desc<"Reverse the"
     " listing to look backwards from the last displayed block of source.">;
   def source_list_file_colon_line : Option<"joint-specifier", "y">, Group<5>,
-    Arg<"FileLineColumn">, Completion<"SourceFile">, 
+    Arg<"FileLineColumn">, Completion<"SourceFile">,
     Desc<"A specifier in the form filename:line[:column] from which to display"
          " source.">;
 }
 
 let Command = "target dependents" in {
   def dependents_no_dependents : Option<"no-dependents", "d">, Group<1>,
     OptionalEnumArg<"Value", "OptionEnumValues(g_dependents_enumaration)">,
     Desc<"Whether or not to load dependents when creating a target. If the "

lldb/source/Utility/ReproducerProvider.cpp

 //===-- Reproducer.cpp ----------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "lldb/Utility/ReproducerProvider.h"
 #include "lldb/Utility/ProcessInfo.h"
 #include "llvm/Support/FileSystem.h"
+#include "llvm/Support/WithColor.h"
 #include "llvm/Support/raw_ostream.h"
 
 using namespace lldb_private;
 using namespace lldb_private::repro;
 using namespace llvm;
 using namespace llvm::yaml;
 
 llvm::Expected<std::unique_ptr<DataRecorder>>
SymbolFileLoader::GetPaths(const UUID *uuid) const {
   auto it = std::lower_bound(m_symbol_files.begin(), m_symbol_files.end(),
                              SymbolFileProvider::Entry(uuid->GetAsString()));
   if (it == m_symbol_files.end())
     return {};
   return std::make_pair<FileSpec, FileSpec>(FileSpec(it->module_path),
                                             FileSpec(it->symbol_path));
 }
 
+void Verifier::Verify(
+    std::function<void(llvm::StringRef)> error_callback,
+    std::function<void(llvm::StringRef)> warning_callback) const {
+  if (!m_loader) {
+    error_callback("invalid loader");
+    return;
+  }
+
+  FileSpec vfs_mapping = m_loader->GetFile<FileProvider::Info>();
+  ErrorOr<std::unique_ptr<MemoryBuffer>> buffer =
+      vfs::getRealFileSystem()->getBufferForFile(vfs_mapping.GetPath());
+  if (!buffer) {
+    error_callback("unable to read files: " + buffer.getError().message());
+    return;
+  }
+
+  IntrusiveRefCntPtr<vfs::FileSystem> vfs = vfs::getVFSFromYAML(
+      std::move(buffer.get()), nullptr, vfs_mapping.GetPath());
+  if (!vfs) {
+    error_callback("unable to initialize the virtual file system");
+    return;
+  }
+
+  auto &redirecting_vfs = static_cast<vfs::RedirectingFileSystem &>(*vfs);
+  redirecting_vfs.setFallthrough(false);
+
+  llvm::Expected<std::string> working_dir =
+      GetDirectoryFrom<WorkingDirectoryProvider>(m_loader);
+  if (working_dir) {
+    if (!vfs->exists(*working_dir))
+      warning_callback("working directory '" + *working_dir + "' not in VFS");
+    vfs->setCurrentWorkingDirectory(*working_dir);
+  } else {
+    warning_callback("no working directory in reproducer: " +
+                     toString(working_dir.takeError()));
+  }
+
+  llvm::Expected<std::string> home_dir =
+      GetDirectoryFrom<HomeDirectoryProvider>(m_loader);
+  if (home_dir) {
+    if (!vfs->exists(*working_dir))
+      warning_callback("home directory '" + *working_dir + "' not in VFS");
+  } else {
+    warning_callback("no home directory in reproducer: " +
+                     toString(working_dir.takeError()));
+  }
+
+  Expected<std::string> symbol_files =
+      m_loader->LoadBuffer<SymbolFileProvider>();
+  if (symbol_files) {
+    std::vector<SymbolFileProvider::Entry> entries;
+    llvm::yaml::Input yin(*symbol_files);
+    yin >> entries;
+    for (const auto &entry : entries) {
+      if (!entry.module_path.empty() && !vfs->exists(entry.module_path)) {
+        warning_callback("'" + entry.module_path + "': module path for " +
+                         entry.uuid + " not in VFS");
+      }
+      if (!entry.symbol_path.empty() && !vfs->exists(entry.symbol_path)) {
+        warning_callback("'" + entry.symbol_path + "': symbol path for " +
+                         entry.uuid + " not in VFS");
+      }
+    }
+  } else {
+    llvm::consumeError(symbol_files.takeError());
+  }
+
+  std::vector<llvm::StringRef> roots = redirecting_vfs.getRoots();
+  for (llvm::StringRef root : roots) {
+    std::error_code ec;
+    vfs::recursive_directory_iterator iter(*vfs, root, ec);
+    vfs::recursive_directory_iterator end;
+    for (; iter != end && !ec; iter.increment(ec)) {
+      ErrorOr<vfs::Status> status = vfs->status(iter->path());
+      if (!status)
+        warning_callback("'" + iter->path().str() +
+                         "': " + status.getError().message());
+    }
+  }
+}
+
 void ProviderBase::anchor() {}
 char CommandProvider::ID = 0;
 char FileProvider::ID = 0;
 char ProviderBase::ID = 0;
 char VersionProvider::ID = 0;
 char WorkingDirectoryProvider::ID = 0;
 char HomeDirectoryProvider::ID = 0;
 char ProcessInfoProvider::ID = 0;

llvm/include/llvm/Support/VirtualFileSystem.h

public:
   std::error_code makeAbsolute(SmallVectorImpl<char> &Path) const override;
 
   directory_iterator dir_begin(const Twine &Dir, std::error_code &EC) override;
 
   void setExternalContentsPrefixDir(StringRef PrefixDir);
 
   StringRef getExternalContentsPrefixDir() const;
 
+  void setFallthrough(bool Fallthrough);
+
+  std::vector<llvm::StringRef> getRoots() const;
+
   void dump(raw_ostream &OS) const;
   void dumpEntry(raw_ostream &OS, Entry *E, int NumSpaces = 0) const;
 #if !defined(NDEBUG) || defined(LLVM_ENABLE_DUMP)
   LLVM_DUMP_METHOD void dump() const;
 #endif
 };
 
 /// Collect all pairs of <virtual path, real path> entries from the

llvm/lib/Support/VirtualFileSystem.cpp

 void RedirectingFileSystem::setExternalContentsPrefixDir(StringRef PrefixDir) {
   ExternalContentsPrefixDir = PrefixDir.str();
 }
 
 StringRef RedirectingFileSystem::getExternalContentsPrefixDir() const {
   return ExternalContentsPrefixDir;
 }
 
+void RedirectingFileSystem::setFallthrough(bool Fallthrough) {
+  IsFallthrough = Fallthrough;
+}
+
+std::vector<StringRef> RedirectingFileSystem::getRoots() const {
+  std::vector<StringRef> R;
+  for (const auto &Root : Roots)
+    R.push_back(Root->getName());
+  return R;
+}
+
 void RedirectingFileSystem::dump(raw_ostream &OS) const {
   for (const auto &Root : Roots)
     dumpEntry(OS, Root.get());
 }
 
 void RedirectingFileSystem::dumpEntry(raw_ostream &OS,
                                       RedirectingFileSystem::Entry *E,
                                       int NumSpaces) const {