[lldb] Add reproducer verifier

[JDevlieghere]

Summary

Add a reproducer verifier that catches:

• Missing or invalid home directory
• Missing or invalid working directory
• Missing or invalid module/symbol paths
• Missing files from the VFS

[JDevlieghere]

This will require a test and maybe some code deduplication in CommandObjectReproducer but I already wanted to put the patch up in case I can't get to that today.

[kastiglione]

lldb/source/Commands/CommandObjectReproducer.cpp

678

Is this lambda missing an assignment to errors?

[JDevlieghere]

lldb/source/Commands/CommandObjectReproducer.cpp

678

Good catch, it is indeed.

[JDevlieghere]

• Add tests
• Extract common code from CommandObjectReproducerDump and CommandObjectReproducerVerify into GetLoaderFromPathOrCurrent
• Address code review feedback

[JDevlieghere]

• Make CHECK-line more specific

[teemperor]

As discussed offline, I think the warnings here seem also useful when just replaying. I guess we should make it clear that things like missing files are fine when replaying a reproducer (e.g., just pointing that out as a 'note' and not warning/error). Beside that this LGTM.

lldb/source/Commands/CommandObjectReproducer.cpp

146

form -> from

156

tot -> to

603

Is that dumping implemented? It's also kind of surprising that verify dumps the current reproducer instead of verifying it.

[JDevlieghere]

• Address code review feedback
• Run the verifier when replaying a reproducer
• Add SBReplayOptions instead of adding another overload
• Add --reproducer-skip-verify flag

[teemperor]

One nit about naming and another small thing, but otherwise this LGTM.

lldb/include/lldb/Utility/Reproducer.h

242

I kinda like verify = true a bit better than skip_verification = false. 'skip' feels like a negation, so you end up checks like if (!skip_verification) (= "if we should not not verify" instead of just if (verify). Same below.

But that's just personal preference so I won't mind the name if you feel strongly about it (and it kinda fits the command line argument names)

lldb/source/Utility/Reproducer.cpp

337

Maybe log the error to llvm::errs? Or is there no symbol-files.yaml when the replay doesn't involve symbol files, so this is expected?

[JDevlieghere]

lldb/source/Utility/Reproducer.cpp

337

Correct, the symbol files are optional.

[davezarzycki]

Hello. I have an auto-bisecting multi-stage bot that has identified this change as breaking release (without assertions) testing on Fedora 33 x86-64. Can we get a quick fix or revert this change for now?

FAIL: lldb-shell :: Reproducer/Functionalities/TestImageList.test (68782 of 69683)
******************** TEST 'lldb-shell :: Reproducer/Functionalities/TestImageList.test' FAILED ********************
Script:
--
: 'RUN: at line 6';   /tmp/_update_lc/r/bin/clang --target=specify-a-target-or-use-a-_host-substitution --target=x86_64-unknown-linux-gnu -pthread -fmodules-cache-path=/tmp/_update_lc/r/lldb-test-build.noindex/module-cache-clang/lldb-shell /home/dave/ro_s/lp/lldb/test/Shell/Reproducer/Functionalities/Inputs/stepping.c -g -o /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.out
: 'RUN: at line 8';   rm -rf /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.txt
: 'RUN: at line 10';   echo "CAPTURE" >> /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.txt
: 'RUN: at line 11';   /tmp/_update_lc/r/bin/lldb --no-lldbinit -S /tmp/_update_lc/r/tools/lldb/test/Shell/lit-lldb-init -x -b  --capture --capture-path /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.repro     -o 'run'     -o 'image list'     -o 'reproducer generate'     /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.out >> /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.txt 2>&1
: 'RUN: at line 17';   echo "REPLAY" >> /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.txt
: 'RUN: at line 18';   /tmp/_update_lc/r/bin/lldb --no-lldbinit -S /tmp/_update_lc/r/tools/lldb/test/Shell/lit-lldb-init -x -b --replay /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.repro >> /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.txt 2>&1
: 'RUN: at line 20';   cat /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.txt | /tmp/_update_lc/r/bin/FileCheck /home/dave/ro_s/lp/lldb/test/Shell/Reproducer/Functionalities/TestImageList.test
--
Exit Code: 1

Command Output (stderr):
--
clang-12: warning: argument unused during compilation: '-fmodules-cache-path=/tmp/_update_lc/r/lldb-test-build.noindex/module-cache-clang/lldb-shell' [-Wunused-command-line-argument]

--

********************
Testing:  0.. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90..
********************
Failed Tests (1):
  lldb-shell :: Reproducer/Functionalities/TestImageList.test


Testing Time: 125.54s
  Unsupported      : 10774
  Passed           : 58806
  Expectedly Failed:   102
  Failed           :     1
FAILED: CMakeFiles/check-all

[teemperor]

Don't see an obvious fix, so I reverted for Jonas in 7c80f2da812e45bbdfa3c8f9ab24440f8ef3362a

[JDevlieghere]

In D86497#2248934, @davezarzycki wrote:

Hello. I have an auto-bisecting multi-stage bot that has identified this change as breaking release (without assertions) testing on Fedora 33 x86-64. Can we get a quick fix or revert this change for now?

FAIL: lldb-shell :: Reproducer/Functionalities/TestImageList.test (68782 of 69683)
******************** TEST 'lldb-shell :: Reproducer/Functionalities/TestImageList.test' FAILED ********************
Script:
--
: 'RUN: at line 6';   /tmp/_update_lc/r/bin/clang --target=specify-a-target-or-use-a-_host-substitution --target=x86_64-unknown-linux-gnu -pthread -fmodules-cache-path=/tmp/_update_lc/r/lldb-test-build.noindex/module-cache-clang/lldb-shell /home/dave/ro_s/lp/lldb/test/Shell/Reproducer/Functionalities/Inputs/stepping.c -g -o /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.out
: 'RUN: at line 8';   rm -rf /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.txt
: 'RUN: at line 10';   echo "CAPTURE" >> /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.txt
: 'RUN: at line 11';   /tmp/_update_lc/r/bin/lldb --no-lldbinit -S /tmp/_update_lc/r/tools/lldb/test/Shell/lit-lldb-init -x -b  --capture --capture-path /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.repro     -o 'run'     -o 'image list'     -o 'reproducer generate'     /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.out >> /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.txt 2>&1
: 'RUN: at line 17';   echo "REPLAY" >> /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.txt
: 'RUN: at line 18';   /tmp/_update_lc/r/bin/lldb --no-lldbinit -S /tmp/_update_lc/r/tools/lldb/test/Shell/lit-lldb-init -x -b --replay /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.repro >> /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.txt 2>&1
: 'RUN: at line 20';   cat /tmp/_update_lc/r/tools/lldb/test/Reproducer/Functionalities/Output/TestImageList.test.tmp.txt | /tmp/_update_lc/r/bin/FileCheck /home/dave/ro_s/lp/lldb/test/Shell/Reproducer/Functionalities/TestImageList.test
--
Exit Code: 1

Command Output (stderr):
--
clang-12: warning: argument unused during compilation: '-fmodules-cache-path=/tmp/_update_lc/r/lldb-test-build.noindex/module-cache-clang/lldb-shell' [-Wunused-command-line-argument]

--

********************
Testing:  0.. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90..
********************
Failed Tests (1):
  lldb-shell :: Reproducer/Functionalities/TestImageList.test


Testing Time: 125.54s
  Unsupported      : 10774
  Passed           : 58806
  Expectedly Failed:   102
  Failed           :     1
FAILED: CMakeFiles/check-all

Hey Dave, I cannot repro this, neither on macOS nor Linux when building in release without assertions. Could you rerun the test manually and copy the output of the different commands? The lit output is really abysmal :(

[davezarzycki]

Line 18 fails: %lldb -x -b --replay %t.repro >> %t.txt 2>&1

error: reproducer replay failed:
warning: home directory '/home/dave' not in VFS

I tried rebuilding in my home directory as opposed to /tmp and the error went away. Did you try building outside of your home directory?

[davezarzycki]

And just to be clear, the source directory in my setup is in the home directory. My cron job / "bot" build just builds in /tmp.

[JDevlieghere]

Thanks, should be fixed by 426fa35b655ffb8647d9d69580a69627c0d19024

lldb/include/lldb/API/SBReproducer.h

 //===-- SBReproducer.h ------------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLDB_API_SBREPRODUCER_H
 #define LLDB_API_SBREPRODUCER_H
 
 #include "lldb/API/SBDefines.h"
 
+namespace lldb_private {
+namespace repro {
+struct ReplayOptions;
+}
+} // namespace lldb_private
+
 namespace lldb {
 
+class LLDB_API SBReplayOptions {
+public:
+  SBReplayOptions();
+  SBReplayOptions(const SBReplayOptions &rhs);
+  ~SBReplayOptions();
+
+  SBReplayOptions &operator=(const SBReplayOptions &rhs);
+
+  void SetVerify(bool verify);
+  bool GetVerify() const;
+
+  void SetCheckVersion(bool check);
+  bool GetCheckVersion() const;
+
+private:
+  std::unique_ptr<lldb_private::repro::ReplayOptions> m_opaque_up;
+};
+
 /// The SBReproducer class is special because it bootstraps the capture and
 /// replay of SB API calls. As a result we cannot rely on any other SB objects
 /// in the interface or implementation of this class.
 class LLDB_API SBReproducer {
 public:
   static const char *Capture();
   static const char *Capture(const char *path);
   static const char *Replay(const char *path);
   static const char *Replay(const char *path, bool skip_version_check);
+  static const char *Replay(const char *path, const SBReplayOptions &options);
   static const char *PassiveReplay(const char *path);
   static const char *GetPath();
   static bool SetAutoGenerate(bool b);
   static bool Generate();
 
   /// The working directory is set to the current working directory when the
   /// reproducers are initialized. This method allows setting a different
   /// working directory. This is used by the API test suite  which temporarily
   /// changes the directory to where the test lives. This is a NO-OP in every
   /// mode but capture.
   static void SetWorkingDirectory(const char *path);
 };
 
 } // namespace lldb
 
 #endif

lldb/include/lldb/Utility/Reproducer.h

private:
   static llvm::Optional<Reproducer> &InstanceImpl();
 
   llvm::Optional<Generator> m_generator;
   llvm::Optional<Loader> m_loader;
 
   mutable std::mutex m_mutex;
 };
 
+class Verifier {
+public:
+  Verifier(Loader *loader) : m_loader(loader) {}
+  void Verify(llvm::function_ref<void(llvm::StringRef)> error_callback,
+              llvm::function_ref<void(llvm::StringRef)> warning_callback,
+              llvm::function_ref<void(llvm::StringRef)> note_callback) const;
+
+private:
+  Loader *m_loader;
+};
+
+struct ReplayOptions {
+  bool verify = true;
+  bool check_version = true;
+};
+
 } // namespace repro
 } // namespace lldb_private
 
 #endif // LLDB_UTILITY_REPRODUCER_H

lldb/source/API/SBReproducer.cpp

 #include "lldb/API/SBReproducer.h"
 #include "lldb/Host/FileSystem.h"
 #include "lldb/lldb-private.h"
 
 using namespace lldb;
 using namespace lldb_private;
 using namespace lldb_private::repro;
 
+SBReplayOptions::SBReplayOptions()
+    : m_opaque_up(std::make_unique<ReplayOptions>()){};
+
+SBReplayOptions::SBReplayOptions(const SBReplayOptions &rhs)
+    : m_opaque_up(std::make_unique<ReplayOptions>(*rhs.m_opaque_up)) {}
+
+SBReplayOptions::~SBReplayOptions() = default;
+
+SBReplayOptions &SBReplayOptions::operator=(const SBReplayOptions &rhs) {
+  if (this == &rhs)
+    return *this;
+  *m_opaque_up = *rhs.m_opaque_up;
+  return *this;
+}
+
+void SBReplayOptions::SetVerify(bool verify) { m_opaque_up->verify = verify; }
+
+bool SBReplayOptions::GetVerify() const { return m_opaque_up->verify; }
+
+void SBReplayOptions::SetCheckVersion(bool check) {
+  m_opaque_up->check_version = check;
+}
+
+bool SBReplayOptions::GetCheckVersion() const {
+  return m_opaque_up->check_version;
+}
+
 SBRegistry::SBRegistry() {
   Registry &R = *this;
 
   RegisterMethods<SBAddress>(R);
   RegisterMethods<SBAttachInfo>(R);
   RegisterMethods<SBBlock>(R);
   RegisterMethods<SBBreakpoint>(R);
   RegisterMethods<SBBreakpointList>(R);
if (auto *l = lldb_private::repro::Reproducer::Instance().GetLoader()) {
     static ReplayData r(std::move(*error_or_file));
     InstrumentationData::Initialize(r.GetDeserializer(), r.GetRegistry());
   }
 
   return nullptr;
 }
 
 const char *SBReproducer::Replay(const char *path) {
-  return SBReproducer::Replay(path, false);
+  SBReplayOptions options;
+  return SBReproducer::Replay(path, options);
 }
 
 const char *SBReproducer::Replay(const char *path, bool skip_version_check) {
+  SBReplayOptions options;
+  options.SetCheckVersion(!skip_version_check);
+  return SBReproducer::Replay(path, options);
+}
+
+const char *SBReproducer::Replay(const char *path,
+                                 const SBReplayOptions &options) {
   static std::string error;
   if (auto e = Reproducer::Initialize(ReproducerMode::Replay, FileSpec(path))) {
     error = llvm::toString(std::move(e));
     return error.c_str();
   }
 
   repro::Loader *loader = repro::Reproducer::Instance().GetLoader();
   if (!loader) {
     error = "unable to get replay loader.";
     return error.c_str();
   }
 
-  if (!skip_version_check) {
+  if (options.GetCheckVersion()) {
     llvm::Expected<std::string> version = loader->LoadBuffer<VersionProvider>();
     if (!version) {
       error = llvm::toString(version.takeError());
       return error.c_str();
     }
     if (lldb_private::GetVersion() != llvm::StringRef(*version).rtrim()) {
       error = "reproducer capture and replay version don't match:\n";
       error.append("reproducer captured with:\n");
       error.append(*version);
       error.append("reproducer replayed with:\n");
       error.append(lldb_private::GetVersion());
       return error.c_str();
     }
   }
 
+  if (options.GetVerify()) {
+    bool verification_failed = false;
+    llvm::raw_string_ostream os(error);
+    auto error_callback = [&](llvm::StringRef error) {
+      verification_failed = true;
+      os << "\nerror: " << error;
+    };
+
+    auto warning_callback = [&](llvm::StringRef warning) {
+      verification_failed = true;
+      os << "\nwarning: " << warning;
+    };
+
+    auto note_callback = [&](llvm::StringRef warning) {};
+
+    Verifier verifier(loader);
+    verifier.Verify(error_callback, warning_callback, note_callback);
+
+    if (verification_failed) {
+      os.flush();
+      return error.c_str();
+    }
+  }
+
   FileSpec file = loader->GetFile<SBProvider::Info>();
   if (!file) {
     error = "unable to get replay data from reproducer.";
     return error.c_str();
   }
 
   SBRegistry registry;
   registry.Replay(file);

lldb/source/Commands/CommandObjectReproducer.cpp

 
 static constexpr OptionEnumValues ReproducerSignalType() {
   return OptionEnumValues(g_reproducer_signaltype);
 }
 
 #define LLDB_OPTIONS_reproducer_xcrash
 #include "CommandOptions.inc"
 
+#define LLDB_OPTIONS_reproducer_verify
+#include "CommandOptions.inc"
+
 template <typename T>
 llvm::Expected<T> static ReadFromYAML(StringRef filename) {
   auto error_or_file = MemoryBuffer::getFile(filename);
   if (auto err = error_or_file.getError()) {
     return errorCodeToError(err);
   }
 
   T t;
   yaml::Input yin((*error_or_file)->getBuffer());
   yin >> t;
 
   if (auto err = yin.error()) {
     return errorCodeToError(err);
   }
 
   return t;
 }
 
+static void SetError(CommandReturnObject &result, Error err) {
+  result.GetErrorStream().Printf("error: %s\n",
+                                 toString(std::move(err)).c_str());
+  result.SetStatus(eReturnStatusFailed);
+}
+
+/// Create a loader from the given path if specified. Otherwise use the current
+/// loader used for replay.
+static Loader *
+GetLoaderFromPathOrCurrent(llvm::Optional<Loader> &loader_storage,
+                           CommandReturnObject &result,
+                           FileSpec reproducer_path) {
+  if (reproducer_path) {
+    loader_storage.emplace(reproducer_path);
+    Loader *loader = &(*loader_storage);
+    if (Error err = loader->LoadIndex()) {
+      // This is a hard error and will set the result to eReturnStatusFailed.
+      SetError(result, std::move(err));
+      return nullptr;
+    }
+    return loader;
+  }
+
+  if (Loader *loader = Reproducer::Instance().GetLoader())
+    return loader;
+
+  // This is a soft error because this is expected to fail during capture.
+  result.SetError("Not specifying a reproducer is only support during replay.");
+  result.SetStatus(eReturnStatusSuccessFinishNoResult);
+  return nullptr;
+}
+
 class CommandObjectReproducerGenerate : public CommandObjectParsed {
 public:
   CommandObjectReproducerGenerate(CommandInterpreter &interpreter)
       : CommandObjectParsed(
             interpreter, "reproducer generate",
             "Generate reproducer on disk. When the debugger is in capture "
             "mode, this command will output the reproducer to a directory on "
             "disk and quit. In replay mode this command in a no-op.",
if (Generator *g = r.GetGenerator()) {
         result.GetOutputStream() << "Auto generate: on\n";
     }
 
     result.SetStatus(eReturnStatusSuccessFinishResult);
     return result.Succeeded();
   }
 };
 
-static void SetError(CommandReturnObject &result, Error err) {
-  result.GetErrorStream().Printf("error: %s\n",
-                                 toString(std::move(err)).c_str());
-  result.SetStatus(eReturnStatusFailed);
-}
-
 class CommandObjectReproducerDump : public CommandObjectParsed {
 public:
   CommandObjectReproducerDump(CommandInterpreter &interpreter)
       : CommandObjectParsed(interpreter, "reproducer dump",
                             "Dump the information contained in a reproducer. "
                             "If no reproducer is specified during replay, it "
                             "dumps the content of the current reproducer.",
                             nullptr) {}

 protected:
   bool DoExecute(Args &command, CommandReturnObject &result) override {
     if (!command.empty()) {
       result.AppendErrorWithFormat("'%s' takes no arguments",
                                    m_cmd_name.c_str());
       return false;
     }
 
-    // If no reproducer path is specified, use the loader currently used for
-    // replay. Otherwise create a new loader just for dumping.
     llvm::Optional<Loader> loader_storage;
-    Loader *loader = nullptr;
-    if (!m_options.file) {
-      loader = Reproducer::Instance().GetLoader();
-      if (loader == nullptr) {
-        result.SetError(
-            "Not specifying a reproducer is only support during replay.");
-        result.SetStatus(eReturnStatusSuccessFinishNoResult);
-        return false;
-      }
-    } else {
-      loader_storage.emplace(m_options.file);
-      loader = &(*loader_storage);
-      if (Error err = loader->LoadIndex()) {
-        SetError(result, std::move(err));
-        return false;
-      }
-    }
-
-    // If we get here we should have a valid loader.
-    assert(loader);
+    Loader *loader =
+        GetLoaderFromPathOrCurrent(loader_storage, result, m_options.file);
+    if (!loader)
 
     switch (m_options.provider) {
     case eReproducerProviderFiles: {
       FileSpec vfs_mapping = loader->GetFile<FileProvider::Info>();
 
       // Read the VFS mapping.
       ErrorOr<std::unique_ptr<MemoryBuffer>> buffer =
           vfs::getRealFileSystem()->getBufferForFile(vfs_mapping.GetPath());
bool DoExecute(Args &command, CommandReturnObject &result) override {
     result.SetStatus(eReturnStatusSuccessFinishNoResult);
     return result.Succeeded();
   }
 
 private:
   CommandOptions m_options;
 };
 
+class CommandObjectReproducerVerify : public CommandObjectParsed {
+public:
+  CommandObjectReproducerVerify(CommandInterpreter &interpreter)
+      : CommandObjectParsed(interpreter, "reproducer verify",
+                            "Verify the contents of a reproducer. "
+                            "If no reproducer is specified during replay, it "
+                            "verifies the content of the current reproducer.",
+                            nullptr) {}
+
+  ~CommandObjectReproducerVerify() override = default;
+
+  Options *GetOptions() override { return &m_options; }
+
+  class CommandOptions : public Options {
+  public:
+    CommandOptions() : Options(), file() {}
+
+    ~CommandOptions() override = default;
+
+    Status SetOptionValue(uint32_t option_idx, StringRef option_arg,
+                          ExecutionContext *execution_context) override {
+      Status error;
+      const int short_option = m_getopt_table[option_idx].val;
+
+      switch (short_option) {
+      case 'f':
+        file.SetFile(option_arg, FileSpec::Style::native);
+        FileSystem::Instance().Resolve(file);
+        break;
+      default:
+        llvm_unreachable("Unimplemented option");
+      }
+
+      return error;
+    }
+
+    void OptionParsingStarting(ExecutionContext *execution_context) override {
+      file.Clear();
+    }
+
+    ArrayRef<OptionDefinition> GetDefinitions() override {
+      return makeArrayRef(g_reproducer_verify_options);
+    }
+
+    FileSpec file;
+  };
+
+protected:
+  bool DoExecute(Args &command, CommandReturnObject &result) override {
+    if (!command.empty()) {
+      result.AppendErrorWithFormat("'%s' takes no arguments",
+                                   m_cmd_name.c_str());
+      return false;
+    }
+
+    llvm::Optional<Loader> loader_storage;
+    Loader *loader =
+        GetLoaderFromPathOrCurrent(loader_storage, result, m_options.file);
+    if (!loader)
+      return false;
+
+    bool errors = false;
+    auto error_callback = [&](llvm::StringRef error) {
+      errors = true;
+      result.AppendError(error);
+    };
+
+    bool warnings = false;
+    auto warning_callback = [&](llvm::StringRef warning) {
+      warnings = true;
+      result.AppendWarning(warning);
+    };
+
+    auto note_callback = [&](llvm::StringRef warning) {
+      result.AppendMessage(warning);
+    };
+
+    Verifier verifier(loader);
+    verifier.Verify(error_callback, warning_callback, note_callback);
+
+    if (warnings || errors) {
+      result.AppendMessage("reproducer verification failed");
+      result.SetStatus(eReturnStatusFailed);
+    } else {
+      result.AppendMessage("reproducer verification succeeded");
+      result.SetStatus(eReturnStatusSuccessFinishResult);
+    }
+
+    return result.Succeeded();
+  }
+
+private:
+  CommandOptions m_options;
+};
+
 CommandObjectReproducer::CommandObjectReproducer(
     CommandInterpreter &interpreter)
     : CommandObjectMultiword(
           interpreter, "reproducer",
           "Commands for manipulating reproducers. Reproducers make it "
           "possible "
           "to capture full debug sessions with all its dependencies. The "
           "resulting reproducer is used to replay the debug session while "
           "debugging the debugger.\n"
           "Because reproducers need the whole the debug session from "
           "beginning to end, you need to launch the debugger in capture or "
           "replay mode, commonly though the command line driver.\n"
           "Reproducers are unrelated record-replay debugging, as you cannot "
           "interact with the debugger during replay.\n",
           "reproducer <subcommand> [<subcommand-options>]") {
   LoadSubCommand(
       "generate",
       CommandObjectSP(new CommandObjectReproducerGenerate(interpreter)));
   LoadSubCommand("status", CommandObjectSP(
                                new CommandObjectReproducerStatus(interpreter)));
   LoadSubCommand("dump",
                  CommandObjectSP(new CommandObjectReproducerDump(interpreter)));
+  LoadSubCommand("verify", CommandObjectSP(
+                               new CommandObjectReproducerVerify(interpreter)));
   LoadSubCommand("xcrash", CommandObjectSP(
                                new CommandObjectReproducerXCrash(interpreter)));
 }
 
 CommandObjectReproducer::~CommandObjectReproducer() = default;

lldb/source/Commands/Options.td

let Command = "reproducer dump" in {
   def reproducer_provider : Option<"provider", "p">, Group<1>,
     EnumArg<"None", "ReproducerProviderType()">,
     Required, Desc<"The reproducer provider to dump.">;
   def reproducer_file : Option<"file", "f">, Group<1>, Arg<"Filename">,
     Desc<"The reproducer path. If a reproducer is replayed and no path is "
     "provided, that reproducer is dumped.">;
 }
 
+let Command = "reproducer verify" in {
+  def reproducer_verify_file : Option<"file", "f">, Group<1>, Arg<"Filename">,
+    Desc<"The reproducer path. If a reproducer is replayed and no path is "
+    "provided, that reproducer is dumped.">;
+}
+
 let Command = "reproducer xcrash" in {
   def reproducer_signal : Option<"signal", "s">, Group<1>,
     EnumArg<"None", "ReproducerSignalType()">,
     Required, Desc<"The signal to crash the debugger.">;
 }
 
 let Command = "memory read" in {
   def memory_read_num_per_line : Option<"num-per-line", "l">, Group<1>,

lldb/source/Utility/Reproducer.cpp

llvm::Error Loader::LoadIndex() {
   return llvm::Error::success();
 }
 
 bool Loader::HasFile(StringRef file) {
   assert(m_loaded);
   auto it = std::lower_bound(m_files.begin(), m_files.end(), file.str());
   return (it != m_files.end()) && (*it == file);
 }
+
+void Verifier::Verify(
+    llvm::function_ref<void(llvm::StringRef)> error_callback,
+    llvm::function_ref<void(llvm::StringRef)> warning_callback,
+    llvm::function_ref<void(llvm::StringRef)> note_callack) const {
+  if (!m_loader) {
+    error_callback("invalid loader");
+    return;
+  }
+
+  FileSpec vfs_mapping = m_loader->GetFile<FileProvider::Info>();
+  ErrorOr<std::unique_ptr<MemoryBuffer>> buffer =
+      vfs::getRealFileSystem()->getBufferForFile(vfs_mapping.GetPath());
+  if (!buffer) {
+    error_callback("unable to read files: " + buffer.getError().message());
+    return;
+  }
+
+  IntrusiveRefCntPtr<vfs::FileSystem> vfs = vfs::getVFSFromYAML(
+      std::move(buffer.get()), nullptr, vfs_mapping.GetPath());
+  if (!vfs) {
+    error_callback("unable to initialize the virtual file system");
+    return;
+  }
+
+  auto &redirecting_vfs = static_cast<vfs::RedirectingFileSystem &>(*vfs);
+  redirecting_vfs.setFallthrough(false);
+
+  {
+    llvm::Expected<std::string> working_dir =
+        GetDirectoryFrom<WorkingDirectoryProvider>(m_loader);
+    if (working_dir) {
+      if (!vfs->exists(*working_dir))
+        warning_callback("working directory '" + *working_dir + "' not in VFS");
+      vfs->setCurrentWorkingDirectory(*working_dir);
+    } else {
+      warning_callback("no working directory in reproducer: " +
+                       toString(working_dir.takeError()));
+    }
+  }
+
+  {
+    llvm::Expected<std::string> home_dir =
+        GetDirectoryFrom<HomeDirectoryProvider>(m_loader);
+    if (home_dir) {
+      if (!vfs->exists(*home_dir))
+        warning_callback("home directory '" + *home_dir + "' not in VFS");
+    } else {
+      warning_callback("no home directory in reproducer: " +
+                       toString(home_dir.takeError()));
+    }
+  }
+
+  {
+    Expected<std::string> symbol_files =
+        m_loader->LoadBuffer<SymbolFileProvider>();
+    if (symbol_files) {
+      std::vector<SymbolFileProvider::Entry> entries;
+      llvm::yaml::Input yin(*symbol_files);
+      yin >> entries;
+      for (const auto &entry : entries) {
+        if (!entry.module_path.empty() && !vfs->exists(entry.module_path)) {
+          warning_callback("'" + entry.module_path + "': module path for " +
+                           entry.uuid + " not in VFS");
+        }
+        if (!entry.symbol_path.empty() && !vfs->exists(entry.symbol_path)) {
+          warning_callback("'" + entry.symbol_path + "': symbol path for " +
+                           entry.uuid + " not in VFS");
+        }
+      }
+    } else {
+      llvm::consumeError(symbol_files.takeError());
+    }
+  }
+
+  // Missing files in the VFS are notes rather than warnings. Because the VFS
+  // is a snapshot, temporary files could have been removed between when they
+  // were recorded and when the reproducer was generated.
+  std::vector<llvm::StringRef> roots = redirecting_vfs.getRoots();
+  for (llvm::StringRef root : roots) {
+    std::error_code ec;
+    vfs::recursive_directory_iterator iter(*vfs, root, ec);
+    vfs::recursive_directory_iterator end;
+    for (; iter != end && !ec; iter.increment(ec)) {
+      ErrorOr<vfs::Status> status = vfs->status(iter->path());
+      if (!status)
+        note_callack("'" + iter->path().str() +
+                     "': " + status.getError().message());
+    }
+  }
+}

lldb/source/Utility/ReproducerProvider.cpp

 //===-- Reproducer.cpp ----------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "lldb/Utility/ReproducerProvider.h"
 #include "lldb/Utility/ProcessInfo.h"
 #include "llvm/Support/FileSystem.h"
+#include "llvm/Support/WithColor.h"
 #include "llvm/Support/raw_ostream.h"
 
 using namespace lldb_private;
 using namespace lldb_private::repro;
 using namespace llvm;
 using namespace llvm::yaml;
 
 llvm::Expected<std::unique_ptr<DataRecorder>>

lldb/test/Shell/Reproducer/TestDebugSymbols.test

 # REQUIRES: system-darwin
 
 # RUN: rm -rf %t.repro
 # RUN: env LLDB_APPLE_DSYMFORUUID_EXECUTABLE=%S/Inputs/dsymforuuid.sh %lldb --capture --capture-path %t.repro -c %S/Inputs/core -o 'reproducer generate'
 
 # RUN: cat %t.repro/symbol-files.yaml | FileCheck %s --check-prefix YAML
 # YAML: AD52358C-94F8-3796-ADD6-B20FFAC00E5C
 # YAML: /path/to/unstripped/executable
 # YAML: /path/to/foo.dSYM/Contents/Resources/DWARF/foo
 
 # RUN: %lldb -b -o 'reproducer dump -p symbol-files -f %t.repro' | FileCheck %s --check-prefix DUMP
 # DUMP: uuid: AD52358C-94F8-3796-ADD6-B20FFAC00E5C
 # DUMP-NEXT: module path: /path/to/unstripped/executable
 # DUMP-NEXT: symbol path: /path/to/foo.dSYM/Contents/Resources/DWARF/foo
+
+# RUN: not %lldb -b -o 'reproducer verify -f %t.repro' 2>&1 | FileCheck %s --check-prefix VERIFY
+# VERIFY: warning: '/path/to/unstripped/executable': module path for AD52358C-94F8-3796-ADD6-B20FFAC00E5C not in VFS
+# VERIFY: warning: '/path/to/foo.dSYM/Contents/Resources/DWARF/foo': symbol path for AD52358C-94F8-3796-ADD6-B20FFAC00E5C not in VFS

lldb/test/Shell/Reproducer/TestVerify.test

+# RUN: rm -rf %t.repro
+# RUN: rm -rf %t.repro2
+# RUN: %clang_host %S/Inputs/simple.c -g -o %t.out
+# RUN: %lldb -x -b -s %S/Inputs/GDBRemoteCapture.in --capture --capture-path %t.repro %t.out
+# RUN: %lldb --replay %t.repro
+
+# RUN: echo "/bogus/home/dir" > %t.repro/home.txt
+# RUN: echo "/bogus/current/working/dir" > %t.repro/cwd.txt
+
+# RUN: not %lldb -b -o 'reproducer verify -f %t.repro' 2>&1 | FileCheck %s
+# CHECK: working directory '/bogus/current/working/dir' not in VFS
+# CHECK: home directory '/bogus/home/dir' not in VFS
+
+# RUN: rm %t.repro/root/%S/Inputs/GDBRemoteCapture.in
+# RUN: echo "CHECK: '%S/Inputs/GDBRemoteCapture.in': No such file or directory" > %t.check
+# RUN: not %lldb -b -o 'reproducer verify -f %t.repro' 2>&1 | FileCheck %t.check
+
+# RUN: not %lldb --replay %t.repro 2>&1 | FileCheck %s
+
+# At this point the reproducer is too broken to ignore the verification issues.
+# Capture a new reproducer and only change the home directory, which is
+# recoverable as far as this test goes.
+
+# RUN: %lldb -x -b -s %S/Inputs/GDBRemoteCapture.in --capture --capture-path %t.repro2 %t.out
+# RUN: echo "/bogus/home/dir" > %t.repro2/home.txt
+# RUN: %lldb --replay %t.repro2 --reproducer-no-verify 2>&1 | FileCheck %s --check-prefix NO-VERIFY
+# NO-VERIFY-NOT: home directory '/bogus/home/dir' not in VFS

lldb/tools/driver/Driver.cpp

EXAMPLES:
   Note: In REPL mode no file is loaded, so commands specified to run after
   loading the file (via -o or -s) will be ignored.)___";
   llvm::outs() << examples << '\n';
 }
 
 llvm::Optional<int> InitializeReproducer(llvm::StringRef argv0,
                                          opt::InputArgList &input_args) {
   if (auto *replay_path = input_args.getLastArg(OPT_replay)) {
-    const bool no_version_check = input_args.hasArg(OPT_no_version_check);
+    SBReplayOptions replay_options;
+    replay_options.SetCheckVersion(!input_args.hasArg(OPT_no_version_check));
+    replay_options.SetVerify(!input_args.hasArg(OPT_no_verification));
     if (const char *error =
-            SBReproducer::Replay(replay_path->getValue(), no_version_check)) {
+            SBReproducer::Replay(replay_path->getValue(), replay_options)) {
       WithColor::error() << "reproducer replay failed: " << error << '\n';
       return 1;
     }
     return 0;
   }
 
   bool capture = input_args.hasArg(OPT_capture);
   bool generate_on_exit = input_args.hasArg(OPT_generate_on_exit);

lldb/tools/driver/Options.td

 def capture_path: Separate<["--", "-"], "capture-path">,
   MetaVarName<"<filename>">,
   HelpText<"Tells the debugger to use the given filename for the reproducer.">;
 def replay: Separate<["--", "-"], "replay">,
   MetaVarName<"<filename>">,
   HelpText<"Tells the debugger to replay a reproducer from <filename>.">;
 def no_version_check: F<"reproducer-no-version-check">,
   HelpText<"Disable the reproducer version check.">;
+def no_verification: F<"reproducer-no-verify">,
+  HelpText<"Disable the reproducer verification.">;
 def no_generate_on_signal: F<"reproducer-no-generate-on-signal">,
   HelpText<"Don't generate reproducer when a signal is received.">;
 def generate_on_exit: F<"reproducer-generate-on-exit">,
   HelpText<"Generate reproducer on exit.">;
 
 def REM : R<["--"], "">;

llvm/include/llvm/Support/VirtualFileSystem.h

public:
   std::error_code makeAbsolute(SmallVectorImpl<char> &Path) const override;
 
   directory_iterator dir_begin(const Twine &Dir, std::error_code &EC) override;
 
   void setExternalContentsPrefixDir(StringRef PrefixDir);
 
   StringRef getExternalContentsPrefixDir() const;
 
+  void setFallthrough(bool Fallthrough);
+
+  std::vector<llvm::StringRef> getRoots() const;
+
   void dump(raw_ostream &OS) const;
   void dumpEntry(raw_ostream &OS, Entry *E, int NumSpaces = 0) const;
 #if !defined(NDEBUG) || defined(LLVM_ENABLE_DUMP)
   LLVM_DUMP_METHOD void dump() const;
 #endif
 };
 
 /// Collect all pairs of <virtual path, real path> entries from the

llvm/lib/Support/VirtualFileSystem.cpp

 void RedirectingFileSystem::setExternalContentsPrefixDir(StringRef PrefixDir) {
   ExternalContentsPrefixDir = PrefixDir.str();
 }
 
 StringRef RedirectingFileSystem::getExternalContentsPrefixDir() const {
   return ExternalContentsPrefixDir;
 }
 
+void RedirectingFileSystem::setFallthrough(bool Fallthrough) {
+  IsFallthrough = Fallthrough;
+}
+
+std::vector<StringRef> RedirectingFileSystem::getRoots() const {
+  std::vector<StringRef> R;
+  for (const auto &Root : Roots)
+    R.push_back(Root->getName());
+  return R;
+}
+
 void RedirectingFileSystem::dump(raw_ostream &OS) const {
   for (const auto &Root : Roots)
     dumpEntry(OS, Root.get());
 }
 
 void RedirectingFileSystem::dumpEntry(raw_ostream &OS,
                                       RedirectingFileSystem::Entry *E,
                                       int NumSpaces) const {