[Driver] Add compiler option to generate a reproducer

[bruno]

Summary

One way to currently test the reproducers is to setup
"FORCE_CLANG_DIAGNOSTICS_CRASH=1" before invoking clang. This simulates
a crash and produces the same contents needed by the reproducers.

The reproducers are specially useful when triaging Modules issues, not
only on crashes, but also for reproducing misleading warnings, errors,
etc. I propose we add '-gen-reproducer' driver option to clang (or
any similar name) and give users a flag option.

Additionally, clang already have -fno-crash-diagnostics, which disables
the crash reproducers. I've decided not to propose "-fcrash-diagnostics"
since it doesn't convey the ideia of reproduction despite a crash.

[v.g.vassilev]

Thanks for working on this, Bruno!

This would be a useful feature to us, too.

I am wondering if there is clearer name than -gen-reproducer.

IIUC, the goal is to extract all environment (system) dependent pieces, copying them to a folder where the compiler can be chroot-ed (via -isysroot). In some sense, what we do seems close to what -header-include-file list_of_includes.txt does and additionally copying the contents of the list_of_includes.txt (and possibly mounting a vfs overlay).

[mehdi_amini]

include/clang/Basic/DiagnosticDriverKinds.td

95

Is it worth it? What about "failing because %1 is set">;

And then later:

Diags.Report(diag::err_drv_force_crash) << "option '-gen-reproducer'";

[bruno]

I am wondering if there is clearer name than -gen-reproducer.

Probably, I'm open to suggestions :-)

IIUC, the goal is to extract all environment (system) dependent pieces, copying them to a folder where the compiler can be chroot-ed (via -isysroot).

It doesn't necessarily needs a -isysroot to work, only if the original invocation used it, then we need to use it as well to ask the VFS (transparently) for the right path locations.

In some sense, what we do seems close to what -header-include-file list_of_includes.txt does and additionally copying the contents of the list_of_includes.txt (and possibly mounting a vfs overlay).

-header-include-file does something similar but more limited, I added some specific callbacks for the module collector back in the day that it might not collect yet though, for instance, it collects module.modulemap's and will soon collect header maps and some other stuff.

It also has some special handling for symlinks; it doesn't copy both files, but generate two yaml entries that point to the same "physical" location.

[bruno]

Update after Mehdi's review!

[bruno]

Ping!

[v.g.vassilev]

My questions were answered. LGTM!

[rsmith]

LGTM with one change.

include/clang/Basic/DiagnosticDriverKinds.td

95

Even though we don't have translations for our diagnostics, it's intended that they be translatable, so you should not hardcode strings like "option " and "environment variable " in the code. Use a %select here instead, maybe? (See http://clang.llvm.org/docs/InternalsManual.html#the-format-string)

docs/UsersManual.rst

 
 .. option:: -fno-crash-diagnostics
 
   Disable auto-generation of preprocessed source files during a clang crash.
 
 The -fno-crash-diagnostics flag can be helpful for speeding the process
 of generating a delta reduced test case.
 
+Clang is also capable of generating preprocessed source file(s) and associated
+run script(s) even without a crash. This is specially useful when trying to
+generate a reproducer for warnings or errors while using modules.
+
+.. option:: -gen-reproducer
+
+  Generates preprocessed source files, a reproducer script and if relevant, a
+  cache containing: built module pcm's and all headers needed to rebuilt the
+  same modules.
+
 .. _rpass:
 
 Options to Emit Optimization Reports
 ------------------------------------
 
 Optimization reports trace, at a high-level, all the major decisions
 done by compiler transformations. For instance, when the inliner
 decides to inline function ``foo()`` into ``bar()``, or the loop unroller

include/clang/Basic/DiagnosticDriverKinds.td

def err_drv_clang_unsupported_opt_cxx_darwin_i386 : Error<
   "the clang compiler does not support '%0' for C++ on Darwin/i386">;
 def err_drv_command_failed : Error<
   "%0 command failed with exit code %1 (use -v to see invocation)">;
 def err_drv_compilationdatabase : Error<
   "compilation database '%0' could not be opened: %1">;
 def err_drv_command_signalled : Error<
   "%0 command failed due to signal (use -v to see invocation)">;
 def err_drv_force_crash : Error<
-  "failing because environment variable '%0' is set">;
+  "failing because %0 is set">;
 def err_drv_invalid_mfloat_abi : Error<
   "invalid float ABI '%0'">;
 def err_drv_invalid_libcxx_deployment : Error<
   "invalid deployment target for -stdlib=libc++ (requires %0 or later)">;
 def err_drv_invalid_argument_to_fdebug_prefix_map : Error<
   "invalid argument '%0' to -fdebug-prefix-map">;
 def err_drv_malformed_sanitizer_blacklist : Error<
   "malformed sanitizer blacklist: '%0'">;

include/clang/Driver/Driver.h

private:
   /// Whether to check that input files exist when constructing compilation
   /// jobs.
   unsigned CheckInputsExist : 1;
 
 public:
   /// Use lazy precompiled headers for PCH support.
   unsigned CCCUsePCH : 1;
 
+  /// Force clang to emit reproducer for driver invocation. This is enabled
+  /// indirectly by setting FORCE_CLANG_DIAGNOSTICS_CRASH environment variable
+  /// or when using the -gen-reproducer driver flag.
+  unsigned GenReproducer : 1;
+
 private:
   /// Certain options suppress the 'no input files' warning.
   unsigned SuppressMissingInputWarning : 1;
 
   std::list<std::string> TempFiles;
   std::list<std::string> ResultFiles;
 
   /// \brief Cache of all the ToolChains in use by the driver.

include/clang/Driver/Options.td

def ccc_arcmt_modify : Flag<["-"], "ccc-arcmt-modify">, InternalDriverOpt,
   HelpText<"Apply modifications to files to conform to ARC">;
 def ccc_arcmt_migrate : Separate<["-"], "ccc-arcmt-migrate">, InternalDriverOpt,
   HelpText<"Apply modifications and produces temporary files that conform to ARC">;
 def arcmt_migrate_report_output : Separate<["-"], "arcmt-migrate-report-output">,
   HelpText<"Output path for the plist report">,  Flags<[CC1Option]>;
 def arcmt_migrate_emit_arc_errors : Flag<["-"], "arcmt-migrate-emit-errors">,
   HelpText<"Emit ARC errors even if the migrator can fix them">,
   Flags<[CC1Option]>;
+def gen_reproducer: Flag<["-"], "gen-reproducer">, InternalDebugOpt,
+  HelpText<"Auto-generates preprocessed source files and a reproduction script">;
 
 def _migrate : Flag<["--"], "migrate">, Flags<[DriverOption]>,
   HelpText<"Run the migrator">;
 def ccc_objcmt_migrate : Separate<["-"], "ccc-objcmt-migrate">,
   InternalDriverOpt,
   HelpText<"Apply modifications and produces temporary files to migrate to "
    "modern ObjC syntax">;
 def objcmt_migrate_literals : Flag<["-"], "objcmt-migrate-literals">, Flags<[CC1Option]>,

 def fcommon : Flag<["-"], "fcommon">, Group<f_Group>;
 def fcompile_resource_EQ : Joined<["-"], "fcompile-resource=">, Group<f_Group>;
 def fconstant_cfstrings : Flag<["-"], "fconstant-cfstrings">, Group<f_Group>;
 def fconstant_string_class_EQ : Joined<["-"], "fconstant-string-class=">, Group<f_Group>;
 def fconstexpr_depth_EQ : Joined<["-"], "fconstexpr-depth=">, Group<f_Group>;
 def fconstexpr_steps_EQ : Joined<["-"], "fconstexpr-steps=">, Group<f_Group>;
 def fconstexpr_backtrace_limit_EQ : Joined<["-"], "fconstexpr-backtrace-limit=">,
                                     Group<f_Group>;
-def fno_crash_diagnostics : Flag<["-"], "fno-crash-diagnostics">, Group<f_clang_Group>, Flags<[NoArgumentUnused]>;
+def fno_crash_diagnostics : Flag<["-"], "fno-crash-diagnostics">, Group<f_clang_Group>, Flags<[NoArgumentUnused]>,
+  HelpText<"Disable auto-generation of preprocessed source files and a script for reproduction during a clang crash">;
 def fcreate_profile : Flag<["-"], "fcreate-profile">, Group<f_Group>;
 def fcxx_exceptions: Flag<["-"], "fcxx-exceptions">, Group<f_Group>,
   HelpText<"Enable C++ exceptions">, Flags<[CC1Option]>;
 def fcxx_modules : Flag <["-"], "fcxx-modules">, Group<f_Group>,
   Flags<[DriverOption]>;
 def fdebug_pass_arguments : Flag<["-"], "fdebug-pass-arguments">, Group<f_Group>;
 def fdebug_pass_structure : Flag<["-"], "fdebug-pass-structure">, Group<f_Group>;
 def fdepfile_entry : Joined<["-"], "fdepfile-entry=">,

lib/Driver/Driver.cpp

: Opts(createDriverOptTable()), Diags(Diags), VFS(std::move(VFS)),
       Mode(GCCMode), SaveTemps(SaveTempsNone), BitcodeEmbed(EmbedNone),
       LTOMode(LTOK_None), ClangExecutable(ClangExecutable),
       SysRoot(DEFAULT_SYSROOT), UseStdLib(true),
       DriverTitle("clang LLVM compiler"), CCPrintOptionsFilename(nullptr),
       CCPrintHeadersFilename(nullptr), CCLogDiagnosticsFilename(nullptr),
       CCCPrintBindings(false), CCPrintHeaders(false), CCLogDiagnostics(false),
       CCGenDiagnostics(false), DefaultTargetTriple(DefaultTargetTriple),
       CCCGenericGCCName(""), CheckInputsExist(true), CCCUsePCH(true),
-      SuppressMissingInputWarning(false) {
+      GenReproducer(false), SuppressMissingInputWarning(false) {
 
   // Provide a sane fallback if no VFS is specified.
   if (!this->VFS)
     this->VFS = vfs::getRealFileSystem();
 
   Name = llvm::sys::path::filename(ClangExecutable);
   Dir = llvm::sys::path::parent_path(ClangExecutable);
   InstalledDir = Dir; // Provide a sensible default installed dir.
Compilation *Driver::BuildCompilation(ArrayRef<const char *> ArgList) {
   // options, either by introducing new ones or by overloading gcc ones like -V
   // or -b.
   CCCPrintPhases = Args.hasArg(options::OPT_ccc_print_phases);
   CCCPrintBindings = Args.hasArg(options::OPT_ccc_print_bindings);
   if (const Arg *A = Args.getLastArg(options::OPT_ccc_gcc_name))
     CCCGenericGCCName = A->getValue();
   CCCUsePCH =
       Args.hasFlag(options::OPT_ccc_pch_is_pch, options::OPT_ccc_pch_is_pth);
+  GenReproducer = Args.hasFlag(options::OPT_gen_reproducer,
+                               options::OPT_fno_crash_diagnostics,
+                               !!::getenv("FORCE_CLANG_DIAGNOSTICS_CRASH"));
   // FIXME: DefaultTargetTriple is used by the target-prefixed calls to as/ld
   // and getToolChain is const.
   if (IsCLMode()) {
     // clang-cl targets MSVC-style Win32.
     llvm::Triple T(DefaultTargetTriple);
     T.setOS(llvm::Triple::Win32);
     T.setVendor(llvm::Triple::PC);
     T.setEnvironment(llvm::Triple::MSVC);

test/Driver/crash-report-crashfile.m

 // REQUIRES: crash-recovery, shell, system-darwin
 // RUN: rm -rf %t
 // RUN: mkdir -p %t/i %t/m %t
 
 // RUN: not env FORCE_CLANG_DIAGNOSTICS_CRASH= TMPDIR=%t TEMP=%t TMP=%t \
-// RUN: %clang -fsyntax-only %s -I %S/Inputs/module -isysroot %/t/i/    \
-// RUN: -fmodules -fmodules-cache-path=%t/m/ -DFOO=BAR 2>&1 | FileCheck %s
+// RUN: %clang -fsyntax-only %s \
+// RUN:   -I %S/Inputs/module -isysroot %/t/i/ \
+// RUN:   -fmodules -fmodules-cache-path=%t/m/ -DFOO=BAR 2>&1 | \
+// RUN:   FileCheck -check-prefix=CRASH_ENV %s
+
+// RUN: not env TMPDIR=%t TEMP=%t TMP=%t \
+// RUN: %clang -gen-reproducer -fsyntax-only %s \
+// RUN:   -I %S/Inputs/module -isysroot %/t/i/ \
+// RUN:   -fmodules -fmodules-cache-path=%t/m/ -DFOO=BAR 2>&1 | \
+// RUN:   FileCheck -check-prefix=CRASH_FLAG %s
 
 @import simple;
 const int x = MODULE_MACRO;
 
-// CHECK: Preprocessed source(s) and associated run script(s) are located at:
-// CHECK-NEXT: note: diagnostic msg: {{.*}}.m
-// CHECK-NEXT: note: diagnostic msg: {{.*}}.cache
-// CHECK-NEXT: note: diagnostic msg: {{.*}}.sh
-// CHECK-NEXT: note: diagnostic msg: Crash backtrace is located in
-// CHECK-NEXT: note: diagnostic msg: {{.*}}Library/Logs/DiagnosticReports{{.*}}
+// CRASH_ENV: failing because environment variable 'FORCE_CLANG_DIAGNOSTICS_CRASH' is set
+// CRASH_ENV: Preprocessed source(s) and associated run script(s) are located at:
+// CRASH_ENV-NEXT: note: diagnostic msg: {{.*}}.m
+// CRASH_ENV-NEXT: note: diagnostic msg: {{.*}}.cache
+// CRASH_ENV-NEXT: note: diagnostic msg: {{.*}}.sh
+// CRASH_ENV-NEXT: note: diagnostic msg: Crash backtrace is located in
+// CRASH_ENV-NEXT: note: diagnostic msg: {{.*}}Library/Logs/DiagnosticReports{{.*}}
+
+// CRASH_FLAG: failing because option '-gen-reproducer' is set
+// CRASH_FLAG: Preprocessed source(s) and associated run script(s) are located at:
+// CRASH_FLAG-NEXT: note: diagnostic msg: {{.*}}.m
+// CRASH_FLAG-NEXT: note: diagnostic msg: {{.*}}.cache
+// CRASH_FLAG-NEXT: note: diagnostic msg: {{.*}}.sh
+// CRASH_FLAG-NEXT: note: diagnostic msg: Crash backtrace is located in
+// CRASH_FLAG-NEXT: note: diagnostic msg: {{.*}}Library/Logs/DiagnosticReports{{.*}}

tools/driver/driver.cpp

int main(int argc_, const char **argv_) {
 
   std::unique_ptr<Compilation> C(TheDriver.BuildCompilation(argv));
   int Res = 0;
   SmallVector<std::pair<int, const Command *>, 4> FailingCommands;
   if (C.get())
     Res = TheDriver.ExecuteCompilation(*C, FailingCommands);
 
   // Force a crash to test the diagnostics.
-  if (::getenv("FORCE_CLANG_DIAGNOSTICS_CRASH")) {
-    Diags.Report(diag::err_drv_force_crash) << "FORCE_CLANG_DIAGNOSTICS_CRASH";
+  if (TheDriver.GenReproducer) {
+    Diags.Report(diag::err_drv_force_crash)
+        << (!::getenv("FORCE_CLANG_DIAGNOSTICS_CRASH")
+                ? "option '-gen-reproducer'"
+                : "environment variable 'FORCE_CLANG_DIAGNOSTICS_CRASH'");
 
     // Pretend that every command failed.
     FailingCommands.clear();
     for (const auto &J : C->getJobs())
       if (const Command *C = dyn_cast<Command>(&J))
         FailingCommands.push_back(std::make_pair(-1, C));
   }