[Driver] Re-run lld with --reproduce when it crashes

[abrachet]

Summary

This was discussed on https://discourse.llvm.org/t/rfc-generating-lld-reproducers-on-crashes/58071/12

When lld crashes, or errors when -gen-reproducer=error and -fcrash-diagnostics=all clang will re-run lld with --reproduce=$temp_file for easily reproducing the crash/error.

[MaskRay]

Thanks. I think this is more appropriate than D102892.
I am thinking that whether this should be opt-in, possibly with an environment variable or option.

clang/include/clang/Driver/Driver.h

511

Remove generateLinkerDiagnostics -.
https://llvm.org/docs/CodingStandards.html#commenting "Don’t duplicate function or class name at the beginning of the comment"

[aganea]

There is currently no testing because, although there is testing for generateCompilationDiagnostics, it is harder to force lld to crash, even if we added a way to crash lld on purpose, the test would need to depend on lld. Looking for suggestions if anyone has any.

I think you add test coverage in LLD, through a environment variable that triggers a crash in any of the drivers & --reproduce to ensure the archive was created.

clang/lib/Driver/Driver.cpp

1550

Instead of adding generateLinkerDiagnostics above, would it be possible to leverage the same codepath as when Clang crashes, ie. initCompilationForDiagnostics() and then add --reproduce when building linker options, based on C.isForDiagnostics()? (and use C.addTempFile())

[abrachet]

Add -fcrash_diagnostics which can accept a level to determine if lld should be re-run with --reproduce

[abrachet]

In D120175#3335453, @aganea wrote:

There is currently no testing because, although there is testing for generateCompilationDiagnostics, it is harder to force lld to crash, even if we added a way to crash lld on purpose, the test would need to depend on lld. Looking for suggestions if anyone has any.

I think you add test coverage in LLD, through a environment variable that triggers a crash in any of the drivers & --reproduce to ensure the archive was created.

lld has tests for --reproduce already. Even if I added an environment variable to crash lld, the test would depend on lld being built, adding a big dependency to clang's tests. Do we think that's worth it?

In D120175#3333948, @MaskRay wrote:

Thanks. I think this is more appropriate than D102892.
I am thinking that whether this should be opt-in, possibly with an environment variable or option.

Sounds good, how does the current flag look?

clang/lib/Driver/Driver.cpp

1550

I was stuck on this for a while, you're right that it would be better to do it this way. Ultimately I ended up deciding it wasn't worth the increase in complexity to make that possible.

Right now generateCompilationDiagnostics chooses it's jobs based on input files so with clang source.c it doesn't get run. Also currently non preprocess-able sources get filtered out and a job isn't created for them. Changing those ended up being a big hassle and I was worried would add edge cases that we're going to be difficult to think of to test.

[abrachet]

Ping :)

[aganea]

In D120175#3338696, @abrachet wrote:

In D120175#3335453, @aganea wrote:

There is currently no testing because, although there is testing for generateCompilationDiagnostics, it is harder to force lld to crash, even if we added a way to crash lld on purpose, the test would need to depend on lld. Looking for suggestions if anyone has any.

I think you add test coverage in LLD, through a environment variable that triggers a crash in any of the drivers & --reproduce to ensure the archive was created.

lld has tests for --reproduce already. Even if I added an environment variable to crash lld, the test would depend on lld being built, adding a big dependency to clang's tests. Do we think that's worth it?

It would be a interesting addition to bind LLVM_LINKER_IS_LLD to a LIT test "feature" -- in a follow-up patch if you're willing to? (see config.available_features.add('z3') and LLVM_WITH_Z3 in clang/test/lit.cfg.py for an example)

clang/docs/UsersManual.rst

670

-fcrash-diagnostics (dash not underscore)

675

Shouldn't we be more explicit and set the option to be "compiler"?

clang/include/clang/Driver/Compilation.h

228

Should probably remove (see other comments)

clang/include/clang/Driver/Driver.h

511

s/lld/LLD and please add missing period.

clang/lib/Driver/Driver.cpp

1507

Uppercase Level.

1521

Do you actually need the redirects here? I figure that if LLD re-runs, we don't need to emit the stdout/stderr once again. Actually calling Compilation::initCompilationForDiagnostics() would disable that.

1554

I wonder if you can move this call to C.initCompilationForDiagnostics() just a bit above if (IsLLD) to be able to call Driver::GetNamedOutputPath in generateLinkerDiagnostics? This will prevent the temp files cleanup.

[abrachet]

Add tests when building with lld

[abrachet]

In D120175#3365001, @aganea wrote:

In D120175#3338696, @abrachet wrote:

In D120175#3335453, @aganea wrote:

There is currently no testing because, although there is testing for generateCompilationDiagnostics, it is harder to force lld to crash, even if we added a way to crash lld on purpose, the test would need to depend on lld. Looking for suggestions if anyone has any.

I think you add test coverage in LLD, through a environment variable that triggers a crash in any of the drivers & --reproduce to ensure the archive was created.

lld has tests for --reproduce already. Even if I added an environment variable to crash lld, the test would depend on lld being built, adding a big dependency to clang's tests. Do we think that's worth it?

It would be a interesting addition to bind LLVM_LINKER_IS_LLD to a LIT test "feature" -- in a follow-up patch if you're willing to? (see config.available_features.add('z3') and LLVM_WITH_Z3 in clang/test/lit.cfg.py for an example)

Ah thanks for the tip. I've done it when "lld" is in LLVM_ENABLE_PROJECTS because I need to have a way to crash it, which I have added to lld.

clang/docs/UsersManual.rst

675

Sure

clang/lib/Driver/Driver.cpp

1521

You're right I've just sent output to /dev/null. I've moved the code bellow the initCompilationForDiagnostics call. However the redirects would normally happen in BuildJobs/ExecuteJobs but I'm not using that here. So I just specified the same redirects as in initCompilationForDiagnostics

1554

Done. I pulled out the specific code I'm interested in from GetNamedOutputPath into Driver::CreateTempFile because it manually chooses the suffix with types::getTypeTempSuffix(JA.getType(), IsCLMode()); which will never be "tar"

[abrachet]

Forgot to add test file to the diff

[aganea]

Thanks for making the changes!

You could probably split this in 3 incremental commits, to save on debugging-after-commit & reverts if things go wrong, but it's up to you!

• One commit for the LLD addition and a very simple test in lld/test/ that triggers config->forceCrash.
• One for "Clang using LLD" LIT option, and a simple test such as clang++ main.cpp -### and check if LLD is there. Then ensuring the bots that have LLD actually are passing the test.
• One for extending fcrash-diagnostics= which is the main purpose of this patch.

clang/lib/Driver/Driver.cpp

1635

TmpName?

lld/ELF/Driver.cpp

1208 ↗

(On Diff #415265)

You could do config->forceCrash = getenv("LLD_CRASH");. I would add a very simple test in LLD as well.

lld/ELF/Options.td

718 ↗

(On Diff #415265)

I think it'd be better if wasn't an explicit option, but a environment variable instead. @MaskRay an opinion on this?

[abrachet]

Split up into 3 separate patches. See D121724 and D121725

[abrachet]

This no longer depends of previous dependent revisions because I found an easier way to check if clang is being built with lld, and since -gen-reproducer changes landed, there is no need to depend on lld crashing, just that we can make an error.

[rnk]

Regarding environment variables, all flags are accessible via the environment if your build system propagates CCC_OVERRIDE_OPTIONS:
https://twitter.com/kastiglione/status/1334348223452876800?lang=en

IMO we ought to document this hidden feature. I've mentioned this feature to various folks and some of them say we shouldn't document it because it's a giant hack, but I think it's a great feature.

On the patch at hand, this sounds good to me, we should do it. :) I agree we can't do this by default. It seems OK to generate a ~10MB pre-processed source file in /tmp every time we crash, but a multi-GB archive for every linker crash would quickly fill up most disks.

[abrachet]

Ping. We recently ran into a linker crash and this would have been useful :)

[aganea]

LGTM!

[MaskRay]

LGTM

(Note: This is a patch which I think would benefit from a second look and 2.5h from approval to push was too short for others to react on this patch.)

[abrachet]

In D120175#3691843, @MaskRay wrote:

LGTM

(Note: This is a patch which I think would benefit from a second look and 2.5h from approval to push was too short for others to react on this patch.)

Ack, I'll give more time in the future.

[thakis]

This breaks tests on mac and win:
http://45.33.8.238/macm1/41545/step_7.txt
http://45.33.8.238/win/63579/step_7.txt

Please take a look and revert for now if it takes a while to fix.

[abrachet]

In D120175#3692032, @thakis wrote:

This breaks tests on mac and win:
http://45.33.8.238/macm1/41545/step_7.txt
http://45.33.8.238/win/63579/step_7.txt

Please take a look and revert for now if it takes a while to fix.

Should be fixed in https://github.com/llvm/llvm-project/commit/9028966a717239cf586d6c4f606965d444176dc4

[probinson]

lld has tests for --reproduce already. Even if I added an environment variable to crash lld, the test would depend on lld being built, adding a big dependency to clang's tests. Do we think that's worth it?

There is now cross-project-tests which would seem to be the appropriate place for a test that wants to run both clang and lld.

[thakis]

clang/lib/Driver/Driver.cpp

1638

This does the wrong thing on Windows, I think (the flag is spelled /reproduce:path there I think?)

[abrachet]

In D120175#3693458, @probinson wrote:

There is now cross-project-tests which would seem to be the appropriate place for a test that wants to run both clang and lld.

Thanks for pointing that out. I'll look into moving the test there

clang/lib/Driver/Driver.cpp

1638

Ah yeah good catch. Unfortunately this wasn't caught upstream because I quickly had to change to explicit target because *-ps4 target doesn't support -fuse-ld and there didn't already exist a PS4 lit feature that I could mark as unsupported. This was actually why you saw test failures on non linux platforms.

I'll make a new review that correctly uses /reproduce:. Thanks again

[probinson]

there didn't already exist a PS4 lit feature that I could mark as unsupported.

? I believe UNSUPPORTED: ps4 should work, because UNSUPPORTED will look at the default target triple.

[thakis]

I noticed that when using -fcrash-diagnostics=all and the linker crashes, clang creates both preprocessor and linker repros:

clang: note: diagnostic msg: /var/folders/w6/wpbtszrs7jl9dc9l5qtdkvg00000gn/T/main-241332.c
clang: note: diagnostic msg: /var/folders/w6/wpbtszrs7jl9dc9l5qtdkvg00000gn/T/linker-crash-945b07.tar
clang: note: diagnostic msg: /var/folders/w6/wpbtszrs7jl9dc9l5qtdkvg00000gn/T/main-241332.sh

Is that intentional? I would've expected to only get the linker repro when the linker crashes.

[abrachet]

In D120175#3902789, @thakis wrote:

I noticed that when using -fcrash-diagnostics=all and the linker crashes, clang creates both preprocessor and linker repros:

clang: note: diagnostic msg: /var/folders/w6/wpbtszrs7jl9dc9l5qtdkvg00000gn/T/main-241332.c
clang: note: diagnostic msg: /var/folders/w6/wpbtszrs7jl9dc9l5qtdkvg00000gn/T/linker-crash-945b07.tar
clang: note: diagnostic msg: /var/folders/w6/wpbtszrs7jl9dc9l5qtdkvg00000gn/T/main-241332.sh

Is that intentional? I would've expected to only get the linker repro when the linker crashes.

I didn't really make a conscious decision one way or the other. I guess reproing all jobs leading to a crash somewhat makes sense, though it also doesn't seem that helpful to get the preprocessed source file. I'll send out a patch to fix this

[thakis]

I didn't really make a conscious decision one way or the other. I guess reproing all jobs leading to a crash somewhat makes sense, though it also doesn't seem that helpful to get the preprocessed source file. I'll send out a patch to fix this

Great, thanks :)

For context, we're trying to use this flag In Production to try and debug a linker crash, but making the compiler repro failed for some reason – and then we wondered why that step has to happen at all :)

[abrachet]

In D120175#3902838, @thakis wrote:

Great, thanks :)

For context, we're trying to use this flag In Production to try and debug a linker crash, but making the compiler repro failed for some reason – and then we wondered why that step has to happen at all :)

Nice, I'm happy folks are using (or trying to use) this in production :). We've found that it has saved us quite a bit of time.

D137289 is up to stop generating the preprocessed sources for linker crashes

clang/docs/UsersManual.rst

 Generally, this only occurs to those living on the `bleeding
 edge <https://llvm.org/releases/download.html#svn>`_. Clang goes to great
 lengths to assist you in filing a bug report. Specifically, Clang
 generates preprocessed source file(s) and associated run script(s) upon
 a crash. These files should be attached to a bug report to ease
 reproducibility of the failure. Below are the command line options to
 control the crash diagnostics.
 
+.. option:: -fcrash-diagnostics=<val>
+
+  Valid values are:
+
+  * ``off`` (Disable auto-generation of preprocessed source files during a clang crash.)
+  * ``compiler`` (Generate diagnostics for compiler crashes (default))
+  * ``all`` (Generate diagnostics for all tools which support it)
+
 .. option:: -fno-crash-diagnostics
 
   Disable auto-generation of preprocessed source files during a clang crash.
 
 The -fno-crash-diagnostics flag can be helpful for speeding the process
 of generating a delta reduced test case.
 
 .. option:: -fcrash-diagnostics-dir=<dir>

clang/include/clang/Driver/Compilation.h

template <typename T, typename... Args> T *MakeAction(Args &&... Arg) {
     return RawPtr;
   }
 
   JobList &getJobs() { return Jobs; }
   const JobList &getJobs() const { return Jobs; }
 
   void addCommand(std::unique_ptr<Command> C) { Jobs.addJob(std::move(C)); }
 
+  llvm::opt::ArgStringList &getTempFiles() { return TempFiles; }
   const llvm::opt::ArgStringList &getTempFiles() const { return TempFiles; }
 
   const ArgStringMap &getResultFiles() const { return ResultFiles; }
 
   const ArgStringMap &getFailureResultFiles() const {
     return FailureResultFiles;
   }
 
   /// Installs a handler that is executed when a compilation job is finished.
   /// The arguments of the callback specify the compilation job as an instance
   /// of class Command and the exit status of the child process executed that
   /// job.
   void setPostCallback(const std::function<void(const Command &, int)> &CB) {
     PostCallback = CB;
   }
 
   /// Returns the sysroot path.

clang/include/clang/Driver/Driver.h

public:
   /// generateCompilationDiagnostics - Generate diagnostics information
   /// including preprocessed source file(s).
   ///
   void generateCompilationDiagnostics(
       Compilation &C, const Command &FailingCommand,
       StringRef AdditionalInformation = "",
       CompilationDiagnosticReport *GeneratedReport = nullptr);
 
   enum class CommandStatus {
     Crash = 1,
     Error,
     Ok,
   };
 
   enum class ReproLevel {
     Off = 0,
     OnCrash = static_cast<int>(CommandStatus::Crash),
InputInfoList BuildJobsForAction(
       bool AtTopLevel, bool MultipleArchs, const char *LinkingOutput,
       std::map<std::pair<const Action *, std::string>, InputInfoList>
           &CachedResults,
       Action::OffloadKind TargetDeviceOffloadKind) const;
 
   /// Returns the default name for linked images (e.g., "a.out").
   const char *getDefaultImageName() const;
 
+  // Creates a temp file with $Prefix-%%%%%%.$Suffix
+  const char *CreateTempFile(Compilation &C, StringRef Prefix, StringRef Suffix,
+                             bool MultipleArchs = false,
+                             StringRef BoundArch = {}) const;
+
   /// GetNamedOutputPath - Return the name to use for the output of
   /// the action \p JA. The result is appended to the compilation's
   /// list of temporary or result files, as appropriate.
   ///
   /// \param C - The compilation.
   /// \param JA - The action of interest.
   /// \param BaseInput - The original input file that this action was
   /// triggered by.

clang/include/clang/Driver/Options.td

 def fconstant_string_class_EQ : Joined<["-"], "fconstant-string-class=">, Group<f_Group>;
 def fconstexpr_depth_EQ : Joined<["-"], "fconstexpr-depth=">, Group<f_Group>;
 def fconstexpr_steps_EQ : Joined<["-"], "fconstexpr-steps=">, Group<f_Group>;
 def fexperimental_new_constant_interpreter : Flag<["-"], "fexperimental-new-constant-interpreter">, Group<f_Group>,
   HelpText<"Enable the experimental new constant interpreter">, Flags<[CC1Option]>,
   MarshallingInfoFlag<LangOpts<"EnableNewConstInterp">>;
 def fconstexpr_backtrace_limit_EQ : Joined<["-"], "fconstexpr-backtrace-limit=">,
                                     Group<f_Group>;
+def fcrash_diagnostics_EQ : Joined<["-"], "fcrash-diagnostics=">, Group<f_clang_Group>, Flags<[NoArgumentUnused, CoreOption]>,
+  HelpText<"Set level of crash diagnostic reporting, (option: off, compiler, all)">;
+def fcrash_diagnostics : Flag<["-"], "fcrash-diagnostics">, Group<f_clang_Group>, Flags<[NoArgumentUnused, CoreOption]>,
+  HelpText<"Enable crash diagnostic reporting (default)">, Alias<fcrash_diagnostics_EQ>, AliasArgs<["compiler"]>;
 def fno_crash_diagnostics : Flag<["-"], "fno-crash-diagnostics">, Group<f_clang_Group>, Flags<[NoArgumentUnused, CoreOption]>,
   Alias<gen_reproducer_eq>, AliasArgs<["off"]>,
   HelpText<"Disable auto-generation of preprocessed source files and a script for reproduction during a clang crash">;
 def fcrash_diagnostics_dir : Joined<["-"], "fcrash-diagnostics-dir=">,
   Group<f_clang_Group>, Flags<[NoArgumentUnused, CoreOption]>,
   HelpText<"Put crash-report files in <dir>">, MetaVarName<"<dir>">;
 def fcreate_profile : Flag<["-"], "fcreate-profile">, Group<f_Group>;
 defm cxx_exceptions: BoolFOption<"cxx-exceptions",

clang/lib/Driver/Driver.cpp

 }
 
 // When clang crashes, produce diagnostic information including the fully
 // preprocessed source file(s).  Request that the developer attach the
 // diagnostic information to a bug report.
 void Driver::generateCompilationDiagnostics(
     Compilation &C, const Command &FailingCommand,
     StringRef AdditionalInformation, CompilationDiagnosticReport *Report) {
   if (C.getArgs().hasArg(options::OPT_fno_crash_diagnostics))
     return;
 
-  // Don't try to generate diagnostics for link or dsymutil jobs.
-  if (FailingCommand.getCreator().isLinkJob() ||
-      FailingCommand.getCreator().isDsymutilJob())
+  unsigned Level = 1;
+  if (Arg *A = C.getArgs().getLastArg(options::OPT_fcrash_diagnostics_EQ)) {
+    Level = llvm::StringSwitch<unsigned>(A->getValue())
+                .Case("off", 0)
+                .Case("compiler", 1)
+                .Case("all", 2)
+                .Default(1);
+  }
+  if (!Level)
+    return;
+
+  // Don't try to generate diagnostics for dsymutil jobs.
+  if (FailingCommand.getCreator().isDsymutilJob())
     return;
 
+  bool IsLLD = false;
+  ArgStringList SavedTemps;
+  if (FailingCommand.getCreator().isLinkJob()) {
+    C.getDefaultToolChain().GetLinkerPath(&IsLLD);
+    if (!IsLLD || Level < 2)
+      return;
+
+    // If lld crashed, we will re-run the same command with the input it used
+    // to have. In that case we should not remove temp files in
+    // initCompilationForDiagnostics yet. They will be added back and removed
+    // later.
+    SavedTemps = std::move(C.getTempFiles());
+    assert(!C.getTempFiles().size());
+  }
+
   // Print the version of the compiler.
   PrintVersion(C, llvm::errs());
 
   // Suppress driver output and emit preprocessor output to temp file.
   CCGenDiagnostics = true;
 
   // Save the original job command(s).
   Command Cmd = FailingCommand;
 
   // Keep track of whether we produce any errors while trying to produce
   // preprocessed sources.
   DiagnosticErrorTrap Trap(Diags);
 
   // Suppress tool output.
   C.initCompilationForDiagnostics();
 
   // Construct the list of inputs.
   InputList Inputs;
   BuildInputs(C.getDefaultToolChain(), C.getArgs(), Inputs);
 
   for (InputList::iterator it = Inputs.begin(), ie = Inputs.end(); it != ie;) {
     bool IgnoreInput = false;
 
void Driver::generateCompilationDiagnostics(
 
   // If any of the preprocessing commands failed, clean up and exit.
   if (!FailingCommands.empty()) {
     Diag(clang::diag::note_drv_command_failed_diag_msg)
         << "Error generating preprocessed source(s).";
     return;
   }
 
+  // If lld failed, rerun it again with --reproduce.
+  if (IsLLD) {
+    const char *TmpName = CreateTempFile(C, "linker-crash", "tar");
+    Command NewLLDInvocation = Cmd;
+    llvm::opt::ArgStringList ArgList = NewLLDInvocation.getArguments();
+    ArgList.push_back(Saver.save(Twine{"--reproduce="} + TmpName).data());
+    NewLLDInvocation.replaceArguments(std::move(ArgList));
+
+    // Redirect stdout/stderr to /dev/null.
+    NewLLDInvocation.Execute({None, {""}, {""}}, nullptr, nullptr);
+  }
+
   const ArgStringList &TempFiles = C.getTempFiles();
   if (TempFiles.empty()) {
     Diag(clang::diag::note_drv_command_failed_diag_msg)
         << "Error generating preprocessed source(s).";
     return;
   }
 
   Diag(clang::diag::note_drv_command_failed_diag_msg)
for (const char *TempFile : TempFiles) {
     if (StringRef(TempFile).endswith(".cache")) {
       // In some cases (modules) we'll dump extra data to help with reproducing
       // the crash into a directory next to the output.
       VFS = llvm::sys::path::filename(TempFile);
       llvm::sys::path::append(VFS, "vfs", "vfs.yaml");
     }
   }
 
+  for (const char *TempFile : SavedTemps)
+    C.addTempFile(TempFile);
+
   // Assume associated files are based off of the first temporary file.
   CrashReportInfo CrashInfo(TempFiles[0], VFS);
 
   llvm::SmallString<128> Script(CrashInfo.Filename);
   llvm::sys::path::replace_extension(Script, "sh");
   std::error_code EC;
   llvm::raw_fd_ostream ScriptOS(Script, EC, llvm::sys::fs::CD_CreateNew,
                                 llvm::sys::fs::FA_Write,
static bool HasPreprocessOutput(const Action &JA) {
   if (isa<OffloadAction>(JA) && isa<PreprocessJobAction>(JA.getInputs()[0]))
     return true;
   if (isa<OffloadBundlingJobAction>(JA) &&
       HasPreprocessOutput(*(JA.getInputs()[0])))
     return true;
   return false;
 }
 
+const char *Driver::CreateTempFile(Compilation &C, StringRef Prefix,
+                                   StringRef Suffix, bool MultipleArchs,
+                                   StringRef BoundArch) const {
+  SmallString<128> TmpName;
+  Arg *A = C.getArgs().getLastArg(options::OPT_fcrash_diagnostics_dir);
+  if (CCGenDiagnostics && A) {
+    SmallString<128> CrashDirectory(A->getValue());
+    if (!getVFS().exists(CrashDirectory))
+      llvm::sys::fs::create_directories(CrashDirectory);
+    llvm::sys::path::append(CrashDirectory, Prefix);
+    const char *Middle = !Suffix.empty() ? "-%%%%%%." : "-%%%%%%";
+    std::error_code EC = llvm::sys::fs::createUniqueFile(
+        CrashDirectory + Middle + Suffix, TmpName);
+    if (EC) {
+      Diag(clang::diag::err_unable_to_make_temp) << EC.message();
+      return "";
+    }
+  } else {
+    if (MultipleArchs && !BoundArch.empty()) {
+      TmpName = GetTemporaryDirectory(Prefix);
+      llvm::sys::path::append(TmpName,
+                              Twine(Prefix) + "-" + BoundArch + "." + Suffix);
+    } else {
+      TmpName = GetTemporaryPath(Prefix, Suffix);
+    }
+  }
+  return C.addTempFile(C.getArgs().MakeArgString(TmpName));
+}
+
 const char *Driver::GetNamedOutputPath(Compilation &C, const JobAction &JA,
                                        const char *BaseInput,
                                        StringRef OrigBoundArch, bool AtTopLevel,
                                        bool MultipleArchs,
                                        StringRef OffloadingPrefix) const {
   std::string BoundArch = OrigBoundArch.str();
   if (is_style_windows(llvm::sys::path::Style::native)) {
     // BoundArch may contains ':', which is invalid in file names on Windows,
const char *Driver::GetNamedOutputPath(Compilation &C, const JobAction &JA,
   }
 
   // Output to a temporary file?
   if ((!AtTopLevel && !isSaveTempsEnabled() &&
        !C.getArgs().hasArg(options::OPT__SLASH_Fo)) ||
       CCGenDiagnostics) {
     StringRef Name = llvm::sys::path::filename(BaseInput);
     std::pair<StringRef, StringRef> Split = Name.split('.');
-    SmallString<128> TmpName;
     const char *Suffix = types::getTypeTempSuffix(JA.getType(), IsCLMode());
-    Arg *A = C.getArgs().getLastArg(options::OPT_fcrash_diagnostics_dir);
-    if (CCGenDiagnostics && A) {
-      SmallString<128> CrashDirectory(A->getValue());
-      if (!getVFS().exists(CrashDirectory))
-        llvm::sys::fs::create_directories(CrashDirectory);
-      llvm::sys::path::append(CrashDirectory, Split.first);
-      const char *Middle = Suffix ? "-%%%%%%." : "-%%%%%%";
-      std::error_code EC = llvm::sys::fs::createUniqueFile(
-          CrashDirectory + Middle + Suffix, TmpName);
-      if (EC) {
-        Diag(clang::diag::err_unable_to_make_temp) << EC.message();
-        return "";
-      }
-    } else {
-      if (MultipleArchs && !BoundArch.empty()) {
-        TmpName = GetTemporaryDirectory(Split.first);
-        llvm::sys::path::append(TmpName,
-                                Split.first + "-" + BoundArch + "." + Suffix);
-      } else {
-        TmpName = GetTemporaryPath(Split.first, Suffix);
-      }
-    }
-    return C.addTempFile(C.getArgs().MakeArgString(TmpName));
+    return CreateTempFile(C, Split.first, Suffix, MultipleArchs, BoundArch);
   }
 
   SmallString<128> BasePath(BaseInput);
   SmallString<128> ExternalPath("");
   StringRef BaseName;
 
   // Dsymutil actions should use the full path.
   if (isa<DsymutilJobAction>(JA) && C.getArgs().hasArg(options::OPT_dsym_dir)) {

clang/test/Driver/crash-report.cpp

 // RUN: cat %t/crash-report-*.sh | FileCheck --check-prefix=CHECKSH %s
 
 // RUN: env TMPDIR=%t TEMP=%t TMP=%t RC_DEBUG_OPTIONS=1                  \
 // RUN:  CC_PRINT_HEADERS=1 CC_LOG_DIAGNOSTICS=1                         \
 // RUN:  not %clang %s @%t.rsp -DFATAL 2>&1 | FileCheck %s
 // RUN: cat %t/crash-report-*.cpp | FileCheck --check-prefix=CHECKSRC %s
 // RUN: cat %t/crash-report-*.sh | FileCheck --check-prefix=CHECKSH %s
 
+// Test manually specifying -fcrash-diagnostics[=[compiler|all]] emits
+// diagnostics
+// RUN: env TMPDIR=%t TEMP=%t TMP=%t RC_DEBUG_OPTIONS=1                  \
+// RUN:  CC_PRINT_HEADERS=1 CC_LOG_DIAGNOSTICS=1                         \
+// RUN:  not %clang %s @%t.rsp -DFATAL -fcrash-diagnostics 2>&1 |        \
+// RUN:  FileCheck %s
+// RUN: cat %t/crash-report-*.cpp | FileCheck --check-prefix=CHECKSRC %s
+// RUN: cat %t/crash-report-*.sh | FileCheck --check-prefix=CHECKSH %s
+
+// RUN: env TMPDIR=%t TEMP=%t TMP=%t RC_DEBUG_OPTIONS=1                   \
+// RUN:  CC_PRINT_HEADERS=1 CC_LOG_DIAGNOSTICS=1                          \
+// RUN:  not %clang %s @%t.rsp -DFATAL -fcrash-diagnostics=compiler 2>&1 |\
+// RUN:  FileCheck %s
+// RUN: cat %t/crash-report-*.cpp | FileCheck --check-prefix=CHECKSRC %s
+// RUN: cat %t/crash-report-*.sh | FileCheck --check-prefix=CHECKSH %s
+
+// RUN: env TMPDIR=%t TEMP=%t TMP=%t RC_DEBUG_OPTIONS=1                  \
+// RUN:  CC_PRINT_HEADERS=1 CC_LOG_DIAGNOSTICS=1                         \
+// RUN:  not %clang %s @%t.rsp -DFATAL -fcrash-diagnostics=all 2>&1 |    \
+// RUN:  FileCheck %s
+// RUN: cat %t/crash-report-*.cpp | FileCheck --check-prefix=CHECKSRC %s
+// RUN: cat %t/crash-report-*.sh | FileCheck --check-prefix=CHECKSH %s
+
 // REQUIRES: crash-recovery
 
 #ifdef PARSER
 #pragma clang __debug parser_crash
 #elif CRASH
 #pragma clang __debug crash
 #elif FATAL
 #pragma clang __debug llvm_fatal_error

clang/test/Driver/lit.local.cfg

+from lit.llvm import llvm_config
+
 config.suffixes = ['.c', '.cpp', '.h', '.m', '.mm', '.S', '.s', '.f90', '.F90', '.f95',
                    '.cu', '.rs', '.cl', '.clcpp', '.hip', '.hlsl']
 config.substitutions = list(config.substitutions)
 config.substitutions.insert(0,
     ('%clang_cc1',
      """*** Do not use 'clang -cc1' in Driver tests. ***""") )
 
 # Remove harmful environmental variables for clang Driver tests.
 # Some might be useful for other tests so they are only removed here.
 driver_overwrite_env_vars = ['MACOSX_DEPLOYMENT_TARGET',
                              'IPHONEOS_DEPLOYMENT_TARGET',
                              'SDKROOT', 'CCC_OVERRIDE_OPTIONS',
                              'CC_PRINT_OPTIONS', 'CC_PRINT_HEADERS',
                              'CC_LOG_DIAGNOSTICS', 'CC_PRINT_PROC_STAT']
 
 for name in driver_overwrite_env_vars:
   if name in config.environment:
     del config.environment[name]
+
+if llvm_config.use_lld(required=False):
+    config.available_features.add('lld')

clang/test/Driver/lld-repro.c

+// REQUIRES: lld
+
+// RUN: not %clang %s -fuse-ld=lld -gen-reproducer=error -fcrash-diagnostics-dir=%t -fcrash-diagnostics=all 2>&1 \
+// RUN:   | FileCheck %s
+
+// check that we still get lld's output
+// CHECK: error: undefined symbol: a
+
+// CHECK: Preprocessed source(s) and associated run script(s) are located at:
+// CHECK-NEXT: note: diagnostic msg: {{.*}}/lld-repro-{{.*}}.c
+// CHECK-NEXT: note: diagnostic msg: {{.*}}/linker-crash-{{.*}}.tar
+// CHECK-NEXT: note: diagnostic msg: {{.*}}/lld-repro-{{.*}}.sh
+// CHECK-NEXT: note: diagnostic msg:
+// CHECK: ********************
+
+// RUN: not %clang %s -fuse-ld=lld -gen-reproducer=error -fcrash-diagnostics-dir=%t -fcrash-diagnostics=compiler 2>&1 \
+// RUN:   | FileCheck %s --check-prefix=NO-LINKER
+// RUN: not %clang %s -fuse-ld=lld -gen-reproducer=error -fcrash-diagnostics-dir=%t 2>&1 \
+// RUN:   | FileCheck %s --check-prefix=NO-LINKER
+
+// NO-LINKER-NOT: Preprocessed source(s) and associated run script(s) are located at:
+
+extern int a;
+int main() {
+  return a;
+}