Add new hidden option -print-on-crash that prints out IR that caused opt pipeline to crash

[aeubanks]

Summary

A new hidden option -print-on-crash that prints the IR as it was upon entering
the last pass when there is a crash.

The IR is saved in its print form before each pass is started and a
signal handler is registered. If the compilation crashes, the signal
handler will print the saved IR to dbgs(). This option
can be modified using -print-module-scope to get the IR for the complete
module. Note that this option only works with the new pass manager.

[jamieschmeiser]

This option will be introduced (along with others) in my tutorial at the LLVM developer's conference in October.

[aeubanks]

I'm worried that this will affect memory usage.
Is --print-before-all not good enough?

[jamieschmeiser]

--print-before-all will print the IR before every pass which can result in the IR being printed literally thousands of times while this option prints it only once. The IR is saved in a string which is cleared each time and re-used so the memory overhead is not great; also this overhead is only hit when the option is requested. It does slow the compilation when used but not as much as --print-before-all and the user does not have as difficult a task in extracting the IR from the output. If the option is false (the default), the overhead is only the checking of the option. Basically, --print-before-all is cumbersome for this purpose while this option is convenient. Also, one could reasonably request a user to retrieve the IR (if they are willing) with this option when reporting a crash while requesting a user to use --print-before-all would not be reasonable.

[jamieschmeiser]

[NFC] Reorder includes and change case of variable as suggested by
clang-tidy/clang-format.

[yrouban]

llvm/lib/Passes/StandardInstrumentations.cpp

346–347

static_cast<PrintCrashIRInstrumentation *>(P)

1035–1040

do we need to remove this signal handler when this is destructed?

[aeubanks]

Ok I guess this makes sense over --print-before-all.

To bikeshed a bit, maybe the option should be named --print-on-crash? For example --print-before-all does not contain "IR" in the name.

It would be nice to include a test. Maybe creating a new pass that crashes upon running could be useful.

Looks like the latest diff on Phab is against the first revision, could that be fixed?

[jamieschmeiser]

For a test, I could create a pass that calls __builtin_trap that is only added to the opt pipeline by a really hidden option. The danger is that if someone uses that option, the compiler would crash, which seems dangerous just for testing. Is that desirable?

[jamieschmeiser]

llvm/lib/Passes/StandardInstrumentations.cpp

1035–1040

I don't think it is necessary because typically, control does not return from the signal handler and destructors are not run.

[yrouban]

llvm/lib/Passes/StandardInstrumentations.cpp

1035–1040

1. The life time of the PrintCrashIRInstrumentation instance is shorter than the scope of signal handlers. So there is a chance when PrintCrashIRInstrumentation is destructed then LLVM is crashed. This results in executing the handler with the deleted instance of PrintCrashIRInstrumentation.
2. AddSignalHandler() has a limited number of handlers. You should not add a new handler for every new instance of PrintCrashIRInstrumentation. I would suggest that only one crash reporting handler is registered and this reporter maintains a global set of live instances of PrintCrashIRInstrumentation.

[jamieschmeiser]

Address review comments.

Change the name of the option to -print-on-crash.
Change the registered signal handler to be a global function that only
references other globals to avoid life-span problems.
Only register a single signal handler for multiple handlers that all refer
to the same global object. This may cause corruption of the output
in some cases due to race conditions but this is deemed acceptable
since this is for developer use, rather than seen by a user.
Create a pass for the new pass manager that always triggers a crash but
is not put into any pipeline to allow testing of the option.
Finally, a test for the option is added.

[yrouban]

llvm/lib/Passes/StandardInstrumentations.cpp

351–354

I would suggest that :

1. SavedIR is an instance field of PrintCrashIRInstrumentation.
2. There is a global set of live PrintCrashIRInstrumentation. Every instance is added in its constructor and deleted in its destructor. The add and delete operations are synchronized.
3. For all live PrintCrashIRInstrumentation instances the crashedIRReporter prints whatever it is supposed to print.

932

this must be done for every call of registerCallbacks disregard to HandlerRegistered.

[aeubanks]

llvm/test/Other/print-on-crash.ll

6

I think the typical way is not --crash opt ....
Not sure if this may REQUIRES: asserts or not

15

maybe check that some or the IR is printed? at least a function name

[jamieschmeiser]

Respond to review comments: use mutex to protect building of vector of crash printers.
Create a global vector of crash printers that the signal handler uses to report the IRs in the event of a crash.
Adding a crash printer and removing it from the vector is protected using mutex locking.

Respond to review comments: change test to use not --crash and make it check for part of IR.

[yrouban]

llvm/lib/Passes/StandardInstrumentations.cpp

348–350

please make these both static private fields of PrintIRInstrumentation.
rename mtx with first uppercase.

942

This way CrashReporters vector is growing unlimited - memory leak.
Please use a pointer set and no registration index will be needed.
E.g. DenseSet<PrintCrashIRInstrumentation *> CrashReporters;

[aeubanks]

llvm/lib/Passes/StandardInstrumentations.cpp

348–350

https://llvm.org/docs/CodingStandards.html#do-not-use-static-constructors

[yrouban]

llvm/lib/Passes/StandardInstrumentations.cpp

348–350

Please elaborate. Do you mean you disagree agree with my suggestion?
I just proposed to hide the mtx and CrashReporters making them static fields of PrintCrashIRInstrumentation:

StandardInstrumentations.h:

class PrintCrashIRInstrumentation {
  ...
  // Avoid races when creating/destroying the crash IR printers.
  static std::mutex mtx;
  // All of the crash reporters that will report on a crash.
  static std::vector<PrintCrashIRInstrumentation *> CrashReporters;
}

StandardInstrumentations.cpp:

std::mutex PrintCrashIRInstrumentation::mtx;
std::vector<PrintCrashIRInstrumentation *> PrintCrashIRInstrumentation::CrashReporters;

This is essentially the same as

std::mutex mtx;
std::vector<PrintCrashIRInstrumentation *> CrashReporters;

[aeubanks]

llvm/lib/Passes/StandardInstrumentations.cpp

348–350

These are global variables with constructors right? LLVM coding guide doesn't allow global constructors. I think that also applies to static members of classes.

And I think adding a signal handler per StandardInstrumentation is fine. Apparently LLVM only allows 8 max signal handlers (Signals.cpp), but we probably won't hit that, especially since this is a hidden option and likely we'll only have one StandardInstrumentation in most cases. We can always revisit later if necessary.

[yrouban]

llvm/lib/Passes/StandardInstrumentations.cpp

348–350

global variables ...
literally yes, but we have plenty of those of type cl::opt.
Another option I see is to create a singleton at the very first use. But it would complicate things.

Adding a signal handler for every instance of StandardInstrumentation is unacceptable for compilers that compile many units in one process. A new instance is created for every call of llvm::runPassPipeline().

[jamieschmeiser]

Respond to review comments: Change global vector to pointer to DenseSet,
change objects from unnamed namespace to private statics.

Also, add assertions and make locking thread-safe.

[aeubanks]

llvm/lib/Passes/StandardInstrumentations.cpp

348–350

It seems you have some experience in globals based on some older llvm-dev threads you were involved in, seems like you have more background on this than me so I won't block on the mutex global (although of course a non-global way would be better if possible and relatively clean).

Ok multiple runs creating multiple StandardInstrumentations makes sense.

[aeubanks]

some nits, but lgtm as long as yrouban approves

llvm/include/llvm/Passes/StandardInstrumentations.h

23

is this include necessary?

llvm/lib/Passes/PassBuilder.cpp

268–278

could you move this right after the noop passes below?

[jamieschmeiser]

Respond to review comments: replace unneeded include with standard include, move code in file.
React to review comments on other PRs: change test prefixes to something more useful.
React to test failures with other PRs: change tests to have regex rather than having PassID in test
Add another test to test interaction with -print-module-scope

[aeubanks]

overall lgtm with some nits, but I'd like yrouban to take a look at the signal handler

llvm/test/Other/print-on-crash.ll

24

nitpicking, but reducing this to
define i32 @main() {
would be ideal

[aeubanks]

llvm/lib/Passes/StandardInstrumentations.cpp

57–58

the last two sentences seem unnecessary. also hidden option doesn't need to be specified here and in the commit description.

So just "Print the IR being processed when a pass crashes" is good enough IMO

1077

why is this inline?

[jamieschmeiser]

Made changes requested by @aeubanks, who is satisfied but would like @yrouban to check the signal handler.

[jamieschmeiser]

@yrouban, @aeubanks indicated that he is satisfied but would like you to sign off on the signal handler.

[aeubanks]

If @yrouban doesn't respond I'd say it's fine to go ahead and submit this, just make sure that the title and summary are properly updated.

[yrouban]

llvm/include/llvm/Passes/StandardInstrumentations.h

156

I would suggest that we use just

static DenseSet<PrintCrashIRInstrumentation *> CrashReporters

It would simplify implementation but take some more bytes for empty DenseSet.

[aeubanks]

the title should probably contain the new --print-on-crash flag name

[aeubanks]

Also, adding a file to dump the IR to would be awesome for debugging crashes, maybe in a future change

[jamieschmeiser]

I think that is a great idea, especially since it is likely it is being captured anyway... Perhaps with -print-on-crash=<filename> with the current behaviour when =<filename> is not specified? As a subsequent change rather than in this one.

[aeubanks]

that sounds good to me

[aeubanks]

Is this ready to land?

Also, on further thought, do we really need all the mutex code? This is supposed to be for debugging anyway, not really used in production code like JITs.

[jamieschmeiser]

Yes, it is ready to land. There was a fair bit of discussion during the review so I left it after it was accepted in case there were still concerns. The mutex code was added because it was requested in review. IIRC, there were concerns about multiple pass managers registering signal handlers. Since it is for debugging, efficiency of the mutex code is not as important.

[jamieschmeiser]

Rebased and updated call to registerBeforeNonSkippedPassCallback. Also removed extra blank line at end of test.

[jamieschmeiser]

The Buildbot has detected a failed build on builder mlir-windows while building llvm.

Full details are available at:

https://urldefense.proofpoint.com/v2/url?u=https-3A__lab.llvm.org_buildbot-23builders_13_builds_6076&d=DwICaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Nr2SnYQn80wzIpml4veK2C-U4ilVZzi0kWYCc2WSLoA&m=CQQdxPSThSw2OucwuObeggJaCzdQ6vt0CFRALzhLKoY&s=IE0fvkJX313_gvazCZdO2fla5o5wb1oOEemSpIR3ZMM&e=

[jamieschmeiser]

The build problem is that __builtin_trap is not recognized in PassBuilder.cpp.

[aeubanks]

maybe an assert(false) is more portable?

[anna]

In D86657#2684482, @aeubanks wrote:

maybe an assert(false) is more portable?

@jamieschmeiser does that help? Wondering if that will allow to land this. This is a good option to have.

[jamieschmeiser]

@anna Sorry, this kind of dropped off my radar. I will check into it and try to land this.

[jamieschmeiser]

Update and change test pass to use assert(false)

[jamieschmeiser]

fix Clang-tidy problems

[aeubanks]

back to the mutex discussion, I feel like it's a very narrow use case to use --print-on-crash with multiple threads
I'd rather land this without the mutex stuff and add the mutex stuff later if we really decide we need it

llvm/lib/Passes/PassBuilder.cpp

445

this should probably be a module pass

450

I'd move the definition into the class definition

llvm/test/Other/print-on-crash.ll

2

we should probably add REQUIRES: asserts to make sure the assert in the pass triggers

4

not true anymore

15

unnecessary comment

[jamieschmeiser]

I agree that the mutex code is overkill, considering that this is intended for debugging. If things go off the rails because of the lacking mutex code, the consequences are not severe since it is expected to crash anyway (or else why would you use this?)
I will removed the mutex code and clean things up based on your comments unless someone raises an objection.

[jamieschmeiser]

As requested, I removed the mutex code and cleaned up a few things.

[aeubanks]

mostly looks good
should we always print the full module? generally if we're looking at some crashed IR, we want to be able to run opt on it, and just printing out a function won't allow us to do that

llvm/include/llvm/Passes/StandardInstrumentations.h

409

we don't need a set anymore right? just one crash reporter

llvm/lib/Passes/PassBuilder.cpp

445

extraneous struct?

[jamieschmeiser]

Regarding always printing the module, I'm not sure that is desirable. The
problematic IR may be very large and the option respects -print-module-scope
so the module IR can be obtained. There are also tools for creating a fleshed-out IR
from the IR for a function.

[aeubanks]

In D86657#3033378, @jamieschmeiser wrote:

There are also tools for creating a fleshed-out IR from the IR for a function.

Can you elaborate?

For now this still lgtm, we can iterate on it later

[jamieschmeiser]

I think I am getting confused with llvm-extract --func=<function> which will extract what is needed for a function from a module IR. However, one could likely use that to get an IR with the necessary addition declarations and then substitute in the IR for the function using this option. I wonder if it might be possible to use the code in llvm-extract to do that automatically? Something to think about...

[aeubanks]

In D86657#3034561, @jamieschmeiser wrote:

I think I am getting confused with llvm-extract --func=<function> which will extract what is needed for a function from a module IR. However, one could likely use that to get an IR with the necessary addition declarations and then substitute in the IR for the function using this option. I wonder if it might be possible to use the code in llvm-extract to do that automatically? Something to think about...

llvm-extract is a bit finnicky, it basically relies on GlobalDCE and has to change function linkage in order to preserve functions.

[aeubanks]

The title needs updating for the option name

Btw I just tried this on a very large file with a (real) LLVM crash and it was very slow and I gave up. It's probably from printing the IR after every pass. Perhaps cloning the module via CloneModule() and keeping that around might be faster?

[DaniilSuchkov]

In D86657#3037670, @aeubanks wrote:

The title needs updating for the option name

Btw I just tried this on a very large file with a (real) LLVM crash and it was very slow and I gave up. It's probably from printing the IR after every pass. Perhaps cloning the module via CloneModule() and keeping that around might be faster?

Keep in mind that:
a) This is still (supposedly) a quality of life improvement over using -print-after-all -print-module-scope. In my experience -print-after-all -print-module-scope in some cases can take hours (and then I have to deal with a few GB long log).
b) There are bugs that can corrupt IR in other modules (like the one I fixed here: D110752), so keeping a live Module object isn't the best idea.

Serializing the module into the binary representation and then de-serizalining and printing it upon crash is an option, though I'm not sure it's robust enough, since the LLVMContext can be arbitrarily broken at that point.

[aeubanks]

In D86657#3037681, @DaniilSuchkov wrote:

In D86657#3037670, @aeubanks wrote:

The title needs updating for the option name

Btw I just tried this on a very large file with a (real) LLVM crash and it was very slow and I gave up. It's probably from printing the IR after every pass. Perhaps cloning the module via CloneModule() and keeping that around might be faster?

Keep in mind that:
a) This is still (supposedly) a quality of life improvement over using -print-after-all -print-module-scope. In my experience -print-after-all -print-module-scope in some cases can take hours (and then I have to deal with a few GB long log).

The current implementation isn't that much better than just -print-after-all -print-module-scope. It's easy to look at a huge log, search for the last instance of "IR Dump", and delete everything from that to the beginning of the file. I think making this quick is very important for it to be much more useful than -print-after-all -print-module-scope.
For example, I had a repro that took 2 seconds to crash. I tried specifying (with this patched in) --print-on-crash, and gave up after several minutes. If could keep the slowdown of turning this on within an order of magnitude of the original repro that'd be super useful.

b) There are bugs that can corrupt IR in other modules (like the one I fixed here: D110752), so keeping a live Module object isn't the best idea.

D110752 is not corrupting IR, it happened because we were looking at a constant's users, which is not a thing that makes sense to do (e.g. look at all users of the constant false).

Serializing the module into the binary representation and then de-serizalining and printing it upon crash is an option, though I'm not sure it's robust enough, since the LLVMContext can be arbitrarily broken at that point.

I guess it's possible that LLVMContext is broken, but that's much rarer than simply seeing a random crash in a pass. I don't think it's worth handling the small chance that something fundamental is broken to slow down the common use case.

But again, still fine to land this and improve on it later if you'd like.

[DaniilSuchkov]

D110752 is not corrupting IR, it happened because we were looking at a constant's users, which is not a thing that makes sense to do (e.g. look at all users of the constant false).

Yep, what it did doesn't make any sense because it was a bug. Since here we're discussing a debugging option, in my opinion it makes sense to take such cases into account.

[jamieschmeiser]

Added the usual filtering options (filter-passes and filter-print-func) with uninteresting passes being saved with a message that they were filtered out. This should speed up things significantly when they are used. If the user isn't aware of the name of the crashing pass, they can use -filter-passes=blah (where blah doesn't exist) and they will get a message saying that the pass was filtered out. They can discover the name from that message and then rerun it with -filter-passes.

[aeubanks]

llvm/lib/Passes/StandardInstrumentations.cpp

18

is this necessary?

[jamieschmeiser]

Remove unnecessary include.

[aeubanks]

still lgtm

[MaskRay]

llvm/lib/Passes/PassBuilder.cpp

445

error: non-void function does not return a value [-Werror,-Wreturn-type]

use llvm_unreachable

[jamieschmeiser]

Failed the lit test.
https://lab.llvm.org/buildbot/#/builders/110/builds/6977
https://lab.llvm.org/buildbot/#/builders/36/builds/13137
https://lab.llvm.org/buildbot/#/builders/52/builds/11218

[jamieschmeiser]

missed @MaskRay 's comment, will add that before attempting a relanding

[fhahn]

I saw the patch got reverted a while ago. Are there any plans for re-submitting the change?

This would be very valuable to have in combination with an option to print the whole module before the crashing pass to a a file or the function causing the crash extracted from the whole module, so there's a reproducer file that can be used easily.

[aeubanks]

I'll reland this using the crash passes added in D120993 once that lands

[jamieschmeiser]

Thanks!

llvm/include/llvm/Passes/StandardInstrumentations.h

 /// This header defines a class that provides bookkeeping for all standard
 /// (i.e in-tree) pass instrumentations.
 ///
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_PASSES_STANDARDINSTRUMENTATIONS_H
 #define LLVM_PASSES_STANDARDINSTRUMENTATIONS_H
 
+#include "llvm/ADT/DenseSet.h"
 #include "llvm/ADT/STLExtras.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/IR/BasicBlock.h"
 #include "llvm/IR/OptBisect.h"
 #include "llvm/IR/PassTimingInfo.h"
 #include "llvm/IR/ValueHandle.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Transforms/IPO/SampleProfileProbe.h"
 
+#include <mutex>
 #include <string>
 #include <utility>
 
 namespace llvm {
 
 class Module;
 class Function;
 class PassInstrumentationCallbacks;

 // 1.  The initial IR processed.
 // 2.  To get the representation of the IR (of type \p T).
 // 3.  When a pass does not change the IR.
 // 4.  When a pass changes the IR (given both before and after representations
 //         of type \p T).
 // 5.  When an IR is invalidated.
 // 6.  When a pass is run on an IR that is not interesting (based on options).
 // 7.  When a pass is ignored (pass manager or adapter pass).
 // 8.  To compare two IR representations (of type \p T).
 template <typename IRUnitT> class ChangeReporter {
 protected:
   ChangeReporter(bool RunInVerboseMode) : VerboseMode(RunInVerboseMode) {}
 
 public:
   virtual ~ChangeReporter();
 
   // Determine if this pass/IR is interesting and if so, save the IR

 class VerifyInstrumentation {
   bool DebugLogging;
 
 public:
   VerifyInstrumentation(bool DebugLogging) : DebugLogging(DebugLogging) {}
   void registerCallbacks(PassInstrumentationCallbacks &PIC);
 };
 
+// Print IR on crash.
+class PrintCrashIRInstrumentation {
+public:
+  PrintCrashIRInstrumentation()
+      : SavedIR("*** Dump of IR Before Last Pass Unknown ***") {}
+  ~PrintCrashIRInstrumentation();
+  void registerCallbacks(PassInstrumentationCallbacks &PIC);
+  void reportCrashIR();
+
+protected:
+  std::string SavedIR;
+
+private:
+  // All of the crash reporters that will report on a crash.
+  static DenseSet<PrintCrashIRInstrumentation *> *CrashReporters;
+  // Crash handler registered when print-on-crash is specified.
+  static void SignalHandler(void *);
+  // Exception-safe locking
+  class MtxLock {
+  public:
+    MtxLock() { Mtx.lock(); }
+    ~MtxLock() { Mtx.unlock(); }
+
+  protected:
+    // Avoid races when creating/destroying the crash IR printers.
+    static std::mutex Mtx;
+  };
+};
+
 /// This class provides an interface to register all the standard pass
 /// instrumentations and manages their state (if any).
 class StandardInstrumentations {
   PrintIRInstrumentation PrintIR;
   PrintPassInstrumentation PrintPass;
   TimePassesHandler TimePasses;
   OptNoneInstrumentation OptNone;
   OptBisectInstrumentation OptBisect;
   PreservedCFGCheckerInstrumentation PreservedCFGChecker;
   IRChangedPrinter PrintChangedIR;
   PseudoProbeVerifier PseudoProbeVerification;
   InLineChangePrinter PrintChangedDiff;
+  PrintCrashIRInstrumentation PrintCrashIR;
   VerifyInstrumentation Verify;
 
   bool VerifyEach;
 
 public:
   StandardInstrumentations(bool DebugLogging, bool VerifyEach = false);
 
   void registerCallbacks(PassInstrumentationCallbacks &PIC);
 
   TimePassesHandler &getTimePasses() { return TimePasses; }
 };
 
 extern template class ChangeReporter<std::string>;
 extern template class TextChangeReporter<std::string>;
 
 extern template class ChangeReporter<ChangedIRData>;
 extern template class TextChangeReporter<ChangedIRData>;
-
 } // namespace llvm
 
 #endif

llvm/lib/Passes/PassBuilder.cpp

static const Regex DefaultAliasRegex(
     "^(default|thinlto-pre-link|thinlto|lto-pre-link|lto)<(O[0123sz])>$");
 
 /// Flag to enable inline deferral during PGO.
 static cl::opt<bool>
     EnablePGOInlineDeferral("enable-npm-pgo-inline-deferral", cl::init(true),
                             cl::Hidden,
                             cl::desc("Enable inline deferral during PGO"));
 
 static cl::opt<bool> EnableMemProfiler("enable-mem-prof", cl::init(false),
                                        cl::Hidden, cl::ZeroOrMore,
                                        cl::desc("Enable memory profiler"));
 
 static cl::opt<bool> PerformMandatoryInliningsFirst(
     "mandatory-inlining-first", cl::init(true), cl::Hidden, cl::ZeroOrMore,
     cl::desc("Perform mandatory inlinings module-wide, before performing "
              "inlining."));
 
 static cl::opt<bool> EnableO3NonTrivialUnswitching(
     "enable-npm-O3-nontrivial-unswitch", cl::init(true), cl::Hidden,
     cl::ZeroOrMore, cl::desc("Enable non-trivial loop unswitching for -O3"));
 
 PipelineTuningOptions::PipelineTuningOptions() {
   LoopInterleaving = true;
   LoopVectorization = true;
   SLPVectorization = false;
   LoopUnrolling = true;
   ForgetAllSCEVInLoopUnroll = ForgetSCEVInLoopUnroll;

 ///
 /// This is for optimization purposes so we don't populate it if we never use
 /// it. This should be updated if new pass instrumentation wants to use the map.
 /// We currently only use this for --print-before/after.
 bool shouldPopulateClassToPassNames() {
   return !printBeforePasses().empty() || !printAfterPasses().empty();
 }
 
+// A pass for testing -print-on-crash.
+// DO NOT USE THIS EXCEPT FOR TESTING!
+class TriggerCrashPass : public PassInfoMixin<TriggerCrashPass> {
+public:
+  PreservedAnalyses run(Function &F, FunctionAnalysisManager &FAM);
+};
+
+// DO NOT USE THIS EXCEPT FOR TESTING!
+PreservedAnalyses TriggerCrashPass::run(Function &, FunctionAnalysisManager &) {
+  __builtin_trap();
+}
+
 } // namespace
 
 PassBuilder::PassBuilder(bool DebugLogging, TargetMachine *TM,
                          PipelineTuningOptions PTO, Optional<PGOOptions> PGOOpt,
                          PassInstrumentationCallbacks *PIC)
     : DebugLogging(DebugLogging), TM(TM), PTO(PTO), PGOOpt(PGOOpt), PIC(PIC) {
   if (TM)
     TM->registerPassBuilderCallbacks(*this, DebugLogging);

llvm/lib/Passes/PassRegistry.def

 FUNCTION_PASS("slp-vectorizer", SLPVectorizerPass())
 FUNCTION_PASS("slsr", StraightLineStrengthReducePass())
 FUNCTION_PASS("speculative-execution", SpeculativeExecutionPass())
 FUNCTION_PASS("spec-phis", SpeculateAroundPHIsPass())
 FUNCTION_PASS("sroa", SROA())
 FUNCTION_PASS("strip-gc-relocates", StripGCRelocates())
 FUNCTION_PASS("structurizecfg", StructurizeCFGPass())
 FUNCTION_PASS("tailcallelim", TailCallElimPass())
+FUNCTION_PASS("trigger-crash", TriggerCrashPass())
 FUNCTION_PASS("unify-loop-exits", UnifyLoopExitsPass())
 FUNCTION_PASS("vector-combine", VectorCombinePass())
 FUNCTION_PASS("verify", VerifierPass())
 FUNCTION_PASS("verify<domtree>", DominatorTreeVerifierPass())
 FUNCTION_PASS("verify<loops>", LoopVerifierPass())
 FUNCTION_PASS("verify<memoryssa>", MemorySSAVerifierPass())
 FUNCTION_PASS("verify<regions>", RegionInfoVerifierPass())
 FUNCTION_PASS("verify<safepoint-ir>", SafepointIRVerifierPass())

llvm/lib/Passes/StandardInstrumentations.cpp

 /// This file defines IR-printing pass instrumentation callbacks as well as
 /// StandardInstrumentations class that manages standard pass instrumentations.
 ///
 //===----------------------------------------------------------------------===//
 
 #include "llvm/Passes/StandardInstrumentations.h"
 #include "llvm/ADT/Any.h"
 #include "llvm/ADT/Optional.h"
+#include "llvm/ADT/SmallString.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/Analysis/CallGraphSCCPass.h"
 #include "llvm/Analysis/LazyCallGraph.h"
 #include "llvm/Analysis/LoopInfo.h"
 #include "llvm/IR/Function.h"
 #include "llvm/IR/Module.h"
 #include "llvm/IR/PassInstrumentation.h"
 #include "llvm/IR/PrintPasses.h"
 #include "llvm/IR/Verifier.h"
 #include "llvm/Support/CommandLine.h"
+#include "llvm/Support/CrashRecoveryContext.h"
 #include "llvm/Support/Debug.h"
 #include "llvm/Support/FormatVariadic.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/Program.h"
+#include "llvm/Support/Signals.h"
 #include "llvm/Support/raw_ostream.h"
 #include <unordered_set>
 #include <vector>
 
 using namespace llvm;
 
 cl::opt<bool> PreservedCFGCheckerInstrumentation::VerifyPreservedCFG(
     "verify-cfg-preserved", cl::Hidden,
 #ifdef NDEBUG
     cl::init(false));
 #else
     cl::init(false));
 #endif
 
 // FIXME: Change `-debug-pass-manager` from boolean to enum type. Similar to
 // `-debug-pass` in legacy PM.
 static cl::opt<bool>
     DebugPMVerbose("debug-pass-manager-verbose", cl::Hidden, cl::init(false),
                    cl::desc("Print all pass management debugging information. "
                             "`-debug-pass-manager` must also be specified"));
 
 // An option that prints out the IR after passes, similar to
 // -print-after-all except that it only prints the IR after passes that
 // change the IR.  Those passes that do not make changes to the IR are
 // reported as not making any changes.  In addition, the initial IR is
 // also reported.  Other hidden options affect the output from this
 // option.  -filter-passes will limit the output to the named passes
 // that actually change the IR and other passes are reported as filtered out.
 // The specified passes will either be reported as making no changes (with
 // no IR reported) or the changed IR will be reported.  Also, the
 // -filter-print-funcs and -print-module-scope options will do similar
 // filtering based on function name, reporting changed IRs as functions(or
PrintChangedBefore("print-before-changed",
                        cl::desc("Print before passes that change them"),
                        cl::init(false), cl::Hidden);
 
 // An option for specifying the diff used by print-changed=[diff | diff-quiet]
 static cl::opt<std::string>
     DiffBinary("print-changed-diff-path", cl::Hidden, cl::init("diff"),
                cl::desc("system diff used by change reporters"));
 
+// An option to print the IR that was being processed when a pass crashes.
+static cl::opt<bool>
+    PrintCrashIR("print-on-crash",
+                 cl::desc("Print the last form of the IR before crash"),
+                 cl::init(false), cl::Hidden);
+
 namespace {
 
 // Perform a system based diff between \p Before and \p After, using
 // \p OldLineFormat, \p NewLineFormat, and \p UnchangedLineFormat
 // to control the formatting of the output.  Return an error message
 // for any failures instead of the diff.
 std::string doSystemDiff(StringRef Before, StringRef After,
                          StringRef OldLineFormat, StringRef NewLineFormat,
void unwrapAndPrint(raw_ostream &OS, Any IR, StringRef Banner,
   if (any_isa<const Loop *>(IR)) {
     const Loop *L = any_cast<const Loop *>(IR);
     assert(L && "Loop should be valid for printing");
     printIR(OS, L, Banner, Brief);
     return;
   }
   llvm_unreachable("Unknown wrapped IR type");
 }
 
 // Return true when this is a pass for which changes should be ignored
 bool isIgnored(StringRef PassID) {
   return isSpecialPass(PassID,
                        {"PassManager", "PassAdaptor", "AnalysisManagerProxy"});
 }
 
 } // namespace
 
 template <typename IRUnitT>
 ChangeReporter<IRUnitT>::~ChangeReporter<IRUnitT>() {
   assert(BeforeStack.empty() && "Problem with Change Printer stack.");
 }
 
 template <typename IRUnitT>
 bool ChangeReporter<IRUnitT>::isInterestingFunction(const Function &F) {
   return isFunctionInPrintList(F.getName());
if (BBGuards)
       BBGuards->try_emplace(intptr_t(&BB), &BB);
     for (auto *Succ : successors(&BB)) {
       Graph[&BB][Succ]++;
       if (BBGuards)
         BBGuards->try_emplace(intptr_t(Succ), Succ);
     }
   }
 }
 
 static void printBBName(raw_ostream &out, const BasicBlock *BB) {
   if (BB->hasName()) {
     out << BB->getName() << "<" << BB << ">";
     return;
   }
 
   if (!BB->getParent()) {
     out << "unnamed_removed<" << BB << ">";
     return;
   }
 
   if (BB == &BB->getParent()->getEntryBlock()) {
     out << "entry"
         << "<" << BB << ">";
     return;
   }
 
   unsigned FuncOrderBlockNum = 0;
void PreservedCFGCheckerInstrumentation::CFG::printDiff(raw_ostream &out,
   }
 }
 
 void PreservedCFGCheckerInstrumentation::registerCallbacks(
     PassInstrumentationCallbacks &PIC) {
   if (!VerifyPreservedCFG)
     return;
 
   PIC.registerBeforeNonSkippedPassCallback([this](StringRef P, Any IR) {
     if (any_isa<const Function *>(IR))
       GraphStackBefore.emplace_back(P, CFG(any_cast<const Function *>(IR)));
     else
       GraphStackBefore.emplace_back(P, None);
   });
 
   PIC.registerAfterPassInvalidatedCallback(
       [this](StringRef P, const PreservedAnalyses &PassPA) {
         auto Before = GraphStackBefore.pop_back_val();
         assert(Before.first == P &&
                "Before and After callbacks must correspond");
         (void)Before;
       });
PIC.registerAfterPassCallback([this](StringRef P, Any IR,
     }
   });
 }
 
 void VerifyInstrumentation::registerCallbacks(
     PassInstrumentationCallbacks &PIC) {
   PIC.registerAfterPassCallback(
       [this](StringRef P, Any IR, const PreservedAnalyses &PassPA) {
         if (isIgnored(P) || P == "VerifierPass")
           return;
         if (any_isa<const Function *>(IR) || any_isa<const Loop *>(IR)) {
           const Function *F;
           if (any_isa<const Loop *>(IR))
             F = any_cast<const Loop *>(IR)->getHeader()->getParent();
           else
             F = any_cast<const Function *>(IR);
           if (DebugLogging)

 
 StandardInstrumentations::StandardInstrumentations(bool DebugLogging,
                                                    bool VerifyEach)
     : PrintPass(DebugLogging), OptNone(DebugLogging),
       PrintChangedIR(PrintChanged == ChangePrinter::PrintChangedVerbose),
       PrintChangedDiff(PrintChanged == ChangePrinter::PrintChangedDiffVerbose),
       Verify(DebugLogging), VerifyEach(VerifyEach) {}
 
+std::mutex PrintCrashIRInstrumentation::MtxLock::Mtx;
+DenseSet<PrintCrashIRInstrumentation *>
+    *PrintCrashIRInstrumentation::CrashReporters = nullptr;
+
+void PrintCrashIRInstrumentation::reportCrashIR() { dbgs() << SavedIR; }
+
+void PrintCrashIRInstrumentation::SignalHandler(void *) {
+  // Called by signal handlers so do not lock here
+  // Are any of PrintCrashIRInstrumentation objects still alive?
+  if (!CrashReporters)
+    return;
+
+  assert(PrintCrashIR && "Did not expect to get here without option set.");
+  for (auto I : *CrashReporters)
+    I->reportCrashIR();
+}
+
+PrintCrashIRInstrumentation::~PrintCrashIRInstrumentation() {
+  if (!PrintCrashIR)
+    return;
+
+  MtxLock Lock;
+  assert(CrashReporters && "Expected CrashReporters to be set");
+
+  // Was this registered?
+  DenseSet<PrintCrashIRInstrumentation *>::iterator I =
+      CrashReporters->find(this);
+  if (I == CrashReporters->end())
+    return;
+  CrashReporters->erase(I);
+  if (!CrashReporters->empty())
+    return;
+  delete CrashReporters;
+  CrashReporters = nullptr;
+}
+
+void PrintCrashIRInstrumentation::registerCallbacks(
+    PassInstrumentationCallbacks &PIC) {
+  if (!PrintCrashIR)
+    return;
+
+  {
+    MtxLock Lock;
+    if (!CrashReporters) {
+      CrashReporters = new DenseSet<PrintCrashIRInstrumentation *>();
+      sys::AddSignalHandler(SignalHandler, nullptr);
+    }
+    CrashReporters->insert(this);
+  }
+  PIC.registerBeforeNonSkippedPassCallback([this](StringRef PassID, Any IR) {
+    assert((MtxLock(), CrashReporters && CrashReporters->find(this) !=
+                                             CrashReporters->end()) &&
+           "Expected CrashReporters to be set and containing this");
+    SavedIR.clear();
+    SmallString<80> Banner =
+        formatv("*** Dump of {0}IR Before Last Pass {1} Started ***",
+                llvm::forcePrintModuleIR() ? "Module " : "", PassID);
+    raw_string_ostream OS(SavedIR);
+    unwrapAndPrint(OS, IR, Banner, llvm::forcePrintModuleIR());
+  });
+}
+
 void StandardInstrumentations::registerCallbacks(
     PassInstrumentationCallbacks &PIC) {
   PrintIR.registerCallbacks(PIC);
   PrintPass.registerCallbacks(PIC);
   TimePasses.registerCallbacks(PIC);
   OptNone.registerCallbacks(PIC);
   OptBisect.registerCallbacks(PIC);
   PreservedCFGChecker.registerCallbacks(PIC);
   PrintChangedIR.registerCallbacks(PIC);
   PseudoProbeVerification.registerCallbacks(PIC);
   if (VerifyEach)
     Verify.registerCallbacks(PIC);
   PrintChangedDiff.registerCallbacks(PIC);
+  PrintCrashIR.registerCallbacks(PIC);
 }
 
 namespace llvm {
 
 template class ChangeReporter<std::string>;
 template class TextChangeReporter<std::string>;
 
 template class ChangeReporter<ChangedIRData>;
 template class TextChangeReporter<ChangedIRData>;
 
 } // namespace llvm

llvm/test/Other/print-on-crash.ll

+; A test that the hidden option -print-on-crash properly sets a signal handler
+; which gets called when a pass crashes.  The trigger-crash pass calls
+; __builtin_trap.
+
+; RUN: not --crash opt -print-on-crash -passes=trigger-crash < %s 2>&1 | FileCheck %s --check-prefix=CHECK_SIMPLE
+
+; A test that the signal handler set by the  hidden option -print-on-crash
+; is not called when no pass crashes.
+
+; RUN: opt -print-on-crash -passes="default<O2>" < %s 2>&1 | FileCheck %s --check-prefix=CHECK_NO_CRASH
+
+; RUN: not --crash opt -print-on-crash -print-module-scope -passes=trigger-crash < %s 2>&1 | FileCheck %s --check-prefix=CHECK_MODULE
+
+; The input corresponds to "int main() { return 0; }" but is irrelevant.
+
+; CHECK_SIMPLE: *** Dump of IR Before Last Pass {{.*}} Started ***
+; CHECK_SIMPLE: @main
+; CHECK_SIMPLE: entry:
+; CHECK_NO_CRASH-NOT: *** Dump of IR
+; CHECK_MODULE: *** Dump of Module IR Before Last Pass {{.*}} Started ***
+; CHECK_MODULE: ; ModuleID = {{.*}}
+
+define i32 @main() {
+entry:
+  %retval = alloca i32, align 4
+  store i32 0, i32* %retval, align 4
+  ret i32 0
+}