[PM][WIP] Enable out-of-tree registration of passes with the new PassBuilder

[philip.pfaffe]

Summary

This is approach tries different angle towards plugin pass registration.

Through the HookManager API, out-of-tree callers may register callbacks with the various stages at which passes are added into pass managers, including parsing of a pass pipeline as well as at extension points within the default -O pipelines.

Registering utilities like require<> and invalidate<> needs to be handled manually by the caller, but I've provided a helper for that.

I've marked this patch WIP because there is one key element missing currently, which is automatically detecting the type of a pipeline given just a pass name. I've yet to come up with a nice solution here. Obvious options are either adding another callback insertion point here, or registering the set of available names (with their IRUnit type); neither of which I don't like to much. I'm happy about suggestions.

[philip.pfaffe]

I've merged the HookManager into the PassBuilder type. To rendezvous the PB with plugins, it is now able to call a well-known function extern "C" void RegisterPluginPasses(PassBuilder &) on the loaded objects. Plugins get to register callbacks there.

A linked-in Polly will not use this mechanism though, which is why I'm invoking it directly in opt's NewPMDriver as a demonstration (other tools will do the same).

[chandlerc]

First round of high-level feedback on this approach, thanks so much for working no this!

include/llvm/Passes/PassBuilder.h

323 ↗

(On Diff #100125)

Generally, I'd like to not add more transitions between public and private. I would prefer the following rough order:

0. Before the first public, any private types / aliases needed to define the public API.

1. Public API
2. Protected API (if it exists)
3. Private API

324 ↗

(On Diff #100125)

Could you sort these based on the order in which they first are used in the default pipeline?

352–355 ↗

(On Diff #100125)

This seems redundant with OptimizerLast. I think in the new PM we should just have OptimizerLast and we should pass the OptimizationLevel as a parameter to *all* of these callbacks.

377–386 ↗

(On Diff #100125)

What do you think about naming these FooT-style? I don't feel strongly about it, but often like it to identify pure type aliases.

380–384 ↗

(On Diff #100125)

I don't think it makes sense to have these be two different kinds of callbacks. At least, not until there are complaints about how annoying it is to ignore the rest.

I think your second signature is pretty good. It lets users that don't care about nested pipelines just use an unnamed parameter and ignore even the struct type, etc.

I would call this callback type something to do with *parsing* though as that is all it does.

386 ↗

(On Diff #100125)

This should be called something to do with *registering*.

407–414 ↗

(On Diff #100125)

I would suggest separating this into a later change. The extension mechanisms should work well out of the box for non-plugin use cases and it lets the review be a bit more focused.

As an example, I'd like to use some of this to implement support for using the new PM in Clang when running sanitizers.

422–425 ↗

(On Diff #100125)

Rather than all four of these plus the huge swath of extension point enums above, how do you feel about a different API:

All of the extension points above have a fixed pass manager type. So what if we have one registerFooCallback for each extension point, completely remove the enum from the public API, and make the pass manager type concrete and precise?

While this would mean more register... functions, not more than we already have enumerators. You should move the comments currently on the enumerators to be on the registration routines. I also generally think it would be best to document the extension points separately from the rest of the registration stuff as its much less boiler-plate-y.

One reason why I prefer this -- with the current API, there are a lot of combinations of pass manager types and enumerations that won't get any compile time error but cannot actually fire in practice.

436–462 ↗

(On Diff #100125)

Why are these public? I would make them implementation details. And in most cases they're called in only one place and could be completely inlined.

489–491 ↗

(On Diff #100125)

I think you should show an example of how to use this here.

494 ↗

(On Diff #100125)

Naming convention suggests fooBarBaz here.

Also, "handle" doesn't really help the reader much IMO. I would name it after the name of the callback above. IE, if above it is something like ParsePassCallbackT I would call this parseAnalysisUtilityPasses.

495 ↗

(On Diff #100125)

There is a type PipelineEntry. I don't think that makes a good parameter name here, especially when the parameter is extracted *from* that type...

[philip.pfaffe]

Addressed review comments.

The two main things are:

• Remove the extension point enum and replace it with individual callbacks
• Get rid of the invoke() functions and inline them everywhere

[mehdi_amini]

include/llvm/Passes/PassBuilder.h

171 ↗

(On Diff #101102)

That's a long block of undocumented code here.

[chandlerc]

Generally this is starting to get pretty close. What are your thoughts on testing? I would suggest a unittest because that should be a good way to simulate an "out of tree" user.

include/llvm/Passes/PassBuilder.h

49 ↗

(On Diff #101102)

Now that this is public, probably need to flesh out its documentation so folks understand the details of how this should be expected to work.

141–142 ↗

(On Diff #101102)

I think this template and all the using aliases of it below don't really help readability enough. These types only show up in one place (the declaration of a function that has a nice comment) and should just be written out in that location IMO.

171 ↗

(On Diff #101102)

Actually, I think all of these are used in essentially two places: a register signature and the vector declaration below. I'm not sure adding an alias just to save that single repeatition is worth it.

I actually find the public functions more readable without the type alias, even though it means a mild amount of repeatition in the container type at the bottom. What do you think Mehdi?

401 ↗

(On Diff #101102)

This one deserves its own dedicated comment rather than being grouped.

402–408 ↗

(On Diff #101102)

I would separate the analysis callbacks from the pipeline parsing callbacks and give them (each) their own documentation.

413–414 ↗

(On Diff #101102)

This one too deserves its own documentation IMO.

lib/Passes/PassBuilder.cpp

275–278 ↗

(On Diff #101102)

Since these are all one-line functions, would it make sense to inline them into the header so we don't have to re-declare all of them?

875 ↗

(On Diff #101102)

SimplifyFPM -> PeepholeFPM ?

899–902 ↗

(On Diff #101102)

We should factor out a helper that adds the peephole passes to an FPM as we do this in a lot of places.

[philip.pfaffe]

Address Review comments

This contains one functional change: I've moved the parseXPassPipeline() functions into the public API, and added the PB instance to the TopLevel parsing callback. The reason is that when I'm being invoked for the TopLevel pipeline, I usually don't actually want to handle all the parsing myself. Instead, I'd only like to set up the PM stack (either by messing with the PipelineElement nest, as PB does, or prepare the stack of PM objects), and then hand this back into the PassBuilder.

Also:

• Inline CallbackT using-decls as well as the function definitions into the callback registration functions in the header.
• Commentary

[philip.pfaffe]

Regarding Testing:
The unit-test included is a bit stale at the moment, because I didn't keep up with the API changes and extensions. Once everyone is happy with the general callback registration API, I'll flesh it out to also test different IRUnits and the things like extension points and top level pipelines. Moreover, currently it only checks if PassBuilder is happy with the pipeline, it doesn't actually verify that the PassManagers are correctly populated.

[philip.pfaffe]

As per Chandler's out-of-band suggestion, remove the parseXPassPipeline() functions from the public API again. Instead, use the existing parseXPipelineCallbacks to determine if a callback accepts a pass name for IRUnit X.

[chandlerc]

Your testing strategy looks good. Comments below. I think this is getting close enough that it makes sense to start hacking on tests for it. Thanks so much for all the work here, I really like how this is turning out!

include/llvm/Passes/PassBuilder.h

369 ↗

(On Diff #101190)

Here and below, can drop the old enumerator before the '-'.

371–385 ↗

(On Diff #101190)

I think we should drop both of these "early" extension points and the "last" extension point.

In the new PM, it is trivial to nest a pass manager, allowing users to easily run passes both before and after the pipeline. This should reduce the number of extension points we have to carry and it removes a semantic complication with the "last" extension point. If we really can't get any of the users of these ported over, we can add them back, but I'd like to be somewhat lazy here and work to address these use cases with nesting and wrapping as long as possible.

466 ↗

(On Diff #101190)

I would expand on this a bit.... You should spell out what 'AA' stands for here, and talk about what the callback is expected to do with the AAManager.

485 ↗

(On Diff #101190)

I think registerAnalysisRegistrationCallback reads slightly better... But not a lot...

504 ↗

(On Diff #101190)

Similarly registerPipelineParsingCallback. Curious if others agree though.

lib/Passes/PassBuilder.cpp

1059 ↗

(On Diff #101190)

Here and below, I would just s/CallbacksArrayT/CallbacksT/g. I don't think Array is really helping... You really mean more Range, but I think plural is enough to signify "hey I can loop over it".

1059–1060 ↗

(On Diff #101190)

Probably worth a comment explaining what craziness we're doing here and why. =]

1267–1275 ↗

(On Diff #101190)

This restructuring seems bad to me...

I think we always want to run the callbacks *last*. The way the parsing logic works, there are two "last" places (one when we have an inner pipeline, one without).

If you're bothered by duplicating the for loop, you could put this in a tiny lambda you declare at the top, and just call it on each path?

1647 ↗

(On Diff #101190)

If we're going to brace one arm, let's brace them all here.

[philip.pfaffe]

Address (most) review comments.

• Mostly comments and rename a few things.
• Move parsing callback invocations to after the builtin passes

Running callbacks last during parsing last means that callers are now unable to "steal" pass names.

[philip.pfaffe]

include/llvm/Passes/PassBuilder.h

371–385 ↗

(On Diff #101190)

I'm not entirely convinced about this. I expect most users of the callbacks won't have access to the outermost PassManagers, so nesting will not be easily possible.

[philip.pfaffe]

Flesh out the tests. Mainly:

• Expand the unittest to handle all IRUnits as well as parsing of a top-level pipeline
• Add a lit test for the extension points.

To enable the latter, I added a few command line options to opt which accept textual pipelines (for appropriate pass managers), which then get inserted into extensions points. This requires 3 additional APIs on the PassBuilder for parsing a pipeline text into something other than a ModulePassManager.

[chandlerc]

Whew, back to this.

Mostly nit picks and comments about testing.

include/llvm/Passes/PassBuilder.h

371–385 ↗

(On Diff #101190)

How so? I'm curious what the issue is here... Callbacks seem like the most complex form of extension, whereas nesting seems much simpler...

480–489 ↗

(On Diff #102210)

I would sink this to after the other pipeline parsing callback registration entry points.

572 ↗

(On Diff #102210)

EP -> Extension Point.

lib/Passes/PassBuilder.cpp

990 ↗

(On Diff #102210)

Here especially (but also elsewhere) I'd put the invokePeepholeEP... line adjacent to the InstCombinePass line (they're the two peepholes) rather than next to the subsequent stuff which is unrelated.

1312 ↗

(On Diff #102210)

No need for blank line here.

1571–1574 ↗

(On Diff #102210)

Should this go after we parse the builtin in AA names?

1678 ↗

(On Diff #102210)

No need for blank line.

1699–1708 ↗

(On Diff #102210)

I'm not sure you want this fallback logic here...

If you want to be able to essentially duplicate the above logic, you want this to *only* succeed at parsing a CGSCC pass pipeline.

Otherwise, you'll never be able to skip the CGSCC layer when you get a function pass (or something nested inside a function pass) and you're trying to build the module layer thing.

1722–1729 ↗

(On Diff #102210)

While less important, same comment as above.

test/Other/new-pm-defaults.ll

29–33 ↗

(On Diff #102210)

What about having just one opt invocation with a no-op pass in *every* extension point? I don't think you lose any coverage, and then you have a much easier check prefix of just CHECK-EP.

tools/opt/NewPMDriver.cpp

101–105 ↗

(On Diff #102210)

Do the polly bits in a follow-up commit?

unittests/IR/HookManagerTest.cpp

23 ↗

(On Diff #102210)

Rather than counting things, I would suggest using gmock for this. It is kind of what gmock was built for. To see how to get value-type wrappers around mocks that you can set expectations on, look at unittests/Transforms/Scalar/LoopPassManagerTest.cpp. You should end up with very similar code to this to template out the mocks over the types you care about, but should be able to use things like EXPECT_CALL to track when 'invalidate' or 'run' are called which seems way better than counting.

83–113 ↗

(On Diff #102210)

I think you can inline all this as lambas below? You shouldn't need a generic lambda because you should have the types inside that template already.

122–127 ↗

(On Diff #102210)

Throughout this test, I would try to comment a bit more heavily. My suggestion would be the following: assume the reader is trying to learn how to use the extension point interface to write their own customizations, and they are relatively unfamiliar with the extension point API, googletest (especially typed tests) and goolemock. You want to have enough comments that they can read this code cold and use it as documentation for how the API should work in practice (as opposed to its spec).

Also, probably need a better name than HookManagerTest?

[philip.pfaffe]

Addressed (most of the) review comments:

• Drop Early and Late EPs
• Fix nits
• Convert the unittest to gmock

Two things are still open and require more discussion:

1. The overloads of parsePassPipeline. These functions implement top level parsing. I think we want automatic nesting inference in all these cases. A use case is for example this: opt can now parse EP pipelines into the various EPs, most of which do not construct a ModulePassManager. Consider running opt -passes-ep-peephole="licm". The Peephole EP inserts passes into a FunctionPassManager, and as with the -passes option, this simplifies typing out the pipeline a lot.
2. A single opt invocation in the new-pm-defaults.ll testcase: Making this a single opt RUN line would certainly simplify the testcase a bit. What I like about the way this looks now, however, is that the CHECK-EP-* prefixes show very clearly what and where the extensions points are.

[chandlerc]

Pretty desperately need to factor the mocking helpers out, but that's not your fault. The actual test code is really clean IMO.

As discussed in IRC, I'd still like to remove the synthesis of inner IR-unit pipelines for EPs and remove the extra parsing APIs. I don't think they pull their weight. But the opt check is fine with a tweak below.

More detailed comments on the unittest mostly.

include/llvm/Passes/PassBuilder.h

472 ↗

(On Diff #105058)

calles -> callers

test/Other/new-pm-defaults.ll

29–33 ↗

(On Diff #102210)

I see, you want to check the *different* extension points.

OK, in this case, I would still name the check prefixes using ALL-CAPS-WITH-DASHES rather than the mixed case you have.

unittests/IR/PassBuilderCallbacksTest.cpp

159 ↗

(On Diff #105058)

API grouping doxygen doesn't seem terribly useful inside the bowels of the test.

270 ↗

(On Diff #105058)

You can't name identifiers with leading _ and a capital letter.

Perhaps TestIRUnitT?

295–300 ↗

(On Diff #105058)

I would inline this into the caller so you can write more precise expectations.

For example:

ASSERT_TRUE(PB.parsePassPipeline(this->PM, PipelineText, true)) << PipelineText;

Seems likely to give a much nicer error when it fails. =]

I guess you could provide wrappers around PB.parsePassPipeline to ease some of the code, but it's not clear it is worth it. The test should be easy to inspect even if a very minor of repetition is involved.

Also, do you need the this-> here?

311–312 ↗

(On Diff #105058)

Why local_unnamed_addr? Seems like this test should really be as simple as possible?

[philip.pfaffe]

Addressed Comments

[chandlerc]

LGTM

I have some thoughts about simplifying the test, but let's ship this. I think those can be done incrementally afterward. Thanks again for all the work here!

test/Other/new-pm-lto-defaults.ll

20 ↗

(On Diff #105688)

nit: Can omit blank line I think?

[grosser]

Very nice! Thanks to everyone for working on this!

llvm/trunk/include/llvm/Passes/PassBuilder.h

 /// construction. The \c PassRegistry.def file specifies how to construct all
 /// of the built-in passes, and those may reference these members during
 /// construction.
 class PassBuilder {
   TargetMachine *TM;
   Optional<PGOOptions> PGOOpt;
 
 public:
+  /// \brief A struct to capture parsed pass pipeline names.
+  ///
+  /// A pipeline is defined as a series of names, each of which may in itself
+  /// recursively contain a nested pipeline. A name is either the name of a pass
+  /// (e.g. "instcombine") or the name of a pipeline type (e.g. "cgscc"). If the
+  /// name is the name of a pass, the InnerPipeline is empty, since passes
+  /// cannot contain inner pipelines. See parsePassPipeline() for a more
+  /// detailed description of the textual pipeline format.
+  struct PipelineElement {
+    StringRef Name;
+    std::vector<PipelineElement> InnerPipeline;
+  };
+
   /// \brief LLVM-provided high-level optimization levels.
   ///
   /// This enumerates the LLVM-provided high-level optimization levels. Each
   /// level has a specific goal and rationale.
   enum OptimizationLevel {
     /// Disable as many optimizations as possible. This doesn't completely
     /// disable the optimizer in all cases, for example always_inline functions
     /// can be required to be inlined for correctness.
public:
   /// build them.
   ModulePassManager buildLTODefaultPipeline(OptimizationLevel Level,
                                             bool DebugLogging = false);
 
   /// Build the default `AAManager` with the default alias analysis pipeline
   /// registered.
   AAManager buildDefaultAAPipeline();
 
-  /// \brief Parse a textual pass pipeline description into a \c ModulePassManager.
+  /// \brief Parse a textual pass pipeline description into a \c
+  /// ModulePassManager.
   ///
   /// The format of the textual pass pipeline description looks something like:
   ///
   ///   module(function(instcombine,sroa),dce,cgscc(inliner,function(...)),...)
   ///
   /// Pass managers have ()s describing the nest structure of passes. All passes
   /// are comma separated. As a special shortcut, if the very first pass is not
   /// a module pass (as a module pass manager is), this will automatically form
   /// the shortest stack of pass managers that allow inserting that first pass.
-  /// So, assuming function passes 'fpassN', CGSCC passes 'cgpassN', and loop passes
-  /// 'lpassN', all of these are valid:
+  /// So, assuming function passes 'fpassN', CGSCC passes 'cgpassN', and loop
+  /// passes 'lpassN', all of these are valid:
   ///
   ///   fpass1,fpass2,fpass3
   ///   cgpass1,cgpass2,cgpass3
   ///   lpass1,lpass2,lpass3
   ///
   /// And they are equivalent to the following (resp.):
   ///
   ///   module(function(fpass1,fpass2,fpass3))
   ///   module(cgscc(cgpass1,cgpass2,cgpass3))
   ///   module(function(loop(lpass1,lpass2,lpass3)))
   ///
   /// This shortcut is especially useful for debugging and testing small pass
-  /// combinations. Note that these shortcuts don't introduce any other magic. If
-  /// the sequence of passes aren't all the exact same kind of pass, it will be
-  /// an error. You cannot mix different levels implicitly, you must explicitly
-  /// form a pass manager in which to nest passes.
+  /// combinations. Note that these shortcuts don't introduce any other magic.
+  /// If the sequence of passes aren't all the exact same kind of pass, it will
+  /// be an error. You cannot mix different levels implicitly, you must
+  /// explicitly form a pass manager in which to nest passes.
   bool parsePassPipeline(ModulePassManager &MPM, StringRef PipelineText,
                          bool VerifyEachPass = true, bool DebugLogging = false);
 
+  /// {{@ Parse a textual pass pipeline description into a specific PassManager
+  ///
+  /// Automatic deduction of an appropriate pass manager stack is not supported.
+  /// For example, to insert a loop pass 'lpass' into a FunctinoPassManager,
+  /// this is the valid pipeline text:
+  ///
+  ///   function(lpass)
+  bool parsePassPipeline(CGSCCPassManager &CGPM, StringRef PipelineText,
+                         bool VerifyEachPass = true, bool DebugLogging = false);
+  bool parsePassPipeline(FunctionPassManager &FPM, StringRef PipelineText,
+                         bool VerifyEachPass = true, bool DebugLogging = false);
+  bool parsePassPipeline(LoopPassManager &LPM, StringRef PipelineText,
+                         bool VerifyEachPass = true, bool DebugLogging = false);
+  /// @}}
+
   /// Parse a textual alias analysis pipeline into the provided AA manager.
   ///
   /// The format of the textual AA pipeline is a comma separated list of AA
   /// pass names:
   ///
   ///   basic-aa,globals-aa,...
   ///
   /// The AA manager is set up such that the provided alias analyses are tried
   /// in the order specified. See the \c AAManaager documentation for details
   /// about the logic used. This routine just provides the textual mapping
   /// between AA names and the analyses to register with the manager.
   ///
   /// Returns false if the text cannot be parsed cleanly. The specific state of
   /// the \p AA manager is unspecified if such an error is encountered and this
   /// returns false.
   bool parseAAPipeline(AAManager &AA, StringRef PipelineText);
 
-private:
-  /// A struct to capture parsed pass pipeline names.
-  struct PipelineElement {
-    StringRef Name;
-    std::vector<PipelineElement> InnerPipeline;
-  };
+  /// \brief Register a callback for a default optimizer pipeline extension
+  /// point
+  ///
+  /// This extension point allows adding passes that perform peephole
+  /// optimizations similar to the instruction combiner. These passes will be
+  /// inserted after each instance of the instruction combiner pass.
+  void registerPeepholeEPCallback(
+      const std::function<void(FunctionPassManager &, OptimizationLevel)> &C) {
+    PeepholeEPCallbacks.push_back(C);
+  }
+
+  /// \brief Register a callback for a default optimizer pipeline extension
+  /// point
+  ///
+  /// This extension point allows adding late loop canonicalization and
+  /// simplification passes. This is the last point in the loop optimization
+  /// pipeline before loop deletion. Each pass added
+  /// here must be an instance of LoopPass.
+  /// This is the place to add passes that can remove loops, such as target-
+  /// specific loop idiom recognition.
+  void registerLateLoopOptimizationsEPCallback(
+      const std::function<void(LoopPassManager &, OptimizationLevel)> &C) {
+    LateLoopOptimizationsEPCallbacks.push_back(C);
+  }
 
+  /// \brief Register a callback for a default optimizer pipeline extension
+  /// point
+  ///
+  /// This extension point allows adding loop passes to the end of the loop
+  /// optimizer.
+  void registerLoopOptimizerEndEPCallback(
+      const std::function<void(LoopPassManager &, OptimizationLevel)> &C) {
+    LoopOptimizerEndEPCallbacks.push_back(C);
+  }
+
+  /// \brief Register a callback for a default optimizer pipeline extension
+  /// point
+  ///
+  /// This extension point allows adding optimization passes after most of the
+  /// main optimizations, but before the last cleanup-ish optimizations.
+  void registerScalarOptimizerLateEPCallback(
+      const std::function<void(FunctionPassManager &, OptimizationLevel)> &C) {
+    ScalarOptimizerLateEPCallbacks.push_back(C);
+  }
+
+  /// \brief Register a callback for a default optimizer pipeline extension
+  /// point
+  ///
+  /// This extension point allows adding CallGraphSCC passes at the end of the
+  /// main CallGraphSCC passes and before any function simplification passes run
+  /// by CGPassManager.
+  void registerCGSCCOptimizerLateEPCallback(
+      const std::function<void(CGSCCPassManager &, OptimizationLevel)> &C) {
+    CGSCCOptimizerLateEPCallbacks.push_back(C);
+  }
+
+  /// \brief Register a callback for a default optimizer pipeline extension
+  /// point
+  ///
+  /// This extension point allows adding optimization passes before the
+  /// vectorizer and other highly target specific optimization passes are
+  /// executed.
+  void registerVectorizerStartEPCallback(
+      const std::function<void(FunctionPassManager &, OptimizationLevel)> &C) {
+    VectorizerStartEPCallbacks.push_back(C);
+  }
+
+  /// \brief Register a callback for parsing an AliasAnalysis Name to populate
+  /// the given AAManager \p AA
+  void registerParseAACallback(
+      const std::function<bool(StringRef Name, AAManager &AA)> &C) {
+    AAParsingCallbacks.push_back(C);
+  }
+
+  /// {{@ Register callbacks for analysis registration with this PassBuilder
+  /// instance.
+  /// Callees register their analyses with the given AnalysisManager objects.
+  void registerAnalysisRegistrationCallback(
+      const std::function<void(CGSCCAnalysisManager &)> &C) {
+    CGSCCAnalysisRegistrationCallbacks.push_back(C);
+  }
+  void registerAnalysisRegistrationCallback(
+      const std::function<void(FunctionAnalysisManager &)> &C) {
+    FunctionAnalysisRegistrationCallbacks.push_back(C);
+  }
+  void registerAnalysisRegistrationCallback(
+      const std::function<void(LoopAnalysisManager &)> &C) {
+    LoopAnalysisRegistrationCallbacks.push_back(C);
+  }
+  void registerAnalysisRegistrationCallback(
+      const std::function<void(ModuleAnalysisManager &)> &C) {
+    ModuleAnalysisRegistrationCallbacks.push_back(C);
+  }
+  /// @}}
+
+  /// {{@ Register pipeline parsing callbacks with this pass builder instance.
+  /// Using these callbacks, callers can parse both a single pass name, as well
+  /// as entire sub-pipelines, and populate the PassManager instance
+  /// accordingly.
+  void registerPipelineParsingCallback(
+      const std::function<bool(StringRef Name, CGSCCPassManager &,
+                               ArrayRef<PipelineElement>)> &C) {
+    CGSCCPipelineParsingCallbacks.push_back(C);
+  }
+  void registerPipelineParsingCallback(
+      const std::function<bool(StringRef Name, FunctionPassManager &,
+                               ArrayRef<PipelineElement>)> &C) {
+    FunctionPipelineParsingCallbacks.push_back(C);
+  }
+  void registerPipelineParsingCallback(
+      const std::function<bool(StringRef Name, LoopPassManager &,
+                               ArrayRef<PipelineElement>)> &C) {
+    LoopPipelineParsingCallbacks.push_back(C);
+  }
+  void registerPipelineParsingCallback(
+      const std::function<bool(StringRef Name, ModulePassManager &,
+                               ArrayRef<PipelineElement>)> &C) {
+    ModulePipelineParsingCallbacks.push_back(C);
+  }
+  /// @}}
+
+  /// \brief Register a callback for a top-level pipeline entry.
+  ///
+  /// If the PassManager type is not given at the top level of the pipeline
+  /// text, this Callback should be used to determine the appropriate stack of
+  /// PassManagers and populate the passed ModulePassManager.
+  void registerParseTopLevelPipelineCallback(
+      const std::function<bool(ModulePassManager &, ArrayRef<PipelineElement>,
+                               bool VerifyEachPass, bool DebugLogging)> &C) {
+    TopLevelPipelineParsingCallbacks.push_back(C);
+  }
+
+private:
   static Optional<std::vector<PipelineElement>>
   parsePipelineText(StringRef Text);
 
   bool parseModulePass(ModulePassManager &MPM, const PipelineElement &E,
                        bool VerifyEachPass, bool DebugLogging);
   bool parseCGSCCPass(CGSCCPassManager &CGPM, const PipelineElement &E,
                       bool VerifyEachPass, bool DebugLogging);
   bool parseFunctionPass(FunctionPassManager &FPM, const PipelineElement &E,
bool parseFunctionPassPipeline(FunctionPassManager &FPM,
                                  ArrayRef<PipelineElement> Pipeline,
                                  bool VerifyEachPass, bool DebugLogging);
   bool parseCGSCCPassPipeline(CGSCCPassManager &CGPM,
                               ArrayRef<PipelineElement> Pipeline,
                               bool VerifyEachPass, bool DebugLogging);
   bool parseModulePassPipeline(ModulePassManager &MPM,
                                ArrayRef<PipelineElement> Pipeline,
                                bool VerifyEachPass, bool DebugLogging);
+
+  void addPGOInstrPasses(ModulePassManager &MPM, bool DebugLogging,
+                         OptimizationLevel Level, bool RunProfileGen,
+                         std::string ProfileGenFile,
+                         std::string ProfileUseFile);
+
+  void invokePeepholeEPCallbacks(FunctionPassManager &, OptimizationLevel);
+
+  // Extension Point callbacks
+  SmallVector<std::function<void(FunctionPassManager &, OptimizationLevel)>, 2>
+      PeepholeEPCallbacks;
+  SmallVector<std::function<void(LoopPassManager &, OptimizationLevel)>, 2>
+      LateLoopOptimizationsEPCallbacks;
+  SmallVector<std::function<void(LoopPassManager &, OptimizationLevel)>, 2>
+      LoopOptimizerEndEPCallbacks;
+  SmallVector<std::function<void(FunctionPassManager &, OptimizationLevel)>, 2>
+      ScalarOptimizerLateEPCallbacks;
+  SmallVector<std::function<void(CGSCCPassManager &, OptimizationLevel)>, 2>
+      CGSCCOptimizerLateEPCallbacks;
+  SmallVector<std::function<void(FunctionPassManager &, OptimizationLevel)>, 2>
+      VectorizerStartEPCallbacks;
+  // Module callbacks
+  SmallVector<std::function<void(ModuleAnalysisManager &)>, 2>
+      ModuleAnalysisRegistrationCallbacks;
+  SmallVector<std::function<bool(StringRef, ModulePassManager &,
+                                 ArrayRef<PipelineElement>)>,
+              2>
+      ModulePipelineParsingCallbacks;
+  SmallVector<std::function<bool(ModulePassManager &, ArrayRef<PipelineElement>,
+                                 bool VerifyEachPass, bool DebugLogging)>,
+              2>
+      TopLevelPipelineParsingCallbacks;
+  // CGSCC callbacks
+  SmallVector<std::function<void(CGSCCAnalysisManager &)>, 2>
+      CGSCCAnalysisRegistrationCallbacks;
+  SmallVector<std::function<bool(StringRef, CGSCCPassManager &,
+                                 ArrayRef<PipelineElement>)>,
+              2>
+      CGSCCPipelineParsingCallbacks;
+  // Function callbacks
+  SmallVector<std::function<void(FunctionAnalysisManager &)>, 2>
+      FunctionAnalysisRegistrationCallbacks;
+  SmallVector<std::function<bool(StringRef, FunctionPassManager &,
+                                 ArrayRef<PipelineElement>)>,
+              2>
+      FunctionPipelineParsingCallbacks;
+  // Loop callbacks
+  SmallVector<std::function<void(LoopAnalysisManager &)>, 2>
+      LoopAnalysisRegistrationCallbacks;
+  SmallVector<std::function<bool(StringRef, LoopPassManager &,
+                                 ArrayRef<PipelineElement>)>,
+              2>
+      LoopPipelineParsingCallbacks;
+  // AA callbacks
+  SmallVector<std::function<bool(StringRef Name, AAManager &AA)>, 2>
+      AAParsingCallbacks;
 };
+
+/// This utility template takes care of adding require<> and invalidate<>
+/// passes for an analysis to a given \c PassManager. It is intended to be used
+/// during parsing of a pass pipeline when parsing a single PipelineName.
+/// When registering a new function analysis FancyAnalysis with the pass
+/// pipeline name "fancy-analysis", a matching ParsePipelineCallback could look
+/// like this:
+///
+/// static bool parseFunctionPipeline(StringRef Name, FunctionPassManager &FPM,
+///                                   ArrayRef<PipelineElement> P) {
+///   if (parseAnalysisUtilityPasses<FancyAnalysis>("fancy-analysis", Name,
+///                                                 FPM))
+///     return true;
+///   return false;
+/// }
+template <typename AnalysisT, typename IRUnitT, typename AnalysisManagerT,
+          typename... ExtraArgTs>
+bool parseAnalysisUtilityPasses(
+    StringRef AnalysisName, StringRef PipelineName,
+    PassManager<IRUnitT, AnalysisManagerT, ExtraArgTs...> &PM) {
+  if (!PipelineName.endswith(">"))
+    return false;
+  // See if this is an invalidate<> pass name
+  if (PipelineName.startswith("invalidate<")) {
+    PipelineName = PipelineName.substr(11, PipelineName.size() - 12);
+    if (PipelineName != AnalysisName)
+      return false;
+    PM.addPass(InvalidateAnalysisPass<AnalysisT>());
+    return true;
+  }
+
+  // See if this is a require<> pass name
+  if (PipelineName.startswith("require<")) {
+    PipelineName = PipelineName.substr(8, PipelineName.size() - 9);
+    if (PipelineName != AnalysisName)
+      return false;
+    PM.addPass(RequireAnalysisPass<AnalysisT, IRUnitT, AnalysisManagerT,
+                                   ExtraArgTs...>());
+    return true;
+  }
+
+  return false;
+}
 }
 
 #endif

llvm/trunk/lib/Passes/PassBuilder.cpp

 
 AnalysisKey NoOpModuleAnalysis::Key;
 AnalysisKey NoOpCGSCCAnalysis::Key;
 AnalysisKey NoOpFunctionAnalysis::Key;
 AnalysisKey NoOpLoopAnalysis::Key;
 
 } // End anonymous namespace.
 
+void PassBuilder::invokePeepholeEPCallbacks(
+    FunctionPassManager &FPM, PassBuilder::OptimizationLevel Level) {
+  for (auto &C : PeepholeEPCallbacks)
+    C(FPM, Level);
+}
+
 void PassBuilder::registerModuleAnalyses(ModuleAnalysisManager &MAM) {
 #define MODULE_ANALYSIS(NAME, CREATE_PASS)                                     \
   MAM.registerPass([&] { return CREATE_PASS; });
 #include "PassRegistry.def"
+
+  for (auto &C : ModuleAnalysisRegistrationCallbacks)
+    C(MAM);
 }
 
 void PassBuilder::registerCGSCCAnalyses(CGSCCAnalysisManager &CGAM) {
 #define CGSCC_ANALYSIS(NAME, CREATE_PASS)                                      \
   CGAM.registerPass([&] { return CREATE_PASS; });
 #include "PassRegistry.def"
+
+  for (auto &C : CGSCCAnalysisRegistrationCallbacks)
+    C(CGAM);
 }
 
 void PassBuilder::registerFunctionAnalyses(FunctionAnalysisManager &FAM) {
 #define FUNCTION_ANALYSIS(NAME, CREATE_PASS)                                   \
   FAM.registerPass([&] { return CREATE_PASS; });
 #include "PassRegistry.def"
+
+  for (auto &C : FunctionAnalysisRegistrationCallbacks)
+    C(FAM);
 }
 
 void PassBuilder::registerLoopAnalyses(LoopAnalysisManager &LAM) {
 #define LOOP_ANALYSIS(NAME, CREATE_PASS)                                       \
   LAM.registerPass([&] { return CREATE_PASS; });
 #include "PassRegistry.def"
+
+  for (auto &C : LoopAnalysisRegistrationCallbacks)
+    C(LAM);
 }
 
 FunctionPassManager
 PassBuilder::buildFunctionSimplificationPipeline(OptimizationLevel Level,
                                                  bool DebugLogging,
                                                  bool PrepareForThinLTO) {
   assert(Level != O0 && "Must request optimizations!");
   FunctionPassManager FPM(DebugLogging);
PassBuilder::buildFunctionSimplificationPipeline(OptimizationLevel Level,
   FPM.addPass(JumpThreadingPass());
   FPM.addPass(CorrelatedValuePropagationPass());
   FPM.addPass(SimplifyCFGPass());
   FPM.addPass(InstCombinePass());
 
   if (!isOptimizingForSize(Level))
     FPM.addPass(LibCallsShrinkWrapPass());
 
+  invokePeepholeEPCallbacks(FPM, Level);
+
   FPM.addPass(TailCallElimPass());
   FPM.addPass(SimplifyCFGPass());
 
   // Form canonically associated expression trees, and simplify the trees using
   // basic mathematical properties. For example, this will form (nearly)
   // minimal multiplication trees.
   FPM.addPass(ReassociatePass());
 
   // Add the primary loop simplification pipeline.
   // FIXME: Currently this is split into two loop pass pipelines because we run
   // some function passes in between them. These can and should be replaced by
   // loop pass equivalenst but those aren't ready yet. Specifically,
   // `SimplifyCFGPass` and `InstCombinePass` are used. We have
   // `LoopSimplifyCFGPass` which isn't yet powerful enough, and the closest to
   // the other we have is `LoopInstSimplify`.
   LoopPassManager LPM1(DebugLogging), LPM2(DebugLogging);
 
   // Rotate Loop - disable header duplication at -Oz
   LPM1.addPass(LoopRotatePass(Level != Oz));
   LPM1.addPass(LICMPass());
   LPM1.addPass(SimpleLoopUnswitchPass());
   LPM2.addPass(IndVarSimplifyPass());
   LPM2.addPass(LoopIdiomRecognizePass());
+
+  for (auto &C : LateLoopOptimizationsEPCallbacks)
+    C(LPM2, Level);
+
   LPM2.addPass(LoopDeletionPass());
   // Do not enable unrolling in PrepareForThinLTO phase during sample PGO
   // because it changes IR to makes profile annotation in back compile
   // inaccurate.
   if (!PrepareForThinLTO || !PGOOpt || PGOOpt->SampleProfileFile.empty())
     LPM2.addPass(LoopUnrollPass::createFull(Level));
 
+  for (auto &C : LoopOptimizerEndEPCallbacks)
+    C(LPM2, Level);
+
   // We provide the opt remark emitter pass for LICM to use. We only need to do
   // this once as it is immutable.
   FPM.addPass(RequireAnalysisPass<OptimizationRemarkEmitterAnalysis, Function>());
   FPM.addPass(createFunctionToLoopPassAdaptor(std::move(LPM1)));
   FPM.addPass(SimplifyCFGPass());
   FPM.addPass(InstCombinePass());
   FPM.addPass(createFunctionToLoopPassAdaptor(std::move(LPM2)));
 
PassBuilder::buildFunctionSimplificationPipeline(OptimizationLevel Level,
   // Delete dead bit computations (instcombine runs after to fold away the dead
   // computations, and then ADCE will run later to exploit any new DCE
   // opportunities that creates).
   FPM.addPass(BDCEPass());
 
   // Run instcombine after redundancy and dead bit elimination to exploit
   // opportunities opened up by them.
   FPM.addPass(InstCombinePass());
+  invokePeepholeEPCallbacks(FPM, Level);
 
   // Re-consider control flow based optimizations after redundancy elimination,
   // redo DCE, etc.
   FPM.addPass(JumpThreadingPass());
   FPM.addPass(CorrelatedValuePropagationPass());
   FPM.addPass(DSEPass());
   FPM.addPass(createFunctionToLoopPassAdaptor(LICMPass()));
 
+  for (auto &C : ScalarOptimizerLateEPCallbacks)
+    C(FPM, Level);
+
   // Finally, do an expensive DCE pass to catch all the dead code exposed by
   // the simplifications and basic cleanup after all the simplifications.
   FPM.addPass(ADCEPass());
   FPM.addPass(SimplifyCFGPass());
   FPM.addPass(InstCombinePass());
+  invokePeepholeEPCallbacks(FPM, Level);
 
   return FPM;
 }
 
-static void addPGOInstrPasses(ModulePassManager &MPM, bool DebugLogging,
+void PassBuilder::addPGOInstrPasses(ModulePassManager &MPM, bool DebugLogging,
                                     PassBuilder::OptimizationLevel Level,
-                              bool RunProfileGen, std::string ProfileGenFile,
+                                    bool RunProfileGen,
+                                    std::string ProfileGenFile,
                                     std::string ProfileUseFile) {
   // Generally running simplification passes and the inliner with an high
   // threshold results in smaller executables, but there may be cases where
   // the size grows, so let's be conservative here and skip this simplification
   // at -Os/Oz.
   if (!isOptimizingForSize(Level)) {
     InlineParams IP;
 
     // In the old pass manager, this is a cl::opt. Should still this be one?
     IP.DefaultThreshold = 75;
 
     // FIXME: The hint threshold has the same value used by the regular inliner.
     // This should probably be lowered after performance testing.
     // FIXME: this comment is cargo culted from the old pass manager, revisit).
     IP.HintThreshold = 325;
 
     CGSCCPassManager CGPipeline(DebugLogging);
 
     CGPipeline.addPass(InlinerPass(IP));
 
     FunctionPassManager FPM;
     FPM.addPass(SROA());
     FPM.addPass(EarlyCSEPass());    // Catch trivial redundancies.
     FPM.addPass(SimplifyCFGPass()); // Merge & remove basic blocks.
     FPM.addPass(InstCombinePass()); // Combine silly sequences.
+    invokePeepholeEPCallbacks(FPM, Level);
 
-    // FIXME: Here the old pass manager inserts peephole extensions.
-    // Add them when they're supported.
     CGPipeline.addPass(createCGSCCToFunctionPassAdaptor(std::move(FPM)));
 
     MPM.addPass(createModuleToPostOrderCGSCCPassAdaptor(std::move(CGPipeline)));
   }
 
   // Delete anything that is now dead to make sure that we don't instrument
   // dead code. Instrumentation can end up keeping dead code around and
   // dramatically increase code size.
PassBuilder::buildModuleSimplificationPipeline(OptimizationLevel Level,
   // Remove any dead arguments exposed by cleanups and constand folding
   // globals.
   MPM.addPass(DeadArgumentEliminationPass());
 
   // Create a small function pass pipeline to cleanup after all the global
   // optimizations.
   FunctionPassManager GlobalCleanupPM(DebugLogging);
   GlobalCleanupPM.addPass(InstCombinePass());
+  invokePeepholeEPCallbacks(GlobalCleanupPM, Level);
+
   GlobalCleanupPM.addPass(SimplifyCFGPass());
   MPM.addPass(createModuleToFunctionPassAdaptor(std::move(GlobalCleanupPM)));
 
   // Add all the requested passes for PGO, if requested.
   if (PGOOpt) {
     assert(PGOOpt->RunProfileGen || !PGOOpt->SampleProfileFile.empty() ||
            !PGOOpt->ProfileUseFile.empty());
     if (PGOOpt->SampleProfileFile.empty())
if (Level == O3)
     MainCGPipeline.addPass(ArgumentPromotionPass());
 
   // Lastly, add the core function simplification pipeline nested inside the
   // CGSCC walk.
   MainCGPipeline.addPass(createCGSCCToFunctionPassAdaptor(
       buildFunctionSimplificationPipeline(Level, DebugLogging,
                                           PrepareForThinLTO)));
 
+  for (auto &C : CGSCCOptimizerLateEPCallbacks)
+    C(MainCGPipeline, Level);
+
   // We wrap the CGSCC pipeline in a devirtualization repeater. This will try
   // to detect when we devirtualize indirect calls and iterate the SCC passes
   // in that case to try and catch knock-on inlining or function attrs
   // opportunities. Then we add it to the module pipeline by walking the SCCs
   // in postorder (or bottom-up).
   MPM.addPass(
       createModuleToPostOrderCGSCCPassAdaptor(createDevirtSCCRepeatedPass(
           std::move(MainCGPipeline), MaxDevirtIterations, DebugLogging)));
PassBuilder::buildModuleOptimizationPipeline(OptimizationLevel Level,
   OptimizePM.addPass(Float2IntPass());
   // FIXME: We need to run some loop optimizations to re-rotate loops after
   // simplify-cfg and others undo their rotation.
 
   // Optimize the loop execution. These passes operate on entire loop nests
   // rather than on each loop in an inside-out manner, and so they are actually
   // function passes.
 
+  for (auto &C : VectorizerStartEPCallbacks)
+    C(OptimizePM, Level);
+
   // First rotate loops that may have been un-rotated by prior passes.
   OptimizePM.addPass(createFunctionToLoopPassAdaptor(LoopRotatePass()));
 
   // Distribute loops to allow partial vectorization.  I.e. isolate dependences
   // into separate loop that would otherwise inhibit vectorization.  This is
   // currently only performed for loops marked with the metadata
   // llvm.loop.distribute=true or when -enable-loop-distribute is specified.
   OptimizePM.addPass(LoopDistributePass());
ModulePassManager PassBuilder::buildLTODefaultPipeline(OptimizationLevel Level,
 
   // Remove unused arguments from functions.
   MPM.addPass(DeadArgumentEliminationPass());
 
   // Reduce the code after globalopt and ipsccp.  Both can open up significant
   // simplification opportunities, and both can propagate functions through
   // function pointers.  When this happens, we often have to resolve varargs
   // calls, etc, so let instcombine do this.
-  // FIXME: add peephole extensions here as the legacy PM does.
-  MPM.addPass(createModuleToFunctionPassAdaptor(InstCombinePass()));
+  FunctionPassManager PeepholeFPM(DebugLogging);
+  PeepholeFPM.addPass(InstCombinePass());
+  invokePeepholeEPCallbacks(PeepholeFPM, Level);
+
+  MPM.addPass(createModuleToFunctionPassAdaptor(std::move(PeepholeFPM)));
 
   // Note: historically, the PruneEH pass was run first to deduce nounwind and
   // generally clean up exception handling overhead. It isn't clear this is
   // valuable as the inliner doesn't currently care whether it is inlining an
   // invoke or a call.
   // Run the inliner now.
   MPM.addPass(createModuleToPostOrderCGSCCPassAdaptor(
       InlinerPass(getInlineParamsFromOptLevel(Level))));
 
   // Optimize globals again after we ran the inliner.
   MPM.addPass(GlobalOptPass());
 
   // Garbage collect dead functions.
   // FIXME: Add ArgumentPromotion pass after once it's ported.
   MPM.addPass(GlobalDCEPass());
 
   FunctionPassManager FPM(DebugLogging);
-
   // The IPO Passes may leave cruft around. Clean up after them.
-  // FIXME: add peephole extensions here as the legacy PM does.
   FPM.addPass(InstCombinePass());
+  invokePeepholeEPCallbacks(FPM, Level);
+
   FPM.addPass(JumpThreadingPass());
 
   // Break up allocas
   FPM.addPass(SROA());
 
   // Run a few AA driver optimizations here and now to cleanup the code.
   MPM.addPass(createModuleToFunctionPassAdaptor(std::move(FPM)));
 
ModulePassManager PassBuilder::buildLTODefaultPipeline(OptimizationLevel Level,
 
   // FIXME: We may want to run SLPVectorizer here.
   // After vectorization, assume intrinsics may tell us more
   // about pointer alignments.
 #if 0
   MainFPM.add(AlignmentFromAssumptionsPass());
 #endif
 
-  // FIXME: add peephole extensions to the PM here.
+  // FIXME: Conditionally run LoadCombine here, after it's ported
+  // (in case we still have this pass, given its questionable usefulness).
+
   MainFPM.addPass(InstCombinePass());
+  invokePeepholeEPCallbacks(MainFPM, Level);
   MainFPM.addPass(JumpThreadingPass());
   MPM.addPass(createModuleToFunctionPassAdaptor(std::move(MainFPM)));
 
   // Create a function that performs CFI checks for cross-DSO calls with
   // targets in the current module.
   MPM.addPass(CrossDSOCFIPass());
 
   // Lower type metadata and the type.test intrinsic. This pass supports

 
 /// Tests whether a pass name starts with a valid prefix for a default pipeline
 /// alias.
 static bool startsWithDefaultPipelineAliasPrefix(StringRef Name) {
   return Name.startswith("default") || Name.startswith("thinlto") ||
          Name.startswith("lto");
 }
 
-static bool isModulePassName(StringRef Name) {
+/// Tests whether registered callbacks will accept a given pass name.
+///
+/// When parsing a pipeline text, the type of the outermost pipeline may be
+/// omitted, in which case the type is automatically determined from the first
+/// pass name in the text. This may be a name that is handled through one of the
+/// callbacks. We check this through the oridinary parsing callbacks by setting
+/// up a dummy PassManager in order to not force the client to also handle this
+/// type of query.
+template <typename PassManagerT, typename CallbacksT>
+static bool callbacksAcceptPassName(StringRef Name, CallbacksT &Callbacks) {
+  if (!Callbacks.empty()) {
+    PassManagerT DummyPM;
+    for (auto &CB : Callbacks)
+      if (CB(Name, DummyPM, {}))
+        return true;
+  }
+  return false;
+}
+
+template <typename CallbacksT>
+static bool isModulePassName(StringRef Name, CallbacksT &Callbacks) {
   // Manually handle aliases for pre-configured pipeline fragments.
   if (startsWithDefaultPipelineAliasPrefix(Name))
     return DefaultAliasRegex.match(Name);
 
   // Explicitly handle pass manager names.
   if (Name == "module")
     return true;
   if (Name == "cgscc")
     return true;
   if (Name == "function")
     return true;
 
   // Explicitly handle custom-parsed pass names.
   if (parseRepeatPassName(Name))
     return true;
 
 #define MODULE_PASS(NAME, CREATE_PASS)                                         \
   if (Name == NAME)                                                            \
     return true;
 #define MODULE_ANALYSIS(NAME, CREATE_PASS)                                     \
   if (Name == "require<" NAME ">" || Name == "invalidate<" NAME ">")           \
     return true;
 #include "PassRegistry.def"
 
-  return false;
+  return callbacksAcceptPassName<ModulePassManager>(Name, Callbacks);
 }
 
-static bool isCGSCCPassName(StringRef Name) {
+template <typename CallbacksT>
+static bool isCGSCCPassName(StringRef Name, CallbacksT &Callbacks) {
   // Explicitly handle pass manager names.
   if (Name == "cgscc")
     return true;
   if (Name == "function")
     return true;
 
   // Explicitly handle custom-parsed pass names.
   if (parseRepeatPassName(Name))
     return true;
   if (parseDevirtPassName(Name))
     return true;
 
 #define CGSCC_PASS(NAME, CREATE_PASS)                                          \
   if (Name == NAME)                                                            \
     return true;
 #define CGSCC_ANALYSIS(NAME, CREATE_PASS)                                      \
   if (Name == "require<" NAME ">" || Name == "invalidate<" NAME ">")           \
     return true;
 #include "PassRegistry.def"
 
-  return false;
+  return callbacksAcceptPassName<CGSCCPassManager>(Name, Callbacks);
 }
 
-static bool isFunctionPassName(StringRef Name) {
+template <typename CallbacksT>
+static bool isFunctionPassName(StringRef Name, CallbacksT &Callbacks) {
   // Explicitly handle pass manager names.
   if (Name == "function")
     return true;
   if (Name == "loop")
     return true;
 
   // Explicitly handle custom-parsed pass names.
   if (parseRepeatPassName(Name))
     return true;
 
 #define FUNCTION_PASS(NAME, CREATE_PASS)                                       \
   if (Name == NAME)                                                            \
     return true;
 #define FUNCTION_ANALYSIS(NAME, CREATE_PASS)                                   \
   if (Name == "require<" NAME ">" || Name == "invalidate<" NAME ">")           \
     return true;
 #include "PassRegistry.def"
 
-  return false;
+  return callbacksAcceptPassName<FunctionPassManager>(Name, Callbacks);
 }
 
-static bool isLoopPassName(StringRef Name) {
+template <typename CallbacksT>
+static bool isLoopPassName(StringRef Name, CallbacksT &Callbacks) {
   // Explicitly handle pass manager names.
   if (Name == "loop")
     return true;
 
   // Explicitly handle custom-parsed pass names.
   if (parseRepeatPassName(Name))
     return true;
 
 #define LOOP_PASS(NAME, CREATE_PASS)                                           \
   if (Name == NAME)                                                            \
     return true;
 #define LOOP_ANALYSIS(NAME, CREATE_PASS)                                       \
   if (Name == "require<" NAME ">" || Name == "invalidate<" NAME ">")           \
     return true;
 #include "PassRegistry.def"
 
-  return false;
+  return callbacksAcceptPassName<LoopPassManager>(Name, Callbacks);
 }
 
 Optional<std::vector<PassBuilder::PipelineElement>>
 PassBuilder::parsePipelineText(StringRef Text) {
   std::vector<PipelineElement> ResultPipeline;
 
   SmallVector<std::vector<PipelineElement> *, 4> PipelineStack = {
       &ResultPipeline};
if (!InnerPipeline.empty()) {
     if (auto Count = parseRepeatPassName(Name)) {
       ModulePassManager NestedMPM(DebugLogging);
       if (!parseModulePassPipeline(NestedMPM, InnerPipeline, VerifyEachPass,
                                    DebugLogging))
         return false;
       MPM.addPass(createRepeatedPass(*Count, std::move(NestedMPM)));
       return true;
     }
+
+    for (auto &C : ModulePipelineParsingCallbacks)
+      if (C(Name, MPM, InnerPipeline))
+        return true;
+
     // Normal passes can't have pipelines.
     return false;
   }
 
   // Manually handle aliases for pre-configured pipeline fragments.
   if (startsWithDefaultPipelineAliasPrefix(Name)) {
     SmallVector<StringRef, 3> Matches;
     if (!DefaultAliasRegex.match(Name, &Matches))
       return false;
     assert(Matches.size() == 3 && "Must capture two matched strings!");
 
     OptimizationLevel L = StringSwitch<OptimizationLevel>(Matches[2])
                               .Case("O0", O0)
                               .Case("O1", O1)
                               .Case("O2", O2)
                               .Case("O3", O3)
                               .Case("Os", Os)
                               .Case("Oz", Oz);
     if (L == O0)
       // At O0 we do nothing at all!
       return true;
 
     if (Matches[1] == "default") {
       MPM.addPass(buildPerModuleDefaultPipeline(L, DebugLogging));
     } else if (Matches[1] == "thinlto-pre-link") {
       MPM.addPass(buildThinLTOPreLinkDefaultPipeline(L, DebugLogging));
#define MODULE_ANALYSIS(NAME, CREATE_PASS)                                     \
   }                                                                            \
   if (Name == "invalidate<" NAME ">") {                                        \
     MPM.addPass(InvalidateAnalysisPass<                                        \
                 std::remove_reference<decltype(CREATE_PASS)>::type>());        \
     return true;                                                               \
   }
 #include "PassRegistry.def"
 
+  for (auto &C : ModulePipelineParsingCallbacks)
+    if (C(Name, MPM, InnerPipeline))
+      return true;
   return false;
 }
 
 bool PassBuilder::parseCGSCCPass(CGSCCPassManager &CGPM,
                                  const PipelineElement &E, bool VerifyEachPass,
                                  bool DebugLogging) {
   auto &Name = E.Name;
   auto &InnerPipeline = E.InnerPipeline;
if (auto MaxRepetitions = parseDevirtPassName(Name)) {
       CGSCCPassManager NestedCGPM(DebugLogging);
       if (!parseCGSCCPassPipeline(NestedCGPM, InnerPipeline, VerifyEachPass,
                                   DebugLogging))
         return false;
       CGPM.addPass(createDevirtSCCRepeatedPass(std::move(NestedCGPM),
                                                *MaxRepetitions, DebugLogging));
       return true;
     }
+
+    for (auto &C : CGSCCPipelineParsingCallbacks)
+      if (C(Name, CGPM, InnerPipeline))
+        return true;
+
     // Normal passes can't have pipelines.
     return false;
   }
 
-  // Now expand the basic registered passes from the .inc file.
 #define CGSCC_PASS(NAME, CREATE_PASS)                                          \
   if (Name == NAME) {                                                          \
     CGPM.addPass(CREATE_PASS);                                                 \
     return true;                                                               \
   }
 #define CGSCC_ANALYSIS(NAME, CREATE_PASS)                                      \
   if (Name == "require<" NAME ">") {                                           \
     CGPM.addPass(RequireAnalysisPass<                                          \
                  std::remove_reference<decltype(CREATE_PASS)>::type,           \
                  LazyCallGraph::SCC, CGSCCAnalysisManager, LazyCallGraph &,    \
                  CGSCCUpdateResult &>());                                      \
     return true;                                                               \
   }                                                                            \
   if (Name == "invalidate<" NAME ">") {                                        \
     CGPM.addPass(InvalidateAnalysisPass<                                       \
                  std::remove_reference<decltype(CREATE_PASS)>::type>());       \
     return true;                                                               \
   }
 #include "PassRegistry.def"
 
+  for (auto &C : CGSCCPipelineParsingCallbacks)
+    if (C(Name, CGPM, InnerPipeline))
+      return true;
   return false;
 }
 
 bool PassBuilder::parseFunctionPass(FunctionPassManager &FPM,
                                     const PipelineElement &E,
                                     bool VerifyEachPass, bool DebugLogging) {
   auto &Name = E.Name;
   auto &InnerPipeline = E.InnerPipeline;
if (!InnerPipeline.empty()) {
     if (auto Count = parseRepeatPassName(Name)) {
       FunctionPassManager NestedFPM(DebugLogging);
       if (!parseFunctionPassPipeline(NestedFPM, InnerPipeline, VerifyEachPass,
                                      DebugLogging))
         return false;
       FPM.addPass(createRepeatedPass(*Count, std::move(NestedFPM)));
       return true;
     }
+
+    for (auto &C : FunctionPipelineParsingCallbacks)
+      if (C(Name, FPM, InnerPipeline))
+        return true;
+
     // Normal passes can't have pipelines.
     return false;
   }
 
-  // Now expand the basic registered passes from the .inc file.
 #define FUNCTION_PASS(NAME, CREATE_PASS)                                       \
   if (Name == NAME) {                                                          \
     FPM.addPass(CREATE_PASS);                                                  \
     return true;                                                               \
   }
 #define FUNCTION_ANALYSIS(NAME, CREATE_PASS)                                   \
   if (Name == "require<" NAME ">") {                                           \
     FPM.addPass(                                                               \
         RequireAnalysisPass<                                                   \
             std::remove_reference<decltype(CREATE_PASS)>::type, Function>());  \
     return true;                                                               \
   }                                                                            \
   if (Name == "invalidate<" NAME ">") {                                        \
     FPM.addPass(InvalidateAnalysisPass<                                        \
                 std::remove_reference<decltype(CREATE_PASS)>::type>());        \
     return true;                                                               \
   }
 #include "PassRegistry.def"
 
+  for (auto &C : FunctionPipelineParsingCallbacks)
+    if (C(Name, FPM, InnerPipeline))
+      return true;
   return false;
 }
 
 bool PassBuilder::parseLoopPass(LoopPassManager &LPM, const PipelineElement &E,
                                 bool VerifyEachPass, bool DebugLogging) {
   StringRef Name = E.Name;
   auto &InnerPipeline = E.InnerPipeline;
 
if (!InnerPipeline.empty()) {
     if (auto Count = parseRepeatPassName(Name)) {
       LoopPassManager NestedLPM(DebugLogging);
       if (!parseLoopPassPipeline(NestedLPM, InnerPipeline, VerifyEachPass,
                                  DebugLogging))
         return false;
       LPM.addPass(createRepeatedPass(*Count, std::move(NestedLPM)));
       return true;
     }
+
+    for (auto &C : LoopPipelineParsingCallbacks)
+      if (C(Name, LPM, InnerPipeline))
+        return true;
+
     // Normal passes can't have pipelines.
     return false;
   }
 
-  // Now expand the basic registered passes from the .inc file.
 #define LOOP_PASS(NAME, CREATE_PASS)                                           \
   if (Name == NAME) {                                                          \
     LPM.addPass(CREATE_PASS);                                                  \
     return true;                                                               \
   }
 #define LOOP_ANALYSIS(NAME, CREATE_PASS)                                       \
   if (Name == "require<" NAME ">") {                                           \
     LPM.addPass(RequireAnalysisPass<                                           \
                 std::remove_reference<decltype(CREATE_PASS)>::type, Loop,      \
                 LoopAnalysisManager, LoopStandardAnalysisResults &,            \
                 LPMUpdater &>());                                              \
     return true;                                                               \
   }                                                                            \
   if (Name == "invalidate<" NAME ">") {                                        \
     LPM.addPass(InvalidateAnalysisPass<                                        \
                 std::remove_reference<decltype(CREATE_PASS)>::type>());        \
     return true;                                                               \
   }
 #include "PassRegistry.def"
 
+  for (auto &C : LoopPipelineParsingCallbacks)
+    if (C(Name, LPM, InnerPipeline))
+      return true;
   return false;
 }
 
 bool PassBuilder::parseAAPassName(AAManager &AA, StringRef Name) {
 #define MODULE_ALIAS_ANALYSIS(NAME, CREATE_PASS)                               \
   if (Name == NAME) {                                                          \
     AA.registerModuleAnalysis<                                                 \
         std::remove_reference<decltype(CREATE_PASS)>::type>();                 \
     return true;                                                               \
   }
 #define FUNCTION_ALIAS_ANALYSIS(NAME, CREATE_PASS)                             \
   if (Name == NAME) {                                                          \
     AA.registerFunctionAnalysis<                                               \
         std::remove_reference<decltype(CREATE_PASS)>::type>();                 \
     return true;                                                               \
   }
 #include "PassRegistry.def"
 
+  for (auto &C : AAParsingCallbacks)
+    if (C(Name, AA))
+      return true;
   return false;
 }
 
 bool PassBuilder::parseLoopPassPipeline(LoopPassManager &LPM,
                                         ArrayRef<PipelineElement> Pipeline,
                                         bool VerifyEachPass,
                                         bool DebugLogging) {
   for (const auto &Element : Pipeline) {
for (const auto &Element : Pipeline) {
     if (!parseModulePass(MPM, Element, VerifyEachPass, DebugLogging))
       return false;
     if (VerifyEachPass)
       MPM.addPass(VerifierPass());
   }
   return true;
 }
 
-// Primary pass pipeline description parsing routine.
+// Primary pass pipeline description parsing routine for a \c ModulePassManager
 // FIXME: Should this routine accept a TargetMachine or require the caller to
 // pre-populate the analysis managers with target-specific stuff?
 bool PassBuilder::parsePassPipeline(ModulePassManager &MPM,
                                     StringRef PipelineText, bool VerifyEachPass,
                                     bool DebugLogging) {
   auto Pipeline = parsePipelineText(PipelineText);
   if (!Pipeline || Pipeline->empty())
     return false;
 
   // If the first name isn't at the module layer, wrap the pipeline up
   // automatically.
   StringRef FirstName = Pipeline->front().Name;
 
-  if (!isModulePassName(FirstName)) {
-    if (isCGSCCPassName(FirstName))
+  if (!isModulePassName(FirstName, ModulePipelineParsingCallbacks)) {
+    if (isCGSCCPassName(FirstName, CGSCCPipelineParsingCallbacks)) {
       Pipeline = {{"cgscc", std::move(*Pipeline)}};
-    else if (isFunctionPassName(FirstName))
+    } else if (isFunctionPassName(FirstName,
+                                  FunctionPipelineParsingCallbacks)) {
       Pipeline = {{"function", std::move(*Pipeline)}};
-    else if (isLoopPassName(FirstName))
+    } else if (isLoopPassName(FirstName, LoopPipelineParsingCallbacks)) {
       Pipeline = {{"function", {{"loop", std::move(*Pipeline)}}}};
-    else
+    } else {
+      for (auto &C : TopLevelPipelineParsingCallbacks)
+        if (C(MPM, *Pipeline, VerifyEachPass, DebugLogging))
+          return true;
+
       // Unknown pass name!
       return false;
     }
+  }
 
   return parseModulePassPipeline(MPM, *Pipeline, VerifyEachPass, DebugLogging);
 }
 
+// Primary pass pipeline description parsing routine for a \c CGSCCPassManager
+bool PassBuilder::parsePassPipeline(CGSCCPassManager &CGPM,
+                                    StringRef PipelineText, bool VerifyEachPass,
+                                    bool DebugLogging) {
+  auto Pipeline = parsePipelineText(PipelineText);
+  if (!Pipeline || Pipeline->empty())
+    return false;
+
+  StringRef FirstName = Pipeline->front().Name;
+  if (!isCGSCCPassName(FirstName, CGSCCPipelineParsingCallbacks))
+    return false;
+
+  return parseCGSCCPassPipeline(CGPM, *Pipeline, VerifyEachPass, DebugLogging);
+}
+
+// Primary pass pipeline description parsing routine for a \c
+// FunctionPassManager
+bool PassBuilder::parsePassPipeline(FunctionPassManager &FPM,
+                                    StringRef PipelineText, bool VerifyEachPass,
+                                    bool DebugLogging) {
+  auto Pipeline = parsePipelineText(PipelineText);
+  if (!Pipeline || Pipeline->empty())
+    return false;
+
+  StringRef FirstName = Pipeline->front().Name;
+  if (!isFunctionPassName(FirstName, FunctionPipelineParsingCallbacks))
+    return false;
+
+  return parseFunctionPassPipeline(FPM, *Pipeline, VerifyEachPass,
+                                   DebugLogging);
+}
+
+// Primary pass pipeline description parsing routine for a \c LoopPassManager
+bool PassBuilder::parsePassPipeline(LoopPassManager &CGPM,
+                                    StringRef PipelineText, bool VerifyEachPass,
+                                    bool DebugLogging) {
+  auto Pipeline = parsePipelineText(PipelineText);
+  if (!Pipeline || Pipeline->empty())
+    return false;
+
+  return parseLoopPassPipeline(CGPM, *Pipeline, VerifyEachPass, DebugLogging);
+}
+
 bool PassBuilder::parseAAPipeline(AAManager &AA, StringRef PipelineText) {
   // If the pipeline just consists of the word 'default' just replace the AA
   // manager with our default one.
   if (PipelineText == "default") {
     AA = buildDefaultAAPipeline();
     return true;
   }
 

llvm/trunk/test/Other/new-pm-defaults.ll

 ; RUN:     | FileCheck %s --check-prefix=CHECK-O --check-prefix=CHECK-Os
 ; RUN: opt -disable-verify -debug-pass-manager \
 ; RUN:     -passes='default<Oz>' -S %s 2>&1 \
 ; RUN:     | FileCheck %s --check-prefix=CHECK-O --check-prefix=CHECK-Oz
 ; RUN: opt -disable-verify -debug-pass-manager \
 ; RUN:     -passes='lto-pre-link<O2>' -S %s 2>&1 \
 ; RUN:     | FileCheck %s --check-prefix=CHECK-O --check-prefix=CHECK-O2
 
+; RUN: opt -disable-verify -debug-pass-manager \
+; RUN:     -passes-ep-peephole='no-op-function' \
+; RUN:     -passes='default<O3>' -S  %s 2>&1 \
+; RUN:     | FileCheck %s --check-prefix=CHECK-O --check-prefix=CHECK-O3 \
+; RUN:     --check-prefix=CHECK-EP-PEEPHOLE
+; RUN: opt -disable-verify -debug-pass-manager \
+; RUN:     -passes-ep-late-loop-optimizations='no-op-loop' \
+; RUN:     -passes='default<O3>' -S  %s 2>&1 \
+; RUN:     | FileCheck %s --check-prefix=CHECK-O --check-prefix=CHECK-O3 \
+; RUN:     --check-prefix=CHECK-EP-LOOP-LATE
+; RUN: opt -disable-verify -debug-pass-manager \
+; RUN:     -passes-ep-loop-optimizer-end='no-op-loop' \
+; RUN:     -passes='default<O3>' -S  %s 2>&1 \
+; RUN:     | FileCheck %s --check-prefix=CHECK-O --check-prefix=CHECK-O3 \
+; RUN:     --check-prefix=CHECK-EP-LOOP-END
+; RUN: opt -disable-verify -debug-pass-manager \
+; RUN:     -passes-ep-scalar-optimizer-late='no-op-function' \
+; RUN:     -passes='default<O3>' -S  %s 2>&1 \
+; RUN:     | FileCheck %s --check-prefix=CHECK-O --check-prefix=CHECK-O3 \
+; RUN:     --check-prefix=CHECK-EP-SCALAR-LATE
+; RUN: opt -disable-verify -debug-pass-manager \
+; RUN:     -passes-ep-cgscc-optimizer-late='no-op-cgscc' \
+; RUN:     -passes='default<O3>' -S  %s 2>&1 \
+; RUN:     | FileCheck %s --check-prefix=CHECK-O --check-prefix=CHECK-O3 \
+; RUN:     --check-prefix=CHECK-EP-CGSCC-LATE
+; RUN: opt -disable-verify -debug-pass-manager \
+; RUN:     -passes-ep-vectorizer-start='no-op-function' \
+; RUN:     -passes='default<O3>' -S  %s 2>&1 \
+; RUN:     | FileCheck %s --check-prefix=CHECK-O --check-prefix=CHECK-O3 \
+; RUN:     --check-prefix=CHECK-EP-VECTORIZER-START
+
 ; CHECK-O: Starting llvm::Module pass manager run.
 ; CHECK-O-NEXT: Running pass: PassManager<{{.*}}Module{{.*}}>
 ; CHECK-O-NEXT: Starting llvm::Module pass manager run.
 ; CHECK-O-NEXT: Running pass: ForceFunctionAttrsPass
 ; CHECK-O-NEXT: Running pass: PassManager<{{.*}}Module{{.*}}>
 ; CHECK-O-NEXT: Starting llvm::Module pass manager run.
 ; CHECK-O-NEXT: Running pass: InferFunctionAttrsPass
 ; CHECK-O-NEXT: Running analysis: TargetLibraryAnalysis

 ; CHECK-O-NEXT: Finished llvm::Function pass manager run.
 ; CHECK-O-NEXT: Running pass: IPSCCPPass
 ; CHECK-O-NEXT: Running pass: GlobalOptPass
 ; CHECK-O-NEXT: Running pass: ModuleToFunctionPassAdaptor<{{.*}}PromotePass>
 ; CHECK-O-NEXT: Running pass: DeadArgumentEliminationPass
 ; CHECK-O-NEXT: Running pass: ModuleToFunctionPassAdaptor<{{.*}}PassManager{{.*}}>
 ; CHECK-O-NEXT: Starting llvm::Function pass manager run.
 ; CHECK-O-NEXT: Running pass: InstCombinePass
+; CHECK-EP-PEEPHOLE-NEXT: Running pass: NoOpFunctionPass
 ; CHECK-O-NEXT: Running pass: SimplifyCFGPass
 ; CHECK-O-NEXT: Finished llvm::Function pass manager run.
 ; CHECK-O-NEXT: Running pass: RequireAnalysisPass<{{.*}}GlobalsAA
 ; CHECK-O-NEXT: Running analysis: GlobalsAA
 ; CHECK-O-NEXT: Running analysis: CallGraphAnalysis
 ; CHECK-O-NEXT: Running pass: RequireAnalysisPass<{{.*}}ProfileSummaryAnalysis
 ; CHECK-O-NEXT: Running analysis: ProfileSummaryAnalysis
 ; CHECK-O-NEXT: Running pass: ModuleToPostOrderCGSCCPassAdaptor<{{.*}}LazyCallGraph{{.*}}>

 ; CHECK-O-NEXT: Running pass: JumpThreadingPass
 ; CHECK-O-NEXT: Running analysis: LazyValueAnalysis
 ; CHECK-O-NEXT: Running pass: CorrelatedValuePropagationPass
 ; CHECK-O-NEXT: Running pass: SimplifyCFGPass
 ; CHECK-O-NEXT: Running pass: InstCombinePass
 ; CHECK-O1-NEXT: Running pass: LibCallsShrinkWrapPass
 ; CHECK-O2-NEXT: Running pass: LibCallsShrinkWrapPass
 ; CHECK-O3-NEXT: Running pass: LibCallsShrinkWrapPass
+; CHECK-EP-PEEPHOLE-NEXT: Running pass: NoOpFunctionPass
 ; CHECK-O-NEXT: Running pass: TailCallElimPass
 ; CHECK-O-NEXT: Running pass: SimplifyCFGPass
 ; CHECK-O-NEXT: Running pass: ReassociatePass
 ; CHECK-O-NEXT: Running pass: RequireAnalysisPass<{{.*}}OptimizationRemarkEmitterAnalysis
 ; CHECK-O-NEXT: Running analysis: OptimizationRemarkEmitterAnalysis
 ; CHECK-O-NEXT: Running pass: FunctionToLoopPassAdaptor<{{.*}}LoopStandardAnalysisResults{{.*}}>
 ; CHECK-O-NEXT: Running analysis: LoopAnalysis
 ; CHECK-O-NEXT: Running analysis: ScalarEvolutionAnalysis
 ; CHECK-O-NEXT: Running analysis: InnerAnalysisManagerProxy
 ; CHECK-O-NEXT: Starting Loop pass manager run.
 ; CHECK-O-NEXT: Running pass: LoopRotatePass
 ; CHECK-O-NEXT: Running pass: LICM
 ; CHECK-O-NEXT: Running analysis: OuterAnalysisManagerProxy
 ; CHECK-O-NEXT: Running pass: SimpleLoopUnswitchPass
 ; CHECK-O-NEXT: Finished Loop pass manager run.
 ; CHECK-O-NEXT: Running pass: SimplifyCFGPass
 ; CHECK-O-NEXT: Running pass: InstCombinePass
 ; CHECK-O-NEXT: Running pass: FunctionToLoopPassAdaptor<{{.*}}LoopStandardAnalysisResults{{.*}}>
 ; CHECK-O-NEXT: Starting Loop pass manager run.
 ; CHECK-O-NEXT: Running pass: IndVarSimplifyPass
 ; CHECK-O-NEXT: Running pass: LoopIdiomRecognizePass
+; CHECK-EP-LOOP-LATE-NEXT: Running pass: NoOpLoopPass
 ; CHECK-O-NEXT: Running pass: LoopDeletionPass
 ; CHECK-O-NEXT: Running pass: LoopUnrollPass
+; CHECK-EP-LOOP-END-NEXT: Running pass: NoOpLoopPass
 ; CHECK-O-NEXT: Finished Loop pass manager run.
 ; CHECK-Os-NEXT: Running pass: MergedLoadStoreMotionPass
 ; CHECK-Os-NEXT: Running pass: GVN
 ; CHECK-Os-NEXT: Running analysis: MemoryDependenceAnalysis
 ; CHECK-Oz-NEXT: Running pass: MergedLoadStoreMotionPass
 ; CHECK-Oz-NEXT: Running pass: GVN
 ; CHECK-Oz-NEXT: Running analysis: MemoryDependenceAnalysis
 ; CHECK-O2-NEXT: Running pass: MergedLoadStoreMotionPass
 ; CHECK-O2-NEXT: Running pass: GVN
 ; CHECK-O2-NEXT: Running analysis: MemoryDependenceAnalysis
 ; CHECK-O3-NEXT: Running pass: MergedLoadStoreMotionPass
 ; CHECK-O3-NEXT: Running pass: GVN
 ; CHECK-O3-NEXT: Running analysis: MemoryDependenceAnalysis
 ; CHECK-O-NEXT: Running pass: MemCpyOptPass
 ; CHECK-O1-NEXT: Running analysis: MemoryDependenceAnalysis
 ; CHECK-O-NEXT: Running pass: SCCPPass
 ; CHECK-O-NEXT: Running pass: BDCEPass
 ; CHECK-O-NEXT: Running analysis: DemandedBitsAnalysis
 ; CHECK-O-NEXT: Running pass: InstCombinePass
+; CHECK-EP-PEEPHOLE-NEXT: Running pass: NoOpFunctionPass
 ; CHECK-O-NEXT: Running pass: JumpThreadingPass
 ; CHECK-O-NEXT: Running pass: CorrelatedValuePropagationPass
 ; CHECK-O-NEXT: Running pass: DSEPass
 ; CHECK-O-NEXT: Running pass: FunctionToLoopPassAdaptor<{{.*}}LICMPass{{.*}}>
+; CHECK-EP-SCALAR-LATE-NEXT: Running pass: NoOpFunctionPass
 ; CHECK-O-NEXT: Running pass: ADCEPass
 ; CHECK-O-NEXT: Running analysis: PostDominatorTreeAnalysis
 ; CHECK-O-NEXT: Running pass: SimplifyCFGPass
 ; CHECK-O-NEXT: Running pass: InstCombinePass
+; CHECK-EP-PEEPHOLE-NEXT: Running pass: NoOpFunctionPass
 ; CHECK-O-NEXT: Finished llvm::Function pass manager run.
+; CHECK-EP-CGSCC-LATE-NEXT: Running pass: NoOpCGSCCPass
 ; CHECK-O-NEXT: Finished CGSCC pass manager run.
 ; CHECK-O-NEXT: Finished llvm::Module pass manager run.
 ; CHECK-O-NEXT: Running pass: PassManager<{{.*}}Module{{.*}}>
 ; CHECK-O-NEXT: Starting llvm::Module pass manager run.
 ; CHECK-O-NEXT: Running pass: GlobalOptPass
 ; CHECK-O-NEXT: Running pass: EliminateAvailableExternallyPass
 ; CHECK-O-NEXT: Running pass: ReversePostOrderFunctionAttrsPass
 ; CHECK-O-NEXT: Running pass: RequireAnalysisPass<{{.*}}GlobalsAA
 ; CHECK-O-NEXT: Running pass: ModuleToFunctionPassAdaptor<{{.*}}PassManager{{.*}}>
 ; CHECK-O-NEXT: Starting llvm::Function pass manager run.
 ; CHECK-O-NEXT: Running pass: Float2IntPass
+; CHECK-EP-VECTORIZER-START-NEXT: Running pass: NoOpFunctionPass
 ; CHECK-O-NEXT: Running pass: FunctionToLoopPassAdaptor<{{.*}}LoopRotatePass
 ; CHECK-O-NEXT: Running pass: LoopDistributePass
 ; CHECK-O-NEXT: Running pass: LoopVectorizePass
 ; CHECK-O-NEXT: Running analysis: BlockFrequencyAnalysis
 ; CHECK-O-NEXT: Running analysis: BranchProbabilityAnalysis
 ; CHECK-O-NEXT: Running pass: LoopLoadEliminationPass
 ; CHECK-O-NEXT: Running analysis: LoopAccessAnalysis
 ; CHECK-O-NEXT: Running pass: InstCombinePass

llvm/trunk/test/Other/new-pm-lto-defaults.ll

 ; RUN:     -passes='lto<O3>' -S  %s 2>&1 \
 ; RUN:     | FileCheck %s --check-prefix=CHECK-O --check-prefix=CHECK-O2
 ; RUN: opt -disable-verify -debug-pass-manager \
 ; RUN:     -passes='lto<Os>' -S %s 2>&1 \
 ; RUN:     | FileCheck %s --check-prefix=CHECK-O --check-prefix=CHECK-O2
 ; RUN: opt -disable-verify -debug-pass-manager \
 ; RUN:     -passes='lto<Oz>' -S %s 2>&1 \
 ; RUN:     | FileCheck %s --check-prefix=CHECK-O --check-prefix=CHECK-O2
+; RUN: opt -disable-verify -debug-pass-manager \
+; RUN:     -passes='lto<O3>' -S  %s -passes-ep-peephole='no-op-function' 2>&1 \
+; RUN:     | FileCheck %s --check-prefix=CHECK-O --check-prefix=CHECK-O2 \
+; RUN:     --check-prefix=CHECK-EP-Peephole
 
 ; CHECK-O: Starting llvm::Module pass manager run.
 ; CHECK-O-NEXT: Running pass: PassManager<{{.*}}Module
 ; CHECK-O-NEXT: Starting llvm::Module pass manager run.
 ; CHECK-O-NEXT: Running pass: GlobalDCEPass
 ; CHECK-O-NEXT: Running pass: ForceFunctionAttrsPass
 ; CHECK-O-NEXT: Running pass: InferFunctionAttrsPass
 ; CHECK-O-NEXT: Running analysis: TargetLibraryAnalysis

 ; CHECK-O-NEXT: Running pass: GlobalSplitPass
 ; CHECK-O-NEXT: Running pass: WholeProgramDevirtPass
 ; CHECK-O2-NEXT: Running pass: GlobalOptPass
 ; CHECK-O2-NEXT: Running pass: ModuleToFunctionPassAdaptor<{{.*}}PromotePass>
 ; CHECK-O2-NEXT: Running analysis: DominatorTreeAnalysis
 ; CHECK-O2-NEXT: Running analysis: AssumptionAnalysis
 ; CHECK-O2-NEXT: Running pass: ConstantMergePass
 ; CHECK-O2-NEXT: Running pass: DeadArgumentEliminationPass
-; CHECK-O2-NEXT: Running pass: ModuleToFunctionPassAdaptor<{{.*}}InstCombinePass>
+; CHECK-O2-NEXT: Running pass: ModuleToFunctionPassAdaptor<{{.*}}PassManager{{.*}}>
+; CHECK-O2-NEXT: Starting llvm::Function pass manager run.
+; CHECK-O2-NEXT: Running pass: InstCombinePass
+; CHECK-EP-Peephole-NEXT: Running pass: NoOpFunctionPass
+; CHECK-O2-NEXT: Finished llvm::Function pass manager run.
 ; CHECK-O2-NEXT: Running pass: ModuleToPostOrderCGSCCPassAdaptor<{{.*}}InlinerPass>
 ; CHECK-O2-NEXT: Running pass: GlobalOptPass
 ; CHECK-O2-NEXT: Running pass: GlobalDCEPass
 ; CHECK-O2-NEXT: Running pass: ModuleToFunctionPassAdaptor<{{.*}}PassManager{{.*}}>
 ; CHECK-O2-NEXT: Starting llvm::Function pass manager run.
 ; CHECK-O2-NEXT: Running pass: InstCombinePass
+; CHECK-EP-Peephole-NEXT: Running pass: NoOpFunctionPass
 ; CHECK-O2-NEXT: Running pass: JumpThreadingPass
 ; CHECK-O2-NEXT: Running analysis: LazyValueAnalysis
 ; CHECK-O2-NEXT: Running pass: SROA on foo
 ; CHECK-O2-NEXT: Finished llvm::Function pass manager run.
 ; CHECK-O2-NEXT: Running pass: ModuleToPostOrderCGSCCPassAdaptor<{{.*}}PostOrderFunctionAttrsPass>
 ; CHECK-O2-NEXT: Running pass: ModuleToFunctionPassAdaptor<{{.*}}PassManager{{.*}}>
 ; CHECK-O2-NEXT: Running analysis: MemoryDependenceAnalysis
 ; CHECK-O2-NEXT: Running analysis: OptimizationRemarkEmitterAnalysis

llvm/trunk/tools/opt/NewPMDriver.cpp

 // use when querying for aliasing information. It only works in concert with
 // the "passes" flag above.
 static cl::opt<std::string>
     AAPipeline("aa-pipeline",
                cl::desc("A textual description of the alias analysis "
                         "pipeline for handling managed aliasing queries"),
                cl::Hidden);
 
+/// {{@ These options accept textual pipeline descriptions which will be
+/// inserted into default pipelines at the respective extension points
+static cl::opt<std::string> PeepholeEPPipeline(
+    "passes-ep-peephole",
+    cl::desc("A textual description of the function pass pipeline inserted at "
+             "the Peephole extension points into default pipelines"),
+    cl::Hidden);
+static cl::opt<std::string> LateLoopOptimizationsEPPipeline(
+    "passes-ep-late-loop-optimizations",
+    cl::desc(
+        "A textual description of the loop pass pipeline inserted at "
+        "the LateLoopOptimizations extension point into default pipelines"),
+    cl::Hidden);
+static cl::opt<std::string> LoopOptimizerEndEPPipeline(
+    "passes-ep-loop-optimizer-end",
+    cl::desc("A textual description of the loop pass pipeline inserted at "
+             "the LoopOptimizerEnd extension point into default pipelines"),
+    cl::Hidden);
+static cl::opt<std::string> ScalarOptimizerLateEPPipeline(
+    "passes-ep-scalar-optimizer-late",
+    cl::desc("A textual description of the function pass pipeline inserted at "
+             "the ScalarOptimizerLate extension point into default pipelines"),
+    cl::Hidden);
+static cl::opt<std::string> CGSCCOptimizerLateEPPipeline(
+    "passes-ep-cgscc-optimizer-late",
+    cl::desc("A textual description of the cgscc pass pipeline inserted at "
+             "the CGSCCOptimizerLate extension point into default pipelines"),
+    cl::Hidden);
+static cl::opt<std::string> VectorizerStartEPPipeline(
+    "passes-ep-vectorizer-start",
+    cl::desc("A textual description of the function pass pipeline inserted at "
+             "the VectorizerStart extension point into default pipelines"),
+    cl::Hidden);
+/// @}}
+
+/// If one of the EPPipeline command line options was given, register callbacks
+/// for parsing and inserting the given pipeline
+static void registerEPCallbacks(PassBuilder &PB, bool VerifyEachPass,
+                                bool DebugLogging) {
+  if (!PeepholeEPPipeline.empty())
+    PB.registerPeepholeEPCallback(
+        [&](FunctionPassManager &PM, PassBuilder::OptimizationLevel Level) {
+          return PB.parsePassPipeline(PM, PeepholeEPPipeline, VerifyEachPass,
+                                      DebugPM);
+        });
+  if (!LateLoopOptimizationsEPPipeline.empty())
+    PB.registerLateLoopOptimizationsEPCallback(
+        [&](LoopPassManager &PM, PassBuilder::OptimizationLevel Level) {
+          return PB.parsePassPipeline(PM, LateLoopOptimizationsEPPipeline,
+                                      VerifyEachPass, DebugPM);
+        });
+  if (!LoopOptimizerEndEPPipeline.empty())
+    PB.registerLoopOptimizerEndEPCallback(
+        [&](LoopPassManager &PM, PassBuilder::OptimizationLevel Level) {
+          return PB.parsePassPipeline(PM, LoopOptimizerEndEPPipeline,
+                                      VerifyEachPass, DebugPM);
+        });
+  if (!ScalarOptimizerLateEPPipeline.empty())
+    PB.registerScalarOptimizerLateEPCallback(
+        [&](FunctionPassManager &PM, PassBuilder::OptimizationLevel Level) {
+          return PB.parsePassPipeline(PM, ScalarOptimizerLateEPPipeline,
+                                      VerifyEachPass, DebugPM);
+        });
+  if (!CGSCCOptimizerLateEPPipeline.empty())
+    PB.registerCGSCCOptimizerLateEPCallback(
+        [&](CGSCCPassManager &PM, PassBuilder::OptimizationLevel Level) {
+          return PB.parsePassPipeline(PM, CGSCCOptimizerLateEPPipeline,
+                                      VerifyEachPass, DebugPM);
+        });
+  if (!VectorizerStartEPPipeline.empty())
+    PB.registerVectorizerStartEPCallback(
+        [&](FunctionPassManager &PM, PassBuilder::OptimizationLevel Level) {
+          return PB.parsePassPipeline(PM, VectorizerStartEPPipeline,
+                                      VerifyEachPass, DebugPM);
+        });
+}
+
 bool llvm::runPassPipeline(StringRef Arg0, Module &M, TargetMachine *TM,
                            tool_output_file *Out,
                            tool_output_file *ThinLTOLinkOut,
                            StringRef PassPipeline, OutputKind OK,
                            VerifierKind VK,
                            bool ShouldPreserveAssemblyUseListOrder,
                            bool ShouldPreserveBitcodeUseListOrder,
                            bool EmitSummaryIndex, bool EmitModuleHash) {
+  bool VerifyEachPass = VK == VK_VerifyEachPass;
   PassBuilder PB(TM);
+  registerEPCallbacks(PB, VerifyEachPass, DebugPM);
 
   // Specially handle the alias analysis manager so that we can register
   // a custom pipeline of AA passes with it.
   AAManager AA;
   if (!PB.parseAAPipeline(AA, AAPipeline)) {
     errs() << Arg0 << ": unable to parse AA pipeline description.\n";
     return false;
   }
bool llvm::runPassPipeline(StringRef Arg0, Module &M, TargetMachine *TM,
   PB.registerFunctionAnalyses(FAM);
   PB.registerLoopAnalyses(LAM);
   PB.crossRegisterProxies(LAM, FAM, CGAM, MAM);
 
   ModulePassManager MPM(DebugPM);
   if (VK > VK_NoVerifier)
     MPM.addPass(VerifierPass());
 
-  if (!PB.parsePassPipeline(MPM, PassPipeline, VK == VK_VerifyEachPass,
-                            DebugPM)) {
+  if (!PB.parsePassPipeline(MPM, PassPipeline, VerifyEachPass, DebugPM)) {
     errs() << Arg0 << ": unable to parse pass pipeline description.\n";
     return false;
   }
 
   if (VK > VK_NoVerifier)
     MPM.addPass(VerifierPass());
 
   // Add any relevant output pass at the end of the pipeline.

llvm/trunk/unittests/IR/CMakeLists.txt

 set(LLVM_LINK_COMPONENTS
   Analysis
   AsmParser
   Core
   Support
+  Passes
   )
 
 set(IRSources
   AsmWriterTest.cpp
   AttributesTest.cpp
   BasicBlockTest.cpp
   ConstantRangeTest.cpp
   ConstantsTest.cpp
   DebugInfoTest.cpp
   DebugTypeODRUniquingTest.cpp
   DominatorTreeTest.cpp
   FunctionTest.cpp
+  PassBuilderCallbacksTest.cpp
   IRBuilderTest.cpp
   InstructionsTest.cpp
   IntrinsicsTest.cpp
   LegacyPassManagerTest.cpp
   MDBuilderTest.cpp
   MetadataTest.cpp
   ModuleTest.cpp
   PassManagerTest.cpp

llvm/trunk/unittests/IR/PassBuilderCallbacksTest.cpp

+//===- unittests/IR/PassBuilderCallbacksTest.cpp - PB Callback Tests --===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include <gmock/gmock.h>
+#include <gtest/gtest.h>
+#include <llvm/Analysis/CGSCCPassManager.h>
+#include <llvm/Analysis/LoopAnalysisManager.h>
+#include <llvm/AsmParser/Parser.h>
+#include <llvm/IR/LLVMContext.h>
+#include <llvm/IR/PassManager.h>
+#include <llvm/Passes/PassBuilder.h>
+#include <llvm/Support/SourceMgr.h>
+#include <llvm/Transforms/Scalar/LoopPassManager.h>
+
+using namespace llvm;
+
+namespace llvm {
+/// Provide an ostream operator for StringRef.
+///
+/// For convenience we provide a custom matcher below for IRUnit's and analysis
+/// result's getName functions, which most of the time returns a StringRef. The
+/// matcher makes use of this operator.
+static std::ostream &operator<<(std::ostream &O, StringRef S) {
+  return O << S.str();
+}
+}
+
+namespace {
+using testing::DoDefault;
+using testing::Return;
+using testing::Expectation;
+using testing::Invoke;
+using testing::WithArgs;
+using testing::_;
+
+/// \brief A CRTP base for analysis mock handles
+///
+/// This class reconciles mocking with the value semantics implementation of the
+/// AnalysisManager. Analysis mock handles should derive from this class and
+/// call \c setDefault() in their constroctur for wiring up the defaults defined
+/// by this base with their mock run() and invalidate() implementations.
+template <typename DerivedT, typename IRUnitT,
+          typename AnalysisManagerT = AnalysisManager<IRUnitT>,
+          typename... ExtraArgTs>
+class MockAnalysisHandleBase {
+public:
+  class Analysis : public AnalysisInfoMixin<Analysis> {
+    friend AnalysisInfoMixin<Analysis>;
+    friend MockAnalysisHandleBase;
+    static AnalysisKey Key;
+
+    DerivedT *Handle;
+
+    Analysis(DerivedT &Handle) : Handle(&Handle) {
+      static_assert(std::is_base_of<MockAnalysisHandleBase, DerivedT>::value,
+                    "Must pass the derived type to this template!");
+    }
+
+  public:
+    class Result {
+      friend MockAnalysisHandleBase;
+
+      DerivedT *Handle;
+
+      Result(DerivedT &Handle) : Handle(&Handle) {}
+
+    public:
+      // Forward invalidation events to the mock handle.
+      bool invalidate(IRUnitT &IR, const PreservedAnalyses &PA,
+                      typename AnalysisManagerT::Invalidator &Inv) {
+        return Handle->invalidate(IR, PA, Inv);
+      }
+    };
+
+    Result run(IRUnitT &IR, AnalysisManagerT &AM, ExtraArgTs... ExtraArgs) {
+      return Handle->run(IR, AM, ExtraArgs...);
+    }
+  };
+
+  Analysis getAnalysis() { return Analysis(static_cast<DerivedT &>(*this)); }
+  typename Analysis::Result getResult() {
+    return typename Analysis::Result(static_cast<DerivedT &>(*this));
+  }
+
+protected:
+  // FIXME: MSVC seems unable to handle a lambda argument to Invoke from within
+  // the template, so we use a boring static function.
+  static bool invalidateCallback(IRUnitT &IR, const PreservedAnalyses &PA,
+                                 typename AnalysisManagerT::Invalidator &Inv) {
+    auto PAC = PA.template getChecker<Analysis>();
+    return !PAC.preserved() &&
+           !PAC.template preservedSet<AllAnalysesOn<IRUnitT>>();
+  }
+
+  /// Derived classes should call this in their constructor to set up default
+  /// mock actions. (We can't do this in our constructor because this has to
+  /// run after the DerivedT is constructed.)
+  void setDefaults() {
+    ON_CALL(static_cast<DerivedT &>(*this),
+            run(_, _, testing::Matcher<ExtraArgTs>(_)...))
+        .WillByDefault(Return(this->getResult()));
+    ON_CALL(static_cast<DerivedT &>(*this), invalidate(_, _, _))
+        .WillByDefault(Invoke(&invalidateCallback));
+  }
+};
+
+/// \brief A CRTP base for pass mock handles
+///
+/// This class reconciles mocking with the value semantics implementation of the
+/// PassManager. Pass mock handles should derive from this class and
+/// call \c setDefault() in their constroctur for wiring up the defaults defined
+/// by this base with their mock run() and invalidate() implementations.
+template <typename DerivedT, typename IRUnitT, typename AnalysisManagerT,
+          typename... ExtraArgTs>
+AnalysisKey MockAnalysisHandleBase<DerivedT, IRUnitT, AnalysisManagerT,
+                                   ExtraArgTs...>::Analysis::Key;
+
+template <typename DerivedT, typename IRUnitT,
+          typename AnalysisManagerT = AnalysisManager<IRUnitT>,
+          typename... ExtraArgTs>
+class MockPassHandleBase {
+public:
+  class Pass : public PassInfoMixin<Pass> {
+    friend MockPassHandleBase;
+
+    DerivedT *Handle;
+
+    Pass(DerivedT &Handle) : Handle(&Handle) {
+      static_assert(std::is_base_of<MockPassHandleBase, DerivedT>::value,
+                    "Must pass the derived type to this template!");
+    }
+
+  public:
+    PreservedAnalyses run(IRUnitT &IR, AnalysisManagerT &AM,
+                          ExtraArgTs... ExtraArgs) {
+      return Handle->run(IR, AM, ExtraArgs...);
+    }
+  };
+
+  Pass getPass() { return Pass(static_cast<DerivedT &>(*this)); }
+
+protected:
+  /// Derived classes should call this in their constructor to set up default
+  /// mock actions. (We can't do this in our constructor because this has to
+  /// run after the DerivedT is constructed.)
+  void setDefaults() {
+    ON_CALL(static_cast<DerivedT &>(*this),
+            run(_, _, testing::Matcher<ExtraArgTs>(_)...))
+        .WillByDefault(Return(PreservedAnalyses::all()));
+  }
+};
+
+/// Mock handles for passes for the IRUnits Module, CGSCC, Function, Loop.
+/// These handles define the appropriate run() mock interface for the respective
+/// IRUnit type.
+template <typename IRUnitT> struct MockPassHandle;
+template <>
+struct MockPassHandle<Loop>
+    : MockPassHandleBase<MockPassHandle<Loop>, Loop, LoopAnalysisManager,
+                         LoopStandardAnalysisResults &, LPMUpdater &> {
+  MOCK_METHOD4(run,
+               PreservedAnalyses(Loop &, LoopAnalysisManager &,
+                                 LoopStandardAnalysisResults &, LPMUpdater &));
+  MockPassHandle() { setDefaults(); }
+};
+
+template <>
+struct MockPassHandle<Function>
+    : MockPassHandleBase<MockPassHandle<Function>, Function> {
+  MOCK_METHOD2(run, PreservedAnalyses(Function &, FunctionAnalysisManager &));
+
+  MockPassHandle() { setDefaults(); }
+};
+
+template <>
+struct MockPassHandle<LazyCallGraph::SCC>
+    : MockPassHandleBase<MockPassHandle<LazyCallGraph::SCC>, LazyCallGraph::SCC,
+                         CGSCCAnalysisManager, LazyCallGraph &,
+                         CGSCCUpdateResult &> {
+  MOCK_METHOD4(run,
+               PreservedAnalyses(LazyCallGraph::SCC &, CGSCCAnalysisManager &,
+                                 LazyCallGraph &G, CGSCCUpdateResult &UR));
+
+  MockPassHandle() { setDefaults(); }
+};
+
+template <>
+struct MockPassHandle<Module>
+    : MockPassHandleBase<MockPassHandle<Module>, Module> {
+  MOCK_METHOD2(run, PreservedAnalyses(Module &, ModuleAnalysisManager &));
+
+  MockPassHandle() { setDefaults(); }
+};
+
+/// Mock handles for analyses for the IRUnits Module, CGSCC, Function, Loop.
+/// These handles define the appropriate run() and invalidate() mock interfaces
+/// for the respective IRUnit type.
+template <typename IRUnitT> struct MockAnalysisHandle;
+template <>
+struct MockAnalysisHandle<Loop>
+    : MockAnalysisHandleBase<MockAnalysisHandle<Loop>, Loop,
+                             LoopAnalysisManager,
+                             LoopStandardAnalysisResults &> {
+
+  MOCK_METHOD3_T(run, typename Analysis::Result(Loop &, LoopAnalysisManager &,
+                                                LoopStandardAnalysisResults &));
+
+  MOCK_METHOD3_T(invalidate, bool(Loop &, const PreservedAnalyses &,
+                                  LoopAnalysisManager::Invalidator &));
+
+  MockAnalysisHandle<Loop>() { this->setDefaults(); }
+};
+
+template <>
+struct MockAnalysisHandle<Function>
+    : MockAnalysisHandleBase<MockAnalysisHandle<Function>, Function> {
+  MOCK_METHOD2(run, Analysis::Result(Function &, FunctionAnalysisManager &));
+
+  MOCK_METHOD3(invalidate, bool(Function &, const PreservedAnalyses &,
+                                FunctionAnalysisManager::Invalidator &));
+
+  MockAnalysisHandle<Function>() { setDefaults(); }
+};
+
+template <>
+struct MockAnalysisHandle<LazyCallGraph::SCC>
+    : MockAnalysisHandleBase<MockAnalysisHandle<LazyCallGraph::SCC>,
+                             LazyCallGraph::SCC, CGSCCAnalysisManager,
+                             LazyCallGraph &> {
+  MOCK_METHOD3(run, Analysis::Result(LazyCallGraph::SCC &,
+                                     CGSCCAnalysisManager &, LazyCallGraph &));
+
+  MOCK_METHOD3(invalidate, bool(LazyCallGraph::SCC &, const PreservedAnalyses &,
+                                CGSCCAnalysisManager::Invalidator &));
+
+  MockAnalysisHandle<LazyCallGraph::SCC>() { setDefaults(); }
+};
+
+template <>
+struct MockAnalysisHandle<Module>
+    : MockAnalysisHandleBase<MockAnalysisHandle<Module>, Module> {
+  MOCK_METHOD2(run, Analysis::Result(Module &, ModuleAnalysisManager &));
+
+  MOCK_METHOD3(invalidate, bool(Module &, const PreservedAnalyses &,
+                                ModuleAnalysisManager::Invalidator &));
+
+  MockAnalysisHandle<Module>() { setDefaults(); }
+};
+
+static std::unique_ptr<Module> parseIR(LLVMContext &C, const char *IR) {
+  SMDiagnostic Err;
+  return parseAssemblyString(IR, Err, C);
+}
+
+template <typename PassManagerT> class PassBuilderCallbacksTest;
+
+/// This test fixture is shared between all the actual tests below and
+/// takes care of setting up appropriate defaults.
+///
+/// The template specialization serves to extract the IRUnit and AM types from
+/// the given PassManagerT.
+template <typename TestIRUnitT, typename... ExtraPassArgTs,
+          typename... ExtraAnalysisArgTs>
+class PassBuilderCallbacksTest<PassManager<
+    TestIRUnitT, AnalysisManager<TestIRUnitT, ExtraAnalysisArgTs...>,
+    ExtraPassArgTs...>> : public testing::Test {
+protected:
+  using IRUnitT = TestIRUnitT;
+  using AnalysisManagerT = AnalysisManager<TestIRUnitT, ExtraAnalysisArgTs...>;
+  using PassManagerT =
+      PassManager<TestIRUnitT, AnalysisManagerT, ExtraPassArgTs...>;
+  using AnalysisT = typename MockAnalysisHandle<IRUnitT>::Analysis;
+
+  LLVMContext Context;
+  std::unique_ptr<Module> M;
+
+  PassBuilder PB;
+  ModulePassManager PM;
+  LoopAnalysisManager LAM;
+  FunctionAnalysisManager FAM;
+  CGSCCAnalysisManager CGAM;
+  ModuleAnalysisManager AM;
+
+  MockPassHandle<IRUnitT> PassHandle;
+  MockAnalysisHandle<IRUnitT> AnalysisHandle;
+
+  static PreservedAnalyses getAnalysisResult(IRUnitT &U, AnalysisManagerT &AM,
+                                             ExtraAnalysisArgTs &&... Args) {
+    (void)AM.template getResult<AnalysisT>(
+        U, std::forward<ExtraAnalysisArgTs>(Args)...);
+    return PreservedAnalyses::all();
+  }
+
+  PassBuilderCallbacksTest()
+      : M(parseIR(Context,
+                  "declare void @bar()\n"
+                  "define void @foo(i32 %n) {\n"
+                  "entry:\n"
+                  "  br label %loop\n"
+                  "loop:\n"
+                  "  %iv = phi i32 [ 0, %entry ], [ %iv.next, %loop ]\n"
+                  "  %iv.next = add i32 %iv, 1\n"
+                  "  tail call void @bar()\n"
+                  "  %cmp = icmp eq i32 %iv, %n\n"
+                  "  br i1 %cmp, label %exit, label %loop\n"
+                  "exit:\n"
+                  "  ret void\n"
+                  "}\n")),
+        PM(true), LAM(true), FAM(true), CGAM(true), AM(true) {
+
+    /// Register a callback for analysis registration.
+    ///
+    /// The callback is a function taking a reference to an AnalyisManager
+    /// object. When called, the callee gets to register its own analyses with
+    /// this PassBuilder instance.
+    PB.registerAnalysisRegistrationCallback([this](AnalysisManagerT &AM) {
+      // Register our mock analysis
+      AM.registerPass([this] { return AnalysisHandle.getAnalysis(); });
+    });
+
+    /// Register a callback for pipeline parsing.
+    ///
+    /// During parsing of a textual pipeline, the PassBuilder will call these
+    /// callbacks for each encountered pass name that it does not know. This
+    /// includes both simple pass names as well as names of sub-pipelines. In
+    /// the latter case, the InnerPipeline is not empty.
+    PB.registerPipelineParsingCallback(
+        [this](StringRef Name, PassManagerT &PM,
+               ArrayRef<PassBuilder::PipelineElement> InnerPipeline) {
+          /// Handle parsing of the names of analysis utilities such as
+          /// require<test-analysis> and invalidate<test-analysis> for our
+          /// analysis mock handle
+          if (parseAnalysisUtilityPasses<AnalysisT>("test-analysis", Name, PM))
+            return true;
+
+          /// Parse the name of our pass mock handle
+          if (Name == "test-transform") {
+            PM.addPass(PassHandle.getPass());
+            return true;
+          }
+          return false;
+        });
+
+    /// Register builtin analyses and cross-register the analysis proxies
+    PB.registerModuleAnalyses(AM);
+    PB.registerCGSCCAnalyses(CGAM);
+    PB.registerFunctionAnalyses(FAM);
+    PB.registerLoopAnalyses(LAM);
+    PB.crossRegisterProxies(LAM, FAM, CGAM, AM);
+  }
+};
+
+/// Define a custom matcher for objects which support a 'getName' method.
+///
+/// LLVM often has IR objects or analysis objects which expose a name
+/// and in tests it is convenient to match these by name for readability.
+/// Usually, this name is either a StringRef or a plain std::string. This
+/// matcher supports any type exposing a getName() method of this form whose
+/// return value is compatible with an std::ostream. For StringRef, this uses
+/// the shift operator defined above.
+///
+/// It should be used as:
+///
+///   HasName("my_function")
+///
+/// No namespace or other qualification is required.
+MATCHER_P(HasName, Name, "") {
+  *result_listener << "has name '" << arg.getName() << "'";
+  return Name == arg.getName();
+}
+
+using ModuleCallbacksTest = PassBuilderCallbacksTest<ModulePassManager>;
+using CGSCCCallbacksTest = PassBuilderCallbacksTest<CGSCCPassManager>;
+using FunctionCallbacksTest = PassBuilderCallbacksTest<FunctionPassManager>;
+using LoopCallbacksTest = PassBuilderCallbacksTest<LoopPassManager>;
+
+/// Test parsing of the name of our mock pass for all IRUnits.
+///
+/// The pass should by default run our mock analysis and then preserve it.
+TEST_F(ModuleCallbacksTest, Passes) {
+  EXPECT_CALL(AnalysisHandle, run(HasName("<string>"), _));
+  EXPECT_CALL(PassHandle, run(HasName("<string>"), _))
+      .WillOnce(Invoke(getAnalysisResult));
+
+  StringRef PipelineText = "test-transform";
+  ASSERT_TRUE(PB.parsePassPipeline(PM, PipelineText, true))
+      << "Pipeline was: " << PipelineText;
+  PM.run(*M, AM);
+}
+
+TEST_F(FunctionCallbacksTest, Passes) {
+  EXPECT_CALL(AnalysisHandle, run(HasName("foo"), _));
+  EXPECT_CALL(PassHandle, run(HasName("foo"), _))
+      .WillOnce(Invoke(getAnalysisResult));
+
+  StringRef PipelineText = "test-transform";
+  ASSERT_TRUE(PB.parsePassPipeline(PM, PipelineText, true))
+      << "Pipeline was: " << PipelineText;
+  PM.run(*M, AM);
+}
+
+TEST_F(LoopCallbacksTest, Passes) {
+  EXPECT_CALL(AnalysisHandle, run(HasName("loop"), _, _));
+  EXPECT_CALL(PassHandle, run(HasName("loop"), _, _, _))
+      .WillOnce(WithArgs<0, 1, 2>(Invoke(getAnalysisResult)));
+
+  StringRef PipelineText = "test-transform";
+  ASSERT_TRUE(PB.parsePassPipeline(PM, PipelineText, true))
+      << "Pipeline was: " << PipelineText;
+  PM.run(*M, AM);
+}
+
+TEST_F(CGSCCCallbacksTest, Passes) {
+  EXPECT_CALL(AnalysisHandle, run(HasName("(foo)"), _, _));
+  EXPECT_CALL(PassHandle, run(HasName("(foo)"), _, _, _))
+      .WillOnce(WithArgs<0, 1, 2>(Invoke(getAnalysisResult)));
+
+  StringRef PipelineText = "test-transform";
+  ASSERT_TRUE(PB.parsePassPipeline(PM, PipelineText, true))
+      << "Pipeline was: " << PipelineText;
+  PM.run(*M, AM);
+}
+
+/// Test parsing of the names of analysis utilities for our mock analysis
+/// for all IRUnits.
+///
+/// We first require<>, then invalidate<> it, expecting the analysis to be run
+/// once and subsequently invalidated.
+TEST_F(ModuleCallbacksTest, AnalysisUtilities) {
+  EXPECT_CALL(AnalysisHandle, run(HasName("<string>"), _));
+  EXPECT_CALL(AnalysisHandle, invalidate(HasName("<string>"), _, _));
+
+  StringRef PipelineText = "require<test-analysis>,invalidate<test-analysis>";
+  ASSERT_TRUE(PB.parsePassPipeline(PM, PipelineText, true))
+      << "Pipeline was: " << PipelineText;
+  PM.run(*M, AM);
+}
+
+TEST_F(CGSCCCallbacksTest, PassUtilities) {
+  EXPECT_CALL(AnalysisHandle, run(HasName("(foo)"), _, _));
+  EXPECT_CALL(AnalysisHandle, invalidate(HasName("(foo)"), _, _));
+
+  StringRef PipelineText = "require<test-analysis>,invalidate<test-analysis>";
+  ASSERT_TRUE(PB.parsePassPipeline(PM, PipelineText, true))
+      << "Pipeline was: " << PipelineText;
+  PM.run(*M, AM);
+}
+
+TEST_F(FunctionCallbacksTest, AnalysisUtilities) {
+  EXPECT_CALL(AnalysisHandle, run(HasName("foo"), _));
+  EXPECT_CALL(AnalysisHandle, invalidate(HasName("foo"), _, _));
+
+  StringRef PipelineText = "require<test-analysis>,invalidate<test-analysis>";
+  ASSERT_TRUE(PB.parsePassPipeline(PM, PipelineText, true))
+      << "Pipeline was: " << PipelineText;
+  PM.run(*M, AM);
+}
+
+TEST_F(LoopCallbacksTest, PassUtilities) {
+  EXPECT_CALL(AnalysisHandle, run(HasName("loop"), _, _));
+  EXPECT_CALL(AnalysisHandle, invalidate(HasName("loop"), _, _));
+
+  StringRef PipelineText = "require<test-analysis>,invalidate<test-analysis>";
+
+  ASSERT_TRUE(PB.parsePassPipeline(PM, PipelineText, true))
+      << "Pipeline was: " << PipelineText;
+  PM.run(*M, AM);
+}
+
+/// Test parsing of the top-level pipeline.
+///
+/// The ParseTopLevelPipeline callback takes over parsing of the entire pipeline
+/// from PassBuilder if it encounters an unknown pipeline entry at the top level
+/// (i.e., the first entry on the pipeline).
+/// This test parses a pipeline named 'another-pipeline', whose only elements
+/// may be the test-transform pass or the analysis utilities
+TEST_F(ModuleCallbacksTest, ParseTopLevelPipeline) {
+  PB.registerParseTopLevelPipelineCallback([this](
+      ModulePassManager &MPM, ArrayRef<PassBuilder::PipelineElement> Pipeline,
+      bool VerifyEachPass, bool DebugLogging) {
+    auto &FirstName = Pipeline.front().Name;
+    auto &InnerPipeline = Pipeline.front().InnerPipeline;
+    if (FirstName == "another-pipeline") {
+      for (auto &E : InnerPipeline) {
+        if (parseAnalysisUtilityPasses<AnalysisT>("test-analysis", E.Name, PM))
+          continue;
+
+        if (E.Name == "test-transform") {
+          PM.addPass(PassHandle.getPass());
+          continue;
+        }
+        return false;
+      }
+    }
+    return true;
+  });
+
+  EXPECT_CALL(AnalysisHandle, run(HasName("<string>"), _));
+  EXPECT_CALL(PassHandle, run(HasName("<string>"), _))
+      .WillOnce(Invoke(getAnalysisResult));
+  EXPECT_CALL(AnalysisHandle, invalidate(HasName("<string>"), _, _));
+
+  StringRef PipelineText =
+      "another-pipeline(test-transform,invalidate<test-analysis>)";
+  ASSERT_TRUE(PB.parsePassPipeline(PM, PipelineText, true))
+      << "Pipeline was: " << PipelineText;
+  PM.run(*M, AM);
+
+  /// Test the negative case
+  PipelineText = "another-pipeline(instcombine)";
+  ASSERT_FALSE(PB.parsePassPipeline(PM, PipelineText, true))
+      << "Pipeline was: " << PipelineText;
+}
+} // end anonymous namespace