[PM] Introduce a reasonable port of the main per-module pass pipeline from the old pass manager in the new one.

[chandlerc]

Summary

I'm not trying to support (initially) the numerous options that are
currently available to customize the pass pipeline. If we end up really
wanting them, we can add them later, but I suspect many are no longer
interesting. The simplicity of omitting them will help a lot as we sort
out what the pipeline should look like in the new PM.

I've also documented to the best of my ability *why* each pass or group
of passes is used so that reading the pipeline is more helpful. In many
cases I think we have some questionable choices of ordering and I've
left FIXME comments in place so we know what to come back and revisit
going forward. But for now, I've left it as similar to the current
pipeline as I could.

Lastly, I've had to comment out several places where passes are not
ported to the new pass manager or where the loop pass infrastructure is
not yet ready. I did at least fix a few bugs in the loop pass
infrastructure uncovered by running the full pipeline, but I didn't want
to go too far in this patch -- I'll come back and re-enable these as the
infrastructure comes online. But I'd like to keep the comments in place
because I don't want to lose track of which passes need to be enabled
and where they go.

One thing that seemed like a significant API improvement was to require
that we don't build pipelines for O0. It seems to have no real benefit.

I've also switched back to returning pass managers by value as at this
API layer it feels much more natural to me for composition. But if
others disagree, I'm happy to go back to an output parameter.

I'm not 100% happy with the testing strategy currently, but it seems at
least OK. I may come back and try to refactor or otherwise improve this
in subsequent patches but I wanted to at least get a good starting point
in place.

[mehdi_amini]

lib/Passes/PassBuilder.cpp

374

Technically the default -O{1,2,3} pipeline starts with populateFunctionPassManager (which runs -simplifycfg -domtree -sroa -early-cse -lower-expect)

773

Shame on you for not supporting ThinLTO! :p
OK, shame on me for not submitting a patch earlier...

[chandlerc]

lib/Passes/PassBuilder.cpp

374

I have that here in EarlyFPM. I just lifted the Force and Infer up because they are designed to be "as early as possible" but were module passes and so challenging before.

773

Happy for you to work on this, or happy to port that part of the existing PMB when I get to it. I mostly wanted to get enough to test stuff "in the wild".

[mehdi_amini]

LGTM, I think that'll be a good starting point for experimentation :)

lib/Passes/PassBuilder.cpp

374

Oh right, I missed it because of the motion of the FunctionAttrs passes.

include/llvm/Analysis/LoopPassManager.h

 /// This header provides classes for managing passes over loops in LLVM IR.
 ///
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_ANALYSIS_LOOPPASSMANAGER_H
 #define LLVM_ANALYSIS_LOOPPASSMANAGER_H
 
 #include "llvm/ADT/STLExtras.h"
+#include "llvm/Analysis/AliasAnalysis.h"
 #include "llvm/Analysis/LoopInfo.h"
+#include "llvm/Analysis/ScalarEvolution.h"
+#include "llvm/Analysis/TargetLibraryInfo.h"
+#include "llvm/IR/Dominators.h"
 #include "llvm/IR/PassManager.h"
 
 namespace llvm {
 
 extern template class PassManager<Loop>;
 /// \brief The loop pass manager.
 ///
 /// See the documentation for the PassManager template for details. It runs a
public:
   /// \brief Runs the loop passes across every loop in the function.
   PreservedAnalyses run(Function &F, FunctionAnalysisManager &AM) {
     // Setup the loop analysis manager from its proxy.
     LoopAnalysisManager &LAM =
         AM.getResult<LoopAnalysisManagerFunctionProxy>(F).getManager();
     // Get the loop structure for this function
     LoopInfo &LI = AM.getResult<LoopAnalysis>(F);
 
+    // Also precompute all of the function analyses used by loop passes.
+    // FIXME: These should be handed into the loop passes when the loop pass
+    // management layer is reworked to follow the design of CGSCC.
+    (void)AM.getResult<AAManager>(F);
+    (void)AM.getResult<DominatorTreeAnalysis>(F);
+    (void)AM.getResult<ScalarEvolutionAnalysis>(F);
+    (void)AM.getResult<TargetLibraryAnalysis>(F);
+
     PreservedAnalyses PA = PreservedAnalyses::all();
 
     // We want to visit the loops in reverse post-order. We'll build the stack
     // of loops to visit in Loops by first walking the loops in pre-order.
     SmallVector<Loop *, 2> Loops;
     SmallVector<Loop *, 2> WorkList(LI.begin(), LI.end());
     while (!WorkList.empty()) {
       Loop *L = WorkList.pop_back_val();

include/llvm/Passes/PassBuilder.h

public:
 
   /// \brief Registers all available loop analysis passes.
   ///
   /// This is an interface that can be used to populate a \c LoopAnalysisManager
   /// with all registered loop analyses. Callers can still manually register any
   /// additional analyses.
   void registerLoopAnalyses(LoopAnalysisManager &LAM);
 
-  /// \brief Add a per-module default optimization pipeline to a pass manager.
+  /// Construct the core LLVM function canonicalization and simplification
+  /// pipeline.
+  ///
+  /// This is a long pipeline and uses most of the per-function optimization
+  /// passes in LLVM to canonicalize and simplify the IR. It is suitable to run
+  /// repeatedly over the IR and is not expected to destroy important
+  /// information about the semantics of the IR.
+  ///
+  /// Note that \p Level cannot be `O0` here. The pipelines produced are
+  /// only intended for use when attempting to optimize code. If frontends
+  /// require some transformations for semantic reasons, they should explicitly
+  /// build them.
+  FunctionPassManager
+  buildFunctionSimplificationPipeline(OptimizationLevel Level,
+                                      bool DebugLogging = false);
+
+  /// Build a per-module default optimization pipeline.
   ///
   /// This provides a good default optimization pipeline for per-module
   /// optimization and code generation without any link-time optimization. It
   /// typically correspond to frontend "-O[123]" options for optimization
   /// levels \c O1, \c O2 and \c O3 resp.
-  void addPerModuleDefaultPipeline(ModulePassManager &MPM,
-                                   OptimizationLevel Level,
+  ///
+  /// Note that \p Level cannot be `O0` here. The pipelines produced are
+  /// only intended for use when attempting to optimize code. If frontends
+  /// require some transformations for semantic reasons, they should explicitly
+  /// build them.
+  ModulePassManager buildPerModuleDefaultPipeline(OptimizationLevel Level,
                                                   bool DebugLogging = false);
 
-  /// \brief Add a pre-link, LTO-targeting default optimization pipeline to
-  /// a pass manager.
+  /// Build a pre-link, LTO-targeting default optimization pipeline to a pass
+  /// manager.
   ///
   /// This adds the pre-link optimizations tuned to work well with a later LTO
   /// run. It works to minimize the IR which needs to be analyzed without
   /// making irreversible decisions which could be made better during the LTO
   /// run.
-  void addLTOPreLinkDefaultPipeline(ModulePassManager &MPM,
-                                    OptimizationLevel Level,
+  ///
+  /// Note that \p Level cannot be `O0` here. The pipelines produced are
+  /// only intended for use when attempting to optimize code. If frontends
+  /// require some transformations for semantic reasons, they should explicitly
+  /// build them.
+  ModulePassManager buildLTOPreLinkDefaultPipeline(OptimizationLevel Level,
                                                    bool DebugLogging = false);
 
-  /// \brief Add an LTO default optimization pipeline to a pass manager.
+  /// Build an LTO default optimization pipeline to a pass manager.
   ///
   /// This provides a good default optimization pipeline for link-time
   /// optimization and code generation. It is particularly tuned to fit well
   /// when IR coming into the LTO phase was first run through \c
   /// addPreLinkLTODefaultPipeline, and the two coordinate closely.
-  void addLTODefaultPipeline(ModulePassManager &MPM, OptimizationLevel Level,
+  ///
+  /// Note that \p Level cannot be `O0` here. The pipelines produced are
+  /// only intended for use when attempting to optimize code. If frontends
+  /// require some transformations for semantic reasons, they should explicitly
+  /// build them.
+  ModulePassManager buildLTODefaultPipeline(OptimizationLevel Level,
                                             bool DebugLogging = false);
 
   /// \brief Parse a textual pass pipeline description into a \c ModulePassManager.
   ///
   /// The format of the textual pass pipeline description looks something like:
   ///
   ///   module(function(instcombine,sroa),dce,cgscc(inliner,function(...)),...)
   ///
   /// Pass managers have ()s describing the nest structure of passes. All passes

include/llvm/Transforms/Scalar/LoopRotation.h

 #include "llvm/Analysis/LoopPassManager.h"
 #include "llvm/IR/PassManager.h"
 
 namespace llvm {
 
 /// A simple loop rotation transformation.
 class LoopRotatePass : public PassInfoMixin<LoopRotatePass> {
 public:
-  LoopRotatePass();
+  LoopRotatePass(bool EnableHeaderDuplication = true);
   PreservedAnalyses run(Loop &L, LoopAnalysisManager &AM);
+
+private:
+  const bool EnableHeaderDuplication;
 };
 }
 
 #endif // LLVM_TRANSFORMS_SCALAR_LOOPROTATION_H

lib/Analysis/LoopPassManager.cpp

 
 PreservedAnalyses llvm::getLoopPassPreservedAnalyses() {
   PreservedAnalyses PA;
   PA.preserve<DominatorTreeAnalysis>();
   PA.preserve<LoopAnalysis>();
   PA.preserve<ScalarEvolutionAnalysis>();
   // TODO: What we really want to do here is preserve an AA category, but that
   // concept doesn't exist yet.
+  PA.preserve<AAManager>();
   PA.preserve<BasicAA>();
   PA.preserve<GlobalsAA>();
   PA.preserve<SCEVAA>();
   return PA;
 }

lib/Passes/PassBuilder.cpp

 #include "llvm/Transforms/Vectorize/SLPVectorizer.h"
 
 #include <type_traits>
 
 using namespace llvm;
 
 static Regex DefaultAliasRegex("^(default|lto-pre-link|lto)<(O[0123sz])>$");
 
+static bool isOptimizingForSize(PassBuilder::OptimizationLevel Level) {
+  switch (Level) {
+  case PassBuilder::O0:
+  case PassBuilder::O1:
+  case PassBuilder::O2:
+  case PassBuilder::O3:
+    return false;
+
+  case PassBuilder::Os:
+  case PassBuilder::Oz:
+    return true;
+  }
+  llvm_unreachable("Invalid optimization level!");
+}
+
 namespace {
 
 /// \brief No-op module pass which does nothing.
 struct NoOpModulePass {
   PreservedAnalyses run(Module &M, ModuleAnalysisManager &) {
     return PreservedAnalyses::all();
   }
   static StringRef name() { return "NoOpModulePass"; }

 }
 
 void PassBuilder::registerLoopAnalyses(LoopAnalysisManager &LAM) {
 #define LOOP_ANALYSIS(NAME, CREATE_PASS)                                       \
   LAM.registerPass([&] { return CREATE_PASS; });
 #include "PassRegistry.def"
 }
 
-void PassBuilder::addPerModuleDefaultPipeline(ModulePassManager &MPM,
-                                              OptimizationLevel Level,
+FunctionPassManager
+PassBuilder::buildFunctionSimplificationPipeline(OptimizationLevel Level,
                                                  bool DebugLogging) {
-  // FIXME: Finish fleshing this out to match the legacy pipelines.
+  assert(Level != O0 && "Must request optimizations!");
+  FunctionPassManager FPM(DebugLogging);
+
+  // Form SSA out of local memory accesses after breaking apart aggregates into
+  // scalars.
+  FPM.addPass(SROA());
+
+  // Catch trivial redundancies
+  FPM.addPass(EarlyCSEPass());
+
+  // Speculative execution if the target has divergent branches; otherwise nop.
+  FPM.addPass(SpeculativeExecutionPass());
+
+  // Optimize based on known information about branches, and cleanup afterward.
+  FPM.addPass(JumpThreadingPass());
+  FPM.addPass(CorrelatedValuePropagationPass());
+  FPM.addPass(SimplifyCFGPass());
+  FPM.addPass(InstCombinePass());
+
+  if (!isOptimizingForSize(Level))
+    FPM.addPass(LibCallsShrinkWrapPass());
+
+  FPM.addPass(TailCallElimPass());
+  FPM.addPass(SimplifyCFGPass());
+
+  // Form canonically associated expression trees, and simplify the trees using
+  // basic mathematical properties. For example, this will form (nearly)
+  // minimal multiplication trees.
+  FPM.addPass(ReassociatePass());
+
+  // Add the primary loop simplification pipeline.
+  // FIXME: Currently this is split into two loop pass pipelines because we run
+  // some function passes in between them. These can and should be replaced by
+  // loop pass equivalenst but those aren't ready yet. Specifically,
+  // `SimplifyCFGPass` and `InstCombinePass` are used. We have
+  // `LoopSimplifyCFGPass` which isn't yet powerful enough, and the closest to
+  // the other we have is `LoopInstSimplify`.
+  LoopPassManager LPM1(DebugLogging), LPM2(DebugLogging);
+
+  // Rotate Loop - disable header duplication at -Oz
+  LPM1.addPass(LoopRotatePass(Level != Oz));
+  LPM1.addPass(LICMPass());
+  // FIXME: Enable these when the loop pass manager can support updating the
+  // loop nest after transformations and we finsih porting the loop passes.
+#if 0
+  LPM1.addPass(LoopUnswitchPass(/* OptimizeForSize */ Level != O3));
+  LPM2.addPass(IndVarSimplifyPass());
+  LPM2.addPass(LoopIdiomPass());
+  LPM2.addPass(LoopDeletionPass());
+  LPM2.addPass(SimpleLoopUnrollPass());
+#endif
+  FPM.addPass(createFunctionToLoopPassAdaptor(std::move(LPM1)));
+  FPM.addPass(SimplifyCFGPass());
+  FPM.addPass(InstCombinePass());
+  FPM.addPass(createFunctionToLoopPassAdaptor(std::move(LPM2)));
+
+  // Eliminate redundancies.
+  if (Level != O1) {
+    // These passes add substantial compile time so skip them at O1.
+    FPM.addPass(MergedLoadStoreMotionPass());
+    FPM.addPass(GVN());
+  }
+
+  // Specially optimize memory movement as it doesn't look like dataflow in SSA.
+  FPM.addPass(MemCpyOptPass());
+
+  // Sparse conditional constant propagation.
+  // FIXME: It isn't clear why we do this *after* loop passes rather than
+  // before...
+  FPM.addPass(SCCPPass());
+
+  // Delete dead bit computations (instcombine runs after to fold away the dead
+  // computations, and then ADCE will run later to exploit any new DCE
+  // opportunities that creates).
+  FPM.addPass(BDCEPass());
+
+  // Run instcombine after redundancy and dead bit elimination to exploit
+  // opportunities opened up by them.
+  FPM.addPass(InstCombinePass());
+
+  // Re-consider control flow based optimizations after redundancy elimination,
+  // redo DCE, etc.
+  FPM.addPass(JumpThreadingPass());
+  FPM.addPass(CorrelatedValuePropagationPass());
+  FPM.addPass(DSEPass());
+  FPM.addPass(createFunctionToLoopPassAdaptor(LICMPass()));
+
+  // Finally, do an expensive DCE pass to catch all the dead code exposed by
+  // the simplifications and basic cleanup after all the simplifications.
+  FPM.addPass(ADCEPass());
+  FPM.addPass(SimplifyCFGPass());
+  FPM.addPass(InstCombinePass());
+
+  return FPM;
+}
+
+ModulePassManager
+PassBuilder::buildPerModuleDefaultPipeline(OptimizationLevel Level,
+                                           bool DebugLogging) {
+  assert(Level != O0 && "Must request optimizations for the default pipeline!");
+  ModulePassManager MPM(DebugLogging);
+
+  // Force any function attributes we want the rest of the pipeline te observe.
+  MPM.addPass(ForceFunctionAttrsPass());
+
+  // Do basic inference of function attributes from known properties of system
+  // libraries and other oracles.
+  MPM.addPass(InferFunctionAttrsPass());
+
+  // Create an early function pass manager to cleanup the output of the
+  // frontend.
   FunctionPassManager EarlyFPM(DebugLogging);
   EarlyFPM.addPass(SimplifyCFGPass());
   EarlyFPM.addPass(SROA());
   EarlyFPM.addPass(EarlyCSEPass());
   EarlyFPM.addPass(LowerExpectIntrinsicPass());
-
+  EarlyFPM.addPass(GVNHoistPass());
   MPM.addPass(createModuleToFunctionPassAdaptor(std::move(EarlyFPM)));
+
+  // Interprocedural constant propagation now that basic cleanup has occured
+  // and prior to optimizing globals.
+  // FIXME: This position in the pipeline hasn't been carefully considered in
+  // years, it should be re-analyzed.
+  MPM.addPass(IPSCCPPass());
+
+  // Optimize globals to try and fold them into constants.
+  MPM.addPass(GlobalOptPass());
+
+  // Promote any localized globals to SSA registers.
+  // FIXME: Should this instead by a run of SROA?
+  // FIXME: We should probably run instcombine and simplify-cfg afterward to
+  // delete control flows that are dead once globals have been folded to
+  // constants.
+  MPM.addPass(createModuleToFunctionPassAdaptor(PromotePass()));
+
+  // Remove any dead arguments exposed by cleanups and constand folding
+  // globals.
+  MPM.addPass(DeadArgumentEliminationPass());
+
+  // Create a small function pass pipeline to cleanup after all the global
+  // optimizations.
+  FunctionPassManager GlobalCleanupPM(DebugLogging);
+  GlobalCleanupPM.addPass(InstCombinePass());
+  GlobalCleanupPM.addPass(SimplifyCFGPass());
+  MPM.addPass(createModuleToFunctionPassAdaptor(std::move(GlobalCleanupPM)));
+
+  // FIXME: Enable this when cross-IR-unit analysis invalidation is working.
+#if 0
+  MPM.addPass(RequireAnalysisPass<GlobalsAA>());
+#endif
+
+  // Now begin the main postorder CGSCC pipeline.
+  // FIXME: The current CGSCC pipeline has its origins in the legacy pass
+  // manager and trying to emulate its precise behavior. Much of this doesn't
+  // make a lot of sense and we should revisit the core CGSCC structure.
+  CGSCCPassManager MainCGPipeline(DebugLogging);
+
+  // Note: historically, the PruneEH pass was run first to deduce nounwind and
+  // generally clean up exception handling overhead. It isn't clear this is
+  // valuable as the inliner doesn't currently care whether it is inlining an
+  // invoke or a call.
+
+  // Run the inliner first. The theory is that we are walking bottom-up and so
+  // the callees have already been fully optimized, and we want to inline them
+  // into the callers so that our optimizations can reflect that.
+  // FIXME; Customize the threshold based on optimization level.
+  MainCGPipeline.addPass(InlinerPass());
+
+  // Now deduce any function attributes based in the current code.
+  MainCGPipeline.addPass(PostOrderFunctionAttrsPass());
+
+  // Lastly, add the core function simplification pipeline nested inside the
+  // CGSCC walk.
+  MainCGPipeline.addPass(createCGSCCToFunctionPassAdaptor(
+      buildFunctionSimplificationPipeline(Level, DebugLogging)));
+
+  MPM.addPass(
+      createModuleToPostOrderCGSCCPassAdaptor(std::move(MainCGPipeline)));
+
+  // This ends the canonicalization and simplification phase of the pipeline.
+  // At this point, we expect to have canonical and simple IR which we begin
+  // *optimizing* for efficient execution going forward.
+
+  // Eliminate externally available functions now that inlining is over -- we
+  // won't emit these anyways.
+  MPM.addPass(EliminateAvailableExternallyPass());
+
+  // Do RPO function attribute inference across the module to forward-propagate
+  // attributes where applicable.
+  // FIXME: Is this really an optimization rather than a canonicalization?
+  MPM.addPass(ReversePostOrderFunctionAttrsPass());
+
+  // Recompute GloblasAA here prior to function passes. This is particularly
+  // useful as the above will have inlined, DCE'ed, and function-attr
+  // propagated everything. We should at this point have a reasonably minimal
+  // and richly annotated call graph. By computing aliasing and mod/ref
+  // information for all local globals here, the late loop passes and notably
+  // the vectorizer will be able to use them to help recognize vectorizable
+  // memory operations.
+  // FIXME: Enable this once analysis invalidation is fully supported.
+#if 0
+  MPM.addPass(Require<GlobalsAA>());
+#endif
+
+  FunctionPassManager OptimizePM(DebugLogging);
+  OptimizePM.addPass(Float2IntPass());
+  // FIXME: We need to run some loop optimizations to re-rotate loops after
+  // simplify-cfg and others undo their rotation.
+
+  // Optimize the loop execution. These passes operate on entire loop nests
+  // rather than on each loop in an inside-out manner, and so they are actually
+  // function passes.
+  OptimizePM.addPass(LoopDistributePass());
+  OptimizePM.addPass(LoopVectorizePass());
+  // FIXME: Need to port Loop Load Elimination and add it here.
+  OptimizePM.addPass(InstCombinePass());
+
+  // Optimize parallel scalar instruction chains into SIMD instructions.
+  OptimizePM.addPass(SLPVectorizerPass());
+
+  // Cleanup after vectorizers.
+  OptimizePM.addPass(SimplifyCFGPass());
+  OptimizePM.addPass(InstCombinePass());
+
+  // Unroll small loops to hide loop backedge latency and saturate any parallel
+  // execution resources of an out-of-order processor.
+  // FIXME: Need to add once loop pass pipeline is available.
+
+  // FIXME: Add the loop sink pass when ported.
+
+  // FIXME: Add cleanup from the loop pass manager when we're forming LCSSA
+  // here.
+
+  // Now that we've vectorized and unrolled loops, we may have more refined
+  // alignment information, try to re-derive it here.
+  OptimizePM.addPass(AlignmentFromAssumptionsPass());
+
+  // ADd the core optimizing pipeline.
+  MPM.addPass(createModuleToFunctionPassAdaptor(std::move(OptimizePM)));
+
+  // Now we need to do some global optimization transforms.
+  // FIXME: It would seem like these should come first in the optimization
+  // pipeline and maybe be the bottom of the canonicalization pipeline? Weird
+  // ordering here.
+  MPM.addPass(GlobalDCEPass());
+  MPM.addPass(ConstantMergePass());
+
+  return MPM;
 }
 
-void PassBuilder::addLTOPreLinkDefaultPipeline(ModulePassManager &MPM,
-                                               OptimizationLevel Level,
-                                               bool DebugLogging) {
+ModulePassManager
+PassBuilder::buildLTOPreLinkDefaultPipeline(OptimizationLevel Level,
+  assert(Level != O0 && "Must request optimizations for the default pipeline!");
   // FIXME: We should use a customized pre-link pipeline!
-  addPerModuleDefaultPipeline(MPM, Level, DebugLogging);
+  return buildPerModuleDefaultPipeline(Level, DebugLogging);
 }
 
-void PassBuilder::addLTODefaultPipeline(ModulePassManager &MPM,
-                                        OptimizationLevel Level,
+ModulePassManager PassBuilder::buildLTODefaultPipeline(OptimizationLevel Level,
                                                        bool DebugLogging) {
+  assert(Level != O0 && "Must request optimizations for the default pipeline!");
+  ModulePassManager MPM(DebugLogging);
+
   // FIXME: Finish fleshing this out to match the legacy LTO pipelines.
   FunctionPassManager LateFPM(DebugLogging);
   LateFPM.addPass(InstCombinePass());
   LateFPM.addPass(SimplifyCFGPass());
 
   MPM.addPass(createModuleToFunctionPassAdaptor(std::move(LateFPM)));
+
+  return MPM;
 }
 
 static Optional<int> parseRepeatPassName(StringRef Name) {
   if (!Name.consume_front("repeat<") || !Name.consume_back(">"))
     return None;
   int Count;
   if (Name.getAsInteger(0, Count) || Count <= 0)
     return None;
if (Name.startswith("default") || Name.startswith("lto")) {
 
     OptimizationLevel L = StringSwitch<OptimizationLevel>(Matches[2])
         .Case("O0", O0)
         .Case("O1", O1)
         .Case("O2", O2)
         .Case("O3", O3)
         .Case("Os", Os)
         .Case("Oz", Oz);
+    if (L == O0)
+      // At O0 we do nothing at all!
+      return true;
 
     if (Matches[1] == "default") {
-      addPerModuleDefaultPipeline(MPM, L, DebugLogging);
+      MPM.addPass(buildPerModuleDefaultPipeline(L, DebugLogging));
     } else if (Matches[1] == "lto-pre-link") {
-      addLTOPreLinkDefaultPipeline(MPM, L, DebugLogging);
+      MPM.addPass(buildLTOPreLinkDefaultPipeline(L, DebugLogging));
     } else {
       assert(Matches[1] == "lto" && "Not one of the matched options!");
-      addLTODefaultPipeline(MPM, L, DebugLogging);
+      MPM.addPass(buildLTODefaultPipeline(L, DebugLogging));
     }
     return true;
   }
 
   // Finally expand the basic registered passes from the .inc file.
 #define MODULE_PASS(NAME, CREATE_PASS)                                         \
   if (Name == NAME) {                                                          \
     MPM.addPass(CREATE_PASS);                                                  \

lib/Transforms/Scalar/LoopRotation.cpp

bool LoopRotate::processLoop(Loop *L) {
   // Restore the loop metadata.
   // NB! We presume LoopRotation DOESN'T ADD its own metadata.
   if ((MadeChange || SimplifiedLatch) && LoopMD)
     L->setLoopID(LoopMD);
 
   return MadeChange;
 }
 
-LoopRotatePass::LoopRotatePass() {}
+LoopRotatePass::LoopRotatePass(bool EnableHeaderDuplication)
+    : EnableHeaderDuplication(EnableHeaderDuplication) {}
 
 PreservedAnalyses LoopRotatePass::run(Loop &L, LoopAnalysisManager &AM) {
   auto &FAM = AM.getResult<FunctionAnalysisManagerLoopProxy>(L).getManager();
   Function *F = L.getHeader()->getParent();
 
   auto *LI = FAM.getCachedResult<LoopAnalysis>(*F);
   const auto *TTI = FAM.getCachedResult<TargetIRAnalysis>(*F);
   auto *AC = FAM.getCachedResult<AssumptionAnalysis>(*F);
   assert((LI && TTI && AC) && "Analyses for loop rotation not available");
 
   // Optional analyses.
   auto *DT = FAM.getCachedResult<DominatorTreeAnalysis>(*F);
   auto *SE = FAM.getCachedResult<ScalarEvolutionAnalysis>(*F);
-  LoopRotate LR(DefaultRotationThreshold, LI, TTI, AC, DT, SE);
+  int Threshold = EnableHeaderDuplication ? DefaultRotationThreshold : 0;
+  LoopRotate LR(Threshold, LI, TTI, AC, DT, SE);
 
   bool Changed = LR.processLoop(&L);
   if (!Changed)
     return PreservedAnalyses::all();
   return getLoopPassPreservedAnalyses();
 }
 
 namespace {

test/Other/new-pass-manager.ll

 ; RUN:     | FileCheck %s --check-prefix=CHECK-CALLGRAPH
 ; CHECK-CALLGRAPH: Starting llvm::Module pass manager run
 ; CHECK-CALLGRAPH: Running pass: RequireAnalysisPass
 ; CHECK-CALLGRAPH: Running analysis: CallGraphAnalysis
 ; CHECK-CALLGRAPH: Finished llvm::Module pass manager run
 
 ; RUN: opt -disable-output -disable-verify -debug-pass-manager \
 ; RUN:     -passes='default<O0>' %s 2>&1 \
-; RUN:     | FileCheck %s --check-prefix=CHECK-O2
+; RUN:     | FileCheck %s --check-prefix=CHECK-O0
+; CHECK-O0: Starting llvm::Module pass manager run
+; CHECK-O0-NEXT: Finished llvm::Module pass manager run
+
 ; RUN: opt -disable-output -disable-verify -debug-pass-manager \
 ; RUN:     -passes='default<O1>' %s 2>&1 \
-; RUN:     | FileCheck %s --check-prefix=CHECK-O2
+; RUN:     | FileCheck %s --check-prefix=CHECK-O --check-prefix=CHECK-O1
 ; RUN: opt -disable-output -disable-verify -debug-pass-manager \
 ; RUN:     -passes='default<O2>' %s 2>&1 \
-; RUN:     | FileCheck %s --check-prefix=CHECK-O2
+; RUN:     | FileCheck %s --check-prefix=CHECK-O --check-prefix=CHECK-O2
 ; RUN: opt -disable-output -disable-verify -debug-pass-manager \
 ; RUN:     -passes='default<Os>' %s 2>&1 \
-; RUN:     | FileCheck %s --check-prefix=CHECK-O2
+; RUN:     | FileCheck %s --check-prefix=CHECK-O --check-prefix=CHECK-Os
 ; RUN: opt -disable-output -disable-verify -debug-pass-manager \
 ; RUN:     -passes='default<Oz>' %s 2>&1 \
-; RUN:     | FileCheck %s --check-prefix=CHECK-O2
+; RUN:     | FileCheck %s --check-prefix=CHECK-O --check-prefix=CHECK-Oz
 ; RUN: opt -disable-output -disable-verify -debug-pass-manager \
 ; RUN:     -passes='lto-pre-link<O2>' %s 2>&1 \
-; RUN:     | FileCheck %s --check-prefix=CHECK-O2
-; CHECK-O2: Starting llvm::Module pass manager run
-; CHECK-O2: Running pass: SimplifyCFGPass
-; CHECK-O2: Running pass: SROA
-; CHECK-O2: Running pass: EarlyCSEPass
-; CHECK-O2: Running pass: LowerExpectIntrinsicPass
+; RUN:     | FileCheck %s --check-prefix=CHECK-O --check-prefix=CHECK-O2
+; CHECK-O: Starting llvm::Module pass manager run
+; CHECK-O: Starting llvm::Module pass manager run
+; CHECK-O: Running pass: ForceFunctionAttrsPass
+; CHECK-O: Running pass: InferFunctionAttrsPass
+; CHECK-O: Starting llvm::Function pass manager run.
+; CHECK-O: Running pass: SimplifyCFGPass
+; CHECK-O: Running pass: SROA
+; CHECK-O: Running pass: EarlyCSEPass
+; CHECK-O: Running pass: LowerExpectIntrinsicPass
+; CHECK-O: Running pass: GVNHoistPass
+; CHECK-O: Finished llvm::Function pass manager run.
+; CHECK-O: Running pass: IPSCCPPass
+; CHECK-O: Running pass: GlobalOptPass
+; CHECK-O: Running pass: ModuleToFunctionPassAdaptor<{{.*}}PromotePass>
+; CHECK-O: Running pass: DeadArgumentEliminationPass
+; CHECK-O: Starting llvm::Function pass manager run.
+; CHECK-O: Running pass: InstCombinePass
+; CHECK-O: Running pass: SimplifyCFGPass
+; CHECK-O: Finished llvm::Function pass manager run.
+; CHECK-O: Starting CGSCC pass manager run.
+; CHECK-O: Starting llvm::Function pass manager run.
+; CHECK-O: Running pass: SROA
+; CHECK-O: Running pass: EarlyCSEPass
+; CHECK-O: Running pass: SpeculativeExecutionPass
+; CHECK-O: Running pass: JumpThreadingPass
+; CHECK-O: Running pass: CorrelatedValuePropagationPass
+; CHECK-O: Running pass: SimplifyCFGPass
+; CHECK-O: Running pass: InstCombinePass
+; CHECK-O1: Running pass: LibCallsShrinkWrapPass
+; CHECK-O2: Running pass: LibCallsShrinkWrapPass
+; CHECK-Os-NOT: Running pass: LibCallsShrinkWrapPass
+; CHECK-Oz-NOT: Running pass: LibCallsShrinkWrapPass
+; CHECK-O: Running pass: TailCallElimPass
+; CHECK-O: Running pass: SimplifyCFGPass
+; CHECK-O: Running pass: ReassociatePass
+; CHECK-O: Starting llvm::Loop pass manager run.
+; CHECK-O: Running pass: LoopRotatePass
+; CHECK-O: Running pass: LICMPass
+; CHECK-O: Finished llvm::Loop pass manager run.
+; CHECK-O: Running pass: SimplifyCFGPass
+; CHECK-O: Running pass: InstCombinePass
+; CHECK-O: Starting llvm::Loop pass manager run.
+; CHECK-O: Finished llvm::Loop pass manager run.
+; CHECK-O: Running pass: MemCpyOptPass
+; CHECK-O: Running pass: SCCPPass
+; CHECK-O: Running pass: BDCEPass
+; CHECK-O: Running pass: InstCombinePass
+; CHECK-O: Running pass: JumpThreadingPass
+; CHECK-O: Running pass: CorrelatedValuePropagationPass
+; CHECK-O: Running pass: DSEPass
+; CHECK-O: Running pass: FunctionToLoopPassAdaptor<{{.*}}LICMPass>
+; CHECK-O: Running pass: ADCEPass
+; CHECK-O: Running pass: SimplifyCFGPass
+; CHECK-O: Running pass: InstCombinePass
+; CHECK-O: Finished llvm::Function pass manager run.
+; CHECK-O: Finished CGSCC pass manager run.
+; CHECK-O: Running pass: EliminateAvailableExternallyPass
+; CHECK-O: Running pass: ReversePostOrderFunctionAttrsPass
+; CHECK-O: Starting llvm::Function pass manager run.
+; CHECK-O: Running pass: Float2IntPass
+; CHECK-O: Running pass: LoopDistributePass
+; CHECK-O: Running pass: LoopVectorizePass
+; CHECK-O: Running pass: InstCombinePass
+; CHECK-O: Running pass: SLPVectorizerPass
+; CHECK-O: Running pass: SimplifyCFGPass
+; CHECK-O: Running pass: InstCombinePass
+; CHECK-O: Running pass: AlignmentFromAssumptionsPass
+; CHECK-O: Finished llvm::Function pass manager run.
+; CHECK-O: Running pass: GlobalDCEPass
+; CHECK-O: Running pass: ConstantMergePass
 
 ; RUN: opt -disable-output -disable-verify -debug-pass-manager \
 ; RUN:     -passes='lto<O2>' %s 2>&1 \
 ; RUN:     | FileCheck %s --check-prefix=CHECK-LTO-O2
 ; CHECK-LTO-O2: Starting llvm::Module pass manager run
 ; CHECK-LTO-O2: Running pass: InstCombinePass
 ; CHECK-LTO-O2: Running pass: SimplifyCFGPass
 

 ; CHECK-REPEAT-LOOP-PASS: Starting llvm::Module pass manager run
 ; CHECK-REPEAT-LOOP-PASS-NEXT: Running pass: ModuleToFunctionPassAdaptor
 ; CHECK-REPEAT-LOOP-PASS-NEXT: Running analysis: InnerAnalysisManagerProxy<{{.*}}>
 ; CHECK-REPEAT-LOOP-PASS-NEXT: Starting llvm::Function pass manager run
 ; CHECK-REPEAT-LOOP-PASS-NEXT: Running pass: FunctionToLoopPassAdaptor
 ; CHECK-REPEAT-LOOP-PASS-NEXT: Running analysis: InnerAnalysisManagerProxy<{{.*}}>
 ; CHECK-REPEAT-LOOP-PASS-NEXT: Running analysis: LoopAnalysis
 ; CHECK-REPEAT-LOOP-PASS-NEXT: Running analysis: DominatorTreeAnalysis
+; CHECK-REPEAT-LOOP-PASS-NEXT: Running analysis: AAManager
+; CHECK-REPEAT-LOOP-PASS-NEXT: Running analysis: TargetLibraryAnalysis
+; CHECK-REPEAT-LOOP-PASS-NEXT: Running analysis: ScalarEvolutionAnalysis
+; CHECK-REPEAT-LOOP-PASS-NEXT: Running analysis: AssumptionAnalysis
 ; CHECK-REPEAT-LOOP-PASS-NEXT: Starting llvm::Loop pass manager run
 ; CHECK-REPEAT-LOOP-PASS-NEXT: Running pass: RepeatedPass
 ; CHECK-REPEAT-LOOP-PASS-NEXT: Starting llvm::Loop pass manager run
 ; CHECK-REPEAT-LOOP-PASS-NEXT: Running pass: NoOpLoopPass
 ; CHECK-REPEAT-LOOP-PASS-NEXT: Finished llvm::Loop pass manager run
 ; CHECK-REPEAT-LOOP-PASS-NEXT: Starting llvm::Loop pass manager run
 ; CHECK-REPEAT-LOOP-PASS-NEXT: Running pass: NoOpLoopPass
 ; CHECK-REPEAT-LOOP-PASS-NEXT: Finished llvm::Loop pass manager run

unittests/Analysis/LoopPassManagerTest.cpp

 //===- llvm/unittest/Analysis/LoopPassManagerTest.cpp - LPM tests ---------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
-#include "gtest/gtest.h"
+#include "llvm/Analysis/AliasAnalysis.h"
+#include "llvm/Analysis/AssumptionCache.h"
 #include "llvm/Analysis/LoopPassManager.h"
+#include "llvm/Analysis/ScalarEvolution.h"
+#include "llvm/Analysis/TargetLibraryInfo.h"
 #include "llvm/AsmParser/Parser.h"
 #include "llvm/IR/Dominators.h"
 #include "llvm/IR/Function.h"
 #include "llvm/IR/LLVMContext.h"
 #include "llvm/IR/Module.h"
 #include "llvm/IR/PassManager.h"
 #include "llvm/Support/SourceMgr.h"
+#include "gtest/gtest.h"
 
 using namespace llvm;
 
 namespace {
 
 class TestLoopAnalysis : public AnalysisInfoMixin<TestLoopAnalysis> {
   friend AnalysisInfoMixin<TestLoopAnalysis>;
   static AnalysisKey Key;
TEST_F(LoopPassManagerTest, Basic) {
   LoopAnalysisManager LAM(true);
   int LoopAnalysisRuns = 0;
   LAM.registerPass([&] { return TestLoopAnalysis(LoopAnalysisRuns); });
 
   FunctionAnalysisManager FAM(true);
   // We need DominatorTreeAnalysis for LoopAnalysis.
   FAM.registerPass([&] { return DominatorTreeAnalysis(); });
   FAM.registerPass([&] { return LoopAnalysis(); });
+  // We also allow loop passes to assume a set of other analyses and so need
+  // those.
+  FAM.registerPass([&] { return AAManager(); });
+  FAM.registerPass([&] { return TargetLibraryAnalysis(); });
+  FAM.registerPass([&] { return ScalarEvolutionAnalysis(); });
+  FAM.registerPass([&] { return AssumptionAnalysis(); });
   FAM.registerPass([&] { return LoopAnalysisManagerFunctionProxy(LAM); });
   LAM.registerPass([&] { return FunctionAnalysisManagerLoopProxy(FAM); });
 
   ModuleAnalysisManager MAM(true);
   MAM.registerPass([&] { return FunctionAnalysisManagerModuleProxy(FAM); });
   FAM.registerPass([&] { return ModuleAnalysisManagerFunctionProxy(MAM); });
 
   ModulePassManager MPM(true);