Refactor the build{Module|Function}SimplificationPipeline to expose optimization phase.

[danielcdh]

Summary

This is in preparation of https://reviews.llvm.org/D36052

[chandlerc]

I definitely lik ethis direction. Minor comments about wording / phrasing of things below.

include/llvm/Passes/PassBuilder.h

74–77

I don't think we want this to be so open ended. Until there is some *other* necessary predicate, we should scope this very narrowly so it isn't tempting to use for other things without careful consideration.

I'd call this ThinLTOPhase. Also, I'd make it an enum class.

78–79

Based on the above, this would likely become None with the comment that it indicates no ThinLTO behavior needed.

81–90

And these can just be PreLink and PostLink as they'll have a qualifier when used: ThinLTOPhase::PreLink.

I think the comments should also be minimal. The details of how PGO being listed here seem risky to just fall out of sync with the reality. I'd either say nothing (because PreLink seems pretty clear) or that they correspond to the pre-link phase of ThinLTO.

[danielcdh]

update

[chandlerc]

LGTM with a comment wording tweak below. Thanks for he refactoring!

include/llvm/Passes/PassBuilder.h

236

"the current optimization phase" -> "the current ThinLTO phase", here and below where the same phrasing is used.

[danielcdh]

update. Thanks for the review!

include/llvm/Passes/PassBuilder.h

public:
   /// name is the name of a pass, the InnerPipeline is empty, since passes
   /// cannot contain inner pipelines. See parsePassPipeline() for a more
   /// detailed description of the textual pipeline format.
   struct PipelineElement {
     StringRef Name;
     std::vector<PipelineElement> InnerPipeline;
   };
 
+  /// \brief LLVM optimization phase.
+  ///
+  /// This enumerates the LLVM high0levle optimization phases.
+  enum OptimizationPhase {
+    /// Regular optimization phase, which generated final assembly/object file.
+    Regular,
+
+    /// Optimization phase invoked by ThinLTO pre-link phase.
+    /// For PGO, it invokes profile instrumentation/use at this phase.
+    /// For SamplePGO, it invokes the first profile annotation at this phase.
+    /// ICP should not be invoked in this phase.
+    PreLinkThinLTO,
+
+    /// Optimization phase invoked by ThinLTO post-link phase.
+    /// For PGO, it invokes ICP in this phase.
+    /// For SamplePGO, it invokes the 2nd profile annotation and then ICP.
+    PostLinkThinLTO
+  };
+
   /// \brief LLVM-provided high-level optimization levels.
   ///
   /// This enumerates the LLVM-provided high-level optimization levels. Each
   /// level has a specific goal and rationale.
   enum OptimizationLevel {
     /// Disable as many optimizations as possible. This doesn't completely
     /// disable the optimizer in all cases, for example always_inline functions
     /// can be required to be inlined for correctness.
public:
   /// repeatedly over the IR and is not expected to destroy important
   /// information about the semantics of the IR.
   ///
   /// Note that \p Level cannot be `O0` here. The pipelines produced are
   /// only intended for use when attempting to optimize code. If frontends
   /// require some transformations for semantic reasons, they should explicitly
   /// build them.
   ///
-  /// \p PrepareForThinLTO indicates whether this is invoked in
-  /// PrepareForThinLTO phase. Special handling is needed for sample PGO to
-  /// ensure profile accurate in the backend profile annotation phase.
+  /// \p Phase indicates the current optimization phase.
   FunctionPassManager
   buildFunctionSimplificationPipeline(OptimizationLevel Level,
-                                      bool DebugLogging = false,
-                                      bool PrepareForThinLTO = false);
+                                      OptimizationPhase Phase,
+                                      bool DebugLogging = false);
 
   /// Construct the core LLVM module canonicalization and simplification
   /// pipeline.
   ///
   /// This pipeline focuses on canonicalizing and simplifying the entire module
   /// of IR. Much like the function simplification pipeline above, it is
   /// suitable to run repeatedly over the IR and is not expected to destroy
   /// important information. It does, however, perform inlining and other
   /// heuristic based simplifications that are not strictly reversible.
   ///
   /// Note that \p Level cannot be `O0` here. The pipelines produced are
   /// only intended for use when attempting to optimize code. If frontends
   /// require some transformations for semantic reasons, they should explicitly
   /// build them.
   ///
-  /// \p PrepareForThinLTO indicates whether this is invoked in
-  /// PrepareForThinLTO phase. Special handling is needed for sample PGO to
-  /// ensure profile accurate in the backend profile annotation phase.
+  /// \p Phase indicates the current optimization phase.
   ModulePassManager
   buildModuleSimplificationPipeline(OptimizationLevel Level,
-                                    bool DebugLogging = false,
-                                    bool PrepareForThinLTO = false);
+                                    OptimizationPhase Phase,
+                                    bool DebugLogging = false);
 
   /// Construct the core LLVM module optimization pipeline.
   ///
   /// This pipeline focuses on optimizing the execution speed of the IR. It
   /// uses cost modeling and thresholds to balance code growth against runtime
   /// improvements. It includes vectorization and other information destroying
   /// transformations. It also cannot generally be run repeatedly on a module
   /// without potentially seriously regressing either runtime performance of

lib/Passes/PassBuilder.cpp

 #include "PassRegistry.def"
 
   for (auto &C : LoopAnalysisRegistrationCallbacks)
     C(LAM);
 }
 
 FunctionPassManager
 PassBuilder::buildFunctionSimplificationPipeline(OptimizationLevel Level,
-                                                 bool DebugLogging,
-                                                 bool PrepareForThinLTO) {
+                                                 OptimizationPhase Phase,
+                                                 bool DebugLogging) {
   assert(Level != O0 && "Must request optimizations!");
   FunctionPassManager FPM(DebugLogging);
 
   // Form SSA out of local memory accesses after breaking apart aggregates into
   // scalars.
   FPM.addPass(SROA());
 
   // Catch trivial redundancies
PassBuilder::buildFunctionSimplificationPipeline(OptimizationLevel Level,
   LPM1.addPass(SimpleLoopUnswitchPass());
   LPM2.addPass(IndVarSimplifyPass());
   LPM2.addPass(LoopIdiomRecognizePass());
 
   for (auto &C : LateLoopOptimizationsEPCallbacks)
     C(LPM2, Level);
 
   LPM2.addPass(LoopDeletionPass());
-  // Do not enable unrolling in PrepareForThinLTO phase during sample PGO
+  // Do not enable unrolling in PreLinkThinLTO phase during sample PGO
   // because it changes IR to makes profile annotation in back compile
   // inaccurate.
-  if (!PrepareForThinLTO || !PGOOpt || PGOOpt->SampleProfileFile.empty())
+  if (Phase != PreLinkThinLTO || !PGOOpt || PGOOpt->SampleProfileFile.empty())
     LPM2.addPass(LoopUnrollPass::createFull(Level));
 
   for (auto &C : LoopOptimizerEndEPCallbacks)
     C(LPM2, Level);
 
   // We provide the opt remark emitter pass for LICM to use. We only need to do
   // this once as it is immutable.
   FPM.addPass(RequireAnalysisPass<OptimizationRemarkEmitterAnalysis, Function>());
getInlineParamsFromOptLevel(PassBuilder::OptimizationLevel Level) {
   auto O3 = PassBuilder::O3;
   unsigned OptLevel = Level > O3 ? 2 : Level;
   unsigned SizeLevel = Level > O3 ? Level - O3 : 0;
   return getInlineParams(OptLevel, SizeLevel);
 }
 
 ModulePassManager
 PassBuilder::buildModuleSimplificationPipeline(OptimizationLevel Level,
-                                               bool DebugLogging,
-                                               bool PrepareForThinLTO) {
+                                               OptimizationPhase Phase,
+                                               bool DebugLogging) {
   ModulePassManager MPM(DebugLogging);
 
   // Do basic inference of function attributes from known properties of system
   // libraries and other oracles.
   MPM.addPass(InferFunctionAttrsPass());
 
   // Create an early function pass manager to cleanup the output of the
   // frontend.
if (!PGOOpt->ProfileGenFile.empty() || !PGOOpt->ProfileUseFile.empty())
       // Instrumentation based PGO (gen and use)
       addPGOInstrPasses(MPM, DebugLogging, Level, PGOOpt->RunProfileGen,
                         PGOOpt->ProfileGenFile, PGOOpt->ProfileUseFile);
     else if (!PGOOpt->SampleProfileFile.empty())
       // SamplePGO use
       MPM.addPass(SampleProfileLoaderPass(PGOOpt->SampleProfileFile));
 
     // Indirect call promotion that promotes intra-module targes only.
-    // Do not enable it in PrepareForThinLTO phase during sample PGO because
+    // Do not enable it in PreLinkThinLTO phase during sample PGO because
     // it changes IR to makes profile annotation in back compile inaccurate.
-    if ((!PrepareForThinLTO && !PGOOpt->SampleProfileFile.empty())
+    if ((Phase != PreLinkThinLTO && !PGOOpt->SampleProfileFile.empty())
         || !PGOOpt->ProfileUseFile.empty())
       MPM.addPass(PGOIndirectCallPromotion(
           false, PGOOpt && !PGOOpt->SampleProfileFile.empty()));
   }
 
   // Require the GlobalsAA analysis for the module so we can query it within
   // the CGSCC pipeline.
   MPM.addPass(RequireAnalysisPass<GlobalsAA, Module>());
PassBuilder::buildModuleSimplificationPipeline(OptimizationLevel Level,
   // Note: historically, the PruneEH pass was run first to deduce nounwind and
   // generally clean up exception handling overhead. It isn't clear this is
   // valuable as the inliner doesn't currently care whether it is inlining an
   // invoke or a call.
 
   // Run the inliner first. The theory is that we are walking bottom-up and so
   // the callees have already been fully optimized, and we want to inline them
   // into the callers so that our optimizations can reflect that.
-  // For PrepareForThinLTO pass, we disable hot-caller heuristic for sample PGO
+  // For PreLinkThinLTO pass, we disable hot-caller heuristic for sample PGO
   // because it makes profile annotation in the backend inaccurate.
   InlineParams IP = getInlineParamsFromOptLevel(Level);
-  if (PrepareForThinLTO && PGOOpt && !PGOOpt->SampleProfileFile.empty())
+  if (Phase == PreLinkThinLTO && PGOOpt && !PGOOpt->SampleProfileFile.empty())
     IP.HotCallSiteThreshold = 0;
   MainCGPipeline.addPass(InlinerPass(IP));
 
   // Now deduce any function attributes based in the current code.
   MainCGPipeline.addPass(PostOrderFunctionAttrsPass());
 
   // When at O3 add argument promotion to the pass pipeline.
   // FIXME: It isn't at all clear why this should be limited to O3.
   if (Level == O3)
     MainCGPipeline.addPass(ArgumentPromotionPass());
 
   // Lastly, add the core function simplification pipeline nested inside the
   // CGSCC walk.
   MainCGPipeline.addPass(createCGSCCToFunctionPassAdaptor(
-      buildFunctionSimplificationPipeline(Level, DebugLogging,
-                                          PrepareForThinLTO)));
+      buildFunctionSimplificationPipeline(Level, Phase, DebugLogging)));
 
   for (auto &C : CGSCCOptimizerLateEPCallbacks)
     C(MainCGPipeline, Level);
 
   // We wrap the CGSCC pipeline in a devirtualization repeater. This will try
   // to detect when we devirtualize indirect calls and iterate the SCC passes
   // in that case to try and catch knock-on inlining or function attrs
   // opportunities. Then we add it to the module pipeline by walking the SCCs
PassBuilder::buildPerModuleDefaultPipeline(OptimizationLevel Level,
 
   // Force any function attributes we want the rest of the pipeline to observe.
   MPM.addPass(ForceFunctionAttrsPass());
 
   if (PGOOpt && PGOOpt->SamplePGOSupport)
     MPM.addPass(createModuleToFunctionPassAdaptor(AddDiscriminatorsPass()));
 
   // Add the core simplification pipeline.
-  MPM.addPass(buildModuleSimplificationPipeline(Level, DebugLogging,
-                                                /*PrepareForThinLTO=*/false));
+  MPM.addPass(buildModuleSimplificationPipeline(Level, Regular, DebugLogging));
 
   // Now add the optimization pipeline.
   MPM.addPass(buildModuleOptimizationPipeline(Level, DebugLogging));
 
   return MPM;
 }
 
 ModulePassManager
 PassBuilder::buildThinLTOPreLinkDefaultPipeline(OptimizationLevel Level,
                                                 bool DebugLogging) {
   assert(Level != O0 && "Must request optimizations for the default pipeline!");
 
   ModulePassManager MPM(DebugLogging);
 
   // Force any function attributes we want the rest of the pipeline to observe.
   MPM.addPass(ForceFunctionAttrsPass());
 
   if (PGOOpt && PGOOpt->SamplePGOSupport)
     MPM.addPass(createModuleToFunctionPassAdaptor(AddDiscriminatorsPass()));
 
   // If we are planning to perform ThinLTO later, we don't bloat the code with
   // unrolling/vectorization/... now. Just simplify the module as much as we
   // can.
-  MPM.addPass(buildModuleSimplificationPipeline(Level, DebugLogging,
-                                                /*PrepareForThinLTO=*/true));
+  MPM.addPass(buildModuleSimplificationPipeline(Level, PreLinkThinLTO,
+                                                DebugLogging));
 
   // Run partial inlining pass to partially inline functions that have
   // large bodies.
   // FIXME: It isn't clear whether this is really the right place to run this
   // in ThinLTO. Because there is another canonicalization and simplification
   // phase that will run after the thin link, running this here ends up with
   // less information than will be available later and it may grow functions in
   // ways that aren't beneficial.
PassBuilder::buildThinLTODefaultPipeline(OptimizationLevel Level,
   // During the ThinLTO backend phase we perform early indirect call promotion
   // here, before globalopt. Otherwise imported available_externally functions
   // look unreferenced and are removed.
   MPM.addPass(PGOIndirectCallPromotion(
       true /* InLTO */, PGOOpt && !PGOOpt->SampleProfileFile.empty() &&
                             !PGOOpt->ProfileUseFile.empty()));
 
   // Add the core simplification pipeline.
-  MPM.addPass(buildModuleSimplificationPipeline(Level, DebugLogging,
-                                                /*PrepareForThinLTO=*/false));
+  MPM.addPass(buildModuleSimplificationPipeline(Level, PostLinkThinLTO,
+                                                DebugLogging));
 
   // Now add the optimization pipeline.
   MPM.addPass(buildModuleOptimizationPipeline(Level, DebugLogging));
 
   return MPM;
 }
 
 ModulePassManager