[Inline][Remark] Annotate inline pass name with link phase information for analysis.

[mingmingl]

Summary

Annotate inline remarks with two dimensional information

1. the link stage, or no LTO.
2. the context / driver (CGSCC, ModuleInliner, etc).

Implementation-wise, the annotation is flag gated; flag is turned off by default.

[aeubanks]

can you put the motivation in the description?

llvm/lib/Analysis/InlineAdvisor.cpp

105

these parens aren't necessary

llvm/lib/Passes/PassBuilderPipelines.cpp

1138–1139

ternary operator is unnecessary

[mingmingl]

Resolve the comments.

Plus, use llvm_unreachable for getAnnotatedPassName helper function since switch statement has default and should return

[mingmingl]

In D125495#3510037, @aeubanks wrote:

can you put the motivation in the description?

add the motivation in Summary.

[mingmingl]

Fix a build error for llvm-project/llvm/lib/Transforms/IPO/SampleProfile.cpp:613:1: error: expected ‘;’ before ‘}’ token

[davidxl]

It might be worth annotating the early inliner for PGO as well.

llvm/lib/Analysis/InlineAdvisor.cpp

60

document possible inline pass names here.

110

why adding 'default' in the name?

llvm/lib/Transforms/IPO/SampleProfile.cpp

595

I think it is better to also add 'sample-loader' into the pass string as well.

[wenlei]

there's also an always inline pass (AlwaysInliner.cpp).

cc @modimo in case this affects inline replay as well. theoretically replay can be more accurate leveraging accurate pass info.

[mingmingl]

In D125495#3510137, @davidxl wrote:

It might be worth annotating the early inliner for PGO as well.

To confirm if I'm understanding it correctly, you mean we should annotate the "early" (i.e., not profile-driven) information for those inline transformations that are are made without PGO profiles [1].

Ask since the early inliner is annotated with prelink/postlink information (same line around line 630 by construction of ModuleInlinerWrapperPass instance)

[1] https://github.com/llvm/llvm-project/blob/e91a73de24d60954700d7ac0293c050ab2cbe90b/llvm/lib/Passes/PassBuilderPipelines.cpp#L630

In D125495#3510215, @wenlei wrote:

there's also an always inline pass (AlwaysInliner.cpp).

cc @modimo in case this affects inline replay as well. theoretically replay can be more accurate leveraging accurate pass info.

thanks for mentioning this!

Re always-inliner, it's currently providing always-inline information in pass remark name [2]. So do we want to annotate the pass name for always-inline?

Re replay inline, the pass name for replay inline advices will remain the same with -annotate-inline-lto-phase on -> the replay inline advisor class implementation remain the same before and after this change.
For my understanding, is the replay inline advisor used for development (conveniently inline the functions according to an external file)? Is the replay inline advisor used for production build?

To share some thoughts as context, the current implementation is simple and annotates prelink/postlink information for scc-inliner and sample-loader inliner

• To elaborate, for scc inliner, only when constructors of default-inline-advisor provides the optional LTOPhase and flag is on, the pass name change.
  • From this perspective, if new implementations of InlineAdvisor/InlineAdvice interface are added, they won't pick up the change inadvertently.
• On the other hand, if we want to carry implementation-specific information for general inline advice (e.g. always, replay, module-scc or scc) , one option is to add new virtual method in InlineAdvisor/InlineAdvice.

[2] https://github.com/llvm/llvm-project/blob/e91a73de24d60954700d7ac0293c050ab2cbe90b/llvm/lib/Transforms/IPO/AlwaysInliner.cpp#L86 and https://github.com/llvm/llvm-project/blob/e91a73de24d60954700d7ac0293c050ab2cbe90b/llvm/lib/Analysis/InlineAdvisor.cpp#L477

llvm/lib/Analysis/InlineAdvisor.cpp

60

Before changing option name (that involves updating tests), shall the option be renamed as 'annotate-scc-inline-lto-phase'.

Regardless, I should probably document the affected inline pass names in description (re always-inline and replay-inline in another comment).

(keep this comment open)

110

'default' was originally added to indicate the particular instance of DefaultInlineAdvice is from DefaultInlineAdvisor (i.e., heuristic-based), and not from ReplayInlineAdvisor [1].

However, I realize this is not necessary now -> ReplayInlineAdvisor won't provide LTOPhase when constructing an advice, so 'default' could be erased. Before making a change (that involves updating tests), shall I call it 'thin-lto-prelink-inline' or 'thin-lto-prelink-scc-inline'?

(keep this comment open)

[1] https://github.com/llvm/llvm-project/blob/b1aed14bfea07508e4b9d864168c1ae6b5b5c665/llvm/lib/Analysis/ReplayInlineAdvisor.cpp#L116

llvm/lib/Transforms/IPO/SampleProfile.cpp

595

The macros has "CSINLINE_DEBUG", which has a substring "sample-profile". So they currently carry the pass information ("sample-profile"). Nevertheless it took me a while to realize it's "sample-profile" (not "sample-loader") when analyzing the remarks.

[kazu]

In D125495#3510338, @mingmingl wrote:

In D125495#3510137, @davidxl wrote:

It might be worth annotating the early inliner for PGO as well.

To confirm if I'm understanding it correctly, you mean we should annotate the "early" (i.e., not profile-driven) information for those inline transformations that are are made without PGO profiles [1].

Ask since the early inliner is annotated with prelink/postlink information (same line around line 630 by construction of ModuleInlinerWrapperPass instance)

I am not David, but yes, it would be nice to be able to differentiate the early inlining and rhw profile-driven inlining in the FDO scenario.

[davidxl]

In D125495#3516513, @kazu wrote:

In D125495#3510338, @mingmingl wrote:

In D125495#3510137, @davidxl wrote:

It might be worth annotating the early inliner for PGO as well.

To confirm if I'm understanding it correctly, you mean we should annotate the "early" (i.e., not profile-driven) information for those inline transformations that are are made without PGO profiles [1].

Ask since the early inliner is annotated with prelink/postlink information (same line around line 630 by construction of ModuleInlinerWrapperPass instance)

I am not David, but yes, it would be nice to be able to differentiate the early inlining and rhw profile-driven inlining in the FDO scenario.

Early-inline is a separate phase used in FDO only (both profile gen and profile-use). Yes, they are not using profile data.

llvm/lib/Analysis/InlineAdvisor.cpp

60

Probably changing the option name to be 'annotate-inline-phase'.

110

There are two dimensions information for annotation: one is the phase, and the other is the inline driver.

For inline phase we have:

Early (FDO only), SampleLoader (AFDO, non thinLTO), SampleLoaderPrelink(AFDO, thinLTO), SampleLoaderPostlink (AFDO, thinLTO), mainInline (main inline phase, non ThinLTO), preLinkInl (thinLTO), postLinkInl(ThinLTO), always ...

For inliner flavors, we have:

BU (bottom up SCC inliner), TD (towndown SCC inliner -- used by SamplerLoaderOnly), ML (ML based inliner), and PriorityBased (module inliner).

To combine them together, we can have something like:

early-BU
sample-TD
sample-prelink-TD
sample-postlink-TD
prelink-BU
postlink-BU
main-BU
main-ML
main-priority
prelink-priority

etc.

[modimo]

In D125495#3510338, @mingmingl wrote:

Re replay inline, the pass name for replay inline advices will remain the same with -annotate-inline-lto-phase on -> the replay inline advisor class implementation remain the same before and after this change.
For my understanding, is the replay inline advisor used for development (conveniently inline the functions according to an external file)? Is the replay inline advisor used for production build?

I've been only using it for development as a way to stabilize inlining when comparing compiler changes that impact function size. That way, we can better examine the impact of the change rather than the resulting inlining differences. It can certainly be used in production but I don't know of a user doing so.

In D125495#3510215, @wenlei wrote:

there's also an always inline pass (AlwaysInliner.cpp).

cc @modimo in case this affects inline replay as well. theoretically replay can be more accurate leveraging accurate pass info.

Replay currently is tied to an entire pass, so if replay is specified for the CGSCC inliner all invocations will perform replay. Importantly, there's no distinction between which instance of the CGSCC inliner the replay remarks came from or uses--I've been manually controlling post-LTO/pre-LTO by changing where the replay/remark flag is passed. With lto phase and inliner name tagging on the remarks generated, replay can now know which exact pass to apply them on and theoretically generate better results.

[mingmingl]

Add struct InlineAdvisorParams.

• Ctor of inliner advisor provides this struct with two dimensional information.
• Inline advice instances query this struct to get the annotation information.

[mingmingl]

The patch annotates SampleProfile inline, as well as the inline passes that talks to DefaultInlineAdvisor to get an advice (CGSCC, ModuleInline, ReplayInline).

Some rationale regarding why not all inline related passes are annotated:

1. The inline advisor interface exposes ORE instance (OptimizationRemarkEmitter) and four interfaces (like recordInliningImpl) to derived classes (protected methods and fields); so derived classes can decide what messages to emit and the pass name to use. From this perspective, non-trivial refactors and changes to existing interface class are necessary to "revoke" the derived classes' access to ORE instance.

2. Relatedly, AlwaysInliner and MLInliner use ORE instance without talking to InlineAdvisor interface (similar to sample-profile pass), which is actually much simpler to modify (that could be done in separate patches)

[mingmingl]

Just fwiw, the sample-profile annotation and default-inline-advisor annotation are orthogonal and gated by two different options. I'm willing to move either one into another patch if a smaller patch is easier to review :-)

[mingmingl]

resolve the following compile failure and run ninja check-all to verify

external/llvm-project/llvm/lib/Analysis/InlineAdvisor.cpp:562:31: error: no member named 'AlwaysInliner' in 'llvm::InlineAdvisorContext'; did you mean 'EarlyInliner'?
  case (InlineAdvisorContext::AlwaysInliner):
        ~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~
                              EarlyInliner

[davidxl]

I suggest splitting out the LTOPhase passing interface changes into a different patch. That one is more mechanical. After which this patch can be greatly simplified.

llvm/include/llvm/Analysis/InlineAdvisor.h

43

Is it possible to define sample loader inliner here too ?

44

AlwaysInline should probably be defined here too.

197

a inline --> an inline

198

If we split out AlwaysInline, then it will be less ambiguous.

llvm/include/llvm/Passes/PassBuilder.h

220

This change can be split out into an independent patch.

llvm/lib/Analysis/InlineAdvisor.cpp

568

In what case it is unspecified?

[mingmingl]

This patch is not ready for review (D126833 is the parent revision, and I'll need to merge D126824)

Reply the previous comments to continue the discussion before revising this patch.

thanks for reviews and comments!

llvm/include/llvm/Analysis/InlineAdvisor.h

43

Done in D126833.

p.s. originally thought sample profile inliner owns an ORE (OptimizationRemarkEmitter) to emit remarks (i.e., without using InlineAdvisor or InlineAdvice). Now make Context class general so all inline passes could provide this context information.

198

If I understand correctly, you mean the MandatoryInlineAdvice (given out by CGSCC inliner pass) [1] should be annotated with always-inline (which is indeed much clearer)

[1] https://github.com/llvm/llvm-project/blob/2dfe41944658a006dc3f89316b1448cfec0131c6/llvm/lib/Analysis/InlineAdvisor.cpp#L612-L617

llvm/include/llvm/Passes/PassBuilder.h

220

To confirm my understanding, is this independent patch supposed to provide InlineAdvisorParams when building the pipeline?

If yes, it sounds like this patch could be reduced to change derived classes (e.g., DefaultInlineAdvisor`) to make use of Optional<InlineAdvisorParam>, but still no-op since pipeline provides NoneType::None.

(keep this comment open)

llvm/lib/Analysis/InlineAdvisor.cpp

110

Introduced struct InlineAdvisorParams so it could convey dimensional information to InlineAdvisor.

As shown in regression tests, the current annotation looks similar to the examples, yet without 1) or 2) for simplicity.

1. TD/BU information is not added.
2. priority information is not added.

Presumably, compiler users could deduce TD/BU from the pass name (sample-profile, CGSCC), and priority information from the compiler options (that they specify on their own).

568

On a second thought UnspecifiedInlinerContext shouldn't exist since InlineAdvisorParams is of type Optional.

In other words, if context isn't specified, InlineAdvisorParams should be NoneType::None.

Removed this entry.

[mingmingl]

Sync base to head.

[kazu]

LGTM with a minor comment fix. Thanks!

llvm/lib/Analysis/InlineAdvisor.cpp

63

Insert 'and' like ModuleInliner, and ReplayInliner.

[mingmingl]

Update the description of option --annotate-inline-phase.

[mingmingl]

In D125495#3607234, @kazu wrote:

LGTM with a minor comment fix. Thanks!

Done. Also rephrase it to reflect the updated meaning (from LTO to {LTO, pass}). Thanks!

llvm/include/llvm/Analysis/InlineAdvisor.h

 ///
 /// - Development mode, for training new models.
 /// In this mode, we trade off runtime performance for flexibility. This mode
 /// requires the full C Tensorflow API library, and evaluates models
 /// dynamically. This mode also permits generating training logs, for offline
 /// training.
 enum class InliningAdvisorMode : int { Default, Release, Development };
 
+enum class InlineAdvisorContext : int {
+  EarlyInliner,
+  CGSCCInliner,
+  ModuleInliner,
+  ReplayCGSCCInliner,
+  ReplaySampleProfileInliner,
+  UnspecifiedInlinerContext,
+};
+
+/// Provides the context information when an instance of inline advisor is
+/// constructed.
+struct InlineAdvisorParams {
+  ThinOrFullLTOPhase LTOPhase;
+
+  InlineAdvisorContext Context;
+};
+
 class InlineAdvisor;
 /// Capture state between an inlining decision having had been made, and
 /// its impact being observable. When collecting model training data, this
 /// allows recording features/decisions/partial reward data sets.
 ///
 /// Derivations of this type are expected to be tightly coupled with their
 /// InliningAdvisors. The base type implements the minimal contractual
 /// obligations.
public:
   /// to prepare for a partial update, based on the optional SCC.
   virtual void onPassExit(LazyCallGraph::SCC *SCC = nullptr) {}
 
   /// Support for printer pass
   virtual void print(raw_ostream &OS) const {
     OS << "Unimplemented InlineAdvisor print\n";
   }
 
+  /// IMPORTANT:
+  ///
+  /// 1. Pass name is annotated only when `--annotate-inline-phase` is true and
+  /// inline advisor constructor provides InlineAdvisorParams.
+  /// 2. When annotated, The annotated inline pass name indicates the pass
+  /// instance where inline decision is made; however the specific pass instance
+  /// may not be the accurate reason.
+  ///    For example, a inline decision in postlink CGSCC pass could be a result
+  ///    of applying heuristics, but could be a result of mandatory inline as
+  ///    well.
+  std::string getAnnotatedInlinePassName();
+
 protected:
-  InlineAdvisor(Module &M, FunctionAnalysisManager &FAM);
+  InlineAdvisor(Module &M, FunctionAnalysisManager &FAM,
+                Optional<InlineAdvisorParams> IAP = NoneType::None);
   virtual std::unique_ptr<InlineAdvice> getAdviceImpl(CallBase &CB) = 0;
   virtual std::unique_ptr<InlineAdvice> getMandatoryAdvice(CallBase &CB,
                                                            bool Advice);
 
   Module &M;
   FunctionAnalysisManager &FAM;
+  const Optional<InlineAdvisorParams> IAP;
   std::unique_ptr<ImportedFunctionsInliningStatistics> ImportedFunctionsStats;
 
   enum class MandatoryInliningKind { NotMandatory, Always, Never };
 
   static MandatoryInliningKind getMandatoryKind(CallBase &CB,
                                                 FunctionAnalysisManager &FAM,
                                                 OptimizationRemarkEmitter &ORE);
 
   OptimizationRemarkEmitter &getCallerORE(CallBase &CB);
 
 private:
   friend class InlineAdvice;
 };
 
 /// The default (manual heuristics) implementation of the InlineAdvisor. This
 /// implementation does not need to keep state between inliner pass runs, and is
 /// reusable as-is for inliner pass test scenarios, as well as for regular use.
 class DefaultInlineAdvisor : public InlineAdvisor {
 public:
   DefaultInlineAdvisor(Module &M, FunctionAnalysisManager &FAM,
-                       InlineParams Params)
-      : InlineAdvisor(M, FAM), Params(Params) {}
+                       InlineParams Params, InlineAdvisorParams IAP)
+      : InlineAdvisor(M, FAM, IAP), Params(Params) {}
 
 private:
   std::unique_ptr<InlineAdvice> getAdviceImpl(CallBase &CB) override;
 
   InlineParams Params;
 };
 
 /// The InlineAdvisorAnalysis is a module pass because the InlineAdvisor
 /// needs to capture state right before inlining commences over a module.
 class InlineAdvisorAnalysis : public AnalysisInfoMixin<InlineAdvisorAnalysis> {
 public:
   static AnalysisKey Key;
   InlineAdvisorAnalysis() = default;
   struct Result {
     Result(Module &M, ModuleAnalysisManager &MAM) : M(M), MAM(MAM) {}
     bool invalidate(Module &, const PreservedAnalyses &PA,
                     ModuleAnalysisManager::Invalidator &) {
       // Check whether the analysis has been explicitly invalidated. Otherwise,
       // it's stateless and remains preserved.
       auto PAC = PA.getChecker<InlineAdvisorAnalysis>();
       return !PAC.preservedWhenStateless();
     }
     bool tryCreate(InlineParams Params, InliningAdvisorMode Mode,
-                   const ReplayInlinerSettings &ReplaySettings);
+                   const ReplayInlinerSettings &ReplaySettings,
+                   InlineAdvisorParams IAP);
     InlineAdvisor *getAdvisor() const { return Advisor.get(); }
 
   private:
     Module &M;
     ModuleAnalysisManager &MAM;
     std::unique_ptr<InlineAdvisor> Advisor;
   };
 

llvm/include/llvm/Analysis/ReplayInlineAdvisor.h

struct ReplayInlinerSettings {
   Scope ReplayScope;
   Fallback ReplayFallback;
   CallSiteFormat ReplayFormat;
 };
 
 /// Get call site location as a string with the given format
 std::string formatCallSiteLocation(DebugLoc DLoc, const CallSiteFormat &Format);
 
-std::unique_ptr<InlineAdvisor> getReplayInlineAdvisor(
-    Module &M, FunctionAnalysisManager &FAM, LLVMContext &Context,
+std::unique_ptr<InlineAdvisor>
+getReplayInlineAdvisor(Module &M, FunctionAnalysisManager &FAM,
+                       LLVMContext &Context,
                        std::unique_ptr<InlineAdvisor> OriginalAdvisor,
-    const ReplayInlinerSettings &ReplaySettings, bool EmitRemarks);
+                       const ReplayInlinerSettings &ReplaySettings,
+                       bool EmitRemarks, InlineAdvisorParams IAP);
 
 /// Replay inline advisor that uses optimization remarks from inlining of
 /// previous build to guide current inlining. This is useful for inliner tuning.
 class ReplayInlineAdvisor : public InlineAdvisor {
 public:
   ReplayInlineAdvisor(Module &M, FunctionAnalysisManager &FAM,
                       LLVMContext &Context,
                       std::unique_ptr<InlineAdvisor> OriginalAdvisor,
                       const ReplayInlinerSettings &ReplaySettings,
-                      bool EmitRemarks);
+                      bool EmitRemarks, InlineAdvisorParams IAP);
   std::unique_ptr<InlineAdvice> getAdviceImpl(CallBase &CB) override;
   bool areReplayRemarksLoaded() const { return HasReplayRemarks; }
 
 private:
   bool hasInlineAdvice(Function &F) const {
     return (ReplaySettings.ReplayScope ==
             ReplayInlinerSettings::Scope::Module) ||
            CallersToReplay.contains(F.getName());

llvm/include/llvm/Passes/PassBuilder.h

public:
   /// transformations. It also cannot generally be run repeatedly on a module
   /// without potentially seriously regressing either runtime performance of
   /// the code or serious code size growth.
   ///
   /// Note that \p Level cannot be `O0` here. The pipelines produced are
   /// only intended for use when attempting to optimize code. If frontends
   /// require some transformations for semantic reasons, they should explicitly
   /// build them.
-  ModulePassManager buildModuleOptimizationPipeline(OptimizationLevel Level,
-                                                    bool LTOPreLink = false);
+  ModulePassManager
+  buildModuleOptimizationPipeline(OptimizationLevel Level,
+                                  ThinOrFullLTOPhase LTOPhase);
 
   /// Build a per-module default optimization pipeline.
   ///
   /// This provides a good default optimization pipeline for per-module
   /// optimization and code generation without any link-time optimization. It
   /// typically correspond to frontend "-O[123]" options for optimization
   /// levels \c O1, \c O2 and \c O3 resp.
   ///
Error parseFunctionPassPipeline(FunctionPassManager &FPM,
                                   ArrayRef<PipelineElement> Pipeline);
   Error parseCGSCCPassPipeline(CGSCCPassManager &CGPM,
                                ArrayRef<PipelineElement> Pipeline);
   Error parseModulePassPipeline(ModulePassManager &MPM,
                                 ArrayRef<PipelineElement> Pipeline);
 
   void addPGOInstrPasses(ModulePassManager &MPM, OptimizationLevel Level,
                          bool RunProfileGen, bool IsCS, std::string ProfileFile,
-                         std::string ProfileRemappingFile);
+                         std::string ProfileRemappingFile,
+                         ThinOrFullLTOPhase LTOPhase);
   void invokePeepholeEPCallbacks(FunctionPassManager &, OptimizationLevel);
 
   // Extension Point callbacks
   SmallVector<std::function<void(FunctionPassManager &, OptimizationLevel)>, 2>
       PeepholeEPCallbacks;
   SmallVector<std::function<void(LoopPassManager &, OptimizationLevel)>, 2>
       LateLoopOptimizationsEPCallbacks;
   SmallVector<std::function<void(LoopPassManager &, OptimizationLevel)>, 2>

llvm/include/llvm/Transforms/IPO/Inliner.h

 /// It should be noted that the legacy inliners do considerably more than this
 /// inliner pass does. They provide logic for manually merging allocas, and
 /// doing considerable DCE including the DCE of dead functions. This pass makes
 /// every attempt to be simpler. DCE of functions requires complex reasoning
 /// about comdat groups, etc. Instead, it is expected that other more focused
 /// passes be composed to achieve the same end result.
 class InlinerPass : public PassInfoMixin<InlinerPass> {
 public:
-  InlinerPass(bool OnlyMandatory = false) : OnlyMandatory(OnlyMandatory) {}
+  InlinerPass(bool OnlyMandatory = false,
+              ThinOrFullLTOPhase LTOPhase = ThinOrFullLTOPhase::None)
+      : OnlyMandatory(OnlyMandatory), LTOPhase(LTOPhase) {}
   InlinerPass(InlinerPass &&Arg) = default;
 
   PreservedAnalyses run(LazyCallGraph::SCC &C, CGSCCAnalysisManager &AM,
                         LazyCallGraph &CG, CGSCCUpdateResult &UR);
 
   void printPipeline(raw_ostream &OS,
                      function_ref<StringRef(StringRef)> MapClassName2PassName);
 
 private:
   InlineAdvisor &getAdvisor(const ModuleAnalysisManagerCGSCCProxy::Result &MAM,
                             FunctionAnalysisManager &FAM, Module &M);
   std::unique_ptr<InlineAdvisor> OwnedAdvisor;
   const bool OnlyMandatory;
+  const ThinOrFullLTOPhase LTOPhase;
 };
 
 /// Module pass, wrapping the inliner pass. This works in conjunction with the
 /// InlineAdvisorAnalysis to facilitate inlining decisions taking into account
 /// module-wide state, that need to keep track of inter-inliner pass runs, for
 /// a given module. An InlineAdvisor is configured and kept alive for the
 /// duration of the ModuleInlinerWrapperPass::run.
 class ModuleInlinerWrapperPass
     : public PassInfoMixin<ModuleInlinerWrapperPass> {
 public:
   ModuleInlinerWrapperPass(
       InlineParams Params = getInlineParams(), bool MandatoryFirst = true,
+      InlineAdvisorParams IAP = {},
       InliningAdvisorMode Mode = InliningAdvisorMode::Default,
       unsigned MaxDevirtIterations = 0);
   ModuleInlinerWrapperPass(ModuleInlinerWrapperPass &&Arg) = default;
 
   PreservedAnalyses run(Module &, ModuleAnalysisManager &);
 
   /// Allow adding more CGSCC passes, besides inlining. This should be called
   /// before run is called, as part of pass pipeline building.
template <class T> void addLateModulePass(T Pass) {
     AfterCGMPM.addPass(std::move(Pass));
   }
 
   void printPipeline(raw_ostream &OS,
                      function_ref<StringRef(StringRef)> MapClassName2PassName);
 
 private:
   const InlineParams Params;
+  const InlineAdvisorParams IAP;
   const InliningAdvisorMode Mode;
   const unsigned MaxDevirtIterations;
   // TODO: Clean this up so we only have one ModulePassManager.
   CGSCCPassManager PM;
   ModulePassManager MPM;
   ModulePassManager AfterCGMPM;
 };
 } // end namespace llvm
 
 #endif // LLVM_TRANSFORMS_IPO_INLINER_H

llvm/include/llvm/Transforms/IPO/ModuleInliner.h

 /// call in every function in the whole module and tries to inline if
 /// profitable. With this module level inliner, it is possible to evaluate more
 /// heuristics in the module level such like PriorityInlineOrder. It can be
 /// tuned with a number of parameters to control what cost model is used and
 /// what tradeoffs are made when making the decision.
 class ModuleInlinerPass : public PassInfoMixin<ModuleInlinerPass> {
 public:
   ModuleInlinerPass(InlineParams Params = getInlineParams(),
-                    InliningAdvisorMode Mode = InliningAdvisorMode::Default)
-      : Params(Params), Mode(Mode){};
+                    InliningAdvisorMode Mode = InliningAdvisorMode::Default,
+                    ThinOrFullLTOPhase LTOPhase = ThinOrFullLTOPhase::None)
+      : Params(Params), Mode(Mode), LTOPhase(LTOPhase){};
   ModuleInlinerPass(ModuleInlinerPass &&Arg) = default;
 
   PreservedAnalyses run(Module &, ModuleAnalysisManager &);
 
 private:
   InlineAdvisor &getAdvisor(const ModuleAnalysisManager &MAM,
                             FunctionAnalysisManager &FAM, Module &M);
   std::unique_ptr<InlineAdvisor> OwnedAdvisor;
   const InlineParams Params;
   const InliningAdvisorMode Mode;
+  const ThinOrFullLTOPhase LTOPhase;
 };
 } // end namespace llvm
 
 #endif // LLVM_TRANSFORMS_IPO_MODULEINLINER_H

llvm/lib/Analysis/InlineAdvisor.cpp

 
 // An integer used to limit the cost of inline deferral.  The default negative
 // number tells shouldBeDeferred to only take the secondary cost into account.
 static cl::opt<int>
     InlineDeferralScale("inline-deferral-scale",
                         cl::desc("Scale to limit the cost of inline deferral"),
                         cl::init(2), cl::Hidden);
 
+static cl::opt<bool> AnnotateInlinePhase(
+    "annotate-inline-phase", cl::Hidden, cl::init(false),
+    cl::desc("If true, annotate inline advisor remarks "
+             "with LTO phase information. Relevant inline instances include "
+             "CGSCCInliner, ModuleInliner, ReplayInliner. Support for missed "
+             "inline remarks needs to be added."));
+
 extern cl::opt<InlinerFunctionImportStatsOpts> InlinerFunctionImportStats;
 
 namespace {
 using namespace llvm::ore;
 class MandatoryInlineAdvice : public InlineAdvice {
 public:
   MandatoryInlineAdvice(InlineAdvisor *Advisor, CallBase &CB,
                         OptimizationRemarkEmitter &ORE,
private:
   }
 
   void recordUnattemptedInliningImpl() override {
     assert(!IsInliningRecommended && "Expected to attempt inlining");
   }
 };
 } // namespace
 
+// FIXME: Consider using getAnnotatedInlinePassName for missed inlines.
 void DefaultInlineAdvice::recordUnsuccessfulInliningImpl(
     const InlineResult &Result) {
   using namespace ore;
   llvm::setInlineRemark(*OriginalCB, std::string(Result.getFailureReason()) +
                                          "; " + inlineCostStr(*OIC));
   ORE.emit([&]() {
     return OptimizationRemarkMissed(DEBUG_TYPE, "NotInlined", DLoc, Block)
            << "'" << NV("Callee", Callee) << "' is not inlined into '"
            << NV("Caller", Caller)
            << "': " << NV("Reason", Result.getFailureReason());
   });
 }
 
 void DefaultInlineAdvice::recordInliningWithCalleeDeletedImpl() {
   if (EmitRemarks)
-    emitInlinedIntoBasedOnCost(ORE, DLoc, Block, *Callee, *Caller, *OIC);
+    emitInlinedIntoBasedOnCost(ORE, DLoc, Block, *Callee, *Caller, *OIC,
+                               /* ForProfileContext= */ false,
+                               Advisor->getAnnotatedInlinePassName().c_str());
 }
 
 void DefaultInlineAdvice::recordInliningImpl() {
   if (EmitRemarks)
-    emitInlinedIntoBasedOnCost(ORE, DLoc, Block, *Callee, *Caller, *OIC);
+    emitInlinedIntoBasedOnCost(ORE, DLoc, Block, *Callee, *Caller, *OIC,
+                               /* ForProfileContext= */ false,
+                               Advisor->getAnnotatedInlinePassName().c_str());
 }
 
 llvm::Optional<llvm::InlineCost> static getDefaultInlineAdvice(
     CallBase &CB, FunctionAnalysisManager &FAM, const InlineParams &Params) {
   Function &Caller = *CB.getCaller();
   ProfileSummaryInfo *PSI =
       FAM.getResult<ModuleAnalysisManagerFunctionProxy>(Caller)
           .getCachedResult<ProfileSummaryAnalysis>(
return llvm::shouldInline(
       Params.EnableDeferral.getValueOr(EnableInlineDeferral));
 }
 
 std::unique_ptr<InlineAdvice>
 DefaultInlineAdvisor::getAdviceImpl(CallBase &CB) {
   auto OIC = getDefaultInlineAdvice(CB, FAM, Params);
   return std::make_unique<DefaultInlineAdvice>(
       this, CB, OIC,
-      FAM.getResult<OptimizationRemarkEmitterAnalysis>(*CB.getCaller()));
+      FAM.getResult<OptimizationRemarkEmitterAnalysis>(*CB.getCaller()),
+      /* EmitRemarks */ true);
 }
 
 InlineAdvice::InlineAdvice(InlineAdvisor *Advisor, CallBase &CB,
                            OptimizationRemarkEmitter &ORE,
                            bool IsInliningRecommended)
     : Advisor(Advisor), Caller(CB.getCaller()), Callee(CB.getCalledFunction()),
       DLoc(CB.getDebugLoc()), Block(CB.getParent()), ORE(ORE),
       IsInliningRecommended(IsInliningRecommended) {}
void InlineAdvice::recordInliningWithCalleeDeleted() {
   recordInlineStatsIfNeeded();
   recordInliningWithCalleeDeletedImpl();
 }
 
 AnalysisKey InlineAdvisorAnalysis::Key;
 
 bool InlineAdvisorAnalysis::Result::tryCreate(
     InlineParams Params, InliningAdvisorMode Mode,
-    const ReplayInlinerSettings &ReplaySettings) {
+    const ReplayInlinerSettings &ReplaySettings, InlineAdvisorParams IAP) {
   auto &FAM = MAM.getResult<FunctionAnalysisManagerModuleProxy>(M).getManager();
   switch (Mode) {
   case InliningAdvisorMode::Default:
     LLVM_DEBUG(dbgs() << "Using default inliner heuristic.\n");
-    Advisor.reset(new DefaultInlineAdvisor(M, FAM, Params));
+    Advisor.reset(new DefaultInlineAdvisor(M, FAM, Params, IAP));
     // Restrict replay to default advisor, ML advisors are stateful so
     // replay will need augmentations to interleave with them correctly.
     if (!ReplaySettings.ReplayFile.empty()) {
       Advisor = llvm::getReplayInlineAdvisor(M, FAM, M.getContext(),
                                              std::move(Advisor), ReplaySettings,
-                                             /* EmitRemarks =*/true);
+                                             /* EmitRemarks =*/true, IAP);
     }
     break;
   case InliningAdvisorMode::Development:
 #ifdef LLVM_HAVE_TF_API
     LLVM_DEBUG(dbgs() << "Using development-mode inliner policy.\n");
     Advisor =
         llvm::getDevelopmentModeAdvisor(M, MAM, [&FAM, Params](CallBase &CB) {
           auto OIC = getDefaultInlineAdvice(CB, FAM, Params);
llvm::emitInlinedInto(
       [&](OptimizationRemark &Remark) {
         if (ForProfileContext)
           Remark << " to match profiling context";
         Remark << " with " << IC;
       },
       PassName);
 }
 
-InlineAdvisor::InlineAdvisor(Module &M, FunctionAnalysisManager &FAM)
-    : M(M), FAM(FAM) {
+InlineAdvisor::InlineAdvisor(Module &M, FunctionAnalysisManager &FAM,
+                             Optional<InlineAdvisorParams> IAP)
+    : M(M), FAM(FAM), IAP(IAP) {
   if (InlinerFunctionImportStats != InlinerFunctionImportStatsOpts::No) {
     ImportedFunctionsStats =
         std::make_unique<ImportedFunctionsInliningStatistics>();
     ImportedFunctionsStats->setModuleInfo(M);
   }
 }
 
 InlineAdvisor::~InlineAdvisor() {
   if (ImportedFunctionsStats) {
     assert(InlinerFunctionImportStats != InlinerFunctionImportStatsOpts::No);
     ImportedFunctionsStats->dump(InlinerFunctionImportStats ==
                                  InlinerFunctionImportStatsOpts::Verbose);
   }
 }
 
 std::unique_ptr<InlineAdvice> InlineAdvisor::getMandatoryAdvice(CallBase &CB,
                                                                 bool Advice) {
   return std::make_unique<MandatoryInlineAdvice>(this, CB, getCallerORE(CB),
                                                  Advice);
 }
 
+static inline const char *getLTOPhase(ThinOrFullLTOPhase LTOPhase) {
+  switch (LTOPhase) {
+  case (ThinOrFullLTOPhase::None):
+    return "main";
+  case (ThinOrFullLTOPhase::ThinLTOPreLink):
+  case (ThinOrFullLTOPhase::FullLTOPreLink):
+    return "prelink";
+  case (ThinOrFullLTOPhase::ThinLTOPostLink):
+  case (ThinOrFullLTOPhase::FullLTOPostLink):
+    return "postlink";
+  }
+  llvm_unreachable("unreachable");
+}
+
+static inline const char *getInlineAdvisorContext(InlineAdvisorContext IAC) {
+  switch (IAC) {
+  case (InlineAdvisorContext::EarlyInliner):
+    return "early-inline";
+  case (InlineAdvisorContext::CGSCCInliner):
+    return "cgscc-inline";
+  case (InlineAdvisorContext::ModuleInliner):
+    return "module-inline";
+  case (InlineAdvisorContext::ReplayCGSCCInliner):
+    return "replay-cgscc-inline";
+  case (InlineAdvisorContext::ReplaySampleProfileInliner):
+    return "replay-sample-profile-inline";
+  case (InlineAdvisorContext::UnspecifiedInlinerContext):
+    return DEBUG_TYPE;
+  }
+  llvm_unreachable("unreachable");
+}
+
+std::string InlineAdvisor::getAnnotatedInlinePassName() {
+  if (!AnnotateInlinePhase || !IAP.hasValue())
+    return DEBUG_TYPE;
+
+  // IAP is constant and initialized in ctor, so compute the annotated name only
+  // once.
+  static const std::string PassName =
+      std::string(getLTOPhase(IAP.getValue().LTOPhase)) + "-" +
+      std::string(getInlineAdvisorContext(IAP.getValue().Context));
+
+  return PassName;
+}
+
 InlineAdvisor::MandatoryInliningKind
 InlineAdvisor::getMandatoryKind(CallBase &CB, FunctionAnalysisManager &FAM,
                                 OptimizationRemarkEmitter &ORE) {
   auto &Callee = *CB.getCalledFunction();
 
   auto GetTLI = [&](Function &F) -> const TargetLibraryInfo & {
     return FAM.getResult<TargetLibraryAnalysis>(F);
   };

llvm/lib/Analysis/ReplayInlineAdvisor.cpp

 
 using namespace llvm;
 
 #define DEBUG_TYPE "replay-inline"
 
 ReplayInlineAdvisor::ReplayInlineAdvisor(
     Module &M, FunctionAnalysisManager &FAM, LLVMContext &Context,
     std::unique_ptr<InlineAdvisor> OriginalAdvisor,
-    const ReplayInlinerSettings &ReplaySettings, bool EmitRemarks)
-    : InlineAdvisor(M, FAM), OriginalAdvisor(std::move(OriginalAdvisor)),
+    const ReplayInlinerSettings &ReplaySettings, bool EmitRemarks,
+    InlineAdvisorParams IAP)
+    : InlineAdvisor(M, FAM, IAP), OriginalAdvisor(std::move(OriginalAdvisor)),
       ReplaySettings(ReplaySettings), EmitRemarks(EmitRemarks) {
 
   auto BufferOrErr = MemoryBuffer::getFileOrSTDIN(ReplaySettings.ReplayFile);
   std::error_code EC = BufferOrErr.getError();
   if (EC) {
     Context.emitError("Could not open remarks file: " + EC.message());
     return;
   }
for (; !LineIt.is_at_eof(); ++LineIt) {
     InlineSitesFromRemarks[Combined] = IsPositiveRemark;
     if (ReplaySettings.ReplayScope == ReplayInlinerSettings::Scope::Function)
       CallersToReplay.insert(Caller);
   }
 
   HasReplayRemarks = true;
 }
 
-std::unique_ptr<InlineAdvisor> llvm::getReplayInlineAdvisor(
-    Module &M, FunctionAnalysisManager &FAM, LLVMContext &Context,
+std::unique_ptr<InlineAdvisor>
+llvm::getReplayInlineAdvisor(Module &M, FunctionAnalysisManager &FAM,
+                             LLVMContext &Context,
                              std::unique_ptr<InlineAdvisor> OriginalAdvisor,
-    const ReplayInlinerSettings &ReplaySettings, bool EmitRemarks) {
+                             const ReplayInlinerSettings &ReplaySettings,
+                             bool EmitRemarks, InlineAdvisorParams IAP) {
   auto Advisor = std::make_unique<ReplayInlineAdvisor>(
-      M, FAM, Context, std::move(OriginalAdvisor), ReplaySettings, EmitRemarks);
+      M, FAM, Context, std::move(OriginalAdvisor), ReplaySettings, EmitRemarks,
+      IAP);
   if (!Advisor->areReplayRemarksLoaded())
     Advisor.reset();
   return Advisor;
 }
 
 std::unique_ptr<InlineAdvice> ReplayInlineAdvisor::getAdviceImpl(CallBase &CB) {
   assert(HasReplayRemarks);
 

llvm/lib/Passes/PassBuilderPipelines.cpp

 void PassBuilder::addRequiredLTOPreLinkPasses(ModulePassManager &MPM) {
   MPM.addPass(CanonicalizeAliasesPass());
   MPM.addPass(NameAnonGlobalPass());
 }
 
 void PassBuilder::addPGOInstrPasses(ModulePassManager &MPM,
                                     OptimizationLevel Level, bool RunProfileGen,
                                     bool IsCS, std::string ProfileFile,
-                                    std::string ProfileRemappingFile) {
+                                    std::string ProfileRemappingFile,
+                                    ThinOrFullLTOPhase LTOPhase) {
   assert(Level != OptimizationLevel::O0 && "Not expecting O0 here!");
   if (!IsCS && !DisablePreInliner) {
     InlineParams IP;
 
     IP.DefaultThreshold = PreInlineThreshold;
 
     // FIXME: The hint threshold has the same value used by the regular inliner
     // when not optimzing for size. This should probably be lowered after
     // performance testing.
     // FIXME: this comment is cargo culted from the old pass manager, revisit).
     IP.HintThreshold = Level.isOptimizingForSize() ? PreInlineThreshold : 325;
-    ModuleInlinerWrapperPass MIWP(IP);
+    ModuleInlinerWrapperPass MIWP(
+        IP, /* MandatoryFirst */ true,
+        InlineAdvisorParams{LTOPhase, InlineAdvisorContext::EarlyInliner});
     CGSCCPassManager &CGPipeline = MIWP.getPM();
 
     FunctionPassManager FPM;
     FPM.addPass(SROAPass());
     FPM.addPass(EarlyCSEPass());    // Catch trivial redundancies.
     FPM.addPass(SimplifyCFGPass(SimplifyCFGOptions().convertSwitchRangeToICmp(
         true)));                    // Merge & remove basic blocks.
     FPM.addPass(InstCombinePass()); // Combine silly sequences.
PassBuilder::buildInlinerPipeline(OptimizationLevel Level,
   // prologue / epilogue.
   if (Phase == ThinOrFullLTOPhase::ThinLTOPreLink && PGOOpt &&
       PGOOpt->Action == PGOOptions::SampleUse)
     IP.HotCallSiteThreshold = 0;
 
   if (PGOOpt)
     IP.EnableDeferral = EnablePGOInlineDeferral;
 
-  ModuleInlinerWrapperPass MIWP(IP, PerformMandatoryInliningsFirst,
-                                UseInlineAdvisor, MaxDevirtIterations);
+  ModuleInlinerWrapperPass MIWP(
+      IP, PerformMandatoryInliningsFirst,
+      InlineAdvisorParams{Phase, InlineAdvisorContext::CGSCCInliner},
 
   // Require the GlobalsAA analysis for the module so we can query it within
   // the CGSCC pipeline.
   MIWP.addModulePass(RequireAnalysisPass<GlobalsAA, Module>());
   // Invalidate AAManager so it can be recreated and pick up the newly available
   // GlobalsAA.
   MIWP.addModulePass(
       createModuleToFunctionPassAdaptor(InvalidateAnalysisPass<AAManager>()));
PassBuilder::buildModuleInlinerPipeline(OptimizationLevel Level,
   // The inline deferral logic is used to avoid losing some
   // inlining chance in future. It is helpful in SCC inliner, in which
   // inlining is processed in bottom-up order.
   // While in module inliner, the inlining order is a priority-based order
   // by default. The inline deferral is unnecessary there. So we disable the
   // inline deferral logic in module inliner.
   IP.EnableDeferral = false;
 
-  MPM.addPass(ModuleInlinerPass(IP, UseInlineAdvisor));
+  MPM.addPass(ModuleInlinerPass(IP, UseInlineAdvisor, Phase));
 
   MPM.addPass(createModuleToFunctionPassAdaptor(
       buildFunctionSimplificationPipeline(Level, Phase),
       PTO.EagerlyInvalidateAnalyses));
 
   MPM.addPass(createModuleToPostOrderCGSCCPassAdaptor(
       CoroSplitPass(Level != OptimizationLevel::O0)));
 
PassBuilder::buildModuleSimplificationPipeline(OptimizationLevel Level,
 
   // Add all the requested passes for instrumentation PGO, if requested.
   if (PGOOpt && Phase != ThinOrFullLTOPhase::ThinLTOPostLink &&
       (PGOOpt->Action == PGOOptions::IRInstr ||
        PGOOpt->Action == PGOOptions::IRUse)) {
     addPGOInstrPasses(MPM, Level,
                       /* RunProfileGen */ PGOOpt->Action == PGOOptions::IRInstr,
                       /* IsCS */ false, PGOOpt->ProfileFile,
-                      PGOOpt->ProfileRemappingFile);
+                      PGOOpt->ProfileRemappingFile, Phase);
     MPM.addPass(PGOIndirectCallPromotion(false, false));
   }
   if (PGOOpt && Phase != ThinOrFullLTOPhase::ThinLTOPostLink &&
       PGOOpt->CSAction == PGOOptions::CSIRInstr)
     MPM.addPass(PGOInstrumentationGenCreateVar(PGOOpt->CSProfileGenFile));
 
   // Synthesize function entry counts for non-PGO compilation.
   if (EnableSyntheticCounts && !PGOOpt)
void PassBuilder::addVectorPasses(OptimizationLevel Level,
   FPM.addPass(AlignmentFromAssumptionsPass());
 
   if (IsFullLTO)
     FPM.addPass(InstCombinePass());
 }
 
 ModulePassManager
 PassBuilder::buildModuleOptimizationPipeline(OptimizationLevel Level,
-                                             bool LTOPreLink) {
+                                             ThinOrFullLTOPhase LTOPhase) {
+  const bool LTOPreLink = (LTOPhase == ThinOrFullLTOPhase::ThinLTOPreLink ||
+                           LTOPhase == ThinOrFullLTOPhase::FullLTOPreLink);
   ModulePassManager MPM;
 
   // Optimize globals now that the module is fully simplified.
   MPM.addPass(GlobalOptPass());
   MPM.addPass(GlobalDCEPass());
 
   // Run partial inlining pass to partially inline functions that have
   // large bodies.
   if (RunPartialInlining)
     MPM.addPass(PartialInlinerPass());
PassBuilder::buildModuleOptimizationPipeline(OptimizationLevel Level,
   // Do a post inline PGO instrumentation and use pass. This is a context
   // sensitive PGO pass. We don't want to do this in LTOPreLink phrase as
   // cross-module inline has not been done yet. The context sensitive
   // instrumentation is after all the inlines are done.
   if (!LTOPreLink && PGOOpt) {
     if (PGOOpt->CSAction == PGOOptions::CSIRInstr)
       addPGOInstrPasses(MPM, Level, /* RunProfileGen */ true,
                         /* IsCS */ true, PGOOpt->CSProfileGenFile,
-                        PGOOpt->ProfileRemappingFile);
+                        PGOOpt->ProfileRemappingFile, LTOPhase);
     else if (PGOOpt->CSAction == PGOOptions::CSIRUse)
       addPGOInstrPasses(MPM, Level, /* RunProfileGen */ false,
                         /* IsCS */ true, PGOOpt->ProfileFile,
-                        PGOOpt->ProfileRemappingFile);
+                        PGOOpt->ProfileRemappingFile, LTOPhase);
   }
 
   // Re-compute GlobalsAA here prior to function passes. This is particularly
   // useful as the above will have inlined, DCE'ed, and function-attr
   // propagated everything. We should at this point have a reasonably minimal
   // and richly annotated call graph. By computing aliasing and mod/ref
   // information for all local globals here, the late loop passes and notably
   // the vectorizer will be able to use them to help recognize vectorizable
PassBuilder::buildPerModuleDefaultPipeline(OptimizationLevel Level,
 
   // Apply module pipeline start EP callback.
   for (auto &C : PipelineStartEPCallbacks)
     C(MPM, Level);
 
   if (PGOOpt && PGOOpt->DebugInfoForProfiling)
     MPM.addPass(createModuleToFunctionPassAdaptor(AddDiscriminatorsPass()));
 
+  const ThinOrFullLTOPhase LTOPhase = LTOPreLink
+                                          ? ThinOrFullLTOPhase::FullLTOPreLink
+                                          : ThinOrFullLTOPhase::None;
   // Add the core simplification pipeline.
-  MPM.addPass(buildModuleSimplificationPipeline(
-      Level, LTOPreLink ? ThinOrFullLTOPhase::FullLTOPreLink
-                        : ThinOrFullLTOPhase::None));
+  MPM.addPass(buildModuleSimplificationPipeline(Level, LTOPhase));
 
   // Now add the optimization pipeline.
-  MPM.addPass(buildModuleOptimizationPipeline(Level, LTOPreLink));
+  MPM.addPass(buildModuleOptimizationPipeline(Level, LTOPhase));
 
   if (PGOOpt && PGOOpt->PseudoProbeForProfiling &&
       PGOOpt->Action == PGOOptions::SampleUse)
     MPM.addPass(PseudoProbeUpdatePass());
 
   // Emit annotation remarks.
   addAnnotationRemarksPass(MPM);
 
ModulePassManager PassBuilder::buildThinLTODefaultPipeline(
   // Force any function attributes we want the rest of the pipeline to observe.
   MPM.addPass(ForceFunctionAttrsPass());
 
   // Add the core simplification pipeline.
   MPM.addPass(buildModuleSimplificationPipeline(
       Level, ThinOrFullLTOPhase::ThinLTOPostLink));
 
   // Now add the optimization pipeline.
-  MPM.addPass(buildModuleOptimizationPipeline(Level));
+  MPM.addPass(buildModuleOptimizationPipeline(
+      Level, ThinOrFullLTOPhase::ThinLTOPostLink));
 
   // Emit annotation remarks.
   addAnnotationRemarksPass(MPM);
 
   return MPM;
 }
 
 ModulePassManager
PassBuilder::buildLTODefaultPipeline(OptimizationLevel Level,
   MPM.addPass(createModuleToFunctionPassAdaptor(std::move(PeepholeFPM),
                                                 PTO.EagerlyInvalidateAnalyses));
 
   // Note: historically, the PruneEH pass was run first to deduce nounwind and
   // generally clean up exception handling overhead. It isn't clear this is
   // valuable as the inliner doesn't currently care whether it is inlining an
   // invoke or a call.
   // Run the inliner now.
-  MPM.addPass(ModuleInlinerWrapperPass(getInlineParamsFromOptLevel(Level)));
+  MPM.addPass(ModuleInlinerWrapperPass(
+      getInlineParamsFromOptLevel(Level),
+      /* MandatoryFirst */ true,
+      InlineAdvisorParams{ThinOrFullLTOPhase::FullLTOPostLink,
+                          InlineAdvisorContext::CGSCCInliner}));
 
   // Optimize globals again after we ran the inliner.
   MPM.addPass(GlobalOptPass());
 
   // Garbage collect dead functions.
   MPM.addPass(GlobalDCEPass());
 
   // If we didn't decide to inline a function, check to see if we can
   // transform it to pass arguments by value instead of by reference.
   MPM.addPass(createModuleToPostOrderCGSCCPassAdaptor(ArgumentPromotionPass()));
 
   FunctionPassManager FPM;
   // The IPO Passes may leave cruft around. Clean up after them.
   FPM.addPass(InstCombinePass());
   invokePeepholeEPCallbacks(FPM, Level);
 
   FPM.addPass(JumpThreadingPass(/*InsertFreezeWhenUnfoldingSelect*/ true));
 
   // Do a post inline PGO instrumentation and use pass. This is a context
   // sensitive PGO pass.
   if (PGOOpt) {
     if (PGOOpt->CSAction == PGOOptions::CSIRInstr)
       addPGOInstrPasses(MPM, Level, /* RunProfileGen */ true,
                         /* IsCS */ true, PGOOpt->CSProfileGenFile,
-                        PGOOpt->ProfileRemappingFile);
+                        PGOOpt->ProfileRemappingFile,
+                        ThinOrFullLTOPhase::FullLTOPostLink);
     else if (PGOOpt->CSAction == PGOOptions::CSIRUse)
       addPGOInstrPasses(MPM, Level, /* RunProfileGen */ false,
                         /* IsCS */ true, PGOOpt->ProfileFile,
-                        PGOOpt->ProfileRemappingFile);
+                        PGOOpt->ProfileRemappingFile,
+                        ThinOrFullLTOPhase::FullLTOPostLink);
   }
 
   // Break up allocas
   FPM.addPass(SROAPass());
 
   // LTO provides additional opportunities for tailcall elimination due to
   // link-time inlining, and visibility of nocapture attribute.
   FPM.addPass(TailCallElimPass());

llvm/lib/Transforms/IPO/Inliner.cpp

if (!IAA) {
     // It should still be possible to run the inliner as a stand-alone SCC pass,
     // for test scenarios. In that case, we default to the
     // DefaultInlineAdvisor, which doesn't need to keep state between SCC pass
     // runs. It also uses just the default InlineParams.
     // In this case, we need to use the provided FAM, which is valid for the
     // duration of the inliner pass, and thus the lifetime of the owned advisor.
     // The one we would get from the MAM can be invalidated as a result of the
     // inliner's activity.
-    OwnedAdvisor =
-        std::make_unique<DefaultInlineAdvisor>(M, FAM, getInlineParams());
+    OwnedAdvisor = std::make_unique<DefaultInlineAdvisor>(
+        M, FAM, getInlineParams(),
+        InlineAdvisorParams{LTOPhase, InlineAdvisorContext::CGSCCInliner});
 
     if (!CGSCCInlineReplayFile.empty())
       OwnedAdvisor = getReplayInlineAdvisor(
           M, FAM, M.getContext(), std::move(OwnedAdvisor),
           ReplayInlinerSettings{CGSCCInlineReplayFile,
                                 CGSCCInlineReplayScope,
                                 CGSCCInlineReplayFallback,
                                 {CGSCCInlineReplayFormat}},
-          /*EmitRemarks=*/true);
+          /*EmitRemarks=*/true,
+          InlineAdvisorParams{LTOPhase,
+                              InlineAdvisorContext::ReplayCGSCCInliner});
 
     return *OwnedAdvisor;
   }
   assert(IAA->getAdvisor() &&
          "Expected a present InlineAdvisorAnalysis also have an "
          "InlineAdvisor initialized");
   return *IAA->getAdvisor();
 }
PreservedAnalyses InlinerPass::run(LazyCallGraph::SCC &InitialC,
   PA.preserve<FunctionAnalysisManagerCGSCCProxy>();
   // We have already invalidated all analyses on modified functions.
   PA.preserveSet<AllAnalysesOn<Function>>();
   return PA;
 }
 
 ModuleInlinerWrapperPass::ModuleInlinerWrapperPass(InlineParams Params,
                                                    bool MandatoryFirst,
+                                                   InlineAdvisorParams IAP,
                                                    InliningAdvisorMode Mode,
                                                    unsigned MaxDevirtIterations)
-    : Params(Params), Mode(Mode), MaxDevirtIterations(MaxDevirtIterations) {
+    : Params(Params), IAP(IAP), Mode(Mode),
+      MaxDevirtIterations(MaxDevirtIterations) {
   // Run the inliner first. The theory is that we are walking bottom-up and so
   // the callees have already been fully optimized, and we want to inline them
   // into the callers so that our optimizations can reflect that.
   // For PreLinkThinLTO pass, we disable hot-caller heuristic for sample PGO
   // because it makes profile annotation in the backend inaccurate.
   if (MandatoryFirst)
     PM.addPass(InlinerPass(/*OnlyMandatory*/ true));
   PM.addPass(InlinerPass());
 }
 
 PreservedAnalyses ModuleInlinerWrapperPass::run(Module &M,
                                                 ModuleAnalysisManager &MAM) {
   auto &IAA = MAM.getResult<InlineAdvisorAnalysis>(M);
   if (!IAA.tryCreate(Params, Mode,
                      {CGSCCInlineReplayFile,
                       CGSCCInlineReplayScope,
                       CGSCCInlineReplayFallback,
-                      {CGSCCInlineReplayFormat}})) {
+                      {CGSCCInlineReplayFormat}},
+                     IAP)) {
     M.getContext().emitError(
         "Could not setup Inlining Advisor for the requested "
         "mode and/or options");
     return PreservedAnalyses::all();
   }
 
   // We wrap the CGSCC pipeline in a devirtualization repeater. This will try
   // to detect when we devirtualize indirect calls and iterate the SCC passes

llvm/lib/Transforms/IPO/ModuleInliner.cpp

if (!IAA) {
     // It should still be possible to run the inliner as a stand-alone module
     // pass, for test scenarios. In that case, we default to the
     // DefaultInlineAdvisor, which doesn't need to keep state between module
     // pass runs. It also uses just the default InlineParams. In this case, we
     // need to use the provided FAM, which is valid for the duration of the
     // inliner pass, and thus the lifetime of the owned advisor. The one we
     // would get from the MAM can be invalidated as a result of the inliner's
     // activity.
-    OwnedAdvisor = std::make_unique<DefaultInlineAdvisor>(M, FAM, Params);
+    OwnedAdvisor = std::make_unique<DefaultInlineAdvisor>(
+        M, FAM, Params,
+        InlineAdvisorParams{LTOPhase, InlineAdvisorContext::ModuleInliner});
 
     return *OwnedAdvisor;
   }
   assert(IAA->getAdvisor() &&
          "Expected a present InlineAdvisorAnalysis also have an "
          "InlineAdvisor initialized");
   return *IAA->getAdvisor();
 }
 
 static bool isKnownLibFunction(Function &F, TargetLibraryInfo &TLI) {
   LibFunc LF;
 
   // Either this is a normal library function or a "vectorizable"
   // function.  Not using the VFDatabase here because this query
   // is related only to libraries handled via the TLI.
   return TLI.getLibFunc(F, LF) ||
          TLI.isKnownVectorFunctionInLibrary(F.getName());
 }
 
 PreservedAnalyses ModuleInlinerPass::run(Module &M,
                                          ModuleAnalysisManager &MAM) {
   LLVM_DEBUG(dbgs() << "---- Module Inliner is Running ---- \n");
 
   auto &IAA = MAM.getResult<InlineAdvisorAnalysis>(M);
-  if (!IAA.tryCreate(Params, Mode, {})) {
+  if (!IAA.tryCreate(
+          Params, Mode, {},
+          InlineAdvisorParams{LTOPhase, InlineAdvisorContext::ModuleInliner})) {
     M.getContext().emitError(
         "Could not setup Inlining Advisor for the requested "
         "mode and/or options");
     return PreservedAnalyses::all();
   }
 
   bool Changed = false;
 

llvm/lib/Transforms/IPO/SampleProfile.cpp

 #include <vector>
 
 using namespace llvm;
 using namespace sampleprof;
 using namespace llvm::sampleprofutil;
 using ProfileCount = Function::ProfileCount;
 #define DEBUG_TYPE "sample-profile"
 #define CSINLINE_DEBUG DEBUG_TYPE "-inline"
+#define PRELINK_CSINLINE_DEBUG_TYPE "prelink-" CSINLINE_DEBUG
+#define POSTLINK_CSINLINE_DEBUG_TYPE "postlink-" CSINLINE_DEBUG
+#define MAIN_CSINLINE_DEBUG_TYPE "main-" CSINLINE_DEBUG
 
 STATISTIC(NumCSInlined,
           "Number of functions inlined with context sensitive profile");
 STATISTIC(NumCSNotInlined,
           "Number of functions not inlined with context sensitive profile");
 STATISTIC(NumMismatchedProfile,
           "Number of functions with CFG mismatched profile");
 STATISTIC(NumMatchedProfile, "Number of functions with CFG matched profile");
MaxNumPromotions("sample-profile-icp-max-prom", cl::init(3), cl::Hidden,
                      cl::ZeroOrMore,
                      cl::desc("Max number of promotions for a single indirect "
                               "call callsite in sample profile loader"));
 
 static cl::opt<bool> OverwriteExistingWeights(
     "overwrite-existing-weights", cl::Hidden, cl::init(false),
     cl::desc("Ignore existing branch weights on IR and always overwrite."));
 
+static cl::opt<bool> AnnotateSampleProfileInlinePhase(
+    "annotate-sample-profile-inline-phase", cl::Hidden, cl::init(false),
+    cl::desc("Annotate LTO phase (prelink / postlink), or main (no LTO) for "
+             "sample-profile inline pass name."));
+
 extern cl::opt<bool> EnableExtTspBlockPlacement;
 
 namespace {
 
 using BlockWeightMap = DenseMap<const BasicBlock *, uint64_t>;
 using EquivalenceClassMap = DenseMap<const BasicBlock *, const BasicBlock *>;
 using Edge = std::pair<const BasicBlock *, const BasicBlock *>;
 using EdgeWeightMap = DenseMap<Edge, uint64_t>;

 
 private:
   SampleProfileLoader SampleLoader;
   AssumptionCacheTracker *ACT = nullptr;
   TargetTransformInfoWrapperPass *TTIWP = nullptr;
   TargetLibraryInfoWrapperPass *TLIWP = nullptr;
 };
 
+static inline const char *getAnnotatedRemarkPassName(ThinOrFullLTOPhase phase) {
+  if (!AnnotateSampleProfileInlinePhase)
+    return CSINLINE_DEBUG;
+
+  switch (phase) {
+  case llvm::ThinOrFullLTOPhase::ThinLTOPreLink:
+  case llvm::ThinOrFullLTOPhase::FullLTOPreLink:
+    return PRELINK_CSINLINE_DEBUG_TYPE;
+  case llvm::ThinOrFullLTOPhase::ThinLTOPostLink:
+  case llvm::ThinOrFullLTOPhase::FullLTOPostLink:
+    return POSTLINK_CSINLINE_DEBUG_TYPE;
+  default:
+    return MAIN_CSINLINE_DEBUG_TYPE;
+  }
+  llvm_unreachable("unreachable path");
+}
+
 } // end anonymous namespace
 
 ErrorOr<uint64_t> SampleProfileLoader::getInstWeight(const Instruction &Inst) {
   if (FunctionSamples::ProfileIsProbeBased)
     return getProbeWeight(Inst);
 
   const DebugLoc &DLoc = Inst.getDebugLoc();
   if (!DLoc)

 }
 
 void SampleProfileLoader::emitOptimizationRemarksForInlineCandidates(
     const SmallVectorImpl<CallBase *> &Candidates, const Function &F,
     bool Hot) {
   for (auto I : Candidates) {
     Function *CalledFunction = I->getCalledFunction();
     if (CalledFunction) {
-      ORE->emit(OptimizationRemarkAnalysis(CSINLINE_DEBUG, "InlineAttempt",
-                                           I->getDebugLoc(), I->getParent())
+      ORE->emit(OptimizationRemarkAnalysis(getAnnotatedRemarkPassName(LTOPhase),
+                                           "InlineAttempt", I->getDebugLoc(),
+                                           I->getParent())
                 << "previous inlining reattempted for "
                 << (Hot ? "hotness: '" : "size: '")
                 << ore::NV("Callee", CalledFunction) << "' into '"
                 << ore::NV("Caller", &F) << "'");
     }
   }
 }
 
bool SampleProfileLoader::tryInlineCandidate(
   CallBase &CB = *Candidate.CallInstr;
   Function *CalledFunction = CB.getCalledFunction();
   assert(CalledFunction && "Expect a callee with definition");
   DebugLoc DLoc = CB.getDebugLoc();
   BasicBlock *BB = CB.getParent();
 
   InlineCost Cost = shouldInlineCandidate(Candidate);
   if (Cost.isNever()) {
-    ORE->emit(OptimizationRemarkAnalysis(CSINLINE_DEBUG, "InlineFail", DLoc, BB)
+    ORE->emit(OptimizationRemarkAnalysis(getAnnotatedRemarkPassName(LTOPhase),
+                                         "InlineFail", DLoc, BB)
               << "incompatible inlining");
     return false;
   }
 
   if (!Cost)
     return false;
 
   InlineFunctionInfo IFI(nullptr, GetAC);
   IFI.UpdateProfile = false;
   if (!InlineFunction(CB, IFI).isSuccess())
     return false;
 
   // Merge the attributes based on the inlining.
   AttributeFuncs::mergeAttributesForInlining(*BB->getParent(),
                                              *CalledFunction);
 
   // The call to InlineFunction erases I, so we can't pass it here.
-  emitInlinedIntoBasedOnCost(*ORE, DLoc, BB, *CalledFunction,
-                             *BB->getParent(), Cost, true, CSINLINE_DEBUG);
+  emitInlinedIntoBasedOnCost(*ORE, DLoc, BB, *CalledFunction, *BB->getParent(),
+                             Cost, true, getAnnotatedRemarkPassName(LTOPhase));
 
   // Now populate the list of newly exposed call sites.
   if (InlinedCallSites) {
     InlinedCallSites->clear();
     for (auto &I : IFI.InlinedCallSites)
       InlinedCallSites->push_back(I);
   }
 
void SampleProfileLoader::promoteMergeNotInlinedContextSamples(
     const Function &F) {
   // Accumulate not inlined callsite information into notInlinedSamples
   for (const auto &Pair : NonInlinedCallSites) {
     CallBase *I = Pair.getFirst();
     Function *Callee = I->getCalledFunction();
     if (!Callee || Callee->isDeclaration())
       continue;
 
-    ORE->emit(OptimizationRemarkAnalysis(CSINLINE_DEBUG, "NotInline",
-                                         I->getDebugLoc(), I->getParent())
+    ORE->emit(OptimizationRemarkAnalysis(getAnnotatedRemarkPassName(LTOPhase),
+                                         "NotInline", I->getDebugLoc(),
+                                         I->getParent())
               << "previous inlining not repeated: '"
               << ore::NV("Callee", Callee) << "' into '"
               << ore::NV("Caller", &F) << "'");
 
     ++NumCSNotInlined;
     const FunctionSamples *FS = Pair.getSecond();
     if (FS->getTotalSamples() == 0 && FS->getEntrySamples() == 0) {
       continue;
bool SampleProfileLoader::doInitialization(Module &M,
 
   if (FAM && !ProfileInlineReplayFile.empty()) {
     ExternalInlineAdvisor = getReplayInlineAdvisor(
         M, *FAM, Ctx, /*OriginalAdvisor=*/nullptr,
         ReplayInlinerSettings{ProfileInlineReplayFile,
                               ProfileInlineReplayScope,
                               ProfileInlineReplayFallback,
                               {ProfileInlineReplayFormat}},
-        /*EmitRemarks=*/false);
+        /*EmitRemarks=*/false,
+        InlineAdvisorParams{LTOPhase,
+                            InlineAdvisorContext::ReplaySampleProfileInliner});
   }
 
   // Apply tweaks if context-sensitive or probe-based profile is available.
   if (Reader->profileIsCS() || Reader->profileIsPreInlined() ||
       Reader->profileIsProbeBased()) {
     if (!UseIterativeBFIInference.getNumOccurrences())
       UseIterativeBFIInference = true;
     if (!SampleProfileUseProfi.getNumOccurrences())

llvm/test/LTO/Resolution/X86/diagnostic-handler-remarks-with-hotness.ll

 ; RUN: llvm-as < %s >%t.bc
 
 ; Check pass remarks emitted to YAML file
 ; RUN: rm -f %t.yaml
 ; RUN: llvm-lto2 run -pass-remarks-output=%t.yaml \
 ; RUN:           -pass-remarks-with-hotness \
 ; RUN:           -r %t.bc,tinkywinky,p \
 ; RUN:           -r %t.bc,patatino,px \
 ; RUN:           -r %t.bc,main,px -o %t.o %t.bc
-; RUN: cat %t.yaml | FileCheck %s -check-prefix=YAML
+; RUN: cat %t.yaml | FileCheck %s -check-prefixes=YAML,YAML-NO-ANNOTATE
 
 ; Check low threshold allows remarks to emit.
 ; RUN: rm -f %t.t300.yaml
 ; RUN: llvm-lto2 run -pass-remarks-output=%t.t300.yaml \
 ; RUN:           -pass-remarks-with-hotness \
 ; RUN:           -pass-remarks-hotness-threshold=300 \
 ; RUN:           -r %t.bc,tinkywinky,p \
 ; RUN:           -r %t.bc,patatino,px \
 ; RUN:           -r %t.bc,main,px -o %t.o %t.bc
-; RUN: FileCheck %s -check-prefix=YAML < %t.t300.yaml
+; RUN: FileCheck %s -check-prefixes=YAML,YAML-NO-ANNOTATE < %t.t300.yaml
 
 ; Check high threshold disallows remarks to emit.
 ; RUN: rm -f %t.t301.yaml
 ; RUN: llvm-lto2 run -pass-remarks-output=%t.t301.yaml \
 ; RUN:           -pass-remarks-with-hotness \
 ; RUN:           -pass-remarks-hotness-threshold=301 \
 ; RUN:           -r %t.bc,tinkywinky,p \
 ; RUN:           -r %t.bc,patatino,px \

 ; Check high threshold disallows remarks to emit.
 ; RUN: llvm-lto2 run -pass-remarks=inline \
 ; RUN:           -pass-remarks-with-hotness \
 ; RUN:           -pass-remarks-hotness-threshold=301 \
 ; RUN:           -r %t.bc,tinkywinky,p \
 ; RUN:           -r %t.bc,patatino,px \
 ; RUN:           -r %t.bc,main,px -o %t.o %t.bc 2>&1 | count 0
 
+; Check that remarks are annotated with LTO phase information with `-annotate-inline-phase`.
+; RUN: rm -f %t.yaml
+; RUN: llvm-lto2 run -pass-remarks-output=%t.yaml \
+; RUN:           -pass-remarks-with-hotness \
+; RUN:           -annotate-inline-phase \
+; RUN:           -r %t.bc,tinkywinky,p \
+; RUN:           -r %t.bc,patatino,px \
+; RUN:           -r %t.bc,main,px -o %t.o %t.bc
+; RUN: cat %t.yaml | FileCheck %s -check-prefixes=YAML,YAML-ANNOTATE
+
+; Run again with `--annotate-inline-phase`.
+; RUN: rm -f %t.t300.yaml
+; RUN: llvm-lto2 run -pass-remarks-output=%t.t300.yaml \
+; RUN:           -pass-remarks-with-hotness \
+; RUN:           -pass-remarks-hotness-threshold=300 \
+; RUN:           -annotate-inline-phase \
+; RUN:           -r %t.bc,tinkywinky,p \
+; RUN:           -r %t.bc,patatino,px \
+; RUN:           -r %t.bc,main,px -o %t.o %t.bc
+; RUN: FileCheck %s -check-prefixes=YAML,YAML-ANNOTATE < %t.t300.yaml
+
 ; YAML: --- !Passed
-; YAML-NEXT: Pass:            inline
+; YAML-NO-ANNOTATE-NEXT: Pass:            inline
+; YAML-ANNOTATE-NEXT: Pass:            postlink-cgscc-inline
+
 ; YAML-NEXT: Name:            Inlined
 ; YAML-NEXT: Function:        main
 ; YAML-NEXT: Hotness:         300
 ; YAML-NEXT: Args:
 ; YAML-NEXT:   - String:          ''''
 ; YAML-NEXT:   - Callee:          tinkywinky
 ; YAML-NEXT:   - String:          ''' inlined into '''
 ; YAML-NEXT:   - Caller:          main

llvm/test/LTO/Resolution/X86/diagnostic-handler-remarks.ll

 ; Test of LTO with opt remarks YAML output.
 
 ; First try with Regular LTO
 ; RUN: llvm-as < %s >%t.bc
 ; RUN: rm -f %t.yaml
 ; RUN: llvm-lto2 run -pass-remarks-output=%t.yaml \
 ; RUN:           -pass-remarks-filter=inline \
 ; RUN:           -r %t.bc,tinkywinky,p \
 ; RUN:           -r %t.bc,patatino,px \
 ; RUN:           -r %t.bc,main,px -o %t.o %t.bc
-; RUN: cat %t.yaml | FileCheck %s -check-prefix=YAML
+; RUN: cat %t.yaml | FileCheck %s -check-prefixes=YAML,YAML-NO-ANNOTATE
 
 ; Try again with ThinLTO
 ; RUN: opt -module-summary %s -o %t.bc
 ; RUN: rm -f %t.thin.1.yaml
 ; RUN: llvm-lto2 run -pass-remarks-output=%t \
 ; RUN:           -pass-remarks-filter=inline \
 ; RUN:           -r %t.bc,tinkywinky,p \
 ; RUN:           -r %t.bc,patatino,px \
 ; RUN:           -r %t.bc,main,px -o %t.o %t.bc
-; RUN: cat %t.thin.1.yaml | FileCheck %s -check-prefix=YAML
+; RUN: cat %t.thin.1.yaml | FileCheck %s -check-prefixes=YAML,YAML-NO-ANNOTATE
+
+; Check that remarks are annotated with LTO phase information with `-annotate-inline-phase`.
+; First try with Regular LTO
+; RUN: llvm-as < %s >%t.bc
+; RUN: rm -f %t.yaml
+; RUN: llvm-lto2 run -pass-remarks-output=%t.yaml \
+; RUN:           -pass-remarks-filter=inline \
+; RUN:           -annotate-inline-phase \
+; RUN:           -r %t.bc,tinkywinky,p \
+; RUN:           -r %t.bc,patatino,px \
+; RUN:           -r %t.bc,main,px -o %t.o %t.bc
+; RUN: cat %t.yaml | FileCheck %s -check-prefixes=YAML,YAML-POSTLINK-LTO
+
+; Try again with ThinLTO
+; RUN: opt -module-summary %s -o %t.bc
+; RUN: rm -f %t.thin.1.yaml
+; RUN: llvm-lto2 run -pass-remarks-output=%t \
+; RUN:           -pass-remarks-filter=inline \
+; RUN:           -annotate-inline-phase \
+; RUN:           -r %t.bc,tinkywinky,p \
+; RUN:           -r %t.bc,patatino,px \
+; RUN:           -r %t.bc,main,px -o %t.o %t.bc
+; RUN: cat %t.thin.1.yaml | FileCheck %s -check-prefixes=YAML,YAML-POSTLINK-LTO
 
 ; YAML:      --- !Passed
-; YAML-NEXT: Pass:            inline
+; YAML-NO-ANNOTATE: Pass:         inline
+; YAML-POSTLINK-LTO-NEXT: Pass:            postlink-cgscc-inline
 ; YAML-NEXT: Name:            Inlined
 ; YAML-NEXT: Function:        main
 ; YAML-NEXT: Args:
 ; YAML-NEXT:   - String:          ''''
 ; YAML-NEXT:   - Callee:          tinkywinky
 ; YAML-NEXT:   - String:          ''' inlined into '''
 ; YAML-NEXT:   - Caller:          main
 ; YAML-NEXT:   - String:          ''''

llvm/test/LTO/X86/diagnostic-handler-remarks-with-hotness.ll

 ; Check that the hotness attribute is included in the optimization record file
 ; with -lto-pass-remarks-with-hotness.
 
 ; RUN: llvm-as < %s >%t.bc
 ; RUN: rm -f %t.yaml %t.t300.yaml %t.t301.yaml
 ; RUN: llvm-lto -lto-pass-remarks-output=%t.yaml \
 ; RUN:          -lto-pass-remarks-with-hotness \
 ; RUN:          -exported-symbol _main -o %t.o %t.bc
-; RUN: cat %t.yaml | FileCheck -check-prefix=YAML %s
+; RUN: cat %t.yaml | FileCheck -check-prefixes=YAML,YAML-NO-ANNOTATE %s
 
 ; RUN: llvm-lto -lto-pass-remarks-output=%t.t300.yaml \
 ; RUN:          -lto-pass-remarks-with-hotness \
 ; RUN:          -lto-pass-remarks-hotness-threshold=300 \
 ; RUN:          -exported-symbol _main -o %t.o %t.bc
-; RUN: FileCheck -check-prefix=YAML %s < %t.t300.yaml
+; RUN: FileCheck -check-prefixes=YAML,YAML-NO-ANNOTATE %s < %t.t300.yaml
 
 ; RUN: llvm-lto -lto-pass-remarks-output=%t.t301.yaml \
 ; RUN:          -lto-pass-remarks-with-hotness \
 ; RUN:          -lto-pass-remarks-hotness-threshold=301 \
 ; RUN:          -exported-symbol _main -o %t.o %t.bc
-; RUN: not FileCheck -check-prefix=YAML %s < %t.t301.yaml
+; RUN: not FileCheck -check-prefixes=YAML,YAML-NO-ANNOTATE %s < %t.t301.yaml
+
+; Check that remarks are annotated with LTO phase information with `-annotate-inline-lto-phase`.
+; RUN: llvm-as < %s >%t.bc
+; RUN: rm -f %t.yaml %t.t300.yaml %t.t301.yaml
+; RUN: llvm-lto -lto-pass-remarks-output=%t.yaml \
+; RUN:          -lto-pass-remarks-with-hotness \
+; RUN:          -annotate-inline-phase \
+; RUN:          -exported-symbol _main -o %t.o %t.bc
+; RUN: cat %t.yaml | FileCheck -check-prefixes=YAML,YAML-ANNOTATE %s
+
+; RUN: llvm-lto -lto-pass-remarks-output=%t.t300.yaml \
+; RUN:          -lto-pass-remarks-with-hotness \
+; RUN:          -lto-pass-remarks-hotness-threshold=300 \
+; RUN:          -annotate-inline-phase \
+; RUN:          -exported-symbol _main -o %t.o %t.bc
+; RUN: FileCheck -check-prefixes=YAML,YAML-ANNOTATE %s < %t.t300.yaml
+
+; RUN: llvm-lto -lto-pass-remarks-output=%t.t301.yaml \
+; RUN:          -lto-pass-remarks-with-hotness \
+; RUN:          -lto-pass-remarks-hotness-threshold=301 \
+; RUN:          -annotate-inline-phase \
+; RUN:          -exported-symbol _main -o %t.o %t.bc
+; RUN: not FileCheck -check-prefixes=YAML,YAML-ANNOTATE %s < %t.t301.yaml
 
 ; YAML:      --- !Passed
-; YAML-NEXT: Pass:            inline
+; YAML-NO-ANNOTATE-NEXT: Pass:            inline
+; YAML-ANNOTATE-NEXT: Pass:            postlink-cgscc-inline
 ; YAML-NEXT: Name:            Inlined
 ; YAML-NEXT: Function:        main
 ; YAML-NEXT: Hotness:         300
 ; YAML-NEXT: Args:
 ; YAML-NEXT:   - String:          ''''
 ; YAML-NEXT:   - Callee:          foo
 ; YAML-NEXT:   - String:          ''' inlined into '''
 ; YAML-NEXT:   - Caller:          main

llvm/test/LTO/X86/diagnostic-handler-remarks.ll

 
 ; Optimization records are collected regardless of the diagnostic handler
 ; RUN: rm -f %t.yaml
 ; RUN: llvm-lto \
 ; RUN:          -lto-pass-remarks-output=%t.yaml \
 ; RUN:          -exported-symbol _func2 \
 ; RUN:          -exported-symbol _main -o %t.o %t.bc 2>&1 | \
 ; RUN:     FileCheck %s -allow-empty
-; RUN: cat %t.yaml | FileCheck %s -check-prefix=YAML
+; RUN: cat %t.yaml | FileCheck %s -check-prefixes=YAML,YAML-NO-ANNOTATE
+
+; Try again with `-annotate-inline-lto-phase`.
+; RUN: rm -f %t.yaml
+; RUN: llvm-lto \
+; RUN:          -annotate-inline-phase \
+; RUN:          -lto-pass-remarks-output=%t.yaml \
+; RUN:          -exported-symbol _func2 \
+; RUN:          -exported-symbol _main -o %t.o %t.bc 2>&1 | \
+; RUN:     FileCheck %s -allow-empty
+; RUN: cat %t.yaml | FileCheck %s -check-prefixes=YAML,YAML-ANNOTATE
 
 ; REMARKS: remark: {{.*}} 'foo' inlined into 'main'
 ; REMARKS: remark: {{.*}} loop not vectorized: cannot prove it is safe to reorder memory operations
 ; REMARKS_DH: llvm-lto: remark: {{.*}} 'foo' inlined into 'main'
 ; REMARKS_DH: llvm-lto: remark: {{.*}} loop not vectorized: cannot prove it is safe to reorder memory operations
 ; CHECK-NOT: remark:
 ; CHECK-NOT: llvm-lto:
 ; NM-NOT: foo
 ; NM: func2
 ; NM: main
 
 ; YAML:      --- !Passed
-; YAML-NEXT: Pass:            inline
+; YAML-NO-ANNOTATE-NEXT: Pass:            inline
+; YAML-ANNOTATE-NEXT: Pass:            postlink-cgscc-inline
 ; YAML-NEXT: Name:            Inlined
 ; YAML-NEXT: Function:        main
 ; YAML-NEXT: Args:
 ; YAML-NEXT:   - String:          ''''
 ; YAML-NEXT:   - Callee:          foo
 ; YAML-NEXT:   - String:          ''' inlined into '''
 ; YAML-NEXT:   - Caller:          main
 ; YAML-NEXT:   - String:          ''''

llvm/test/ThinLTO/X86/diagnostic-handler-remarks-with-hotness.ll

 ; RUN: opt -module-summary %s -o %t1.bc
 ; RUN: opt -module-summary %p/Inputs/diagnostic-handler-remarks.ll -o %t2.bc
 
 ; Check that the hotness attribute is included in the optimization record file
 ; with -lto-pass-remarks-with-hotness.
 
 ; RUN: llvm-lto -thinlto-action=run \
 ; RUN:          -lto-pass-remarks-output=%t.yaml \
 ; RUN:          -lto-pass-remarks-with-hotness \
 ; RUN:          -exported-symbol _func2 \
 ; RUN:          -exported-symbol _main %t1.bc %t2.bc 2>&1 | \
 ; RUN:     FileCheck %s -allow-empty
 ; CHECK-NOT: remark:
 ; CHECK-NOT: llvm-lto:
 
+; Verify that bar is imported 'and' inlined into 'foo'
+; RUN: cat %t.yaml.thin.0.yaml | FileCheck %s -check-prefixes=YAML1,YAML1-NO-ANNOTATE
 
 ; Verify that bar is imported 'and' inlined into 'foo'
-; RUN: cat %t.yaml.thin.0.yaml | FileCheck %s -check-prefix=YAML1
+; RUN: cat %t.yaml.thin.1.yaml | FileCheck %s -check-prefixes=YAML2,YAML2-NO-ANNOTATE
+
+; Run again with -annotate-inline-phase
+
+; RUN: llvm-lto -thinlto-action=run \
+; RUN:          -lto-pass-remarks-output=%t.yaml \
+; RUN:          -annotate-inline-phase \
+; RUN:          -lto-pass-remarks-with-hotness \
+; RUN:          -exported-symbol _func2 \
+; RUN:          -exported-symbol _main %t1.bc %t2.bc 2>&1 | \
+; RUN:     FileCheck %s -allow-empty
+; CHECK-NOT: remark:
+; CHECK-NOT: llvm-lto:
+
+; Verify that pass name is annotated with LTO phase information.
+; RUN: cat %t.yaml.thin.0.yaml | FileCheck %s -check-prefixes=YAML1,YAML1-ANNOTATE
+
 ; YAML1:      --- !Passed
 ; YAML1:      --- !Passed
-; YAML1-NEXT: Pass:            inline
+; YAML1-NO-ANNOTATE-NEXT: Pass: inline
+; YAML1-ANNOTATE-NEXT: Pass:   postlink-cgscc-inline
 ; YAML1-NEXT: Name:            Inlined
 ; YAML1-NEXT: Function:        main
 ; YAML1-NEXT: Hotness:         50
 ; YAML1-NEXT: Args:
 ; YAML1-NEXT:   - String:          ''''
 ; YAML1-NEXT:   - Callee:          foo
 ; YAML1-NEXT:   - String:          ''' inlined into '
 ; YAML1-NEXT:   - Caller:          main
 ; YAML1-NEXT:   - String:          ''''
 ; YAML1-NEXT:   - String:          ' with '
 ; YAML1-NEXT:   - String:          '(cost='
 ; YAML1-NEXT:   - Cost:            '-30'
 ; YAML1-NEXT:   - String:          ', threshold='
 ; YAML1-NEXT:   - Threshold:       '375'
 ; YAML1-NEXT:   - String:          ')'
 ; YAML1-NEXT: ...
 
 
 ; Verify that bar is imported 'and' inlined into 'foo'
-; RUN: cat %t.yaml.thin.1.yaml | FileCheck %s -check-prefix=YAML2
+; RUN: cat %t.yaml.thin.1.yaml | FileCheck %s -check-prefixes=YAML2,YAML2-ANNOTATE
 ; YAML2:      --- !Passed
-; YAML2-NEXT: Pass:            inline
+; YAML2-NO-ANNOTATE-NEXT: Pass:            inline
+; YAML2-ANNOTATE-NEXT: Pass:            postlink-cgscc-inline
 ; YAML2-NEXT: Name:            Inlined
 ; YAML2-NEXT: Function:        foo
 ; YAML2-NEXT: Args:
 ; YAML2-NEXT:   - String:          ''''
 ; YAML2-NEXT:   - Callee:          bar
 ; YAML2-NEXT:   - String:          ''' inlined into '
 ; YAML2-NEXT:   - Caller:          foo
 ; YAML2-NEXT:   - String:          ''''

llvm/test/ThinLTO/X86/diagnostic-handler-remarks.ll

 ; RUN:          -exported-symbol _func2 \
 ; RUN:          -exported-symbol _main %t1.bc %t2.bc 2>&1 | \
 ; RUN:     FileCheck %s -allow-empty
 ; CHECK-NOT: remark:
 ; CHECK-NOT: llvm-lto:
 
 
 ; Verify that bar is imported 'and' inlined into 'foo'
-; RUN: cat %t.yaml.thin.0.yaml | FileCheck %s -check-prefix=YAML1
+; RUN: cat %t.yaml.thin.0.yaml | FileCheck %s -check-prefixes=YAML1,YAML1-NO-ANNOTATE
+
+; Verify that bar is imported 'and' inlined into 'foo'
+; RUN: cat %t.yaml.thin.1.yaml | FileCheck %s -check-prefixes=YAML2,YAML2-NO-ANNOTATE
+
+; Try again with -annotate-inline-phase
+; RUN: rm -f %t.yaml.thin.0.yaml %t.yaml.thin.1.yaml
+; RUN: llvm-lto -thinlto-action=run \
+; RUN:          -annotate-inline-phase \
+; RUN:          -lto-pass-remarks-output=%t.yaml \
+; RUN:          -lto-pass-remarks-filter=inline \
+; RUN:          -lto-pass-remarks-format=yaml \
+; RUN:          -exported-symbol _func2 \
+; RUN:          -exported-symbol _main %t1.bc %t2.bc 2>&1 | \
+; RUN:     FileCheck %s -allow-empty
+; CHECK-NOT: remark:
+; CHECK-NOT: llvm-lto:
+
+; Verify thati inline happens the same way and pass name are annotated with LTO phase information.
+; RUN: cat %t.yaml.thin.0.yaml | FileCheck %s -check-prefixes=YAML1,YAML1-ANNOTATE
+
+; Verify that bar is imported 'and' inlined into 'foo'
+; RUN: cat %t.yaml.thin.1.yaml | FileCheck %s -check-prefixes=YAML2,YAML2-ANNOTATE
+
 ; YAML1:      --- !Passed
 ; YAML1:      --- !Passed
-; YAML1-NEXT: Pass:            inline
+; YAML1-NO-ANNOTATE: Pass: inline
+; YAML1-ANNOTATE-NEXT: Pass:   postlink-cgscc-inline
 ; YAML1-NEXT: Name:            Inlined
 ; YAML1-NEXT: Function:        main
 ; YAML1-NEXT: Args:
 ; YAML1-NEXT:   - String:          ''''
 ; YAML1-NEXT:   - Callee:          foo
 ; YAML1-NEXT:   - String:          ''' inlined into '
 ; YAML1-NEXT:   - Caller:          main
 ; YAML1-NEXT:   - String:          ''''
 ; YAML1-NEXT:   - String:          ' with '
 ; YAML1-NEXT:   - String:          '(cost='
 ; YAML1-NEXT:   - Cost:            '-30'
 ; YAML1-NEXT:   - String:          ', threshold='
 ; YAML1-NEXT:   - Threshold:       '375'
 ; YAML1-NEXT:   - String:          ')'
 ; YAML1-NEXT: ...
 
-
-; Verify that bar is imported 'and' inlined into 'foo'
-; RUN: cat %t.yaml.thin.1.yaml | FileCheck %s -check-prefix=YAML2
 ; YAML2: --- !Passed
-; YAML2-NEXT: Pass:            inline
+; YAML2-NO-ANNOTATE: Pass: inline
+; YAML2-ANNOTATE-NEXT: Pass:   postlink-cgscc-inline
 ; YAML2-NEXT: Name:            Inlined
 ; YAML2-NEXT: Function:        foo
 ; YAML2-NEXT: Args:
 ; YAML2-NEXT:   - String:          ''''
 ; YAML2-NEXT:   - Callee:          bar
 ; YAML2-NEXT:   - String:          ''' inlined into '
 ; YAML2-NEXT:   - Caller:          foo
 ; YAML2-NEXT:   - String:          ''''

llvm/test/Transforms/Inline/optimization-remarks-passed-yaml.ll

 ; RUN: opt < %s -S -inline -pass-remarks-output=%t -pass-remarks=inline \
 ; RUN:    -pass-remarks-missed=inline -pass-remarks-analysis=inline \
 ; RUN:    -pass-remarks-with-hotness 2>&1 | FileCheck %s
-; RUN: cat %t | FileCheck -check-prefix=YAML %s
+; RUN: cat %t | FileCheck -check-prefixes=YAML,YAML-NO-ANNOTATE %s
 
 ; RUN: opt < %s -S -passes=inline -pass-remarks-output=%t -pass-remarks=inline \
 ; RUN:    -pass-remarks-missed=inline -pass-remarks-analysis=inline \
 ; RUN:    -pass-remarks-with-hotness 2>&1 | FileCheck %s
-; RUN: cat %t | FileCheck -check-prefix=YAML %s
+; RUN: cat %t | FileCheck -check-prefixes=YAML,YAML-NO-ANNOTATE %s
 
 ; RUN: opt < %s -S -passes=inliner-wrapper -pass-remarks-output=%t -pass-remarks=inline \
 ; RUN:    -pass-remarks-missed=inline -pass-remarks-analysis=inline \
+; RUN:    -annotate-inline-phase=false \
 ; RUN:    -pass-remarks-with-hotness 2>&1 | FileCheck %s
-; RUN: cat %t | FileCheck -check-prefix=YAML %s
+; RUN: cat %t | FileCheck -check-prefixes=YAML,YAML-NO-ANNOTATE %s
+
+; RUN: opt < %s -S -passes=inliner-wrapper -pass-remarks-output=%t -pass-remarks=inline \
+; RUN:    -pass-remarks-missed=inline -pass-remarks-analysis=inline \
+; RUN:    -annotate-inline-phase \
+; RUN:    -pass-remarks-with-hotness 2>&1 | FileCheck %s
+; RUN: cat %t | FileCheck -check-prefixes=YAML,YAML-ANNOTATE %s
 
 ; Check the YAML file for inliner-generated passed and analysis remarks.  This
 ; is the input:
 
 ;  1     int foo() { return 1; }
 ;  2
 ;  3     int bar() {
 ;  4       return foo();
 ;  5     }
 
 ; CHECK: remark: /tmp/s.c:4:10: 'foo' inlined into 'bar' with (cost={{[0-9\-]+}}, threshold={{[0-9]+}}) at callsite bar:1:10; (hotness: 30)
 
 ; YAML:      --- !Passed
-; YAML-NEXT: Pass:            inline
+; YAML-NO-ANNOTATE-NEXT: Pass:            inline
+; YAML-ANNOTATE-NEXT: Pass:            main-early-inline
 ; YAML-NEXT: Name:            Inlined
 ; YAML-NEXT: DebugLoc:        { File: '/tmp/s.c', Line: 4, Column: 10 }
 ; YAML-NEXT: Function:        bar
 ; YAML-NEXT: Hotness:         30
 ; YAML-NEXT: Args:
 ; YAML-NEXT:   - String: ''''
 ; YAML-NEXT:   - Callee: foo
 ; YAML-NEXT:     DebugLoc:        { File: '/tmp/s.c', Line: 1, Column: 0 }

llvm/test/Transforms/SampleProfile/remarks.ll

 ; RUN: opt < %s -sample-profile -sample-profile-file=%S/Inputs/remarks.prof -S -pass-remarks=sample-profile -pass-remarks-output=%t.opt.yaml 2>&1 | FileCheck %s
-; RUN: FileCheck %s -check-prefix=YAML < %t.opt.yaml
+; RUN: FileCheck %s -check-prefixes=YAML,YAML-NO-ANNOTATE < %t.opt.yaml
 ; RUN: opt < %s -passes=sample-profile -sample-profile-file=%S/Inputs/remarks.prof -S -pass-remarks=sample-profile -pass-remarks-output=%t.opt.yaml 2>&1 | FileCheck %s
-; RUN: FileCheck %s -check-prefix=YAML < %t.opt.yaml
+; RUN: FileCheck %s -check-prefixes=YAML,YAML-NO-ANNOTATE < %t.opt.yaml
+
+; RUN: opt < %s -sample-profile -annotate-sample-profile-inline-phase -sample-profile-file=%S/Inputs/remarks.prof -S -pass-remarks=sample-profile -pass-remarks-output=%t.opt.yaml 2>&1 | FileCheck %s
+; RUN: FileCheck %s -check-prefixes=YAML,YAML-ANNOTATE < %t.opt.yaml
+; RUN: opt < %s -passes=sample-profile -annotate-sample-profile-inline-phase -sample-profile-file=%S/Inputs/remarks.prof -S -pass-remarks=sample-profile -pass-remarks-output=%t.opt.yaml 2>&1 | FileCheck %s
+; RUN: FileCheck %s -check-prefixes=YAML,YAML-ANNOTATE < %t.opt.yaml
 
 ; Original test case.
 ;
 ;     1    #include <stdlib.h>
 ;     2
 ;     3    long long foo() {
 ;     4      long long int sum = 0;
 ;     5      for (int i = 0; i < 500000000; i++)

 ; The back edge for the loop is the hottest edge in the loop subgraph.
 ; CHECK: remark: remarks.cc:6:9: most popular destination for conditional branches at remarks.cc:5:3
 
 ; The predicate almost always chooses the 'else' branch.
 ; CHECK: remark: remarks.cc:9:15: most popular destination for conditional branches at remarks.cc:6:9
 
 ; Checking to see if YAML file is generated and contains remarks
 ;YAML:       --- !Passed
-;YAML-NEXT:  Pass:            sample-profile-inline
+;YAML-NO-ANNOTATE-NEXT:  Pass:            sample-profile-inline
+;YAML-ANNOTATE-NEXT:  Pass:            main-sample-profile-inline
 ;YAML-NEXT:  Name:            Inlined
 ;YAML-NEXT:  DebugLoc:        { File: remarks.cc, Line: 13, Column: 21 }
 ;YAML-NEXT:  Function:        main
 ;YAML-NEXT:  Args:
 ;YAML-NEXT:    - String:          ''''
 ;YAML-NEXT:    - Callee:          _Z3foov
 ;YAML-NEXT:      DebugLoc:        { File: remarks.cc, Line: 3, Column: 0 }
 ;YAML-NEXT:    - String:          ''' inlined into '

 ;YAML-NEXT:    - String:          main
 ;YAML-NEXT:    - String:          ':'
 ;YAML-NEXT:    - Line:            '0'
 ;YAML-NEXT:    - String:          ':'
 ;YAML-NEXT:    - Column:          '21'
 ;YAML-NEXT:    - String:          ';'
 ;YAML-NEXT:  ...
 ;YAML:       --- !Passed
-;YAML-NEXT:  Pass:            sample-profile-inline
+;YAML-NO-ANNOTATE-NEXT:  Pass:            sample-profile-inline
+;YAML-ANNOTATE-NEXT:  Pass:            main-sample-profile-inline
 ;YAML-NEXT:  Name:            AlwaysInline
 ;YAML-NEXT:  DebugLoc:        { File: remarks.cc, Line: 9, Column: 19 }
 ;YAML-NEXT:  Function:        main
 ;YAML-NEXT:  Args:
 ;YAML-NEXT:    - String:          ''''
 ;YAML-NEXT:    - Callee:          rand
 ;YAML-NEXT:      DebugLoc:        { File: remarks.cc, Line: 90, Column: 0 }
 ;YAML-NEXT:    - String:          ''' inlined into '''