[OptDiag] Updating Remarks in SampleProfile

[efriedma]

Summary

Updating remark API to newer OptimizationDiagnosticInfo API. This allows remarks to show up in diagnostic yaml file, and enables use of opt-viewer tool.

Hotness information for remarks (L505 and L751) do not display hotness information, most likely due to profile information not being propagated yet. Unsure if this is the desired outcome.

[anemet]

Thanks very much for working on this! Please assign https://bugs.llvm.org/show_bug.cgi?id=33794 to you. Also check out my note in the bug.

Hotness information for remarks (L505 and L751) do not display hotness information, most likely due to profile information not being propagated yet. Unsure if this is the desired outcome.

You need to use -pass-remarks-with-hotness for that.

lib/Transforms/IPO/SampleProfile.cpp

165–181

Rather than passing and ORE reference around like this, can't we make it a member to this class?

504–514

Should these be OptimizationRemarkAnalysis instead? These aren't really optimization performed (or missed) i.e. either negative or positive.

505

You probably want to stream this out as named-value argument. Looks for NV in Inliner.cpp for an example.

506–507

I don't know much about the internals of sample-based profiling but is LineOffset or Discriminator actionable to the user? I.e. isn't this just a debugging aid if something goes wrong?

681–682

Formatting only?

750–755

Again see Inliner.cpp how to stream these out as named-value arguments.

1259–1262

You can stream out the instruction like we stream them out I think in LICM as a named-value argument. This may include the LLVM IR name of the instruction which may or may not work in this case, so you may need to tweak DiagnosticInfoOptimizationBase::Argument to do the right thing here.

1487

For the new pass manager path (when you enter via SampleProfileLoaderPass::run we shouldn't create ORE on the file but rather get it from FunctionAnalysisManagerModuleProxy. See IndirectCallPromotion.cpp for an example.

1508–1509

Commit formatting change in otherwise unmodified code separately.

test/Transforms/SampleProfile/remarks.ll

30–40

Please check the full records with YAML-NEXT.

[tarunr]

Thanks for reviewing and for pointing me to examples.

-pass-remarks-with-hotness doesn't seem to be helping me in getting hotness information in these remarks.

lib/Transforms/IPO/SampleProfile.cpp

504–514

Yes, I think that this makes sense

506–507

The LineOffset and Discriminator may be useful for the user to better determine how the compiler applied the profile to the source. Though, I agree it doesn't seem very actionable to the user

1259–1262

Currently it looks like the opcode of the instruction gets streamed out in DiagnosticInfoOptimizationBase::Argument. Do you mean that the original source code of the instruction or location of the instruction should be streamed out instead?

[tarunr]

This update makes OptimizationRemarkEmitter part of the class, adds ORE to the promoteIndirectCall, and streams out remark messages as name-valued arguments.

[anemet]

In D36127#837482, @tarunr wrote:

-pass-remarks-with-hotness doesn't seem to be helping me in getting hotness information in these remarks.

It does something (after applying your patch):

./bin/opt -sample-profile -sample-profile-file=../test/Transforms/SampleProfile/Inputs/remarks.prof -S -pass-remarks-output=- -pass-remarks-with-hotness ../test/Transforms/SampleProfile/remarks.ll -o /dev/null | grep Hotness
Hotness: 0
Hotness: 0

but the 0 values are probably wrong. You probably need to study/debug this. The hotness should come from BFI which sample profiler should be updating.

lib/Transforms/IPO/SampleProfile.cpp

254

Comments end in period.

730

I think that this last use of the function that didn't use to pass ORE. Please remove the default argument from the declaration.

750–754

We can't pass I here because it's remove by the time we get here? If yes, please comment.

750–755

You didn't address this. See the Inliner or the Argument ctors; you can just stream out the Function itself.

1263–1265

I think that the way we should handle this is like: << NV("CondBranchesLoc", BranchLoc)

And then add a new ctor Arguments(StringRef Key, DebugLoc DL) where you set Val to DL.getFileName() + DL.getLine() + DL.getCol() and also Loc to DL. I think this will appear correctly both on the compiler output and in YAML/HTML.

If you want to make that this works correctly in YAML/HTML, you can run opt-viewer on the YAML file. You will have to split out the original C source from the comment into a source file specified in the debug info.

1270

Just stream out a DebugLoc() in you can't find. Don't the change the name of the field.

1441

Default argument on the definition? Please remove.

[anemet]

In D36127#837621, @anemet wrote:

In D36127#837482, @tarunr wrote:

-pass-remarks-with-hotness doesn't seem to be helping me in getting hotness information in these remarks.

It does something (after applying your patch):

./bin/opt -sample-profile -sample-profile-file=../test/Transforms/SampleProfile/Inputs/remarks.prof -S -pass-remarks-output=- -pass-remarks-with-hotness ../test/Transforms/SampleProfile/remarks.ll -o /dev/null | grep Hotness
Hotness: 0
Hotness: 0

but the 0 values are probably wrong. You probably need to study/debug this. The hotness should come from BFI which sample profiler should be updating.

Actually thinking a bit more about this, I don't think we can expect correct profile counts in BFI at this point. This pass adds the necessary metadata to get profile count on a subsequent run of BFI. @davidxl, am I right about this?

So I think for now we should just add a comment about this.

[efriedma]

(Tarun won't have time to finish this, so I'm taking it over.)

[efriedma]

Address review comments.

[anemet]

lib/Transforms/IPO/SampleProfile.cpp

1260

Why do you output the instruction? Its location should already be carried by remark location. It's also weird to see the instruction name in the remark:

CHECK: remark: nolocinfo.c:3:5: ret is the most popular destination for conditional branches at <UNKNOWN LOCATION>

[efriedma]

Don't emit instruction names for "most popular destination" remarks.

[anemet]

LGTM, thanks!

include/llvm/IR/DiagnosticInfo.h

struct Argument {
     Argument(StringRef Key, const Value *V);
     Argument(StringRef Key, const Type *T);
     Argument(StringRef Key, StringRef S);
     Argument(StringRef Key, int N);
     Argument(StringRef Key, int64_t N);
     Argument(StringRef Key, unsigned N);
     Argument(StringRef Key, uint64_t N);
     Argument(StringRef Key, bool B) : Key(Key), Val(B ? "true" : "false") {}
+    Argument(StringRef Key, DebugLoc dl);
   };
 
   /// \p PassName is the name of the pass emitting this diagnostic. \p
   /// RemarkName is a textual identifier for the remark (single-word,
   /// camel-case). \p Fn is the function where the diagnostic is being emitted.
   /// \p Loc is the location information to use in the diagnostic. If line table
   /// information is available, the diagnostic will include the source code
   /// location.

include/llvm/Transforms/Instrumentation.h

 // Count is the profile count value that F is the target function.
 // These two values are used to update the branch weight.
 // If \p AttachProfToDirectCall is true, a prof metadata is attached to the
 // new direct call to contain \p Count.
 // Returns the promoted direct call instruction.
 Instruction *promoteIndirectCall(Instruction *Inst, Function *F, uint64_t Count,
                                  uint64_t TotalCount,
                                  bool AttachProfToDirectCall,
-                                 OptimizationRemarkEmitter *ORE = nullptr);
+                                 OptimizationRemarkEmitter *ORE);
 
 /// Options for the frontend instrumentation based profiling pass.
 struct InstrProfOptions {
   // Add the 'noredzone' attribute to added runtime library calls.
   bool NoRedZone = false;
 
   // Do counter register promotion
   bool DoCounterPromotion = false;

lib/IR/DiagnosticInfo.cpp

DiagnosticInfoOptimizationBase::Argument::Argument(StringRef Key, int64_t N)
     : Key(Key), Val(itostr(N)) {}
 
 DiagnosticInfoOptimizationBase::Argument::Argument(StringRef Key, unsigned N)
     : Key(Key), Val(utostr(N)) {}
 
 DiagnosticInfoOptimizationBase::Argument::Argument(StringRef Key, uint64_t N)
     : Key(Key), Val(utostr(N)) {}
 
+DiagnosticInfoOptimizationBase::Argument::Argument(StringRef Key, DebugLoc Loc)
+    : Key(Key), Loc(Loc) {
+  if (Loc) {
+    Val = (Loc->getFilename() + ":" + Twine(Loc.getLine()) + ":" +
+           Twine(Loc.getCol())).str();
+  } else {
+    Val = "<UNKNOWN LOCATION>";
+  }
+}
+
 void DiagnosticInfoOptimizationBase::print(DiagnosticPrinter &DP) const {
   DP << getLocationStr() << ": " << getMsg();
   if (Hotness)
     DP << " (hotness: " << *Hotness << ")";
 }
 
 OptimizationRemark::OptimizationRemark(const char *PassName,
                                        StringRef RemarkName,

lib/Transforms/IPO/SampleProfile.cpp

 
 #include "llvm/Transforms/SampleProfile.h"
 #include "llvm/ADT/DenseMap.h"
 #include "llvm/ADT/SmallPtrSet.h"
 #include "llvm/ADT/SmallSet.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/Analysis/AssumptionCache.h"
 #include "llvm/Analysis/LoopInfo.h"
+#include "llvm/Analysis/OptimizationDiagnosticInfo.h"
 #include "llvm/Analysis/PostDominators.h"
 #include "llvm/IR/Constants.h"
 #include "llvm/IR/DebugInfo.h"
 #include "llvm/IR/DiagnosticInfo.h"
 #include "llvm/IR/Dominators.h"
 #include "llvm/IR/Function.h"
 #include "llvm/IR/GlobalValue.h"
 #include "llvm/IR/InstIterator.h"

 /// This pass reads profile data from the file specified by
 /// -sample-profile-file and annotates every affected function with the
 /// profile information found in that file.
 class SampleProfileLoader {
 public:
   SampleProfileLoader(StringRef Name = SampleProfileFile)
       : DT(nullptr), PDT(nullptr), LI(nullptr), ACT(nullptr), Reader(),
         Samples(nullptr), Filename(Name), ProfileIsValid(false),
-        TotalCollectedSamples(0) {}
+        TotalCollectedSamples(0), ORE(nullptr) {}
 
   bool doInitialization(Module &M);
-  bool runOnModule(Module &M);
+  bool runOnModule(Module &M, ModuleAnalysisManager *AM);
   void setACT(AssumptionCacheTracker *A) { ACT = A; }
 
   void dump() { Reader->dump(); }
 
 protected:
-  bool runOnFunction(Function &F);
+  bool runOnFunction(Function &F, ModuleAnalysisManager *AM);
   unsigned getFunctionLoc(Function &F);
   bool emitAnnotations(Function &F);
   ErrorOr<uint64_t> getInstWeight(const Instruction &I);
   ErrorOr<uint64_t> getBlockWeight(const BasicBlock *BB);
   const FunctionSamples *findCalleeFunctionSamples(const Instruction &I) const;
   std::vector<const FunctionSamples *>
   findIndirectCallFunctionSamples(const Instruction &I) const;
   const FunctionSamples *findFunctionSamples(const Instruction &I) const;
   bool inlineHotFunctions(Function &F,
                           DenseSet<GlobalValue::GUID> &ImportGUIDs);
   void printEdgeWeight(raw_ostream &OS, Edge E);
   void printBlockWeight(raw_ostream &OS, const BasicBlock *BB) const;
   void printBlockEquivalence(raw_ostream &OS, const BasicBlock *BB);
   bool computeBlockWeights(Function &F);
   void findEquivalenceClasses(Function &F);
   template <bool IsPostDom>
   void findEquivalencesFor(BasicBlock *BB1, ArrayRef<BasicBlock *> Descendants,
                            DominatorTreeBase<BasicBlock, IsPostDom> *DomTree);
 
   void propagateWeights(Function &F);
   uint64_t visitEdge(Edge E, unsigned *NumUnknownEdges, Edge *UnknownEdge);
   void buildEdges(Function &F);
   bool propagateThroughEdges(Function &F, bool UpdateBlockCount);
   void computeDominanceAndLoopInfo(Function &F);
   unsigned getOffset(const DILocation *DIL) const;
   void clearFunctionData();
 
   /// \brief Map basic blocks to their computed weights.
protected:
   /// \brief Flag indicating whether the profile input loaded successfully.
   bool ProfileIsValid;
 
   /// \brief Total number of samples collected in this profile.
   ///
   /// This is the sum of all the samples collected in all the functions executed
   /// at runtime.
   uint64_t TotalCollectedSamples;
+
+  /// \brief Optimization Remark Emitter used to emit diagnostic remarks.
+  OptimizationRemarkEmitter *ORE;
 };
 
 class SampleProfileLoaderLegacyPass : public ModulePass {
 public:
   // Class identification, replacement for typeinfo
   static char ID;
 
   SampleProfileLoaderLegacyPass(StringRef Name = SampleProfileFile)
ErrorOr<uint64_t> SampleProfileLoader::getInstWeight(const Instruction &Inst) {
   const DILocation *DIL = DLoc;
   uint32_t LineOffset = getOffset(DIL);
   uint32_t Discriminator = DIL->getBaseDiscriminator();
   ErrorOr<uint64_t> R = FS->findSamplesAt(LineOffset, Discriminator);
   if (R) {
     bool FirstMark =
         CoverageTracker.markSamplesUsed(FS, LineOffset, Discriminator, R.get());
     if (FirstMark) {
-      const Function *F = Inst.getParent()->getParent();
-      LLVMContext &Ctx = F->getContext();
-      emitOptimizationRemark(
-          Ctx, DEBUG_TYPE, *F, DLoc,
-          Twine("Applied ") + Twine(*R) +
-              " samples from profile (offset: " + Twine(LineOffset) +
-              ((Discriminator) ? Twine(".") + Twine(Discriminator) : "") + ")");
+      if (Discriminator)
+        ORE->emit(OptimizationRemarkAnalysis(DEBUG_TYPE, "AppliedSamples", &Inst)
+                  << "Applied " << ore::NV("NumSamples", *R)
+                  << " samples from profile (offset: "
+                  << ore::NV("LineOffset", LineOffset) << "."
+                  << ore::NV("Discriminator", Discriminator) << ")");
+      else
+        ORE->emit(OptimizationRemarkAnalysis(DEBUG_TYPE, "AppliedSamples", &Inst)
+                  << "Applied " << ore::NV("NumSamples", *R)
+                  << " samples from profile (offset: "
+                  << ore::NV("LineOffset", LineOffset) << ")");
     }
     DEBUG(dbgs() << "    " << DLoc.getLine() << "."
                  << DIL->getBaseDiscriminator() << ":" << Inst
                  << " (line offset: " << LineOffset << "."
                  << DIL->getBaseDiscriminator() << " - weight: " << R.get()
                  << ")\n");
   }
   return R;

 /// \param ImportGUIDs a set to be updated to include all GUIDs that come
 ///     from a different module but inlined in the profiled binary.
 ///
 /// \returns True if there is any inline happened.
 bool SampleProfileLoader::inlineHotFunctions(
     Function &F, DenseSet<GlobalValue::GUID> &ImportGUIDs) {
   DenseSet<Instruction *> PromotedInsns;
   bool Changed = false;
-  LLVMContext &Ctx = F.getContext();
   std::function<AssumptionCache &(Function &)> GetAssumptionCache = [&](
       Function &F) -> AssumptionCache & { return ACT->getAssumptionCache(F); };
   while (true) {
     bool LocalChanged = false;
     SmallVector<Instruction *, 10> CIS;
     for (auto &BB : F) {
       bool Hot = false;
       SmallVector<Instruction *, 10> Candidates;
       for (auto &I : BB.getInstList()) {
         const FunctionSamples *FS = nullptr;
         if ((isa<CallInst>(I) || isa<InvokeInst>(I)) &&
for (auto I : CIS) {
             continue;
           CalledFunction = R->getValue();
           if (CalledFunction && isLegalToPromote(I, CalledFunction, &Reason)) {
             // The indirect target was promoted and inlined in the profile, as a
             // result, we do not have profile info for the branch probability.
             // We set the probability to 80% taken to indicate that the static
             // call is likely taken.
             DI = dyn_cast<Instruction>(
-                promoteIndirectCall(I, CalledFunction, 80, 100, false)
+                promoteIndirectCall(I, CalledFunction, 80, 100, false, ORE)
                     ->stripPointerCasts());
             PromotedInsns.insert(I);
           } else {
             DEBUG(dbgs() << "\nFailed to promote indirect call to "
                          << CalleeFunctionName << " because " << Reason
                          << "\n");
             continue;
           }
         }
       if (!CalledFunction || !CalledFunction->getSubprogram()) {
         findCalleeFunctionSamples(*I)->findImportedFunctions(
             ImportGUIDs, F.getParent(),
             Samples->getTotalSamples() * SampleProfileHotThreshold / 100);
         continue;
       }
       DebugLoc DLoc = I->getDebugLoc();
+      BasicBlock *BB = I->getParent();
       if (InlineFunction(CallSite(DI), IFI)) {
         LocalChanged = true;
-        emitOptimizationRemark(Ctx, DEBUG_TYPE, F, DLoc,
-                               Twine("inlined hot callee '") +
-                                   CalledFunction->getName() + "' into '" +
-                                   F.getName() + "'");
+        // The call to InlineFunction erases DI, so we can't pass it here.
+        ORE->emit(OptimizationRemark(DEBUG_TYPE, "HotInline", DLoc, BB)
+                  << "inlined hot callee '"
+                  << ore::NV("Callee", CalledFunction) << "' into '"
+                  << ore::NV("Caller", &F) << "'");
       }
     }
     if (LocalChanged) {
       Changed = true;
     } else {
       break;
     }
   }
   return Changed;
for (auto &BI : F) {
 
     DebugLoc BranchLoc = TI->getDebugLoc();
     DEBUG(dbgs() << "\nGetting weights for branch at line "
                  << ((BranchLoc) ? Twine(BranchLoc.getLine())
                                  : Twine("<UNKNOWN LOCATION>"))
                  << ".\n");
     SmallVector<uint32_t, 4> Weights;
     uint32_t MaxWeight = 0;
-    DebugLoc MaxDestLoc;
+    Instruction *MaxDestInst;
     for (unsigned I = 0; I < TI->getNumSuccessors(); ++I) {
       BasicBlock *Succ = TI->getSuccessor(I);
       Edge E = std::make_pair(BB, Succ);
       uint64_t Weight = EdgeWeights[E];
       DEBUG(dbgs() << "\t"; printEdgeWeight(dbgs(), E));
       // Use uint32_t saturated arithmetic to adjust the incoming weights,
       // if needed. Sample counts in profiles are 64-bit unsigned values,
       // but internally branch weights are expressed as 32-bit values.
       if (Weight > std::numeric_limits<uint32_t>::max()) {
         DEBUG(dbgs() << " (saturated due to uint32_t overflow)");
         Weight = std::numeric_limits<uint32_t>::max();
       }
       // Weight is added by one to avoid propagation errors introduced by
       // 0 weights.
       Weights.push_back(static_cast<uint32_t>(Weight + 1));
       if (Weight != 0) {
         if (Weight > MaxWeight) {
           MaxWeight = Weight;
-          MaxDestLoc = Succ->getFirstNonPHIOrDbgOrLifetime()->getDebugLoc();
+          MaxDestInst = Succ->getFirstNonPHIOrDbgOrLifetime();
         }
       }
     }
 
     uint64_t TempWeight;
     // Only set weights if there is at least one non-zero weight.
     // In any other case, let the analyzer set weights.
     // Do not set weights if the weights are present. In ThinLTO, the profile
     // annotation is done twice. If the first annotation already set the
     // weights, the second pass does not need to set it.
     if (MaxWeight > 0 && !TI->extractProfTotalWeight(TempWeight)) {
       DEBUG(dbgs() << "SUCCESS. Found non-zero weights.\n");
       TI->setMetadata(llvm::LLVMContext::MD_prof,
                       MDB.createBranchWeights(Weights));
-      emitOptimizationRemark(
-          Ctx, DEBUG_TYPE, F, MaxDestLoc,
-          Twine("most popular destination for conditional branches at ") +
-              ((BranchLoc) ? Twine(BranchLoc->getFilename() + ":" +
-                                   Twine(BranchLoc.getLine()) + ":" +
-                                   Twine(BranchLoc.getCol()))
-                           : Twine("<UNKNOWN LOCATION>")));
+      ORE->emit(OptimizationRemark(DEBUG_TYPE, "PopularDest", MaxDestInst)
+                << "most popular destination for conditional branches at "
+                << ore::NV("CondBranchesLoc", BranchLoc));
     } else {
       DEBUG(dbgs() << "SKIPPED. All branch weights are zero.\n");
     }
   }
 }
 
 /// \brief Get the line number for the function header.
 ///
 /// This looks up function \p F in the current compilation unit and
 /// retrieves the line number where the function is defined. This is
 /// line 0 for all the samples read from the profile file. Every line
 /// number is relative to this line.
 ///
 /// \param F  Function object to query.
 ///
 /// \returns the line number where \p F is defined. If it returns 0,
 ///          it means that there is no debug information available for \p F.

 ModulePass *llvm::createSampleProfileLoaderPass() {
   return new SampleProfileLoaderLegacyPass(SampleProfileFile);
 }
 
 ModulePass *llvm::createSampleProfileLoaderPass(StringRef Name) {
   return new SampleProfileLoaderLegacyPass(Name);
 }
 
-bool SampleProfileLoader::runOnModule(Module &M) {
+bool SampleProfileLoader::runOnModule(Module &M, ModuleAnalysisManager *AM) {
   if (!ProfileIsValid)
     return false;
 
   // Compute the total number of samples collected in this profile.
   for (const auto &I : Reader->getProfiles())
     TotalCollectedSamples += I.second.getTotalSamples();
 
   // Populate the symbol map.
if (pos != std::string::npos) {
         r.first->second = nullptr;
     }
   }
 
   bool retval = false;
   for (auto &F : M)
     if (!F.isDeclaration()) {
       clearFunctionData();
-      retval |= runOnFunction(F);
+      retval |= runOnFunction(F, AM);
     }
   if (M.getProfileSummary() == nullptr)
     M.setProfileSummary(Reader->getSummary().getMD(M.getContext()));
   return retval;
 }
 
 bool SampleProfileLoaderLegacyPass::runOnModule(Module &M) {
   // FIXME: pass in AssumptionCache correctly for the new pass manager.
   SampleLoader.setACT(&getAnalysis<AssumptionCacheTracker>());
-  return SampleLoader.runOnModule(M);
+  return SampleLoader.runOnModule(M, nullptr);
 }
 
-bool SampleProfileLoader::runOnFunction(Function &F) {
+bool SampleProfileLoader::runOnFunction(Function &F, ModuleAnalysisManager *AM) {
   F.setEntryCount(0);
+  std::unique_ptr<OptimizationRemarkEmitter> OwnedORE;
+  if (AM) {
+    auto &FAM =
+        AM->getResult<FunctionAnalysisManagerModuleProxy>(*F.getParent())
+            .getManager();
+    ORE = &FAM.getResult<OptimizationRemarkEmitterAnalysis>(F);
+  } else {
+    OwnedORE = make_unique<OptimizationRemarkEmitter>(&F);
+    ORE = OwnedORE.get();
+  }
   Samples = Reader->getSamplesFor(F);
   if (Samples && !Samples->empty())
     return emitAnnotations(F);
   return false;
 }
 
 PreservedAnalyses SampleProfileLoaderPass::run(Module &M,
                                                ModuleAnalysisManager &AM) {
 
   SampleProfileLoader SampleLoader(
       ProfileFileName.empty() ? SampleProfileFile : ProfileFileName);
 
   SampleLoader.doInitialization(M);
 
-  if (!SampleLoader.runOnModule(M))
+  if (!SampleLoader.runOnModule(M, &AM))
     return PreservedAnalyses::all();
 
   return PreservedAnalyses::none();
 }

test/Transforms/SampleProfile/cov-zero-samples.ll

-; RUN: opt < %s -instcombine -sample-profile -sample-profile-file=%S/Inputs/cov-zero-samples.prof -sample-profile-check-record-coverage=100 -pass-remarks=sample-profile -o /dev/null 2>&1 | FileCheck %s
-; RUN: opt < %s -passes="function(instcombine),sample-profile" -sample-profile-file=%S/Inputs/cov-zero-samples.prof -sample-profile-check-record-coverage=100 -pass-remarks=sample-profile -o /dev/null 2>&1 | FileCheck %s
+; RUN: opt < %s -instcombine -sample-profile -sample-profile-file=%S/Inputs/cov-zero-samples.prof -sample-profile-check-record-coverage=100 -pass-remarks=sample-profile -pass-remarks-analysis=sample-profile -o /dev/null 2>&1 | FileCheck %s
+; RUN: opt < %s -passes="function(instcombine),sample-profile" -sample-profile-file=%S/Inputs/cov-zero-samples.prof -sample-profile-check-record-coverage=100 -pass-remarks=sample-profile -pass-remarks-analysis=sample-profile -o /dev/null 2>&1 | FileCheck %s
 ;
 ; CHECK: remark: cov-zero-samples.cc:9:29: Applied 404065 samples from profile (offset: 2.1)
 ; CHECK: remark: cov-zero-samples.cc:10:9: Applied 443089 samples from profile (offset: 3)
 ; CHECK: remark: cov-zero-samples.cc:10:36: Applied 0 samples from profile (offset: 3.1)
 ; CHECK: remark: cov-zero-samples.cc:11:12: Applied 404066 samples from profile (offset: 4)
 ; CHECK: remark: cov-zero-samples.cc:13:25: Applied 0 samples from profile (offset: 6)
 ; CHECK: remark: cov-zero-samples.cc:14:3: Applied 0 samples from profile (offset: 7)
 ; CHECK: remark: cov-zero-samples.cc:10:9: most popular destination for conditional branches at cov-zero-samples.cc:9:3

test/Transforms/SampleProfile/inline-coverage.ll

-; RUN: opt < %s -instcombine -sample-profile -sample-profile-file=%S/Inputs/inline-coverage.prof -sample-profile-check-record-coverage=100 -sample-profile-check-sample-coverage=110 -pass-remarks=sample-profile -o /dev/null 2>&1 | FileCheck %s
-; RUN: opt < %s -passes="function(instcombine),sample-profile" -sample-profile-file=%S/Inputs/inline-coverage.prof -sample-profile-check-record-coverage=100 -sample-profile-check-sample-coverage=110 -pass-remarks=sample-profile -o /dev/null 2>&1 | FileCheck %s
+; RUN: opt < %s -instcombine -sample-profile -sample-profile-file=%S/Inputs/inline-coverage.prof -sample-profile-check-record-coverage=100 -sample-profile-check-sample-coverage=110 -pass-remarks=sample-profile -pass-remarks-analysis=sample-profile -o /dev/null 2>&1 | FileCheck %s
+; RUN: opt < %s -passes="function(instcombine),sample-profile" -sample-profile-file=%S/Inputs/inline-coverage.prof -sample-profile-check-record-coverage=100 -sample-profile-check-sample-coverage=110 -pass-remarks=sample-profile -pass-remarks-analysis=sample-profile -o /dev/null 2>&1 | FileCheck %s
 ;
 ; Original code:
 ;
 ;     1    #include <stdlib.h>
 ;     2
 ;     3    long long int foo(long i) {
 ;     4      return rand() * i;
 ;     5    }

test/Transforms/SampleProfile/remarks.ll

-; RUN: opt < %s -sample-profile -sample-profile-file=%S/Inputs/remarks.prof -S -pass-remarks=sample-profile 2>&1 | FileCheck %s
+; RUN: opt < %s -sample-profile -sample-profile-file=%S/Inputs/remarks.prof -S -pass-remarks=sample-profile -pass-remarks-output=%t.opt.yaml 2>&1 | FileCheck %s
 ; RUN: opt < %s -passes=sample-profile -sample-profile-file=%S/Inputs/remarks.prof -S -pass-remarks=sample-profile 2>&1 | FileCheck %s
-;
+; RUN: FileCheck %s -check-prefix=YAML < %t.opt.yaml
 ; Original test case.
 ;
 ;     1    #include <stdlib.h>
 ;     2
 ;     3    long long foo() {
 ;     4      long long int sum = 0;
 ;     5      for (int i = 0; i < 500000000; i++)
 ;     6        if (i < 1000)

 ; CHECK: remark: remarks.cc:13:21: inlined hot callee '_Z3foov' into 'main'
 
 ; The back edge for the loop is the hottest edge in the loop subgraph.
 ; CHECK: remark: remarks.cc:6:9: most popular destination for conditional branches at remarks.cc:5:3
 
 ; The predicate almost always chooses the 'else' branch.
 ; CHECK: remark: remarks.cc:9:15: most popular destination for conditional branches at remarks.cc:6:9
 
+; Checking to see if YAML file is generated and contains remarks
+;YAML:       --- !Passed
+;YAML-NEXT:  Pass:            sample-profile
+;YAML-NEXT:  Name:            HotInline
+;YAML-NEXT:  DebugLoc:        { File: remarks.cc, Line: 13, Column: 21 }
+;YAML-NEXT:  Function:        main
+;YAML-NEXT:  Args:
+;YAML-NEXT:    - String:          'inlined hot callee '''
+;YAML-NEXT:    - Callee:          _Z3foov
+;YAML-NEXT:      DebugLoc:        { File: remarks.cc, Line: 3, Column: 0 }
+;YAML-NEXT:    - String:          ''' into '''
+;YAML-NEXT:    - Caller:          main
+;YAML-NEXT:        DebugLoc:        { File: remarks.cc, Line: 13, Column: 0 }
+;YAML-NEXT:    - String:          ''''
+;YAML-NEXT:  ...
+;YAML:  --- !Analysis
+;YAML-NEXT:  Pass:            sample-profile
+;YAML-NEXT:  Name:            AppliedSamples
+;YAML-NEXT:  DebugLoc:        { File: remarks.cc, Line: 5, Column: 8 }
+;YAML-NEXT:  Function:        main
+;YAML-NEXT:  Args:
+;YAML-NEXT:    - String:          'Applied '
+;YAML-NEXT:    - NumSamples:      '18305'
+;YAML-NEXT:    - String:          ' samples from profile (offset: '
+;YAML-NEXT:    - LineOffset:      '2'
+;YAML-NEXT:    - String:          ')'
+;YAML-NEXT:  ...
+;YAML:  --- !Passed
+;YAML-NEXT:  Pass:            sample-profile
+;YAML-NEXT:  Name:            PopularDest
+;YAML-NEXT:  DebugLoc:        { File: remarks.cc, Line: 6, Column: 9 }
+;YAML-NEXT:  Function:        main
+;YAML-NEXT:  Args:
+;YAML-NEXT:    - String:          'most popular destination for conditional branches at '
+;YAML-NEXT:    - CondBranchesLoc: 'remarks.cc:5:3'
+;YAML-NEXT:      DebugLoc:        { File: remarks.cc, Line: 5, Column: 3 }
+;YAML-NEXT:  ...
+
 ; Function Attrs: nounwind uwtable
 define i64 @_Z3foov() #0 !dbg !4 {
 entry:
   %sum = alloca i64, align 8
   %i = alloca i32, align 4
   %0 = bitcast i64* %sum to i8*, !dbg !19
   call void @llvm.lifetime.start.p0i8(i64 8, i8* %0) #4, !dbg !19
   call void @llvm.dbg.declare(metadata i64* %sum, metadata !9, metadata !20), !dbg !21