[Remarks][2/2] Expand remarks hotness threshold option support in more tools

[weiwang]

Summary

This is the #2 of 2 changes that make remarks hotness threshold option
available in more tools. The changes also allow the threshold to sync with
hotness threshold from profile summary with special value 'auto'.

This change expands remarks hotness threshold option
-fdiagnostics-hotness-threshold in clang and *-remarks-hotness-threshold in
other tools to utilize hotness threshold from profile summary.

Remarks hotness filtering relies on several driver options. Table below lists
how different options are correlated and affect final remarks outputs:

| profile | hotness | threshold | remarks printed |
|---------|---------|-----------|-----------------|
| No      | No      | No        | All             |
| No      | No      | Yes       | None            |
| No      | Yes     | No        | All             |
| No      | Yes     | Yes       | None            |
| Yes     | No      | No        | All             |
| Yes     | No      | Yes       | None            |
| Yes     | Yes     | No        | All             |
| Yes     | Yes     | Yes       | >=threshold     |

In the presence of profile summary, it is often more desirable to directly use
the hotness threshold from profile summary. The new argument value 'auto'
indicates threshold will be synced with hotness threshold from profile summary
during compilation. The "auto" threshold relies on the availability of profile
summary. In case of missing such information, no remarks will be generated.

[weiwang]

This is the 3rd of 3 dependent patches:

1. [lld] Enable remarks hotness filtering in lld: https://reviews.llvm.org/D85809
2. [clang] Pass-through remarks options to lld: https://reviews.llvm.org/D85810
3. [remarks] Optimization remarks hotness filtering from profile summary: https://reviews.llvm.org/D85808

[tejohnson]

In D85808#2212297, @weiwang wrote:

This is the 3rd of 3 dependent patches:

1. [lld] Enable remarks hotness filtering in lld: https://reviews.llvm.org/D85809
2. [clang] Pass-through remarks options to lld: https://reviews.llvm.org/D85810
3. [remarks] Optimization remarks hotness filtering from profile summary: https://reviews.llvm.org/D85808

You can relate these as Parent/Child patches in Phabricator. See "Edit Related Revisions" on the top right. It helps reviewers to see the relationship and keep track of what patches are still open.

Note these relationships get set up automatically when you upload a patch with "Depends on Dxxxxx." in the description.

lld/ELF/Config.h

280 ↗

(On Diff #284966)

Since this field is being added in patch D85809 as an unsigned, why not add it as llvm::Optional<> there to start with instead of adding and then changing?

Ditto for a number of other places in this patch.

[weiwang]

In D85808#2214272, @tejohnson wrote:

In D85808#2212297, @weiwang wrote:

This is the 3rd of 3 dependent patches:

1. [lld] Enable remarks hotness filtering in lld: https://reviews.llvm.org/D85809
2. [clang] Pass-through remarks options to lld: https://reviews.llvm.org/D85810
3. [remarks] Optimization remarks hotness filtering from profile summary: https://reviews.llvm.org/D85808

You can relate these as Parent/Child patches in Phabricator. See "Edit Related Revisions" on the top right. It helps reviewers to see the relationship and keep track of what patches are still open.

Note these relationships get set up automatically when you upload a patch with "Depends on Dxxxxx." in the description.

Thanks for the tip.

lld/ELF/Config.h

280 ↗

(On Diff #284966)

Thanks for the feedback! It does seem awkward. When doing the splitting, I tried to make the 3 patches look more self-contained from each other. The Optional type change seems unrelated with the first patch.

[tejohnson]

lld/ELF/Config.h

280 ↗

(On Diff #284966)

I would recommend going straight to Optional in the first patch, and just note there that the type is needed for the follow on patch. That will avoid unnecessary churn.

[weiwang]

update:

1. reorganize code splitting with parent diff.
2. format and style change.

[tejohnson]

llvm/lib/Analysis/OptimizationRemarkEmitter.cpp

103–110

Can you clarify what you mean by this only happening once?

[weiwang]

llvm/lib/Analysis/OptimizationRemarkEmitter.cpp

103–110

The check of Context.isDiagnosticsHotnessThresholdSetFromPSI() guarantees that ProfileSummaryInfoWrapperPass and setDiagnosticsHotnessThreshold would only be called once. Once the threshold is set from PSI, the check always returns false. But strictly speaking, since PSI is immutable, it only happens once with/without the check.

[weiwang]

@tejohnson @MaskRay Do you have other comments to the change?

[tejohnson]

Largely LGTM, except that the clang changes are missing a corresponding test.

[weiwang]

update test case for clang option pass-through

[tejohnson]

Thanks for adding the Driver test. I was thinking of something to test the CompilerInvocation changes, similar to your test using opt, that ensures the option has the desired behavior when invoked via clang. Looks like there is an existing test clang/test/Frontend/optimization-remark-with-hotness.c that perhaps could be extended or leveraged?

[weiwang]

1. Add clang test with remarks output.
2. Fix a missing dependency on PSI in legacy pass manager.

[weiwang]

In D85808#2403587, @tejohnson wrote:

Thanks for adding the Driver test. I was thinking of something to test the CompilerInvocation changes, similar to your test using opt, that ensures the option has the desired behavior when invoked via clang. Looks like there is an existing test clang/test/Frontend/optimization-remark-with-hotness.c that perhaps could be extended or leveraged?

Thanks for the suggestion. I have added a new test case for clang. The source file used is the same one in the opt test case.

[weiwang]

@tejohnson @MaskRay Do you have other comments?

[weiwang]

rebase

[tejohnson]

lgtm with a couple of minor nits noted below that you can fix before submitting

clang/include/clang/Basic/CodeGenOptions.h

353

nit: s/emarks/remarks/

358

nit: s/threhold/threshold/

clang/test/Frontend/remarks-hotness.cpp

28

nit: move this down by the corresponding REMARKS line below? (makes the difference between the 2 cases clearer)

[weiwang]

1. Fix typo.
2. Minor order adjustment in testcase.

[weiwang]

In D85808#2424588, @tejohnson wrote:

lgtm with a couple of minor nits noted below that you can fix before submitting

Thanks for pointing them out. Really appreciate it!

[jansvoboda11]

I'm seeing build failures that might be triggered by this patch: http://lab.llvm.org:8011/#/builders/109/builds/3714

[hans]

In D85808#2424908, @jansvoboda11 wrote:

I'm seeing build failures that might be triggered by this patch: http://lab.llvm.org:8011/#/builders/109/builds/3714

It seems to affect a lot of the main bots (random example: http://lab.llvm.org:8011/#/builders/52/builds/1963)
And here's another one: http://45.33.8.238/linux/34102/step_12.txt
We also hit this in Chromium.

I think David just fixed it in https://github.com/llvm/llvm-project/commit/09d82fa95f4561a6a2ce80bce00209018ba70c24

[weiwang]

Ah, I am sorry. Thanks for fixing it.

clang/include/clang/Basic/CodeGenOptions.h

public:
   /// coverage pass should actually not be instrumented.
   std::vector<std::string> SanitizeCoverageBlocklistFiles;
 
   /// Executable and command-line used to create a given CompilerInvocation.
   /// Most of the time this will be the full -cc1 command.
   const char *Argv0 = nullptr;
   ArrayRef<const char *> CommandLineArgs;
 
+  /// The minimum hotness value a diagnostic needs in order to be included in
+  /// optimization diagnostics.
+  ///
+  /// The threshold is an Optional value, which maps to one of the 3 states:
+  /// 1. 0            => threshold disabled. All remarks will be printed.
+  /// 2. positive int => manual threshold by user. Remarks with hotness exceed
+  ///                    threshold will be printed.
+  /// 3. None         => 'auto' threshold by user. The actual value is not
+  ///                    available at command line, but will be synced with
+  ///                    hotness threshold from profile summary during
+  ///                    compilation.
+  ///
+  /// If threshold option is not specified, it is disabled by default.
+  Optional<uint64_t> DiagnosticsHotnessThreshold = 0;
+
 public:
   // Define accessors/mutators for code generation options of enumeration type.
 #define CODEGENOPT(Name, Bits, Default)
 #define ENUM_CODEGENOPT(Name, Type, Bits, Default) \
   Type get##Name() const { return static_cast<Type>(Name); } \
   void set##Name(Type Value) { Name = static_cast<unsigned>(Value); }
 #include "clang/Basic/CodeGenOptions.def"
 

clang/include/clang/Basic/CodeGenOptions.def

 
 /// Number of path components to strip when emitting checks. (0 == full
 /// filename)
 VALUE_CODEGENOPT(EmitCheckPathComponentsToStrip, 32, 0)
 
 /// Whether to report the hotness of the code region for optimization remarks.
 CODEGENOPT(DiagnosticsWithHotness, 1, 0)
 
-/// The minimum hotness value a diagnostic needs in order to be included in
-/// optimization diagnostics.
-VALUE_CODEGENOPT(DiagnosticsHotnessThreshold, 32, 0)
-
 /// Whether copy relocations support is available when building as PIE.
 CODEGENOPT(PIECopyRelocations, 1, 0)
 
 /// Whether we should use the undefined behaviour optimization for control flow
 /// paths that reach the end of a function without executing a required return.
 CODEGENOPT(StrictReturn, 1, 1)
 
 /// Whether emit extra debug info for sample pgo profile collection.

clang/include/clang/Basic/DiagnosticDriverKinds.td

 def err_drv_unable_to_remove_file : Error<
   "unable to remove file: %0">;
 def err_drv_unable_to_set_working_directory : Error <
   "unable to set working directory: %0">;
 def err_drv_command_failure : Error<
   "unable to execute command: %0">;
 def err_drv_invalid_darwin_version : Error<
   "invalid Darwin version number: %0">;
+def err_drv_invalid_diagnotics_hotness_threshold : Error<
+  "invalid argument in '%0', only integer or 'auto' is supported">;
 def err_drv_missing_argument : Error<
   "argument to '%0' is missing (expected %1 value%s1)">;
 def err_drv_invalid_Xarch_argument_with_args : Error<
   "invalid Xarch argument: '%0', options requiring arguments are unsupported">;
 def err_drv_Xopenmp_target_missing_triple : Error<
   "cannot deduce implicit triple value for -Xopenmp-target, specify triple using -Xopenmp-target=<triple>">;
 def err_drv_invalid_Xopenmp_target_with_args : Error<
   "invalid -Xopenmp-target argument: '%0', options requiring arguments are unsupported">;

clang/include/clang/Driver/Options.td

 def fdiagnostics_parseable_fixits : Flag<["-"], "fdiagnostics-parseable-fixits">, Group<f_clang_Group>,
     Flags<[CoreOption, CC1Option]>, HelpText<"Print fix-its in machine parseable form">;
 def fdiagnostics_print_source_range_info : Flag<["-"], "fdiagnostics-print-source-range-info">,
     Group<f_clang_Group>,  Flags<[CC1Option]>,
     HelpText<"Print source range spans in numeric form">;
 def fdiagnostics_show_hotness : Flag<["-"], "fdiagnostics-show-hotness">, Group<f_Group>,
     Flags<[CC1Option]>, HelpText<"Enable profile hotness information in diagnostic line">;
 def fdiagnostics_hotness_threshold_EQ : Joined<["-"], "fdiagnostics-hotness-threshold=">,
-    Group<f_Group>, Flags<[CC1Option]>, MetaVarName<"<number>">,
-    HelpText<"Prevent optimization remarks from being output if they do not have at least this profile count">;
+    Group<f_Group>, Flags<[CC1Option]>, MetaVarName<"<value>">,
+    HelpText<"Prevent optimization remarks from being output if they do not have at least this profile count. "
+    "Use 'auto' to apply the threshold from profile summary">;
 def fdiagnostics_show_option : Flag<["-"], "fdiagnostics-show-option">, Group<f_Group>,
     HelpText<"Print option name with mappable diagnostics">;
 def fdiagnostics_show_note_include_stack : Flag<["-"], "fdiagnostics-show-note-include-stack">,
     Group<f_Group>, Flags<[CC1Option]>, HelpText<"Display include stacks for diagnostic notes">;
 def fdiagnostics_format_EQ : Joined<["-"], "fdiagnostics-format=">, Group<f_clang_Group>;
 def fdiagnostics_show_category_EQ : Joined<["-"], "fdiagnostics-show-category=">, Group<f_clang_Group>;
 def fdiagnostics_show_template_tree : Flag<["-"], "fdiagnostics-show-template-tree">,
     Group<f_Group>, Flags<[CC1Option]>,

clang/lib/Frontend/CompilerInvocation.cpp

 //===- CompilerInvocation.cpp ---------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "clang/Frontend/CompilerInvocation.h"
 #include "TestModuleFileExtension.h"
 #include "clang/Basic/Builtins.h"
 #include "clang/Basic/CharInfo.h"
 #include "clang/Basic/CodeGenOptions.h"
 #include "clang/Basic/CommentOptions.h"
 #include "clang/Basic/DebugInfoOptions.h"
 #include "clang/Basic/Diagnostic.h"
+#include "clang/Basic/DiagnosticDriver.h"
 #include "clang/Basic/DiagnosticOptions.h"
 #include "clang/Basic/FileSystemOptions.h"
 #include "clang/Basic/LLVM.h"
 #include "clang/Basic/LangOptions.h"
 #include "clang/Basic/LangStandard.h"
 #include "clang/Basic/ObjCRuntime.h"
 #include "clang/Basic/Sanitizers.h"
 #include "clang/Basic/SourceLocation.h"

 #include "llvm/Linker/Linker.h"
 #include "llvm/MC/MCTargetOptions.h"
 #include "llvm/Option/Arg.h"
 #include "llvm/Option/ArgList.h"
 #include "llvm/Option/OptSpecifier.h"
 #include "llvm/Option/OptTable.h"
 #include "llvm/Option/Option.h"
 #include "llvm/ProfileData/InstrProfReader.h"
+#include "llvm/Remarks/HotnessThresholdParser.h"
 #include "llvm/Support/CodeGen.h"
 #include "llvm/Support/Compiler.h"
 #include "llvm/Support/Error.h"
 #include "llvm/Support/ErrorHandling.h"
 #include "llvm/Support/ErrorOr.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/Host.h"
 #include "llvm/Support/MathExtras.h"
bool UsingProfile = UsingSampleProfile ||
       (Opts.getProfileUse() != CodeGenOptions::ProfileNone);
 
   if (Opts.DiagnosticsWithHotness && !UsingProfile &&
       // An IR file will contain PGO as metadata
       IK.getLanguage() != Language::LLVM_IR)
     Diags.Report(diag::warn_drv_diagnostics_hotness_requires_pgo)
         << "-fdiagnostics-show-hotness";
 
-  Opts.DiagnosticsHotnessThreshold = getLastArgUInt64Value(
-      Args, options::OPT_fdiagnostics_hotness_threshold_EQ, 0);
-  if (Opts.DiagnosticsHotnessThreshold > 0 && !UsingProfile)
+  // Parse remarks hotness threshold. Valid value is either integer or 'auto'.
+  if (auto *arg =
+          Args.getLastArg(options::OPT_fdiagnostics_hotness_threshold_EQ)) {
+    auto ResultOrErr =
+        llvm::remarks::parseHotnessThresholdOption(arg->getValue());
+
+    if (!ResultOrErr) {
+      Diags.Report(diag::err_drv_invalid_diagnotics_hotness_threshold)
+          << "-fdiagnostics-hotness-threshold=";
+    } else {
+      Opts.DiagnosticsHotnessThreshold = *ResultOrErr;
+      if ((!Opts.DiagnosticsHotnessThreshold.hasValue() ||
+           Opts.DiagnosticsHotnessThreshold.getValue() > 0) &&
+          !UsingProfile)
         Diags.Report(diag::warn_drv_diagnostics_hotness_requires_pgo)
             << "-fdiagnostics-hotness-threshold=";
+    }
+  }
 
   // If the user requested to use a sample profile for PGO, then the
   // backend will need to track source location information so the profile
   // can be incorporated into the IR.
   if (UsingSampleProfile)
     NeedLocTracking = true;
 
   // If the user requested a flag that requires source locations available in

clang/test/Driver/opt-record.c

 // No pass-through: lto is disabled
 // RUN: %clang -target x86_64 -### -o FOO -fdiagnostics-hotness-threshold=100 -fsave-optimization-record %s 2>&1 | FileCheck %s -check-prefix=CHECK-NOPASS
 
 // Pass-through:
 // RUN: %clang -target x86_64-linux -### -fuse-ld=lld -flto -fdiagnostics-hotness-threshold=100 -fsave-optimization-record -foptimization-record-passes=inline %s 2>&1 | FileCheck %s -check-prefix=CHECK-PASS-A
 // RUN: %clang -target x86_64-linux -### -o FOO -fuse-ld=gold -flto -fdiagnostics-hotness-threshold=100 -fsave-optimization-record -foptimization-record-passes=inline %s 2>&1 | FileCheck %s -check-prefix=CHECK-PASS
 // RUN: %clang -target x86_64-linux -### -o FOO -fuse-ld=lld -flto=thin -fdiagnostics-hotness-threshold=100 -fsave-optimization-record=some-format -foptimization-record-file=FOO.txt %s 2>&1 | FileCheck %s -check-prefix=CHECK-PASS-CUSTOM
 // RUN: %clang -target x86_64-linux -### -o FOO -fuse-ld=lld -flto=thin -fdiagnostics-hotness-threshold=100 -Rpass=inline -Rpass-missed=inline -Rpass-analysis=inline %s 2>&1 | FileCheck %s -check-prefix=CHECK-PASS-RPASS
+// RUN: %clang -target x86_64-linux -### -o FOO -fuse-ld=lld -flto=thin -fdiagnostics-hotness-threshold=auto -Rpass=inline -Rpass-missed=inline -Rpass-analysis=inline %s 2>&1 | FileCheck %s -check-prefix=CHECK-PASS-AUTO
 
 // CHECK-NOPASS-NOT: "--plugin-opt=opt-remarks-filename="
 // CHECK-NOPASS-NOT: "--plugin-opt=opt-remarks-passes=inline"
 // CHECK-NOPASS-NOT: "--plugin-opt=opt-remarks-format=yaml"
 // CHECK-NOPASS-NOT: "--plugin-opt=opt-remarks-hotness-threshold=100"
 
 // CHECK-PASS-A:      "--plugin-opt=opt-remarks-filename=a.out.opt.ld.yaml"
 // CHECK-PASS-A-SAME: "--plugin-opt=opt-remarks-passes=inline"
 // CHECK-PASS-A-SAME: "--plugin-opt=opt-remarks-format=yaml"
 // CHECK-PASS-A-SAME: "--plugin-opt=opt-remarks-hotness-threshold=100"
 
 // CHECK-PASS:      "--plugin-opt=opt-remarks-filename=FOO.opt.ld.yaml"
 // CHECK-PASS-SAME: "--plugin-opt=opt-remarks-passes=inline"
 // CHECK-PASS-SAME: "--plugin-opt=opt-remarks-format=yaml"
 // CHECK-PASS-SAME: "--plugin-opt=opt-remarks-hotness-threshold=100"
 
 // CHECK-PASS-CUSTOM:      "--plugin-opt=opt-remarks-filename=FOO.txt.opt.ld.some-format"
 // CHECK-PASS-CUSTOM-SAME: "--plugin-opt=opt-remarks-format=some-format"
 // CHECK-PASS-CUSTOM-SAME: "--plugin-opt=opt-remarks-hotness-threshold=100"
 
 // CHECK-PASS-RPASS:      "--plugin-opt=-pass-remarks=inline"
 // CHECK-PASS-RPASS-SAME: "--plugin-opt=-pass-remarks-missed=inline"
 // CHECK-PASS-RPASS-SAME: "--plugin-opt=-pass-remarks-analysis=inline"
 // CHECK-PASS-RPASS-SAME: "--plugin-opt=opt-remarks-hotness-threshold=100"
+
+// CHECK-PASS-AUTO:   "--plugin-opt=opt-remarks-hotness-threshold=auto"

clang/test/Frontend/Inputs/remarks-hotness.prof

+_Z7callee1v:600:600
+ 1: 600
+_Z7callee2v:1:1
+ 1: 1
+_Z7caller1v:400:400
+ 1: 400
+_Z7caller2v:1:1
+ 1: 1

clang/test/Frontend/remarks-hotness.cpp

+// Without hotness threshold, print both hot and cold remarks.
+// RUN: %clang_cc1 -triple x86_64-linux %s -emit-llvm-only -O3 \
+// RUN:     -fprofile-sample-use=%S/Inputs/remarks-hotness.prof \
+// RUN:     -Rpass=inline -Rpass-analysis=inline -Rpass-missed=inline \
+// RUN:     -fexperimental-new-pass-manager -fdiagnostics-show-hotness 2>&1 \
+// RUN:     | FileCheck -check-prefix=REMARKS %s
+
+// With auto hotness threshold, only print hot remarks.
+// RUN: %clang_cc1 -triple x86_64-linux %s -emit-llvm-only -O3 \
+// RUN:     -fprofile-sample-use=%S/Inputs/remarks-hotness.prof \
+// RUN:     -Rpass=inline -Rpass-analysis=inline -Rpass-missed=inline \
+// RUN:     -fexperimental-new-pass-manager -fdiagnostics-show-hotness \
+// RUN:     -fdiagnostics-hotness-threshold=auto 2>&1 \
+// RUN:     | FileCheck -check-prefix=HOT_CALL %s
+
+int callee1() {
+  return 1;
+}
+
+__attribute__((noinline)) int callee2() {
+  return 2;
+}
+
+// REMARKS: _Z7callee1v inlined into _Z7caller1v
+// HOT_CALL: _Z7callee1v inlined into _Z7caller1v
+int caller1() {
+  return callee1();
+}
+
+// REMARKS: _Z7callee2v not inlined into _Z7caller2v
+// HOT_CALL-NOT: _Z7callee2v not inlined into _Z7caller2v
+int caller2() {
+  return callee2();
+}

llvm/include/llvm/Analysis/ProfileSummaryInfo.h

 /// isColdCount that tells whether a given profile count is considered hot/cold
 /// based on the profile summary. This also provides convenience methods to
 /// check whether a function is hot or cold.
 
 // FIXME: Provide convenience methods to determine hotness/coldness of other IR
 // units. This would require making this depend on BFI.
 class ProfileSummaryInfo {
 private:
-  Module &M;
+  const Module &M;
   std::unique_ptr<ProfileSummary> Summary;
   void computeThresholds();
   // Count thresholds to answer isHotCount and isColdCount queries.
   Optional<uint64_t> HotCountThreshold, ColdCountThreshold;
   // True if the working set size of the code is considered huge,
   // because the number of profile counts required to reach the hot
   // percentile is above a huge threshold.
   Optional<bool> HasHugeWorkingSetSize;
   // True if the working set size of the code is considered large,
   // because the number of profile counts required to reach the hot
   // percentile is above a large threshold.
   Optional<bool> HasLargeWorkingSetSize;
   // Compute the threshold for a given cutoff.
   Optional<uint64_t> computeThreshold(int PercentileCutoff) const;
   // The map that caches the threshold values. The keys are the percentile
   // cutoff values and the values are the corresponding threshold values.
   mutable DenseMap<int, uint64_t> ThresholdCache;
 
 public:
-  ProfileSummaryInfo(Module &M) : M(M) { refresh(); }
+  ProfileSummaryInfo(const Module &M) : M(M) { refresh(); }
+
   ProfileSummaryInfo(ProfileSummaryInfo &&Arg) = default;
 
   /// If no summary is present, attempt to refresh.
   void refresh();
 
   /// Returns true if profile summary is available.
   bool hasProfileSummary() const { return Summary != nullptr; }
 

llvm/include/llvm/IR/LLVMContext.h

#undef LLVM_FIXED_MD_KIND
   ///                summary, threshold will be kept at "MAX", effectively
   ///                suppresses all remarks output.
   uint64_t getDiagnosticsHotnessThreshold() const;
 
   /// Set the minimum hotness value a diagnostic needs in order to be
   /// included in optimization diagnostics.
   void setDiagnosticsHotnessThreshold(Optional<uint64_t> Threshold);
 
+  /// Return if hotness threshold is requested from PSI.
+  bool isDiagnosticsHotnessThresholdSetFromPSI() const;
+
   /// The "main remark streamer" used by all the specialized remark streamers.
   /// This streamer keeps generic remark metadata in memory throughout the life
   /// of the LLVMContext. This metadata may be emitted in a section in object
   /// files depending on the format requirements.
   ///
   /// All specialized remark streamers should convert remarks to
   /// llvm::remarks::Remark and emit them through this streamer.
   remarks::RemarkStreamer *getMainRemarkStreamer();

llvm/include/llvm/IR/Module.h

/// @}
   /// @name Utility functions for querying and setting PGO summary
   /// @{
 
   /// Attach profile summary metadata to this module.
   void setProfileSummary(Metadata *M, ProfileSummary::Kind Kind);
 
   /// Returns profile summary metadata. When IsCS is true, use the context
   /// sensitive profile summary.
-  Metadata *getProfileSummary(bool IsCS);
+  Metadata *getProfileSummary(bool IsCS) const;
   /// @}
 
   /// Returns whether semantic interposition is to be respected.
   bool getSemanticInterposition() const;
   bool noSemanticInterposition() const;
 
   /// Set whether semantic interposition is to be respected.
   void setSemanticInterposition(bool);

llvm/lib/Analysis/OptimizationRemarkEmitter.cpp

 // that by using this service passes become dependent on BFI as well.  BFI is
 // used to compute the "hotness" of the diagnostic message.
 //===----------------------------------------------------------------------===//
 
 #include "llvm/Analysis/OptimizationRemarkEmitter.h"
 #include "llvm/Analysis/BranchProbabilityInfo.h"
 #include "llvm/Analysis/LazyBlockFrequencyInfo.h"
 #include "llvm/Analysis/LoopInfo.h"
+#include "llvm/Analysis/ProfileSummaryInfo.h"
 #include "llvm/IR/DiagnosticInfo.h"
 #include "llvm/IR/Dominators.h"
 #include "llvm/IR/LLVMContext.h"
 #include "llvm/InitializePasses.h"
 
 using namespace llvm;
 
 OptimizationRemarkEmitter::OptimizationRemarkEmitter(const Function *F)
OptimizationRemarkEmitterWrapperPass::OptimizationRemarkEmitterWrapperPass()
     : FunctionPass(ID) {
   initializeOptimizationRemarkEmitterWrapperPassPass(
       *PassRegistry::getPassRegistry());
 }
 
 bool OptimizationRemarkEmitterWrapperPass::runOnFunction(Function &Fn) {
   BlockFrequencyInfo *BFI;
 
-  if (Fn.getContext().getDiagnosticsHotnessRequested())
+  auto &Context = Fn.getContext();
+  if (Context.getDiagnosticsHotnessRequested()) {
     BFI = &getAnalysis<LazyBlockFrequencyInfoPass>().getBFI();
-  else
+    // Get hotness threshold from PSI. This should only happen once.
+    if (Context.isDiagnosticsHotnessThresholdSetFromPSI()) {
+      if (ProfileSummaryInfo *PSI =
+              &getAnalysis<ProfileSummaryInfoWrapperPass>().getPSI())
+        Context.setDiagnosticsHotnessThreshold(
+            PSI->getOrCompHotCountThreshold());
+    }
+  } else
     BFI = nullptr;
 
   ORE = std::make_unique<OptimizationRemarkEmitter>(&Fn, BFI);
   return false;
 }
 
 void OptimizationRemarkEmitterWrapperPass::getAnalysisUsage(
     AnalysisUsage &AU) const {
   LazyBlockFrequencyInfoPass::getLazyBFIAnalysisUsage(AU);
+  AU.addRequired<ProfileSummaryInfoWrapperPass>();
   AU.setPreservesAll();
 }
 
 AnalysisKey OptimizationRemarkEmitterAnalysis::Key;
 
 OptimizationRemarkEmitter
 OptimizationRemarkEmitterAnalysis::run(Function &F,
                                        FunctionAnalysisManager &AM) {
   BlockFrequencyInfo *BFI;
+  auto &Context = F.getContext();
 
-  if (F.getContext().getDiagnosticsHotnessRequested())
+  if (Context.getDiagnosticsHotnessRequested()) {
     BFI = &AM.getResult<BlockFrequencyAnalysis>(F);
-  else
+    // Get hotness threshold from PSI. This should only happen once.
+    if (Context.isDiagnosticsHotnessThresholdSetFromPSI()) {
+      auto &MAMProxy = AM.getResult<ModuleAnalysisManagerFunctionProxy>(F);
+      if (ProfileSummaryInfo *PSI =
+              MAMProxy.getCachedResult<ProfileSummaryAnalysis>(*F.getParent()))
+        Context.setDiagnosticsHotnessThreshold(
+            PSI->getOrCompHotCountThreshold());
+    }
+  } else
     BFI = nullptr;
 
   return OptimizationRemarkEmitter(&F, BFI);
 }
 
 char OptimizationRemarkEmitterWrapperPass::ID = 0;
 static const char ore_name[] = "Optimization Remark Emitter";
 #define ORE_NAME "opt-remark-emitter"
 
 INITIALIZE_PASS_BEGIN(OptimizationRemarkEmitterWrapperPass, ORE_NAME, ore_name,
                       false, true)
 INITIALIZE_PASS_DEPENDENCY(LazyBFIPass)
+INITIALIZE_PASS_DEPENDENCY(ProfileSummaryInfoWrapperPass)
 INITIALIZE_PASS_END(OptimizationRemarkEmitterWrapperPass, ORE_NAME, ore_name,
                     false, true)

llvm/lib/IR/LLVMContext.cpp

 }
 bool LLVMContext::getDiagnosticsHotnessRequested() const {
   return pImpl->DiagnosticsHotnessRequested;
 }
 
 void LLVMContext::setDiagnosticsHotnessThreshold(Optional<uint64_t> Threshold) {
   pImpl->DiagnosticsHotnessThreshold = Threshold;
 }
+
 uint64_t LLVMContext::getDiagnosticsHotnessThreshold() const {
   return pImpl->DiagnosticsHotnessThreshold.getValueOr(UINT64_MAX);
 }
 
+bool LLVMContext::isDiagnosticsHotnessThresholdSetFromPSI() const {
+  return !pImpl->DiagnosticsHotnessThreshold.hasValue();
+}
+
 remarks::RemarkStreamer *LLVMContext::getMainRemarkStreamer() {
   return pImpl->MainRemarkStreamer.get();
 }
 const remarks::RemarkStreamer *LLVMContext::getMainRemarkStreamer() const {
   return const_cast<LLVMContext *>(this)->getMainRemarkStreamer();
 }
 void LLVMContext::setMainRemarkStreamer(
     std::unique_ptr<remarks::RemarkStreamer> RemarkStreamer) {

llvm/lib/IR/LLVMRemarkStreamer.cpp

 
 Expected<std::unique_ptr<ToolOutputFile>> llvm::setupLLVMOptimizationRemarks(
     LLVMContext &Context, StringRef RemarksFilename, StringRef RemarksPasses,
     StringRef RemarksFormat, bool RemarksWithHotness,
     Optional<uint64_t> RemarksHotnessThreshold) {
   if (RemarksWithHotness)
     Context.setDiagnosticsHotnessRequested(true);
 
-  if (RemarksHotnessThreshold)
-    Context.setDiagnosticsHotnessThreshold(RemarksHotnessThreshold);
 
   if (RemarksFilename.empty())
     return nullptr;
 
   Expected<remarks::Format> Format = remarks::parseFormat(RemarksFormat);
   if (Error E = Format.takeError())
     return make_error<LLVMRemarkSetupFormatError>(std::move(E));
 

 
 Error llvm::setupLLVMOptimizationRemarks(
     LLVMContext &Context, raw_ostream &OS, StringRef RemarksPasses,
     StringRef RemarksFormat, bool RemarksWithHotness,
     Optional<uint64_t> RemarksHotnessThreshold) {
   if (RemarksWithHotness)
     Context.setDiagnosticsHotnessRequested(true);
 
-  if (RemarksHotnessThreshold)
-    Context.setDiagnosticsHotnessThreshold(RemarksHotnessThreshold);
 
   Expected<remarks::Format> Format = remarks::parseFormat(RemarksFormat);
   if (Error E = Format.takeError())
     return make_error<LLVMRemarkSetupFormatError>(std::move(E));
 
   Expected<std::unique_ptr<remarks::RemarkSerializer>> RemarkSerializer =
       remarks::createRemarkSerializer(*Format,
                                       remarks::SerializerMode::Separate, OS);

llvm/lib/IR/Module.cpp

 
 void Module::setProfileSummary(Metadata *M, ProfileSummary::Kind Kind) {
   if (Kind == ProfileSummary::PSK_CSInstr)
     setModuleFlag(ModFlagBehavior::Error, "CSProfileSummary", M);
   else
     setModuleFlag(ModFlagBehavior::Error, "ProfileSummary", M);
 }
 
-Metadata *Module::getProfileSummary(bool IsCS) {
+Metadata *Module::getProfileSummary(bool IsCS) const {
   return (IsCS ? getModuleFlag("CSProfileSummary")
                : getModuleFlag("ProfileSummary"));
 }
 
 bool Module::getSemanticInterposition() const {
   Metadata *MF = getModuleFlag("SemanticInterposition");
 
   auto *Val = cast_or_null<ConstantAsMetadata>(MF);

llvm/test/Other/optimization-remarks-auto.ll

+;; This test verifies 'auto' hotness threshold when profile summary is available.
+;;
+;; new PM
+; RUN: rm -f %t.yaml %t.hot.yaml
+; RUN: opt < %s --disable-output --enable-new-pm \
+; RUN: --passes='inline' \
+; RUN: --pass-remarks-output=%t.yaml --pass-remarks-filter='inline' \
+; RUN: --pass-remarks-with-hotness
+; RUN: FileCheck %s -check-prefix=YAML-PASS < %t.yaml
+; RUN: FileCheck %s -check-prefix=YAML-MISS < %t.yaml
+
+;; test 'auto' threshold
+; RUN: opt < %s --disable-output --enable-new-pm \
+; RUN: --passes='module(print-profile-summary,cgscc(inline))' \
+; RUN: --pass-remarks-output=%t.hot.yaml --pass-remarks-filter='inline' \
+; RUN: --pass-remarks-with-hotness --pass-remarks-hotness-threshold=auto 2>&1 | FileCheck %s
+; RUN: FileCheck %s -check-prefix=YAML-PASS < %t.hot.yaml
+; RUN: not FileCheck %s -check-prefix=YAML-MISS < %t.hot.yaml
+
+; RUN: opt < %s --disable-output --enable-new-pm \
+; RUN: --passes='module(print-profile-summary,cgscc(inline))' \
+; RUN: --pass-remarks=inline --pass-remarks-missed=inline --pass-remarks-analysis=inline \
+; RUN: --pass-remarks-with-hotness --pass-remarks-hotness-threshold=auto 2>&1 | FileCheck %s -check-prefix=CHECK-RPASS
+
+; YAML-PASS:      --- !Passed
+; YAML-PASS-NEXT: Pass:            inline
+; YAML-PASS-NEXT: Name:            Inlined
+; YAML-PASS-NEXT: Function:        caller1
+; YAML-PASS-NEXT: Hotness:         400
+
+; YAML-MISS:      --- !Missed
+; YAML-MISS-NEXT: Pass:            inline
+; YAML-MISS-NEXT: Name:            NeverInline
+; YAML-MISS-NEXT: Function:        caller2
+; YAML-MISS-NEXT: Hotness:         1
+
+; CHECK-RPASS: callee1 inlined into caller1 with (cost=-30, threshold=4500) (hotness: 400)
+; CHECK-RPASS-NOT: callee2 not inlined into caller2 because it should never be inlined (cost=never): noinline function attribute (hotness: 1)
+
+define void @callee1() !prof !20 {
+; CHECK: callee1 :hot
+entry:
+  ret void
+}
+
+; Function Attrs: noinline
+define void @callee2() noinline !prof !21 {
+; CHECK: callee2 :cold
+entry:
+  ret void
+}
+
+define void @caller1() !prof !20 {
+; CHECK: caller1 :hot
+entry:
+  call void @callee1()
+  ret void
+}
+
+define void @caller2() !prof !21 {
+; CHECK: caller2 :cold
+entry:
+  call void @callee2()
+  ret void
+}
+
+!llvm.module.flags = !{!1}
+!20 = !{!"function_entry_count", i64 400}
+!21 = !{!"function_entry_count", i64 1}
+
+!1 = !{i32 1, !"ProfileSummary", !2}
+!2 = !{!3, !4, !5, !6, !7, !8, !9, !10}
+!3 = !{!"ProfileFormat", !"InstrProf"}
+!4 = !{!"TotalCount", i64 10000}
+!5 = !{!"MaxCount", i64 10}
+!6 = !{!"MaxInternalCount", i64 1}
+!7 = !{!"MaxFunctionCount", i64 1000}
+!8 = !{!"NumCounts", i64 3}
+!9 = !{!"NumFunctions", i64 3}
+!10 = !{!"DetailedSummary", !11}
+!11 = !{!12, !13, !14}
+!12 = !{i32 10000, i64 100, i32 1}
+!13 = !{i32 999000, i64 100, i32 1}
+!14 = !{i32 999999, i64 1, i32 2}
+

llvm/test/Transforms/SampleProfile/Inputs/remarks-hotness.prof

+_Z7callee1v:600:600
+ 1: 600
+_Z7callee2v:1:1
+ 1: 1
+_Z7caller1v:400:400
+ 1: 400
+_Z7caller2v:1:1
+ 1: 1

llvm/test/Transforms/SampleProfile/remarks-hotness.ll

+;; This test verifies 'auto' hotness threshold when profile file is provided.
+;;
+;; new PM
+; RUN: rm -f %t.yaml %t.hot.yaml
+; RUN: opt %s --enable-new-pm --passes='sample-profile,cgscc(inline)' \
+; RUN: --sample-profile-file=%S/Inputs/remarks-hotness.prof \
+; RUN: -S --pass-remarks-filter=inline --pass-remarks-output=%t.yaml \
+; RUN: -pass-remarks-with-hotness --disable-output
+; RUN: FileCheck %s -check-prefix=YAML-PASS < %t.yaml
+; RUN: FileCheck %s -check-prefix=YAML-MISS < %t.yaml
+
+;; test 'auto' threshold
+; RUN: opt %s --enable-new-pm --passes='sample-profile,cgscc(inline)' \
+; RUN: --sample-profile-file=%S/Inputs/remarks-hotness.prof \
+; RUN: -S --pass-remarks-filter=inline --pass-remarks-output=%t.hot.yaml \
+; RUN: --pass-remarks-with-hotness --pass-remarks-hotness-threshold=auto --disable-output
+; RUN: FileCheck %s -check-prefix=YAML-PASS < %t.hot.yaml
+; RUN: not FileCheck %s -check-prefix=YAML-MISS < %t.hot.yaml
+
+; RUN: opt %s --enable-new-pm --passes='sample-profile,cgscc(inline)' \
+; RUN: --sample-profile-file=%S/Inputs/remarks-hotness.prof \
+; RUN: -S --pass-remarks=inline --pass-remarks-missed=inline --pass-remarks-analysis=inline \
+; RUN: --pass-remarks-with-hotness --pass-remarks-hotness-threshold=auto --disable-output 2>&1 | FileCheck %s -check-prefix=CHECK-RPASS
+
+; YAML-PASS:      --- !Passed
+; YAML-PASS-NEXT: Pass:            inline
+; YAML-PASS-NEXT: Name:            Inlined
+; YAML-PASS-NEXT: DebugLoc:        { File: remarks-hotness.cpp, Line: 10, Column: 10 }
+; YAML-PASS-NEXT: Function:        _Z7caller1v
+; YAML-PASS-NEXT: Hotness:         401
+
+; YAML-MISS:      --- !Missed
+; YAML-MISS-NEXT: Pass:            inline
+; YAML-MISS-NEXT: Name:            NeverInline
+; YAML-MISS-NEXT: DebugLoc:        { File: remarks-hotness.cpp, Line: 14, Column: 10 }
+; YAML-MISS-NEXT: Function:        _Z7caller2v
+; YAML-MISS-NEXT: Hotness:         2
+
+; CHECK-RPASS: _Z7callee1v inlined into _Z7caller1v with (cost=-30, threshold=4500) at callsite _Z7caller1v:1 (hotness: 401)
+; CHECK-RPASS-NOT: _Z7callee2v not inlined into _Z7caller2v because it should never be inlined (cost=never): noinline function attribute (hotness: 2)
+
+; ModuleID = 'remarks-hotness.cpp'
+source_filename = "remarks-hotness.cpp"
+target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+; Function Attrs: use-sample-profile
+define dso_local i32 @_Z7callee1v() #0 !dbg !7 {
+  ret i32 1, !dbg !11
+}
+
+; Function Attrs: noinline nounwind uwtable use-sample-profile
+define dso_local i32 @_Z7callee2v() #1 !dbg !12 {
+  ret i32 2, !dbg !13
+}
+
+; Function Attrs: use-sample-profile
+define dso_local i32 @_Z7caller1v() #0 !dbg !14 {
+  %1 = call i32 @_Z7callee1v(), !dbg !15
+  ret i32 %1, !dbg !16
+}
+
+; Function Attrs: use-sample-profile
+define dso_local i32 @_Z7caller2v() #0 !dbg !17 {
+  %1 = call i32 @_Z7callee2v(), !dbg !18
+  ret i32 %1, !dbg !19
+}
+
+attributes #0 = { "use-sample-profile" }
+attributes #1 = { noinline nounwind uwtable "use-sample-profile" }
+
+!llvm.dbg.cu = !{!0}
+!llvm.module.flags = !{!3, !4, !5}
+!llvm.ident = !{!6}
+
+!0 = distinct !DICompileUnit(language: DW_LANG_C_plus_plus_14, file: !1, isOptimized: false, runtimeVersion: 0, emissionKind: FullDebug, enums: !2, splitDebugInlining: false, debugInfoForProfiling: true, nameTableKind: None)
+!1 = !DIFile(filename: "remarks-hotness.cpp", directory: ".")
+!2 = !{}
+!3 = !{i32 7, !"Dwarf Version", i32 4}
+!4 = !{i32 2, !"Debug Info Version", i32 3}
+!5 = !{i32 1, !"wchar_size", i32 4}
+!6 = !{!"clang version 11.0.0"}
+!7 = distinct !DISubprogram(name: "callee1", linkageName: "_Z7callee1v", scope: !1, file: !1, line: 1, type: !8, scopeLine: 1, flags: DIFlagPrototyped, spFlags: DISPFlagDefinition, unit: !0, retainedNodes: !2)
+!8 = !DISubroutineType(types: !9)
+!9 = !{!10}
+!10 = !DIBasicType(name: "int", size: 32, encoding: DW_ATE_signed)
+!11 = !DILocation(line: 2, column: 3, scope: !7)
+!12 = distinct !DISubprogram(name: "callee2", linkageName: "_Z7callee2v", scope: !1, file: !1, line: 5, type: !8, scopeLine: 5, flags: DIFlagPrototyped, spFlags: DISPFlagDefinition, unit: !0, retainedNodes: !2)
+!13 = !DILocation(line: 6, column: 3, scope: !12)
+!14 = distinct !DISubprogram(name: "caller1", linkageName: "_Z7caller1v", scope: !1, file: !1, line: 9, type: !8, scopeLine: 9, flags: DIFlagPrototyped, spFlags: DISPFlagDefinition, unit: !0, retainedNodes: !2)
+!15 = !DILocation(line: 10, column: 10, scope: !14)
+!16 = !DILocation(line: 10, column: 3, scope: !14)
+!17 = distinct !DISubprogram(name: "caller2", linkageName: "_Z7caller2v", scope: !1, file: !1, line: 13, type: !8, scopeLine: 13, flags: DIFlagPrototyped, spFlags: DISPFlagDefinition, unit: !0, retainedNodes: !2)
+!18 = !DILocation(line: 14, column: 10, scope: !17)
+!19 = !DILocation(line: 14, column: 3, scope: !17)
+