[Remarks][1/2] Expand remarks hotness threshold option support in more tools

[weiwang]

Summary

This is the #1 of 2 changes that make remarks hotness threshold option
available in more tools. The changes also allow the threshold to sync with
hotness threshold from profile summary with special value 'auto'.

This change modifies the interface of lto::setupLLVMOptimizationRemarks() to
accept remarks hotness threshold. Update all the tools that use it with remarks
hotness threshold options:

• lld: '--opt-remarks-hotness-threshold='
• llvm-lto2: '--pass-remarks-hotness-threshold='
• llvm-lto: '--lto-pass-remarks-hotness-threshold='
• gold plugin: '-plugin-opt=opt-remarks-hotness-threshold='

NOTE: Internal remarks hotness threshold type is updated to accommodate the next change. The 'auto' value is allowed, but has no effect.

[weiwang]

This is the 1st of 3 dependent patches:

1. [lld] Enable remarks hotness filtering in lld: https://reviews.llvm.org/D85809
2. [clang] Pass-through remarks options to lld: https://reviews.llvm.org/D85810
3. [remarks] Optimization remarks hotness filtering from profile summary: https://reviews.llvm.org/D85808

[weiwang]

clang-format

[thegameg]

LGTM, thanks for plugging all these options through!

[tejohnson]

The changes to *LTOCodeGenerator and the test using llvm-lto are unrelated to lld (they are the old LTO API used by ld64). So they should be moved to a different patch or the patch name changed. Would be good to add llvm-lto2 tests which exercise the new LTO API changed in LTO.cpp, LTOBackend.cpp etc. Note those changes are broader than just lld (also used by gold-plugin).

[weiwang]

changes:

1. revert the change made to llvm-lto.
2. add the new option to llvm-lto2 wit test cases.

ld.gold's behavior is not affected by the change: No threshold is enforced.

[weiwang]

In D85809#2214246, @tejohnson wrote:

The changes to *LTOCodeGenerator and the test using llvm-lto are unrelated to lld (they are the old LTO API used by ld64). So they should be moved to a different patch or the patch name changed. Would be good to add llvm-lto2 tests which exercise the new LTO API changed in LTO.cpp, LTOBackend.cpp etc. Note those changes are broader than just lld (also used by gold-plugin).

I have removed the change to llvm-lto and added support in llvm-lto2. The new option is not exposed to ld.gold, so it is not affected.

[MaskRay]

lld/test/ELF/lto/opt-remarks.ll

10

For continuation lines, we usually indent by 2 columns.

This file was created in 2017, so some lines do not obey this rule. For new RUN lines you probably need to follow the rule.

15

Replace cat ... | cmd with cmd < ...

Actually FileCheck supports --input-file, but since it is long, some people don't use it.

llvm/test/LTO/Resolution/X86/diagnostic-handler-remarks-with-hotness.ll

20

cat ... | cmd => cmd < ...

[MaskRay]

llvm/include/llvm/LTO/Config.h

124

Add "0 means no threshold".

[tejohnson]

In D85809#2217341, @weiwang wrote:

In D85809#2214246, @tejohnson wrote:

The changes to *LTOCodeGenerator and the test using llvm-lto are unrelated to lld (they are the old LTO API used by ld64). So they should be moved to a different patch or the patch name changed. Would be good to add llvm-lto2 tests which exercise the new LTO API changed in LTO.cpp, LTOBackend.cpp etc. Note those changes are broader than just lld (also used by gold-plugin).

I have removed the change to llvm-lto and added support in llvm-lto2. The new option is not exposed to ld.gold, so it is not affected.

You might as well have the legacy LTO and llvm-lto support, presumably it could be useful for ld64 etc that use it. Since you already implemented it.

And I suggest adding the support to gold-plugin.cpp as well (added a comment to one of your follow on patches to this effect as well).

[thegameg]

In D85809#2217341, @weiwang wrote:

I have removed the change to llvm-lto

It would be nice if you can bring that back (maybe in a separate patch as Teresa suggested). I can see it being useful through ld64/libLTO, and for consistency it doesn't sound like it would cause any harm, since these options are passed as "debug options" through -mllvm.

[tejohnson]

In D85809#2218725, @thegameg wrote:

In D85809#2217341, @weiwang wrote:

I have removed the change to llvm-lto

It would be nice if you can bring that back (maybe in a separate patch as Teresa suggested). I can see it being useful through ld64/libLTO, and for consistency it doesn't sound like it would cause any harm, since these options are passed as "debug options" through -mllvm.

Just to clarify, I wasn't suggesting putting it in another patch, but just bringing it back here (plus adding in the gold-plugin support). The other patch I was referring to was the follow on patch D85810 where I also suggested adding support to gold-plugin.cpp.

[thegameg]

In D85809#2218742, @tejohnson wrote:

Just to clarify, I wasn't suggesting putting it in another patch, but just bringing it back here (plus adding in the gold-plugin support). The other patch I was referring to was the follow on patch D85810 where I also suggested adding support to gold-plugin.cpp.

Ah, looks like I missed your comment from this morning. I agree with that.

[weiwang]

update:
add remarks hotness threshold to all the tools uses lto::setupLLVMOptimizationRemarks

[weiwang]

format update

[weiwang]

format update

[MaskRay]

lld/ELF/Config.h

282

This should be moved above

llvm::StringRef optRemarksFilename;
llvm::StringRef optRemarksPasses;
llvm::StringRef optRemarksFormat;

lld/ELF/Options.td

550

<integer> might be preciser.

llvm/tools/gold/gold-plugin.cpp

303

Drop unneeded inner braces

943

stray ;

[weiwang]

lld/ELF/Config.h

282

Thanks for the comments! I will upload another revision to fix this, and also address Teresa's comment in https://reviews.llvm.org/D85808.

[weiwang]

update:

1. move internal remarks hotness threshold type change and its related handling code from child diff into this one. With this move, remarks hotness threshold now accepts 'auto' value, but its handling is currently disabled.
2. format change.

[weiwang]

Try to fix a windows build error. No msvc available locally, so even with this change, the build could still fail.

Suspected a naming conflict between llvm::Optional class and llvm::Optional enum defined in CommandLine.h.

[weiwang]

try to fix windows build error. Use fully qualified name for enum value Optional.

[weiwang]

try to fix windows build linking error.

[weiwang]

Really appreciate the feedbacks. If there is no more, could anyone approve 3 changes? Thanks.

[weiwang]

@tejohnson @MaskRay Do you have other comments to the change?

[MaskRay]

lld/ELF/Config.h

115

Don't repeat the include/llvm/LTO/Config.h comment here

lld just forwards these options. Their help messages should be in llvm/LTO/

lld/ELF/Driver.cpp

995

delete trailing period

llvm/include/llvm/Remarks/HotnessThresholdParser.h

20

empty line before namespace

llvm/include/llvm/Support/CommandLine.h

1484

Unrelated change

[weiwang]

Address Comments.

[weiwang]

Thanks @MaskRay ! I am really sorry I didn't catch up your comments earlier. My email somehow stops giving me notifications on diff updates.

llvm/include/llvm/Support/CommandLine.h

1484

This change is need for window msvc build test to pass. Previously the msvc build failed during template instantiation at cl::opt<Optional<uint64_t>, false, remarks::HotnessThresholdParser>. The first argument for the base class ctor is an enum type llvm::cl::Optional, but msvc somehow tries to map it to llvm::Optional. I am not sure why the compiler thinks they are the same. I've tried several ways to fix this, but it only works by explicitly giving its fully qualified name. Linux build does not have this issue.

[weiwang]

@MaskRay @tejohnson, do you have other comments regarding this change and its dependent?

[tejohnson]

LGTM, but please wait for @MaskRay since he had the most recent comments on this patch.

[weiwang]

Thanks! I will wait for his input.

[MaskRay]

lld/ELF/Options.td

550

This is not done

llvm/include/llvm/LTO/Config.h

128

Does swapping the meaning of 0 and None make more sense? None looks like a better default value.

llvm/include/llvm/Remarks/HotnessThresholdParser.h

48

Regarding the MSVC compiliation issue. If you use llvm::Optional here, does it work?

llvm/test/LTO/Resolution/X86/diagnostic-handler-remarks-with-hotness.ll

12

Full stop.

30

We don't use not FileCheck. If you want to check a pattern does not exist, use https://llvm.org/docs/CommandGuide/FileCheck.html#the-check-not-directive

53

Please fix the not FIleCheck

[weiwang]

lld/ELF/Options.td

550

Since the value can be integer or 'auto', maybe <value> is more appropriate here?

[weiwang]

llvm/test/LTO/Resolution/X86/diagnostic-handler-remarks-with-hotness.ll

30

Sorry, I am not sure how to apply "CHECK-EMPTY" in this case. The input is empty, and I think "CHECK-EMPTY" assumes a non-empty input (requires a "CHECK" line above it).

[weiwang]

llvm/include/llvm/Remarks/HotnessThresholdParser.h

48

Good point! I completed overlooked this place. I will give it a try in the next revision.

[MaskRay]

llvm/test/LTO/Resolution/X86/diagnostic-handler-remarks-with-hotness.ll

30

Use | count 0 to check empty output

[weiwang]

llvm/include/llvm/LTO/Config.h

128

None means unknown, more of a transient value. It would be updated by the actual hotness value from PSI later. Also currently 0 is treated as disabled, I figured keeping its current meaning requires less change.

[weiwang]

update test case

[weiwang]

update to address MSVC build error

[weiwang]

llvm/include/llvm/Remarks/HotnessThresholdParser.h

48

Well, just tried it, looks like the windows build failed at the same place:
https://buildkite.com/llvm-project/premerge-checks/builds/17124

FAILED: lib/LTO/CMakeFiles/LLVMLTO.dir/LTOCodeGenerator.cpp.obj 
sccache C:\BuildTools\VC\Tools\MSVC\14.27.29110\bin\Hostx64\x64\cl.exe  /nologo /TP -DBUILD_EXAMPLES -DGTEST_HAS_RTTI=0 -DUNICODE -D_CRT_NONSTDC_NO_DEPRECATE -D_CRT_NONSTDC_NO_WARNINGS -D_CRT_SECURE_NO_DEPRECATE -D_CRT_SECURE_NO_WARNINGS -D_HAS_EXCEPTIONS=0 -D_SCL_SECURE_NO_DEPRECATE -D_SCL_SECURE_NO_WARNINGS -D_UNICODE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Ilib\LTO -IC:\ws\w64\llvm-project\premerge-checks\llvm\lib\LTO -Iinclude -IC:\ws\w64\llvm-project\premerge-checks\llvm\include /DWIN32 /D_WINDOWS   /Zc:inline /Zc:__cplusplus /Zc:strictStrings /Oi /Zc:rvalueCast /bigobj /W4 -wd4141 -wd4146 -wd4244 -wd4267 -wd4291 -wd4351 -wd4456 -wd4457 -wd4458 -wd4459 -wd4503 -wd4624 -wd4722 -wd4100 -wd4127 -wd4512 -wd4505 -wd4610 -wd4510 -wd4702 -wd4245 -wd4706 -wd4310 -wd4701 -wd4703 -wd4389 -wd4611 -wd4805 -wd4204 -wd4577 -wd4091 -wd4592 -wd4319 -wd4709 -wd4324 -w14062 -we4238 /Gw /MD /O2 /Ob2    /EHs-c- /GR- -UNDEBUG -std:c++14 /showIncludes /Folib\LTO\CMakeFiles\LLVMLTO.dir\LTOCodeGenerator.cpp.obj /Fdlib\LTO\CMakeFiles\LLVMLTO.dir\LLVMLTO.pdb /FS -c C:\ws\w64\llvm-project\premerge-checks\llvm\lib\LTO\LTOCodeGenerator.cpp
C:\ws\w64\llvm-project\premerge-checks\llvm\include\llvm/Support/CommandLine.h(1484): error C2275: 'llvm::Optional<uint64_t>': illegal use of this type as an expression
C:\ws\w64\llvm-project\premerge-checks\llvm\lib\LTO\LTOCodeGenerator.cpp(97): note: see reference to function template instantiation 'llvm::cl::opt<llvm::Optional<uint64_t>,false,llvm::remarks::HotnessThresholdParser>::opt<char[35],llvm::cl::desc,llvm::cl::value_desc,llvm::cl::initializer<int>,llvm::cl::OptionHidden>(const char (&)[35],const llvm::cl::desc &,const llvm::cl::value_desc &,const llvm::cl::initializer<int> &,const llvm::cl::OptionHidden &)' being compiled
C:\ws\w64\llvm-project\premerge-checks\llvm\lib\LTO\LTOCodeGenerator.cpp(92): note: see reference to function template instantiation 'llvm::cl::opt<llvm::Optional<uint64_t>,false,llvm::remarks::HotnessThresholdParser>::opt<char[35],llvm::cl::desc,llvm::cl::value_desc,llvm::cl::initializer<int>,llvm::cl::OptionHidden>(const char (&)[35],const llvm::cl::desc &,const llvm::cl::value_desc &,const llvm::cl::initializer<int> &,const llvm::cl::OptionHidden &)' being compiled

I am going to revert it back.

[weiwang]

revert back MSVC fix

[weiwang]

@tejohnson @MaskRay Do you have other comments?

[weiwang]

rebase

[tejohnson]

lgtm

lld/ELF/Config.h

struct Configuration {
   llvm::StringRef ltoAAPipeline;
   llvm::StringRef ltoCSProfileFile;
   llvm::StringRef ltoNewPmPasses;
   llvm::StringRef ltoObjPath;
   llvm::StringRef ltoSampleProfile;
   llvm::StringRef mapFile;
   llvm::StringRef outputFile;
   llvm::StringRef optRemarksFilename;
+
+  // The minimum hotness value a diagnostic needs in order to be included in
+  // optimization diagnostics.
+  //
+  // The threshold is an Optional value, which maps to one of the 3 states:
+  // 1. 0            => threshold disabled. All emarks will be printed.
+  // 2. positive int => manual threshold by user. Remarks with hotness exceed
+  //                    threshold will be printed.
+  // 3. None         => 'auto' threshold by user. The actual value is not
+  //                    available at command line, but will be synced with
+  //                    hotness threhold from profile summary during
+  //                    compilation.
+  //
+  // If threshold option is not specified, it is disabled by default.
+  llvm::Optional<uint64_t> optRemarksHotnessThreshold = 0;
   llvm::StringRef optRemarksPasses;
   llvm::StringRef optRemarksFormat;
   llvm::StringRef progName;
   llvm::StringRef printArchiveStats;
   llvm::StringRef printSymbolOrder;
   llvm::StringRef soName;
   llvm::StringRef sysroot;
   llvm::StringRef thinLTOCacheDir;
struct Configuration {
   uint64_t mipsGotSize;
   uint64_t zStackSize;
   unsigned ltoPartitions;
   unsigned ltoo;
   unsigned optimize;
   StringRef thinLTOJobs;
   unsigned timeTraceGranularity;
   int32_t splitStackAdjustSize;
 
   // The following config options do not directly correspond to any
   // particular command line options.
 
   // True if we need to pass through relocations in input files to the
   // output file. Usually false because we consume relocations.
   bool copyRelocs;
 
   // True if the target is ELF64. False if ELF32.

lld/ELF/Driver.cpp

 #include "lld/Common/Memory.h"
 #include "lld/Common/Strings.h"
 #include "lld/Common/TargetOptionsCommandFlags.h"
 #include "lld/Common/Version.h"
 #include "llvm/ADT/SetVector.h"
 #include "llvm/ADT/StringExtras.h"
 #include "llvm/ADT/StringSwitch.h"
 #include "llvm/LTO/LTO.h"
+#include "llvm/Remarks/HotnessThresholdParser.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/Compression.h"
 #include "llvm/Support/GlobPattern.h"
 #include "llvm/Support/LEB128.h"
 #include "llvm/Support/Parallel.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/TarWriter.h"
 #include "llvm/Support/TargetSelect.h"
static void readConfigs(opt::InputArgList &args) {
   config->mmapOutputFile =
       args.hasFlag(OPT_mmap_output_file, OPT_no_mmap_output_file, true);
   config->nmagic = args.hasFlag(OPT_nmagic, OPT_no_nmagic, false);
   config->noinhibitExec = args.hasArg(OPT_noinhibit_exec);
   config->nostdlib = args.hasArg(OPT_nostdlib);
   config->oFormatBinary = isOutputFormatBinary(args);
   config->omagic = args.hasFlag(OPT_omagic, OPT_no_omagic, false);
   config->optRemarksFilename = args.getLastArgValue(OPT_opt_remarks_filename);
+
+  // Parse remarks hotness threshold. Valid value is either integer or 'auto'.
+  if (auto *arg = args.getLastArg(OPT_opt_remarks_hotness_threshold)) {
+    auto resultOrErr = remarks::parseHotnessThresholdOption(arg->getValue());
+    if (!resultOrErr)
+      error(arg->getSpelling() + ": invalid argument '" + arg->getValue() +
+            "', only integer or 'auto' is supported.");
+    else
+      config->optRemarksHotnessThreshold = *resultOrErr;
+  }
+
   config->optRemarksPasses = args.getLastArgValue(OPT_opt_remarks_passes);
   config->optRemarksWithHotness = args.hasArg(OPT_opt_remarks_with_hotness);
   config->optRemarksFormat = args.getLastArgValue(OPT_opt_remarks_format);
   config->optimize = args::getInteger(args, OPT_O, 1);
   config->orphanHandling = getOrphanHandling(args);
   config->outputFile = args.getLastArgValue(OPT_o);
   config->pie = args.hasFlag(OPT_pie, OPT_no_pie, false);
   config->printIcfSections =

lld/ELF/LTO.cpp

static lto::Config createConfig() {
   // Set up a custom pipeline if we've been asked to.
   c.OptPipeline = std::string(config->ltoNewPmPasses);
   c.AAPipeline = std::string(config->ltoAAPipeline);
 
   // Set up optimization remarks if we've been asked to.
   c.RemarksFilename = std::string(config->optRemarksFilename);
   c.RemarksPasses = std::string(config->optRemarksPasses);
   c.RemarksWithHotness = config->optRemarksWithHotness;
+  c.RemarksHotnessThreshold = config->optRemarksHotnessThreshold;
   c.RemarksFormat = std::string(config->optRemarksFormat);
 
   c.SampleProfile = std::string(config->ltoSampleProfile);
   c.UseNewPM = config->ltoNewPassManager;
   c.DebugPassManager = config->ltoDebugPassManager;
   c.DwoDir = std::string(config->dwoDir);
 
   c.HasWholeProgramVisibility = config->ltoWholeProgramVisibility;

lld/ELF/Options.td

 def lto_sample_profile: JJ<"lto-sample-profile=">,
   HelpText<"Sample profile file path">;
 def lto_whole_program_visibility: FF<"lto-whole-program-visibility">,
   HelpText<"Asserts that the LTO link has whole program visibility">;
 def disable_verify: F<"disable-verify">;
 defm mllvm: Eq<"mllvm", "Additional arguments to forward to LLVM's option processing">;
 def opt_remarks_filename: Separate<["--"], "opt-remarks-filename">,
   HelpText<"YAML output file for optimization remarks">;
+defm opt_remarks_hotness_threshold: EEq<"opt-remarks-hotness-threshold",
+  "Minimum profile count required for an optimization remark to be output."
+  " Use 'auto' to apply the threshold from profile summary.">,
+  MetaVarName<"<value>">;
 def opt_remarks_passes: Separate<["--"], "opt-remarks-passes">,
   HelpText<"Regex for the passes that need to be serialized to the output file">;
 def opt_remarks_with_hotness: FF<"opt-remarks-with-hotness">,
   HelpText<"Include hotness information in the optimization remarks file">;
 def opt_remarks_format: Separate<["--"], "opt-remarks-format">,
   HelpText<"The format used for serializing remarks (default: YAML)">;
 def save_temps: F<"save-temps">;
 def lto_basic_block_sections: JJ<"lto-basic-block-sections=">,
def: F<"plugin-opt=new-pass-manager">,
   Alias<lto_new_pass_manager>, HelpText<"Alias for --lto-new-pass-manager">;
 def: F<"plugin-opt=cs-profile-generate">,
   Alias<lto_cs_profile_generate>, HelpText<"Alias for --lto-cs-profile-generate">;
 def: J<"plugin-opt=cs-profile-path=">,
   Alias<lto_cs_profile_file>, HelpText<"Alias for --lto-cs-profile-file">;
 def: J<"plugin-opt=obj-path=">,
   Alias<lto_obj_path_eq>,
   HelpText<"Alias for --lto-obj-path=">;
+def: J<"plugin-opt=opt-remarks-filename=">,
+  Alias<opt_remarks_filename>,
+  HelpText<"Alias for --opt-remarks-filename">;
+def: J<"plugin-opt=opt-remarks-passes=">,
+  Alias<opt_remarks_passes>,
+  HelpText<"Alias for --opt-remarks-passes">;
+def: J<"plugin-opt=opt-remarks-format=">,
+  Alias<opt_remarks_format>,
+  HelpText<"Alias for --opt-remarks-format">;
+def: F<"plugin-opt=opt-remarks-with-hotness">,
+  Alias<opt_remarks_with_hotness>,
+  HelpText<"Alias for --opt-remarks-with_hotness">;
+def: J<"plugin-opt=opt-remarks-hotness-threshold=">,
+  Alias<opt_remarks_hotness_threshold>,
+  HelpText<"Alias for --opt-remarks-hotness-threshold">;
 def: J<"plugin-opt=sample-profile=">,
   Alias<lto_sample_profile>, HelpText<"Alias for --lto-sample-profile">;
 def: F<"plugin-opt=save-temps">, Alias<save_temps>, HelpText<"Alias for --save-temps">;
 def: F<"plugin-opt=thinlto-emit-imports-files">,
   Alias<thinlto_emit_imports_files>,
   HelpText<"Alias for --thinlto-emit-imports-files">;
 def: F<"plugin-opt=thinlto-index-only">,
   Alias<thinlto_index_only>,

lld/test/ELF/lto/opt-remarks.ll

 ; REQUIRES: x86
 ; RUN: llvm-as %s -o %t.o
 
-; RUN: rm -f %t.yaml
+; RUN: rm -f %t.yaml %t1.yaml %t.hot.yaml %t.t300.yaml %t.t301.yaml
 ; RUN: ld.lld --opt-remarks-filename %t.yaml %t.o -o %t -shared -save-temps
 ; RUN: llvm-dis %t.0.4.opt.bc -o - | FileCheck %s
 ; RUN: ld.lld --opt-remarks-with-hotness --opt-remarks-filename %t.hot.yaml \
 ; RUN:  %t.o -o %t -shared
+; RUN: ld.lld --opt-remarks-with-hotness --opt-remarks-hotness-threshold=300 \
+; RUN:   --opt-remarks-filename %t.t300.yaml %t.o -o %t -shared
+; RUN: ld.lld --opt-remarks-with-hotness --opt-remarks-hotness-threshold=301 \
+; RUN:   --opt-remarks-filename %t.t301.yaml %t.o -o %t -shared
 ; RUN: cat %t.yaml | FileCheck %s -check-prefix=YAML
 ; RUN: cat %t.hot.yaml | FileCheck %s -check-prefix=YAML-HOT
+; RUN: FileCheck %s -check-prefix=YAML-HOT < %t.t300.yaml
+; RUN: count 0 < %t.t301.yaml
 ; RUN: ld.lld --opt-remarks-filename %t1.yaml --opt-remarks-passes inline %t.o \
 ; RUN: -o /dev/null -shared
 ; RUN: cat %t1.yaml | FileCheck %s -check-prefix=YAML-PASSES
 ; RUN: ld.lld --opt-remarks-filename %t1.yaml --opt-remarks-format yaml %t.o \
 ; RUN: -o /dev/null -shared
-; RUN: cat %t.yaml | FileCheck %s -check-prefix=YAML
+; RUN: FileCheck %s -check-prefix=YAML < %t1.yaml
 
 ; Check that @tinkywinky is inlined after optimizations.
 ; CHECK-LABEL: define i32 @main
 ; CHECK-NEXT:  %a.i = {{.*}}call i32 @patatino()
 ; CHECK-NEXT:  ret i32 %a.i
 ; CHECK-NEXT: }
 
 ; YAML:      --- !Passed

llvm/include/llvm/IR/LLVMContext.h

#undef LLVM_FIXED_MD_KIND
   /// Return if a code hotness metric should be included in optimization
   /// diagnostics.
   bool getDiagnosticsHotnessRequested() const;
   /// Set if a code hotness metric should be included in optimization
   /// diagnostics.
   void setDiagnosticsHotnessRequested(bool Requested);
 
   /// Return the minimum hotness value a diagnostic would need in order
-  /// to be included in optimization diagnostics. If there is no minimum, this
-  /// returns None.
+  /// to be included in optimization diagnostics.
+  ///
+  /// Three possible return values:
+  /// 0            - threshold is disabled. Everything will be printed out.
+  /// positive int - threshold is set.
+  /// UINT64_MAX   - threshold is not yet set, and needs to be synced from
+  ///                profile summary. Note that in case of missing profile
+  ///                summary, threshold will be kept at "MAX", effectively
+  ///                suppresses all remarks output.
   uint64_t getDiagnosticsHotnessThreshold() const;
 
   /// Set the minimum hotness value a diagnostic needs in order to be
   /// included in optimization diagnostics.
-  void setDiagnosticsHotnessThreshold(uint64_t Threshold);
+  void setDiagnosticsHotnessThreshold(Optional<uint64_t> Threshold);
 
   /// The "main remark streamer" used by all the specialized remark streamers.
   /// This streamer keeps generic remark metadata in memory throughout the life
   /// of the LLVMContext. This metadata may be emitted in a section in object
   /// files depending on the format requirements.
   ///
   /// All specialized remark streamers should convert remarks to
   /// llvm::remarks::Remark and emit them through this streamer.

llvm/include/llvm/IR/LLVMRemarkStreamer.h

using LLVMRemarkSetupErrorInfo<
       LLVMRemarkSetupFormatError>::LLVMRemarkSetupErrorInfo;
 };
 
 /// Setup optimization remarks that output to a file.
 Expected<std::unique_ptr<ToolOutputFile>>
 setupLLVMOptimizationRemarks(LLVMContext &Context, StringRef RemarksFilename,
                              StringRef RemarksPasses, StringRef RemarksFormat,
                              bool RemarksWithHotness,
-                             unsigned RemarksHotnessThreshold = 0);
+                             Optional<uint64_t> RemarksHotnessThreshold = 0);
 
 /// Setup optimization remarks that output directly to a raw_ostream.
 /// \p OS is managed by the caller and should be open for writing as long as \p
 /// Context is streaming remarks to it.
-Error setupLLVMOptimizationRemarks(LLVMContext &Context, raw_ostream &OS,
-                                   StringRef RemarksPasses,
-                                   StringRef RemarksFormat,
-                                   bool RemarksWithHotness,
-                                   unsigned RemarksHotnessThreshold = 0);
+Error setupLLVMOptimizationRemarks(
+    LLVMContext &Context, raw_ostream &OS, StringRef RemarksPasses,
+    StringRef RemarksFormat, bool RemarksWithHotness,
+    Optional<uint64_t> RemarksHotnessThreshold = 0);
 
 } // end namespace llvm
 
 #endif // LLVM_IR_LLVMREMARKSTREAMER_H

llvm/include/llvm/LTO/Config.h

struct Config {
   std::string RemarksFilename = "";
 
   /// Optimization remarks pass filter.
   std::string RemarksPasses = "";
 
   /// Whether to emit optimization remarks with hotness informations.
   bool RemarksWithHotness = false;
 
+  /// The minimum hotness value a diagnostic needs in order to be included in
+  /// optimization diagnostics.
+  ///
+  /// The threshold is an Optional value, which maps to one of the 3 states:
+  /// 1. 0            => threshold disabled. All emarks will be printed.
+  /// 2. positive int => manual threshold by user. Remarks with hotness exceed
+  ///                    threshold will be printed.
+  /// 3. None         => 'auto' threshold by user. The actual value is not
+  ///                    available at command line, but will be synced with
+  ///                    hotness threhold from profile summary during
+  ///                    compilation.
+  ///
+  /// If threshold option is not specified, it is disabled by default.
+  llvm::Optional<uint64_t> RemarksHotnessThreshold = 0;
+
   /// The format used for serializing remarks (default: YAML).
   std::string RemarksFormat = "";
 
   /// Whether to emit the pass manager debuggging informations.
   bool DebugPassManager = false;
 
   /// Statistics output file path.
   std::string StatsFile;

llvm/include/llvm/LTO/LTO.h

 /// Given the original \p Path to an output file, replace any path
 /// prefix matching \p OldPrefix with \p NewPrefix. Also, create the
 /// resulting directory if it does not yet exist.
 std::string getThinLTOOutputFile(const std::string &Path,
                                  const std::string &OldPrefix,
                                  const std::string &NewPrefix);
 
 /// Setup optimization remarks.
-Expected<std::unique_ptr<ToolOutputFile>>
-setupLLVMOptimizationRemarks(LLVMContext &Context, StringRef RemarksFilename,
-                             StringRef RemarksPasses, StringRef RemarksFormat,
-                             bool RemarksWithHotness, int Count = -1);
+Expected<std::unique_ptr<ToolOutputFile>> setupLLVMOptimizationRemarks(
+    LLVMContext &Context, StringRef RemarksFilename, StringRef RemarksPasses,
+    StringRef RemarksFormat, bool RemarksWithHotness,
+    Optional<uint64_t> RemarksHotnessThreshold = 0, int Count = -1);
 
 /// Setups the output file for saving statistics.
 Expected<std::unique_ptr<ToolOutputFile>>
 setupStatsFile(StringRef StatsFilename);
 
 class LTO;
 struct SymbolResolution;
 class ThinBackendProc;

llvm/include/llvm/Remarks/HotnessThresholdParser.h

+//===- HotnessThresholdParser.h - Parser for hotness threshold --*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+///
+/// \file
+/// This file implements a simple parser to decode commandline option for
+/// remarks hotness threshold that supports both int and a special 'auto' value.
+///
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_REMARKS_HOTNESSTHRESHOLDPARSER_H
+#define LLVM_REMARKS_HOTNESSTHRESHOLDPARSER_H
+
+#include "llvm/ADT/Optional.h"
+#include "llvm/Support/CommandLine.h"
+namespace llvm {
+namespace remarks {
+
+// Parse remarks hotness threshold argument value.
+// Valid option values are
+// 1. integer: manually specified threshold; or
+// 2. string 'auto': automatically get threshold from profile summary.
+//
+// Return None Optional if 'auto' is specified, indicating the value will
+// be filled later during PSI.
+inline Expected<Optional<uint64_t>> parseHotnessThresholdOption(StringRef Arg) {
+  if (Arg == "auto")
+    return None;
+
+  int64_t Val;
+  if (Arg.getAsInteger(10, Val))
+    return createStringError(llvm::inconvertibleErrorCode(),
+                             "Not an integer: %s", Arg.data());
+
+  // Negative integer effectively means no threshold
+  return Val < 0 ? 0 : Val;
+}
+
+// A simple CL parser for '*-remarks-hotness-threshold='
+class HotnessThresholdParser : public cl::parser<Optional<uint64_t>> {
+public:
+  HotnessThresholdParser(cl::Option &O) : cl::parser<Optional<uint64_t>>(O) {}
+
+  bool parse(cl::Option &O, StringRef ArgName, StringRef Arg,
+             Optional<uint64_t> &V) {
+    auto ResultOrErr = parseHotnessThresholdOption(Arg);
+    if (!ResultOrErr)
+      return O.error("Invalid argument '" + Arg +
+                     "', only integer or 'auto' is supported.");
+
+    V = *ResultOrErr;
+    return false;
+  }
+};
+
+} // namespace remarks
+} // namespace llvm
+#endif // LLVM_REMARKS_HOTNESSTHRESHOLDPARSER_H

llvm/include/llvm/Support/CommandLine.h

public:
   template <class T> DataType &operator=(const T &Val) {
     this->setValue(Val);
     Callback(Val);
     return this->getValue();
   }
 
   template <class... Mods>
   explicit opt(const Mods &... Ms)
-      : Option(Optional, NotHidden), Parser(*this) {
+      : Option(llvm::cl::Optional, NotHidden), Parser(*this) {
     apply(this, Ms...);
     done();
   }
 
   void setCallback(
       std::function<void(const typename ParserClass::parser_data_type &)> CB) {
     Callback = CB;
   }

llvm/lib/IR/LLVMContext.cpp

 
 void LLVMContext::setDiagnosticsHotnessRequested(bool Requested) {
   pImpl->DiagnosticsHotnessRequested = Requested;
 }
 bool LLVMContext::getDiagnosticsHotnessRequested() const {
   return pImpl->DiagnosticsHotnessRequested;
 }
 
-void LLVMContext::setDiagnosticsHotnessThreshold(uint64_t Threshold) {
+void LLVMContext::setDiagnosticsHotnessThreshold(Optional<uint64_t> Threshold) {
   pImpl->DiagnosticsHotnessThreshold = Threshold;
 }
 uint64_t LLVMContext::getDiagnosticsHotnessThreshold() const {
-  return pImpl->DiagnosticsHotnessThreshold;
+  return pImpl->DiagnosticsHotnessThreshold.getValueOr(UINT64_MAX);
 }
 
 remarks::RemarkStreamer *LLVMContext::getMainRemarkStreamer() {
   return pImpl->MainRemarkStreamer.get();
 }
 const remarks::RemarkStreamer *LLVMContext::getMainRemarkStreamer() const {
   return const_cast<LLVMContext *>(this)->getMainRemarkStreamer();
 }

llvm/lib/IR/LLVMContextImpl.h

public:
   /// The main remark streamer used by all the other streamers (e.g. IR, MIR,
   /// frontends, etc.). This should only be used by the specific streamers, and
   /// never directly.
   std::unique_ptr<remarks::RemarkStreamer> MainRemarkStreamer;
 
   std::unique_ptr<DiagnosticHandler> DiagHandler;
   bool RespectDiagnosticFilters = false;
   bool DiagnosticsHotnessRequested = false;
-  uint64_t DiagnosticsHotnessThreshold = 0;
+  /// The minimum hotness value a diagnostic needs in order to be included in
+  /// optimization diagnostics.
+  ///
+  /// The threshold is an Optional value, which maps to one of the 3 states:
+  /// 1). 0            => threshold disabled. All emarks will be printed.
+  /// 2). positive int => manual threshold by user. Remarks with hotness exceed
+  ///                     threshold will be printed.
+  /// 3). None         => 'auto' threshold by user. The actual value is not
+  ///                     available at command line, but will be synced with
+  ///                     hotness threhold from profile summary during
+  ///                     compilation.
+  ///
+  /// State 1 and 2 are considered as terminal states. State transition is
+  /// only allowed from 3 to 2, when the threshold is first synced with profile
+  /// summary. This ensures that the threshold is set only once and stays
+  /// constant.
+  ///
+  /// If threshold option is not specified, it is disabled (0) by default.
+  Optional<uint64_t> DiagnosticsHotnessThreshold = 0;
+
   /// The specialized remark streamer used by LLVM's OptimizationRemarkEmitter.
   std::unique_ptr<LLVMRemarkStreamer> LLVMRS;
 
   LLVMContext::YieldCallbackTy YieldCallback = nullptr;
   void *YieldOpaqueHandle = nullptr;
 
   using IntMapTy =
       DenseMap<APInt, std::unique_ptr<ConstantInt>, DenseMapAPIntKeyInfo>;

llvm/lib/IR/LLVMRemarkStreamer.cpp

 
 char LLVMRemarkSetupFileError::ID = 0;
 char LLVMRemarkSetupPatternError::ID = 0;
 char LLVMRemarkSetupFormatError::ID = 0;
 
 Expected<std::unique_ptr<ToolOutputFile>> llvm::setupLLVMOptimizationRemarks(
     LLVMContext &Context, StringRef RemarksFilename, StringRef RemarksPasses,
     StringRef RemarksFormat, bool RemarksWithHotness,
-    unsigned RemarksHotnessThreshold) {
+    Optional<uint64_t> RemarksHotnessThreshold) {
   if (RemarksWithHotness)
     Context.setDiagnosticsHotnessRequested(true);
 
   if (RemarksHotnessThreshold)
     Context.setDiagnosticsHotnessThreshold(RemarksHotnessThreshold);
 
   if (RemarksFilename.empty())
     return nullptr;
Expected<std::unique_ptr<ToolOutputFile>> llvm::setupLLVMOptimizationRemarks(
 
   if (!RemarksPasses.empty())
     if (Error E = Context.getMainRemarkStreamer()->setFilter(RemarksPasses))
       return make_error<LLVMRemarkSetupPatternError>(std::move(E));
 
   return std::move(RemarksFile);
 }
 
-Error llvm::setupLLVMOptimizationRemarks(LLVMContext &Context, raw_ostream &OS,
-                                         StringRef RemarksPasses,
-                                         StringRef RemarksFormat,
-                                         bool RemarksWithHotness,
-                                         unsigned RemarksHotnessThreshold) {
+Error llvm::setupLLVMOptimizationRemarks(
+    LLVMContext &Context, raw_ostream &OS, StringRef RemarksPasses,
+    StringRef RemarksFormat, bool RemarksWithHotness,
+    Optional<uint64_t> RemarksHotnessThreshold) {
   if (RemarksWithHotness)
     Context.setDiagnosticsHotnessRequested(true);
 
   if (RemarksHotnessThreshold)
     Context.setDiagnosticsHotnessThreshold(RemarksHotnessThreshold);
 
   Expected<remarks::Format> Format = remarks::parseFormat(RemarksFormat);
   if (Error E = Format.takeError())

llvm/lib/LTO/LTO.cpp

Error LTO::run(AddStreamFn AddStream, NativeObjectCache Cache) {
 
   return Result;
 }
 
 Error LTO::runRegularLTO(AddStreamFn AddStream) {
   // Setup optimization remarks.
   auto DiagFileOrErr = lto::setupLLVMOptimizationRemarks(
       RegularLTO.CombinedModule->getContext(), Conf.RemarksFilename,
-      Conf.RemarksPasses, Conf.RemarksFormat, Conf.RemarksWithHotness);
+      Conf.RemarksPasses, Conf.RemarksFormat, Conf.RemarksWithHotness,
+      Conf.RemarksHotnessThreshold);
   if (!DiagFileOrErr)
     return DiagFileOrErr.takeError();
 
   // Finalize linking of regular LTO modules containing summaries now that
   // we have computed liveness information.
   for (auto &M : RegularLTO.ModsWithSummaries)
     if (Error Err = linkRegularLTO(std::move(M),
                                    /*LivenessFromIndex=*/true))
for (auto &Mod : ModuleMap) {
     ++Task;
   }
 
   return BackendProc->wait();
 }
 
 Expected<std::unique_ptr<ToolOutputFile>> lto::setupLLVMOptimizationRemarks(
     LLVMContext &Context, StringRef RemarksFilename, StringRef RemarksPasses,
-    StringRef RemarksFormat, bool RemarksWithHotness, int Count) {
+    StringRef RemarksFormat, bool RemarksWithHotness,
+    Optional<uint64_t> RemarksHotnessThreshold, int Count) {
   std::string Filename = std::string(RemarksFilename);
   // For ThinLTO, file.opt.<format> becomes
   // file.opt.<format>.thin.<num>.<format>.
   if (!Filename.empty() && Count != -1)
     Filename =
         (Twine(Filename) + ".thin." + llvm::utostr(Count) + "." + RemarksFormat)
             .str();
 
   auto ResultOrErr = llvm::setupLLVMOptimizationRemarks(
-      Context, Filename, RemarksPasses, RemarksFormat, RemarksWithHotness);
+      Context, Filename, RemarksPasses, RemarksFormat, RemarksWithHotness,
+      RemarksHotnessThreshold);
   if (Error E = ResultOrErr.takeError())
     return std::move(E);
 
   if (*ResultOrErr)
     (*ResultOrErr)->keep();
 
   return ResultOrErr;
 }

llvm/lib/LTO/LTOBackend.cpp

Error lto::thinBackend(const Config &Conf, unsigned Task, AddStreamFn AddStream,
   if (!TOrErr)
     return TOrErr.takeError();
 
   std::unique_ptr<TargetMachine> TM = createTargetMachine(Conf, *TOrErr, Mod);
 
   // Setup optimization remarks.
   auto DiagFileOrErr = lto::setupLLVMOptimizationRemarks(
       Mod.getContext(), Conf.RemarksFilename, Conf.RemarksPasses,
-      Conf.RemarksFormat, Conf.RemarksWithHotness, Task);
+      Conf.RemarksFormat, Conf.RemarksWithHotness, Conf.RemarksHotnessThreshold,
+      Task);
   if (!DiagFileOrErr)
     return DiagFileOrErr.takeError();
   auto DiagnosticOutputFile = std::move(*DiagFileOrErr);
 
   // Set the partial sample profile ratio in the profile summary module flag of
   // the module, if applicable.
   Mod.setPartialSampleProfileRatio(CombinedIndex);
 

llvm/lib/LTO/LTOCodeGenerator.cpp

 #include "llvm/InitializePasses.h"
 #include "llvm/LTO/LTO.h"
 #include "llvm/LTO/legacy/LTOModule.h"
 #include "llvm/LTO/legacy/UpdateCompilerUsed.h"
 #include "llvm/Linker/Linker.h"
 #include "llvm/MC/MCAsmInfo.h"
 #include "llvm/MC/MCContext.h"
 #include "llvm/MC/SubtargetFeature.h"
+#include "llvm/Remarks/HotnessThresholdParser.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/Host.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/Signals.h"
 #include "llvm/Support/TargetRegistry.h"
 #include "llvm/Support/TargetSelect.h"
 #include "llvm/Support/ToolOutputFile.h"

 #endif
     cl::Hidden);
 
 cl::opt<bool> RemarksWithHotness(
     "lto-pass-remarks-with-hotness",
     cl::desc("With PGO, include profile count in optimization remarks"),
     cl::Hidden);
 
+cl::opt<Optional<uint64_t>, false, remarks::HotnessThresholdParser>
+    RemarksHotnessThreshold(
+        "lto-pass-remarks-hotness-threshold",
+        cl::desc("Minimum profile count required for an "
+                 "optimization remark to be output."
+                 " Use 'auto' to apply the threshold from profile summary."),
+        cl::value_desc("uint or 'auto'"), cl::init(0), cl::Hidden);
+
 cl::opt<std::string>
     RemarksFilename("lto-pass-remarks-output",
                     cl::desc("Output filename for pass remarks"),
                     cl::value_desc("filename"));
 
 cl::opt<std::string>
     RemarksPasses("lto-pass-remarks-filter",
                   cl::desc("Only record optimization remarks from passes whose "

 
 /// Optimize merged modules using various IPO passes
 bool LTOCodeGenerator::optimize(bool DisableVerify, bool DisableInline,
                                 bool DisableGVNLoadPRE,
                                 bool DisableVectorization) {
   if (!this->determineTarget())
     return false;
 
-  auto DiagFileOrErr =
-      lto::setupLLVMOptimizationRemarks(Context, RemarksFilename, RemarksPasses,
-                                        RemarksFormat, RemarksWithHotness);
+  auto DiagFileOrErr = lto::setupLLVMOptimizationRemarks(
+      Context, RemarksFilename, RemarksPasses, RemarksFormat,
+      RemarksWithHotness, RemarksHotnessThreshold);
   if (!DiagFileOrErr) {
     errs() << "Error: " << toString(DiagFileOrErr.takeError()) << "\n";
     report_fatal_error("Can't get an output file for the remarks");
   }
   DiagnosticOutputFile = std::move(*DiagFileOrErr);
 
   // Setup output file to emit statistics.
   auto StatsFileOrErr = lto::setupStatsFile(LTOStatsFile);

llvm/lib/LTO/ThinLTOCodeGenerator.cpp

 #include "llvm/IR/Mangler.h"
 #include "llvm/IR/PassTimingInfo.h"
 #include "llvm/IR/Verifier.h"
 #include "llvm/IRReader/IRReader.h"
 #include "llvm/LTO/LTO.h"
 #include "llvm/LTO/SummaryBasedOptimizations.h"
 #include "llvm/MC/SubtargetFeature.h"
 #include "llvm/Object/IRObjectFile.h"
+#include "llvm/Remarks/HotnessThresholdParser.h"
 #include "llvm/Support/CachePruning.h"
 #include "llvm/Support/Debug.h"
 #include "llvm/Support/Error.h"
 #include "llvm/Support/FileUtilities.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/SHA1.h"
 #include "llvm/Support/SmallVectorMemoryBuffer.h"
 #include "llvm/Support/TargetRegistry.h"

 #define DEBUG_TYPE "thinlto"
 
 namespace llvm {
 // Flags -discard-value-names, defined in LTOCodeGenerator.cpp
 extern cl::opt<bool> LTODiscardValueNames;
 extern cl::opt<std::string> RemarksFilename;
 extern cl::opt<std::string> RemarksPasses;
 extern cl::opt<bool> RemarksWithHotness;
+extern cl::opt<Optional<uint64_t>, false, remarks::HotnessThresholdParser>
+    RemarksHotnessThreshold;
 extern cl::opt<std::string> RemarksFormat;
 }
 
 namespace {
 
 // Default to using all available threads in the system, but using only one
 // thred per core, as indicated by the usage of
 // heavyweight_hardware_concurrency() below.
for (auto IndexCount : ModulesOrdering) {
           }
         }
 
         LLVMContext Context;
         Context.setDiscardValueNames(LTODiscardValueNames);
         Context.enableDebugTypeODRUniquing();
         auto DiagFileOrErr = lto::setupLLVMOptimizationRemarks(
             Context, RemarksFilename, RemarksPasses, RemarksFormat,
-            RemarksWithHotness, count);
+            RemarksWithHotness, RemarksHotnessThreshold, count);
         if (!DiagFileOrErr) {
           errs() << "Error: " << toString(DiagFileOrErr.takeError()) << "\n";
           report_fatal_error("ThinLTO: Can't get an output file for the "
                              "remarks");
         }
 
         // Parse module now
         auto TheModule = loadModuleFromInput(Mod.get(), Context, false,

llvm/test/LTO/Resolution/X86/diagnostic-handler-remarks-with-hotness.ll

 ; RUN: llvm-as < %s >%t.bc
 
 ; Check pass remarks emitted to YAML file
 ; RUN: rm -f %t.yaml
 ; RUN: llvm-lto2 run -pass-remarks-output=%t.yaml \
 ; RUN:           -pass-remarks-with-hotness \
 ; RUN:           -r %t.bc,tinkywinky,p \
 ; RUN:           -r %t.bc,patatino,px \
 ; RUN:           -r %t.bc,main,px -o %t.o %t.bc
 ; RUN: cat %t.yaml | FileCheck %s -check-prefix=YAML
 
+; Check low threshold allows remarks to emit
+; RUN: rm -f %t.t300.yaml
+; RUN: llvm-lto2 run -pass-remarks-output=%t.t300.yaml \
+; RUN:           -pass-remarks-with-hotness \
+; RUN:           -pass-remarks-hotness-threshold=300 \
+; RUN:           -r %t.bc,tinkywinky,p \
+; RUN:           -r %t.bc,patatino,px \
+; RUN:           -r %t.bc,main,px -o %t.o %t.bc
+; RUN: FileCheck %s -check-prefix=YAML < %t.t300.yaml
+
+; Check high threshold disallows remarks to emit
+; RUN: rm -f %t.t301.yaml
+; RUN: llvm-lto2 run -pass-remarks-output=%t.t301.yaml \
+; RUN:           -pass-remarks-with-hotness \
+; RUN:           -pass-remarks-hotness-threshold=301 \
+; RUN:           -r %t.bc,tinkywinky,p \
+; RUN:           -r %t.bc,patatino,px \
+; RUN:           -r %t.bc,main,px -o %t.o %t.bc
+; RUN: not FileCheck %s -check-prefix=YAML < %t.t301.yaml
+
 ; Check pass remarks emitted to stderr
 ; RUN: llvm-lto2 run -pass-remarks=inline \
 ; RUN:           -pass-remarks-with-hotness \
 ; RUN:           -r %t.bc,tinkywinky,p \
 ; RUN:           -r %t.bc,patatino,px \
 ; RUN:           -r %t.bc,main,px -o %t.o %t.bc 2>&1 | FileCheck %s
 
+; Check low threshold allows remarks to emit
+; RUN: llvm-lto2 run -pass-remarks=inline \
+; RUN:           -pass-remarks-with-hotness \
+; RUN:           -pass-remarks-hotness-threshold=300 \
+; RUN:           -r %t.bc,tinkywinky,p \
+; RUN:           -r %t.bc,patatino,px \
+; RUN:           -r %t.bc,main,px -o %t.o %t.bc 2>&1 | FileCheck %s
+
+; Check high threshold disallows remarks to emit
+; RUN: llvm-lto2 run -pass-remarks=inline \
+; RUN:           -pass-remarks-with-hotness \
+; RUN:           -pass-remarks-hotness-threshold=301 \
+; RUN:           -r %t.bc,tinkywinky,p \
+; RUN:           -r %t.bc,patatino,px \
+; RUN:           -r %t.bc,main,px -o %t.o %t.bc 2>&1 | not FileCheck %s
+
 ; YAML: --- !Passed
 ; YAML-NEXT: Pass:            inline
 ; YAML-NEXT: Name:            Inlined
 ; YAML-NEXT: Function:        main
 ; YAML-NEXT: Hotness:         300
 ; YAML-NEXT: Args:
 ; YAML-NEXT:   - Callee:          tinkywinky
 ; YAML-NEXT:   - String:          ' inlined into '

llvm/test/LTO/X86/diagnostic-handler-remarks-with-hotness.ll

 ; Check that the hotness attribute is included in the optimization record file
 ; with -lto-pass-remarks-with-hotness.
 
 ; RUN: llvm-as < %s >%t.bc
-; RUN: rm -f %t.yaml
+; RUN: rm -f %t.yaml %t.t300.yaml %t.t301.yaml
 ; RUN: llvm-lto -lto-pass-remarks-output=%t.yaml \
 ; RUN:          -lto-pass-remarks-with-hotness \
 ; RUN:          -exported-symbol _main -o %t.o %t.bc
 ; RUN: cat %t.yaml | FileCheck -check-prefix=YAML %s
 
+; RUN: llvm-lto -lto-pass-remarks-output=%t.t300.yaml \
+; RUN:          -lto-pass-remarks-with-hotness \
+; RUN:          -lto-pass-remarks-hotness-threshold=300 \
+; RUN:          -exported-symbol _main -o %t.o %t.bc
+; RUN: FileCheck -check-prefix=YAML %s < %t.t300.yaml
+
+; RUN: llvm-lto -lto-pass-remarks-output=%t.t301.yaml \
+; RUN:          -lto-pass-remarks-with-hotness \
+; RUN:          -lto-pass-remarks-hotness-threshold=301 \
+; RUN:          -exported-symbol _main -o %t.o %t.bc
+; RUN: not FileCheck -check-prefix=YAML %s < %t.t301.yaml
+
 ; YAML:      --- !Passed
 ; YAML-NEXT: Pass:            inline
 ; YAML-NEXT: Name:            Inlined
 ; YAML-NEXT: Function:        main
 ; YAML-NEXT: Hotness:         300
 ; YAML-NEXT: Args:
 ; YAML-NEXT:   - Callee:          foo
 ; YAML-NEXT:   - String:          ' inlined into '

llvm/test/tools/gold/X86/opt-remarks.ll

 ; Test plugin options for opt-remarks.
 ; RUN: llvm-as %s -o %t.o
 ; RUN: %gold -m elf_x86_64 -plugin %llvmshlibdir/LLVMgold%shlibext -shared \
 ; RUN:    -plugin-opt=save-temps \
 ; RUN:    -plugin-opt=opt-remarks-passes=inline \
 ; RUN:    -plugin-opt=opt-remarks-format=yaml \
 ; RUN:    -plugin-opt=opt-remarks-filename=%t.yaml %t.o -o %t2.o 2>&1
 ; RUN: llvm-dis %t2.o.0.4.opt.bc -o - | FileCheck %s
 ; RUN: %gold -m elf_x86_64 -plugin %llvmshlibdir/LLVMgold%shlibext -shared \
 ; RUN:    -plugin-opt=opt-remarks-passes=inline \
 ; RUN:    -plugin-opt=opt-remarks-format=yaml \
 ; RUN:    -plugin-opt=opt-remarks-with-hotness \
 ; RUN:    -plugin-opt=opt-remarks-filename=%t.hot.yaml %t.o -o %t2.o 2>&1
+; RUN: %gold -m elf_x86_64 -plugin %llvmshlibdir/LLVMgold%shlibext -shared \
+; RUN:    -plugin-opt=opt-remarks-passes=inline \
+; RUN:    -plugin-opt=opt-remarks-format=yaml \
+; RUN:    -plugin-opt=opt-remarks-with-hotness \
+; RUN:    -plugin-opt=opt-remarks-hotness-threshold=300 \
+; RUN:    -plugin-opt=opt-remarks-filename=%t.t300.yaml %t.o -o %t2.o 2>&1
+; RUN: %gold -m elf_x86_64 -plugin %llvmshlibdir/LLVMgold%shlibext -shared \
+; RUN:    -plugin-opt=opt-remarks-passes=inline \
+; RUN:    -plugin-opt=opt-remarks-format=yaml \
+; RUN:    -plugin-opt=opt-remarks-with-hotness \
+; RUN:    -plugin-opt=opt-remarks-hotness-threshold=301 \
+; RUN:    -plugin-opt=opt-remarks-filename=%t.t301.yaml %t.o -o %t2.o 2>&1
 ; RUN: cat %t.yaml | FileCheck %s -check-prefix=YAML
 ; RUN: cat %t.hot.yaml | FileCheck %s -check-prefix=YAML-HOT
+; RUN: FileCheck %s -check-prefix=YAML-HOT < %t.t300.yaml
+; RUN: count 0 < %t.t301.yaml
 
 ; Check that @f is inlined after optimizations.
 ; CHECK-LABEL: define i32 @_start
 ; CHECK-NEXT:  %a.i = tail call i32 @bar()
 ; CHECK-NEXT:  ret i32 %a.i
 ; CHECK-NEXT: }
 
 ; YAML: --- !Missed

 ; YAML-NEXT:   - String:          ' with '
 ; YAML-NEXT:   - String:          '(cost='
 ; YAML-NEXT:   - Cost:            '0'
 ; YAML-NEXT:   - String:          ', threshold='
 ; YAML-NEXT:   - Threshold:       '337'
 ; YAML-NEXT:   - String:          ')'
 ; YAML-NEXT: ...
 
-; YAML-HOT: ...
 ; YAML-HOT: --- !Passed
 ; YAML-HOT: Pass:            inline
 ; YAML-HOT-NEXT: Name:            Inlined
 ; YAML-HOT-NEXT: Function:        _start
 ; YAML-HOT-NEXT: Hotness:         300
 ; YAML-HOT-NEXT: Args:
 ; YAML-HOT-NEXT:   - Callee:          f
 ; YAML-HOT-NEXT:   - String:          ' inlined into '

llvm/tools/gold/gold-plugin.cpp

 #include "llvm/Bitcode/BitcodeWriter.h"
 #include "llvm/CodeGen/CommandFlags.h"
 #include "llvm/Config/config.h" // plugin-api.h requires HAVE_STDINT_H
 #include "llvm/IR/Constants.h"
 #include "llvm/IR/DiagnosticPrinter.h"
 #include "llvm/LTO/Caching.h"
 #include "llvm/LTO/LTO.h"
 #include "llvm/Object/Error.h"
+#include "llvm/Remarks/HotnessThresholdParser.h"
 #include "llvm/Support/CachePruning.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/Host.h"
 #include "llvm/Support/ManagedStatic.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/TargetSelect.h"
 #include "llvm/Support/Threading.h"
 #include "llvm/Support/raw_ostream.h"
 #include <list>
 #include <map>
 #include <plugin-api.h>
 #include <string>
 #include <system_error>
 #include <utility>
 #include <vector>
 
 // FIXME: remove this declaration when we stop maintaining Ubuntu Quantal and
 // Precise and Debian Wheezy (binutils 2.23 is required)
 #define LDPO_PIE 3
#endif
   static std::string stats_file;
   // Asserts that LTO link has whole program visibility
   static bool whole_program_visibility = false;
 
   // Optimization remarks filename, accepted passes and hotness options
   static std::string RemarksFilename;
   static std::string RemarksPasses;
   static bool RemarksWithHotness = false;
+  static Optional<uint64_t> RemarksHotnessThreshold = 0;
   static std::string RemarksFormat;
 
   // Context sensitive PGO options.
   static std::string cs_profile_path;
   static bool cs_pgo_gen = false;
 
   static void process_plugin_option(const char *opt_)
   {
static void process_plugin_option(const char *opt_)
     } else if (opt.consume_front("dwo_dir=")) {
       dwo_dir = std::string(opt);
     } else if (opt.consume_front("opt-remarks-filename=")) {
       RemarksFilename = std::string(opt);
     } else if (opt.consume_front("opt-remarks-passes=")) {
       RemarksPasses = std::string(opt);
     } else if (opt == "opt-remarks-with-hotness") {
       RemarksWithHotness = true;
+    } else if (opt.consume_front("opt-remarks-hotness-threshold=")) {
+      auto ResultOrErr = remarks::parseHotnessThresholdOption(opt);
+      if (!ResultOrErr)
+        message(LDPL_FATAL, "Invalid remarks hotness threshold: %s", opt);
+      else
+        RemarksHotnessThreshold = *ResultOrErr;
     } else if (opt.consume_front("opt-remarks-format=")) {
       RemarksFormat = std::string(opt);
     } else if (opt.consume_front("stats-file=")) {
       stats_file = std::string(opt);
     } else {
       // Save this option to pass to the code generator.
       // ParseCommandLineOptions() expects argv[0] to be program name. Lazily
       // add that.
static std::unique_ptr<LTO> createLTO(IndexWriteCallback OnIndexWrite,
   Conf.RunCSIRInstr = options::cs_pgo_gen;
 
   Conf.DwoDir = options::dwo_dir;
 
   // Set up optimization remarks handling.
   Conf.RemarksFilename = options::RemarksFilename;
   Conf.RemarksPasses = options::RemarksPasses;
   Conf.RemarksWithHotness = options::RemarksWithHotness;
+  Conf.RemarksHotnessThreshold = options::RemarksHotnessThreshold;
   Conf.RemarksFormat = options::RemarksFormat;
 
   // Use new pass manager if set in driver
   Conf.UseNewPM = options::new_pass_manager;
   // Debug new pass manager if requested
   Conf.DebugPassManager = options::debug_pass_manager;
 
   Conf.HasWholeProgramVisibility = options::whole_program_visibility;
 

llvm/tools/llc/llc.cpp

 #include "llvm/IR/LLVMRemarkStreamer.h"
 #include "llvm/IR/LegacyPassManager.h"
 #include "llvm/IR/Module.h"
 #include "llvm/IR/Verifier.h"
 #include "llvm/IRReader/IRReader.h"
 #include "llvm/InitializePasses.h"
 #include "llvm/MC/SubtargetFeature.h"
 #include "llvm/Pass.h"
+#include "llvm/Remarks/HotnessThresholdParser.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/Debug.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/FormattedStream.h"
 #include "llvm/Support/Host.h"
 #include "llvm/Support/InitLLVM.h"
 #include "llvm/Support/ManagedStatic.h"
 #include "llvm/Support/PluginLoader.h"

 
 static cl::list<std::string> IncludeDirs("I", cl::desc("include search path"));
 
 static cl::opt<bool> RemarksWithHotness(
     "pass-remarks-with-hotness",
     cl::desc("With PGO, include profile count in optimization remarks"),
     cl::Hidden);
 
-static cl::opt<unsigned>
-    RemarksHotnessThreshold("pass-remarks-hotness-threshold",
-                            cl::desc("Minimum profile count required for "
-                                     "an optimization remark to be output"),
-                            cl::Hidden);
+static cl::opt<Optional<uint64_t>, false, remarks::HotnessThresholdParser>
+    RemarksHotnessThreshold(
+        "pass-remarks-hotness-threshold",
+                 "an optimization remark to be output. "
+                 "Use 'auto' to apply the threshold from profile summary."),
+        cl::value_desc("N or 'auto'"), cl::init(0), cl::Hidden);
 
 static cl::opt<std::string>
     RemarksFilename("pass-remarks-output",
                     cl::desc("Output filename for pass remarks"),
                     cl::value_desc("filename"));
 
 static cl::opt<std::string>
     RemarksPasses("pass-remarks-filter",

llvm/tools/llvm-lto2/llvm-lto2.cpp

 //===----------------------------------------------------------------------===//
 
 #include "llvm/Bitcode/BitcodeReader.h"
 #include "llvm/CodeGen/CommandFlags.h"
 #include "llvm/IR/DiagnosticPrinter.h"
 #include "llvm/LTO/Caching.h"
 #include "llvm/LTO/LTO.h"
 #include "llvm/Passes/PassPlugin.h"
+#include "llvm/Remarks/HotnessThresholdParser.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/InitLLVM.h"
 #include "llvm/Support/PluginLoader.h"
 #include "llvm/Support/TargetSelect.h"
 #include "llvm/Support/Threading.h"
 
 using namespace llvm;
static cl::opt<std::string> DefaultTriple(
     cl::desc(
         "Replace unspecified target triples in input files with this triple"));
 
 static cl::opt<bool> RemarksWithHotness(
     "pass-remarks-with-hotness",
     cl::desc("With PGO, include profile count in optimization remarks"),
     cl::Hidden);
 
+cl::opt<Optional<uint64_t>, false, remarks::HotnessThresholdParser>
+    RemarksHotnessThreshold(
+        "pass-remarks-hotness-threshold",
+        cl::desc("Minimum profile count required for an "
+                 "optimization remark to be output."
+                 " Use 'auto' to apply the threshold from profile summary."),
+        cl::value_desc("uint or 'auto'"), cl::init(0), cl::Hidden);
+
 static cl::opt<std::string>
     RemarksFilename("pass-remarks-output",
                     cl::desc("Output filename for pass remarks"),
                     cl::value_desc("filename"));
 
 static cl::opt<std::string>
     RemarksPasses("pass-remarks-filter",
                   cl::desc("Only record optimization remarks from passes whose "
static int run(int argc, char **argv) {
   if (SaveTemps)
     check(Conf.addSaveTemps(OutputFilename + "."),
           "Config::addSaveTemps failed");
 
   // Optimization remarks.
   Conf.RemarksFilename = RemarksFilename;
   Conf.RemarksPasses = RemarksPasses;
   Conf.RemarksWithHotness = RemarksWithHotness;
+  Conf.RemarksHotnessThreshold = RemarksHotnessThreshold;
   Conf.RemarksFormat = RemarksFormat;
 
   Conf.SampleProfile = SamplePGOFile;
   Conf.CSIRProfile = CSPGOFile;
   Conf.RunCSIRInstr = RunCSIRInstr;
 
   // Run a custom pipeline, if asked for.
   Conf.OptPipeline = OptPipeline;

llvm/tools/opt/opt.cpp

 #include "llvm/IR/LegacyPassNameParser.h"
 #include "llvm/IR/Module.h"
 #include "llvm/IR/Verifier.h"
 #include "llvm/IRReader/IRReader.h"
 #include "llvm/InitializePasses.h"
 #include "llvm/LinkAllIR.h"
 #include "llvm/LinkAllPasses.h"
 #include "llvm/MC/SubtargetFeature.h"
+#include "llvm/Remarks/HotnessThresholdParser.h"
 #include "llvm/Support/Debug.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/Host.h"
 #include "llvm/Support/InitLLVM.h"
 #include "llvm/Support/PluginLoader.h"
 #include "llvm/Support/SourceMgr.h"
 #include "llvm/Support/SystemUtils.h"
 #include "llvm/Support/TargetRegistry.h"
TimeTraceFile("time-trace-file",
                     cl::desc("Specify time trace file destination"),
                     cl::value_desc("filename"));
 
 static cl::opt<bool> RemarksWithHotness(
     "pass-remarks-with-hotness",
     cl::desc("With PGO, include profile count in optimization remarks"),
     cl::Hidden);
 
-static cl::opt<unsigned>
-    RemarksHotnessThreshold("pass-remarks-hotness-threshold",
-                            cl::desc("Minimum profile count required for "
-                                     "an optimization remark to be output"),
-                            cl::Hidden);
+static cl::opt<Optional<uint64_t>, false, remarks::HotnessThresholdParser>
+    RemarksHotnessThreshold(
+        "pass-remarks-hotness-threshold",
+                 "an optimization remark to be output. "
+                 "Use 'auto' to apply the threshold from profile summary."),
+        cl::value_desc("N or 'auto'"), cl::init(0), cl::Hidden);
 
 static cl::opt<std::string>
     RemarksFilename("pass-remarks-output",
                     cl::desc("Output filename for pass remarks"),
                     cl::value_desc("filename"));
 
 static cl::opt<std::string>
     RemarksPasses("pass-remarks-filter",