[NFC] Refactor MBB hotness/coldness into templated PSI functions

[shenhan]

Summary

[NFC] In D152399, we calculate BPI->BFI in MachineFunctionSplit pass just to use PSI->isFunctionHotInCallGraph, which is expensive. Instead, we can implement this directly with MBFI.

Reviewer @wenlei mentioned in his commet, that machine_size_opts already has isFunctionColdInCallGraph, isFunctionHotInCallGraphNthPercentile, etc implemented. These can be refactored and reused across MFS and machine size opts.

This CL does this - it refactors out those internal static functions into PSI as templated functions, so they can be accessed easily.

[wenlei]

Thanks for the follow up. It doesn't feel like this belongs to MBFI either. BFI for IR doesn't contain any of the hotness query - instead these are all part of PSI.

I'm wondering if we can make some template functions in PSI that cover both IR and MIR hotness queries, since BFI and MBFI have very similar APIs. We have precedence for PGO code sharing between IR and MIR (SampleProfileLoaderBaseImpl<BT>).

[shenhan]

In D152758#4416070, @wenlei wrote:

Thanks for the follow up. It doesn't feel like this belongs to MBFI either. BFI for IR doesn't contain any of the hotness query - instead these are all part of PSI.

I'm wondering if we can make some template functions in PSI that cover both IR and MIR hotness queries, since BFI and MBFI have very similar APIs. We have precedence for PGO code sharing between IR and MIR (SampleProfileLoaderBaseImpl<BT>).

Thanks. That makes sense. I've reverted all changes to MBFI. And moved the code to PSI with templates to accommodate Function/MachineFunction, BasicBlock/MachineBasicBlock, BFI/MBFI. Also moved some function implementations from the cpp file to the header, as they are templated. (I have not chosen to template the whole PSI class, because currently I see limited usage for such changes. (And the scope of template-PSI would require a serials of CLs.)).

[wenlei]

+@davidxl

llvm/include/llvm/Analysis/ProfileSummaryInfo.h

137

This is better down through template specialization. Check how SampleProfileInference<BT>::findUnlikelyJumps handles Invoke in IR.

You can extract only this part into a small helper, and specialize that helper based on template type (i.e. check call/invoke for IR, no-op for MIR)

160

same here

279

same here

283

same here

llvm/lib/Analysis/ProfileSummaryInfo.cpp

98–100

nit: this comment seem lost?

[davidxl]

llvm/include/llvm/Analysis/ProfileSummaryInfo.h

137

Agree. If we plan to have this for MBFI in the future, we can just extract this part out as a helper function too.

161

Can this part share code with the Hot checking counter part?

[shenhan]

llvm/include/llvm/Analysis/ProfileSummaryInfo.h

137

Done.

161

Done by extracting part of this into templated function "findTotalCallCount", leaving hot/cold checking part unchanged. This keeps code simple and also easy to understand.

[shenhan]

Build failure is caused by one time out case:
Timed Out Tests (1):

MLIR :: Examples/standalone/test.toy

Unlikely to be caused by this change. Rerunning locally and will retry the bot later.

[wenlei]

llvm/include/llvm/Analysis/ProfileSummaryInfo.h

269

Call and Invoke are specific to IR version, so checking that in generic implementation is a bit weird. I'd suggest let generic implementation return std::nullopt, and have IR version check on Call and Invoke.

269

also nit: findTotalCallCount->getTotalCallCount to be consistent with getProfileCount.

[shenhan]

llvm/include/llvm/Analysis/ProfileSummaryInfo.h

269

Good point. Done.

[wenlei]

llvm/lib/CodeGen/MachineSizeOpts.cpp

26–27

Now with PSI directly supporting IR and MIR, do we still need MachineBasicBlockBFIAdapter?

This function below (main use of adapter) can now interface with templated PSI APIs (isHotBlockNthPercentile/isColdBlockNthPercentile/isColdBlock) directly without going through adapter.

template<typename AdapterT, typename BlockTOrBlockFreq, typename BFIT>
bool shouldOptimizeForSizeImpl(BlockTOrBlockFreq BBOrBlockFreq, ProfileSummaryInfo *PSI,
                               BFIT *BFI, PGSOQueryType QueryType)

[shenhan]

llvm/lib/CodeGen/MachineSizeOpts.cpp

26–27

Thanks. Good hint.

Removed "AdapterT" template-parameter from "shouldOptimizeForSizeImpl" and "shouldFuncOptimizeForSizeImpl".

This also leads to the removal of "MachineBasicBlockBFIAdapter" class. Now MachineSizeOpts.cpp looks neat.

[shenhan]

llvm/include/llvm/Analysis/ProfileSummaryInfo.h

127

Also renamed all occurrences of template type name "FreqInfoType" -> "BFIT", so this is in accordance with existing code.

[shenhan]

Previous build bot failure was caused by 31af18bccea95fe1ae8aa2c51cf7c8e92a1c208e, which was later reverted by c9a8a0e8a9b267fae0352af352e9c735fef019f7. Synced this patch past problematic CLs.

[wenlei]

Thanks for the refactoring and cleanup. LGTM.

llvm/include/llvm/Analysis/ProfileSummaryInfo.h

126–143

nit: FunctionType->FuncT to be consistent with BFIT and other template definitions.

[shenhan]

Thanks @wenlei for the review.

llvm/include/llvm/Analysis/ProfileSummaryInfo.h

126–143

Done for all occurrences.

[aidengrossman]

This seems to have broken quite a few builders. Do you mind reverting if you don't have a quick fix available?

llvm/include/llvm/Analysis/ProfileSummaryInfo.h

 // information.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_ANALYSIS_PROFILESUMMARYINFO_H
 #define LLVM_ANALYSIS_PROFILESUMMARYINFO_H
 
 #include "llvm/ADT/DenseMap.h"
+#include "llvm/Analysis/BlockFrequencyInfo.h"
+#include "llvm/CodeGen/MachineFunction.h"
+#include "llvm/IR/Function.h"
+#include "llvm/IR/Instructions.h"
 #include "llvm/IR/PassManager.h"
 #include "llvm/IR/ProfileSummary.h"
 #include "llvm/Pass.h"
 #include <memory>
 #include <optional>
 
 namespace llvm {
 class BasicBlock;
-class BlockFrequencyInfo;
 class CallBase;
-class Function;
 
 /// Analysis providing profile information.
 ///
 /// This is an immutable analysis pass that provides ability to query global
 /// (program-level) profile information. The main APIs are isHotCount and
 /// isColdCount that tells whether a given profile count is considered hot/cold
 /// based on the profile summary. This also provides convenience methods to
 /// check whether a function is hot or cold.
std::optional<uint64_t> getProfileCount(const CallBase &CallInst,
                                           BlockFrequencyInfo *BFI,
                                           bool AllowSynthetic = false) const;
   /// Returns true if module \c M has partial-profile sample profile.
   bool hasPartialSampleProfile() const;
   /// Returns true if the working set size of the code is considered huge.
   bool hasHugeWorkingSetSize() const;
   /// Returns true if the working set size of the code is considered large.
   bool hasLargeWorkingSetSize() const;
-  /// Returns true if \p F has hot function entry.
-  bool isFunctionEntryHot(const Function *F) const;
+  /// Returns true if \p F has hot function entry. If it returns false, it
+  /// either means it is not hot or it is unknown whether it is hot or not (for
+  /// example, no profile data is available).
+  template <typename FuncT> bool isFunctionEntryHot(const FuncT *F) const {
+    if (!F || !hasProfileSummary())
+      return false;
+    std::optional<Function::ProfileCount> FunctionCount = getEntryCount(F);
+    // FIXME: The heuristic used below for determining hotness is based on
+    // preliminary SPEC tuning for inliner. This will eventually be a
+    // convenience method that calls isHotCount.
+    return FunctionCount && isHotCount(FunctionCount->getCount());
+  }
+
   /// Returns true if \p F contains hot code.
-  bool isFunctionHotInCallGraph(const Function *F,
-                                BlockFrequencyInfo &BFI) const;
+  template <typename FuncT, typename BFIT>
+  bool isFunctionHotInCallGraph(const FuncT *F, BFIT &BFI) const {
+    if (!F || !hasProfileSummary())
+      return false;
+    if (auto FunctionCount = getEntryCount(F))
+      if (isHotCount(FunctionCount->getCount()))
+        return true;
+
+    if (auto TotalCallCount = getTotalCallCount(F)) {
+      if (isHotCount(*TotalCallCount))
+        return true;
+    }
+
+    for (const auto &BB : *F)
+      if (isHotBlock(&BB, &BFI))
+        return true;
+    return false;
+  }
   /// Returns true if \p F has cold function entry.
   bool isFunctionEntryCold(const Function *F) const;
   /// Returns true if \p F contains only cold code.
-  bool isFunctionColdInCallGraph(const Function *F,
-                                 BlockFrequencyInfo &BFI) const;
+  template <typename FuncT, typename BFIT>
+  bool isFunctionColdInCallGraph(const FuncT *F, BFIT &BFI) const {
+    if (!F || !hasProfileSummary())
+      return false;
+    if (auto FunctionCount = getEntryCount(F))
+      if (!isColdCount(FunctionCount->getCount()))
+        return false;
+
+    if (auto TotalCallCount = getTotalCallCount(F)) {
+      if (!isColdCount(*TotalCallCount))
+        return false;
+    }
+
+    for (const auto &BB : *F)
+      if (!isColdBlock(&BB, &BFI))
+        return false;
+    return true;
+  }
   /// Returns true if the hotness of \p F is unknown.
   bool isFunctionHotnessUnknown(const Function &F) const;
   /// Returns true if \p F contains hot code with regard to a given hot
   /// percentile cutoff value.
+  template <typename FuncT, typename BFIT>
   bool isFunctionHotInCallGraphNthPercentile(int PercentileCutoff,
-                                             const Function *F,
-                                             BlockFrequencyInfo &BFI) const;
+                                             const FuncT *F, BFIT &BFI) const {
+    return isFunctionHotOrColdInCallGraphNthPercentile<true, FuncT, BFIT>(
+        PercentileCutoff, F, BFI);
+  }
   /// Returns true if \p F contains cold code with regard to a given cold
   /// percentile cutoff value.
+  template <typename FuncT, typename BFIT>
   bool isFunctionColdInCallGraphNthPercentile(int PercentileCutoff,
-                                              const Function *F,
-                                              BlockFrequencyInfo &BFI) const;
+                                              const FuncT *F, BFIT &BFI) const {
+    return isFunctionHotOrColdInCallGraphNthPercentile<false, FuncT, BFIT>(
+        PercentileCutoff, F, BFI);
+  }
   /// Returns true if count \p C is considered hot.
   bool isHotCount(uint64_t C) const;
   /// Returns true if count \p C is considered cold.
   bool isColdCount(uint64_t C) const;
   /// Returns true if count \p C is considered hot with regard to a given
   /// hot percentile cutoff value.
   /// PercentileCutoff is encoded as a 6 digit decimal fixed point number, where
   /// the first two digits are the whole part. E.g. 995000 for 99.5 percentile.
   bool isHotCountNthPercentile(int PercentileCutoff, uint64_t C) const;
   /// Returns true if count \p C is considered cold with regard to a given
   /// cold percentile cutoff value.
   /// PercentileCutoff is encoded as a 6 digit decimal fixed point number, where
   /// the first two digits are the whole part. E.g. 995000 for 99.5 percentile.
   bool isColdCountNthPercentile(int PercentileCutoff, uint64_t C) const;
+
   /// Returns true if BasicBlock \p BB is considered hot.
-  bool isHotBlock(const BasicBlock *BB, BlockFrequencyInfo *BFI) const;
+  template <typename BBType, typename BFIT>
+  bool isHotBlock(const BBType *BB, BFIT *BFI) const {
+    auto Count = BFI->getBlockProfileCount(BB);
+    return Count && isHotCount(*Count);
+  }
+
   /// Returns true if BasicBlock \p BB is considered cold.
-  bool isColdBlock(const BasicBlock *BB, BlockFrequencyInfo *BFI) const;
-  /// Returns true if BasicBlock \p BB is considered hot with regard to a given
-  /// hot percentile cutoff value.
-  /// PercentileCutoff is encoded as a 6 digit decimal fixed point number, where
-  /// the first two digits are the whole part. E.g. 995000 for 99.5 percentile.
-  bool isHotBlockNthPercentile(int PercentileCutoff, const BasicBlock *BB,
-                               BlockFrequencyInfo *BFI) const;
+  template <typename BBType, typename BFIT>
+  bool isColdBlock(const BBType *BB, BFIT *BFI) const {
+    auto Count = BFI->getBlockProfileCount(BB);
+    return Count && isColdCount(*Count);
+  }
+
+  template <typename BFIT>
+  bool isColdBlock(BlockFrequency BlockFreq, const BFIT *BFI) const {
+    auto Count = BFI->getProfileCountFromFreq(BlockFreq.getFrequency());
+    return Count && isColdCount(*Count);
+  }
+
+  template <typename BBType, typename BFIT>
+  bool isHotBlockNthPercentile(int PercentileCutoff, const BBType *BB,
+                               BFIT *BFI) const {
+    return isHotOrColdBlockNthPercentile<true, BBType, BFIT>(PercentileCutoff,
+                                                             BB, BFI);
+  }
+
+  template <typename BFIT>
+  bool isHotBlockNthPercentile(int PercentileCutoff, BlockFrequency BlockFreq,
+                               BFIT *BFI) const {
+    return isHotOrColdBlockNthPercentile<true, BFIT>(PercentileCutoff,
+                                                     BlockFreq, BFI);
+  }
+
   /// Returns true if BasicBlock \p BB is considered cold with regard to a given
   /// cold percentile cutoff value.
   /// PercentileCutoff is encoded as a 6 digit decimal fixed point number, where
   /// the first two digits are the whole part. E.g. 995000 for 99.5 percentile.
-  bool isColdBlockNthPercentile(int PercentileCutoff, const BasicBlock *BB,
-                                BlockFrequencyInfo *BFI) const;
+  template <typename BBType, typename BFIT>
+  bool isColdBlockNthPercentile(int PercentileCutoff, const BBType *BB,
+                                BFIT *BFI) const {
+    return isHotOrColdBlockNthPercentile<false, BBType, BFIT>(PercentileCutoff,
+                                                              BB, BFI);
+  }
+  template <typename BFIT>
+  bool isColdBlockNthPercentile(int PercentileCutoff, BlockFrequency BlockFreq,
+                                BFIT *BFI) const {
+    return isHotOrColdBlockNthPercentile<false, BFIT>(PercentileCutoff,
+                                                      BlockFreq, BFI);
+  }
   /// Returns true if the call site \p CB is considered hot.
   bool isHotCallSite(const CallBase &CB, BlockFrequencyInfo *BFI) const;
   /// Returns true if call site \p CB is considered cold.
   bool isColdCallSite(const CallBase &CB, BlockFrequencyInfo *BFI) const;
   /// Returns HotCountThreshold if set. Recompute HotCountThreshold
   /// if not set.
   uint64_t getOrCompHotCountThreshold() const;
   /// Returns ColdCountThreshold if set. Recompute HotCountThreshold
   /// if not set.
   uint64_t getOrCompColdCountThreshold() const;
   /// Returns HotCountThreshold if set.
   uint64_t getHotCountThreshold() const {
     return HotCountThreshold.value_or(0);
   }
   /// Returns ColdCountThreshold if set.
   uint64_t getColdCountThreshold() const {
     return ColdCountThreshold.value_or(0);
   }
 
- private:
-   template <bool isHot>
-   bool isFunctionHotOrColdInCallGraphNthPercentile(
-       int PercentileCutoff, const Function *F, BlockFrequencyInfo &BFI) const;
-   template <bool isHot>
-   bool isHotOrColdCountNthPercentile(int PercentileCutoff, uint64_t C) const;
-   template <bool isHot>
-   bool isHotOrColdBlockNthPercentile(int PercentileCutoff,
-                                      const BasicBlock *BB,
-                                      BlockFrequencyInfo *BFI) const;
+  template <typename FuncT>
+  std::optional<uint64_t> getTotalCallCount(const FuncT *F) const {
+    return std::nullopt;
+  }
+
+  template <>
+  std::optional<uint64_t> getTotalCallCount<Function>(const Function *F) const {
+    if (!hasSampleProfile())
+      return std::nullopt;
+    uint64_t TotalCallCount = 0;
+    for (const auto &BB : *F) {
+      for (const auto &I : BB) {
+        if (isa<CallInst>(I) || isa<InvokeInst>(I)) {
+          if (auto CallCount = getProfileCount(cast<CallBase>(I), nullptr)) {
+            TotalCallCount += *CallCount;
+          }
+        }
+      }
+    }
+    return TotalCallCount;
+  }
+
+  template <bool isHot, typename FuncT, typename BFIT>
+  bool isFunctionHotOrColdInCallGraphNthPercentile(int PercentileCutoff,
+                                                   const FuncT *F,
+                                                   BFIT &FI) const {
+    if (!F || !hasProfileSummary())
+      return false;
+    if (auto FunctionCount = getEntryCount(F)) {
+      if (isHot &&
+          isHotCountNthPercentile(PercentileCutoff, FunctionCount->getCount()))
+        return true;
+      if (!isHot && !isColdCountNthPercentile(PercentileCutoff,
+                                              FunctionCount->getCount()))
+        return false;
+    }
+    if (auto TotalCallCount = getTotalCallCount(F)) {
+      if (isHot && isHotCountNthPercentile(PercentileCutoff, *TotalCallCount))
+        return true;
+      if (!isHot &&
+          !isColdCountNthPercentile(PercentileCutoff, *TotalCallCount))
+        return false;
+    }
+    for (const auto &BB : *F) {
+      if (isHot && isHotBlockNthPercentile(PercentileCutoff, &BB, &FI))
+        return true;
+      if (!isHot && !isColdBlockNthPercentile(PercentileCutoff, &BB, &FI))
+        return false;
+    }
+    return !isHot;
+  }
+
+
+  template <bool isHot, typename BBType, typename BFIT>
+  bool isHotOrColdBlockNthPercentile(int PercentileCutoff, const BBType *BB,
+                                     BFIT *BFI) const {
+    auto Count = BFI->getBlockProfileCount(BB);
+    if (isHot)
+      return Count && isHotCountNthPercentile(PercentileCutoff, *Count);
+    else
+      return Count && isColdCountNthPercentile(PercentileCutoff, *Count);
+  }
+
+  template <bool isHot, typename BFIT>
+                                     BlockFrequency BlockFreq,
+                                     BFIT *BFI) const {
+    auto Count = BFI->getProfileCountFromFreq(BlockFreq.getFrequency());
+    if (isHot)
+      return Count && isHotCountNthPercentile(PercentileCutoff, *Count);
+    else
+      return Count && isColdCountNthPercentile(PercentileCutoff, *Count);
+  }
+
+  template <typename FuncT>
+  std::optional<Function::ProfileCount> getEntryCount(const FuncT *F) const {
+    return F->getEntryCount();
+  }
+
+  template <>
+  std::optional<Function::ProfileCount>
+  getEntryCount<MachineFunction>(const MachineFunction *F) const {
+    return F->getFunction().getEntryCount();
+  }
 };
 
 /// An analysis pass based on legacy pass manager to deliver ProfileSummaryInfo.
 class ProfileSummaryInfoWrapperPass : public ImmutablePass {
   std::unique_ptr<ProfileSummaryInfo> PSI;
 
 public:
   static char ID;

llvm/include/llvm/Transforms/Utils/SizeOpts.h

return PGSOColdCodeOnly ||
          (PSI->hasInstrumentationProfile() && PGSOColdCodeOnlyForInstrPGO) ||
          (PSI->hasSampleProfile() &&
           ((!PSI->hasPartialSampleProfile() && PGSOColdCodeOnlyForSamplePGO) ||
            (PSI->hasPartialSampleProfile() &&
             PGSOColdCodeOnlyForPartialSamplePGO))) ||
          (PGSOLargeWorkingSetSizeOnly && !PSI->hasLargeWorkingSetSize());
 }
 
-template<typename AdapterT, typename FuncT, typename BFIT>
+template <typename FuncT, typename BFIT>
 bool shouldFuncOptimizeForSizeImpl(const FuncT *F, ProfileSummaryInfo *PSI,
                                    BFIT *BFI, PGSOQueryType QueryType) {
   assert(F);
   if (!PSI || !BFI || !PSI->hasProfileSummary())
     return false;
   if (ForcePGSO)
     return true;
   if (!EnablePGSO)
     return false;
   if (isPGSOColdCodeOnly(PSI))
-    return AdapterT::isFunctionColdInCallGraph(F, PSI, *BFI);
+    return PSI->isFunctionColdInCallGraph(F, *BFI);
   if (PSI->hasSampleProfile())
     // The "isCold" check seems to work better for Sample PGO as it could have
     // many profile-unannotated functions.
-    return AdapterT::isFunctionColdInCallGraphNthPercentile(
-        PgsoCutoffSampleProf, F, PSI, *BFI);
-  return !AdapterT::isFunctionHotInCallGraphNthPercentile(PgsoCutoffInstrProf,
-                                                          F, PSI, *BFI);
+    return PSI->isFunctionColdInCallGraphNthPercentile(PgsoCutoffSampleProf, F,
+                                                       *BFI);
+  return !PSI->isFunctionHotInCallGraphNthPercentile(PgsoCutoffInstrProf, F,
+                                                     *BFI);
 }
 
-template<typename AdapterT, typename BlockTOrBlockFreq, typename BFIT>
-bool shouldOptimizeForSizeImpl(BlockTOrBlockFreq BBOrBlockFreq, ProfileSummaryInfo *PSI,
-                               BFIT *BFI, PGSOQueryType QueryType) {
+template <typename BlockTOrBlockFreq, typename BFIT>
+bool shouldOptimizeForSizeImpl(BlockTOrBlockFreq BBOrBlockFreq,
+                               ProfileSummaryInfo *PSI, BFIT *BFI,
+                               PGSOQueryType QueryType) {
   if (!PSI || !BFI || !PSI->hasProfileSummary())
     return false;
   if (ForcePGSO)
     return true;
   if (!EnablePGSO)
     return false;
   if (isPGSOColdCodeOnly(PSI))
-    return AdapterT::isColdBlock(BBOrBlockFreq, PSI, BFI);
+    return PSI->isColdBlock(BBOrBlockFreq, BFI);
   if (PSI->hasSampleProfile())
     // The "isCold" check seems to work better for Sample PGO as it could have
     // many profile-unannotated functions.
-    return AdapterT::isColdBlockNthPercentile(PgsoCutoffSampleProf,
-                                              BBOrBlockFreq, PSI, BFI);
-  return !AdapterT::isHotBlockNthPercentile(PgsoCutoffInstrProf, BBOrBlockFreq,
-                                            PSI, BFI);
+    return PSI->isColdBlockNthPercentile(PgsoCutoffSampleProf, BBOrBlockFreq,
+                                         BFI);
+  return !PSI->isHotBlockNthPercentile(PgsoCutoffInstrProf, BBOrBlockFreq, BFI);
 }
 
 /// Returns true if function \p F is suggested to be size-optimized based on the
 /// profile.
 bool shouldOptimizeForSize(const Function *F, ProfileSummaryInfo *PSI,
                            BlockFrequencyInfo *BFI,
                            PGSOQueryType QueryType = PGSOQueryType::Other);
 

llvm/lib/Analysis/ProfileSummaryInfo.cpp

if (Call.extractProfTotalWeight(TotalCount))
       return TotalCount;
     return std::nullopt;
   }
   if (BFI)
     return BFI->getBlockProfileCount(Call.getParent(), AllowSynthetic);
   return std::nullopt;
 }
 
-/// Returns true if the function's entry is hot. If it returns false, it
-/// either means it is not hot or it is unknown whether it is hot or not (for
-/// example, no profile data is available).
-bool ProfileSummaryInfo::isFunctionEntryHot(const Function *F) const {
-  if (!F || !hasProfileSummary())
-    return false;
-  auto FunctionCount = F->getEntryCount();
-  // FIXME: The heuristic used below for determining hotness is based on
-  // preliminary SPEC tuning for inliner. This will eventually be a
-  // convenience method that calls isHotCount.
-  return FunctionCount && isHotCount(FunctionCount->getCount());
-}
-
-/// Returns true if the function contains hot code. This can include a hot
-/// function entry count, hot basic block, or (in the case of Sample PGO)
-/// hot total call edge count.
-/// If it returns false, it either means it is not hot or it is unknown
-/// (for example, no profile data is available).
-bool ProfileSummaryInfo::isFunctionHotInCallGraph(
-    const Function *F, BlockFrequencyInfo &BFI) const {
-  if (!F || !hasProfileSummary())
-    return false;
-  if (auto FunctionCount = F->getEntryCount())
-    if (isHotCount(FunctionCount->getCount()))
-      return true;
-
-  if (hasSampleProfile()) {
-    uint64_t TotalCallCount = 0;
-    for (const auto &BB : *F)
-      for (const auto &I : BB)
-        if (isa<CallInst>(I) || isa<InvokeInst>(I))
-          if (auto CallCount = getProfileCount(cast<CallBase>(I), nullptr))
-            TotalCallCount += *CallCount;
-    if (isHotCount(TotalCallCount))
-      return true;
-  }
-  for (const auto &BB : *F)
-    if (isHotBlock(&BB, &BFI))
-      return true;
-  return false;
-}
-
-/// Returns true if the function only contains cold code. This means that
-/// the function entry and blocks are all cold, and (in the case of Sample PGO)
-/// the total call edge count is cold.
-/// If it returns false, it either means it is not cold or it is unknown
-/// (for example, no profile data is available).
-bool ProfileSummaryInfo::isFunctionColdInCallGraph(
-    const Function *F, BlockFrequencyInfo &BFI) const {
-  if (!F || !hasProfileSummary())
-    return false;
-  if (auto FunctionCount = F->getEntryCount())
-    if (!isColdCount(FunctionCount->getCount()))
-      return false;
-
-  if (hasSampleProfile()) {
-    uint64_t TotalCallCount = 0;
-    for (const auto &BB : *F)
-      for (const auto &I : BB)
-        if (isa<CallInst>(I) || isa<InvokeInst>(I))
-          if (auto CallCount = getProfileCount(cast<CallBase>(I), nullptr))
-            TotalCallCount += *CallCount;
-    if (!isColdCount(TotalCallCount))
-      return false;
-  }
-  for (const auto &BB : *F)
-    if (!isColdBlock(&BB, &BFI))
-      return false;
-  return true;
-}
-
 bool ProfileSummaryInfo::isFunctionHotnessUnknown(const Function &F) const {
   assert(hasPartialSampleProfile() && "Expect partial sample profile");
   return !F.getEntryCount();
 }
 
-template <bool isHot>
-bool ProfileSummaryInfo::isFunctionHotOrColdInCallGraphNthPercentile(
-    int PercentileCutoff, const Function *F, BlockFrequencyInfo &BFI) const {
-  if (!F || !hasProfileSummary())
-    return false;
-  if (auto FunctionCount = F->getEntryCount()) {
-    if (isHot &&
-        isHotCountNthPercentile(PercentileCutoff, FunctionCount->getCount()))
-      return true;
-    if (!isHot &&
-        !isColdCountNthPercentile(PercentileCutoff, FunctionCount->getCount()))
-      return false;
-  }
-  if (hasSampleProfile()) {
-    uint64_t TotalCallCount = 0;
-    for (const auto &BB : *F)
-      for (const auto &I : BB)
-        if (isa<CallInst>(I) || isa<InvokeInst>(I))
-          if (auto CallCount = getProfileCount(cast<CallBase>(I), nullptr))
-            TotalCallCount += *CallCount;
-    if (isHot && isHotCountNthPercentile(PercentileCutoff, TotalCallCount))
-      return true;
-    if (!isHot && !isColdCountNthPercentile(PercentileCutoff, TotalCallCount))
-      return false;
-  }
-  for (const auto &BB : *F) {
-    if (isHot && isHotBlockNthPercentile(PercentileCutoff, &BB, &BFI))
-      return true;
-    if (!isHot && !isColdBlockNthPercentile(PercentileCutoff, &BB, &BFI))
-      return false;
-  }
-  return !isHot;
-}
-
-// Like isFunctionHotInCallGraph but for a given cutoff.
-bool ProfileSummaryInfo::isFunctionHotInCallGraphNthPercentile(
-    int PercentileCutoff, const Function *F, BlockFrequencyInfo &BFI) const {
-  return isFunctionHotOrColdInCallGraphNthPercentile<true>(
-      PercentileCutoff, F, BFI);
-}
-
-bool ProfileSummaryInfo::isFunctionColdInCallGraphNthPercentile(
-    int PercentileCutoff, const Function *F, BlockFrequencyInfo &BFI) const {
-  return isFunctionHotOrColdInCallGraphNthPercentile<false>(
-      PercentileCutoff, F, BFI);
-}
-
 /// Returns true if the function's entry is a cold. If it returns false, it
 /// either means it is not cold or it is unknown whether it is cold or not (for
 /// example, no profile data is available).
 bool ProfileSummaryInfo::isFunctionEntryCold(const Function *F) const {
   if (!F)
     return false;
   if (F->hasFnAttribute(Attribute::Cold))
     return true;

 uint64_t ProfileSummaryInfo::getOrCompHotCountThreshold() const {
   return HotCountThreshold.value_or(UINT64_MAX);
 }
 
 uint64_t ProfileSummaryInfo::getOrCompColdCountThreshold() const {
   return ColdCountThreshold.value_or(0);
 }
 
-bool ProfileSummaryInfo::isHotBlock(const BasicBlock *BB,
-                                    BlockFrequencyInfo *BFI) const {
-  auto Count = BFI->getBlockProfileCount(BB);
-  return Count && isHotCount(*Count);
-}
-
-bool ProfileSummaryInfo::isColdBlock(const BasicBlock *BB,
-                                     BlockFrequencyInfo *BFI) const {
-  auto Count = BFI->getBlockProfileCount(BB);
-  return Count && isColdCount(*Count);
-}
-
-template <bool isHot>
-bool ProfileSummaryInfo::isHotOrColdBlockNthPercentile(
-    int PercentileCutoff, const BasicBlock *BB, BlockFrequencyInfo *BFI) const {
-  auto Count = BFI->getBlockProfileCount(BB);
-  if (isHot)
-    return Count && isHotCountNthPercentile(PercentileCutoff, *Count);
-  else
-    return Count && isColdCountNthPercentile(PercentileCutoff, *Count);
-}
-
-bool ProfileSummaryInfo::isHotBlockNthPercentile(
-    int PercentileCutoff, const BasicBlock *BB, BlockFrequencyInfo *BFI) const {
-  return isHotOrColdBlockNthPercentile<true>(PercentileCutoff, BB, BFI);
-}
-
-bool ProfileSummaryInfo::isColdBlockNthPercentile(
-    int PercentileCutoff, const BasicBlock *BB, BlockFrequencyInfo *BFI) const {
-  return isHotOrColdBlockNthPercentile<false>(PercentileCutoff, BB, BFI);
-}
-
 bool ProfileSummaryInfo::isHotCallSite(const CallBase &CB,
                                        BlockFrequencyInfo *BFI) const {
   auto C = getProfileCount(CB, BFI);
   return C && isHotCount(*C);
 }
 
 bool ProfileSummaryInfo::isColdCallSite(const CallBase &CB,
                                         BlockFrequencyInfo *BFI) const {

llvm/lib/CodeGen/MachineSizeOpts.cpp

 
 using namespace llvm;
 
 extern cl::opt<bool> EnablePGSO;
 extern cl::opt<bool> PGSOLargeWorkingSetSizeOnly;
 extern cl::opt<bool> ForcePGSO;
 extern cl::opt<int> PgsoCutoffInstrProf;
 extern cl::opt<int> PgsoCutoffSampleProf;
 
-namespace {
-namespace machine_size_opts_detail {
-
-/// Like ProfileSummaryInfo::isColdBlock but for MachineBasicBlock.
-bool isColdBlock(const MachineBasicBlock *MBB,
-                 ProfileSummaryInfo *PSI,
-                 const MachineBlockFrequencyInfo *MBFI) {
-  auto Count = MBFI->getBlockProfileCount(MBB);
-  return Count && PSI->isColdCount(*Count);
-}
-
-bool isColdBlock(BlockFrequency BlockFreq,
-                 ProfileSummaryInfo *PSI,
-                 const MachineBlockFrequencyInfo *MBFI) {
-  auto Count = MBFI->getProfileCountFromFreq(BlockFreq.getFrequency());
-  return Count && PSI->isColdCount(*Count);
-}
-
-/// Like ProfileSummaryInfo::isHotBlockNthPercentile but for MachineBasicBlock.
-static bool isHotBlockNthPercentile(int PercentileCutoff,
-                                    const MachineBasicBlock *MBB,
-                                    ProfileSummaryInfo *PSI,
-                                    const MachineBlockFrequencyInfo *MBFI) {
-  auto Count = MBFI->getBlockProfileCount(MBB);
-  return Count && PSI->isHotCountNthPercentile(PercentileCutoff, *Count);
-}
-
-static bool isHotBlockNthPercentile(int PercentileCutoff,
-                                    BlockFrequency BlockFreq,
-                                    ProfileSummaryInfo *PSI,
-                                    const MachineBlockFrequencyInfo *MBFI) {
-  auto Count = MBFI->getProfileCountFromFreq(BlockFreq.getFrequency());
-  return Count && PSI->isHotCountNthPercentile(PercentileCutoff, *Count);
-}
-
-static bool isColdBlockNthPercentile(int PercentileCutoff,
-                                     const MachineBasicBlock *MBB,
-                                     ProfileSummaryInfo *PSI,
-                                     const MachineBlockFrequencyInfo *MBFI) {
-  auto Count = MBFI->getBlockProfileCount(MBB);
-  return Count && PSI->isColdCountNthPercentile(PercentileCutoff, *Count);
-}
-
-static bool isColdBlockNthPercentile(int PercentileCutoff,
-                                     BlockFrequency BlockFreq,
-                                     ProfileSummaryInfo *PSI,
-                                     const MachineBlockFrequencyInfo *MBFI) {
-  auto Count = MBFI->getProfileCountFromFreq(BlockFreq.getFrequency());
-  return Count && PSI->isColdCountNthPercentile(PercentileCutoff, *Count);
-}
-
-/// Like ProfileSummaryInfo::isFunctionColdInCallGraph but for
-/// MachineFunction.
-bool isFunctionColdInCallGraph(
-    const MachineFunction *MF,
-    ProfileSummaryInfo *PSI,
-    const MachineBlockFrequencyInfo &MBFI) {
-  if (auto FunctionCount = MF->getFunction().getEntryCount())
-    if (!PSI->isColdCount(FunctionCount->getCount()))
-      return false;
-  for (const auto &MBB : *MF)
-    if (!isColdBlock(&MBB, PSI, &MBFI))
-      return false;
-  return true;
-}
-
-/// Like ProfileSummaryInfo::isFunctionHotInCallGraphNthPercentile but for
-/// MachineFunction.
-bool isFunctionHotInCallGraphNthPercentile(
-    int PercentileCutoff,
-    const MachineFunction *MF,
-    ProfileSummaryInfo *PSI,
-    const MachineBlockFrequencyInfo &MBFI) {
-  if (auto FunctionCount = MF->getFunction().getEntryCount())
-    if (PSI->isHotCountNthPercentile(PercentileCutoff,
-                                     FunctionCount->getCount()))
-      return true;
-  for (const auto &MBB : *MF)
-    if (isHotBlockNthPercentile(PercentileCutoff, &MBB, PSI, &MBFI))
-      return true;
-  return false;
-}
-
-bool isFunctionColdInCallGraphNthPercentile(
-    int PercentileCutoff, const MachineFunction *MF, ProfileSummaryInfo *PSI,
-    const MachineBlockFrequencyInfo &MBFI) {
-  if (auto FunctionCount = MF->getFunction().getEntryCount())
-    if (!PSI->isColdCountNthPercentile(PercentileCutoff,
-                                       FunctionCount->getCount()))
-      return false;
-  for (const auto &MBB : *MF)
-    if (!isColdBlockNthPercentile(PercentileCutoff, &MBB, PSI, &MBFI))
-      return false;
-  return true;
-}
-} // namespace machine_size_opts_detail
-
-struct MachineBasicBlockBFIAdapter {
-  static bool isFunctionColdInCallGraph(const MachineFunction *MF,
-                                        ProfileSummaryInfo *PSI,
-                                        const MachineBlockFrequencyInfo &MBFI) {
-    return machine_size_opts_detail::isFunctionColdInCallGraph(MF, PSI, MBFI);
-  }
-  static bool isFunctionHotInCallGraphNthPercentile(
-      int CutOff,
-      const MachineFunction *MF,
-      ProfileSummaryInfo *PSI,
-      const MachineBlockFrequencyInfo &MBFI) {
-    return machine_size_opts_detail::isFunctionHotInCallGraphNthPercentile(
-        CutOff, MF, PSI, MBFI);
-  }
-  static bool isFunctionColdInCallGraphNthPercentile(
-      int CutOff, const MachineFunction *MF, ProfileSummaryInfo *PSI,
-      const MachineBlockFrequencyInfo &MBFI) {
-    return machine_size_opts_detail::isFunctionColdInCallGraphNthPercentile(
-        CutOff, MF, PSI, MBFI);
-  }
-  static bool isColdBlock(const MachineBasicBlock *MBB,
-                          ProfileSummaryInfo *PSI,
-                          const MachineBlockFrequencyInfo *MBFI) {
-    return machine_size_opts_detail::isColdBlock(MBB, PSI, MBFI);
-  }
-  static bool isColdBlock(BlockFrequency BlockFreq,
-                          ProfileSummaryInfo *PSI,
-                          const MachineBlockFrequencyInfo *MBFI) {
-    return machine_size_opts_detail::isColdBlock(BlockFreq, PSI, MBFI);
-  }
-  static bool isHotBlockNthPercentile(int CutOff,
-                                      const MachineBasicBlock *MBB,
-                                      ProfileSummaryInfo *PSI,
-                                      const MachineBlockFrequencyInfo *MBFI) {
-    return machine_size_opts_detail::isHotBlockNthPercentile(
-        CutOff, MBB, PSI, MBFI);
-  }
-  static bool isHotBlockNthPercentile(int CutOff,
-                                      BlockFrequency BlockFreq,
-                                      ProfileSummaryInfo *PSI,
-                                      const MachineBlockFrequencyInfo *MBFI) {
-    return machine_size_opts_detail::isHotBlockNthPercentile(
-        CutOff, BlockFreq, PSI, MBFI);
-  }
-  static bool isColdBlockNthPercentile(int CutOff, const MachineBasicBlock *MBB,
-                                       ProfileSummaryInfo *PSI,
-                                       const MachineBlockFrequencyInfo *MBFI) {
-    return machine_size_opts_detail::isColdBlockNthPercentile(CutOff, MBB, PSI,
-                                                              MBFI);
-  }
-  static bool isColdBlockNthPercentile(int CutOff, BlockFrequency BlockFreq,
-                                       ProfileSummaryInfo *PSI,
-                                       const MachineBlockFrequencyInfo *MBFI) {
-    return machine_size_opts_detail::isColdBlockNthPercentile(CutOff, BlockFreq,
-                                                              PSI, MBFI);
-  }
-};
-} // end anonymous namespace
-
 bool llvm::shouldOptimizeForSize(const MachineFunction *MF,
                                  ProfileSummaryInfo *PSI,
                                  const MachineBlockFrequencyInfo *MBFI,
                                  PGSOQueryType QueryType) {
-  return shouldFuncOptimizeForSizeImpl<MachineBasicBlockBFIAdapter>(
-      MF, PSI, MBFI, QueryType);
+  return shouldFuncOptimizeForSizeImpl(MF, PSI, MBFI, QueryType);
 }
 
 bool llvm::shouldOptimizeForSize(const MachineBasicBlock *MBB,
                                  ProfileSummaryInfo *PSI,
                                  const MachineBlockFrequencyInfo *MBFI,
                                  PGSOQueryType QueryType) {
   assert(MBB);
-  return shouldOptimizeForSizeImpl<MachineBasicBlockBFIAdapter>(
-      MBB, PSI, MBFI, QueryType);
+  return shouldOptimizeForSizeImpl(MBB, PSI, MBFI, QueryType);
 }
 
 bool llvm::shouldOptimizeForSize(const MachineBasicBlock *MBB,
                                  ProfileSummaryInfo *PSI,
                                  MBFIWrapper *MBFIW,
                                  PGSOQueryType QueryType) {
   assert(MBB);
   if (!PSI || !MBFIW)
     return false;
   BlockFrequency BlockFreq = MBFIW->getBlockFreq(MBB);
-  return shouldOptimizeForSizeImpl<MachineBasicBlockBFIAdapter>(
-      BlockFreq, PSI, &MBFIW->getMBFI(), QueryType);
+  return shouldOptimizeForSizeImpl(BlockFreq, PSI, &MBFIW->getMBFI(),
+                                   QueryType);
 }

llvm/lib/Transforms/Utils/SizeOpts.cpp

static bool isColdBlockNthPercentile(int CutOff, const BasicBlock *BB,
     return PSI->isColdBlockNthPercentile(CutOff, BB, BFI);
   }
 };
 } // end anonymous namespace
 
 bool llvm::shouldOptimizeForSize(const Function *F, ProfileSummaryInfo *PSI,
                                  BlockFrequencyInfo *BFI,
                                  PGSOQueryType QueryType) {
-  return shouldFuncOptimizeForSizeImpl<BasicBlockBFIAdapter>(F, PSI, BFI,
-                                                             QueryType);
+  return shouldFuncOptimizeForSizeImpl(F, PSI, BFI, QueryType);
 }
 
 bool llvm::shouldOptimizeForSize(const BasicBlock *BB, ProfileSummaryInfo *PSI,
                                  BlockFrequencyInfo *BFI,
                                  PGSOQueryType QueryType) {
   assert(BB);
-  return shouldOptimizeForSizeImpl<BasicBlockBFIAdapter>(BB, PSI, BFI,
-                                                         QueryType);
+  return shouldOptimizeForSizeImpl(BB, PSI, BFI, QueryType);
 }