[llvm-profdata] Extend support of --topn to sample profiles

[wenlei]

hot-func-list of sample pgo is the equivalent of topn for instr pgo. Sample pgo diverged from instr pgo here in the sense we use hot-func-list to print a hot function list where "hot" is defined by the dyn-instr percentile based threshold. So simply adding topn might be confusing, as using topn alone for sample pgo won't print the hot function list, unlike instr pgo.

if you want to grow or trim the hot function list, you could use the threshold cutoff switches. Ideally we should unify the switches for both instr pgo and sample pgo.

llvm/tools/llvm-profdata/llvm-profdata.cpp

2352–2353

nit: TopN

[modimo]

In D110449#3021837, @wenlei wrote:

hot-func-list of sample pgo is the equivalent of topn for instr pgo. Sample pgo diverged from instr pgo here in the sense we use hot-func-list to print a hot function list where "hot" is defined by the dyn-instr percentile based threshold. So simply adding topn might be confusing, as using topn alone for sample pgo won't print the hot function list, unlike instr pgo.

if you want to grow or trim the hot function list, you could use the threshold cutoff switches. Ideally we should unify the switches for both instr pgo and sample pgo.

My usage of topn was to be able to compare and analyze a fixed number of functions rather than defining different cut-off values to get similar effects across many different binaries. That being said, this was all running in a script so I could've also had the script do this pruning instead of llvm-profdata.

Unifying switches was the main reason I added this to prof-data, since topn being documented but being a complete no-op in sample profile threw me off.

[wenlei]

In D110449#3021864, @modimo wrote:

In D110449#3021837, @wenlei wrote:

hot-func-list of sample pgo is the equivalent of topn for instr pgo. Sample pgo diverged from instr pgo here in the sense we use hot-func-list to print a hot function list where "hot" is defined by the dyn-instr percentile based threshold. So simply adding topn might be confusing, as using topn alone for sample pgo won't print the hot function list, unlike instr pgo.

if you want to grow or trim the hot function list, you could use the threshold cutoff switches. Ideally we should unify the switches for both instr pgo and sample pgo.

My usage of topn was to be able to compare and analyze a fixed number of functions rather than defining different cut-off values to get similar effects across many different binaries. That being said, this was all running in a script so I could've also had the script do this pruning instead of llvm-profdata.

Unifying switches was the main reason I added this to prof-data, since topn being documented but being a complete no-op in sample profile threw me off.

topn alone would still not do anything with your change. perhaps when topn is on, the hot func list should be printed. llvm-profdata has many switches that are only applicable to either instr. pgo or sample pgo (showcs for ipgo, show-prof-sym-list for spgo, etc..)

llvm/tools/llvm-profdata/llvm-profdata.cpp

2453–2454

ShowHotFuncList || TopN would make it more consistent between instr pgo and sample pgo.

[modimo]

Have topn also output the top function list.

[wenlei]

lgtm with a nit, thanks.

llvm/test/tools/llvm-profdata/sample-topn.test

1 ↗

(On Diff #374989)

I think it's better to merge the tests into sample-hot-func-list.test. This is improving hot func list and the tests also use the same inputs.

[modimo]

Merge tests back into sample-hot-func-list.test

[modimo]

Newline

llvm/test/tools/llvm-profdata/sample-hot-func-list.test

 ; RUN: llvm-profdata show --sample --hot-func-list %S/Inputs/sample-hot-func-list.proftext | FileCheck %s --match-full-lines --strict-whitespace
 
 ;      CHECK:8 out of 10 functions with profile (80.00%) are considered hot functions (max sample >= 470).
 ; CHECK-NEXT:355251 out of 356026 profile counts (99.78%) are from hot functions.
 ; CHECK-NEXT: Total sample (%)       Max sample        Entry sample    Function name
 ; CHECK-NEXT: 184019 (51.69%)        2300              534             main
 ; CHECK-NEXT: 97401 (27.36%)         10640             3035            Func3
 ; CHECK-NEXT: 20305 (5.70%)          1000              1000            _Z3bazi
 ; CHECK-NEXT: 20301 (5.70%)          1437              1437            _Z3bari
 ; CHECK-NEXT: 17043 (4.79%)          3105              1594            Func2
 ; CHECK-NEXT: 7711 (2.17%)           610               610             _Z3fooi
 ; CHECK-NEXT: 6948 (1.95%)           3507              470             Func5
 ; CHECK-NEXT: 1523 (0.43%)           563               169             Func1
 
-
 ; RUN: llvm-profdata show --sample --hot-func-list %S/Inputs/cs-sample.proftext | FileCheck %s --match-full-lines --strict-whitespace --check-prefix=CS
 
 ;      CS:2 out of 8 functions with profile (25.00%) are considered hot functions (max sample >= 23324).
 ; CS-NEXT:1968152 out of 1968919 profile counts (99.96%) are from hot functions.
 ; CS-NEXT: Total sample (%)       Max sample        Entry sample    Function name
 ; CS-NEXT: 1467299 (74.52%)       287884            11              main:3 @ _Z5funcAi:1 @ _Z8funcLeafi
 ; CS-NEXT: 500853 (25.44%)        74946             20              main:3.1 @ _Z5funcBi:1 @ _Z8funcLeafi
+
+
+; RUN: llvm-profdata show --sample --topn=2 %S/Inputs/sample-hot-func-list.proftext | FileCheck %s --match-full-lines --strict-whitespace --check-prefix=TOPN
+
+;      TOPN:8 out of 10 functions with profile (80.00%) are considered hot functions (max sample >= 470).
+; TOPN-NEXT:355251 out of 356026 profile counts (99.78%) are from hot functions.
+; TOPN-NEXT: Total sample (%)       Max sample        Entry sample    Function name
+; TOPN-NEXT: 184019 (51.69%)        2300              534             main
+; TOPN-NEXT: 97401 (27.36%)         10640             3035            Func3
+
+; RUN: llvm-profdata show --sample --topn=1 %S/Inputs/cs-sample.proftext | FileCheck %s --match-full-lines --strict-whitespace --check-prefix=CS-TOPN
+
+;      CS-TOPN:2 out of 8 functions with profile (25.00%) are considered hot functions (max sample >= 23324).
+; CS-TOPN-NEXT:1968152 out of 1968919 profile counts (99.96%) are from hot functions.
+; CS-TOPN-NEXT: Total sample (%)       Max sample        Entry sample    Function name
+; CS-TOPN-NEXT: 1467299 (74.52%)       287884            11              main:3 @ _Z5funcAi:1 @ _Z8funcLeafi

llvm/tools/llvm-profdata/llvm-profdata.cpp

 // and at least 4. Besides, users can optionally give a HotFuncMetric string to
 // print out or let it be an empty string.
 static void dumpHotFunctionList(const std::vector<std::string> &ColumnTitle,
                                 const std::vector<int> &ColumnOffset,
                                 const std::vector<HotFuncInfo> &PrintValues,
                                 uint64_t HotFuncCount, uint64_t TotalFuncCount,
                                 uint64_t HotProfCount, uint64_t TotalProfCount,
                                 const std::string &HotFuncMetric,
-                                raw_fd_ostream &OS) {
+                                uint32_t TopNFunctions, raw_fd_ostream &OS) {
   assert(ColumnOffset.size() == ColumnTitle.size() &&
          "ColumnOffset and ColumnTitle should have the same size");
   assert(ColumnTitle.size() >= 4 &&
          "ColumnTitle should have at least 4 elements");
   assert(TotalFuncCount > 0 &&
          "There should be at least one function in the profile");
   double TotalProfPercent = 0;
   if (TotalProfCount > 0)
FOS << HotProfCount << " out of " << TotalProfCount << " profile counts ("
       << format("%.2f%%", TotalProfPercent) << ") are from hot functions.\n";
 
   for (size_t I = 0; I < ColumnTitle.size(); ++I) {
     FOS.PadToColumn(ColumnOffset[I]);
     FOS << ColumnTitle[I];
   }
   FOS << "\n";
 
-  for (const HotFuncInfo &R : PrintValues) {
+  uint32_t Count = 0;
+  for (const auto &R : PrintValues) {
+    if (TopNFunctions && (Count++ == TopNFunctions))
+      break;
     FOS.PadToColumn(ColumnOffset[0]);
     FOS << R.TotalCount << " (" << format("%.2f%%", R.TotalCountPercent) << ")";
     FOS.PadToColumn(ColumnOffset[1]);
     FOS << R.MaxCount;
     FOS.PadToColumn(ColumnOffset[2]);
     FOS << R.EntryCount;
     FOS.PadToColumn(ColumnOffset[3]);
     FOS << R.FuncName << "\n";
   }
 }
 
 static int showHotFunctionList(const sampleprof::SampleProfileMap &Profiles,
-                               ProfileSummary &PS, raw_fd_ostream &OS) {
+                               ProfileSummary &PS, uint32_t TopN,
+                               raw_fd_ostream &OS) {
   using namespace sampleprof;
 
   const uint32_t HotFuncCutoff = 990000;
   auto &SummaryVector = PS.getDetailedSummary();
   uint64_t MinCountThreshold = 0;
   for (const ProfileSummaryEntry &SummaryEntry : SummaryVector) {
     if (SummaryEntry.Cutoff == HotFuncCutoff) {
       MinCountThreshold = SummaryEntry.MinCount;
double TotalSamplePercent =
             ? (Func.getTotalSamples() * 100.0) / ProfileTotalSample
             : 0;
     PrintValues.emplace_back(HotFuncInfo(
         Func.getContext().toString(), Func.getTotalSamples(),
         TotalSamplePercent, FuncPair.second.second, Func.getEntrySamples()));
   }
   dumpHotFunctionList(ColumnTitle, ColumnOffset, PrintValues, HotFuncCount,
                       Profiles.size(), HotFuncSample, ProfileTotalSample,
-                      Metric, OS);
+                      Metric, TopN, OS);
 
   return 0;
 }
 
 static int showSampleProfile(const std::string &Filename, bool ShowCounts,
-                             bool ShowAllFunctions, bool ShowDetailedSummary,
+                             uint32_t TopN, bool ShowAllFunctions,
+                             bool ShowDetailedSummary,
                              const std::string &ShowFunction,
                              bool ShowProfileSymbolList,
                              bool ShowSectionInfoOnly, bool ShowHotFuncList,
                              raw_fd_ostream &OS) {
   using namespace sampleprof;
   LLVMContext Context;
   auto ReaderOrErr =
       SampleProfileReader::create(Filename, Context, FSDiscriminatorPassOption);
if (ShowProfileSymbolList) {
     ReaderList->dump(OS);
   }
 
   if (ShowDetailedSummary) {
     auto &PS = Reader->getSummary();
     PS.printSummary(OS);
     PS.printDetailedSummary(OS);
   }
 
-  if (ShowHotFuncList)
-    showHotFunctionList(Reader->getProfiles(), Reader->getSummary(), OS);
+  if (ShowHotFuncList || TopN)
+    showHotFunctionList(Reader->getProfiles(), Reader->getSummary(), TopN, OS);
 
   return 0;
 }
 
 static int show_main(int argc, const char *argv[]) {
   cl::opt<std::string> Filename(cl::Positional, cl::Required,
                                 cl::desc("<profdata-file>"));
 
static int show_main(int argc, const char *argv[]) {
 
   if (ProfileKind == instr)
     return showInstrProfile(
         Filename, ShowCounts, TopNFunctions, ShowIndirectCallTargets,
         ShowMemOPSizes, ShowDetailedSummary, DetailedSummaryCutoffs,
         ShowAllFunctions, ShowCS, ValueCutoff, OnlyListBelow, ShowFunction,
         TextFormat, ShowBinaryIds, OS);
   else
-    return showSampleProfile(Filename, ShowCounts, ShowAllFunctions,
-                             ShowDetailedSummary, ShowFunction,
-                             ShowProfileSymbolList, ShowSectionInfoOnly,
-                             ShowHotFuncList, OS);
+    return showSampleProfile(Filename, ShowCounts, TopNFunctions,
+                             ShowAllFunctions, ShowDetailedSummary,
+                             ShowFunction, ShowProfileSymbolList,
+                             ShowSectionInfoOnly, ShowHotFuncList, OS);
 }
 
 int main(int argc, const char *argv[]) {
   InitLLVM X(argc, argv);
 
   StringRef ProgName(sys::path::filename(argv[0]));
   if (argc > 1) {
     int (*func)(int, const char *[]) = nullptr;