[CSSPGO] llvm-profdata support for CS profile.

[hoy]

Summary

Context-sensitive AutoFDO profile has a different name scheme where full calling contexts are encoded as function names. When processing CS proifle, llvm-profdata should use full contexts instead of leaf function names.

[wenlei]

llvm/tools/llvm-profdata/llvm-profdata.cpp

701

I see explicit hasValue() call used all over the place for Optional values, and I also feel it's more readable to spell it out instead of relying on operator(). Not sure if it's an established convention though.

[hoy]

llvm/tools/llvm-profdata/llvm-profdata.cpp

701

Not sure about the convention. There are also usages like this (using operator()) in LLVM. But agreed it's more clear with hasValue(). Will change to that.

[hoy]

Addressing Wenlei's comment.

[wenlei]

lgtm.

[wlei]

LGTM.

[wmi]

llvm/tools/llvm-profdata/llvm-profdata.cpp

1907–1910

This is not added for cs instrument fdo profile. Could you update the the description to differentiate with csspgo?

1917–1920

Is it better to update the description for csspgo since the function filter will be used as context filter? similar for value-cutoff and similarity-cutoff.

[hoy]

llvm/tools/llvm-profdata/llvm-profdata.cpp

1907–1910

Good point, the switch is for CSPGO only.

1917–1920

Sounds good. They are applied to contexts with csspgo.

[hoy]

Addressing Wei's comments.

[wmi]

LGTM.

llvm/test/tools/llvm-profdata/sample-hot-func-list.test

 ; RUN: llvm-profdata show --sample --hot-func-list %S/Inputs/sample-hot-func-list.proftext | FileCheck %s --match-full-lines --strict-whitespace
 
 ;      CHECK:8 out of 10 functions with profile (80.00%) are considered hot functions (max sample >= 470).
 ; CHECK-NEXT:355251 out of 356026 profile counts (99.78%) are from hot functions.
 ; CHECK-NEXT: Total sample (%)       Max sample        Entry sample    Function name
 ; CHECK-NEXT: 184019 (51.69%)        2300              534             main
 ; CHECK-NEXT: 97401 (27.36%)         10640             3035            Func3
 ; CHECK-NEXT: 20305 (5.70%)          1000              1000            _Z3bazi
 ; CHECK-NEXT: 20301 (5.70%)          1437              1437            _Z3bari
 ; CHECK-NEXT: 17043 (4.79%)          3105              1594            Func2
 ; CHECK-NEXT: 7711 (2.17%)           610               610             _Z3fooi
 ; CHECK-NEXT: 6948 (1.95%)           3507              470             Func5
 ; CHECK-NEXT: 1523 (0.43%)           563               169             Func1
+
+
+; RUN: llvm-profdata show --sample --hot-func-list %S/Inputs/cs-sample.proftext | FileCheck %s --match-full-lines --strict-whitespace --check-prefix=CS
+
+;      CS:2 out of 8 functions with profile (25.00%) are considered hot functions (max sample >= 23324).
+; CS-NEXT:1968152 out of 1968919 profile counts (99.96%) are from hot functions.
+; CS-NEXT: Total sample (%)       Max sample        Entry sample    Function name
+; CS-NEXT: 1467299 (74.52%)       287884            11              main:3 @ _Z5funcAi:1 @ _Z8funcLeafi
+; CS-NEXT: 500853 (25.44%)        74946             20              main:3.1 @ _Z5funcBi:1 @ _Z8funcLeafi

llvm/test/tools/llvm-profdata/sample-overlap.test

 ; OVERLAP4:    hot blocks unique in base profile: 9
 ; OVERLAP4:    hot blocks unique in test profile: 9
 ; OVERLAP4:Function-level details:
 ; OVERLAP4:Base weight    Test weight    Similarity   Overlap   Base unique   Test unique   Base samples   Test samples   Function name
 ; OVERLAP4:78.15%         70.17%         23.33%       11.18%    66.14%        74.64%        10896          7166           main
 
 ; RUN: llvm-profdata overlap --sample %S/Inputs/sample-overlap-0.proftext %S/Inputs/sample-overlap-5.proftext | FileCheck %s --check-prefix=OVERLAP5  --match-full-lines --strict-whitespace
 ; OVERLAP5:Sum of sample counts for profile {{.*}}/Inputs/sample-overlap-5.proftext is 0.
+
+
+; RUN: llvm-profdata overlap --sample %S/Inputs/cs-sample.proftext %S/Inputs/cs-sample.proftext | FileCheck %s --check-prefix=CS  --match-full-lines --strict-whitespace
+; CS:Program level:
+; CS:  Whole program profile similarity: 100.000%
+; CS:  Whole program sample overlap: 100.000%
+; CS:    percentage of samples unique in base profile: 0.000%
+; CS:    percentage of samples unique in test profile: 0.000%
+; CS:    total samples in base profile: 772562
+; CS:    total samples in test profile: 772562
+; CS:  Function overlap: 100.000%
+; CS:    overlap functions: 8
+; CS:    functions unique in base profile: 0
+; CS:    functions unique in test profile: 0
+; CS:  Hot-function overlap: 100.000%
+; CS:    overlap hot functions: 2
+; CS:    hot functions unique in base profile: 0
+; CS:    hot functions unique in test profile: 0
+; CS:  Hot-block overlap: 100.000%
+; CS:    overlap hot blocks: 6
+; CS:    hot blocks unique in base profile: 0
+; CS:    hot blocks unique in test profile: 0

llvm/tools/llvm-profdata/llvm-profdata.cpp

mergeSampleProfile(const WeightedFileVector &Inputs, SymbolRemapper *Remapper,
                    StringRef ProfileSymbolListFile, bool CompressAllSections,
                    bool UseMD5, bool GenPartialProfile, FailureMode FailMode) {
   using namespace sampleprof;
   StringMap<FunctionSamples> ProfileMap;
   SmallVector<std::unique_ptr<sampleprof::SampleProfileReader>, 5> Readers;
   LLVMContext Context;
   sampleprof::ProfileSymbolList WriterList;
   Optional<bool> ProfileIsProbeBased;
+  Optional<bool> ProfileIsCS;
   for (const auto &Input : Inputs) {
     auto ReaderOrErr = SampleProfileReader::create(Input.Filename, Context);
     if (std::error_code EC = ReaderOrErr.getError()) {
       warnOrExitGivenError(FailMode, EC, Input.Filename);
       continue;
     }
 
     // We need to keep the readers around until after all the files are
     // read so that we do not lose the function names stored in each
     // reader's memory. The function names are needed to write out the
     // merged profile map.
     Readers.push_back(std::move(ReaderOrErr.get()));
     const auto Reader = Readers.back().get();
     if (std::error_code EC = Reader->read()) {
       warnOrExitGivenError(FailMode, EC, Input.Filename);
       Readers.pop_back();
       continue;
     }
 
     StringMap<FunctionSamples> &Profiles = Reader->getProfiles();
-    if (ProfileIsProbeBased &&
+    if (ProfileIsProbeBased.hasValue() &&
         ProfileIsProbeBased != FunctionSamples::ProfileIsProbeBased)
       exitWithError(
           "cannot merge probe-based profile with non-probe-based profile");
     ProfileIsProbeBased = FunctionSamples::ProfileIsProbeBased;
+    if (ProfileIsCS.hasValue() && ProfileIsCS != FunctionSamples::ProfileIsCS)
+      exitWithError("cannot merge CS profile with non-CS profile");
+    ProfileIsCS = FunctionSamples::ProfileIsCS;
     for (StringMap<FunctionSamples>::iterator I = Profiles.begin(),
                                               E = Profiles.end();
          I != E; ++I) {
       sampleprof_error Result = sampleprof_error::success;
       FunctionSamples Remapped =
           Remapper ? remapSamples(I->second, *Remapper, Result)
                    : FunctionSamples();
       FunctionSamples &Samples = Remapper ? Remapped : I->second;

 }
 
 void SampleOverlapAggregator::computeSampleProfileOverlap(raw_fd_ostream &OS) {
   using namespace sampleprof;
 
   StringMap<const FunctionSamples *> BaseFuncProf;
   const auto &BaseProfiles = BaseReader->getProfiles();
   for (const auto &BaseFunc : BaseProfiles) {
-    BaseFuncProf.try_emplace(BaseFunc.second.getName(), &(BaseFunc.second));
+    BaseFuncProf.try_emplace(BaseFunc.second.getNameWithContext(),
+                             &(BaseFunc.second));
   }
   ProfOverlap.UnionCount = BaseFuncProf.size();
 
   const auto &TestProfiles = TestReader->getProfiles();
   for (const auto &TestFunc : TestProfiles) {
     SampleOverlapStats FuncOverlap;
-    FuncOverlap.TestName = TestFunc.second.getName();
+    FuncOverlap.TestName = TestFunc.second.getNameWithContext();
     assert(TestStats.count(FuncOverlap.TestName) &&
            "TestStats should have records for all functions in test profile "
            "except inlinees");
     FuncOverlap.TestSample = TestStats[FuncOverlap.TestName].SampleSum;
 
     const auto Match = BaseFuncProf.find(FuncOverlap.TestName);
     if (Match == BaseFuncProf.end()) {
       const FuncSampleStats &FuncStats = TestStats[FuncOverlap.TestName];
if (Match == BaseFuncProf.end()) {
 
       ++ProfOverlap.UnionCount;
       ProfOverlap.UnionSample += FuncStats.SampleSum;
     } else {
       ++ProfOverlap.OverlapCount;
 
       // Two functions match with each other. Compute function-level overlap and
       // aggregate them into profile-level overlap.
-      FuncOverlap.BaseName = Match->second->getName();
+      FuncOverlap.BaseName = Match->second->getNameWithContext();
       assert(BaseStats.count(FuncOverlap.BaseName) &&
              "BaseStats should have records for all functions in base profile "
              "except inlinees");
       FuncOverlap.BaseSample = BaseStats[FuncOverlap.BaseName].SampleSum;
 
       FuncOverlap.Similarity = computeSampleFunctionOverlap(
           Match->second, &TestFunc.second, &FuncOverlap, FuncOverlap.BaseSample,
           FuncOverlap.TestSample);
if (TestStats[FuncOverlap.TestName].MaxSample >= FuncFilter.ValueCutoff ||
       FuncOverlap.TestWeight =
           static_cast<double>(FuncOverlap.TestSample) / ProfOverlap.TestSample;
       FuncSimilarityDump.emplace(FuncOverlap.BaseWeight, FuncOverlap);
     }
   }
 
   // Traverse through functions in base profile but not in test profile.
   for (const auto &F : BaseFuncProf) {
-    assert(BaseStats.count(F.second->getName()) &&
+    assert(BaseStats.count(F.second->getNameWithContext()) &&
            "BaseStats should have records for all functions in base profile "
            "except inlinees");
-    const FuncSampleStats &FuncStats = BaseStats[F.second->getName()];
+    const FuncSampleStats &FuncStats =
+        BaseStats[F.second->getNameWithContext()];
     ++ProfOverlap.BaseUniqueCount;
     ProfOverlap.BaseUniqueSample += FuncStats.SampleSum;
 
     updateHotBlockOverlap(FuncStats.SampleSum, 0, FuncStats.HotBlockCount);
 
     double FuncSimilarity = computeSampleFunctionOverlap(
         nullptr, nullptr, nullptr, FuncStats.SampleSum, 0);
     ProfOverlap.Similarity +=

 
 void SampleOverlapAggregator::initializeSampleProfileOverlap() {
   const auto &BaseProf = BaseReader->getProfiles();
   for (const auto &I : BaseProf) {
     ++ProfOverlap.BaseCount;
     FuncSampleStats FuncStats;
     getFuncSampleStats(I.second, FuncStats, BaseHotThreshold);
     ProfOverlap.BaseSample += FuncStats.SampleSum;
-    BaseStats.try_emplace(I.second.getName(), FuncStats);
+    BaseStats.try_emplace(I.second.getNameWithContext(), FuncStats);
   }
 
   const auto &TestProf = TestReader->getProfiles();
   for (const auto &I : TestProf) {
     ++ProfOverlap.TestCount;
     FuncSampleStats FuncStats;
     getFuncSampleStats(I.second, FuncStats, TestHotThreshold);
     ProfOverlap.TestSample += FuncStats.SampleSum;
-    TestStats.try_emplace(I.second.getName(), FuncStats);
+    TestStats.try_emplace(I.second.getNameWithContext(), FuncStats);
   }
 
   ProfOverlap.BaseName = StringRef(BaseFilename);
   ProfOverlap.TestName = StringRef(TestFilename);
 }
 
 void SampleOverlapAggregator::dumpFuncSimilarity(raw_fd_ostream &OS) const {
   using namespace sampleprof;
std::error_code SampleOverlapAggregator::loadProfiles() {
 
   if (std::error_code EC = BaseReader->read())
     exitWithErrorCode(EC, BaseFilename);
   if (std::error_code EC = TestReader->read())
     exitWithErrorCode(EC, TestFilename);
   if (BaseReader->profileIsProbeBased() != TestReader->profileIsProbeBased())
     exitWithError(
         "cannot compare probe-based profile with non-probe-based profile");
+  if (BaseReader->profileIsCS() != TestReader->profileIsCS())
+    exitWithError("cannot compare CS profile with non-CS profile");
 
   // Load BaseHotThreshold and TestHotThreshold as 99-percentile threshold in
   // profile summary.
   const uint64_t HotCutoff = 990000;
   ProfileSummary &BasePS = BaseReader->getSummary();
   for (const auto &SummaryEntry : BasePS.getDetailedSummary()) {
     if (SummaryEntry.Cutoff == HotCutoff) {
       BaseHotThreshold = SummaryEntry.MinCount;

 
 static int overlap_main(int argc, const char *argv[]) {
   cl::opt<std::string> BaseFilename(cl::Positional, cl::Required,
                                     cl::desc("<base profile file>"));
   cl::opt<std::string> TestFilename(cl::Positional, cl::Required,
                                     cl::desc("<test profile file>"));
   cl::opt<std::string> Output("output", cl::value_desc("output"), cl::init("-"),
                               cl::desc("Output file"));
   cl::alias OutputA("o", cl::desc("Alias for --output"), cl::aliasopt(Output));
-  cl::opt<bool> IsCS("cs", cl::init(false),
-                     cl::desc("For context sensitive counts"));
+  cl::opt<bool> IsCS(
+      "cs", cl::init(false),
+      cl::desc("For context sensitive PGO counts. Does not work with CSSPGO."));
   cl::opt<unsigned long long> ValueCutoff(
       "value-cutoff", cl::init(-1),
       cl::desc(
-          "Function level overlap information for every function in test "
+          "Function level overlap information for every function (with calling "
+          "context for csspgo) in test "
           "profile with max count value greater then the parameter value"));
   cl::opt<std::string> FuncNameFilter(
       "function",
-      cl::desc("Function level overlap information for matching functions"));
+      cl::desc("Function level overlap information for matching functions. For "
+               "CSSPGO this takes a a function name with calling context"));
   cl::opt<unsigned long long> SimilarityCutoff(
       "similarity-cutoff", cl::init(0),
-      cl::desc(
-          "For sample profiles, list function names for overlapped functions "
+      cl::desc("For sample profiles, list function names (with calling context "
+               "for csspgo) for overlapped functions "
                "with similarities below the cutoff (percentage times 10000)."));
   cl::opt<ProfileKinds> ProfileKind(
       cl::desc("Profile kind:"), cl::init(instr),
       cl::values(clEnumVal(instr, "Instrumentation profile (default)"),
                  clEnumVal(sample, "Sample profile")));
   cl::ParseCommandLineOptions(argc, argv, "LLVM profile data overlap tool\n");
 
   std::error_code EC;
   raw_fd_ostream OS(Output.data(), EC, sys::fs::OF_Text);
std::string Metric =
       std::string("max sample >= ") + std::to_string(MinCountThreshold);
   std::vector<HotFuncInfo> PrintValues;
   for (const auto &FuncPair : HotFunc) {
     const FunctionSamples &Func = *FuncPair.second.first;
     double TotalSamplePercent =
         (ProfileTotalSample > 0)
             ? (Func.getTotalSamples() * 100.0) / ProfileTotalSample
             : 0;
-    PrintValues.emplace_back(
-        HotFuncInfo(Func.getName(), Func.getTotalSamples(), TotalSamplePercent,
-                    FuncPair.second.second, Func.getEntrySamples()));
+    PrintValues.emplace_back(HotFuncInfo(
+        Func.getNameWithContext(), Func.getTotalSamples(), TotalSamplePercent,
   }
   dumpHotFunctionList(ColumnTitle, ColumnOffset, PrintValues, HotFuncCount,
                       Profiles.size(), HotFuncSample, ProfileTotalSample,
                       Metric, OS);
 
   return 0;
 }