[SampleFDO] Add flag for non-dedicated profile.

[wmi]

Summary

The common profile usage is to collect profile from a target and then use the profile to guide the optimized build for the same target dedicatedly. There are some cases that no profile can be collected for a target. In those cases, although no dedicated profile is available, it is possible to have some generic profile to optimize common libraries and utilities. A flag is needed to tell the non-dedicated profile from the dedicated profile apart, so compiler can use different strategy for them.

[davidxl]

llvm/include/llvm/IR/ProfileSummary.h

54

--> If 'Dedicated' is false, it means the profile is for common/shared code. The common profile is merged from profiles collected from running other targets.

llvm/include/llvm/ProfileData/SampleProf.h

174

SecFlagPartial or SecFlagShared?

llvm/include/llvm/ProfileData/SampleProfWriter.h

223

setPartialProfile or setSharedProfile?

llvm/lib/ProfileData/SampleProfReader.cpp

850

partial or shared?

[wmi]

llvm/include/llvm/ProfileData/SampleProf.h

174

Change to SecFlagPartial here.

[wmi]

Address David's comment.

[wenlei]

Thanks for the patch! We're also thinking about differentiating default profile (from other targets) vs custom profile (from same target) for optimizations, so very curious about your compiler heuristic changes after this. :) I am guessing ProfileSampleAccurate and ProfileAccurateForSymsInList will one of those changes?

[wmi]

In D77426#1961284, @wenlei wrote:

Thanks for the patch! We're also thinking about differentiating default profile (from other targets) vs custom profile (from same target) for optimizations, so very curious about your compiler heuristic changes after this. :) I am guessing ProfileSampleAccurate and ProfileAccurateForSymsInList will one of those changes?

ProfileSampleAccurate and ProfileAccurateForSymsInList are already off by default so it is not urgent to guard the logic with the partial profile flag unless ProfileSampleAccurate and ProfileAccurateForSymsInList will be enabled by default.

An example for compiler heuristic to change is ProfileSummaryInfo::hasLargeWorkingSetSize/hasHugeWorkingSetSize. If the profile is a partial profile, those functions should return false in any case because there is no way to know the actual working set size from a partial profile.

[davidxl]

lgtm

[wenlei]

In D77426#1964396, @wmi wrote:

In D77426#1961284, @wenlei wrote:

Thanks for the patch! We're also thinking about differentiating default profile (from other targets) vs custom profile (from same target) for optimizations, so very curious about your compiler heuristic changes after this. :) I am guessing ProfileSampleAccurate and ProfileAccurateForSymsInList will one of those changes?

ProfileSampleAccurate and ProfileAccurateForSymsInList are already off by default so it is not urgent to guard the logic with the partial profile flag unless ProfileSampleAccurate and ProfileAccurateForSymsInList will be enabled by default.

An example for compiler heuristic to change is ProfileSummaryInfo::hasLargeWorkingSetSize/hasHugeWorkingSetSize. If the profile is a partial profile, those functions should return false in any case because there is no way to know the actual working set size from a partial profile.

Thanks for clarification, looking forward to the coming heuristic improvements.

llvm/include/llvm/IR/ProfileSummary.h

 public:
   enum Kind { PSK_Instr, PSK_CSInstr, PSK_Sample };
 
 private:
   const Kind PSK;
   SummaryEntryVector DetailedSummary;
   uint64_t TotalCount, MaxCount, MaxInternalCount, MaxFunctionCount;
   uint32_t NumCounts, NumFunctions;
+  /// If 'Partial' is false, it means the profile being used to optimize
+  /// a target is collected from the same target.
+  /// If 'Partial' is true, it means the profile is for common/shared
+  /// code. The common profile is usually merged from profiles collected
+  /// from running other targets.
+  bool Partial = false;
   /// Return detailed summary as metadata.
   Metadata *getDetailedSummaryMD(LLVMContext &Context);
 
 public:
   static const int Scale = 1000000;
 
   ProfileSummary(Kind K, SummaryEntryVector DetailedSummary,
                  uint64_t TotalCount, uint64_t MaxCount,
public:
   static ProfileSummary *getFromMD(Metadata *MD);
   SummaryEntryVector &getDetailedSummary() { return DetailedSummary; }
   uint32_t getNumFunctions() { return NumFunctions; }
   uint64_t getMaxFunctionCount() { return MaxFunctionCount; }
   uint32_t getNumCounts() { return NumCounts; }
   uint64_t getTotalCount() { return TotalCount; }
   uint64_t getMaxCount() { return MaxCount; }
   uint64_t getMaxInternalCount() { return MaxInternalCount; }
+  void setPartialProfile(bool PP) { Partial = PP; }
+  bool isPartialProfile() { return Partial; }
 };
 
 } // end namespace llvm
 
 #endif // LLVM_IR_PROFILESUMMARY_H

llvm/include/llvm/ProfileData/SampleProf.h

 
 // Section specific flags are defined here.
 // !!!Note: Everytime a new enum class is created here, please add
 // a new check in verifySecFlag.
 enum class SecNameTableFlags : uint32_t {
   SecFlagInValid = 0,
   SecFlagMD5Name = (1 << 0)
 };
+enum class SecProfSummaryFlags : uint32_t {
+  SecFlagInValid = 0,
+  /// SecFlagPartial means the profile is for common/shared code.
+  /// The common profile is usually merged from profiles collected
+  /// from running other targets.
+  SecFlagPartial = (1 << 0)
+};
 
 // Verify section specific flag is used for the correct section.
 template <class SecFlagType>
 static inline void verifySecFlag(SecType Type, SecFlagType Flag) {
   // No verification is needed for common flags.
   if (std::is_same<SecCommonFlags, SecFlagType>())
     return;
 
   // Verification starts here for section specific flag.
   bool IsFlagLegal = false;
   switch (Type) {
   case SecNameTable:
     IsFlagLegal = std::is_same<SecNameTableFlags, SecFlagType>();
     break;
+  case SecProfSummary:
+    IsFlagLegal = std::is_same<SecProfSummaryFlags, SecFlagType>();
+    break;
   default:
     break;
   }
   if (!IsFlagLegal)
     llvm_unreachable("Misuse of a flag in an incompatible section");
 }
 
 template <class SecFlagType>

llvm/include/llvm/ProfileData/SampleProfWriter.h

public:
   /// Create a new stream writer based on the value of \p Format.
   /// For testing.
   static ErrorOr<std::unique_ptr<SampleProfileWriter>>
   create(std::unique_ptr<raw_ostream> &OS, SampleProfileFormat Format);
 
   virtual void setProfileSymbolList(ProfileSymbolList *PSL) {}
   virtual void setToCompressAllSections() {}
   virtual void setUseMD5() {}
+  virtual void setPartialProfile() {}
 
 protected:
   SampleProfileWriter(std::unique_ptr<raw_ostream> &OS)
       : OutputStream(std::move(OS)) {}
 
   /// Write a file header for the profile file.
   virtual std::error_code
   writeHeader(const StringMap<FunctionSamples> &ProfileMap) = 0;
public:
   };
 
   // Set to use MD5 to represent string in NameTable.
   virtual void setUseMD5() override {
     UseMD5 = true;
     addSectionFlag(SecNameTable, SecNameTableFlags::SecFlagMD5Name);
   }
 
+  // Set the profile to be partial. It means the profile is for
+  // common/shared code. The common profile is usually merged from
+  // profiles collected from running other targets.
+  virtual void setPartialProfile() override {
+    addSectionFlag(SecProfSummary, SecProfSummaryFlags::SecFlagPartial);
+  }
+
 private:
   virtual void initSectionHdrLayout() override {
     // Note that SecFuncOffsetTable section is written after SecLBRProfile
     // in the profile, but is put before SecLBRProfile in SectionHdrLayout.
     //
     // This is because sample reader follows the order of SectionHdrLayout to
     // read each section, to read function profiles on demand sample reader
     // need to get the offset of each function profile first.

llvm/lib/ProfileData/SampleProfReader.cpp

 std::error_code SampleProfileReaderExtBinary::readOneSection(
     const uint8_t *Start, uint64_t Size, const SecHdrTableEntry &Entry) {
   Data = Start;
   End = Start + Size;
   switch (Entry.Type) {
   case SecProfSummary:
     if (std::error_code EC = readSummary())
       return EC;
+    if (hasSecFlag(Entry, SecProfSummaryFlags::SecFlagPartial))
+      Summary->setPartialProfile(true);
     break;
   case SecNameTable:
     if (std::error_code EC = readNameTableSec(
             hasSecFlag(Entry, SecNameTableFlags::SecFlagMD5Name)))
       return EC;
     break;
   case SecLBRProfile:
     if (std::error_code EC = readFuncProfiles())
uint64_t SampleProfileReaderExtBinaryBase::getFileSize() {
   // SecHdrTable to calculate the file size.
   uint64_t FileSize = 0;
   for (auto &Entry : SecHdrTable) {
     FileSize = std::max(Entry.Offset + Entry.Size, FileSize);
   }
   return FileSize;
 }
 
+static std::string getSecFlagsStr(const SecHdrTableEntry &Entry) {
+  std::string Flags;
+  if (hasSecFlag(Entry, SecCommonFlags::SecFlagCompress))
+    Flags.append("{compressed,");
+  else
+    Flags.append("{");
+
+  switch (Entry.Type) {
+  case SecNameTable:
+    if (hasSecFlag(Entry, SecNameTableFlags::SecFlagMD5Name))
+      Flags.append("md5,");
+    break;
+  case SecProfSummary:
+    if (hasSecFlag(Entry, SecProfSummaryFlags::SecFlagPartial))
+      Flags.append("partial,");
+    break;
+  default:
+    break;
+  }
+  char &last = Flags.back();
+  if (last == ',')
+    last = '}';
+  else
+    Flags.append("}");
+  return Flags;
+}
+
 bool SampleProfileReaderExtBinaryBase::dumpSectionInfo(raw_ostream &OS) {
   uint64_t TotalSecsSize = 0;
   for (auto &Entry : SecHdrTable) {
     OS << getSecName(Entry.Type) << " - Offset: " << Entry.Offset
-       << ", Size: " << Entry.Size << "\n";
+       << ", Size: " << Entry.Size << ", Flags: " << getSecFlagsStr(Entry)
+       << "\n";
+    ;
     TotalSecsSize += getSectionSize(Entry.Type);
   }
   uint64_t HeaderSize = SecHdrTable.front().Offset;
   assert(HeaderSize + TotalSecsSize == getFileSize() &&
          "Size of 'header + sections' doesn't match the total size of profile");
 
   OS << "Header Size: " << HeaderSize << "\n";
   OS << "Total Sections Size: " << TotalSecsSize << "\n";

llvm/test/tools/llvm-profdata/show-prof-info.test

+; RUN: llvm-profdata merge -sample -extbinary -use-md5 -compress-all-sections -partial-profile -prof-sym-list=%S/Inputs/profile-symbol-list-1.text %S/Inputs/sample-profile.proftext -o %t.1.output
+; RUN: wc -c < %t.1.output > %t.txt
+; RUN: llvm-profdata show -sample -show-sec-info-only %t.1.output >> %t.txt
+; RUN: FileCheck %s --input-file=%t.txt
+; CHECK: [[FILESIZE:.*]]
+; To check llvm-profdata shows the correct flags for ProfileSummarySection.
+; CHECK: ProfileSummarySection {{.*}} Flags: {compressed,partial}
+; To check llvm-profdata shows the correct flags for NameTableSection.
+; CHECK: NameTableSection {{.*}} Flags: {compressed,md5}
+; To check llvm-profdata shows the correct file size.
+; CHECK: [[FILESIZE]]

llvm/test/tools/llvm-profdata/show-prof-size.test

-; RUN: llvm-profdata merge -sample -extbinary -prof-sym-list=%S/Inputs/profile-symbol-list-1.text %S/Inputs/sample-profile.proftext -o %t.1.output
-; RUN: wc -c < %t.1.output > %t.txt
-; RUN: llvm-profdata show -sample -show-sec-info-only %t.1.output >> %t.txt
-; RUN: FileCheck %s --input-file=%t.txt
-; Check llvm-profdata shows the correct file size.
-; CHECK: [[FILESIZE:.*]]
-; CHECK: [[FILESIZE]]

llvm/tools/llvm-profdata/llvm-profdata.cpp

static void populateProfileSymbolList(MemoryBuffer *Buffer,
   for (StringRef symbol : SymbolVec)
     PSL.add(symbol);
 }
 
 static void handleExtBinaryWriter(sampleprof::SampleProfileWriter &Writer,
                                   ProfileFormat OutputFormat,
                                   MemoryBuffer *Buffer,
                                   sampleprof::ProfileSymbolList &WriterList,
-                                  bool CompressAllSections, bool UseMD5) {
+                                  bool CompressAllSections, bool UseMD5,
+                                  bool PartialProfile) {
   populateProfileSymbolList(Buffer, WriterList);
   if (WriterList.size() > 0 && OutputFormat != PF_Ext_Binary)
     warn("Profile Symbol list is not empty but the output format is not "
          "ExtBinary format. The list will be lost in the output. ");
 
   Writer.setProfileSymbolList(&WriterList);
 
   if (CompressAllSections) {
     if (OutputFormat != PF_Ext_Binary)
       warn("-compress-all-section is ignored. Specify -extbinary to enable it");
     else
       Writer.setToCompressAllSections();
   }
   if (UseMD5) {
     if (OutputFormat != PF_Ext_Binary)
       warn("-use-md5 is ignored. Specify -extbinary to enable it");
     else
       Writer.setUseMD5();
   }
+  if (PartialProfile) {
+    if (OutputFormat != PF_Ext_Binary)
+      warn("-partial-profile is ignored. Specify -extbinary to enable it");
+    else
+      Writer.setPartialProfile();
+  }
 }
 
 static void
 mergeSampleProfile(const WeightedFileVector &Inputs, SymbolRemapper *Remapper,
                    StringRef OutputFilename, ProfileFormat OutputFormat,
                    StringRef ProfileSymbolListFile, bool CompressAllSections,
-                   bool UseMD5, FailureMode FailMode) {
+                   bool UseMD5, bool PartialProfile, FailureMode FailMode) {
   using namespace sampleprof;
   StringMap<FunctionSamples> ProfileMap;
   SmallVector<std::unique_ptr<sampleprof::SampleProfileReader>, 5> Readers;
   LLVMContext Context;
   sampleprof::ProfileSymbolList WriterList;
   for (const auto &Input : Inputs) {
     auto ReaderOrErr = SampleProfileReader::create(Input.Filename, Context);
     if (std::error_code EC = ReaderOrErr.getError()) {
mergeSampleProfile(const WeightedFileVector &Inputs, SymbolRemapper *Remapper,
   if (std::error_code EC = WriterOrErr.getError())
     exitWithErrorCode(EC, OutputFilename);
 
   auto Writer = std::move(WriterOrErr.get());
   // WriterList will have StringRef refering to string in Buffer.
   // Make sure Buffer lives as long as WriterList.
   auto Buffer = getInputFileBuf(ProfileSymbolListFile);
   handleExtBinaryWriter(*Writer, OutputFormat, Buffer.get(), WriterList,
-                        CompressAllSections, UseMD5);
+                        CompressAllSections, UseMD5, PartialProfile);
   Writer->write(ProfileMap);
 }
 
 static WeightedFile parseWeightedFile(const StringRef &WeightedFilename) {
   StringRef WeightStr, FileName;
   std::tie(WeightStr, FileName) = WeightedFilename.split(',');
 
   uint64_t Weight;
static int merge_main(int argc, const char *argv[]) {
   cl::opt<bool> CompressAllSections(
       "compress-all-sections", cl::init(false), cl::Hidden,
       cl::desc("Compress all sections when writing the profile (only "
                "meaningful for -extbinary)"));
   cl::opt<bool> UseMD5(
       "use-md5", cl::init(false), cl::Hidden,
       cl::desc("Choose to use MD5 to represent string in name table (only "
                "meaningful for -extbinary)"));
+  cl::opt<bool> PartialProfile(
+      "partial-profile", cl::init(false), cl::Hidden,
+      cl::desc("Set the profile to be a partial profile (only meaningful "
+               "for -extbinary)"));
 
   cl::ParseCommandLineOptions(argc, argv, "LLVM profile data merger\n");
 
   WeightedFileVector WeightedInputs;
   for (StringRef Filename : InputFilenames)
     addWeightedInput(WeightedInputs, {std::string(Filename), 1});
   for (StringRef WeightedFilename : WeightedInputFilenames)
     addWeightedInput(WeightedInputs, parseWeightedFile(WeightedFilename));
if (!RemappingFile.empty())
     Remapper = SymbolRemapper::create(RemappingFile);
 
   if (ProfileKind == instr)
     mergeInstrProfile(WeightedInputs, Remapper.get(), OutputFilename,
                       OutputFormat, OutputSparse, NumThreads, FailureMode);
   else
     mergeSampleProfile(WeightedInputs, Remapper.get(), OutputFilename,
                        OutputFormat, ProfileSymbolListFile, CompressAllSections,
-                       UseMD5, FailureMode);
+                       UseMD5, PartialProfile, FailureMode);
 
   return 0;
 }
 
 /// Computer the overlap b/w profile BaseFilename and profile TestFilename.
 static void overlapInstrProfile(const std::string &BaseFilename,
                                 const std::string &TestFilename,
                                 const OverlapFuncFilters &FuncFilter,