[memprof] Print out profile build ids in the error message.

[snehasish]

Summary

When no --profiled-binary flag is provided we can print out the build
ids of the modules in the profile. This can help the user fetch the
correct binary from e.g. remote object store.

[snehasish]

Remove unused include.

[tejohnson]

llvm/lib/ProfileData/RawMemProfReader.cpp

204

Should we pass the Path instead of "" for the report() Context parameter? If not useful, probably document the constant "" parameter.

542

Why do we need them in the order they were stored?

554

formatting seems off - run through clang format.

[snehasish]

Address comments.

[snehasish]

Thanks for the quick review!

llvm/lib/ProfileData/RawMemProfReader.cpp

204

Opted to document the param since I don't think printing the profile path is useful (the binary path is unset).

542

If we keep them in order then the profiled binary build id *should* be the first entry in this list. We rely on the sanitizer tooling (which calls dl_iterate_phdr [1]) to maintain the order which is read from the program headers and we preserve the order in the profile. Added a short comment.

[1] https://github.com/llvm/llvm-project/blob/main/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cpp#L737

"The first object visited by callback is the main program."
https://man7.org/linux/man-pages/man3/dl_iterate_phdr.3.html

[tejohnson]

lgtm

llvm/lib/ProfileData/RawMemProfReader.cpp

542

Thanks for the clarification. I would just add the man page link and excerpt to the comment to document why it should be first.

[snehasish]

Rebase, address comment.

llvm/include/llvm/ProfileData/RawMemProfReader.h

public:
 
   // Create a RawMemProfReader after sanity checking the contents of the file at
   // \p Path. The binary from which the profile has been collected is specified
   // via a path in \p ProfiledBinary.
   static Expected<std::unique_ptr<RawMemProfReader>>
   create(const Twine &Path, const StringRef ProfiledBinary,
          bool KeepName = false);
 
+  // Returns a list of build ids recorded in the segment information.
+  static std::vector<std::string> peekBuildIds(MemoryBuffer *DataBuffer);
+
   using GuidMemProfRecordPair = std::pair<GlobalValue::GUID, MemProfRecord>;
   using Iterator = InstrProfIterator<GuidMemProfRecordPair, RawMemProfReader>;
   Iterator end() { return Iterator(); }
   Iterator begin() {
     Iter = FunctionProfileData.begin();
     return Iterator(this);
   }
 

llvm/lib/ProfileData/RawMemProfReader.cpp

 
 #include <algorithm>
 #include <cstdint>
 #include <memory>
 #include <type_traits>
 
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/DenseMap.h"
+#include "llvm/ADT/DenseMapInfo.h"
 #include "llvm/ADT/SetVector.h"
+#include "llvm/ADT/SmallSet.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/ADT/StringExtras.h"
+#include "llvm/ADT/Twine.h"
 #include "llvm/DebugInfo/DWARF/DWARFContext.h"
 #include "llvm/DebugInfo/Symbolize/SymbolizableModule.h"
 #include "llvm/DebugInfo/Symbolize/SymbolizableObjectFile.h"
 #include "llvm/Object/Binary.h"
 #include "llvm/Object/BuildID.h"
 #include "llvm/Object/ELFObjectFile.h"
 #include "llvm/Object/ObjectFile.h"
 #include "llvm/ProfileData/InstrProf.h"
 #include "llvm/ProfileData/MemProf.h"
 #include "llvm/ProfileData/MemProfData.inc"
 #include "llvm/ProfileData/RawMemProfReader.h"
 #include "llvm/Support/Debug.h"
 #include "llvm/Support/Endian.h"
 #include "llvm/Support/Error.h"
+#include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/Path.h"
 
 #define DEBUG_TYPE "memprof"
 
 namespace llvm {
 namespace memprof {
 namespace {
 template <class T = uint64_t> inline T alignedRead(const char *Ptr) {
RawMemProfReader::create(const Twine &Path, const StringRef ProfiledBinary,
   auto BufferOr = MemoryBuffer::getFileOrSTDIN(Path);
   if (std::error_code EC = BufferOr.getError())
     return report(errorCodeToError(EC), Path.getSingleStringRef());
 
   std::unique_ptr<MemoryBuffer> Buffer(BufferOr.get().release());
   if (Error E = checkBuffer(*Buffer))
     return report(std::move(E), Path.getSingleStringRef());
 
-  if (ProfiledBinary.empty())
+  if (ProfiledBinary.empty()) {
+    // Peek the build ids to print a helpful error message.
+    const std::vector<std::string> BuildIds = peekBuildIds(Buffer.get());
+    std::string ErrorMessage(
+        R"(Path to profiled binary is empty, expected binary with one of the following build ids:
+)");
+    for (const auto &Id : BuildIds) {
+      ErrorMessage += "\n BuildId: ";
+      ErrorMessage += Id;
+    }
     return report(
-        errorCodeToError(make_error_code(std::errc::invalid_argument)),
-        "Path to profiled binary is empty!");
+        make_error<StringError>(ErrorMessage, inconvertibleErrorCode()),
+        /*Context=*/"");
+  }
 
   auto BinaryOr = llvm::object::createBinary(ProfiledBinary);
   if (!BinaryOr) {
     return report(BinaryOr.takeError(), ProfiledBinary);
   }
 
   // Use new here since constructor is private.
   std::unique_ptr<RawMemProfReader> Reader(
Error RawMemProfReader::symbolizeAndFilterStackFrames() {
   if (StackMap.empty())
     return make_error<InstrProfError>(
         instrprof_error::malformed,
         "no entries in callstack map after symbolization");
 
   return Error::success();
 }
 
+std::vector<std::string>
+RawMemProfReader::peekBuildIds(MemoryBuffer *DataBuffer) {
+  const char *Next = DataBuffer->getBufferStart();
+  // Use a set + vector since a profile file may contain multiple raw profile
+  // dumps, each with segment information. We want them unique and in order they
+  // were stored in the profile; the profiled binary should be the first entry.
+  // The runtime uses dl_iterate_phdr and the "... first object visited by
+  // callback is the main program."
+  // https://man7.org/linux/man-pages/man3/dl_iterate_phdr.3.html
+  std::vector<std::string> BuildIds;
+  llvm::SmallSet<StringRef, 4> BuildIdsSet;
+  while (Next < DataBuffer->getBufferEnd()) {
+    auto *Header = reinterpret_cast<const memprof::Header *>(Next);
+
+    const llvm::SmallVector<SegmentEntry> Entries =
+        readSegmentEntries(Next + Header->SegmentOffset);
+
+    for (const auto &Entry : Entries) {
+      const std::string Id = getBuildIdString(Entry);
+      if (BuildIdsSet.contains(Id))
+        continue;
+      BuildIds.push_back(Id);
+      BuildIdsSet.insert(BuildIds.back());
+    }
+
+    Next += Header->TotalSize;
+  }
+  return BuildIds;
+}
+
 Error RawMemProfReader::readRawProfile(
     std::unique_ptr<MemoryBuffer> DataBuffer) {
   const char *Next = DataBuffer->getBufferStart();
 
   while (Next < DataBuffer->getBufferEnd()) {
     auto *Header = reinterpret_cast<const memprof::Header *>(Next);
 
     // Read in the segment information, check whether its the same across all

llvm/test/tools/llvm-profdata/memprof-buildid.test

 REQUIRES: x86_64-linux
 
 To update the inputs used below run Inputs/update_memprof_inputs.sh /path/to/updated/clang
 RUN: llvm-readelf --notes %p/Inputs/buildid.memprofexe > %t1.txt
 RUN: llvm-profdata show --memory %p/Inputs/buildid.memprofraw --profiled-binary %p/Inputs/buildid.memprofexe -o -  > %t2.txt
 RUN: cat %t1.txt %t2.txt | FileCheck %s
 
+Test that we print out the profile build ids when --profiled-binary is empty. 
+RUN: not llvm-profdata show --memory %p/Inputs/buildid.memprofraw -o - 2> %t3.txt
+RUN: cat %t1.txt %t3.txt | FileCheck %s
+
 COM: First extract the id from the llvm-readelf output.
 CHECK: Build ID: [[ID:[[:xdigit:]]+]]
 
 COM: Then match it with the profdata output.
-CHECK: BuildId: {{.*}}[[ID]]
+CHECK-COUNT-1: BuildId: {{.*}}[[ID]]