[memprof] Record BuildIDs in the raw profile.

[snehasish]

Summary

This patch adds support for recording BuildIds usng the sanitizer
ListOfModules API. We add another entry to the SegmentEntry struct and
change the memprof raw version.

[snehasish]

Remove commented code.

[tejohnson]

compiler-rt/include/profile/MemProfData.inc

58–59

Do we need to record the size or can we just pad with zeros if less than the max? Is it ever not 32 in practice? Looks like it is defined to 32 in sanitizer_common.h.

[snehasish]

Looking into the regenerating the profiles for the PGOProfile tests.

compiler-rt/include/profile/MemProfData.inc

58–59

I think we need it. In practice, we usually have 16 or 20 byte buildids [1] but they could be of arbitrary length if it's provided by the user rather than generated by the linker. Also the build id is a raw sequence of bytes. Padding with zeros is not enough to distinguish the length if the build id itself contains leading/trailing zero bytes.

[1] https://github.com/llvm/llvm-project/blob/8a5d4eb775c644d8683f24817d44c510d2b853b7/lld/ELF/Writer.cpp#L2938-L2975

[tejohnson]

compiler-rt/include/profile/MemProfData.inc

58–59

Ok got it. I thought sanitizer_common.h always used 32, but I now see another interface where it can be variable (but max of 32, which matches the handling here).

[tejohnson]

lgtm - but does this depend on D145644 and require some additional test changes?

[snehasish]

In D145190#4181608, @tejohnson wrote:

lgtm - but does this depend on D145644 and require some additional test changes?

Yes, I will rebase this patch once that is submitted. I need to look at the LLVM.Transforms/PGOProfile::memprof.ll test failures.

[snehasish]

Update raw profiles, add another test for buildids only.

[snehasish]

In D145190#4181789, @snehasish wrote:

In D145190#4181608, @tejohnson wrote:

lgtm - but does this depend on D145644 and require some additional test changes?

Yes, I will rebase this patch once that is submitted. I need to look at the LLVM.Transforms/PGOProfile::memprof.ll test failures.

@tejohnson I've rebased the patch and updated the raw profiles for all the tests with the new version. Also added a new test for llvm-profdata to check the buildid we serialize in the raw profile. No changes to the code from prior diffs. PTAL, thanks!

[snehasish]

Update test regex.

[tejohnson]

lgtm

[snehasish]

Reopening since I missed the updates to the memprof unit tests in compiler-rt. This was caught by the buildbots. As part of the changes I modified the SerializeToRawProfile API slightly to make it easier to test. Also regenerated the raw profiles and binaries again since the runtime changed.

@tejohnson PTAL, thanks!

[snehasish]

Update the unittest in compiler-rt which was caught by the build bot failures.

[tejohnson]

lgtm

compiler-rt/include/profile/MemProfData.inc

  *
  * This file has two identical copies. The primary copy lives in LLVM and
  * the other one sits in compiler-rt/include/profile directory. To make changes
  * in this file, first modify the primary copy and copy it over to compiler-rt.
  * Testing of any change in this file can start only after the two copies are
  * synced up.
  *
 \*===----------------------------------------------------------------------===*/
+#include <string.h>
 
 #ifdef _MSC_VER
 #define PACKED(...) __pragma(pack(push,1)) __VA_ARGS__ __pragma(pack(pop))
 #else
 #define PACKED(...) __VA_ARGS__ __attribute__((__packed__))
 #endif
 
 // A 64-bit magic number to uniquely identify the raw binary memprof profile file.
 #define MEMPROF_RAW_MAGIC_64                                                                        \
   ((uint64_t)255 << 56 | (uint64_t)'m' << 48 | (uint64_t)'p' << 40 | (uint64_t)'r' << 32 |          \
    (uint64_t)'o' << 24 | (uint64_t)'f' << 16 | (uint64_t)'r' << 8 | (uint64_t)129)
 
 // The version number of the raw binary format.
-#define MEMPROF_RAW_VERSION 2ULL
+#define MEMPROF_RAW_VERSION 3ULL
+
+#define MEMPROF_BUILDID_MAX_SIZE 32ULL
 
 namespace llvm {
 namespace memprof {
 // A struct describing the header used for the raw binary memprof profile format.
 PACKED(struct Header {
   uint64_t Magic;
   uint64_t Version;
   uint64_t TotalSize;
   uint64_t SegmentOffset;
   uint64_t MIBOffset;
   uint64_t StackOffset;
 });
 
-
 // A struct describing the information necessary to describe a /proc/maps
 // segment entry for a particular binary/library identified by its build id.
 PACKED(struct SegmentEntry {
   uint64_t Start;
   uint64_t End;
   uint64_t Offset;
-  // This field is unused until sanitizer procmaps support for build ids for
-  // Linux-Elf is implemented.
-  uint8_t BuildId[32] = {0};
+  uint64_t BuildIdSize;
+  uint8_t BuildId[MEMPROF_BUILDID_MAX_SIZE] = {0};
 
-  SegmentEntry(uint64_t S, uint64_t E, uint64_t O) :
-    Start(S), End(E), Offset(O) {}
+  // This constructor is only used in tests so don't set the BuildId.
+  SegmentEntry(uint64_t S, uint64_t E, uint64_t O)
+      : Start(S), End(E), Offset(O), BuildIdSize(0) {}
 
   SegmentEntry(const SegmentEntry& S) {
     Start = S.Start;
     End = S.End;
     Offset = S.Offset;
+    BuildIdSize = S.BuildIdSize;
+    memcpy(BuildId, S.BuildId, S.BuildIdSize);
   }
 
   SegmentEntry& operator=(const SegmentEntry& S) {
     Start = S.Start;
     End = S.End;
     Offset = S.Offset;
+    BuildIdSize = S.BuildIdSize;
+    memcpy(BuildId, S.BuildId, S.BuildIdSize);
     return *this;
   }
 
   bool operator==(const SegmentEntry& S) const {
-    return Start == S.Start &&
-           End == S.End &&
-           Offset == S.Offset;
+    return Start == S.Start && End == S.End && Offset == S.Offset &&
+           BuildIdSize == S.BuildIdSize &&
+           memcmp(BuildId, S.BuildId, S.BuildIdSize) == 0;
   }
 });
 
 // Packed struct definition for MSVC. We can't use the PACKED macro defined in
 // MemProfData.inc since it would mean we are embedding a directive (the
 // #include for MIBEntryDef) into the macros which is undefined behaviour.
 #ifdef _MSC_VER
 __pragma(pack(push,1))

compiler-rt/lib/memprof/memprof_allocator.cpp

 #include "memprof_mibmap.h"
 #include "memprof_rawprofile.h"
 #include "memprof_stack.h"
 #include "memprof_thread.h"
 #include "profile/MemProfData.inc"
 #include "sanitizer_common/sanitizer_allocator_checks.h"
 #include "sanitizer_common/sanitizer_allocator_interface.h"
 #include "sanitizer_common/sanitizer_allocator_report.h"
+#include "sanitizer_common/sanitizer_common.h"
 #include "sanitizer_common/sanitizer_errno.h"
 #include "sanitizer_common/sanitizer_file.h"
 #include "sanitizer_common/sanitizer_flags.h"
 #include "sanitizer_common/sanitizer_internal_defs.h"
-#include "sanitizer_common/sanitizer_procmaps.h"
 #include "sanitizer_common/sanitizer_stackdepot.h"
 
 #include <sched.h>
 #include <time.h>
 
 namespace __memprof {
 namespace {
 using ::llvm::memprof::MemInfoBlock;
if (print_text) {
       MIBMap.ForEach(PrintCallback,
                      reinterpret_cast<void *>(flags()->print_terse));
       StackDepotPrintAll();
     } else {
       // Serialize the contents to a raw profile. Format documented in
       // memprof_rawprofile.h.
       char *Buffer = nullptr;
 
-      MemoryMappingLayout Layout(/*cache_enabled=*/true);
-      u64 BytesSerialized = SerializeToRawProfile(MIBMap, Layout, Buffer);
+      __sanitizer::ListOfModules List;
+      List.init();
+      ArrayRef<LoadedModule> Modules(List.begin(), List.end());
+      u64 BytesSerialized = SerializeToRawProfile(MIBMap, Modules, Buffer);
       CHECK(Buffer && BytesSerialized && "could not serialize to buffer");
       report_file.Write(Buffer, BytesSerialized);
     }
 
     allocator.ForceUnlock();
   }
 
   // Inserts any blocks which have been allocated but not yet deallocated.

compiler-rt/lib/memprof/memprof_rawprofile.h

 #ifndef MEMPROF_RAWPROFILE_H_
 #define MEMPROF_RAWPROFILE_H_
 
 #include "memprof_mibmap.h"
-#include "sanitizer_common/sanitizer_procmaps.h"
+#include "sanitizer_common/sanitizer_common.h"
 
 namespace __memprof {
 // Serialize the in-memory representation of the memprof profile to the raw
 // binary format. The format itself is documented memprof_rawprofile.cpp.
-u64 SerializeToRawProfile(MIBMapTy &BlockCache, MemoryMappingLayoutBase &Layout,
+u64 SerializeToRawProfile(MIBMapTy &BlockCache, ArrayRef<LoadedModule> Modules,
                           char *&Buffer);
 } // namespace __memprof
 
 #endif // MEMPROF_RAWPROFILE_H_

compiler-rt/lib/memprof/memprof_rawprofile.cpp

 void RecordStackId(const uptr Key, UNUSED LockedMemInfoBlock *const &MIB,
                    void *Arg) {
   // No need to touch the MIB value here since we are only recording the key.
   auto *StackIds = reinterpret_cast<Vector<u64> *>(Arg);
   StackIds->PushBack(Key);
 }
 } // namespace
 
-u64 SegmentSizeBytes(MemoryMappingLayoutBase &Layout) {
+u64 SegmentSizeBytes(ArrayRef<LoadedModule> Modules) {
   u64 NumSegmentsToRecord = 0;
-  MemoryMappedSegment segment;
-  for (Layout.Reset(); Layout.Next(&segment);)
-    if (segment.IsReadable() && segment.IsExecutable())
+  for (const auto &Module : Modules) {
+    for (const auto &Segment : Module.ranges()) {
+      if (Segment.executable)
         NumSegmentsToRecord++;
+    }
+  }
 
   return sizeof(u64) // A header which stores the number of records.
          + sizeof(SegmentEntry) * NumSegmentsToRecord;
 }
 
 // The segment section uses the following format:
 // ---------- Segment Info
 // Num Entries
 // ---------- Segment Entry
 // Start
 // End
 // Offset
-// BuildID 32B
+// UuidSize
+// Uuid 32B
 // ----------
 // ...
-void SerializeSegmentsToBuffer(MemoryMappingLayoutBase &Layout,
+void SerializeSegmentsToBuffer(ArrayRef<LoadedModule> Modules,
                                const u64 ExpectedNumBytes, char *&Buffer) {
   char *Ptr = Buffer;
   // Reserve space for the final count.
   Ptr += sizeof(u64);
 
   u64 NumSegmentsRecorded = 0;
-  MemoryMappedSegment segment;
 
-  for (Layout.Reset(); Layout.Next(&segment);) {
-    if (segment.IsReadable() && segment.IsExecutable()) {
-      // TODO: Record segment.uuid when it is implemented for Linux-Elf.
-      SegmentEntry Entry(segment.start, segment.end, segment.offset);
+  for (const auto &Module : Modules) {
+    for (const auto &Segment : Module.ranges()) {
+      if (Segment.executable) {
+        SegmentEntry Entry(Segment.beg, Segment.end, Module.base_address());
+        CHECK(Module.uuid_size() <= MEMPROF_BUILDID_MAX_SIZE);
+        Entry.BuildIdSize = Module.uuid_size();
+        memcpy(Entry.BuildId, Module.uuid(), Module.uuid_size());
         memcpy(Ptr, &Entry, sizeof(SegmentEntry));
         Ptr += sizeof(SegmentEntry);
         NumSegmentsRecorded++;
       }
     }
-
+  }
   // Store the number of segments we recorded in the space we reserved.
   *((u64 *)Buffer) = NumSegmentsRecorded;
   CHECK(ExpectedNumBytes >= static_cast<u64>(Ptr - Buffer) &&
         "Expected num bytes != actual bytes written");
 }
 
 u64 StackSizeBytes(const Vector<u64> &StackIds) {
   u64 NumBytesToWrite = sizeof(u64);

 // ---------- Stack Entry
 // Num Stacks
 // PC1
 // PC2
 // ...
 // ----------
 // Optional Padding Bytes
 // ...
-u64 SerializeToRawProfile(MIBMapTy &MIBMap, MemoryMappingLayoutBase &Layout,
+u64 SerializeToRawProfile(MIBMapTy &MIBMap, ArrayRef<LoadedModule> Modules,
                           char *&Buffer) {
   // Each section size is rounded up to 8b since the first entry in each section
   // is a u64 which holds the number of entries in the section by convention.
-  const u64 NumSegmentBytes = RoundUpTo(SegmentSizeBytes(Layout), 8);
+  const u64 NumSegmentBytes = RoundUpTo(SegmentSizeBytes(Modules), 8);
 
   Vector<u64> StackIds;
   MIBMap.ForEach(RecordStackId, reinterpret_cast<void *>(&StackIds));
   // The first 8b are for the total number of MIB records. Each MIB record is
   // preceded by a 8b stack id which is associated with stack frames in the next
   // section.
   const u64 NumMIBInfoBytes = RoundUpTo(
       sizeof(u64) + StackIds.Size() * (sizeof(u64) + sizeof(MemInfoBlock)), 8);
u64 SerializeToRawProfile(MIBMapTy &MIBMap, ArrayRef<LoadedModule> Modules,
   Header header{MEMPROF_RAW_MAGIC_64,
                 MEMPROF_RAW_VERSION,
                 static_cast<u64>(TotalSizeBytes),
                 sizeof(Header),
                 sizeof(Header) + NumSegmentBytes,
                 sizeof(Header) + NumSegmentBytes + NumMIBInfoBytes};
   Ptr = WriteBytes(header, Ptr);
 
-  SerializeSegmentsToBuffer(Layout, NumSegmentBytes, Ptr);
+  SerializeSegmentsToBuffer(Modules, NumSegmentBytes, Ptr);
   Ptr += NumSegmentBytes;
 
   SerializeMIBInfoToBuffer(MIBMap, StackIds, NumMIBInfoBytes, Ptr);
   Ptr += NumMIBInfoBytes;
 
   SerializeStackToBuffer(StackIds, NumStackBytes, Ptr);
 
   return TotalSizeBytes;
 }
 
 } // namespace __memprof

compiler-rt/lib/memprof/tests/rawprofile.cpp

 #include "sanitizer_common/sanitizer_stacktrace.h"
 #include "gmock/gmock.h"
 #include "gtest/gtest.h"
 
 namespace {
 
 using ::__memprof::MIBMapTy;
 using ::__memprof::SerializeToRawProfile;
-using ::__sanitizer::MemoryMappedSegment;
-using ::__sanitizer::MemoryMappingLayoutBase;
 using ::__sanitizer::StackDepotPut;
 using ::__sanitizer::StackTrace;
 using ::llvm::memprof::MemInfoBlock;
-using ::testing::_;
-using ::testing::Action;
-using ::testing::DoAll;
-using ::testing::Return;
-using ::testing::SetArgPointee;
-
-class MockMemoryMappingLayout final : public MemoryMappingLayoutBase {
-public:
-  MOCK_METHOD(bool, Next, (MemoryMappedSegment *), (override));
-  MOCK_METHOD(void, Reset, (), (override));
-};
 
 uint64_t PopulateFakeMap(const MemInfoBlock &FakeMIB, uint64_t StackPCBegin,
                          MIBMapTy &FakeMap) {
   constexpr int kSize = 5;
   uint64_t array[kSize];
   for (int i = 0; i < kSize; i++) {
     array[i] = StackPCBegin + i;
   }
template <class T = uint64_t> T Read(char *&Buffer) {
   assert(reinterpret_cast<size_t>(Buffer) % sizeof(T) == 0 &&
          "Unaligned read!");
   T t = *reinterpret_cast<T *>(Buffer);
   Buffer += sizeof(T);
   return t;
 }
 
 TEST(MemProf, Basic) {
-  MockMemoryMappingLayout Layout;
-  MemoryMappedSegment FakeSegment;
-  memset(&FakeSegment, 0, sizeof(FakeSegment));
-  FakeSegment.start = 0x10;
-  FakeSegment.end = 0x20;
-  FakeSegment.offset = 0x10;
-  uint8_t uuid[__sanitizer::kModuleUUIDSize] = {0xC, 0x0, 0xF, 0xF, 0xE, 0xE};
-  memcpy(FakeSegment.uuid, uuid, __sanitizer::kModuleUUIDSize);
-  FakeSegment.protection =
-      __sanitizer::kProtectionExecute | __sanitizer::kProtectionRead;
-
-  const Action<bool(MemoryMappedSegment *)> SetSegment =
-      DoAll(SetArgPointee<0>(FakeSegment), Return(true));
-  EXPECT_CALL(Layout, Next(_))
-      .WillOnce(SetSegment)
-      .WillOnce(Return(false))
-      .WillOnce(SetSegment)
-      .WillRepeatedly(Return(false));
-
-  EXPECT_CALL(Layout, Reset).Times(2);
+  __sanitizer::LoadedModule FakeModule;
+  FakeModule.addAddressRange(/*begin=*/0x10, /*end=*/0x20, /*executable=*/true,
+                             /*writable=*/false, /*name=*/"");
+  const char uuid[MEMPROF_BUILDID_MAX_SIZE] = {0xC, 0x0, 0xF, 0xF, 0xE, 0xE};
+  FakeModule.setUuid(uuid, MEMPROF_BUILDID_MAX_SIZE);
+  __sanitizer::ArrayRef<__sanitizer::LoadedModule> Modules(&FakeModule,
+                                                           (&FakeModule) + 1);
 
   MIBMapTy FakeMap;
   MemInfoBlock FakeMIB;
   // Since we want to override the constructor set vals to make it easier to
   // test.
   memset(&FakeMIB, 0, sizeof(MemInfoBlock));
   FakeMIB.AllocCount = 0x1;
   FakeMIB.TotalAccessCount = 0x2;
 
   uint64_t FakeIds[2];
   FakeIds[0] = PopulateFakeMap(FakeMIB, /*StackPCBegin=*/2, FakeMap);
   FakeIds[1] = PopulateFakeMap(FakeMIB, /*StackPCBegin=*/3, FakeMap);
 
   char *Ptr = nullptr;
-  uint64_t NumBytes = SerializeToRawProfile(FakeMap, Layout, Ptr);
+  uint64_t NumBytes = SerializeToRawProfile(FakeMap, Modules, Ptr);
   const char *Buffer = Ptr;
 
   ASSERT_GT(NumBytes, 0ULL);
   ASSERT_TRUE(Ptr);
 
   // Check the header.
   EXPECT_THAT(Read(Ptr), MEMPROF_RAW_MAGIC_64);
   EXPECT_THAT(Read(Ptr), MEMPROF_RAW_VERSION);
   const uint64_t TotalSize = Read(Ptr);
   const uint64_t SegmentOffset = Read(Ptr);
   const uint64_t MIBOffset = Read(Ptr);
   const uint64_t StackOffset = Read(Ptr);
 
   // ============= Check sizes and padding.
   EXPECT_EQ(TotalSize, NumBytes);
   EXPECT_EQ(TotalSize % 8, 0ULL);
 
   // Should be equal to the size of the raw profile header.
   EXPECT_EQ(SegmentOffset, 48ULL);
 
-  // We expect only 1 segment entry, 8b for the count and 56b for SegmentEntry
+  // We expect only 1 segment entry, 8b for the count and 64b for SegmentEntry
   // in memprof_rawprofile.cpp.
-  EXPECT_EQ(MIBOffset - SegmentOffset, 64ULL);
+  EXPECT_EQ(MIBOffset - SegmentOffset, 72ULL);
 
-  EXPECT_EQ(MIBOffset, 112ULL);
+  EXPECT_EQ(MIBOffset, 120ULL);
   // We expect 2 mib entry, 8b for the count and sizeof(uint64_t) +
   // sizeof(MemInfoBlock) contains stack id + MeminfoBlock.
   EXPECT_EQ(StackOffset - MIBOffset, 8 + 2 * (8 + sizeof(MemInfoBlock)));
 
-  EXPECT_EQ(StackOffset, 336ULL);
+  EXPECT_EQ(StackOffset, 408ULL);
   // We expect 2 stack entries, with 5 frames - 8b for total count,
   // 2 * (8b for id, 8b for frame count and 5*8b for fake frames).
   // Since this is the last section, there may be additional padding at the end
   // to make the total profile size 8b aligned.
   EXPECT_GE(TotalSize - StackOffset, 8ULL + 2 * (8 + 8 + 5 * 8));
 
   // ============= Check contents.
-  // The Uuid field is not yet populated on Linux-Elf by the sanitizer procmaps
-  // library, so we expect it to be filled with 0 for now.
-  unsigned char ExpectedSegmentBytes[64] = {
-      0x01, 0, 0, 0, 0, 0, 0, 0, // Number of entries
-      0x10, 0, 0, 0, 0, 0, 0, 0, // Start
-      0x20, 0, 0, 0, 0, 0, 0, 0, // End
-      0x10, 0, 0, 0, 0, 0, 0, 0, // Offset
-      0x0,                       // Uuid
+  unsigned char ExpectedSegmentBytes[72] = {
+      0x01, 0,   0,   0,   0,   0,  0, 0, // Number of entries
+      0x10, 0,   0,   0,   0,   0,  0, 0, // Start
+      0x20, 0,   0,   0,   0,   0,  0, 0, // End
+      0x0,  0,   0,   0,   0,   0,  0, 0, // Offset
+      0x20, 0,   0,   0,   0,   0,  0, 0, // UuidSize
+      0xC,  0x0, 0xF, 0xF, 0xE, 0xE       // Uuid
   };
-  EXPECT_EQ(memcmp(Buffer + SegmentOffset, ExpectedSegmentBytes, 64), 0);
+  EXPECT_EQ(memcmp(Buffer + SegmentOffset, ExpectedSegmentBytes, 72), 0);
 
   // Check that the number of entries is 2.
   EXPECT_EQ(*reinterpret_cast<const uint64_t *>(Buffer + MIBOffset), 2ULL);
   // Check that stack id is set.
   EXPECT_EQ(*reinterpret_cast<const uint64_t *>(Buffer + MIBOffset + 8),
             FakeIds[0]);
 
   // Only check a few fields of the first MemInfoBlock.
TEST(MemProf, Basic) {
   EXPECT_EQ(
       *reinterpret_cast<const uint64_t *>(Buffer + StackOffset + 8 + 6 * 8 + 8),
       FakeIds[1]);
 
   EXPECT_EQ(memcmp(Buffer + StackOffset + 16 + 6 * 8 + 8, ExpectedStackBytes[1],
                    sizeof(ExpectedStackBytes[1])),
             0);
 }
-
 } // namespace

llvm/include/llvm/ProfileData/MemProfData.inc

  *
  * This file has two identical copies. The primary copy lives in LLVM and
  * the other one sits in compiler-rt/include/profile directory. To make changes
  * in this file, first modify the primary copy and copy it over to compiler-rt.
  * Testing of any change in this file can start only after the two copies are
  * synced up.
  *
 \*===----------------------------------------------------------------------===*/
+#include <string.h>
 
 #ifdef _MSC_VER
 #define PACKED(...) __pragma(pack(push,1)) __VA_ARGS__ __pragma(pack(pop))
 #else
 #define PACKED(...) __VA_ARGS__ __attribute__((__packed__))
 #endif
 
 // A 64-bit magic number to uniquely identify the raw binary memprof profile file.
 #define MEMPROF_RAW_MAGIC_64                                                                        \
   ((uint64_t)255 << 56 | (uint64_t)'m' << 48 | (uint64_t)'p' << 40 | (uint64_t)'r' << 32 |          \
    (uint64_t)'o' << 24 | (uint64_t)'f' << 16 | (uint64_t)'r' << 8 | (uint64_t)129)
 
 // The version number of the raw binary format.
-#define MEMPROF_RAW_VERSION 2ULL
+#define MEMPROF_RAW_VERSION 3ULL
+
+#define MEMPROF_BUILDID_MAX_SIZE 32ULL
 
 namespace llvm {
 namespace memprof {
 // A struct describing the header used for the raw binary memprof profile format.
 PACKED(struct Header {
   uint64_t Magic;
   uint64_t Version;
   uint64_t TotalSize;
   uint64_t SegmentOffset;
   uint64_t MIBOffset;
   uint64_t StackOffset;
 });
 
-
 // A struct describing the information necessary to describe a /proc/maps
 // segment entry for a particular binary/library identified by its build id.
 PACKED(struct SegmentEntry {
   uint64_t Start;
   uint64_t End;
   uint64_t Offset;
-  // This field is unused until sanitizer procmaps support for build ids for
-  // Linux-Elf is implemented.
-  uint8_t BuildId[32] = {0};
+  uint64_t BuildIdSize;
+  uint8_t BuildId[MEMPROF_BUILDID_MAX_SIZE] = {0};
 
-  SegmentEntry(uint64_t S, uint64_t E, uint64_t O) :
-    Start(S), End(E), Offset(O) {}
+  // This constructor is only used in tests so don't set the BuildId.
+  SegmentEntry(uint64_t S, uint64_t E, uint64_t O)
+      : Start(S), End(E), Offset(O), BuildIdSize(0) {}
 
   SegmentEntry(const SegmentEntry& S) {
     Start = S.Start;
     End = S.End;
     Offset = S.Offset;
+    BuildIdSize = S.BuildIdSize;
+    memcpy(BuildId, S.BuildId, S.BuildIdSize);
   }
 
   SegmentEntry& operator=(const SegmentEntry& S) {
     Start = S.Start;
     End = S.End;
     Offset = S.Offset;
+    BuildIdSize = S.BuildIdSize;
+    memcpy(BuildId, S.BuildId, S.BuildIdSize);
     return *this;
   }
 
   bool operator==(const SegmentEntry& S) const {
-    return Start == S.Start &&
-           End == S.End &&
-           Offset == S.Offset;
+    return Start == S.Start && End == S.End && Offset == S.Offset &&
+           BuildIdSize == S.BuildIdSize &&
+           memcmp(BuildId, S.BuildId, S.BuildIdSize) == 0;
   }
 });
 
 // Packed struct definition for MSVC. We can't use the PACKED macro defined in
 // MemProfData.inc since it would mean we are embedding a directive (the
 // #include for MIBEntryDef) into the macros which is undefined behaviour.
 #ifdef _MSC_VER
 __pragma(pack(push,1))

llvm/lib/ProfileData/RawMemProfReader.cpp

bool isRuntimePath(const StringRef Path) {
   // This list should be updated in case new files with additional interceptors
   // are added to the memprof runtime.
   return Filename.equals("memprof_malloc_linux.cpp") ||
          Filename.equals("memprof_interceptors.cpp") ||
          Filename.equals("memprof_new_delete.cpp");
 }
 
 std::string getBuildIdString(const SegmentEntry &Entry) {
-  constexpr size_t Size = sizeof(Entry.BuildId) / sizeof(uint8_t);
-  constexpr uint8_t Zeros[Size] = {0};
   // If the build id is unset print a helpful string instead of all zeros.
-  if (memcmp(Entry.BuildId, Zeros, Size) == 0)
+  if (Entry.BuildIdSize == 0)
     return "<None>";
 
   std::string Str;
   raw_string_ostream OS(Str);
-  for (size_t I = 0; I < Size; I++) {
+  for (size_t I = 0; I < Entry.BuildIdSize; I++) {
     OS << format_hex_no_prefix(Entry.BuildId[I], 2);
   }
   return OS.str();
 }
 } // namespace
 
 Expected<std::unique_ptr<RawMemProfReader>>
 RawMemProfReader::create(const Twine &Path, const StringRef ProfiledBinary,

llvm/test/tools/llvm-profdata/Inputs/update_memprof_inputs.sh

 DEFAULT_MEMPROF_FLAGS="-fuse-ld=lld -Wl,--no-rosegment -gmlt -fdebug-info-for-profiling -fmemory-profile -mno-omit-leaf-frame-pointer -fno-omit-frame-pointer -fno-optimize-sibling-calls -m64 -Wl,-build-id -no-pie"
 
 # Map each test to their source and any additional flags separated by ; 
 declare -A INPUTS
 INPUTS["basic"]="BASIC"
 INPUTS["inline"]="INLINE"
 INPUTS["multi"]="MULTI"
 INPUTS["pic"]="BASIC;-pie"
+INPUTS["buildid"]="BASIC;-Wl,-build-id=sha1"
 
 for name in "${!INPUTS[@]}"; do
   IFS=";" read -r src flags <<< "${INPUTS[$name]}"
   echo "${!src}" > ${OUTDIR}/${name}.c
   ${CLANG} ${DEFAULT_MEMPROF_FLAGS} ${flags} ${OUTDIR}/${name}.c -o ${OUTDIR}/${name}.memprofexe
   env MEMPROF_OPTIONS=log_path=stdout ${OUTDIR}/${name}.memprofexe > ${OUTDIR}/${name}.memprofraw
   rm ${OUTDIR}/${name}.c
 done

llvm/test/tools/llvm-profdata/memprof-basic.test

 REQUIRES: x86_64-linux
 
 To update the inputs used below run Inputs/update_memprof_inputs.sh /path/to/updated/clang
 RUN: llvm-profdata show --memory %p/Inputs/basic.memprofraw --profiled-binary %p/Inputs/basic.memprofexe -o - | FileCheck %s
 
 We expect 2 MIB entries, 1 each for the malloc calls in the program. Any
 additional allocations which do not originate from the main binary are pruned.
 
 CHECK:  MemprofProfile:
 CHECK-NEXT:   Summary:
-CHECK-NEXT:     Version: 2
+CHECK-NEXT:     Version: 3
 CHECK-NEXT:     NumSegments: {{[0-9]+}}
 CHECK-NEXT:     NumMibInfo: 2
 CHECK-NEXT:     NumAllocFunctions: 1
 CHECK-NEXT:     NumStackOffsets: 2
 CHECK-NEXT:   Segments:
 CHECK-NEXT:   -
-CHECK-NEXT:     BuildId: <None>
-CHECK-NEXT:     Start: 0x{{[0-9]+}}
-CHECK-NEXT:     End: 0x{{[0-9]+}}
-CHECK-NEXT:     Offset: 0x{{[0-9]+}}
+CHECK-NEXT:     BuildId: {{[[:xdigit:]]+}}
+CHECK-NEXT:     Start: 0x{{[[:xdigit:]]+}}
+CHECK-NEXT:     End: 0x{{[[:xdigit:]]+}}
+CHECK-NEXT:     Offset: 0x{{[[:xdigit:]]+}}
 CHECK-NEXT:   -
 
 CHECK:   Records:
 CHECK-NEXT:   -
 CHECK-NEXT:     FunctionGUID: {{[0-9]+}}
 CHECK-NEXT:     AllocSites:
 CHECK-NEXT:     -
 CHECK-NEXT:       Callstack:

llvm/test/tools/llvm-profdata/memprof-buildid.test

+REQUIRES: x86_64-linux
+
+To update the inputs used below run Inputs/update_memprof_inputs.sh /path/to/updated/clang
+RUN: llvm-readelf --notes %p/Inputs/buildid.memprofexe > %t1.txt
+RUN: llvm-profdata show --memory %p/Inputs/buildid.memprofraw --profiled-binary %p/Inputs/buildid.memprofexe -o -  > %t2.txt
+RUN: cat %t1.txt %t2.txt | FileCheck %s
+
+COM: First extract the id from the llvm-readelf output.
+CHECK: Build ID: [[ID:[[:xdigit:]]+]]
+
+COM: Then match it with the profdata output.
+CHECK: BuildId: {{.*}}[[ID]]

llvm/test/tools/llvm-profdata/memprof-inline.test

 REQUIRES: x86_64-linux
 
 To update the inputs used below run Inputs/update_memprof_inputs.sh /path/to/updated/clang
 RUN: llvm-profdata show --memory %p/Inputs/inline.memprofraw --profiled-binary %p/Inputs/inline.memprofexe | FileCheck %s
 
 CHECK:  MemprofProfile:
 CHECK-NEXT:  Summary:
-CHECK-NEXT:    Version: 2
+CHECK-NEXT:    Version: 3
 CHECK-NEXT:    NumSegments: {{[0-9]+}}
 CHECK-NEXT:    NumMibInfo: 2
 CHECK-NEXT:    NumAllocFunctions: 2
 CHECK-NEXT:    NumStackOffsets: 1
 CHECK-NEXT:  Segments:
 CHECK-NEXT:  -
-CHECK-NEXT:    BuildId: <None>
-CHECK-NEXT:    Start: 0x{{[0-9]+}}
-CHECK-NEXT:    End: 0x{{[0-9]+}}
-CHECK-NEXT:    Offset: 0x{{[0-9]+}}
+CHECK-NEXT:    BuildId: {{[[:xdigit:]]+}}
+CHECK-NEXT:    Start: 0x{{[[:xdigit:]]+}}
+CHECK-NEXT:    End: 0x{{[[:xdigit:]]+}}
+CHECK-NEXT:    Offset: 0x{{[[:xdigit:]]+}}
 CHECK-NEXT:  -
 
 CHECK:  Records:
 CHECK-NEXT:  -
 CHECK-NEXT:    FunctionGUID: 15505678318020221912
 CHECK-NEXT:    AllocSites:
 CHECK-NEXT:    -
 CHECK-NEXT:      Callstack:

llvm/test/tools/llvm-profdata/memprof-multi.test

 REQUIRES: x86_64-linux
 
 To update the inputs used below run Inputs/update_memprof_inputs.sh /path/to/updated/clang
 RUN: llvm-profdata show --memory %p/Inputs/multi.memprofraw --profiled-binary %p/Inputs/multi.memprofexe -o - | FileCheck %s
 
 We expect 2 MIB entries, 1 each for the malloc calls in the program.
 
 CHECK:  MemprofProfile:
 CHECK-NEXT:  Summary:
-CHECK-NEXT:    Version: 2
+CHECK-NEXT:    Version: 3
 CHECK-NEXT:    NumSegments: {{[0-9]+}}
 CHECK-NEXT:    NumMibInfo: 2
 CHECK-NEXT:    NumAllocFunctions: 1
 CHECK-NEXT:    NumStackOffsets: 2
 
 CHECK:        SymbolName: main
 CHECK-NEXT:     LineOffset: 1
 CHECK-NEXT:     Column: 21
 
 CHECK:        SymbolName: main
 CHECK-NEXT:     LineOffset: 5
 CHECK-NEXT:     Column: 15