BreakpadRecords: Address post-commit feedback

[labath]

Summary

This addresses the issues raised in D56844. It removes the accessors from the
breakpad record structures by making the fields public. Also, I refactor the
UUID parsing code to remove hard-coded constants.

[labath]

rebase

[clayborg]

Let me know what you think of the UUID code and if you want to include that in this patch

source/Plugins/ObjectFile/Breakpad/BreakpadRecords.cpp

67

rename to "consume_hex_integer" or an extra parameter for the base instead of hard coding to 16?

111–113

For Apple platforms Breakpad actually incorrectly byte swaps the first 32 bits and the next two 16 bit values. I have a patch for this, but since this is moving, it would be great to get that fix in here. Also, many linux breakpad file have the UUID field present but set to all zeroes which is bad as UUID parsing code will cause multiple modules to claim a UUID of all zeroes is valid and causes all modules with such a UUID to just use the first one it finds. The code I have in my unsubmitted patch is:

if (pdb70_uuid->Age == 0) {
  bool all_zeroes = true;
  for (size_t i=0; all_zeroes && i<sizeof(pdb70_uuid->Uuid); ++i)
    all_zeroes = pdb70_uuid->Uuid[i] == 0;
  // Many times UUIDs are not filled in at all, so avoid claiming that
  // all such libraries have a valid UUID that is all zeroes.
  if (all_zeroes)
    return UUID();
  if (arch.GetTriple().getVendor() == llvm::Triple::Apple) {
    // Breakpad incorrectly byte swaps the first 32 bit and next 2 16 bit
    // values in the UUID field. Undo this so we can match things up
    // with our symbol files
    uint8_t apple_uuid[16];
    // Byte swap the first 32 bits
    apple_uuid[0] = pdb70_uuid->Uuid[3];
    apple_uuid[1] = pdb70_uuid->Uuid[2];
    apple_uuid[2] = pdb70_uuid->Uuid[1];
    apple_uuid[3] = pdb70_uuid->Uuid[0];
    // Byte swap the next 16 bit value
    apple_uuid[4] = pdb70_uuid->Uuid[5];
    apple_uuid[5] = pdb70_uuid->Uuid[4];
    // Byte swap the next 16 bit value
    apple_uuid[6] = pdb70_uuid->Uuid[7];
    apple_uuid[7] = pdb70_uuid->Uuid[6];
    for (size_t i=8; i<sizeof(pdb70_uuid->Uuid); ++i)
      apple_uuid[i] = pdb70_uuid->Uuid[i];
    return UUID::fromData(apple_uuid, sizeof(apple_uuid));
  } else
    return UUID::fromData(pdb70_uuid->Uuid, sizeof(pdb70_uuid->Uuid));
} else
  return UUID::fromData(pdb70_uuid, sizeof(*pdb70_uuid));

[labath]

source/Plugins/ObjectFile/Breakpad/BreakpadRecords.cpp

67

I'll rename the function before submitting.

111–113

The byte swapping is already handled in this code (that's why it's this complicated). My impression was that the swapping is not apple-specific, but rather depends where you read the UUID from (MODULE record has it swapped, INFO record doesn't). Though that may depend on how we chose to interpret the raw bytes in other UUID sources (object files, pdb files, ...).

As for the all-zero case, that should be easily fixable, by changing fromData to fromOptionalData, but I'm surprised that this is necessary, as I was under the impression that breakpad invent's its own UUIDs in case the original object file doesn't have them. (This would actually be the better case, as otherwise we'd have to figure out how to tell the fictional and non-fictional uuids apart.) @lemo: Can you shed any light on this?

[clayborg]

source/Plugins/ObjectFile/Breakpad/BreakpadRecords.cpp

87–98

This is OK as long as the UUIDs for ELF don't fall into this category. I am able to match up UUIDs for ELF just fine for breakpad files for Android.

[labath]

source/Plugins/ObjectFile/Breakpad/BreakpadRecords.cpp

87–98

Normally on linux you should always have the INFO record, which will have the unmangled UUID, and which we will give preference to if it is available.

If for some reason we don't find an INFO record, then we will use the version from the MODULE record, which we will manually unmangle. But that should be the right thing to do as it matches what the breakpad generator does. (You can see this by looking at a file which has both of these records -- they will differ in that the first one will be mangled and will have an extra zero at the end).

[lemo]

Looks good - the all-zero UUID case is a bit unfortunate but it seems we have to handle it (like Greg suggested)

source/Plugins/ObjectFile/Breakpad/BreakpadRecords.cpp

69

0; ?

source/Plugins/ObjectFile/Breakpad/BreakpadRecords.h

48

coding-convention-wise: should these definitions use struct instead of class?

59–61

const method qualifier?

72

ditto

[labath]

After some internal discussion, it seems that the situation with the all-zero UUIDs is as follows:

• breakpad symbol files do not attach a special meaning to a zero UUID - if a file does not have a build-id, the dump_syms tool will use a hash of the first page of the text section (or something equally silly)
• minidump files may treat the missing build-id by replacing it with zeroes depending on the tool used to produce the minidump: breakpad doesn't do that (it does the same hash as above), crashpad does.

So it seems like there is nothing to do here. Maybe the UUID reading code in ProcessMinidump needs revising though.

source/Plugins/ObjectFile/Breakpad/BreakpadRecords.cpp

69

That is not necessary, as to_integer initializes it. Perhaps more importantly, not initializing this allows tools like msan and valgrind to actually detect the cases when you end up using an uninitialized value.

source/Plugins/ObjectFile/Breakpad/BreakpadRecords.h

48

I don't think we have a strict rule about this case, but personally, when something starts using inheritance, I tend to think of it as a class.

59–61

This is a free function, not a method. :)

source/Plugins/ObjectFile/Breakpad/BreakpadRecords.h

 };
 
 llvm::StringRef toString(Record::Kind K);
 inline llvm::raw_ostream &operator<<(llvm::raw_ostream &OS, Record::Kind K) {
   OS << toString(K);
   return OS;
 }
 
 class ModuleRecord : public Record {
 public:
   static llvm::Optional<ModuleRecord> parse(llvm::StringRef Line);
   ModuleRecord(llvm::Triple::OSType OS, llvm::Triple::ArchType Arch, UUID ID)
       : Record(Module), OS(OS), Arch(Arch), ID(std::move(ID)) {}
 
-  llvm::Triple::OSType getOS() const { return OS; }
-  llvm::Triple::ArchType getArch() const { return Arch; }
-  const UUID &getID() const { return ID; }
-
-private:
   llvm::Triple::OSType OS;
   llvm::Triple::ArchType Arch;
   UUID ID;
 };
 
-bool operator==(const ModuleRecord &L, const ModuleRecord &R);
+bool operator==(const ModuleRecord &L, const ModuleRecord &R) {
+  return L.OS == R.OS && L.Arch == R.Arch && L.ID == R.ID;
+}
 llvm::raw_ostream &operator<<(llvm::raw_ostream &OS, const ModuleRecord &R);
 
 class InfoRecord : public Record {
 public:
   static llvm::Optional<InfoRecord> parse(llvm::StringRef Line);
   InfoRecord(UUID ID) : Record(Info), ID(std::move(ID)) {}
 
-  const UUID &getID() const { return ID; }
-
-private:
   UUID ID;
 };
 
 inline bool operator==(const InfoRecord &L, const InfoRecord &R) {
-  return L.getID() == R.getID();
+  return L.ID == R.ID;
 }
 llvm::raw_ostream &operator<<(llvm::raw_ostream &OS, const InfoRecord &R);
 
 class FuncRecord : public Record {
 public:
   static llvm::Optional<FuncRecord> parse(llvm::StringRef Line);
   FuncRecord(bool Multiple, lldb::addr_t Address, lldb::addr_t Size,
              lldb::addr_t ParamSize, llvm::StringRef Name)
       : Record(Module), Multiple(Multiple), Address(Address), Size(Size),
         ParamSize(ParamSize), Name(Name) {}
 
-  bool getMultiple() const { return Multiple; }
-  lldb::addr_t getAddress() const { return Address; }
-  lldb::addr_t getSize() const { return Size; }
-  lldb::addr_t getParamSize() const { return ParamSize; }
-  llvm::StringRef getName() const { return Name; }
-
-private:
   bool Multiple;
   lldb::addr_t Address;
   lldb::addr_t Size;
   lldb::addr_t ParamSize;
   llvm::StringRef Name;
 };
 
 bool operator==(const FuncRecord &L, const FuncRecord &R);
 llvm::raw_ostream &operator<<(llvm::raw_ostream &OS, const FuncRecord &R);
 
 class PublicRecord : public Record {
 public:
   static llvm::Optional<PublicRecord> parse(llvm::StringRef Line);
   PublicRecord(bool Multiple, lldb::addr_t Address, lldb::addr_t ParamSize,
                llvm::StringRef Name)
       : Record(Module), Multiple(Multiple), Address(Address),
         ParamSize(ParamSize), Name(Name) {}
 
-  bool getMultiple() const { return Multiple; }
-  lldb::addr_t getAddress() const { return Address; }
-  lldb::addr_t getParamSize() const { return ParamSize; }
-  llvm::StringRef getName() const { return Name; }
-
-private:
   bool Multiple;
   lldb::addr_t Address;
   lldb::addr_t ParamSize;
   llvm::StringRef Name;
 };
 
 bool operator==(const PublicRecord &L, const PublicRecord &R);
 llvm::raw_ostream &operator<<(llvm::raw_ostream &OS, const PublicRecord &R);
 
 } // namespace breakpad
 } // namespace lldb_private
 
 #endif // LLDB_PLUGINS_OBJECTFILE_BREAKPAD_BREAKPADRECORDS_H

source/Plugins/ObjectFile/Breakpad/BreakpadRecords.cpp

return llvm::StringSwitch<Triple::ArchType>(str)
       .Case("s390", Triple::systemz)
       .Case("sparc", Triple::sparc)
       .Case("sparcv9", Triple::sparcv9)
       .Case("x86", Triple::x86)
       .Case("x86_64", Triple::x86_64)
       .Default(Triple::UnknownArch);
 }
 
-static llvm::StringRef consume_front(llvm::StringRef &str, size_t n) {
-  llvm::StringRef result = str.take_front(n);
-  str = str.drop_front(n);
-  return result;
+/// Return the number of hex digits needed to encode an (POD) object of a given
+/// type.
+template <typename T> static constexpr size_t hex_digits() {
+  return 2 * sizeof(T);
+}
+
+/// Consume the right number of digits from the input StringRef and convert it
+/// to the endian-specific integer N. Return true on success.
+template <typename T> static bool consume_hex_integer(llvm::StringRef &str, T &N) {
+  llvm::StringRef chunk = str.take_front(hex_digits<T>());
+  uintmax_t t;
+  if (!to_integer(chunk, t, 16))
+    return false;
+  N = t;
+  str = str.drop_front(hex_digits<T>());
+  return true;
 }
 
 static UUID parseModuleId(llvm::Triple::OSType os, llvm::StringRef str) {
-  struct uuid_data {
-    llvm::support::ulittle32_t uuid1;
-    llvm::support::ulittle16_t uuid2[2];
-    uint8_t uuid3[8];
+  struct data_t {
+    struct uuid_t {
+      llvm::support::ulittle32_t part1;
+      llvm::support::ulittle16_t part2[2];
+      uint8_t part3[8];
+    } uuid;
     llvm::support::ulittle32_t age;
   } data;
   static_assert(sizeof(data) == 20, "");
   // The textual module id encoding should be between 33 and 40 bytes long,
   // depending on the size of the age field, which is of variable length.
   // The first three chunks of the id are encoded in big endian, so we need to
   // byte-swap those.
-  if (str.size() < 33 || str.size() > 40)
+  if (str.size() <= hex_digits<data_t::uuid_t>() ||
+      str.size() > hex_digits<data_t>())
     return UUID();
-  uint32_t t;
-  if (to_integer(consume_front(str, 8), t, 16))
-    data.uuid1 = t;
-  else
+  if (!consume_hex_integer(str, data.uuid.part1))
     return UUID();
-  for (int i = 0; i < 2; ++i) {
-    if (to_integer(consume_front(str, 4), t, 16))
-      data.uuid2[i] = t;
-    else
+  for (auto &t : data.uuid.part2) {
+    if (!consume_hex_integer(str, t))
       return UUID();
   }
-  for (int i = 0; i < 8; ++i) {
-    if (!to_integer(consume_front(str, 2), data.uuid3[i], 16))
+  for (auto &t : data.uuid.part3) {
+    if (!consume_hex_integer(str, t))
       return UUID();
   }
-  if (to_integer(str, t, 16))
-    data.age = t;
-  else
+  uint32_t age;
+  if (!to_integer(str, age, 16))
     return UUID();
+  data.age = age;
 
   // On non-windows, the age field should always be zero, so we don't include to
   // match the native uuid format of these platforms.
-  return UUID::fromData(&data, os == llvm::Triple::Win32 ? 20 : 16);
+  return UUID::fromData(&data, os == llvm::Triple::Win32 ? sizeof(data)
+                                                         : sizeof(data.uuid));
 }
 
 Record::Kind Record::classify(llvm::StringRef Line) {
   Token Tok = toToken(getToken(Line).first);
   switch (Tok) {
   case Token::Module:
     return Record::Module;
   case Token::Info:
     return Record::Info;
llvm::Optional<ModuleRecord> ModuleRecord::parse(llvm::StringRef Line) {
   std::tie(Str, Line) = getToken(Line);
   UUID ID = parseModuleId(OS, Str);
   if (!ID)
     return llvm::None;
 
   return ModuleRecord(OS, Arch, std::move(ID));
 }
 
-bool breakpad::operator==(const ModuleRecord &L, const ModuleRecord &R) {
-  return L.getOS() == R.getOS() && L.getArch() == R.getArch() &&
-         L.getID() == R.getID();
-}
 llvm::raw_ostream &breakpad::operator<<(llvm::raw_ostream &OS,
                                         const ModuleRecord &R) {
-  return OS << "MODULE " << llvm::Triple::getOSTypeName(R.getOS()) << " "
-            << llvm::Triple::getArchTypeName(R.getArch()) << " "
-            << R.getID().GetAsString();
+  return OS << "MODULE " << llvm::Triple::getOSTypeName(R.OS) << " "
+            << llvm::Triple::getArchTypeName(R.Arch) << " "
+            << R.ID.GetAsString();
 }
 
 llvm::Optional<InfoRecord> InfoRecord::parse(llvm::StringRef Line) {
   // INFO CODE_ID 554889E55DC3CCCCCCCCCCCCCCCCCCCC [a.exe]
   llvm::StringRef Str;
   std::tie(Str, Line) = getToken(Line);
   if (toToken(Str) != Token::Info)
     return llvm::None;
if (Line.trim().empty()) {
     if (Str.empty() || ID.SetFromStringRef(Str, Str.size() / 2) != Str.size())
       return llvm::None;
   }
   return InfoRecord(std::move(ID));
 }
 
 llvm::raw_ostream &breakpad::operator<<(llvm::raw_ostream &OS,
                                         const InfoRecord &R) {
-  return OS << "INFO CODE_ID " << R.getID().GetAsString();
+  return OS << "INFO CODE_ID " << R.ID.GetAsString();
 }
 
 static bool parsePublicOrFunc(llvm::StringRef Line, bool &Multiple,
                               lldb::addr_t &Address, lldb::addr_t *Size,
                               lldb::addr_t &ParamSize, llvm::StringRef &Name) {
   // PUBLIC [m] address param_size name
   // or
   // FUNC [m] address size param_size name
llvm::Optional<FuncRecord> FuncRecord::parse(llvm::StringRef Line) {
 
   if (parsePublicOrFunc(Line, Multiple, Address, &Size, ParamSize, Name))
     return FuncRecord(Multiple, Address, Size, ParamSize, Name);
 
   return llvm::None;
 }
 
 bool breakpad::operator==(const FuncRecord &L, const FuncRecord &R) {
-  return L.getMultiple() == R.getMultiple() &&
-         L.getAddress() == R.getAddress() && L.getSize() == R.getSize() &&
-         L.getParamSize() == R.getParamSize() && L.getName() == R.getName();
+  return L.Multiple == R.Multiple && L.Address == R.Address &&
+         L.Size == R.Size && L.ParamSize == R.ParamSize && L.Name == R.Name;
 }
 llvm::raw_ostream &breakpad::operator<<(llvm::raw_ostream &OS,
                                         const FuncRecord &R) {
   return OS << llvm::formatv("FUNC {0}{1:x-} {2:x-} {3:x-} {4}",
-                             R.getMultiple() ? "m " : "", R.getAddress(),
-                             R.getSize(), R.getParamSize(), R.getName());
+                             R.Multiple ? "m " : "", R.Address, R.Size,
+                             R.ParamSize, R.Name);
 }
 
 llvm::Optional<PublicRecord> PublicRecord::parse(llvm::StringRef Line) {
   bool Multiple;
   lldb::addr_t Address, ParamSize;
   llvm::StringRef Name;
 
   if (parsePublicOrFunc(Line, Multiple, Address, nullptr, ParamSize, Name))
     return PublicRecord(Multiple, Address, ParamSize, Name);
 
   return llvm::None;
 }
 
 bool breakpad::operator==(const PublicRecord &L, const PublicRecord &R) {
-  return L.getMultiple() == R.getMultiple() &&
-         L.getAddress() == R.getAddress() &&
-         L.getParamSize() == R.getParamSize() && L.getName() == R.getName();
+  return L.Multiple == R.Multiple && L.Address == R.Address &&
+         L.ParamSize == R.ParamSize && L.Name == R.Name;
 }
 llvm::raw_ostream &breakpad::operator<<(llvm::raw_ostream &OS,
                                         const PublicRecord &R) {
   return OS << llvm::formatv("PUBLIC {0}{1:x-} {2:x-} {3}",
-                             R.getMultiple() ? "m " : "", R.getAddress(),
-                             R.getParamSize(), R.getName());
+                             R.Multiple ? "m " : "", R.Address, R.ParamSize,
+                             R.Name);
 }
 
 llvm::StringRef breakpad::toString(Record::Kind K) {
   switch (K) {
   case Record::Module:
     return "MODULE";
   case Record::Info:
     return "INFO";

source/Plugins/ObjectFile/Breakpad/ObjectFileBreakpad.cpp

 llvm::Optional<Header> Header::parse(llvm::StringRef text) {
   llvm::StringRef line;
   std::tie(line, text) = text.split('\n');
   auto Module = ModuleRecord::parse(line);
   if (!Module)
     return llvm::None;
 
   llvm::Triple triple;
-  triple.setArch(Module->getArch());
-  triple.setOS(Module->getOS());
+  triple.setArch(Module->Arch);
+  triple.setOS(Module->OS);
 
   std::tie(line, text) = text.split('\n');
 
   auto Info = InfoRecord::parse(line);
-  UUID uuid = Info && Info->getID() ? Info->getID() : Module->getID();
+  UUID uuid = Info && Info->ID ? Info->ID : Module->ID;
   return Header{ArchSpec(triple), std::move(uuid)};
 }
 
 void ObjectFileBreakpad::Initialize() {
   PluginManager::RegisterPlugin(GetPluginNameStatic(),
                                 GetPluginDescriptionStatic(), CreateInstance,
                                 CreateMemoryInstance, GetModuleSpecifications);
 }

source/Plugins/SymbolFile/Breakpad/SymbolFileBreakpad.cpp

symbols.try_emplace(
         /*is_trampoline*/ false, /*is_artificial*/ false,
         AddressRange(section_sp, address - section_sp->GetFileAddress(),
                      size.getValueOr(0)),
         size.hasValue(), /*contains_linker_annotations*/ false, /*flags*/ 0);
   };
 
   for (llvm::StringRef line : lines(*m_obj_file, Record::Func)) {
     if (auto record = FuncRecord::parse(line))
-      add_symbol(record->getAddress(), record->getSize(), record->getName());
+      add_symbol(record->Address, record->Size, record->Name);
   }
 
   for (llvm::StringRef line : lines(*m_obj_file, Record::Public)) {
     if (auto record = PublicRecord::parse(line))
-      add_symbol(record->getAddress(), llvm::None, record->getName());
+      add_symbol(record->Address, llvm::None, record->Name);
     else
       LLDB_LOG(log, "Failed to parse: {0}. Skipping record.", line);
   }
 
   for (auto &KV : symbols)
     symtab.AddSymbol(std::move(KV.second));
   symtab.CalculateSymbolSizes();
 }