[Symbolizer] Add Build ID flag to llvm-symbolizer.

[mysterymath]

Summary

This adds a --build-id=<hex build ID> flag to llvm-symbolizer. If --obj
is unspecified, this will attempt to look up the provided build ID using
whatever mechanisms are available to the Symbolizer (typically,
debuginfod). The semantics are then as if the found binary were given
using the --obj flag.

[mysterymath]

Update docs.

[mysterymath]

Consistent caps for "build ID."

[mysterymath]

More caps.

[phosek]

llvm/docs/CommandGuide/llvm-symbolizer.rst

189

I wonder whether it wouldn't be less error prone to diagnose the case when user uses both --obj and --build-id as an error?

llvm/tools/llvm-symbolizer/llvm-symbolizer.cpp

282

This is only invoked from one location so I'd consider inlining it.

[jhenderson]

llvm/docs/CommandGuide/llvm-symbolizer.rst

189

+1: I think it should be an error if a user specifies both, unless you have a good use-case for it.

llvm/lib/DebugInfo/Symbolize/Symbolize.cpp

395–396

Don't think you want this blank line.

llvm/tools/llvm-symbolizer/llvm-symbolizer.cpp

385–388

This is untested.

Also, the error format is not consistent with the LLVM coding standards or what llvm-symbolizer prints when there's a missing file, as far as I can see.

[mysterymath]

• Refactored build ID lookup functionality into LLVMSymbolizer. This makes build IDs a first class means to specify modules. In particular, this produces consistent error behavior between paths and build IDs. This will also make it straightforward to extend the stdin syntax to support alternating build IDs and paths.

• Made --build-id and --obj mutually exclusive.

• Added tests.

llvm/tools/llvm-symbolizer/llvm-symbolizer.cpp

385–388

There were some considerable semantic divergences between --obj and --build-id; the former would return EXIT_SUCCESS and keep going, producing only one error, while the latter would immediately exit with a nonzero status.

The current approach also doesn't extend well to handling multiple build IDs specified on a line-by-line basis, which is a near-term TODO.

Accordingly, I've reworked this functionality to take place inside the Symbolizer, rather than in the wrapping tool. This makes BuildIDs a first-class mechanism that can be used to specify modules; it thus intrinsically has similar error behavior to the other ways of specifying modules. This will also make it easy to build syntax into the stdin mechanism to switch out the module specifier type.

[jhenderson]

Thanks for the rework. I've added some basic comments, but I think @phosek (or someone else) would be better to continue this review beyond these points.

llvm/docs/CommandGuide/llvm-symbolizer.rst

189

I'd delete "family" since "-e" etc are just aliases, and therefore it's implied.

240–241

llvm/include/llvm/DebugInfo/Symbolize/Symbolize.h

61–67

Not necessarily something that you need to do right now, but nonetheless worth thinking about: it's clear by adding a third variation of each of these methods that some sort of refactoring would be appropriate to simplify extensibility of this code. In this case, I'd probably make each of the different input types (object, module name, build ID) a separate class, with the relevant methods attached. symbolizeCode would then just become something like:

template <typename T>
Expected<DILineInfo> symbolizeCode(const T &Input, object::SectionedAddress ModuleOffset) {
  return Input.symbolizeCode(ModuleOffset, ...);
}

and so on. Adding a new input kind wouldn't then require modifying this class again, keeping this class stable, and just adding new classes for each input kind.

llvm/lib/DebugInfo/Symbolize/Symbolize.cpp

389

[mysterymath]

Address nits.

llvm/include/llvm/DebugInfo/Symbolize/Symbolize.h

61–67

Yeah, this occurred to me too. I'll take a stab at this afterwards and see if I can design a clean abstraction.

[jhenderson]

LGTM, but please wait for @phosek.

[phosek]

llvm/lib/DebugInfo/Symbolize/Symbolize.cpp

439

IIUC this slightly changes the semantics. Currently, even if we failed to download a file (for example because of the intermittent network error), we would try again on then next symbolization request. With this change, we would cache the error result (that is, an empty string) after the first attempt and never retry. Maybe it would be better to drop this line to preserve the current semantics?

llvm/tools/llvm-symbolizer/llvm-symbolizer.cpp

368

It might be more efficient to check if both --obj and --build-id was specified before parsing --build-id and allocating memory for build ID, but it only makes a difference for the error path so it's not particularly important.

[mysterymath]

Retry build ID lookups each time, but only report failures once.

llvm/lib/DebugInfo/Symbolize/Symbolize.cpp

439

This is slightly tricky: we should retry the network request if it failed previously, but we probably shouldn't spam the console with repeated error messages for the build ID, since that differs from the way errors work for paths (one per path).

I've tried to keep the error behavior while still retrying the lookup under the hood. If a retry succeeds, the data becomes available for all future symbolization request; otherwise, the error is silenced.

[mysterymath]

Always call through to getOrFindDebugBinary, even if there were a previous
error.

[mysterymath]

Make the missing build ID test treat stdout and stderr separately.

Fix the bugs this uncovered.

[phosek]

LGTM

llvm/lib/DebugInfo/Symbolize/Symbolize.cpp

439

Personally I'd prefer getting an error on every unsuccessful attempt; that may be more spammy but avoids hiding potentially useful information from the user. If the errors go to stderr, user can always redirect it to /dev/null if needed. Either solution is fine with me though.

llvm/tools/llvm-symbolizer/llvm-symbolizer.cpp

367–368

I'd use hasArg.

[mysterymath]

• Sync from main.
• Use hasArg.
• Report buildID lookup failures every time.

[mysterymath]

llvm/tools/llvm-symbolizer/llvm-symbolizer.cpp

367–368

There's a use to having a value that means "flag unset": you can unset flags via concatenation, which can help when using utilities in scripts. E.g., an environment variable is set up with a ton of flags outside your control, and you want to remove exactly one of them. This can require either some tricky string manipulation or just appending "--obj= ", depending on the semantics of '--obj'.

That being said, I did a quick random sampling of flags in LLVM, and it's wildly variable whether or not a given flag offers this property. At least some of that is probably due to GCC backcompat concerns, but it's definitely not a property a user can rely on across the board, so doesn't seem to be much point in making more flags behave that way.

[MaskRay]

llvm/lib/DebugInfo/Symbolize/Symbolize.cpp

532

There was a -Wunused-variable warning in -DLLVM_ENABLE_ASSERTIONS=off builds. Fixed by f8701a30f648dd07d472c108f90fa675c1198c46

llvm/docs/CommandGuide/llvm-symbolizer.rst

.. option:: --adjust-vma <offset>
   This can be used to perform lookups as if the object were relocated by the
   offset.
 
 .. option:: --basenames, -s
 
   Print just the file's name without any directories, instead of the
   absolute path.
 
+.. option:: --build-id
+
+  Look up the object using the given build ID, specified as a hexadecimal
+  string. Mutually exclusive with :option:`--obj`.
+
 .. _llvm-symbolizer-opt-C:
 
 .. option:: --demangle, -C
 
   Print demangled function names, if the names are mangled (e.g. the mangled
   name `_Z3bazv` becomes `baz()`, whilst the non-mangled name `foz` is printed
   as is). Defaults to true.
 

 
 .. option:: --no-demangle
 
   Don't print demangled function names.
 
 .. option:: --obj <path>, --exe, -e
 
   Path to object file to be symbolized. If ``-`` is specified, read the object
-  directly from the standard input stream.
+  directly from the standard input stream. Mutually exclusive with
+  :option:`--build-id`.
 
 .. _llvm-symbolizer-opt-output-style:
 
 .. option:: --output-style <LLVM|GNU|JSON>
 
   Specify the preferred output style. Defaults to ``LLVM``. When the output
   style is set to ``GNU``, the tool follows the style of GNU's **addr2line**.
   The differences from the ``LLVM`` style are:

llvm/include/llvm/DebugInfo/Symbolize/Symbolize.h

 //===- Symbolize.h ----------------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // Header for LLVM symbolization library.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_DEBUGINFO_SYMBOLIZE_SYMBOLIZE_H
 #define LLVM_DEBUGINFO_SYMBOLIZE_SYMBOLIZE_H
 
+#include "llvm/ADT/StringMap.h"
 #include "llvm/DebugInfo/Symbolize/SymbolizableModule.h"
 #include "llvm/Object/Binary.h"
 #include "llvm/Object/ELFObjectFile.h"
 #include "llvm/Object/ObjectFile.h"
 #include "llvm/Support/Error.h"
 #include <algorithm>
 #include <cstdint>
 #include <map>
public:
   };
 
   LLVMSymbolizer() = default;
   LLVMSymbolizer(const Options &Opts) : Opts(Opts) {}
 
   ~LLVMSymbolizer() { flush(); }
 
   // Overloads accepting ObjectFile does not support COFF currently
   Expected<DILineInfo> symbolizeCode(const ObjectFile &Obj,
                                      object::SectionedAddress ModuleOffset);
   Expected<DILineInfo> symbolizeCode(const std::string &ModuleName,
                                      object::SectionedAddress ModuleOffset);
+  Expected<DILineInfo> symbolizeCode(ArrayRef<uint8_t> BuildID,
+                                     object::SectionedAddress ModuleOffset);
   Expected<DIInliningInfo>
   symbolizeInlinedCode(const ObjectFile &Obj,
                        object::SectionedAddress ModuleOffset);
   Expected<DIInliningInfo>
   symbolizeInlinedCode(const std::string &ModuleName,
                        object::SectionedAddress ModuleOffset);
+  Expected<DIInliningInfo>
+  symbolizeInlinedCode(ArrayRef<uint8_t> BuildID,
+                       object::SectionedAddress ModuleOffset);
 
   Expected<DIGlobal> symbolizeData(const ObjectFile &Obj,
                                    object::SectionedAddress ModuleOffset);
   Expected<DIGlobal> symbolizeData(const std::string &ModuleName,
                                    object::SectionedAddress ModuleOffset);
+  Expected<DIGlobal> symbolizeData(ArrayRef<uint8_t> BuildID,
+                                   object::SectionedAddress ModuleOffset);
   Expected<std::vector<DILocal>>
   symbolizeFrame(const ObjectFile &Obj, object::SectionedAddress ModuleOffset);
   Expected<std::vector<DILocal>>
   symbolizeFrame(const std::string &ModuleName,
                  object::SectionedAddress ModuleOffset);
+  Expected<std::vector<DILocal>>
+  symbolizeFrame(ArrayRef<uint8_t> BuildID,
+                 object::SectionedAddress ModuleOffset);
   void flush();
 
   static std::string
   DemangleName(const std::string &Name,
                const SymbolizableModule *DbiModuleDescriptor);
 
 private:
+  bool getOrFindDebugBinary(const ArrayRef<uint8_t> BuildID,
+                            std::string &Result);
+
   // Bundles together object file with code/data and object file with
   // corresponding debug info. These objects can be the same.
   using ObjectPair = std::pair<const ObjectFile *, const ObjectFile *>;
 
   template <typename T>
   Expected<DILineInfo>
   symbolizeCodeCommon(const T &ModuleSpecifier,
                       object::SectionedAddress ModuleOffset);
private:
 
   /// Returns a SymbolizableModule or an error if loading debug info failed.
   /// Only one attempt is made to load a module, and errors during loading are
   /// only reported once. Subsequent calls to get module info for a module that
   /// failed to load will return nullptr.
   Expected<SymbolizableModule *>
   getOrCreateModuleInfo(const std::string &ModuleName);
   Expected<SymbolizableModule *> getOrCreateModuleInfo(const ObjectFile &Obj);
+  Expected<SymbolizableModule *>
+  getOrCreateModuleInfo(ArrayRef<uint8_t> BuildID);
 
   Expected<SymbolizableModule *>
   createModuleInfo(const ObjectFile *Obj, std::unique_ptr<DIContext> Context,
                    StringRef ModuleName);
 
   ObjectFile *lookUpDsymFile(const std::string &Path,
                              const MachOObjectFile *ExeObj,
                              const std::string &ArchName);
private:
   /// Return a pointer to object file at specified path, for a specified
   /// architecture (e.g. if path refers to a Mach-O universal binary, only one
   /// object file from it will be returned).
   Expected<ObjectFile *> getOrCreateObject(const std::string &Path,
                                            const std::string &ArchName);
 
   std::map<std::string, std::unique_ptr<SymbolizableModule>, std::less<>>
       Modules;
+  StringMap<std::string> BuildIDPaths;
 
   /// Contains cached results of getOrCreateObjectPair().
   std::map<std::pair<std::string, std::string>, ObjectPair>
       ObjectPairForPathArch;
 
   /// Contains parsed binary for each path, or parsing error.
   std::map<std::string, OwningBinary<Binary>> BinaryForPath;
 

llvm/lib/DebugInfo/Symbolize/Symbolize.cpp

 }
 
 Expected<DILineInfo>
 LLVMSymbolizer::symbolizeCode(const std::string &ModuleName,
                               object::SectionedAddress ModuleOffset) {
   return symbolizeCodeCommon(ModuleName, ModuleOffset);
 }
 
+Expected<DILineInfo>
+LLVMSymbolizer::symbolizeCode(ArrayRef<uint8_t> BuildID,
+                              object::SectionedAddress ModuleOffset) {
+  return symbolizeCodeCommon(BuildID, ModuleOffset);
+}
+
 template <typename T>
 Expected<DIInliningInfo> LLVMSymbolizer::symbolizeInlinedCodeCommon(
     const T &ModuleSpecifier, object::SectionedAddress ModuleOffset) {
   auto InfoOrErr = getOrCreateModuleInfo(ModuleSpecifier);
   if (!InfoOrErr)
     return InfoOrErr.takeError();
 
   SymbolizableModule *Info = *InfoOrErr;

 }
 
 Expected<DIInliningInfo>
 LLVMSymbolizer::symbolizeInlinedCode(const std::string &ModuleName,
                                      object::SectionedAddress ModuleOffset) {
   return symbolizeInlinedCodeCommon(ModuleName, ModuleOffset);
 }
 
+Expected<DIInliningInfo>
+LLVMSymbolizer::symbolizeInlinedCode(ArrayRef<uint8_t> BuildID,
+                                     object::SectionedAddress ModuleOffset) {
+  return symbolizeInlinedCodeCommon(BuildID, ModuleOffset);
+}
+
 template <typename T>
 Expected<DIGlobal>
 LLVMSymbolizer::symbolizeDataCommon(const T &ModuleSpecifier,
                                     object::SectionedAddress ModuleOffset) {
 
   auto InfoOrErr = getOrCreateModuleInfo(ModuleSpecifier);
   if (!InfoOrErr)
     return InfoOrErr.takeError();

 }
 
 Expected<DIGlobal>
 LLVMSymbolizer::symbolizeData(const std::string &ModuleName,
                               object::SectionedAddress ModuleOffset) {
   return symbolizeDataCommon(ModuleName, ModuleOffset);
 }
 
+Expected<DIGlobal>
+LLVMSymbolizer::symbolizeData(ArrayRef<uint8_t> BuildID,
+                              object::SectionedAddress ModuleOffset) {
+  return symbolizeDataCommon(BuildID, ModuleOffset);
+}
+
 template <typename T>
 Expected<std::vector<DILocal>>
 LLVMSymbolizer::symbolizeFrameCommon(const T &ModuleSpecifier,
                                      object::SectionedAddress ModuleOffset) {
   auto InfoOrErr = getOrCreateModuleInfo(ModuleSpecifier);
   if (!InfoOrErr)
     return InfoOrErr.takeError();
 

 }
 
 Expected<std::vector<DILocal>>
 LLVMSymbolizer::symbolizeFrame(const std::string &ModuleName,
                                object::SectionedAddress ModuleOffset) {
   return symbolizeFrameCommon(ModuleName, ModuleOffset);
 }
 
+Expected<std::vector<DILocal>>
+LLVMSymbolizer::symbolizeFrame(ArrayRef<uint8_t> BuildID,
+                               object::SectionedAddress ModuleOffset) {
+  return symbolizeFrameCommon(BuildID, ModuleOffset);
+}
+
 void LLVMSymbolizer::flush() {
   ObjectForUBPathAndArch.clear();
   BinaryForPath.clear();
   ObjectPairForPathArch.clear();
   Modules.clear();
+  BuildIDPaths.clear();
 }
 
 namespace {
-
 // For Path="/path/to/foo" and Basename="foo" assume that debug info is in
 // /path/to/foo.dSYM/Contents/Resources/DWARF/foo.
 // For Path="/path/to/bar.dSYM" and Basename="foo" assume that debug info is in
 // /path/to/bar.dSYM/Contents/Resources/DWARF/foo.
 std::string getDarwinDWARFResourceForPath(const std::string &Path,
                                           const std::string &Basename) {
   SmallString<16> ResourceName = StringRef(Path);
   if (sys::path::extension(Path) != ".dSYM") {
Optional<ArrayRef<uint8_t>> getBuildID(const ELFObjectFileBase *Obj) {
   else if (auto *O = dyn_cast<ELFObjectFile<ELF64LE>>(Obj))
     BuildID = getBuildID(O->getELFFile());
   else if (auto *O = dyn_cast<ELFObjectFile<ELF64BE>>(Obj))
     BuildID = getBuildID(O->getELFFile());
   else
     llvm_unreachable("unsupported file format");
   return BuildID;
 }
+} // end anonymous namespace
+
+static StringRef getBuildIDStr(ArrayRef<uint8_t> BuildID) {
+  return StringRef(reinterpret_cast<const char *>(BuildID.data()),
+                   BuildID.size());
+}
+
+bool LLVMSymbolizer::getOrFindDebugBinary(const ArrayRef<uint8_t> BuildID,
+                                          std::string &Result) {
+  StringRef BuildIDStr = getBuildIDStr(BuildID);
+  auto I = BuildIDPaths.find(BuildIDStr);
+  if (I != BuildIDPaths.end()) {
+    // If an error was recorded.
+    if (I->second.empty())
+      return false;
+    Result = I->second;
+    return true;
+  }
 
-bool findDebugBinary(const std::vector<std::string> &DebugFileDirectory,
-                     const ArrayRef<uint8_t> BuildID, std::string &Result) {
   auto getDebugPath = [&](StringRef Directory) {
     SmallString<128> Path{Directory};
     sys::path::append(Path, ".build-id",
                       llvm::toHex(BuildID[0], /*LowerCase=*/true),
                       llvm::toHex(BuildID.slice(1), /*LowerCase=*/true));
     Path += ".debug";
     return Path;
   };
-  if (DebugFileDirectory.empty()) {
+  auto recordPath = [&](StringRef Path) {
+    Result = Path.str();
+    auto InsertResult = BuildIDPaths.insert({BuildIDStr, Result});
+    assert(InsertResult.second);
+  };
+
+  if (Opts.DebugFileDirectory.empty()) {
     SmallString<128> Path = getDebugPath(
 #if defined(__NetBSD__)
         // Try /usr/libdata/debug/.build-id/../...
         "/usr/libdata/debug"
 #else
         // Try /usr/lib/debug/.build-id/../...
         "/usr/lib/debug"
 #endif
     );
     if (llvm::sys::fs::exists(Path)) {
-      Result = std::string(Path.str());
+      recordPath(Path);
       return true;
     }
   } else {
-    for (const auto &Directory : DebugFileDirectory) {
+    for (const auto &Directory : Opts.DebugFileDirectory) {
       // Try <debug-file-directory>/.build-id/../...
       SmallString<128> Path = getDebugPath(Directory);
       if (llvm::sys::fs::exists(Path)) {
-        Result = std::string(Path.str());
+        recordPath(Path);
         return true;
       }
     }
   }
   // Try debuginfod client cache and known servers.
   Expected<std::string> PathOrErr = getCachedOrDownloadDebuginfo(BuildID);
   if (!PathOrErr) {
     consumeError(PathOrErr.takeError());
+    // Record that an error occurred.
+    recordPath("");
     return false;
   }
-  Result = *PathOrErr;
+  recordPath(*PathOrErr);
   return true;
 }
 
-} // end anonymous namespace
-
 ObjectFile *LLVMSymbolizer::lookUpDsymFile(const std::string &ExePath,
                                            const MachOObjectFile *MachExeObj,
                                            const std::string &ArchName) {
   // On Darwin we may find DWARF in separate object file in
   // resource directory.
   std::vector<std::string> DsymPaths;
   StringRef Filename = sys::path::filename(ExePath);
   DsymPaths.push_back(
ObjectFile *LLVMSymbolizer::lookUpBuildIDObject(const std::string &Path,
                                                 const ELFObjectFileBase *Obj,
                                                 const std::string &ArchName) {
   auto BuildID = getBuildID(Obj);
   if (!BuildID)
     return nullptr;
   if (BuildID->size() < 2)
     return nullptr;
   std::string DebugBinaryPath;
-  if (!findDebugBinary(Opts.DebugFileDirectory, *BuildID, DebugBinaryPath))
+  if (!getOrFindDebugBinary(*BuildID, DebugBinaryPath))
     return nullptr;
   auto DbgObjOrErr = getOrCreateObject(DebugBinaryPath, ArchName);
   if (!DbgObjOrErr) {
     consumeError(DbgObjOrErr.takeError());
     return nullptr;
   }
   return DbgObjOrErr.get();
 }
 
 Expected<LLVMSymbolizer::ObjectPair>
 LLVMSymbolizer::getOrCreateObjectPair(const std::string &Path,
                                       const std::string &ArchName) {
   auto I = ObjectPairForPathArch.find(std::make_pair(Path, ArchName));
   if (I != ObjectPairForPathArch.end())
     return I->second;
 
   auto ObjOrErr = getOrCreateObject(Path, ArchName);
   if (!ObjOrErr) {
     ObjectPairForPathArch.emplace(std::make_pair(Path, ArchName),
                                   ObjectPair(nullptr, nullptr));
     return ObjOrErr.takeError();
   }
 
   ObjectFile *Obj = ObjOrErr.get();
   assert(Obj != nullptr);
   ObjectFile *DbgObj = nullptr;
 
   if (auto MachObj = dyn_cast<const MachOObjectFile>(Obj))
     DbgObj = lookUpDsymFile(Path, MachObj, ArchName);
   else if (auto ELFObj = dyn_cast<const ELFObjectFileBase>(Obj))
     DbgObj = lookUpBuildIDObject(Path, ELFObj, ArchName);
   if (!DbgObj)
     DbgObj = lookUpDebuglinkObject(Path, Obj, ArchName);
LLVMSymbolizer::getOrCreateModuleInfo(const ObjectFile &Obj) {
   if (I != Modules.end())
     return I->second.get();
 
   std::unique_ptr<DIContext> Context = DWARFContext::create(Obj);
   // FIXME: handle COFF object with PDB info to use PDBContext
   return createModuleInfo(&Obj, std::move(Context), ObjName);
 }
 
+Expected<SymbolizableModule *>
+LLVMSymbolizer::getOrCreateModuleInfo(ArrayRef<uint8_t> BuildID) {
+  StringRef BuildIDStr = getBuildIDStr(BuildID);
+  auto I = BuildIDPaths.find(BuildIDStr);
+  if (I != BuildIDPaths.end() && I->second.empty()) {
+    // An error has already been reported.
+    return nullptr;
+  }
+
+  std::string Path;
+  if (!getOrFindDebugBinary(BuildID, Path)) {
+    // An error has not yet been reported.
+    return createStringError(errc::no_such_file_or_directory,
+                             Twine("could not find build ID '") +
+                                 toHex(BuildID) + "'");
+  }
+  return getOrCreateModuleInfo(Path);
+}
+
 namespace {
 
 // Undo these various manglings for Win32 extern "C" functions:
 // cdecl       - _foo
 // stdcall     - _foo@12
 // fastcall    - @foo@12
 // vectorcall  - foo@@12
 // These are all different linkage names for 'foo'.

llvm/test/tools/llvm-symbolizer/debuginfod-bad-build-id.test

+RUN: not llvm-symbolizer --build-id=not_a_hex_string 0x1234 2>&1 | FileCheck %s
+
+CHECK: --build-id=: expected a build ID, but got 'not_a_hex_string'

llvm/test/tools/llvm-symbolizer/debuginfod-build-id-and-obj.test

+RUN: not llvm-symbolizer --build-id=abc --obj=bad 0x1234 2>&1 | FileCheck %s
+
+CHECK: error: cannot specify both --build-id and --obj

llvm/test/tools/llvm-symbolizer/debuginfod-missing-build-id.test

+RUN: llvm-symbolizer --build-id=abad 0x1234 0x5678 2>&1 | FileCheck %s
+
+CHECK: LLVMSymbolizer: error reading file: could not find build ID 'ABAD'
+CHECK: ??
+CHECK: ??:0:0
+CHECK-NOT: LLVMSymbolizer
+CHECK: ??
+CHECK: ??:0:0

llvm/test/tools/llvm-symbolizer/debuginfod.test

 RUN: llvm-objcopy --keep-section=.debug_info %p/Inputs/addr.exe \
 RUN:   %t/llvmcache-9800707741016212219
 
 # The symbolizer should call the debuginfod client library, which finds the
 # debuginfo placed in the cache, enabling symbolization of the address.
 RUN: env DEBUGINFOD_CACHE_PATH=%t llvm-symbolizer \
 RUN:   --obj=%t/addr.exe 0x40054d | FileCheck %s --check-prefix=FOUND
 FOUND: {{[/\]+}}tmp{{[/\]+}}x.c:14:0
+
+# This should also work if the build ID is provided.
+RUN: env DEBUGINFOD_CACHE_PATH=%t llvm-symbolizer \
+RUN:   --build-id=127da749021c1fc1a58cba734a1f542cbe2b7ce4 0x40054d | \
+RUN:   FileCheck %s --check-prefix=FOUND

llvm/tools/llvm-symbolizer/Opts.td

 def grp_mach_o : OptionGroup<"kind">,
                  HelpText<"llvm-symbolizer Mach-O Specific Options">;
 
 def addresses : F<"addresses", "Show address before line information">;
 defm adjust_vma
     : Eq<"adjust-vma", "Add specified offset to object file addresses">,
       MetaVarName<"<offset>">;
 def basenames : Flag<["--"], "basenames">, HelpText<"Strip directory names from paths">;
+defm build_id : Eq<"build-id", "Build ID used to look up the object file">;
 defm debug_file_directory : Eq<"debug-file-directory", "Path to directory where to look for debug files">, MetaVarName<"<dir>">;
 defm default_arch
     : Eq<"default-arch", "Default architecture (for multi-arch objects)">,
       Group<grp_mach_o>;
 defm demangle : B<"demangle", "Demangle function names", "Don't demangle function names">;
 def functions : F<"functions", "Print function name for a given address">;
 def functions_EQ : Joined<["--"], "functions=">, HelpText<"Print function name for a given address">, Values<"none,short,linkage">;
 def help : F<"help", "Display this help">;

llvm/tools/llvm-symbolizer/llvm-symbolizer.cpp

 // tuples (module name, module offset) to code locations (function name,
 // file, line number, column number). It is targeted for compiler-rt tools
 // (especially AddressSanitizer and ThreadSanitizer) that can use it
 // to symbolize stack traces in their error reports.
 //
 //===----------------------------------------------------------------------===//
 
 #include "Opts.inc"
+#include "llvm/ADT/StringExtras.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/Config/config.h"
 #include "llvm/DebugInfo/Symbolize/DIPrinter.h"
 #include "llvm/DebugInfo/Symbolize/Symbolize.h"
 #include "llvm/Debuginfod/HTTPClient.h"
 #include "llvm/Option/Arg.h"
 #include "llvm/Option/ArgList.h"
 #include "llvm/Option/Option.h"

 enum class OutputStyle { LLVM, GNU, JSON };
 
 enum class Command {
   Code,
   Data,
   Frame,
 };
 
-static bool parseCommand(StringRef BinaryName, bool IsAddr2Line,
-                         StringRef InputString, Command &Cmd,
+static bool parseCommand(StringRef BinaryName, ArrayRef<uint8_t> BuildID,
+                         bool IsAddr2Line, StringRef InputString, Command &Cmd,
                          std::string &ModuleName, uint64_t &ModuleOffset) {
   const char kDelimiters[] = " \n\r";
   ModuleName = "";
   if (InputString.consume_front("CODE ")) {
     Cmd = Command::Code;
   } else if (InputString.consume_front("DATA ")) {
     Cmd = Command::Data;
   } else if (InputString.consume_front("FRAME ")) {
     Cmd = Command::Frame;
   } else {
     // If no cmd, assume it's CODE.
     Cmd = Command::Code;
   }
   const char *Pos = InputString.data();
   // Skip delimiters and parse input filename (if needed).
-  if (BinaryName.empty()) {
+  if (BinaryName.empty() && BuildID.empty()) {
     Pos += strspn(Pos, kDelimiters);
     if (*Pos == '"' || *Pos == '\'') {
       char Quote = *Pos;
       Pos++;
       const char *End = strchr(Pos, Quote);
       if (!End)
         return false;
       ModuleName = std::string(Pos, End - Pos);
static bool parseCommand(StringRef BinaryName, ArrayRef<uint8_t> BuildID,
   StringRef Offset(Pos, OffsetLength);
   // GNU addr2line assumes the offset is hexadecimal and allows a redundant
   // "0x" or "0X" prefix; do the same for compatibility.
   if (IsAddr2Line)
     Offset.consume_front("0x") || Offset.consume_front("0X");
   return !Offset.getAsInteger(IsAddr2Line ? 16 : 0, ModuleOffset);
 }
 
-static void symbolizeInput(const opt::InputArgList &Args, uint64_t AdjustVMA,
-                           bool IsAddr2Line, OutputStyle Style,
-                           StringRef InputString, LLVMSymbolizer &Symbolizer,
-                           DIPrinter &Printer) {
-  Command Cmd;
-  std::string ModuleName;
-  uint64_t Offset = 0;
-  if (!parseCommand(Args.getLastArgValue(OPT_obj_EQ), IsAddr2Line,
-                    StringRef(InputString), Cmd, ModuleName, Offset)) {
-    Printer.printInvalidCommand({ModuleName, None}, InputString);
-    return;
-  }
-
+template <typename T>
+void executeCommand(StringRef ModuleName, const T &ModuleSpec, Command Cmd,
+                    uint64_t Offset, uint64_t AdjustVMA, bool ShouldInline,
+                    OutputStyle Style, LLVMSymbolizer &Symbolizer,
   uint64_t AdjustedOffset = Offset - AdjustVMA;
+  object::SectionedAddress Address = {AdjustedOffset,
+                                      object::SectionedAddress::UndefSection};
   if (Cmd == Command::Data) {
-    Expected<DIGlobal> ResOrErr = Symbolizer.symbolizeData(
-        ModuleName, {AdjustedOffset, object::SectionedAddress::UndefSection});
+    Expected<DIGlobal> ResOrErr = Symbolizer.symbolizeData(ModuleSpec, Address);
     print({ModuleName, Offset}, ResOrErr, Printer);
   } else if (Cmd == Command::Frame) {
-    Expected<std::vector<DILocal>> ResOrErr = Symbolizer.symbolizeFrame(
-        ModuleName, {AdjustedOffset, object::SectionedAddress::UndefSection});
+    Expected<std::vector<DILocal>> ResOrErr =
+        Symbolizer.symbolizeFrame(ModuleSpec, Address);
     print({ModuleName, Offset}, ResOrErr, Printer);
-  } else if (Args.hasFlag(OPT_inlines, OPT_no_inlines, !IsAddr2Line)) {
-    Expected<DIInliningInfo> ResOrErr = Symbolizer.symbolizeInlinedCode(
-        ModuleName, {AdjustedOffset, object::SectionedAddress::UndefSection});
+  } else if (ShouldInline) {
+    Expected<DIInliningInfo> ResOrErr =
+        Symbolizer.symbolizeInlinedCode(ModuleSpec, Address);
     print({ModuleName, Offset}, ResOrErr, Printer);
   } else if (Style == OutputStyle::GNU) {
     // With PrintFunctions == FunctionNameKind::LinkageName (default)
     // and UseSymbolTable == true (also default), Symbolizer.symbolizeCode()
     // may override the name of an inlined function with the name of the topmost
     // caller function in the inlining chain. This contradicts the existing
     // behavior of addr2line. Symbolizer.symbolizeInlinedCode() overrides only
     // the topmost function, which suits our needs better.
-    Expected<DIInliningInfo> ResOrErr = Symbolizer.symbolizeInlinedCode(
-        ModuleName, {AdjustedOffset, object::SectionedAddress::UndefSection});
+    Expected<DIInliningInfo> ResOrErr =
+        Symbolizer.symbolizeInlinedCode(ModuleSpec, Address);
     Expected<DILineInfo> Res0OrErr =
         !ResOrErr
             ? Expected<DILineInfo>(ResOrErr.takeError())
             : ((ResOrErr->getNumberOfFrames() == 0) ? DILineInfo()
                                                     : ResOrErr->getFrame(0));
     print({ModuleName, Offset}, Res0OrErr, Printer);
   } else {
-    Expected<DILineInfo> ResOrErr = Symbolizer.symbolizeCode(
-        ModuleName, {AdjustedOffset, object::SectionedAddress::UndefSection});
+    Expected<DILineInfo> ResOrErr =
+        Symbolizer.symbolizeCode(ModuleSpec, Address);
     print({ModuleName, Offset}, ResOrErr, Printer);
   }
 }
 
+static void symbolizeInput(const opt::InputArgList &Args,
+                           ArrayRef<uint8_t> BuildID, uint64_t AdjustVMA,
+                           bool IsAddr2Line, OutputStyle Style,
+                           StringRef InputString, LLVMSymbolizer &Symbolizer,
+                           DIPrinter &Printer) {
+  Command Cmd;
+  std::string ModuleName;
+  uint64_t Offset = 0;
+  if (!parseCommand(Args.getLastArgValue(OPT_obj_EQ), BuildID, IsAddr2Line,
+                    StringRef(InputString), Cmd, ModuleName, Offset)) {
+    Printer.printInvalidCommand({ModuleName, None}, InputString);
+    return;
+  }
+  bool ShouldInline = Args.hasFlag(OPT_inlines, OPT_no_inlines, !IsAddr2Line);
+  if (!BuildID.empty()) {
+    assert(ModuleName.empty());
+    std::string BuildIDStr = toHex(BuildID);
+    executeCommand(BuildIDStr, BuildID, Cmd, Offset, AdjustVMA, ShouldInline,
+                   Style, Symbolizer, Printer);
+  } else {
+    executeCommand(ModuleName, ModuleName, Cmd, Offset, AdjustVMA, ShouldInline,
+                   Style, Symbolizer, Printer);
+  }
+}
+
 static void printHelp(StringRef ToolName, const SymbolizerOptTable &Tbl,
                       raw_ostream &OS) {
   const char HelpText[] = " [options] addresses...";
   Tbl.printHelp(OS, (ToolName + HelpText).str().c_str(),
                 ToolName.str().c_str());
   // TODO Replace this with OptTable API once it adds extrahelp support.
   OS << "\nPass @FILE as argument to read options from FILE.\n";
 }
static FunctionNameKind decideHowToPrintFunctions(const opt::InputArgList &Args,
   if (const opt::Arg *A = Args.getLastArg(OPT_functions_EQ))
     return StringSwitch<FunctionNameKind>(A->getValue())
         .Case("none", FunctionNameKind::None)
         .Case("short", FunctionNameKind::ShortName)
         .Default(FunctionNameKind::LinkageName);
   return IsAddr2Line ? FunctionNameKind::None : FunctionNameKind::LinkageName;
 }
 
+SmallVector<uint8_t> parseBuildIDArg(const opt::InputArgList &Args, int ID) {
+  if (const opt::Arg *A = Args.getLastArg(ID)) {
+    StringRef V(A->getValue());
+    std::string Bytes;
+    if (!tryGetFromHex(V, Bytes)) {
+      errs() << A->getSpelling() + ": expected a build ID, but got '" + V +
+                    "'\n";
+      exit(1);
+    }
+    ArrayRef<uint8_t> BuildID(reinterpret_cast<const uint8_t *>(Bytes.data()),
+                              Bytes.size());
+    return SmallVector<uint8_t>(BuildID.begin(), BuildID.end());
+  }
+  return {};
+}
+
 int main(int argc, char **argv) {
   InitLLVM X(argc, argv);
   // The HTTPClient must be initialized for use by the debuginfod client.
   HTTPClient::initialize();
   sys::InitializeCOMRAII COM(sys::COMThreadingMode::MultiThreaded);
 
   bool IsAddr2Line = sys::path::stem(argv[0]).contains("addr2line");
   BumpPtrAllocator A;
if (const opt::Arg *A = Args.getLastArg(OPT_output_style_EQ)) {
     if (strcmp(A->getValue(), "GNU") == 0)
       Style = OutputStyle::GNU;
     else if (strcmp(A->getValue(), "JSON") == 0)
       Style = OutputStyle::JSON;
     else
       Style = OutputStyle::LLVM;
   }
 
+  SmallVector<uint8_t> BuildID = parseBuildIDArg(Args, OPT_build_id_EQ);
+  if (!BuildID.empty() && !Args.getLastArgValue(OPT_obj_EQ).empty()) {
+    errs() << "error: cannot specify both --build-id and --obj\n";
+    return EXIT_FAILURE;
+  }
+
   LLVMSymbolizer Symbolizer(Opts);
   std::unique_ptr<DIPrinter> Printer;
   if (Style == OutputStyle::GNU)
     Printer = std::make_unique<GNUPrinter>(outs(), errs(), Config);
   else if (Style == OutputStyle::JSON)
     Printer = std::make_unique<JSONPrinter>(outs(), Config);
   else
     Printer = std::make_unique<LLVMPrinter>(outs(), errs(), Config);
 
   std::vector<std::string> InputAddresses = Args.getAllArgValues(OPT_INPUT);
   if (InputAddresses.empty()) {
     const int kMaxInputStringLength = 1024;
     char InputString[kMaxInputStringLength];
 
     while (fgets(InputString, sizeof(InputString), stdin)) {
       // Strip newline characters.
       std::string StrippedInputString(InputString);
       llvm::erase_if(StrippedInputString,
                      [](char c) { return c == '\r' || c == '\n'; });
-      symbolizeInput(Args, AdjustVMA, IsAddr2Line, Style, StrippedInputString,
-                     Symbolizer, *Printer);
+      symbolizeInput(Args, BuildID, AdjustVMA, IsAddr2Line, Style,
+                     StrippedInputString, Symbolizer, *Printer);
       outs().flush();
     }
   } else {
     Printer->listBegin();
     for (StringRef Address : InputAddresses)
-      symbolizeInput(Args, AdjustVMA, IsAddr2Line, Style, Address, Symbolizer,
-                     *Printer);
+      symbolizeInput(Args, BuildID, AdjustVMA, IsAddr2Line, Style, Address,
+                     Symbolizer, *Printer);
     Printer->listEnd();
   }
 
   return 0;
 }