[llvm-cov] Optionally fail on missing binary ID

[mysterymath]

Summary

This adds the --check-binary-id flag that makes sure that an object file
is available for every binary ID mentioned in the given profile. This
should help make the tool more robust in CI environments where it's
expected that coverage mappings should be available for every object
contributing to the profile.

[gulfem]

llvm/lib/ProfileData/Coverage/CoverageMapping.cpp

435

Just a cosmetic suggestion: I am not a big fan of reading a 4-level nested structure.
I was thinking that you can just use if (PathOpt) instead and swap then and else parts. You can also get rid of continue.
This is just a nit-pick, and feel free to land it without that if you prefer.

[mysterymath]

Rearrange if.

compiler-rt/test/profile/Linux/binary-id-lookup.c

 // REQUIRES: linux
 // RUN: split-file %s %t
 // RUN: %clang_profgen -Wl,--build-id=0x12345678 -fcoverage-mapping -O2 -shared %t/foo.c -o %t/libfoo.so
 // RUN: %clang_profgen -Wl,--build-id=0xabcd1234 -fcoverage-mapping -O2 %t/main.c -L%t -lfoo -o %t.main
 // RUN: rm -rf %t.profdir
 // RUN: env LLVM_PROFILE_FILE=%t.profdir/default_%m.profraw LD_LIBRARY_PATH=%t %run %t.main
 // RUN: mkdir -p %t/.build-id/12 %t/.build-id/ab
 // RUN: cp %t/libfoo.so %t/.build-id/12/345678.debug
 // RUN: cp %t.main %t/.build-id/ab/cd1234.debug
 // RUN: llvm-profdata merge -o %t.profdata %t.profdir/default_*.profraw
+
 // RUN: llvm-cov show -instr-profile %t.profdata -debug-file-directory %t | FileCheck %s
 // RUN: llvm-cov show -instr-profile %t.profdata %t/libfoo.so -sources %t/foo.c -object %t.main | FileCheck %s --check-prefix=FOO-ONLY
 // RUN: llvm-cov show -instr-profile %t.profdata -debug-file-directory %t -sources %t/foo.c | FileCheck %s --check-prefix=FOO-ONLY
 // RUN: llvm-cov show -instr-profile %t.profdata -debug-file-directory %t %t/libfoo.so -sources %t/foo.c | FileCheck %s --check-prefix=FOO-ONLY
+
+// RUN: rm %t/.build-id/ab/cd1234.debug
+// RUN: llvm-cov show -instr-profile %t.profdata -debug-file-directory %t %t.main | FileCheck %s
+// RUN: llvm-cov show -instr-profile %t.profdata -debug-file-directory %t | FileCheck %s --check-prefix=FOO-ONLY
+// RUN: not llvm-cov show -instr-profile %t.profdata -debug-file-directory %t --check-binary-ids 2>&1 | FileCheck %s --check-prefix=MISSING-BINARY-ID -DFILENAME=%t.profdata
+
 // RUN: echo "bad" > %t/.build-id/ab/cd1234.debug
 // RUN: llvm-cov show -instr-profile %t.profdata -debug-file-directory %t %t.main | FileCheck %s
+
 // RUN: not llvm-cov show -instr-profile %t.profdata -debug-file-directory %t/empty 2>&1 | FileCheck %s --check-prefix=NODATA
 
 // CHECK: 1| 1|void foo(void) {}
 // CHECK: 2| 1|void bar(void) {}
 // CHECK: 3| 1|int main() {
 
 // FOO-ONLY: 1| 1|void foo(void) {}
+// MISSING-BINARY-ID: error: Failed to load coverage: '[[FILENAME]]': Missing binary ID: abcd1234
 // NODATA: error: Failed to load coverage: '': No coverage data found
 
 //--- foo.c
 void foo(void) {}
 
 //--- main.c
 void foo(void);
 void bar(void) {}
 int main() {
   foo();
   bar();
   return 0;
 }

llvm/docs/CommandGuide/llvm-cov.rst

 but not in any object given on the command line. Defaults to true if debuginfod
 is compiled in and configured via the DEBUGINFOD_URLS environment variable.
 
 .. option:: -debug-file-directory=<dir>
 
 Provides local directories to search for objects corresponding to binary IDs in
 the profile (as with debuginfod). Defaults to system build ID directories.
 
+.. option:: -check-binary-id
+
+Fail if an object file cannot be found for a binary ID present in the profile,
+neither on the command line nor via binary ID lookup.
+
 .. program:: llvm-cov report
 
 .. _llvm-cov-report:
 
 REPORT COMMAND
 --------------
 
 SYNOPSIS

 the command line, so long as debuginfod is compiled in and configured via
 DEBUGINFOD_URLS.
 
 .. option:: -debug-file-directory=<dir>
 
 Provides a directory to search for objects corresponding to binary IDs in the
 profile.
 
+.. option:: -check-binary-id
+
+Fail if an object file cannot be found for a binary ID present in the profile,
+neither on the command line nor via binary ID lookup.
+
 .. program:: llvm-cov export
 
 .. _llvm-cov-export:
 
 EXPORT COMMAND
 --------------
 
 SYNOPSIS

 attempted by default for binary IDs present in the profile but not provided on
 the command line, so long as debuginfod is compiled in and configured via
 DEBUGINFOD_URLS.
 
 .. option:: -debug-file-directory=<dir>
 
 Provides a directory to search for objects corresponding to binary IDs in the
 profile.
+
+.. option:: -check-binary-id
+
+Fail if an object file cannot be found for a binary ID present in the profile,
+neither on the command line nor via binary ID lookup.

llvm/include/llvm/ProfileData/Coverage/CoverageMapping.h

public:
 
   /// Load the coverage mapping from the given object files and profile. If
   /// \p Arches is non-empty, it must specify an architecture for each object.
   /// Ignores non-instrumented object files unless all are not instrumented.
   static Expected<std::unique_ptr<CoverageMapping>>
   load(ArrayRef<StringRef> ObjectFilenames, StringRef ProfileFilename,
        vfs::FileSystem &FS, ArrayRef<StringRef> Arches = std::nullopt,
        StringRef CompilationDir = "",
-       const object::BuildIDFetcher *BIDFetcher = nullptr);
+       const object::BuildIDFetcher *BIDFetcher = nullptr,
+       bool CheckBinaryIDs = false);
 
   /// The number of functions that couldn't have their profiles mapped.
   ///
   /// This is a count of functions whose profile is out of date or otherwise
   /// can't be associated with any coverage information.
   unsigned getMismatchedCount() const { return FuncHashMismatches.size(); }
 
   /// A hash mismatch occurs when a profile record for a symbol does not have

llvm/lib/ProfileData/Coverage/CoverageMapping.cpp

 // code coverage.
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/ProfileData/Coverage/CoverageMapping.h"
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/DenseMap.h"
 #include "llvm/ADT/SmallBitVector.h"
+#include "llvm/ADT/SmallString.h"
 #include "llvm/ADT/SmallVector.h"
+#include "llvm/ADT/StringExtras.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/Object/BuildID.h"
 #include "llvm/ProfileData/Coverage/CoverageMappingReader.h"
 #include "llvm/ProfileData/InstrProfReader.h"
 #include "llvm/Support/Debug.h"
 #include "llvm/Support/Errc.h"
 #include "llvm/Support/Error.h"
 #include "llvm/Support/ErrorHandling.h"
llvm::append_range(*FoundBinaryIDs,
                        }));
   }
   DataFound |= !Readers.empty();
   if (Error E = loadFromReaders(Readers, ProfileReader, Coverage))
     return createFileError(Filename, std::move(E));
   return Error::success();
 }
 
-Expected<std::unique_ptr<CoverageMapping>>
-CoverageMapping::load(ArrayRef<StringRef> ObjectFilenames,
-                      StringRef ProfileFilename, vfs::FileSystem &FS,
-                      ArrayRef<StringRef> Arches, StringRef CompilationDir,
-                      const object::BuildIDFetcher *BIDFetcher) {
+Expected<std::unique_ptr<CoverageMapping>> CoverageMapping::load(
+    ArrayRef<StringRef> ObjectFilenames, StringRef ProfileFilename,
+    vfs::FileSystem &FS, ArrayRef<StringRef> Arches, StringRef CompilationDir,
+    const object::BuildIDFetcher *BIDFetcher, bool CheckBinaryIDs) {
   auto ProfileReaderOrErr = IndexedInstrProfReader::create(ProfileFilename, FS);
   if (Error E = ProfileReaderOrErr.takeError())
     return createFileError(ProfileFilename, std::move(E));
   auto ProfileReader = std::move(ProfileReaderOrErr.get());
   auto Coverage = std::unique_ptr<CoverageMapping>(new CoverageMapping());
   bool DataFound = false;
 
   auto GetArch = [&](size_t Idx) {
if (!ProfileBinaryIDs.empty()) {
       std::set_difference(
           ProfileBinaryIDs.begin(), ProfileBinaryIDs.end(),
           FoundBinaryIDs.begin(), FoundBinaryIDs.end(),
           std::inserter(BinaryIDsToFetch, BinaryIDsToFetch.end()), Compare);
     }
 
     for (object::BuildIDRef BinaryID : BinaryIDsToFetch) {
       std::optional<std::string> PathOpt = BIDFetcher->fetch(BinaryID);
-      if (!PathOpt)
+      if (!PathOpt) {
+        if (CheckBinaryIDs) {
+          return createFileError(
+              ProfileFilename,
+              createStringError(errc::no_such_file_or_directory,
+                                "Missing binary ID: " +
+                                    llvm::toHex(BinaryID, /*LowerCase=*/true)));
+        }
         continue;
+      }
       std::string Path = std::move(*PathOpt);
       StringRef Arch = Arches.size() == 1 ? Arches.front() : StringRef();
       if (Error E = loadFromFile(Path, Arch, CompilationDir, *ProfileReader,
                                  *Coverage, DataFound))
         return std::move(E);
     }
   }
 

llvm/tools/llvm-cov/CodeCoverage.cpp

private:
   std::mutex LoadedSourceFilesLock;
   std::vector<std::pair<std::string, std::unique_ptr<MemoryBuffer>>>
       LoadedSourceFiles;
 
   /// Allowlist from -name-allowlist to be used for filtering.
   std::unique_ptr<SpecialCaseList> NameAllowlist;
 
   std::unique_ptr<object::BuildIDFetcher> BIDFetcher;
+
+  bool CheckBinaryIDs;
 };
 }
 
 static std::string getErrorString(const Twine &Message, StringRef Whence,
                                   bool Warning) {
   std::string Str = (Warning ? "warning" : "error");
   Str += ": ";
   if (!Whence.empty())

 }
 
 std::unique_ptr<CoverageMapping> CodeCoverageTool::load() {
   for (StringRef ObjectFilename : ObjectFilenames)
     if (modifiedTimeGT(ObjectFilename, PGOFilename))
       warning("profile data may be out of date - object is newer",
               ObjectFilename);
   auto FS = vfs::getRealFileSystem();
-  auto CoverageOrErr =
-      CoverageMapping::load(ObjectFilenames, PGOFilename, *FS, CoverageArches,
-                            ViewOpts.CompilationDirectory, BIDFetcher.get());
+  auto CoverageOrErr = CoverageMapping::load(
+      ObjectFilenames, PGOFilename, *FS, CoverageArches,
+      ViewOpts.CompilationDirectory, BIDFetcher.get(), CheckBinaryIDs);
   if (Error E = CoverageOrErr.takeError()) {
     error("Failed to load coverage: " + toString(std::move(E)));
     return nullptr;
   }
   auto Coverage = std::move(CoverageOrErr.get());
   unsigned Mismatched = Coverage->getMismatchedCount();
   if (Mismatched) {
     warning(Twine(Mismatched) + " functions have mismatched data");
cl::opt<unsigned> NumThreads(
       cl::desc("Number of merge threads to use (default: autodetect)"));
   cl::alias NumThreadsA("j", cl::desc("Alias for --num-threads"),
                         cl::aliasopt(NumThreads));
 
   cl::opt<std::string> CompilationDirectory(
       "compilation-dir", cl::init(""),
       cl::desc("Directory used as a base for relative coverage mapping paths"));
 
+  cl::opt<bool> CheckBinaryIDs(
+      "check-binary-ids", cl::desc("Fail if an object couldn't be found for a "
+                                   "binary ID in the profile"));
+
   auto commandLineParser = [&, this](int argc, const char **argv) -> int {
     cl::ParseCommandLineOptions(argc, argv, "LLVM code coverage tool\n");
     ViewOpts.Debug = DebugDump;
     if (Debuginfod) {
       HTTPClient::initialize();
       BIDFetcher = std::make_unique<DebuginfodFetcher>(DebugFileDirectory);
     } else {
       BIDFetcher = std::make_unique<object::BuildIDFetcher>(DebugFileDirectory);
     }
+    this->CheckBinaryIDs = CheckBinaryIDs;
 
     if (!CovFilename.empty())
       ObjectFilenames.emplace_back(CovFilename);
     for (const std::string &Filename : CovFilenames)
       ObjectFilenames.emplace_back(Filename);
     if (ObjectFilenames.empty() && !Debuginfod && DebugFileDirectory.empty()) {
       errs() << "No filenames specified!\n";
       ::exit(1);