Add inclusion filename filters to llvm-cov

[tinloaf]

Summary

Currently, llvm-cov only supports exclusion of files (based on regular expressions). This adds the possibility to also specify a "whitelist" of files via regex. All files not matching one of the supplied regular expressions will be excluded from coverage reporting.

[tinloaf]

I tested this locally and it seems to work; however, I would feel better if I had some tests for this (and the ignore-regex filters, too).

As far as I can tell, there would be two steps to adding tests to this:

• Add a test script at (e.g.) llvm/test/tools/llvm-cov/filename_filters.test
• Add some input data for this test script in the form of a "proftext" file at (e.g.) llvm/test/tools/llvm-cov/Inputs/filename_filters.proftext

However, I know neither the syntax of the .test scripts nor the syntax of the proftext format. If I generate some "proftext" file by running llvm-profdata -text …, I can see the file names in the resulting output file, but no paths. Searching for "proftext llvm" or "proftext format" does give me lots of other tests using this format, but no docs on the format itsef. If somebody could point me in the right direction, I'll gladly add some tests.

[tinloaf]

• Fix logic for empty test sets

[tinloaf]

• Reformat

[tinloaf]

• Suppress clang-tidy diagnostics for push_back

[vsk]

Please take a look at this patch by Michael Daniels that implements a similar feature:
bfed824b57d14e2ba98ddbaf1a1410cf04a3e279

It includes a test and documentation updates that could be folded into this patch. (The patch was reverted in 8d24d72f7f8b7a111f96510fc6d62b05bfb7dbec due to buildbot failures I didn't have the bandwidth to investigate.)

llvm/tools/llvm-cov/CodeCoverage.cpp

 //===- CodeCoverage.cpp - Coverage tool based on profiling instrumentation-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //

 #include "llvm/Support/SpecialCaseList.h"
 #include "llvm/Support/ThreadPool.h"
 #include "llvm/Support/Threading.h"
 #include "llvm/Support/ToolOutputFile.h"
 #include "llvm/Support/VirtualFileSystem.h"
 
 #include <functional>
 #include <map>
+#include <memory>
 #include <system_error>
 
 using namespace llvm;
 using namespace coverage;
 
 void exportCoverageDataToJson(const coverage::CoverageMapping &CoverageMapping,
                               const CoverageViewOptions &Options,
                               raw_ostream &OS);
 
 namespace {
 /// The implementation of the coverage tool.
 class CodeCoverageTool {
 public:
   enum Command {
     /// The show command.
     Show,
     /// The report command.
     Report,
     /// The export command.
     Export
   };
 
   int run(Command Cmd, int argc, const char **argv);
 
+  CodeCoverageTool() {
+    FilenameFilters.push_back(IgnoreFilenameFilters);
+    FilenameFilters.push_back(IncludeFilenameFilters);
+  }
+
 private:
   /// Print the error message to the error output stream.
   void error(const Twine &Message, StringRef Whence = "");
 
   /// Print the warning message to the error output stream.
   void warning(const Twine &Message, StringRef Whence = "");
 
   /// Convert \p Path into an absolute path and append it to the list
int doReport(int argc, const char **argv,
                CommandLineParserType commandLineParser);
 
   int doExport(int argc, const char **argv,
                CommandLineParserType commandLineParser);
 
   std::vector<StringRef> ObjectFilenames;
   CoverageViewOptions ViewOpts;
   CoverageFiltersMatchAll Filters;
-  CoverageFilters IgnoreFilenameFilters;
+
+  std::shared_ptr<CoverageFilters> IgnoreFilenameFilters =
+      std::make_shared<CoverageFilters>();
+  // The inclusion-regex-filters individually match if the inclusion regex
+  // does *not* match the filename (i.e., *this* regex should not cause the
+  // inclusion of *that* file). Thus, for a file to be included, *all* the
+  // inclusion filters must match (i.e., not want to include the file).
+  std::shared_ptr<CoverageFilters> IncludeFilenameFilters =
+      std::make_shared<CoverageFiltersMatchAll>(false);
+
+  // A file is to be excluded if either the exclusion-filters match or
+  // the inclusion-filters match (which match if the file is *not* to be
+  // included, see above)
+  CoverageFilters FilenameFilters;
 
   /// True if InputSourceFiles are provided.
   bool HadSourceFiles = false;
 
   /// The path to the indexed profile.
   std::string PGOFilename;
 
   /// A list of input source files.
private:
   /// A container for input source file buffers.
   std::mutex LoadedSourceFilesLock;
   std::vector<std::pair<std::string, std::unique_ptr<MemoryBuffer>>>
       LoadedSourceFiles;
 
   /// Whitelist from -name-whitelist to be used for filtering.
   std::unique_ptr<SpecialCaseList> NameWhitelist;
 };
-}
+} // namespace
 
 static std::string getErrorString(const Twine &Message, StringRef Whence,
                                   bool Warning) {
   std::string Str = (Warning ? "warning" : "error");
   Str += ": ";
   if (!Whence.empty())
     Str += Whence.str() + ": ";
   Str += Message.str() + "\n";

 
 void CodeCoverageTool::addCollectedPath(const std::string &Path) {
   SmallString<128> EffectivePath(Path);
   if (std::error_code EC = sys::fs::make_absolute(EffectivePath)) {
     error(EC.message(), Path);
     return;
   }
   sys::path::remove_dots(EffectivePath, /*remove_dot_dots=*/true);
-  if (!IgnoreFilenameFilters.matchesFilename(EffectivePath))
+  if (!FilenameFilters.matchesFilename(EffectivePath))
     SourceFiles.emplace_back(EffectivePath.str());
   HadSourceFiles = !SourceFiles.empty();
 }
 
 void CodeCoverageTool::collectPaths(const std::string &Path) {
   llvm::sys::fs::file_status Status;
   llvm::sys::fs::status(Path, Status);
   if (!llvm::sys::fs::exists(Status)) {
     if (PathRemapping)
       addCollectedPath(Path);
     else
       warning("Source file doesn't exist, proceeded by ignoring it.", Path);
     return;
   }
 
   if (llvm::sys::fs::is_regular_file(Status)) {
     addCollectedPath(Path);
     return;
   }
 
   if (llvm::sys::fs::is_directory(Status)) {
     std::error_code EC;
-    for (llvm::sys::fs::recursive_directory_iterator F(Path, EC), E;
-         F != E; F.increment(EC)) {
+    for (llvm::sys::fs::recursive_directory_iterator F(Path, EC), E; F != E;
+         F.increment(EC)) {
 
       auto Status = F->status();
       if (!Status) {
         warning(Status.getError().message(), F->path());
         continue;
       }
 
       if (Status->type() == llvm::sys::fs::file_type::regular_file)
CodeCoverageTool::createFunctionView(const FunctionRecord &Function,
   if (FunctionCoverage.empty())
     return nullptr;
   auto SourceBuffer = getSourceFile(FunctionCoverage.getFilename());
   if (!SourceBuffer)
     return nullptr;
 
   auto Branches = FunctionCoverage.getBranches();
   auto Expansions = FunctionCoverage.getExpansions();
-  auto View = SourceCoverageView::create(DC.demangle(Function.Name),
-                                         SourceBuffer.get(), ViewOpts,
-                                         std::move(FunctionCoverage));
+  auto View =
+      SourceCoverageView::create(DC.demangle(Function.Name), SourceBuffer.get(),
+                                 ViewOpts, std::move(FunctionCoverage));
   attachExpansionSubViews(*View, Expansions, Coverage);
   attachBranchSubViews(*View, DC.demangle(Function.Name), Branches,
                        SourceBuffer.get(), FunctionCoverage);
 
   return View;
 }
 
 std::unique_ptr<SourceCoverageView>
cl::list<std::string> NameRegexFilters(
       cl::ZeroOrMore, cl::cat(FilteringCategory));
 
   cl::list<std::string> IgnoreFilenameRegexFilters(
       "ignore-filename-regex", cl::Optional,
       cl::desc("Skip source code files with file paths that match the given "
                "regular expression"),
       cl::ZeroOrMore, cl::cat(FilteringCategory));
 
+  cl::list<std::string> FilenameRegexFilters(
+      "filename-regex", cl::Optional,
+      cl::desc("If given, only report coverage vor source code files matching "
+               "the given regular expression"),
+      cl::ZeroOrMore, cl::cat(FilteringCategory));
+
   cl::opt<double> RegionCoverageLtFilter(
       "region-coverage-lt", cl::Optional,
       cl::desc("Show code coverage only for functions with region coverage "
                "less than the given threshold"),
       cl::cat(FilteringCategory));
 
   cl::opt<double> RegionCoverageGtFilter(
       "region-coverage-gt", cl::Optional,
cl::opt<cl::boolOrDefault> UseColor(
       "use-color", cl::desc("Emit colored output (default=autodetect)"),
       cl::init(cl::BOU_UNSET));
 
   cl::list<std::string> DemanglerOpts(
       "Xdemangler", cl::desc("<demangler-path>|<demangler-option>"));
 
   cl::opt<bool> RegionSummary(
       "show-region-summary", cl::Optional,
-      cl::desc("Show region statistics in summary table"),
-      cl::init(true));
+      cl::desc("Show region statistics in summary table"), cl::init(true));
 
   cl::opt<bool> BranchSummary(
       "show-branch-summary", cl::Optional,
       cl::desc("Show branch condition statistics in summary table"),
       cl::init(true));
 
   cl::opt<bool> InstantiationSummary(
       "show-instantiation-summary", cl::Optional,
if (!NameFilterFiles.empty()) {
       NameWhitelist = SpecialCaseList::create(
           NameFilterFiles, *vfs::getRealFileSystem(), SpecialCaseListErr);
       if (!NameWhitelist)
         error(SpecialCaseListErr);
     }
 
     // Create the function filters
     if (!NameFilters.empty() || NameWhitelist || !NameRegexFilters.empty()) {
-      auto NameFilterer = std::make_unique<CoverageFilters>();
+      auto NameFilterer = std::make_shared<CoverageFilters>();
       for (const auto &Name : NameFilters)
-        NameFilterer->push_back(std::make_unique<NameCoverageFilter>(Name));
+        NameFilterer->push_back(std::make_shared<NameCoverageFilter>(Name));
       if (NameWhitelist)
         NameFilterer->push_back(
-            std::make_unique<NameWhitelistCoverageFilter>(*NameWhitelist));
+            std::make_shared<NameWhitelistCoverageFilter>(*NameWhitelist));
       for (const auto &Regex : NameRegexFilters)
         NameFilterer->push_back(
-            std::make_unique<NameRegexCoverageFilter>(Regex));
+            std::make_shared<NameRegexCoverageFilter>(Regex));
       Filters.push_back(std::move(NameFilterer));
     }
 
     if (RegionCoverageLtFilter.getNumOccurrences() ||
         RegionCoverageGtFilter.getNumOccurrences() ||
         LineCoverageLtFilter.getNumOccurrences() ||
         LineCoverageGtFilter.getNumOccurrences()) {
-      auto StatFilterer = std::make_unique<CoverageFilters>();
+      auto StatFilterer = std::make_shared<CoverageFilters>();
       if (RegionCoverageLtFilter.getNumOccurrences())
-        StatFilterer->push_back(std::make_unique<RegionCoverageFilter>(
+        StatFilterer->push_back(std::make_shared<RegionCoverageFilter>(
             RegionCoverageFilter::LessThan, RegionCoverageLtFilter));
       if (RegionCoverageGtFilter.getNumOccurrences())
-        StatFilterer->push_back(std::make_unique<RegionCoverageFilter>(
+        StatFilterer->push_back(std::make_shared<RegionCoverageFilter>(
             RegionCoverageFilter::GreaterThan, RegionCoverageGtFilter));
       if (LineCoverageLtFilter.getNumOccurrences())
-        StatFilterer->push_back(std::make_unique<LineCoverageFilter>(
+        StatFilterer->push_back(std::make_shared<LineCoverageFilter>(
             LineCoverageFilter::LessThan, LineCoverageLtFilter));
       if (LineCoverageGtFilter.getNumOccurrences())
-        StatFilterer->push_back(std::make_unique<LineCoverageFilter>(
+        StatFilterer->push_back(std::make_shared<LineCoverageFilter>(
             RegionCoverageFilter::GreaterThan, LineCoverageGtFilter));
       Filters.push_back(std::move(StatFilterer));
     }
 
     // Create the ignore filename filters.
     for (const auto &RE : IgnoreFilenameRegexFilters)
-      IgnoreFilenameFilters.push_back(
-          std::make_unique<NameRegexCoverageFilter>(RE));
+      IgnoreFilenameFilters->push_back(
+          std::make_shared<NameRegexCoverageFilter>(RE));
+    // 'Filters' do always exclude files/functions, thus an inclusion-filter
+    // should match if the file name does *not* match the regex. Invert the
+    // filter!
+    for (const auto &RE : FilenameRegexFilters)
+      IncludeFilenameFilters->push_back(
+          std::make_shared<InvertedCoverageFilter>(
+              std::make_unique<NameRegexCoverageFilter>(RE)));
 
     if (!Arches.empty()) {
       for (const std::string &Arch : Arches) {
         if (Triple(Arch).getArch() == llvm::Triple::ArchType::UnknownArch) {
           error("Unknown architecture: " + Arch);
           return 1;
         }
         CoverageArches.emplace_back(Arch);
int CodeCoverageTool::doShow(int argc, const char **argv,
   if (!Coverage)
     return 1;
 
   auto Printer = CoveragePrinter::create(ViewOpts);
 
   if (SourceFiles.empty() && !HadSourceFiles)
     // Get the source files from the function coverage mapping.
     for (StringRef Filename : Coverage->getUniqueSourceFiles()) {
-      if (!IgnoreFilenameFilters.matchesFilename(Filename))
+      if (!FilenameFilters.matchesFilename(Filename))
         SourceFiles.push_back(std::string(Filename));
     }
 
   // Create an index out of the source files.
   if (ViewOpts.hasOutputDirectory()) {
     if (Error E = Printer->createIndexFile(SourceFiles, *Coverage, Filters)) {
       error("Could not create index file!", toString(std::move(E)));
       return 1;
int CodeCoverageTool::doReport(int argc, const char **argv,
 
   auto Coverage = load();
   if (!Coverage)
     return 1;
 
   CoverageReport Report(ViewOpts, *Coverage.get());
   if (!ShowFunctionSummaries) {
     if (SourceFiles.empty())
-      Report.renderFileReports(llvm::outs(), IgnoreFilenameFilters);
+      Report.renderFileReports(llvm::outs(), FilenameFilters);
     else
       Report.renderFileReports(llvm::outs(), SourceFiles);
   } else {
     if (SourceFiles.empty()) {
       error("Source files must be specified when -show-functions=true is "
             "specified");
       return 1;
     }
if (!Coverage) {
     error("Could not load coverage information");
     return 1;
   }
 
   std::unique_ptr<CoverageExporter> Exporter;
 
   switch (ViewOpts.Format) {
   case CoverageViewOptions::OutputFormat::Text:
-    Exporter = std::make_unique<CoverageExporterJson>(*Coverage.get(),
-                                                       ViewOpts, outs());
+    Exporter = std::make_unique<CoverageExporterJson>(*Coverage.get(), ViewOpts,
+                                                      outs());
     break;
   case CoverageViewOptions::OutputFormat::HTML:
     // Unreachable because we should have gracefully terminated with an error
     // above.
     llvm_unreachable("Export in HTML is not supported!");
   case CoverageViewOptions::OutputFormat::Lcov:
-    Exporter = std::make_unique<CoverageExporterLcov>(*Coverage.get(),
-                                                       ViewOpts, outs());
+    Exporter = std::make_unique<CoverageExporterLcov>(*Coverage.get(), ViewOpts,
+                                                      outs());
     break;
   }
 
   if (SourceFiles.empty())
-    Exporter->renderRoot(IgnoreFilenameFilters);
+    Exporter->renderRoot(FilenameFilters);
   else
     Exporter->renderRoot(SourceFiles);
 
   return 0;
 }
 
 int showMain(int argc, const char *argv[]) {
   CodeCoverageTool Tool;

llvm/tools/llvm-cov/CoverageFilters.h

 //===- CoverageFilters.h - Function coverage mapping filters --------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
public:
 
   /// Return true if the function passes the requirements of this filter.
   virtual bool matches(const coverage::CoverageMapping &CM,
                        const coverage::FunctionRecord &Function) const {
     return true;
   }
 
   /// Return true if the filename passes the requirements of this filter.
-  virtual bool matchesFilename(StringRef Filename) const {
-    return true;
+  virtual bool matchesFilename(StringRef Filename) const { return true; }
+};
+
+class InvertedCoverageFilter : public CoverageFilter {
+private:
+  std::unique_ptr<CoverageFilter> BaseFilter;
+
+public:
+  InvertedCoverageFilter(std::unique_ptr<CoverageFilter> Base)
+      : BaseFilter(std::move(Base)) {}
+
+  /// Return true if the function passes the requirements of this filter.
+  bool matches(const coverage::CoverageMapping &CM,
+               const coverage::FunctionRecord &Function) const override {
+    return !BaseFilter->matches(CM, Function);
+  }
+
+  /// Return true if the filename passes the requirements of this filter.
+  bool matchesFilename(StringRef Filename) const override {
+    return !BaseFilter->matchesFilename(Filename);
   }
 };
 
 /// Matches functions that contain a specific string in their name.
 class NameCoverageFilter : public CoverageFilter {
   StringRef Name;
 
 public:
bool matches(const coverage::CoverageMapping &CM,
                const coverage::FunctionRecord &Function) const override;
 };
 
 /// A collection of filters.
 /// Matches functions that match any filters contained
 /// in an instance of this class.
 class CoverageFilters : public CoverageFilter {
 protected:
-  std::vector<std::unique_ptr<CoverageFilter>> Filters;
+  std::vector<std::shared_ptr<CoverageFilter>> Filters;
+  const bool ValueIfEmpty;
 
 public:
+  /// Create a new match-any filter
+  ///
+  /// @param ValueIfEmptyIn Set to the whether this filter should match
+  /// anything/nothing if it is emtpy.
+  CoverageFilters(bool ValueIfEmptyIn = false) : ValueIfEmpty(ValueIfEmptyIn) {}
+
   /// Append a filter to this collection.
-  void push_back(std::unique_ptr<CoverageFilter> Filter);
+  // NOLINTNEXTLINE(readability-identifier-naming)
+  void push_back(std::shared_ptr<CoverageFilter> Filter);
 
   bool empty() const { return Filters.empty(); }
 
   bool matches(const coverage::CoverageMapping &CM,
                const coverage::FunctionRecord &Function) const override;
 
   bool matchesFilename(StringRef Filename) const override;
 };
 
 /// A collection of filters.
 /// Matches functions that match all of the filters contained
 /// in an instance of this class.
 class CoverageFiltersMatchAll : public CoverageFilters {
 public:
+  /// Create a new match-all filter
+  ///
+  /// @param ValueIfEmptyIn Set to the whether this filter should match
+  /// anything/nothing if it is emtpy.
+  CoverageFiltersMatchAll(bool ValueIfEmptyIn = true)
+      : CoverageFilters(ValueIfEmptyIn) {}
   bool matches(const coverage::CoverageMapping &CM,
                const coverage::FunctionRecord &Function) const override;
+
+  bool matchesFilename(StringRef Filename) const override;
 };
 
 } // namespace llvm
 
 #endif // LLVM_COV_COVERAGEFILTERS_H

llvm/tools/llvm-cov/CoverageFilters.cpp

 //===- CoverageFilters.cpp - Function coverage mapping filters ------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //

 
 bool LineCoverageFilter::matches(
     const coverage::CoverageMapping &CM,
     const coverage::FunctionRecord &Function) const {
   return PassesThreshold(FunctionCoverageSummary::get(CM, Function)
                              .LineCoverage.getPercentCovered());
 }
 
-void CoverageFilters::push_back(std::unique_ptr<CoverageFilter> Filter) {
+// NOLINTNEXTLINE(readability-identifier-naming)
+void CoverageFilters::push_back(std::shared_ptr<CoverageFilter> Filter) {
   Filters.push_back(std::move(Filter));
 }
 
 bool CoverageFilters::matches(const coverage::CoverageMapping &CM,
                               const coverage::FunctionRecord &Function) const {
+  if (empty()) {
+    return ValueIfEmpty;
+  }
+
   for (const auto &Filter : Filters) {
     if (Filter->matches(CM, Function))
       return true;
   }
   return false;
 }
 
 bool CoverageFilters::matchesFilename(StringRef Filename) const {
+  if (empty()) {
+    return ValueIfEmpty;
+  }
+
   for (const auto &Filter : Filters) {
     if (Filter->matchesFilename(Filename))
       return true;
   }
   return false;
 }
 
 bool CoverageFiltersMatchAll::matches(
     const coverage::CoverageMapping &CM,
     const coverage::FunctionRecord &Function) const {
+  if (empty()) {
+    return ValueIfEmpty;
+  }
+
   for (const auto &Filter : Filters) {
     if (!Filter->matches(CM, Function))
       return false;
   }
   return true;
 }
+
+bool CoverageFiltersMatchAll::matchesFilename(StringRef Filename) const {
+  if (empty()) {
+    return ValueIfEmpty;
+  }
+
+  for (const auto &Filter : Filters) {
+    if (!Filter->matchesFilename(Filename))
+      return false;
+  }
+  return true;
+}