[llvm-cov] New parameters to set coverage watermark

[tanjinhua]

Summary

Add a pairs of parameters to set coverage watermark for llvm-cov, and user can change the percentage thresholds marked with different colors in the report.

[tanjinhua]

bug fix

[tanjinhua]

add reviewer

[tanjinhua]

format code with clang-format

[tanjinhua]

I'm not sure if we should support parameters in this direction, I'll add test cases if you guys think so.

[tanjinhua]

update commit details

[tanjinhua]

em... Any one know should I invite other guys to review this? Or do I still need to do anything else?

[phosek]

I'd prefer using "high" and "low" watermark instead of "medium" and "low".

[davidxl]

Please add a test case. The documentation for llvm-cov tool also needs update.

[tanjinhua]

add test cases and documentation

[tanjinhua]

fix test case

[tanjinhua]

Done, could you help me review again? So thanks!
@phosek @davidxl

[tanjinhua]

Or anyone can help to review? Thanks a lot!

[tanjinhua]

Hi, @phosek @davidxl, could you help me review again? So thanks!

[phosek]

LGTM

[tanjinhua]

Hi, this is my first commit for llvm-project, I think this commit can try to push this branch to main now?
Refer to this link: https://llvm.org/docs/Phabricator.html#committing-a-change, is it ok to just leave a comment here, or what I do for next step?
If anyone can help me, so THX!

[tanjinhua]

Hi @phosek, I'm fresh to llvm-project, could you teach me how to do next step?

[phosek]

In D116876#3347627, @tanjinhua wrote:

Hi @phosek, I'm fresh to llvm-project, could you teach me how to do next step?

If you don't have a commit access, I can land the change for you.

[tanjinhua]

In D116876#3361570, @phosek wrote:

In D116876#3347627, @tanjinhua wrote:

Hi @phosek, I'm fresh to llvm-project, could you teach me how to do next step?

If you don't have a commit access, I can land the change for you.

Thanks, I don't have any permission to push, and I have try to ask here: https://discourse.llvm.org/t/ask-help-to-review-or-merge-my-commit/60702/5, lol
Could you help me land it, thanks a lot!

llvm/docs/CommandGuide/llvm-cov.rst

  threshold.
 
 .. option:: -path-equivalence=<from>,<to>
 
  Map the paths in the coverage data to local source file paths. This allows you
  to generate the coverage data on one machine, and then use llvm-cov on a
  different machine where you have the same files on a different path.
 
+.. option:: -coverage-watermark=<high>,<low>
+
+ Set high and low watermarks for coverage in html format output. This allows you
+ to set the high and low watermark of coverage as desired, green when
+ coverage >= high, red when coverage < low, and yellow otherwise. Both high and
+ low should be between 0-100 and high > low.
+
 .. program:: llvm-cov report
 
 .. _llvm-cov-report:
 
 REPORT COMMAND
 --------------
 
 SYNOPSIS

llvm/test/tools/llvm-cov/coverage_watermark.c

+// RUN: not llvm-cov show --instr-profile=/dev/null -coverage-watermark=foo /dev/null 2>&1 | FileCheck --check-prefix=INVALID-ARG %s
+// INVALID-ARG: error: -coverage-watermark: invalid argument 'foo', must be in format 'high,low'
+
+// RUN: not llvm-cov show --instr-profile=/dev/null -coverage-watermark=a,2 /dev/null 2>&1 | FileCheck --check-prefix=INVALID-HIGH %s
+// INVALID-HIGH: error: -coverage-watermark: invalid number 'a', invalid value for 'high'
+
+// RUN: not llvm-cov show --instr-profile=/dev/null -coverage-watermark=10,b /dev/null 2>&1 | FileCheck --check-prefix=INVALID-LOW %s
+// INVALID-LOW: error: -coverage-watermark: invalid number 'b', invalid value for 'low'
+
+// RUN: not llvm-cov show --instr-profile=/dev/null -coverage-watermark=10,20 /dev/null 2>&1 | FileCheck --check-prefix=INVALID-ARRANGE %s
+// INVALID-ARRANGE: error: -coverage-watermark: invalid number range '10,20', must be both high and low should be between 0-100, and high > low
+
+// RUN: llvm-cov show %S/Inputs/templateInstantiations.covmapping -instr-profile %t.profdata -format html -show-region-summary -show-instantiation-summary -o %t.html.dir -path-equivalence=/tmp,%S %S/showTemplateInstantiations.cpp
+// RUN: FileCheck -check-prefix=ORIGIN %s -input-file %t.html.dir/index.html
+
+// ORIGIN: <td class='column-entry-red'>
+// ORIGIN: 50.00% (1/2)
+// ORIGIN: <td class='column-entry-red'>
+// ORIGIN: 33.33% (1/3)
+// ORIGIN: <td class='column-entry-red'>
+// ORIGIN: 41.67% (5/12)
+// ORIGIN: <td class='column-entry-red'>
+// ORIGIN: 16.67% (1/6)
+
+// RUN: llvm-cov show %S/Inputs/templateInstantiations.covmapping -instr-profile %t.profdata -format html -show-region-summary -show-instantiation-summary -o %t.html.dir -path-equivalence=/tmp,%S -coverage-watermark 50,30 %S/showTemplateInstantiations.cpp
+// RUN: FileCheck -check-prefix=LOWER %s -input-file %t.html.dir/index.html
+
+// LOWER: <td class='column-entry-green'>
+// LOWER: 50.00% (1/2)
+// LOWER: <td class='column-entry-yellow'>
+// LOWER: 33.33% (1/3)
+// LOWER: <td class='column-entry-yellow'>
+// LOWER: 41.67% (5/12)
+// LOWER: <td class='column-entry-red'>
+// LOWER: 16.67% (1/6)
+
+int main() {}

llvm/tools/llvm-cov/CodeCoverage.cpp

cl::opt<uint32_t> TabSize(
       "tab-size", cl::init(2),
       cl::desc(
           "Set tab expansion size for html coverage reports (default = 2)"));
 
   cl::opt<std::string> ProjectTitle(
       "project-title", cl::Optional,
       cl::desc("Set project title for the coverage report"));
 
+  cl::opt<std::string> CovWatermark(
+      "coverage-watermark", cl::Optional,
+      cl::desc("<high>,<low> value indicate thresholds for high and low"
+               "coverage watermark"));
+
   auto Err = commandLineParser(argc, argv);
   if (Err)
     return Err;
 
   if (ViewOpts.Format == CoverageViewOptions::OutputFormat::Lcov) {
     error("Lcov format should be used with 'llvm-cov export'.");
     return 1;
   }
 
+  ViewOpts.HighCovWatermark = 100.0;
+  ViewOpts.LowCovWatermark = 80.0;
+  if (!CovWatermark.empty()) {
+    auto WaterMarkPair = StringRef(CovWatermark).split(',');
+    if (WaterMarkPair.first.empty() || WaterMarkPair.second.empty()) {
+      error("invalid argument '" + CovWatermark +
+                "', must be in format 'high,low'",
+            "-coverage-watermark");
+      return 1;
+    }
+
+    char *EndPointer = nullptr;
+    ViewOpts.HighCovWatermark =
+        strtod(WaterMarkPair.first.begin(), &EndPointer);
+    if (EndPointer != WaterMarkPair.first.end()) {
+      error("invalid number '" + WaterMarkPair.first +
+                "', invalid value for 'high'",
+            "-coverage-watermark");
+      return 1;
+    }
+
+    ViewOpts.LowCovWatermark =
+        strtod(WaterMarkPair.second.begin(), &EndPointer);
+    if (EndPointer != WaterMarkPair.second.end()) {
+      error("invalid number '" + WaterMarkPair.second +
+                "', invalid value for 'low'",
+            "-coverage-watermark");
+      return 1;
+    }
+
+    if (ViewOpts.HighCovWatermark > 100 || ViewOpts.LowCovWatermark < 0 ||
+        ViewOpts.HighCovWatermark <= ViewOpts.LowCovWatermark) {
+      error(
+          "invalid number range '" + CovWatermark +
+              "', must be both high and low should be between 0-100, and high "
+              "> low",
+          "-coverage-watermark");
+      return 1;
+    }
+  }
+
   ViewOpts.ShowLineNumbers = true;
   ViewOpts.ShowLineStats = ShowLineExecutionCounts.getNumOccurrences() != 0 ||
                            !ShowRegions || ShowBestLineRegionsCounts;
   ViewOpts.ShowRegionMarkers = ShowRegions || ShowBestLineRegionsCounts;
   ViewOpts.ShowExpandedRegions = ShowExpansions;
   ViewOpts.ShowBranchCounts =
       ShowBranches == CoverageViewOptions::BranchOutputType::Count;
   ViewOpts.ShowBranchPercents =

llvm/tools/llvm-cov/CoverageViewOptions.h

struct CoverageViewOptions {
   BranchOutputType ShowBranches;
   std::string ShowOutputDirectory;
   std::vector<std::string> DemanglerOpts;
   uint32_t TabSize;
   std::string ProjectTitle;
   std::string CreatedTimeStr;
   unsigned NumThreads;
   std::string CompilationDirectory;
+  float HighCovWatermark;
+  float LowCovWatermark;
 
   /// Change the output's stream color if the colors are enabled.
   ColoredRawOstream colored_ostream(raw_ostream &OS,
                                     raw_ostream::Colors Color) const {
     return llvm::colored_ostream(OS, Color, Colors);
   }
 
   /// Check if an output directory has been specified.

llvm/tools/llvm-cov/SourceCoverageViewHTML.cpp

 /// Render a file coverage summary (\p FCS) in a table row. If \p IsTotals is
 /// false, link the summary to \p SF.
 void CoveragePrinterHTML::emitFileSummary(raw_ostream &OS, StringRef SF,
                                           const FileCoverageSummary &FCS,
                                           bool IsTotals) const {
   SmallVector<std::string, 8> Columns;
 
   // Format a coverage triple and add the result to the list of columns.
-  auto AddCoverageTripleToColumn = [&Columns](unsigned Hit, unsigned Total,
-                                              float Pctg) {
+  auto AddCoverageTripleToColumn =
+      [&Columns, this](unsigned Hit, unsigned Total, float Pctg) {
         std::string S;
         {
           raw_string_ostream RSO{S};
           if (Total)
             RSO << format("%*.2f", 7, Pctg) << "% ";
           else
             RSO << "- ";
           RSO << '(' << Hit << '/' << Total << ')';
         }
         const char *CellClass = "column-entry-yellow";
-    if (Hit == Total)
+        if (Pctg >= Opts.HighCovWatermark)
           CellClass = "column-entry-green";
-    else if (Pctg < 80.0)
+        else if (Pctg < Opts.LowCovWatermark)
           CellClass = "column-entry-red";
         Columns.emplace_back(tag("td", tag("pre", S), CellClass));
       };
 
   // Simplify the display file path, and wrap it in a link if requested.
   std::string Filename;
   if (IsTotals) {
     Filename = std::string(SF);
   } else {
     Filename = buildLinkToFile(SF, FCS);
   }