In January, I posted a basic design outline on llvm-dev for supporting Branch Coverage as part of LLVM Source-based Code Coverage. In general, I think this is a straightforward extension of source-based code coverage to measure True/False execution counts of branch-generating conditions in the source code.
Note that by "branch coverage", I am referring to a granular level of measurement for each leaf-level boolean expression (i.e. "condition") that may compose larger boolean expressions using logical operators. This is an equivalent level of granularity to what is reported by GCOV. This granular level of branch coverage I implemented is amenable to further extension for MC/DC analysis. I am not referring to higher-level "decision coverage".
I apologize for the lengthy review -- there are three main sections, but they're interconnected. Most of the other changes are for testing (which, I confirmed, is sufficient to provide coverage of everything that was added). This is my first phabricator review, so please give me suggestions or improvement!
Examples of output for Text and HTML:
Text: f{F12395372}
HTML (as PDF):
Three main chunks of work:
- Adding and using "Branch Regions":
Extending the coverage mapping format with a notion of a "branch region" kind with an additional alternate "FalseCount" counter. Both counters can then be used to measure "True" and "False" branch counts. Also extend the Writer/Reader encoding routines to comprehend it.
clang/lib/CodeGen/CoverageMappingGen.cpp clang/lib/CodeGen/CoverageMappingGen.h llvm/include/llvm/ProfileData/Coverage/CoverageMapping.h llvm/lib/ProfileData/Coverage/CoverageMappingWriter.cpp llvm/lib/ProfileData/Coverage/CoverageMappingReader.cpp
- Instrumenting logical operators ("&&", "||"):
The existing counter instrumentation can be leveraged for most branch regions. Logical operators are a bit special and require an additional counter.
Consider: "X = C1 || C2;"
For logical operators, an execution counter is already emitted to track the execution count for the right-hand-side of the expression (C2), and because of logical operator short-circuit semantics, this counter can tell you whether they left-hand-side (C1) evaluated to True or False (depending on whether it is logical-AND or logical-OR). However, this doesn't present enough information to tell you how many times the right-hand-side (C2) evaluated to True or False. Therefore, we need to instrument an additional counter here that is dependent on the operator semantics.
Note that there is room for optimization here: when a boolean expression with logical operators is used with a control-flow statement (like if-stmt), we can leverage the "Then" block counter. And in cases where you have multiple levels of nested logical operators, we can re-use counters. For this review, I have chosen to keep the design straightforward and will defer these optimizations to a future patch.
Since the new counters extend the profile format, I bumped the profile version to allow backward compatibility with older profile versions.
clang/lib/CodeGen/CGExprScalar.cpp clang/lib/CodeGen/CGStmt.cpp clang/lib/CodeGen/CodeGenFunction.cpp clang/lib/CodeGen/CodeGenFunction.h clang/lib/CodeGen/CodeGenPGO.cpp llvm/include/llvm/ProfileData/InstrProf.h llvm/include/llvm/ProfileData/InstrProfData.inc
- Visualization (llvm-cov):
Extend the llvm-cov built-in notion of "SubView" to visualize branches for whole files, individual functions, and macro expansions (which are recursive because macros can be based on other macros). SubViews keep the visualization regions visually distinct.
Extend summary reports to include branch coverage
Enable branch coverage data to be exportable into JSON and LCOV formats.
llvm/lib/ProfileData/Coverage/CoverageMapping.cpp llvm/tools/llvm-cov/CodeCoverage.cpp llvm/tools/llvm-cov/CoverageReport.cpp llvm/tools/llvm-cov/CoverageSummaryInfo.cpp llvm/tools/llvm-cov/CoverageSummaryInfo.h llvm/tools/llvm-cov/CoverageViewOptions.h llvm/tools/llvm-cov/SourceCoverageView.cpp llvm/tools/llvm-cov/SourceCoverageView.h llvm/tools/llvm-cov/SourceCoverageViewHTML.cpp llvm/tools/llvm-cov/SourceCoverageViewHTML.h llvm/tools/llvm-cov/SourceCoverageViewText.cpp llvm/tools/llvm-cov/SourceCoverageViewText.h llvm/tools/llvm-cov/CoverageExporterJson.cpp llvm/tools/llvm-cov/CoverageExporterLcov.cpp
This fixes a defect that resulted in the wrong counter being instrumented for each Case.