[InstrProf] Add single byte coverage mode

[ellis]

Summary

Use the llvm flag -pgo-function-entry-coverage to create single byte "counters" to track functions coverage. This mode has significantly less size overhead in both code and data because

• We mark a function as "covered" with a store instead of an increment which generally requires fewer assembly instructions
• We use a single byte per function rather than 8 bytes per block

The trade off of course is that this mode only tells you if a function has been covered. This is useful, for example, to detect dead code.

When combined with debug info correlation [0] we are able to create an instrumented Clang binary that is only 150M (the vanilla Clang binary is 143M). That is an overhead of 7M (4.9%) compared to the default instrumentation (without value profiling) which has an overhead of 31M (21.7%).

[0] https://groups.google.com/g/llvm-dev/c/r03Z6JoN7d4

[kyulee]

I will come back to review this code later once the dependent patches land.
In the high-level discussion, this is really a function coverage because it probes the function header, but the flag or terminology inter-mixed just a coverage.
For the code (not just function) coverage, I think we might want to have an option to inject this probe in the block edge as well so that the default/code-coverage can be fully functional.
Of course, this could be a follow-up, but just head-up on how this can be factored in this design.

compiler-rt/test/profile/instrprof-coverage.c

2

Can you add a test case using the debug data -debug-info?

llvm/lib/Transforms/Instrumentation/PGOInstrumentation.cpp

478

Hmm. It appears to continue adding extra parameters.
I wonder whether we can refactor this given more features are coming.

[ellis]

In D116180#3211818, @kyulee wrote:

In the high-level discussion, this is really a function coverage because it probes the function header, but the flag or terminology inter-mixed just a coverage.

You're right, I should update the flag name to make it more clear that this is only a function entry coverage.

For the code (not just function) coverage, I think we might want to have an option to inject this probe in the block edge as well so that the default/code-coverage can be fully functional.
Of course, this could be a follow-up, but just head-up on how this can be factored in this design.

I've experimented with instrumenting clang with block coverage vs block counts. It turns out that full block coverage actually requires a larger counters section than regular block counts. This is because, for block counts, we can infer some missing block counts from summing up their parents or children. However, we cannot do this when using boolean values in the same way. So every basic block must be probed. (There are some cases where you can infer some missing block coverage, but I haven't thought deeply about when you can do this. And I suspect this is rare.) Also, there was slightly less assembly code from block counts rather than block coverage, but that only made the overall size difference negligible between the two.

Basically, block coverage has about the same size overhead, but gives significantly less information than block counts. We could allow for counters to be 16 or 8 bits to save space in the block counters mode, but that would be left for a later diff.

[ellis]

Rebase and add test to instrprof-debug-info-correlate.c.

[kyulee]

llvm/lib/Transforms/Instrumentation/InstrProfiling.cpp

1025

I think we can just delete this condition.
Without DebugInfoCorrelate, Counters are referenced by Data which is being explicitly held as below via CompilerUsedVars.push_back(Data);.
With DebugInfoCorrelate, now Counters can be optimized away since they are local globals while there are no explicit use.
I think the reason the Coverage case (write only) does not work while the Counter case (read/write) works is simply because the global opt is smart enough to optimize the Coverage case but not the Counter case -- in theory there is no explicit use in the chain of references on Counters (other than compiler-rt/runtime), and the compiler may optimize them.

[ellis]

llvm/lib/Transforms/Instrumentation/InstrProfiling.cpp

1025

Yep, I agree. I have a separate diff D115981 to fix this and will update this code after that lands.

[MaskRay]

I know that you have posted many patches in this area and regular reviewers (e.g. davidxl, vsk) are not involved.
It is a holiday season for many folks so some may not be active.
For some intrusive/significant changes worth getting attention from them before committed, even after you get approval from your colleague(?).

[ellis]

In D116180#3214457, @MaskRay wrote:

I know that you have posted many patches in this area and regular reviewers (e.g. davidxl, vsk) are not involved.
It is a holiday season for many folks so some may not be active.
For some intrusive/significant changes worth getting attention from them before committed, even after you get approval from your colleague(?).

Makes sense to me because I posted this just before the holidays. I can hold off landing this and the previous patch until around the second week of January (I'm taking most of next week off anyway).

[MaskRay]

In D116180#3214466, @ellis wrote:

In D116180#3214457, @MaskRay wrote:

I know that you have posted many patches in this area and regular reviewers (e.g. davidxl, vsk) are not involved.
It is a holiday season for many folks so some may not be active.
For some intrusive/significant changes worth getting attention from them before committed, even after you get approval from your colleague(?).

Makes sense to me because I posted this just before the holidays. I can hold off landing this and the previous patch until around the second week of January (I'm taking most of next week off anyway).

Thanks! They do not subscribe to these changes so they may not even know. (perhaps you mentioned the Differential links on some thread on llvm-dev, but it is pretty easy to miss this piece of information.)
I know that prudently picking reviewers is not easy: you need to do some git log archaeology who contributed/reviewed code in the area and may likely have an opinion.
I remember that they both commented on a previous proposal from you, so they likely have an opinion.

[davidxl]

llvm/docs/LangRef.rst

13140

probably name it llvm.instrprof.entry_cover to reflect the semantics. In theory you can have block level coverage mode too.

llvm/include/llvm/IR/IntrinsicInst.h

1179

Split out this in different patch?

llvm/include/llvm/ProfileData/InstrProfReader.h

484

entry coverage is compatiable with IR pgo -- so perhaps add an assertion somewhere.

llvm/lib/Transforms/Instrumentation/InstrProfiling.cpp

942

Split out the change into a helper function to make it cleaner.

llvm/lib/Transforms/Instrumentation/PGOInstrumentation.cpp

258

make the variable name matching the option name

llvm/test/Transforms/PGOProfile/coverage.ll

12

Just check there is one instance of instrinsic call per function?

llvm/tools/llvm-profdata/llvm-profdata.cpp

2093

Add a test case for the coverage mode.

2159

should probably by pass the rest of dump in coverage mode.

[kyulee]

llvm/lib/Transforms/Instrumentation/InstrProfiling.cpp

947

Can we just use Constant::getAllOnesValue() like Constant::getNullValue() with the array type?

[ellis]

Rebase and split NFC changes in another diff.

[ellis]

compiler-rt/test/profile/instrprof-coverage.c

2

I've added this to the above test instrprof-debug-info-correlate.c.

llvm/docs/LangRef.rst

13140

I agree. Renaming to llvm.instrprof.entry_cover.

llvm/include/llvm/IR/IntrinsicInst.h

1179

Created a parent diff

llvm/lib/Transforms/Instrumentation/InstrProfiling.cpp

947

Unfortunately, ConstantArray::get() takes an array so we cannot give it just a Constant.

https://llvm.org/doxygen/classllvm_1_1ConstantArray.html#a0900dacdc7ad8e3ea0cc92993c7fd422

Also, getAllOnesValue() does not appear to accept an array type yet.

llvm/test/Transforms/PGOProfile/coverage.ll

12

Yeah, this is just a sanity check that it isn't called in other blocks.

llvm/tools/llvm-profdata/llvm-profdata.cpp

2093

Added show-covered.test.

[ellis]

Rebase

[ellis]

Fix class name

[kyulee]

llvm/lib/Transforms/Instrumentation/InstrProfiling.cpp

947

FixedVectorType instead of ArrayType seems the same semantic, which appears to work.

[ellis]

Split change into new function

[ellis]

llvm/lib/Transforms/Instrumentation/InstrProfiling.cpp

947

I would like to avoid using different array types for function entry cover and block counters. From the docs (https://llvm.org/docs/LangRef.html#vector-type) it seems that the vector type is intended for SIMD instructions and we won't them on these globals.

I think a better solution would be to expand getAllOnesValue() to take an array type, but I haven't looked into whether that is realistic.

[ellis]

Rebase

[ellis]

llvm/lib/Transforms/Instrumentation/PGOInstrumentation.cpp

478

I've created a D118097 to refactor this

[phosek]

This change appears to be implementing func-cov mode proposed in https://groups.google.com/g/llvm-dev/c/r03Z6JoN7d4. Do you plan on implementing the support for block-cov and func-cnt modes as well? I'm asking because we're interested in the block-cov mode, but also because I'm somewhat concerned about the orthogonality of these modes in the implementation.

For example, this change introduces the VARIANT_MASK_BYTE_ENTRY_COVERAGE flag to track whether the func-cov mode is used. This means that we're going to need two more bits for block-cov and func-cnt (three in total). Wouldn't it make more sense to instead have VARIANT_MASK_BYTE_COVERAGE and VARIANT_MASK_ENTRY_COVERAGE flags which could be combined to represent all four modes, requiring only two bits in total?

[ellis]

In D116180#3268478, @phosek wrote:

This change appears to be implementing func-cov mode proposed in https://groups.google.com/g/llvm-dev/c/r03Z6JoN7d4. Do you plan on implementing the support for block-cov and func-cnt modes as well? I'm asking because we're interested in the block-cov mode, but also because I'm somewhat concerned about the orthogonality of these modes in the implementation.

For example, this change introduces the VARIANT_MASK_BYTE_ENTRY_COVERAGE flag to track whether the func-cov mode is used. This means that we're going to need two more bits for block-cov and func-cnt (three in total). Wouldn't it make more sense to instead have VARIANT_MASK_BYTE_COVERAGE and VARIANT_MASK_ENTRY_COVERAGE flags which could be combined to represent all four modes, requiring only two bits in total?

I was only planning on adding func-cov. A while back I did a small experiment and found very little binary size difference between block-cov and full block-counts. This is because block-counts can instrument a subset of blocks and infer counts from the rest. With block-cov we cannot do this in the same way so we need to instrument every block. There might be a smarter way to infer some block coverage, but I don't know it.

I decided to use just VARIANT_MASK_BYTE_ENTRY_COVERAGE for simplicity, but I'm open to using two flags like you suggested.

Can I ask why you are interested in block-cov rather than using full block counters? If binary size is a concern then I would suggest inferring coverage from 8 bit counters, assuming overflow isn't a problem. If speed is the main concern then maybe block-cov is the right way to go.

[phosek]

In D116180#3269931, @ellis wrote:

In D116180#3268478, @phosek wrote:

This change appears to be implementing func-cov mode proposed in https://groups.google.com/g/llvm-dev/c/r03Z6JoN7d4. Do you plan on implementing the support for block-cov and func-cnt modes as well? I'm asking because we're interested in the block-cov mode, but also because I'm somewhat concerned about the orthogonality of these modes in the implementation.

For example, this change introduces the VARIANT_MASK_BYTE_ENTRY_COVERAGE flag to track whether the func-cov mode is used. This means that we're going to need two more bits for block-cov and func-cnt (three in total). Wouldn't it make more sense to instead have VARIANT_MASK_BYTE_COVERAGE and VARIANT_MASK_ENTRY_COVERAGE flags which could be combined to represent all four modes, requiring only two bits in total?

I was only planning on adding func-cov. A while back I did a small experiment and found very little binary size difference between block-cov and full block-counts. This is because block-counts can instrument a subset of blocks and infer counts from the rest. With block-cov we cannot do this in the same way so we need to instrument every block. There might be a smarter way to infer some block coverage, but I don't know it.

I decided to use just VARIANT_MASK_BYTE_ENTRY_COVERAGE for simplicity, but I'm open to using two flags like you suggested.

Can I ask why you are interested in block-cov rather than using full block counters? If binary size is a concern then I would suggest inferring coverage from 8 bit counters, assuming overflow isn't a problem. If speed is the main concern then maybe block-cov is the right way to go.

We're mostly concerned about performance so block-cov is still of interest to us, but this is a useful insight, thanks! If it's not too much of an overhead, I'd prefer using the two flags so we can introduce the block-cov mode later.

[ellis]

In D116180#3270200, @phosek wrote:

In D116180#3269931, @ellis wrote:

In D116180#3268478, @phosek wrote:

This change appears to be implementing func-cov mode proposed in https://groups.google.com/g/llvm-dev/c/r03Z6JoN7d4. Do you plan on implementing the support for block-cov and func-cnt modes as well? I'm asking because we're interested in the block-cov mode, but also because I'm somewhat concerned about the orthogonality of these modes in the implementation.

For example, this change introduces the VARIANT_MASK_BYTE_ENTRY_COVERAGE flag to track whether the func-cov mode is used. This means that we're going to need two more bits for block-cov and func-cnt (three in total). Wouldn't it make more sense to instead have VARIANT_MASK_BYTE_COVERAGE and VARIANT_MASK_ENTRY_COVERAGE flags which could be combined to represent all four modes, requiring only two bits in total?

I was only planning on adding func-cov. A while back I did a small experiment and found very little binary size difference between block-cov and full block-counts. This is because block-counts can instrument a subset of blocks and infer counts from the rest. With block-cov we cannot do this in the same way so we need to instrument every block. There might be a smarter way to infer some block coverage, but I don't know it.

I decided to use just VARIANT_MASK_BYTE_ENTRY_COVERAGE for simplicity, but I'm open to using two flags like you suggested.

Can I ask why you are interested in block-cov rather than using full block counters? If binary size is a concern then I would suggest inferring coverage from 8 bit counters, assuming overflow isn't a problem. If speed is the main concern then maybe block-cov is the right way to go.

We're mostly concerned about performance so block-cov is still of interest to us, but this is a useful insight, thanks! If it's not too much of an overhead, I'd prefer using the two flags so we can introduce the block-cov mode later.

Sounds good! I'll use two flags but leave the other cases unimplemented for now.

[ellis]

Use two flags for single byte coverage and function entry coverage.

[ellis]

@davidxl @phosek @kyulee Can I get another look at this?

llvm/docs/LangRef.rst

13140

I renamed it to llvm.instrprof.cover because this would be used for block coverage.

llvm/include/llvm/ProfileData/InstrProfReader.h

484

I added an assertion in PGOInstrumentation.cpp.

[kyulee]

LGTM. I'm approving this but suggest for other reviewers'. Also I left a few comments about naming counters that may follow up.

llvm/lib/Transforms/Instrumentation/InstrProfiling.cpp

700–733

Strictly speaking, it's not a counter for coverage because a store instead of an adder will be used. Same as in the description of intrinsic, and all other api names using counters.
A probe or something might be a better name. However, it seems very invasive to change all the counter name, which may confuse the current (main) semantic for edge counter.
I suggest to just keep as it but I'm also open to other reviewer's opinions.

[davidxl]

compiler-rt/lib/profile/InstrProfilingMerge.c

161

is there a test case for this?

[ellis]

Rebase ontop of D115393

compiler-rt/lib/profile/InstrProfilingMerge.c

161

Added the test instrprof-merge-entry-cover.c and found a bug in __llvm_profile_reset_counters(). Thanks for the reminder!

llvm/lib/Transforms/Instrumentation/InstrProfiling.cpp

700–733

Yeah, I would like to use "probe", but "counters" is used all of the place in the source. I can bring this up in https://llvm.discourse.group/ to see if we are willing to make this change.

[snehasish]

llvm/include/llvm/ProfileData/InstrProf.h

280

At a glance, this looks similar to the BB enum value. If this is coverage only can we make it explicit in the name? Perhaps something like FunctionEntryCoverageOnly. IIUC the comment implies "only instrument the function entry for coverage" as opposed to whether the the entry basic block has instrumentation for profiling.

I like the idea of having better naming for the enum values and I'll send out a separate patch renaming the others once this is submitted.

[ellis]

llvm/include/llvm/ProfileData/InstrProf.h

280

This is actually not for coverage. FunctionEntryOnly only instruments the entry basic block, whereas BB guarantees that the entry basic block is instrumented, along with other blocks.

[snehasish]

llvm/include/llvm/ProfileData/InstrProf.h

280

I see. So should we allow profiles with BB and FunctionEntryOnly to be merged? If not can you update the logic in mergeProfileKind in InstrProfWriter.h?

[ellis]

Make BB and FunctionEntryOnly incompatible.

[ellis]

Fix build error

[MaskRay]

In D116180#3276998, @kyulee wrote:

LGTM. I'm approving this but suggest for other reviewers'. Also I left a few comments about naming counters that may follow up.

Thanks for contributing a new coverage mode. I believe it will be useful to some groups.
That said, process-wise, I think committing this just 6hr after this comment was probably too quick. 6hr did not give stakeholders enough time to read through a major new feature.
Some stakeholders may have other commitments so don't respond timely. But if you gave more time with the patch in an "approved" state, they would probably want to comment.

I think my main question is why does a coverage mode need to piggyback on PGOInstrument.cpp which is mainly for optimizations.

From the functionality implemented by -pgo-function-entry-coverage, I don't see how it can be used for optimization.

[ellis]

In D116180#3558453, @MaskRay wrote:

In D116180#3276998, @kyulee wrote:

LGTM. I'm approving this but suggest for other reviewers'. Also I left a few comments about naming counters that may follow up.

Thanks for contributing a new coverage mode. I believe it will be useful to some groups.
That said, process-wise, I think committing this just 6hr after this comment was probably too quick. 6hr did not give stakeholders enough time to read through a major new feature.
Some stakeholders may have other commitments so don't respond timely. But if you gave more time with the patch in an "approved" state, they would probably want to comment.

I think my main question is why does a coverage mode need to piggyback on PGOInstrument.cpp which is mainly for optimizations.

From the functionality implemented by -pgo-function-entry-coverage, I don't see how it can be used for optimization.

I apologize for moving too quickly to land. In the future, I'll try to allow more time for others to review and respect their time.

As for your question, I've posted this RFC [0] describing our plans for lightweight instrumentation. Basically, we were planning to create a few instrumentation modes that trade off profile precision for a more lightweight instrumented binary. 1) block coverage, 2) function entry count, and 3) function entry coverage. As you've seen, D124490 implements block coverage and the profiles are useful for optimization in D125743 which uses block liveness to guide the machine outliner.

Function entry coverage implemented in this diff was an important step towards block coverage instrumentation. Both modes share a lot of code, which is a big reason why I added this in PGOInstrumentation.cpp. Unlike block coverage instrumentation, function entry coverage is not precise enough to properly guide optimizations, so its main use is to detect dead code.

[0] https://groups.google.com/g/llvm-dev/c/r03Z6JoN7d4

compiler-rt/include/profile/InstrProfData.inc

 #define INSTR_PROF_COVMAP_VERSION 5
 
 /* Profile version is always of type uint64_t. Reserve the upper 8 bits in the
  * version for other variants of profile. We set the lowest bit of the upper 8
  * bits (i.e. bit 56) to 1 to indicate if this is an IR-level instrumentation
  * generated profile, and 0 if this is a Clang FE generated profile.
  * 1 in bit 57 indicates there are context-sensitive records in the profile.
  * The 59th bit indicates whether to use debug info to correlate profiles.
+ * The 60th bit indicates if counters are single bytes representing function
+ * entry coverage.
  */
 #define VARIANT_MASKS_ALL 0xff00000000000000ULL
 #define GET_VERSION(V) ((V) & ~VARIANT_MASKS_ALL)
 #define VARIANT_MASK_IR_PROF (0x1ULL << 56)
 #define VARIANT_MASK_CSIR_PROF (0x1ULL << 57)
 #define VARIANT_MASK_INSTR_ENTRY (0x1ULL << 58)
 #define VARIANT_MASK_DBG_CORRELATE (0x1ULL << 59)
+#define VARIANT_MASK_BYTE_ENTRY_COVERAGE (0x1ULL << 60)
 #define INSTR_PROF_RAW_VERSION_VAR __llvm_profile_raw_version
 #define INSTR_PROF_PROFILE_RUNTIME_VAR __llvm_profile_runtime
 #define INSTR_PROF_PROFILE_COUNTER_BIAS_VAR __llvm_profile_counter_bias
 
 /* The variable that holds the name of the profile data
  * specified via command line. */
 #define INSTR_PROF_PROFILE_NAME_VAR __llvm_profile_filename
 

compiler-rt/lib/profile/InstrProfilingBuffer.c

 
 COMPILER_RT_VISIBILITY
 uint64_t __llvm_profile_get_data_size(const __llvm_profile_data *Begin,
                                       const __llvm_profile_data *End) {
   return __llvm_profile_get_num_data(Begin, End) * sizeof(__llvm_profile_data);
 }
 
 COMPILER_RT_VISIBILITY size_t __llvm_profile_counter_entry_size(void) {
+  if (__llvm_profile_get_version() & VARIANT_MASK_BYTE_ENTRY_COVERAGE)
+    return sizeof(uint8_t);
   return sizeof(uint64_t);
 }
 
 COMPILER_RT_VISIBILITY
 uint64_t __llvm_profile_get_num_counters(const char *Begin, const char *End) {
   intptr_t BeginI = (intptr_t)Begin, EndI = (intptr_t)End;
   return ((EndI + __llvm_profile_counter_entry_size() - 1) - BeginI) /
          __llvm_profile_counter_entry_size();

compiler-rt/lib/profile/InstrProfilingMerge.c

for (SrcData = SrcDataStart,
     // record.
     CountersDelta -= sizeof(*SrcData);
     unsigned NC = SrcData->NumCounters;
     if (NC == 0)
       return 1;
     if (SrcCounters < SrcCountersStart || SrcCounters >= SrcNameStart ||
         (SrcCounters + __llvm_profile_counter_entry_size() * NC) > SrcNameStart)
       return 1;
-    for (unsigned I = 0; I < NC; I++)
+    for (unsigned I = 0; I < NC; I++) {
+      if (__llvm_profile_get_version() & VARIANT_MASK_BYTE_ENTRY_COVERAGE) {
+        // A value of zero signifies the function is covered.
+        DstCounters[I] &= SrcCounters[I];
+      } else {
         ((uint64_t *)DstCounters)[I] += ((uint64_t *)SrcCounters)[I];
+      }
+    }
 
     /* Now merge value profile data. */
     if (!VPMergeHook)
       continue;
 
     for (unsigned I = 0; I <= IPVK_Last; I++)
       NVK += (SrcData->NumValueSites[I] != 0);
 

compiler-rt/test/profile/Darwin/instrprof-debug-info-correlate.c

 // REQUIRES: zlib
 
 // Value profiling is currently not supported in lightweight mode.
 // RUN: %clang_pgogen -o %t -g -mllvm --debug-info-correlate -mllvm --disable-vp=true %S/../Inputs/instrprof-debug-info-correlate-main.cpp %S/../Inputs/instrprof-debug-info-correlate-foo.cpp
 // RUN: env LLVM_PROFILE_FILE=%t.proflite %run %t
 // RUN: llvm-profdata merge -o %t.profdata --debug-info=%t.dSYM %t.proflite
 
 // RUN: %clang_pgogen -o %t.normal -mllvm --disable-vp=true %S/../Inputs/instrprof-debug-info-correlate-main.cpp %S/../Inputs/instrprof-debug-info-correlate-foo.cpp
 // RUN: env LLVM_PROFILE_FILE=%t.profraw %run %t.normal
 // RUN: llvm-profdata merge -o %t.normal.profdata %t.profraw
 
 // RUN: diff %t.normal.profdata %t.profdata
+
+// RUN: %clang_pgogen -o %t.cov -g -mllvm --debug-info-correlate -mllvm -pgo-coverage-instrumentation -mllvm --disable-vp=true %S/../Inputs/instrprof-debug-info-correlate-main.cpp %S/../Inputs/instrprof-debug-info-correlate-foo.cpp
+// RUN: env LLVM_PROFILE_FILE=%t.cov.proflite %run %t.cov
+// RUN: llvm-profdata merge -o %t.cov.profdata --debug-info=%t.cov.dSYM %t.cov.proflite
+
+// RUN: %clang_pgogen -o %t.cov.normal -mllvm --pgo-coverage-instrumentation -mllvm --disable-vp=true %S/../Inputs/instrprof-debug-info-correlate-main.cpp %S/../Inputs/instrprof-debug-info-correlate-foo.cpp
+// RUN: env LLVM_PROFILE_FILE=%t.cov.profraw %run %t.cov.normal
+// RUN: llvm-profdata merge -o %t.cov.normal.profdata %t.cov.profraw
+
+// RUN: diff %t.cov.normal.profdata %t.cov.profdata

compiler-rt/test/profile/Inputs/instrprof-debug-info-correlate-bar.h

 int foo(int);
+int unused(int);
 
 inline int bar(int a) {
   while (a > 100)
     a /= 2;
   return a;
 }

compiler-rt/test/profile/Inputs/instrprof-debug-info-correlate-foo.cpp

 #include "instrprof-debug-info-correlate-bar.h"
 
 int foo(int a) {
   if (a % 2)
     return 4 * a + 1;
   return bar(a);
 }
+
+int unused(int a) { return a * a; }

compiler-rt/test/profile/Inputs/instrprof-debug-info-correlate-main.cpp

 #include "instrprof-debug-info-correlate-bar.h"
 
 typedef int (*FP)(int);
-FP Fps[2] = {foo, bar};
+FP Fps[3] = {foo, bar, unused};
 
 int main() {
   for (int i = 0; i < 5; i++)
     Fps[i % 2](i);
   return 0;
 }

compiler-rt/test/profile/Linux/instrprof-debug-info-correlate.c

 // REQUIRES: zlib
 
 // Value profiling is currently not supported in lightweight mode.
 // RUN: %clang_pgogen -o %t -g -mllvm --debug-info-correlate -mllvm --disable-vp=true %S/../Inputs/instrprof-debug-info-correlate-main.cpp %S/../Inputs/instrprof-debug-info-correlate-foo.cpp
 // RUN: env LLVM_PROFILE_FILE=%t.proflite %run %t
 // RUN: llvm-profdata merge -o %t.profdata --debug-info=%t %t.proflite
 
 // RUN: %clang_pgogen -o %t.normal -mllvm --disable-vp=true %S/../Inputs/instrprof-debug-info-correlate-main.cpp %S/../Inputs/instrprof-debug-info-correlate-foo.cpp
 // RUN: env LLVM_PROFILE_FILE=%t.profraw %run %t.normal
 // RUN: llvm-profdata merge -o %t.normal.profdata %t.profraw
 
 // RUN: diff %t.normal.profdata %t.profdata
+
+// RUN: %clang_pgogen -o %t.cov -g -mllvm --debug-info-correlate -mllvm -pgo-coverage-instrumentation -mllvm --disable-vp=true %S/../Inputs/instrprof-debug-info-correlate-main.cpp %S/../Inputs/instrprof-debug-info-correlate-foo.cpp
+// RUN: env LLVM_PROFILE_FILE=%t.cov.proflite %run %t.cov
+// RUN: llvm-profdata merge -o %t.cov.profdata --debug-info=%t.cov %t.cov.proflite
+
+// RUN: %clang_pgogen -o %t.cov.normal -mllvm --pgo-coverage-instrumentation -mllvm --disable-vp=true %S/../Inputs/instrprof-debug-info-correlate-main.cpp %S/../Inputs/instrprof-debug-info-correlate-foo.cpp
+// RUN: env LLVM_PROFILE_FILE=%t.cov.profraw %run %t.cov.normal
+// RUN: llvm-profdata merge -o %t.cov.normal.profdata %t.cov.profraw
+
+// RUN: diff %t.cov.normal.profdata %t.cov.profdata

compiler-rt/test/profile/instrprof-coverage.c

+// RUN: %clang_pgogen -mllvm -pgo-coverage-instrumentation %s -o %t.out
+// RUN: env LLVM_PROFILE_FILE=%t.profraw %run %t.out
+// RUN: llvm-profdata merge -o %t.profdata %t.profraw
+// RUN: llvm-profdata show --covered %t.profdata | FileCheck %s --check-prefix CHECK --implicit-check-not goo
+
+int foo(int i) { return 4 * i + 1; }
+int bar(int i) { return 4 * i + 2; }
+int goo(int i) { return 4 * i + 3; }
+
+int main(int argc, char *argv[]) {
+  foo(5);
+  argc ? bar(6) : goo(7);
+  return 0;
+}
+
+// CHECK: main
+// CHECK: foo
+// CHECK: bar

llvm/docs/LangRef.rst

 intrinsic.
 
 The last argument specifies the value of the increment of the counter variable.
 
 Semantics:
 """"""""""
 See description of '``llvm.instrprof.increment``' intrinsic.
 
+'``llvm.instrprof.cover``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare void @llvm.instrprof.cover(i8* <name>, i64 <hash>)
+
+Overview:
+"""""""""
+
+The '``llvm.instrprof.cover``' intrinsic is used to implement function entry
+coverage instrumentation.
+
+Arguments:
+""""""""""
+The arguments are the same as the first two arguments of
+'``llvm.instrprof.increment``'.
+
+Semantics:
+""""""""""
+Similar to the '``llvm.instrprof.increment``' intrinsic, but it stores zero to
+the profiling variable to signify that the function has been covered. We store
+zero because this is more efficient on some targets.
 
 '``llvm.instrprof.value.profile``' Intrinsic
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Syntax:
 """""""
 
 ::

llvm/include/llvm/IR/IntrinsicInst.h

public:
   static bool classof(const Value *V) {
     return isa<IntrinsicInst>(V) && classof(cast<IntrinsicInst>(V));
   }
 
   Value *getDest() const { return const_cast<Value *>(getArgOperand(0)); }
   Value *getSrc() const { return const_cast<Value *>(getArgOperand(1)); }
 };
 
-/// This represents the llvm.instrprof_increment intrinsic.
-class InstrProfIncrementInst : public IntrinsicInst {
+/// A base class for all instrprof intrinsics.
+class InstrProfBase : public IntrinsicInst {
 public:
-  static bool classof(const IntrinsicInst *I) {
-    return I->getIntrinsicID() == Intrinsic::instrprof_increment;
-  }
-  static bool classof(const Value *V) {
-    return isa<IntrinsicInst>(V) && classof(cast<IntrinsicInst>(V));
-  }
-
+  // The name of the instrumented function.
   GlobalVariable *getName() const {
     return cast<GlobalVariable>(
         const_cast<Value *>(getArgOperand(0))->stripPointerCasts());
   }
-
+  // The hash of the CFG for the instrumented function.
   ConstantInt *getHash() const {
     return cast<ConstantInt>(const_cast<Value *>(getArgOperand(1)));
   }
+  // The number of counters for the instrumented function.
+  ConstantInt *getNumCounters() const;
+  // The index of the counter that this instruction acts on.
+  Value *getIndex() const;
+};
 
-  ConstantInt *getNumCounters() const {
-    return cast<ConstantInt>(const_cast<Value *>(getArgOperand(2)));
+/// This represents the llvm.instrprof.cover intrinsic.
+class InstrProfCoverInst : public InstrProfBase {
+public:
+  static bool classof(const IntrinsicInst *I) {
+    return I->getIntrinsicID() == Intrinsic::instrprof_cover;
   }
-
-  ConstantInt *getIndex() const {
-    return cast<ConstantInt>(const_cast<Value *>(getArgOperand(3)));
+  static bool classof(const Value *V) {
+    return isa<IntrinsicInst>(V) && classof(cast<IntrinsicInst>(V));
   }
+};
 
+/// This represents the llvm.instrprof.increment intrinsic.
+class InstrProfIncrementInst : public InstrProfBase {
+public:
+  static bool classof(const IntrinsicInst *I) {
+    return I->getIntrinsicID() == Intrinsic::instrprof_increment;
+  }
+  static bool classof(const Value *V) {
+    return isa<IntrinsicInst>(V) && classof(cast<IntrinsicInst>(V));
+  }
   Value *getStep() const;
 };
 
+/// This represents the llvm.instrprof.increment.step intrinsic.
 class InstrProfIncrementInstStep : public InstrProfIncrementInst {
 public:
   static bool classof(const IntrinsicInst *I) {
     return I->getIntrinsicID() == Intrinsic::instrprof_increment_step;
   }
   static bool classof(const Value *V) {
     return isa<IntrinsicInst>(V) && classof(cast<IntrinsicInst>(V));
   }
 };
 
-/// This represents the llvm.instrprof_value_profile intrinsic.
-class InstrProfValueProfileInst : public IntrinsicInst {
+/// This represents the llvm.instrprof.value.profile intrinsic.
+class InstrProfValueProfileInst : public InstrProfBase {
 public:
   static bool classof(const IntrinsicInst *I) {
     return I->getIntrinsicID() == Intrinsic::instrprof_value_profile;
   }
   static bool classof(const Value *V) {
     return isa<IntrinsicInst>(V) && classof(cast<IntrinsicInst>(V));
   }
 
-  GlobalVariable *getName() const {
-    return cast<GlobalVariable>(
-        const_cast<Value *>(getArgOperand(0))->stripPointerCasts());
-  }
-
-  ConstantInt *getHash() const {
-    return cast<ConstantInt>(const_cast<Value *>(getArgOperand(1)));
-  }
-
   Value *getTargetValue() const {
     return cast<Value>(const_cast<Value *>(getArgOperand(2)));
   }
 
   ConstantInt *getValueKind() const {
     return cast<ConstantInt>(const_cast<Value *>(getArgOperand(3)));
   }
 

llvm/include/llvm/IR/Intrinsics.td

def int_experimental_noalias_scope_decl
     : DefaultAttrsIntrinsic<[], [llvm_metadata_ty],
         [IntrInaccessibleMemOnly]>; // blocks LICM and some more
 
 // Stack Protector Intrinsic - The stackprotector intrinsic writes the stack
 // guard to the correct place on the stack frame.
 def int_stackprotector : DefaultAttrsIntrinsic<[], [llvm_ptr_ty, llvm_ptrptr_ty], []>;
 def int_stackguard : DefaultAttrsIntrinsic<[llvm_ptr_ty], [], []>;
 
+// A cover for instrumentation based profiling.
+def int_instrprof_cover : Intrinsic<[], [llvm_ptr_ty, llvm_i64_ty]>;
+
 // A counter increment for instrumentation based profiling.
 def int_instrprof_increment : Intrinsic<[],
                                         [llvm_ptr_ty, llvm_i64_ty,
                                          llvm_i32_ty, llvm_i32_ty]>;
 
 // A counter increment with step for instrumentation based profiling.
 def int_instrprof_increment_step : Intrinsic<[],
                                         [llvm_ptr_ty, llvm_i64_ty,

llvm/include/llvm/ProfileData/InstrProf.h

 /// function's raw name. This should only apply to internal linkage functions
 /// declared by users only.
 void createPGOFuncNameMetadata(Function &F, StringRef PGOFuncName);
 
 /// Check if we can use Comdat for profile variables. This will eliminate
 /// the duplicated profile variables for Comdat functions.
 bool needsComdatForCounter(const Function &F, const Module &M);
 
 const std::error_category &instrprof_category();
 
 enum class instrprof_error {
   success = 0,
   eof,
   unrecognized_format,
   bad_magic,
   bad_header,
   unsupported_version,

 // Parse MemOP Size range option.
 void getMemOPSizeRangeFromOption(StringRef Str, int64_t &RangeStart,
                                  int64_t &RangeLast);
 
 // Create a COMDAT variable INSTR_PROF_RAW_VERSION_VAR to make the runtime
 // aware this is an ir_level profile so it can set the version flag.
 GlobalVariable *createIRLevelProfileFlagVar(Module &M, bool IsCS,
                                             bool InstrEntryBBEnabled,
-                                            bool DebugInfoCorrelate);
+                                            bool DebugInfoCorrelate,
+                                            bool CoverageInstrumentation);
 
 // Create the variable for the profile file name.
 void createProfileFileNameVar(Module &M, StringRef InstrProfileOutput);
 
 // Whether to compress function names in profile records, and filenames in
 // code coverage mappings. Used by the Instrumentation library and unit tests.
 extern cl::opt<bool> DoInstrProfNameCompression;
 
 } // end namespace llvm
 #endif // LLVM_PROFILEDATA_INSTRPROF_H

llvm/include/llvm/ProfileData/InstrProfData.inc

 #define INSTR_PROF_COVMAP_VERSION 5
 
 /* Profile version is always of type uint64_t. Reserve the upper 8 bits in the
  * version for other variants of profile. We set the lowest bit of the upper 8
  * bits (i.e. bit 56) to 1 to indicate if this is an IR-level instrumentation
  * generated profile, and 0 if this is a Clang FE generated profile.
  * 1 in bit 57 indicates there are context-sensitive records in the profile.
  * The 59th bit indicates whether to use debug info to correlate profiles.
+ * The 60th bit indicates if counters are single bytes representing function
+ * entry coverage.
  */
 #define VARIANT_MASKS_ALL 0xff00000000000000ULL
 #define GET_VERSION(V) ((V) & ~VARIANT_MASKS_ALL)
 #define VARIANT_MASK_IR_PROF (0x1ULL << 56)
 #define VARIANT_MASK_CSIR_PROF (0x1ULL << 57)
 #define VARIANT_MASK_INSTR_ENTRY (0x1ULL << 58)
 #define VARIANT_MASK_DBG_CORRELATE (0x1ULL << 59)
+#define VARIANT_MASK_BYTE_ENTRY_COVERAGE (0x1ULL << 60)
 #define INSTR_PROF_RAW_VERSION_VAR __llvm_profile_raw_version
 #define INSTR_PROF_PROFILE_RUNTIME_VAR __llvm_profile_runtime
 #define INSTR_PROF_PROFILE_COUNTER_BIAS_VAR __llvm_profile_counter_bias
 
 /* The variable that holds the name of the profile data
  * specified via command line. */
 #define INSTR_PROF_PROFILE_NAME_VAR __llvm_profile_filename
 

llvm/include/llvm/ProfileData/InstrProfReader.h

public:
 
   virtual bool hasCSIRLevelProfile() const = 0;
 
   virtual bool instrEntryBBEnabled() const = 0;
 
   /// Return true if we must provide debug info to create PGO profiles.
   virtual bool useDebugInfoCorrelate() const { return false; }
 
+  /// Return true if the profile has single byte counters representing function
+  /// entry coverage.
+  virtual bool useSingleByteEntryCoverage() const = 0;
+
   /// Return the PGO symtab. There are three different readers:
   /// Raw, Text, and Indexed profile readers. The first two types
   /// of readers are used only by llvm-profdata tool, while the indexed
   /// profile reader is also used by llvm-cov tool and the compiler (
   /// backend or frontend). Since creating PGO symtab can create
   /// significant runtime and memory overhead (as it touches data
   /// for the whole program), InstrProfSymtab for the indexed profile
   /// reader should be created on demand and it is recommended to be

 private:
   /// The profile data file contents.
   std::unique_ptr<MemoryBuffer> DataBuffer;
   /// Iterator over the profile data.
   line_iterator Line;
   bool IsIRLevelProfile = false;
   bool HasCSIRLevelProfile = false;
   bool InstrEntryBBEnabled = false;
+  bool UseSingleByteEntryCoverage = false;
 
   Error readValueProfileData(InstrProfRecord &Record);
 
 public:
   TextInstrProfReader(std::unique_ptr<MemoryBuffer> DataBuffer_)
       : DataBuffer(std::move(DataBuffer_)), Line(*DataBuffer, true, '#') {}
   TextInstrProfReader(const TextInstrProfReader &) = delete;
   TextInstrProfReader &operator=(const TextInstrProfReader &) = delete;
 
   /// Return true if the given buffer is in text instrprof format.
   static bool hasFormat(const MemoryBuffer &Buffer);
 
   bool isIRLevelProfile() const override { return IsIRLevelProfile; }
 
   bool hasCSIRLevelProfile() const override { return HasCSIRLevelProfile; }
 
   bool instrEntryBBEnabled() const override { return InstrEntryBBEnabled; }
 
+  bool useSingleByteEntryCoverage() const override {
+    return UseSingleByteEntryCoverage;
+  }
+
   /// Read the header.
   Error readHeader() override;
 
   /// Read a single record.
   Error readNextRecord(NamedInstrProfRecord &Record) override;
 
   InstrProfSymtab &getSymtab() override {
     assert(Symtab.get());
public:
   bool instrEntryBBEnabled() const override {
     return (Version & VARIANT_MASK_INSTR_ENTRY) != 0;
   }
 
   bool useDebugInfoCorrelate() const override {
     return (Version & VARIANT_MASK_DBG_CORRELATE) != 0;
   }
 
+  bool useSingleByteEntryCoverage() const override {
+    return (Version & VARIANT_MASK_BYTE_ENTRY_COVERAGE) != 0;
+  }
+
   InstrProfSymtab &getSymtab() override {
     assert(Symtab.get());
     return *Symtab.get();
   }
 
 private:
   Error createSymtab(InstrProfSymtab &Symtab);
   Error readNextHeader(const char *CurrentPos);
const char *getNextHeaderPos() const {
       assert(atEnd());
       return (const char *)ValueDataStart;
   }
 
   StringRef getName(uint64_t NameRef) const {
     return Symtab->getFuncName(swap(NameRef));
   }
 
-  int getCounterTypeSize() const { return sizeof(uint64_t); }
+  int getCounterTypeSize() const {
+    return useSingleByteEntryCoverage() ? sizeof(uint8_t) : sizeof(uint64_t);
+  }
 };
 
 using RawInstrProfReader32 = RawInstrProfReader<uint32_t>;
 using RawInstrProfReader64 = RawInstrProfReader<uint64_t>;
 
 namespace IndexedInstrProf {
 
 enum class HashT : uint32_t;
virtual Error getRecords(StringRef FuncName,
                                      ArrayRef<NamedInstrProfRecord> &Data) = 0;
   virtual void advanceToNextKey() = 0;
   virtual bool atEnd() const = 0;
   virtual void setValueProfDataEndianness(support::endianness Endianness) = 0;
   virtual uint64_t getVersion() const = 0;
   virtual bool isIRLevelProfile() const = 0;
   virtual bool hasCSIRLevelProfile() const = 0;
   virtual bool instrEntryBBEnabled() const = 0;
+  virtual bool useSingleByteEntryCoverage() const = 0;
   virtual Error populateSymtab(InstrProfSymtab &) = 0;
 };
 
 using OnDiskHashTableImplV3 =
     OnDiskIterableChainedHashTable<InstrProfLookupTrait>;
 
 template <typename HashTableImpl>
 class InstrProfReaderItaniumRemapper;
public:
   bool hasCSIRLevelProfile() const override {
     return (FormatVersion & VARIANT_MASK_CSIR_PROF) != 0;
   }
 
   bool instrEntryBBEnabled() const override {
     return (FormatVersion & VARIANT_MASK_INSTR_ENTRY) != 0;
   }
 
+  bool useSingleByteEntryCoverage() const override {
+    return (FormatVersion & VARIANT_MASK_BYTE_ENTRY_COVERAGE) != 0;
+  }
+
   Error populateSymtab(InstrProfSymtab &Symtab) override {
     return Symtab.create(HashTable->keys());
   }
 };
 
 /// Name matcher supporting fuzzy matching of symbol names to names in profiles.
 class InstrProfReaderRemapper {
 public:
public:
   bool hasCSIRLevelProfile() const override {
     return Index->hasCSIRLevelProfile();
   }
 
   bool instrEntryBBEnabled() const override {
     return Index->instrEntryBBEnabled();
   }
 
+  bool useSingleByteEntryCoverage() const override {
+    return Index->useSingleByteEntryCoverage();
+  }
+
   /// Return true if the given buffer is in an indexed instrprof format.
   static bool hasFormat(const MemoryBuffer &DataBuffer);
 
   /// Read the file header.
   Error readHeader() override;
   /// Read a single record.
   Error readNextRecord(NamedInstrProfRecord &Record) override;
 

llvm/include/llvm/ProfileData/InstrProfWriter.h

public:
   // PF_IRLevelWithCS is the profile from context sensitive IR instrumentation.
   enum ProfKind { PF_Unknown = 0, PF_FE, PF_IRLevel, PF_IRLevelWithCS };
 
 private:
   bool Sparse;
   StringMap<ProfilingData> FunctionData;
   ProfKind ProfileKind = PF_Unknown;
   bool InstrEntryBBEnabled;
+  bool UseSingleByteEntryCoverage;
   // Use raw pointer here for the incomplete type object.
   InstrProfRecordWriterTrait *InfoObj;
 
 public:
-  InstrProfWriter(bool Sparse = false, bool InstrEntryBBEnabled = false);
+  InstrProfWriter(bool Sparse = false, bool InstrEntryBBEnabled = false,
+                  bool UseSingleByteEntryCoverage = false);
   ~InstrProfWriter();
 
   StringMap<ProfilingData> &getProfileData() { return FunctionData; }
 
   /// Add function counts for the given function. If there are already counts
   /// for this function and the hash and number of counts match, each counter is
   /// summed. Optionally scale counts by \p Weight.
   void addRecord(NamedInstrProfRecord &&I, uint64_t Weight,
Error setIsIRLevelProfile(bool IsIRLevel, bool WithCS) {
     // profile, set the kind to PF_IRLevelWithCS.
     if (ProfileKind == PF_IRLevel && WithCS)
       ProfileKind = PF_IRLevelWithCS;
 
     return Error::success();
   }
 
   void setInstrEntryBBEnabled(bool Enabled) { InstrEntryBBEnabled = Enabled; }
+  void setUseSingleByteEntryCoverage(bool Enabled) {
+    UseSingleByteEntryCoverage = Enabled;
+  }
   // Internal interface for testing purpose only.
   void setValueProfDataEndianness(support::endianness Endianness);
   void setOutputSparse(bool Sparse);
   // Compute the overlap b/w this object and Other. Program level result is
   // stored in Overlap and function level result is stored in FuncLevelOverlap.
   void overlapRecord(NamedInstrProfRecord &&Other, OverlapStats &Overlap,
                      OverlapStats &FuncLevelOverlap,
                      const OverlapFuncFilters &FuncFilter);

llvm/include/llvm/Transforms/Instrumentation/InstrProfiling.h

 /// pass.
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_TRANSFORMS_INSTRUMENTATION_INSTRPROFILING_H
 #define LLVM_TRANSFORMS_INSTRUMENTATION_INSTRPROFILING_H
 
 #include "llvm/ADT/DenseMap.h"
 #include "llvm/ADT/StringRef.h"
+#include "llvm/IR/IRBuilder.h"
 #include "llvm/IR/IntrinsicInst.h"
 #include "llvm/IR/PassManager.h"
 #include "llvm/ProfileData/InstrProf.h"
 #include "llvm/Transforms/Instrumentation.h"
 #include <cstddef>
 #include <cstdint>
 #include <cstring>
 #include <vector>
private:
   bool isRuntimeCounterRelocationEnabled() const;
 
   /// Returns true if profile counter update register promotion is enabled.
   bool isCounterPromotionEnabled() const;
 
   /// Count the number of instrumented value sites for the function.
   void computeNumValueSiteCounts(InstrProfValueProfileInst *Ins);
 
-  /// Replace instrprof_value_profile with a call to runtime library.
+  /// Replace instrprof.value.profile with a call to runtime library.
   void lowerValueProfileInst(InstrProfValueProfileInst *Ins);
 
-  /// Replace instrprof_increment with an increment of the appropriate value.
+  /// Replace instrprof.cover with a store instruction to the coverage byte.
+  void lowerCover(InstrProfCoverInst *Inc);
+
+  /// Replace instrprof.increment with an increment of the appropriate value.
   void lowerIncrement(InstrProfIncrementInst *Inc);
 
   /// Force emitting of name vars for unused functions.
   void lowerCoverageData(GlobalVariable *CoverageNamesVar);
 
+  /// Compute the address of the counter value that this profiling instruction
+  /// acts on.
+  Value *getCounterAddress(InstrProfBase *I);
+
   /// Get the region counters for an increment, creating them if necessary.
   ///
   /// If the counter array doesn't yet exist, the profile data variables
   /// referring to them will also be created.
-  GlobalVariable *getOrCreateRegionCounters(InstrProfIncrementInst *Inc);
+  GlobalVariable *getOrCreateRegionCounters(InstrProfBase *Inc);
 
   /// Emit the section with compressed function names.
   void emitNameData();
 
   /// Emit value nodes section for value profiling.
   void emitVNodes();
 
   /// Emit runtime registration functions for each profile data variable.

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

case Intrinsic::experimental_gc_statepoint:
     LowerStatepoint(cast<GCStatepointInst>(I));
     return;
   case Intrinsic::experimental_gc_result:
     visitGCResult(cast<GCResultInst>(I));
     return;
   case Intrinsic::experimental_gc_relocate:
     visitGCRelocate(cast<GCRelocateInst>(I));
     return;
+  case Intrinsic::instrprof_cover:
+    llvm_unreachable("instrprof failed to lower a cover");
   case Intrinsic::instrprof_increment:
     llvm_unreachable("instrprof failed to lower an increment");
   case Intrinsic::instrprof_value_profile:
     llvm_unreachable("instrprof failed to lower a value profiling call");
   case Intrinsic::localescape: {
     MachineFunction &MF = DAG.getMachineFunction();
     const TargetInstrInfo *TII = DAG.getSubtarget().getInstrInfo();
 

llvm/lib/IR/IntrinsicInst.cpp

if (LastLow == NameTable.end())
     return -1;
   StringRef NameFound = *LastLow;
   if (Name == NameFound ||
       (Name.startswith(NameFound) && Name[NameFound.size()] == '.'))
     return LastLow - NameTable.begin();
   return -1;
 }
 
+ConstantInt *InstrProfBase::getNumCounters() const {
+  if (InstrProfCoverInst::classof(this)) {
+    const auto *M = getModule();
+    return ConstantInt::get(Type::getInt64Ty(M->getContext()), 1);
+  } else if (InstrProfValueProfileInst::classof(this)) {
+    llvm_unreachable("InstrProfValueProfileInst does not have counters!");
+  }
+  return cast<ConstantInt>(const_cast<Value *>(getArgOperand(2)));
+}
+
+Value *InstrProfBase::getIndex() const {
+  if (InstrProfCoverInst::classof(this)) {
+    const auto *M = getModule();
+    return ConstantInt::get(Type::getInt64Ty(M->getContext()), 0);
+  } else if (InstrProfValueProfileInst::classof(this)) {
+    llvm_unreachable("Please use InstrProfValueProfileInst::getIndex()");
+  }
+  return const_cast<Value *>(getArgOperand(3));
+}
+
 Value *InstrProfIncrementInst::getStep() const {
   if (InstrProfIncrementInstStep::classof(this)) {
     return const_cast<Value *>(getArgOperand(4));
   }
   const Module *M = getModule();
   LLVMContext &Context = M->getContext();
   return ConstantInt::get(Type::getInt64Ty(Context), 1);
 }

llvm/lib/ProfileData/InstrProf.cpp

bool canRenameComdatFunc(const Function &F, bool CheckAddressTaken) {
   }
   return true;
 }
 
 // Create a COMDAT variable INSTR_PROF_RAW_VERSION_VAR to make the runtime
 // aware this is an ir_level profile so it can set the version flag.
 GlobalVariable *createIRLevelProfileFlagVar(Module &M, bool IsCS,
                                             bool InstrEntryBBEnabled,
-                                            bool DebugInfoCorrelate) {
+                                            bool DebugInfoCorrelate,
+                                            bool CoverageInstrumentation) {
   const StringRef VarName(INSTR_PROF_QUOTE(INSTR_PROF_RAW_VERSION_VAR));
   Type *IntTy64 = Type::getInt64Ty(M.getContext());
   uint64_t ProfileVersion = (INSTR_PROF_RAW_VERSION | VARIANT_MASK_IR_PROF);
   if (IsCS)
     ProfileVersion |= VARIANT_MASK_CSIR_PROF;
   if (InstrEntryBBEnabled)
     ProfileVersion |= VARIANT_MASK_INSTR_ENTRY;
   if (DebugInfoCorrelate)
     ProfileVersion |= VARIANT_MASK_DBG_CORRELATE;
+  if (CoverageInstrumentation)
+    ProfileVersion |= VARIANT_MASK_BYTE_ENTRY_COVERAGE;
   auto IRLevelVersionVariable = new GlobalVariable(
       M, IntTy64, true, GlobalValue::WeakAnyLinkage,
       Constant::getIntegerValue(IntTy64, APInt(64, ProfileVersion)), VarName);
   IRLevelVersionVariable->setVisibility(GlobalValue::DefaultVisibility);
   Triple TT(M.getTargetTriple());
   if (TT.supportsCOMDAT()) {
     IRLevelVersionVariable->setLinkage(GlobalValue::ExternalLinkage);
     IRLevelVersionVariable->setComdat(M.getOrInsertComdat(VarName));

llvm/lib/ProfileData/InstrProfReader.cpp

return error(instrprof_error::malformed,
                  ("number of counters " + Twine(NumCounters) +
                   " is greater than the maximum number of counters " +
                   Twine(MaxNumCounters))
                      .str());
 
   Record.Counts.clear();
   Record.Counts.reserve(NumCounters);
   for (uint32_t I = 0; I < NumCounters; I++) {
-    const auto *CounterValue = reinterpret_cast<const uint64_t *>(
-        CountersStart + CounterBaseOffset + I * getCounterTypeSize());
+    const char *Ptr =
+        CountersStart + CounterBaseOffset + I * getCounterTypeSize();
+    if (useSingleByteEntryCoverage()) {
+      // A value of zero signifies the block is covered.
+      Record.Counts.push_back(*Ptr == 0 ? 1 : 0);
+    } else {
+      const auto *CounterValue = reinterpret_cast<const uint64_t *>(Ptr);
       Record.Counts.push_back(swap(*CounterValue));
     }
+  }
 
   return success();
 }
 
 template <class IntPtrT>
 Error RawInstrProfReader<IntPtrT>::readValueProfilingData(
     InstrProfRecord &Record) {
   Record.clearValueData();

llvm/lib/ProfileData/InstrProfWriter.cpp

for (const auto &ProfileData : *V) {
       VDataPtr->swapBytesFromHost(ValueProfDataEndianness);
       Out.write((const char *)VDataPtr.get(), S);
     }
   }
 };
 
 } // end namespace llvm
 
-InstrProfWriter::InstrProfWriter(bool Sparse, bool InstrEntryBBEnabled)
+InstrProfWriter::InstrProfWriter(bool Sparse, bool InstrEntryBBEnabled,
+                                 bool UseSingleByteEntryCoverage)
     : Sparse(Sparse), InstrEntryBBEnabled(InstrEntryBBEnabled),
+      UseSingleByteEntryCoverage(UseSingleByteEntryCoverage),
       InfoObj(new InstrProfRecordWriterTrait()) {}
 
 InstrProfWriter::~InstrProfWriter() { delete InfoObj; }
 
 // Internal interface for testing purpose only.
 void InstrProfWriter::setValueProfDataEndianness(
     support::endianness Endianness) {
   InfoObj->ValueProfDataEndianness = Endianness;
Error InstrProfWriter::writeImpl(ProfOStream &OS) {
   if (ProfileKind == PF_IRLevel)
     Header.Version |= VARIANT_MASK_IR_PROF;
   if (ProfileKind == PF_IRLevelWithCS) {
     Header.Version |= VARIANT_MASK_IR_PROF;
     Header.Version |= VARIANT_MASK_CSIR_PROF;
   }
   if (InstrEntryBBEnabled)
     Header.Version |= VARIANT_MASK_INSTR_ENTRY;
+  if (UseSingleByteEntryCoverage)
+    Header.Version |= VARIANT_MASK_BYTE_ENTRY_COVERAGE;
 
   Header.Unused = 0;
   Header.HashType = static_cast<uint64_t>(IndexedInstrProf::HashType);
   Header.HashOffset = 0;
   int N = sizeof(IndexedInstrProf::Header) / sizeof(uint64_t);
 
   // Only write out all the fields except 'HashOffset'. We need
   // to remember the offset of that field to allow back patching

llvm/lib/Transforms/Instrumentation/InstrProfiling.cpp

 #include "llvm/IR/Constants.h"
 #include "llvm/IR/DIBuilder.h"
 #include "llvm/IR/DerivedTypes.h"
 #include "llvm/IR/DiagnosticInfo.h"
 #include "llvm/IR/Dominators.h"
 #include "llvm/IR/Function.h"
 #include "llvm/IR/GlobalValue.h"
 #include "llvm/IR/GlobalVariable.h"
-#include "llvm/IR/IRBuilder.h"
 #include "llvm/IR/Instruction.h"
 #include "llvm/IR/Instructions.h"
 #include "llvm/IR/IntrinsicInst.h"
 #include "llvm/IR/Module.h"
 #include "llvm/IR/Type.h"
 #include "llvm/InitializePasses.h"
 #include "llvm/Pass.h"
 #include "llvm/ProfileData/InstrProf.h"
INITIALIZE_PASS_END(InstrProfilingLegacyPass, "instrprof",
                     false)
 
 ModulePass *
 llvm::createInstrProfilingLegacyPass(const InstrProfOptions &Options,
                                      bool IsCS) {
   return new InstrProfilingLegacyPass(Options, IsCS);
 }
 
-static InstrProfIncrementInst *castToIncrementInst(Instruction *Instr) {
-  InstrProfIncrementInst *Inc = dyn_cast<InstrProfIncrementInstStep>(Instr);
-  if (Inc)
-    return Inc;
-  return dyn_cast<InstrProfIncrementInst>(Instr);
-}
-
 bool InstrProfiling::lowerIntrinsics(Function *F) {
   bool MadeChange = false;
   PromotionCandidates.clear();
   for (BasicBlock &BB : *F) {
     for (Instruction &Instr : llvm::make_early_inc_range(BB)) {
-      InstrProfIncrementInst *Inc = castToIncrementInst(&Instr);
-      if (Inc) {
-        lowerIncrement(Inc);
+      if (auto *IPC = dyn_cast<InstrProfCoverInst>(&Instr)) {
+        lowerCover(IPC);
+        MadeChange = true;
+      } else if (auto *IPI = dyn_cast<InstrProfIncrementInst>(&Instr)) {
+        lowerIncrement(IPI);
+        MadeChange = true;
+      } else if (auto *IPIS = dyn_cast<InstrProfIncrementInstStep>(&Instr)) {
+        lowerIncrement(IPIS);
         MadeChange = true;
-      } else if (auto *Ind = dyn_cast<InstrProfValueProfileInst>(&Instr)) {
-        lowerValueProfileInst(Ind);
+      } else if (auto *IPVP = dyn_cast<InstrProfValueProfileInst>(&Instr)) {
+        lowerValueProfileInst(IPVP);
         MadeChange = true;
       }
     }
   }
 
   if (!MadeChange)
     return false;
 
static bool needsRuntimeHookUnconditionally(const Triple &TT) {
   if (TT.isOSFuchsia())
     return false;
 
   return true;
 }
 
 /// Check if the module contains uses of any profiling intrinsics.
 static bool containsProfilingIntrinsics(Module &M) {
-  if (auto *F = M.getFunction(
-          Intrinsic::getName(llvm::Intrinsic::instrprof_increment)))
-    if (!F->use_empty())
-      return true;
-  if (auto *F = M.getFunction(
-          Intrinsic::getName(llvm::Intrinsic::instrprof_increment_step)))
-    if (!F->use_empty())
-      return true;
-  if (auto *F = M.getFunction(
-          Intrinsic::getName(llvm::Intrinsic::instrprof_value_profile)))
-    if (!F->use_empty())
-      return true;
+  auto containsIntrinsic = [&](int ID) {
+    if (auto *F = M.getFunction(Intrinsic::getName(ID)))
+      return !F->use_empty();
     return false;
+  };
+  return containsIntrinsic(llvm::Intrinsic::instrprof_cover) ||
+         containsIntrinsic(llvm::Intrinsic::instrprof_increment) ||
+         containsIntrinsic(llvm::Intrinsic::instrprof_increment_step) ||
+         containsIntrinsic(llvm::Intrinsic::instrprof_value_profile);
 }
 
 bool InstrProfiling::run(
     Module &M, std::function<const TargetLibraryInfo &(Function &F)> GetTLI) {
   this->M = &M;
   this->GetTLI = std::move(GetTLI);
   NamesVar = nullptr;
   NamesSize = 0;
Call = Builder.CreateCall(
         Args, OpBundles);
   }
   if (auto AK = TLI->getExtAttrForI32Param(false))
     Call->addParamAttr(2, AK);
   Ind->replaceAllUsesWith(Call);
   Ind->eraseFromParent();
 }
 
-void InstrProfiling::lowerIncrement(InstrProfIncrementInst *Inc) {
-  GlobalVariable *Counters = getOrCreateRegionCounters(Inc);
+Value *InstrProfiling::getCounterAddress(InstrProfBase *I) {
+  auto *Counters = getOrCreateRegionCounters(I);
+  IRBuilder<> Builder(I);
+
+  Value *Addr;
+  if (auto *ConstIndex = dyn_cast<ConstantInt>(I->getIndex())) {
+    uint32_t IndexVal = ConstIndex->getZExtValue();
+    Addr = Builder.CreateConstInBoundsGEP2_32(Counters->getValueType(),
+                                              Counters, 0, IndexVal);
+  } else {
+    Addr = Builder.CreateInBoundsGEP(Counters->getValueType(), Counters,
+                                     {Builder.getInt32(0), I->getIndex()});
+  }
 
-  IRBuilder<> Builder(Inc);
-  uint64_t Index = Inc->getIndex()->getZExtValue();
-  Value *Addr = Builder.CreateConstInBoundsGEP2_32(Counters->getValueType(),
-                                                   Counters, 0, Index);
+  if (!isRuntimeCounterRelocationEnabled())
+    return Addr;
 
-  if (isRuntimeCounterRelocationEnabled()) {
-    Type *Int64Ty = Type::getInt64Ty(M->getContext());
-    Type *Int64PtrTy = Type::getInt64PtrTy(M->getContext());
-    Function *Fn = Inc->getParent()->getParent();
-    Instruction &I = Fn->getEntryBlock().front();
-    LoadInst *LI = dyn_cast<LoadInst>(&I);
-    if (!LI) {
-      IRBuilder<> Builder(&I);
-      GlobalVariable *Bias =
-          M->getGlobalVariable(getInstrProfCounterBiasVarName());
-      if (!Bias) {
-        // Compiler must define this variable when runtime counter relocation
-        // is being used. Runtime has a weak external reference that is used
-        // to check whether that's the case or not.
-        Bias = new GlobalVariable(
-            *M, Int64Ty, false, GlobalValue::LinkOnceODRLinkage,
-            Constant::getNullValue(Int64Ty), getInstrProfCounterBiasVarName());
-        Bias->setVisibility(GlobalVariable::HiddenVisibility);
-        // A definition that's weak (linkonce_odr) without being in a COMDAT
-        // section wouldn't lead to link errors, but it would lead to a dead
-        // data word from every TU but one. Putting it in COMDAT ensures there
-        // will be exactly one data slot in the link.
-        if (TT.supportsCOMDAT())
-          Bias->setComdat(M->getOrInsertComdat(Bias->getName()));
-      }
-      LI = Builder.CreateLoad(Int64Ty, Bias);
-    }
-    auto *Add = Builder.CreateAdd(Builder.CreatePtrToInt(Addr, Int64Ty), LI);
-    Addr = Builder.CreateIntToPtr(Add, Int64PtrTy);
-  }
+  Function *Fn = I->getParent()->getParent();
+  Instruction &EntryI = Fn->getEntryBlock().front();
+  LoadInst *LI = dyn_cast<LoadInst>(&EntryI);
+    IRBuilder<> EntryBuilder(&EntryI);
+    auto *Bias = M->getGlobalVariable(getInstrProfCounterBiasVarName());
+    LI = EntryBuilder.CreateLoad(Int64Ty, Bias);
+  return Builder.CreateIntToPtr(Add, Addr->getType());
 
+void InstrProfiling::lowerCover(InstrProfCoverInst *CoverInstruction) {
+  auto *Addr = getCounterAddress(CoverInstruction);
+  // We store zero to represent that this block is covered.
+  IRBuilder<> Builder(CoverInstruction);
+  Builder.CreateStore(Builder.getInt8(0), Addr);
+  CoverInstruction->eraseFromParent();
+}
+
+void InstrProfiling::lowerIncrement(InstrProfIncrementInst *Inc) {
+  auto *Addr = getCounterAddress(Inc);
+
+  IRBuilder<> Builder(Inc);
+  auto Index = cast<Constant>(Inc->getIndex());
   if (Options.Atomic || AtomicCounterUpdateAll ||
-      (Index == 0 && AtomicFirstCounter)) {
+      (Index->isZeroValue() && AtomicFirstCounter)) {
     Builder.CreateAtomicRMW(AtomicRMWInst::Add, Addr, Inc->getStep(),
                             MaybeAlign(), AtomicOrdering::Monotonic);
   } else {
     Value *IncStep = Inc->getStep();
     Value *Load = Builder.CreateLoad(IncStep->getType(), Addr, "pgocount");
     auto *Count = Builder.CreateAdd(Load, Inc->getStep());
     auto *Store = Builder.CreateStore(Count, Addr);
     if (isCounterPromotionEnabled())
for (unsigned I = 0, E = Names->getNumOperands(); I < E; ++I) {
     Name->setLinkage(GlobalValue::PrivateLinkage);
     ReferencedNames.push_back(Name);
     NC->dropAllReferences();
   }
   CoverageNamesVar->eraseFromParent();
 }
 
 /// Get the name of a profiling variable for a particular function.
-static std::string getVarName(InstrProfIncrementInst *Inc, StringRef Prefix,
+static std::string getVarName(InstrProfBase *Inc, StringRef Prefix,
                               bool &Renamed) {
   StringRef NamePrefix = getInstrProfNameVarPrefix();
   StringRef Name = Inc->getName()->getName().substr(NamePrefix.size());
   Function *F = Inc->getParent()->getParent();
   Module *M = F->getParent();
   if (!DoHashBasedCounterSplit || !isIRPGOFlagSet(M) ||
       !canRenameComdatFunc(*F)) {
     Renamed = false;
static bool needsRuntimeRegistrationOfSectionRange(const Triple &TT) {
   // Use linker script magic to get data/cnts/name start/end.
   if (TT.isOSLinux() || TT.isOSFreeBSD() || TT.isOSNetBSD() ||
       TT.isOSSolaris() || TT.isOSFuchsia() || TT.isPS4CPU() || TT.isOSWindows())
     return false;
 
   return true;
 }
 
-GlobalVariable *
-InstrProfiling::getOrCreateRegionCounters(InstrProfIncrementInst *Inc) {
+GlobalVariable *InstrProfiling::getOrCreateRegionCounters(InstrProfBase *Inc) {
   GlobalVariable *NamePtr = Inc->getName();
   auto &PD = ProfileDataMap[NamePtr];
   if (PD.RegionCounters)
     return PD.RegionCounters;
 
   // Match the linkage and visibility of the name global.
   Function *Fn = Inc->getParent()->getParent();
   GlobalValue::LinkageTypes Linkage = NamePtr->getLinkage();
if (UseComdat) {
       if (!NeedComdat)
         C->setSelectionKind(Comdat::NoDeduplicate);
       GV->setComdat(C);
     }
   };
 
   uint64_t NumCounters = Inc->getNumCounters()->getZExtValue();
   LLVMContext &Ctx = M->getContext();
-  ArrayType *CounterTy = ArrayType::get(Type::getInt64Ty(Ctx), NumCounters);
 
   // Create the counters variable.
-  auto *CounterPtr =
+  GlobalVariable *CounterPtr;
+  if (isa<InstrProfCoverInst>(Inc)) {
+    auto *CounterTy = Type::getInt8Ty(Ctx);
+    auto *CounterArrTy = ArrayType::get(CounterTy, NumCounters);
+    // Initialize to all ones to signify that all blocks are uncovered.
+    std::vector<Constant *> InitialValues(NumCounters,
+                                          Constant::getAllOnesValue(CounterTy));
+    CounterPtr = new GlobalVariable(
+        *M, CounterArrTy, false, Linkage,
+        ConstantArray::get(CounterArrTy, InitialValues), CntsVarName);
+    CounterPtr->setAlignment(Align(1));
+  } else {
+    auto *CounterTy = ArrayType::get(Type::getInt64Ty(Ctx), NumCounters);
+    CounterPtr =
         new GlobalVariable(*M, CounterTy, false, Linkage,
                            Constant::getNullValue(CounterTy), CntsVarName);
+    CounterPtr->setAlignment(Align(8));
+  }
   CounterPtr->setVisibility(Visibility);
   CounterPtr->setSection(
       getInstrProfSectionName(IPSK_cnts, TT.getObjectFormat()));
-  CounterPtr->setAlignment(Align(8));
   MaybeSetComdat(CounterPtr);
   CounterPtr->setLinkage(Linkage);
   PD.RegionCounters = CounterPtr;
   if (DebugInfoCorrelate) {
     if (auto *SP = Fn->getSubprogram()) {
       DIBuilder DB(*M, true, SP->getUnit());
       Metadata *FunctionNameAnnotation[] = {
           MDString::get(Ctx, InstrProfCorrelator::FunctionNameAttributeName),
if (NS > 0 && ValueProfileStaticAlloc &&
     ValuesVar->setSection(
         getInstrProfSectionName(IPSK_vals, TT.getObjectFormat()));
     ValuesVar->setAlignment(Align(8));
     MaybeSetComdat(ValuesVar);
     ValuesPtrExpr =
         ConstantExpr::getBitCast(ValuesVar, Type::getInt8PtrTy(Ctx));
   }
 
-  if (DebugInfoCorrelate)
+  if (DebugInfoCorrelate) {
+    if (isa<InstrProfCoverInst>(Inc)) {
+      // Mark this global as used so optimizations don't remove it since it is
+      // never read.
+      CompilerUsedVars.push_back(PD.RegionCounters);
+    }
     return PD.RegionCounters;
+  }
 
   // Create data variable.
   auto *IntPtrTy = M->getDataLayout().getIntPtrType(M->getContext());
   auto *Int16Ty = Type::getInt16Ty(Ctx);
   auto *Int16ArrayTy = ArrayType::get(Int16Ty, IPVK_Last + 1);
   Type *DataTypes[] = {
 #define INSTR_PROF_DATA(Type, LLVMType, Name, Init) LLVMType,
 #include "llvm/ProfileData/InstrProfData.inc"

llvm/lib/Transforms/Instrumentation/PGOInstrumentation.cpp

EmitBranchProbability("pgo-emit-branch-prob", cl::init(false), cl::Hidden,
                                    "branch probability will be emitted as "
                                    "optimization remarks: -{Rpass|"
                                    "pass-remarks}=pgo-instrumentation"));
 
 static cl::opt<bool> PGOInstrumentEntry(
     "pgo-instrument-entry", cl::init(false), cl::Hidden,
     cl::desc("Force to instrument function entry basicblock."));
 
+static cl::opt<bool> CoverageInstrumentation(
+    "pgo-coverage-instrumentation", cl::init(false), cl::Hidden, cl::ZeroOrMore,
+    cl::desc("Use this option to enable function coverage instrumentation."));
+
 static cl::opt<bool>
     PGOFixEntryCount("pgo-fix-entry-count", cl::init(true), cl::Hidden,
                      cl::desc("Fix function entry count in profile use."));
 
 static cl::opt<bool> PGOVerifyHotBFI(
     "pgo-verify-hot-bfi", cl::init(false), cl::Hidden,
     cl::desc("Print out the non-match BFI count if a hot raw profile count "
              "becomes non-hot, or a cold raw profile count becomes hot. "
initializePGOInstrumentationGenCreateVarLegacyPassPass(
         *PassRegistry::getPassRegistry());
   }
 
 private:
   bool runOnModule(Module &M) override {
     createProfileFileNameVar(M, InstrProfileOutput);
     // The variable in a comdat may be discarded by LTO. Ensure the
     // declaration will be retained.
-    appendToCompilerUsed(M, createIRLevelProfileFlagVar(M, /*IsCS=*/true,
-                                                        PGOInstrumentEntry,
-                                                        DebugInfoCorrelate));
+    appendToCompilerUsed(M, createIRLevelProfileFlagVar(
+                                M, /*IsCS=*/true, PGOInstrumentEntry,
+                                DebugInfoCorrelate, CoverageInstrumentation));
     return false;
   }
   std::string InstrProfileOutput;
 };
 
 } // end anonymous namespace
 
 char PGOInstrumentationGenLegacyPass::ID = 0;
static void instrumentOneFunc(
     std::unordered_multimap<Comdat *, GlobalValue *> &ComdatMembers,
     bool IsCS) {
   // Split indirectbr critical edges here before computing the MST rather than
   // later in getInstrBB() to avoid invalidating it.
   SplitIndirectBrCriticalEdges(F, BPI, BFI);
 
   FuncPGOInstrumentation<PGOEdge, BBInfo> FuncInfo(
       F, TLI, ComdatMembers, true, BPI, BFI, IsCS, PGOInstrumentEntry);
+
+  Type *I8PtrTy = Type::getInt8PtrTy(M->getContext());
+  auto Name = ConstantExpr::getBitCast(FuncInfo.FuncNameVar, I8PtrTy);
+  auto CFGHash = ConstantInt::get(Type::getInt64Ty(M->getContext()),
+                                  FuncInfo.FunctionHash);
+  if (CoverageInstrumentation) {
+    auto &EntryBB = F.getEntryBlock();
+    IRBuilder<> Builder(&EntryBB, EntryBB.getFirstInsertionPt());
+    Builder.CreateCall(Intrinsic::getDeclaration(M, Intrinsic::instrprof_cover),
+                       {Name, CFGHash});
+    return;
+  }
+
   std::vector<BasicBlock *> InstrumentBBs;
   FuncInfo.getInstrumentBBs(InstrumentBBs);
   unsigned NumCounters =
       InstrumentBBs.size() + FuncInfo.SIVisitor.getNumOfSelectInsts();
 
   uint32_t I = 0;
-  Type *I8PtrTy = Type::getInt8PtrTy(M->getContext());
   for (auto *InstrBB : InstrumentBBs) {
     IRBuilder<> Builder(InstrBB, InstrBB->getFirstInsertionPt());
     assert(Builder.GetInsertPoint() != InstrBB->end() &&
            "Cannot get the Instrumentation point");
+    auto Index = Builder.getInt32(I++);
+    // llvm.instrprof.increment(i8* <name>, i64 <hash>, i32 <num-counters>,
+    //                          i32 <index>)
     Builder.CreateCall(
         Intrinsic::getDeclaration(M, Intrinsic::instrprof_increment),
-        {ConstantExpr::getBitCast(FuncInfo.FuncNameVar, I8PtrTy),
-         Builder.getInt64(FuncInfo.FunctionHash), Builder.getInt32(NumCounters),
-         Builder.getInt32(I++)});
+        {Name, CFGHash, Builder.getInt32(NumCounters), Index});
   }
 
   // Now instrument select instructions:
   FuncInfo.SIVisitor.instrumentSelects(F, &I, NumCounters, FuncInfo.FuncNameVar,
                                        FuncInfo.FunctionHash);
   assert(I == NumCounters);
 
   if (DisableValueProfiling)
if (BFI->isIrrLoopHeader(&BB) || isIndirectBrTarget(&BB)) {
       Instruction *TI = BB.getTerminator();
       const UseBBInfo &BBCountInfo = getBBInfo(&BB);
       setIrrLoopHeaderMetadata(M, TI, BBCountInfo.CountValue);
     }
   }
 }
 
 void SelectInstVisitor::instrumentOneSelectInst(SelectInst &SI) {
+  if (CoverageInstrumentation)
+    return;
   Module *M = F.getParent();
   IRBuilder<> Builder(&SI);
   Type *Int64Ty = Builder.getInt64Ty();
   Type *I8PtrTy = Builder.getInt8PtrTy();
   auto *Step = Builder.CreateZExt(SI.getCondition(), Int64Ty);
   Builder.CreateCall(
       Intrinsic::getDeclaration(M, Intrinsic::instrprof_increment_step),
       {ConstantExpr::getBitCast(FuncNameVar, I8PtrTy),

 static bool InstrumentAllFunctions(
     Module &M, function_ref<TargetLibraryInfo &(Function &)> LookupTLI,
     function_ref<BranchProbabilityInfo *(Function &)> LookupBPI,
     function_ref<BlockFrequencyInfo *(Function &)> LookupBFI, bool IsCS) {
   // For the context-sensitve instrumentation, we should have a separated pass
   // (before LTO/ThinLTO linking) to create these variables.
   if (!IsCS)
     createIRLevelProfileFlagVar(M, /*IsCS=*/false, PGOInstrumentEntry,
-                                DebugInfoCorrelate);
+                                DebugInfoCorrelate, CoverageInstrumentation);
   std::unordered_multimap<Comdat *, GlobalValue *> ComdatMembers;
   collectComdatMembers(M, ComdatMembers);
 
   for (auto &F : M) {
     if (F.isDeclaration())
       continue;
     if (F.hasFnAttribute(llvm::Attribute::NoProfile))
       continue;
     auto &TLI = LookupTLI(F);
     auto *BPI = LookupBPI(F);
     auto *BFI = LookupBFI(F);
     instrumentOneFunc(F, &M, TLI, BPI, BFI, ComdatMembers, IsCS);
   }
   return true;
 }
 
 PreservedAnalyses
 PGOInstrumentationGenCreateVar::run(Module &M, ModuleAnalysisManager &AM) {
   createProfileFileNameVar(M, CSInstrName);
   // The variable in a comdat may be discarded by LTO. Ensure the declaration
   // will be retained.
-  appendToCompilerUsed(M, createIRLevelProfileFlagVar(M, /*IsCS=*/true,
-                                                      PGOInstrumentEntry,
-                                                      DebugInfoCorrelate));
+  appendToCompilerUsed(M, createIRLevelProfileFlagVar(
+                              M, /*IsCS=*/true, PGOInstrumentEntry,
+                              DebugInfoCorrelate, CoverageInstrumentation));
   return PreservedAnalyses::all();
 }
 
 bool PGOInstrumentationGenLegacyPass::runOnModule(Module &M) {
   if (skipModule(M))
     return false;
 
   auto LookupTLI = [this](Function &F) -> TargetLibraryInfo & {
if (!PGOReader->hasCSIRLevelProfile() && IsCS)
     return false;
 
   // TODO: might need to change the warning once the clang option is finalized.
   if (!PGOReader->isIRLevelProfile()) {
     Ctx.diagnose(DiagnosticInfoPGOProfile(
         ProfileFileName.data(), "Not an IR level instrumentation profile"));
     return false;
   }
+  if (PGOReader->useSingleByteEntryCoverage()) {
+    Ctx.diagnose(DiagnosticInfoPGOProfile(
+        ProfileFileName.data(),
+        "Cannot use function entry coverage profiles for optimization"));
+    return false;
+  }
 
   // Add the profile summary (read from the header of the indexed summary) here
   // so that we can use it below when reading counters (which checks if the
   // function should be marked with a cold or inlinehint attribute).
   M.setProfileSummary(PGOReader->getSummary(IsCS).getMD(M.getContext()),
                       IsCS ? ProfileSummary::PSK_CSInstr
                            : ProfileSummary::PSK_Instr);
   PSI->refresh();

llvm/test/Instrumentation/InstrProfiling/coverage.ll

+; RUN: opt < %s -instrprof -S | FileCheck %s
+
+target triple = "aarch64-unknown-linux-gnu"
+
+@__profn_foo = private constant [3 x i8] c"foo"
+; CHECK: @__profc_foo = private global [1 x i8] c"\FF", section "__llvm_prf_cnts", comdat, align 1
+@__profn_bar = private constant [3 x i8] c"bar"
+; CHECK: @__profc_bar = private global [1 x i8] c"\FF", section "__llvm_prf_cnts", comdat, align 1
+
+define void @_Z3foov() {
+  call void @llvm.instrprof.cover(i8* getelementptr inbounds ([3 x i8], [3 x i8]* @__profn_foo, i32 0, i32 0), i64 12345678)
+  ; CHECK: store i8 0, i8* getelementptr inbounds ([1 x i8], [1 x i8]* @__profc_foo, i32 0, i32 0), align 1
+  ret void
+}
+
+%class.A = type { i32 (...)** }
+define dso_local void @_Z3barv(%class.A* nocapture nonnull align 8 %0) unnamed_addr #0 align 2 {
+  call void @llvm.instrprof.cover(i8* getelementptr inbounds ([3 x i8], [3 x i8]* @__profn_bar, i32 0, i32 0), i64 87654321)
+  ; CHECK: store i8 0, i8* getelementptr inbounds ([1 x i8], [1 x i8]* @__profc_bar, i32 0, i32 0), align 1
+  ret void
+}
+
+declare void @llvm.instrprof.cover(i8*, i64)

llvm/test/Instrumentation/InstrProfiling/debug-info-correlate-coverage.ll

+; RUN: opt < %s -instrprof -debug-info-correlate -S | opt --O2 -S | FileCheck %s
+
+@__profn_foo = private constant [3 x i8] c"foo"
+; CHECK:      @__profc_foo
+
+define  void @_Z3foov() !dbg !12 {
+  call void @llvm.instrprof.cover(i8* getelementptr inbounds ([3 x i8], [3 x i8]* @__profn_foo, i32 0, i32 0), i64 12345678)
+  ; CHECK: store i8 0, i8* getelementptr inbounds ([1 x i8], [1 x i8]* @__profc_foo
+  ret void
+}
+
+declare void @llvm.instrprof.cover(i8*, i64)
+
+!llvm.dbg.cu = !{!0}
+!llvm.module.flags = !{!2, !3, !4, !5, !6, !7, !8, !9, !10}
+!llvm.ident = !{!11}
+
+!0 = distinct !DICompileUnit(language: DW_LANG_C_plus_plus_14, file: !1, producer: "clang version 14.0.0", isOptimized: false, runtimeVersion: 0, emissionKind: FullDebug, splitDebugInlining: false, nameTableKind: None)
+!1 = !DIFile(filename: "debug-info-correlate-coverage.cpp", directory: "")
+!2 = !{i32 7, !"Dwarf Version", i32 4}
+!3 = !{i32 2, !"Debug Info Version", i32 3}
+!4 = !{i32 1, !"wchar_size", i32 4}
+!5 = !{i32 1, !"branch-target-enforcement", i32 0}
+!6 = !{i32 1, !"sign-return-address", i32 0}
+!7 = !{i32 1, !"sign-return-address-all", i32 0}
+!8 = !{i32 1, !"sign-return-address-with-bkey", i32 0}
+!9 = !{i32 7, !"uwtable", i32 1}
+!10 = !{i32 7, !"frame-pointer", i32 1}
+!11 = !{!"clang version 14.0.0"}
+!12 = distinct !DISubprogram(name: "foo", linkageName: "_Z3foov", scope: !13, file: !13, line: 1, type: !14, scopeLine: 1, flags: DIFlagPrototyped, spFlags: DISPFlagDefinition, unit: !0, retainedNodes: !16)
+!13 = !DIFile(filename: "debug-info-correlate-coverage.cpp", directory: "")
+!14 = !DISubroutineType(types: !15)
+!15 = !{null}
+!16 = !{}

llvm/test/Transforms/PGOProfile/coverage.ll

+; RUN: opt < %s -passes=pgo-instr-gen -pgo-coverage-instrumentation -S | FileCheck %s
+target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+define i32 @foo(i32 %i) {
+entry:
+  ; CHECK: call void @llvm.instrprof.cover({{.*}})
+  %cmp = icmp sgt i32 %i, 0
+  br i1 %cmp, label %if.then, label %if.else
+
+if.then:
+  ; CHECK-NOT: llvm.instrprof.cover(
+  %add = add nsw i32 %i, 2
+  %s = select i1 %cmp, i32 %add, i32 0
+  br label %if.end
+
+if.else:
+  %sub = sub nsw i32 %i, 2
+  br label %if.end
+
+if.end:
+  %retv = phi i32 [ %add, %if.then ], [ %sub, %if.else ]
+  ret i32 %retv
+}
+
+; CHECK: declare void @llvm.instrprof.cover(

llvm/tools/llvm-profdata/llvm-profdata.cpp

if (Error E = WC->Writer.setIsIRLevelProfile(IsIRProfile, HasCSIRProfile)) {
     WC->Errors.emplace_back(
         make_error<StringError>(
             "Merge IR generated profile with Clang generated profile.",
             std::error_code()),
         Filename);
     return;
   }
   WC->Writer.setInstrEntryBBEnabled(Reader->instrEntryBBEnabled());
+  WC->Writer.setUseSingleByteEntryCoverage(
+      Reader->useSingleByteEntryCoverage());
 
   for (auto &I : *Reader) {
     if (Remapper)
       I.Name = (*Remapper)(I.Name);
     const StringRef FuncName = I.Name;
     bool Reported = false;
     WC->Writer.addRecord(std::move(I), Input.Weight, [&](Error E) {
       if (Reported) {
static void showValueSitesStats(raw_fd_ostream &OS, uint32_t VK,
 
   OS << "  Value sites histogram:\n\tNumTargets, SiteCount\n";
   for (unsigned I = 0; I < Stats.ValueSitesHistogram.size(); I++) {
     if (Stats.ValueSitesHistogram[I] > 0)
       OS << "\t" << I + 1 << ", " << Stats.ValueSitesHistogram[I] << "\n";
   }
 }
 
 static int showInstrProfile(const std::string &Filename, bool ShowCounts,
                             uint32_t TopN, bool ShowIndirectCallTargets,
                             bool ShowMemOPSizes, bool ShowDetailedSummary,
                             std::vector<uint32_t> DetailedSummaryCutoffs,
                             bool ShowAllFunctions, bool ShowCS,
                             uint64_t ValueCutoff, bool OnlyListBelow,
                             const std::string &ShowFunction, bool TextFormat,
-                            bool ShowBinaryIds, raw_fd_ostream &OS) {
+                            bool ShowBinaryIds, bool ShowCovered,
+                            raw_fd_ostream &OS) {
   auto ReaderOrErr = InstrProfReader::create(Filename);
   std::vector<uint32_t> Cutoffs = std::move(DetailedSummaryCutoffs);
   if (ShowDetailedSummary && Cutoffs.empty()) {
     Cutoffs = {800000, 900000, 950000, 990000, 999000, 999900, 999990};
   }
   InstrProfSummaryBuilder Builder(std::move(Cutoffs));
   if (Error E = ReaderOrErr.takeError())
     exitWithError(std::move(E), Filename);
if (doTextFormatDump) {
       InstrProfWriter::writeRecordInText(Func.Name, Func.Hash, Func, Symtab,
                                          OS);
       continue;
     }
 
     assert(Func.Counts.size() > 0 && "function missing entry counter");
     Builder.addRecord(Func);
 
+    if (ShowCovered)
+      if (std::any_of(Func.Counts.begin(), Func.Counts.end(),
+                      [](uint64_t C) { return C; }))
+        OS << "  " << Func.Name << "\n";
+
     uint64_t FuncMax = 0;
     uint64_t FuncSum = 0;
     for (size_t I = 0, E = Func.Counts.size(); I < E; ++I) {
       if (Func.Counts[I] == (uint64_t)-1)
         continue;
       FuncMax = std::max(FuncMax, Func.Counts[I]);
       FuncSum += Func.Counts[I];
     }
cl::opt<bool> ShowProfileSymbolList(
       cl::desc("Show profile symbol list if it exists in the profile. "));
   cl::opt<bool> ShowSectionInfoOnly(
       "show-sec-info-only", cl::init(false),
       cl::desc("Show the information of each section in the sample profile. "
                "The flag is only usable when the sample profile is in "
                "extbinary format"));
   cl::opt<bool> ShowBinaryIds("binary-ids", cl::init(false),
                               cl::desc("Show binary ids in the profile. "));
+  cl::opt<bool> ShowCovered(
+      "covered", cl::init(false),
+      cl::desc("Show only the functions that have been executed."));
 
   cl::ParseCommandLineOptions(argc, argv, "LLVM profile data summary\n");
 
   if (Filename == OutputFilename) {
     errs() << sys::path::filename(argv[0])
            << ": Input file name cannot be the same as the output file name!\n";
     return 1;
   }
 
   std::error_code EC;
   raw_fd_ostream OS(OutputFilename.data(), EC, sys::fs::OF_TextWithCRLF);
   if (EC)
     exitWithErrorCode(EC, OutputFilename);
 
   if (ShowAllFunctions && !ShowFunction.empty())
     WithColor::warning() << "-function argument ignored: showing all functions\n";
 
   if (ProfileKind == instr)
     return showInstrProfile(
         Filename, ShowCounts, TopNFunctions, ShowIndirectCallTargets,
         ShowMemOPSizes, ShowDetailedSummary, DetailedSummaryCutoffs,
         ShowAllFunctions, ShowCS, ValueCutoff, OnlyListBelow, ShowFunction,
-        TextFormat, ShowBinaryIds, OS);
+        TextFormat, ShowBinaryIds, ShowCovered, OS);
   if (ProfileKind == sample)
     return showSampleProfile(Filename, ShowCounts, TopNFunctions,
                              ShowAllFunctions, ShowDetailedSummary,
                              ShowFunction, ShowProfileSymbolList,
                              ShowSectionInfoOnly, ShowHotFuncList, OS);
   return showMemProfProfile(Filename, OS);
 }