[InstrProf] Add single byte coverage mode

[ellis]

Summary

Use the llvm flag -pgo-function-entry-coverage to create single byte "counters" to track functions coverage. This mode has significantly less size overhead in both code and data because

• We mark a function as "covered" with a store instead of an increment which generally requires fewer assembly instructions
• We use a single byte per function rather than 8 bytes per block

The trade off of course is that this mode only tells you if a function has been covered. This is useful, for example, to detect dead code.

When combined with debug info correlation [0] we are able to create an instrumented Clang binary that is only 150M (the vanilla Clang binary is 143M). That is an overhead of 7M (4.9%) compared to the default instrumentation (without value profiling) which has an overhead of 31M (21.7%).

[0] https://groups.google.com/g/llvm-dev/c/r03Z6JoN7d4

[kyulee]

I will come back to review this code later once the dependent patches land.
In the high-level discussion, this is really a function coverage because it probes the function header, but the flag or terminology inter-mixed just a coverage.
For the code (not just function) coverage, I think we might want to have an option to inject this probe in the block edge as well so that the default/code-coverage can be fully functional.
Of course, this could be a follow-up, but just head-up on how this can be factored in this design.

compiler-rt/test/profile/instrprof-coverage.c

2

Can you add a test case using the debug data -debug-info?

llvm/lib/Transforms/Instrumentation/PGOInstrumentation.cpp

479

Hmm. It appears to continue adding extra parameters.
I wonder whether we can refactor this given more features are coming.

[ellis]

In D116180#3211818, @kyulee wrote:

In the high-level discussion, this is really a function coverage because it probes the function header, but the flag or terminology inter-mixed just a coverage.

You're right, I should update the flag name to make it more clear that this is only a function entry coverage.

For the code (not just function) coverage, I think we might want to have an option to inject this probe in the block edge as well so that the default/code-coverage can be fully functional.
Of course, this could be a follow-up, but just head-up on how this can be factored in this design.

I've experimented with instrumenting clang with block coverage vs block counts. It turns out that full block coverage actually requires a larger counters section than regular block counts. This is because, for block counts, we can infer some missing block counts from summing up their parents or children. However, we cannot do this when using boolean values in the same way. So every basic block must be probed. (There are some cases where you can infer some missing block coverage, but I haven't thought deeply about when you can do this. And I suspect this is rare.) Also, there was slightly less assembly code from block counts rather than block coverage, but that only made the overall size difference negligible between the two.

Basically, block coverage has about the same size overhead, but gives significantly less information than block counts. We could allow for counters to be 16 or 8 bits to save space in the block counters mode, but that would be left for a later diff.

[ellis]

Rebase and add test to instrprof-debug-info-correlate.c.

[kyulee]

llvm/lib/Transforms/Instrumentation/InstrProfiling.cpp

1025

I think we can just delete this condition.
Without DebugInfoCorrelate, Counters are referenced by Data which is being explicitly held as below via CompilerUsedVars.push_back(Data);.
With DebugInfoCorrelate, now Counters can be optimized away since they are local globals while there are no explicit use.
I think the reason the Coverage case (write only) does not work while the Counter case (read/write) works is simply because the global opt is smart enough to optimize the Coverage case but not the Counter case -- in theory there is no explicit use in the chain of references on Counters (other than compiler-rt/runtime), and the compiler may optimize them.

[ellis]

llvm/lib/Transforms/Instrumentation/InstrProfiling.cpp

1025

Yep, I agree. I have a separate diff D115981 to fix this and will update this code after that lands.

[MaskRay]

I know that you have posted many patches in this area and regular reviewers (e.g. davidxl, vsk) are not involved.
It is a holiday season for many folks so some may not be active.
For some intrusive/significant changes worth getting attention from them before committed, even after you get approval from your colleague(?).

[ellis]

In D116180#3214457, @MaskRay wrote:

I know that you have posted many patches in this area and regular reviewers (e.g. davidxl, vsk) are not involved.
It is a holiday season for many folks so some may not be active.
For some intrusive/significant changes worth getting attention from them before committed, even after you get approval from your colleague(?).

Makes sense to me because I posted this just before the holidays. I can hold off landing this and the previous patch until around the second week of January (I'm taking most of next week off anyway).

[MaskRay]

In D116180#3214466, @ellis wrote:

In D116180#3214457, @MaskRay wrote:

I know that you have posted many patches in this area and regular reviewers (e.g. davidxl, vsk) are not involved.
It is a holiday season for many folks so some may not be active.
For some intrusive/significant changes worth getting attention from them before committed, even after you get approval from your colleague(?).

Makes sense to me because I posted this just before the holidays. I can hold off landing this and the previous patch until around the second week of January (I'm taking most of next week off anyway).

Thanks! They do not subscribe to these changes so they may not even know. (perhaps you mentioned the Differential links on some thread on llvm-dev, but it is pretty easy to miss this piece of information.)
I know that prudently picking reviewers is not easy: you need to do some git log archaeology who contributed/reviewed code in the area and may likely have an opinion.
I remember that they both commented on a previous proposal from you, so they likely have an opinion.

[davidxl]

llvm/docs/LangRef.rst

13150

probably name it llvm.instrprof.entry_cover to reflect the semantics. In theory you can have block level coverage mode too.

llvm/include/llvm/IR/IntrinsicInst.h

1229–1230

Split out this in different patch?

llvm/include/llvm/ProfileData/InstrProfReader.h

527

entry coverage is compatiable with IR pgo -- so perhaps add an assertion somewhere.

llvm/lib/Transforms/Instrumentation/InstrProfiling.cpp

959

Split out the change into a helper function to make it cleaner.

llvm/lib/Transforms/Instrumentation/PGOInstrumentation.cpp

258

make the variable name matching the option name

llvm/test/Transforms/PGOProfile/coverage.ll

13

Just check there is one instance of instrinsic call per function?

llvm/tools/llvm-profdata/llvm-profdata.cpp

2088

Add a test case for the coverage mode.

2154

should probably by pass the rest of dump in coverage mode.

[kyulee]

llvm/lib/Transforms/Instrumentation/InstrProfiling.cpp

964

Can we just use Constant::getAllOnesValue() like Constant::getNullValue() with the array type?

[ellis]

Rebase and split NFC changes in another diff.

[ellis]

compiler-rt/test/profile/instrprof-coverage.c

2

I've added this to the above test instrprof-debug-info-correlate.c.

llvm/docs/LangRef.rst

13150

I agree. Renaming to llvm.instrprof.entry_cover.

llvm/include/llvm/IR/IntrinsicInst.h

1229–1230

Created a parent diff

llvm/lib/Transforms/Instrumentation/InstrProfiling.cpp

964

Unfortunately, ConstantArray::get() takes an array so we cannot give it just a Constant.

https://llvm.org/doxygen/classllvm_1_1ConstantArray.html#a0900dacdc7ad8e3ea0cc92993c7fd422

Also, getAllOnesValue() does not appear to accept an array type yet.

llvm/test/Transforms/PGOProfile/coverage.ll

13

Yeah, this is just a sanity check that it isn't called in other blocks.

llvm/tools/llvm-profdata/llvm-profdata.cpp

2088

Added show-covered.test.

[ellis]

Rebase

[ellis]

Fix class name

[kyulee]

llvm/lib/Transforms/Instrumentation/InstrProfiling.cpp

964

FixedVectorType instead of ArrayType seems the same semantic, which appears to work.

[ellis]

Split change into new function

[ellis]

llvm/lib/Transforms/Instrumentation/InstrProfiling.cpp

964

I would like to avoid using different array types for function entry cover and block counters. From the docs (https://llvm.org/docs/LangRef.html#vector-type) it seems that the vector type is intended for SIMD instructions and we won't them on these globals.

I think a better solution would be to expand getAllOnesValue() to take an array type, but I haven't looked into whether that is realistic.

[ellis]

Rebase

[ellis]

llvm/lib/Transforms/Instrumentation/PGOInstrumentation.cpp

479

I've created a D118097 to refactor this

[phosek]

This change appears to be implementing func-cov mode proposed in https://groups.google.com/g/llvm-dev/c/r03Z6JoN7d4. Do you plan on implementing the support for block-cov and func-cnt modes as well? I'm asking because we're interested in the block-cov mode, but also because I'm somewhat concerned about the orthogonality of these modes in the implementation.

For example, this change introduces the VARIANT_MASK_BYTE_ENTRY_COVERAGE flag to track whether the func-cov mode is used. This means that we're going to need two more bits for block-cov and func-cnt (three in total). Wouldn't it make more sense to instead have VARIANT_MASK_BYTE_COVERAGE and VARIANT_MASK_ENTRY_COVERAGE flags which could be combined to represent all four modes, requiring only two bits in total?

[ellis]

In D116180#3268478, @phosek wrote:

This change appears to be implementing func-cov mode proposed in https://groups.google.com/g/llvm-dev/c/r03Z6JoN7d4. Do you plan on implementing the support for block-cov and func-cnt modes as well? I'm asking because we're interested in the block-cov mode, but also because I'm somewhat concerned about the orthogonality of these modes in the implementation.

For example, this change introduces the VARIANT_MASK_BYTE_ENTRY_COVERAGE flag to track whether the func-cov mode is used. This means that we're going to need two more bits for block-cov and func-cnt (three in total). Wouldn't it make more sense to instead have VARIANT_MASK_BYTE_COVERAGE and VARIANT_MASK_ENTRY_COVERAGE flags which could be combined to represent all four modes, requiring only two bits in total?

I was only planning on adding func-cov. A while back I did a small experiment and found very little binary size difference between block-cov and full block-counts. This is because block-counts can instrument a subset of blocks and infer counts from the rest. With block-cov we cannot do this in the same way so we need to instrument every block. There might be a smarter way to infer some block coverage, but I don't know it.

I decided to use just VARIANT_MASK_BYTE_ENTRY_COVERAGE for simplicity, but I'm open to using two flags like you suggested.

Can I ask why you are interested in block-cov rather than using full block counters? If binary size is a concern then I would suggest inferring coverage from 8 bit counters, assuming overflow isn't a problem. If speed is the main concern then maybe block-cov is the right way to go.

[phosek]

In D116180#3269931, @ellis wrote:

In D116180#3268478, @phosek wrote:

This change appears to be implementing func-cov mode proposed in https://groups.google.com/g/llvm-dev/c/r03Z6JoN7d4. Do you plan on implementing the support for block-cov and func-cnt modes as well? I'm asking because we're interested in the block-cov mode, but also because I'm somewhat concerned about the orthogonality of these modes in the implementation.

For example, this change introduces the VARIANT_MASK_BYTE_ENTRY_COVERAGE flag to track whether the func-cov mode is used. This means that we're going to need two more bits for block-cov and func-cnt (three in total). Wouldn't it make more sense to instead have VARIANT_MASK_BYTE_COVERAGE and VARIANT_MASK_ENTRY_COVERAGE flags which could be combined to represent all four modes, requiring only two bits in total?

I was only planning on adding func-cov. A while back I did a small experiment and found very little binary size difference between block-cov and full block-counts. This is because block-counts can instrument a subset of blocks and infer counts from the rest. With block-cov we cannot do this in the same way so we need to instrument every block. There might be a smarter way to infer some block coverage, but I don't know it.

I decided to use just VARIANT_MASK_BYTE_ENTRY_COVERAGE for simplicity, but I'm open to using two flags like you suggested.

Can I ask why you are interested in block-cov rather than using full block counters? If binary size is a concern then I would suggest inferring coverage from 8 bit counters, assuming overflow isn't a problem. If speed is the main concern then maybe block-cov is the right way to go.

We're mostly concerned about performance so block-cov is still of interest to us, but this is a useful insight, thanks! If it's not too much of an overhead, I'd prefer using the two flags so we can introduce the block-cov mode later.

[ellis]

In D116180#3270200, @phosek wrote:

In D116180#3269931, @ellis wrote:

In D116180#3268478, @phosek wrote:

This change appears to be implementing func-cov mode proposed in https://groups.google.com/g/llvm-dev/c/r03Z6JoN7d4. Do you plan on implementing the support for block-cov and func-cnt modes as well? I'm asking because we're interested in the block-cov mode, but also because I'm somewhat concerned about the orthogonality of these modes in the implementation.

For example, this change introduces the VARIANT_MASK_BYTE_ENTRY_COVERAGE flag to track whether the func-cov mode is used. This means that we're going to need two more bits for block-cov and func-cnt (three in total). Wouldn't it make more sense to instead have VARIANT_MASK_BYTE_COVERAGE and VARIANT_MASK_ENTRY_COVERAGE flags which could be combined to represent all four modes, requiring only two bits in total?

I was only planning on adding func-cov. A while back I did a small experiment and found very little binary size difference between block-cov and full block-counts. This is because block-counts can instrument a subset of blocks and infer counts from the rest. With block-cov we cannot do this in the same way so we need to instrument every block. There might be a smarter way to infer some block coverage, but I don't know it.

I decided to use just VARIANT_MASK_BYTE_ENTRY_COVERAGE for simplicity, but I'm open to using two flags like you suggested.

Can I ask why you are interested in block-cov rather than using full block counters? If binary size is a concern then I would suggest inferring coverage from 8 bit counters, assuming overflow isn't a problem. If speed is the main concern then maybe block-cov is the right way to go.

We're mostly concerned about performance so block-cov is still of interest to us, but this is a useful insight, thanks! If it's not too much of an overhead, I'd prefer using the two flags so we can introduce the block-cov mode later.

Sounds good! I'll use two flags but leave the other cases unimplemented for now.

[ellis]

Use two flags for single byte coverage and function entry coverage.

[ellis]

@davidxl @phosek @kyulee Can I get another look at this?

llvm/docs/LangRef.rst

13150

I renamed it to llvm.instrprof.cover because this would be used for block coverage.

llvm/include/llvm/ProfileData/InstrProfReader.h

527

I added an assertion in PGOInstrumentation.cpp.

[kyulee]

LGTM. I'm approving this but suggest for other reviewers'. Also I left a few comments about naming counters that may follow up.

llvm/lib/Transforms/Instrumentation/InstrProfiling.cpp

696–726

Strictly speaking, it's not a counter for coverage because a store instead of an adder will be used. Same as in the description of intrinsic, and all other api names using counters.
A probe or something might be a better name. However, it seems very invasive to change all the counter name, which may confuse the current (main) semantic for edge counter.
I suggest to just keep as it but I'm also open to other reviewer's opinions.

[davidxl]

compiler-rt/lib/profile/InstrProfilingMerge.c

161

is there a test case for this?

[ellis]

Rebase ontop of D115393

compiler-rt/lib/profile/InstrProfilingMerge.c

161

Added the test instrprof-merge-entry-cover.c and found a bug in __llvm_profile_reset_counters(). Thanks for the reminder!

llvm/lib/Transforms/Instrumentation/InstrProfiling.cpp

696–726

Yeah, I would like to use "probe", but "counters" is used all of the place in the source. I can bring this up in https://llvm.discourse.group/ to see if we are willing to make this change.

[snehasish]

llvm/include/llvm/ProfileData/InstrProf.h

289

At a glance, this looks similar to the BB enum value. If this is coverage only can we make it explicit in the name? Perhaps something like FunctionEntryCoverageOnly. IIUC the comment implies "only instrument the function entry for coverage" as opposed to whether the the entry basic block has instrumentation for profiling.

I like the idea of having better naming for the enum values and I'll send out a separate patch renaming the others once this is submitted.

[ellis]

llvm/include/llvm/ProfileData/InstrProf.h

289

This is actually not for coverage. FunctionEntryOnly only instruments the entry basic block, whereas BB guarantees that the entry basic block is instrumented, along with other blocks.

[snehasish]

llvm/include/llvm/ProfileData/InstrProf.h

289

I see. So should we allow profiles with BB and FunctionEntryOnly to be merged? If not can you update the logic in mergeProfileKind in InstrProfWriter.h?

[ellis]

Make BB and FunctionEntryOnly incompatible.

[ellis]

Fix build error

[MaskRay]

In D116180#3276998, @kyulee wrote:

LGTM. I'm approving this but suggest for other reviewers'. Also I left a few comments about naming counters that may follow up.

Thanks for contributing a new coverage mode. I believe it will be useful to some groups.
That said, process-wise, I think committing this just 6hr after this comment was probably too quick. 6hr did not give stakeholders enough time to read through a major new feature.
Some stakeholders may have other commitments so don't respond timely. But if you gave more time with the patch in an "approved" state, they would probably want to comment.

I think my main question is why does a coverage mode need to piggyback on PGOInstrument.cpp which is mainly for optimizations.

From the functionality implemented by -pgo-function-entry-coverage, I don't see how it can be used for optimization.

[ellis]

In D116180#3558453, @MaskRay wrote:

In D116180#3276998, @kyulee wrote:

LGTM. I'm approving this but suggest for other reviewers'. Also I left a few comments about naming counters that may follow up.

Thanks for contributing a new coverage mode. I believe it will be useful to some groups.
That said, process-wise, I think committing this just 6hr after this comment was probably too quick. 6hr did not give stakeholders enough time to read through a major new feature.
Some stakeholders may have other commitments so don't respond timely. But if you gave more time with the patch in an "approved" state, they would probably want to comment.

I think my main question is why does a coverage mode need to piggyback on PGOInstrument.cpp which is mainly for optimizations.

From the functionality implemented by -pgo-function-entry-coverage, I don't see how it can be used for optimization.

I apologize for moving too quickly to land. In the future, I'll try to allow more time for others to review and respect their time.

As for your question, I've posted this RFC [0] describing our plans for lightweight instrumentation. Basically, we were planning to create a few instrumentation modes that trade off profile precision for a more lightweight instrumented binary. 1) block coverage, 2) function entry count, and 3) function entry coverage. As you've seen, D124490 implements block coverage and the profiles are useful for optimization in D125743 which uses block liveness to guide the machine outliner.

Function entry coverage implemented in this diff was an important step towards block coverage instrumentation. Both modes share a lot of code, which is a big reason why I added this in PGOInstrumentation.cpp. Unlike block coverage instrumentation, function entry coverage is not precise enough to properly guide optimizations, so its main use is to detect dead code.

[0] https://groups.google.com/g/llvm-dev/c/r03Z6JoN7d4

compiler-rt/include/profile/InstrProfData.inc

 #define INSTR_PROF_COVMAP_VERSION 5
 
 /* Profile version is always of type uint64_t. Reserve the upper 8 bits in the
  * version for other variants of profile. We set the lowest bit of the upper 8
  * bits (i.e. bit 56) to 1 to indicate if this is an IR-level instrumentation
  * generated profile, and 0 if this is a Clang FE generated profile.
  * 1 in bit 57 indicates there are context-sensitive records in the profile.
  * The 59th bit indicates whether to use debug info to correlate profiles.
+ * The 60th bit indicates single byte coverage instrumentation.
+ * The 61st bit indicates function entry instrumentation only.
  */
 #define VARIANT_MASKS_ALL 0xff00000000000000ULL
 #define GET_VERSION(V) ((V) & ~VARIANT_MASKS_ALL)
 #define VARIANT_MASK_IR_PROF (0x1ULL << 56)
 #define VARIANT_MASK_CSIR_PROF (0x1ULL << 57)
 #define VARIANT_MASK_INSTR_ENTRY (0x1ULL << 58)
 #define VARIANT_MASK_DBG_CORRELATE (0x1ULL << 59)
+#define VARIANT_MASK_BYTE_COVERAGE (0x1ULL << 60)
+#define VARIANT_MASK_FUNCTION_ENTRY_ONLY (0x1ULL << 61)
 #define INSTR_PROF_RAW_VERSION_VAR __llvm_profile_raw_version
 #define INSTR_PROF_PROFILE_RUNTIME_VAR __llvm_profile_runtime
 #define INSTR_PROF_PROFILE_COUNTER_BIAS_VAR __llvm_profile_counter_bias
 
 /* The variable that holds the name of the profile data
  * specified via command line. */
 #define INSTR_PROF_PROFILE_NAME_VAR __llvm_profile_filename
 

compiler-rt/lib/profile/InstrProfiling.c

 COMPILER_RT_VISIBILITY uint64_t __llvm_profile_get_version(void) {
   return INSTR_PROF_RAW_VERSION_VAR;
 }
 
 COMPILER_RT_VISIBILITY void __llvm_profile_reset_counters(void) {
   char *I = __llvm_profile_begin_counters();
   char *E = __llvm_profile_end_counters();
 
-  memset(I, 0, E - I);
+  char ResetValue =
+      (__llvm_profile_get_version() & VARIANT_MASK_BYTE_COVERAGE) ? 0xFF : 0;
+  memset(I, ResetValue, E - I);
 
   const __llvm_profile_data *DataBegin = __llvm_profile_begin_data();
   const __llvm_profile_data *DataEnd = __llvm_profile_end_data();
   const __llvm_profile_data *DI;
   for (DI = DataBegin; DI < DataEnd; ++DI) {
     uint64_t CurrentVSiteCount = 0;
     uint32_t VKI, i;
     if (!DI->Values)

compiler-rt/lib/profile/InstrProfilingBuffer.c

 
 COMPILER_RT_VISIBILITY
 uint64_t __llvm_profile_get_data_size(const __llvm_profile_data *Begin,
                                       const __llvm_profile_data *End) {
   return __llvm_profile_get_num_data(Begin, End) * sizeof(__llvm_profile_data);
 }
 
 COMPILER_RT_VISIBILITY size_t __llvm_profile_counter_entry_size(void) {
+  if (__llvm_profile_get_version() & VARIANT_MASK_BYTE_COVERAGE)
+    return sizeof(uint8_t);
   return sizeof(uint64_t);
 }
 
 COMPILER_RT_VISIBILITY
 uint64_t __llvm_profile_get_num_counters(const char *Begin, const char *End) {
   intptr_t BeginI = (intptr_t)Begin, EndI = (intptr_t)End;
   return ((EndI + __llvm_profile_counter_entry_size() - 1) - BeginI) /
          __llvm_profile_counter_entry_size();

compiler-rt/lib/profile/InstrProfilingMerge.c

for (SrcData = SrcDataStart,
     // record.
     CountersDelta -= sizeof(*SrcData);
     unsigned NC = SrcData->NumCounters;
     if (NC == 0)
       return 1;
     if (SrcCounters < SrcCountersStart || SrcCounters >= SrcNameStart ||
         (SrcCounters + __llvm_profile_counter_entry_size() * NC) > SrcNameStart)
       return 1;
-    for (unsigned I = 0; I < NC; I++)
+    for (unsigned I = 0; I < NC; I++) {
+      if (__llvm_profile_get_version() & VARIANT_MASK_BYTE_COVERAGE) {
+        // A value of zero signifies the function is covered.
+        DstCounters[I] &= SrcCounters[I];
+      } else {
         ((uint64_t *)DstCounters)[I] += ((uint64_t *)SrcCounters)[I];
+      }
+    }
 
     /* Now merge value profile data. */
     if (!VPMergeHook)
       continue;
 
     for (unsigned I = 0; I <= IPVK_Last; I++)
       NVK += (SrcData->NumValueSites[I] != 0);
 

compiler-rt/test/profile/Darwin/instrprof-debug-info-correlate.c

 // Value profiling is currently not supported in lightweight mode.
 // RUN: %clang_pgogen -o %t -g -mllvm --debug-info-correlate -mllvm --disable-vp=true %S/../Inputs/instrprof-debug-info-correlate-main.cpp %S/../Inputs/instrprof-debug-info-correlate-foo.cpp
 // RUN: env LLVM_PROFILE_FILE=%t.proflite %run %t
 // RUN: llvm-profdata merge -o %t.profdata --debug-info=%t.dSYM %t.proflite
 
 // RUN: %clang_pgogen -o %t.normal -mllvm --disable-vp=true %S/../Inputs/instrprof-debug-info-correlate-main.cpp %S/../Inputs/instrprof-debug-info-correlate-foo.cpp
 // RUN: env LLVM_PROFILE_FILE=%t.profraw %run %t.normal
 // RUN: llvm-profdata merge -o %t.normal.profdata %t.profraw
 
 // RUN: diff %t.normal.profdata %t.profdata
+
+// RUN: %clang_pgogen -o %t.cov -g -mllvm --debug-info-correlate -mllvm -pgo-function-entry-coverage -mllvm --disable-vp=true %S/../Inputs/instrprof-debug-info-correlate-main.cpp %S/../Inputs/instrprof-debug-info-correlate-foo.cpp
+// RUN: env LLVM_PROFILE_FILE=%t.cov.proflite %run %t.cov
+// RUN: llvm-profdata merge -o %t.cov.profdata --debug-info=%t.cov.dSYM %t.cov.proflite
+
+// RUN: %clang_pgogen -o %t.cov.normal -mllvm --pgo-function-entry-coverage -mllvm --disable-vp=true %S/../Inputs/instrprof-debug-info-correlate-main.cpp %S/../Inputs/instrprof-debug-info-correlate-foo.cpp
+// RUN: env LLVM_PROFILE_FILE=%t.cov.profraw %run %t.cov.normal
+// RUN: llvm-profdata merge -o %t.cov.normal.profdata %t.cov.profraw
+
+// RUN: diff %t.cov.normal.profdata %t.cov.profdata

compiler-rt/test/profile/Inputs/instrprof-debug-info-correlate-bar.h

 int foo(int);
+int unused(int);
 
 inline int bar(int a) {
   while (a > 100)
     a /= 2;
   return a;
 }

compiler-rt/test/profile/Inputs/instrprof-debug-info-correlate-foo.cpp

 #include "instrprof-debug-info-correlate-bar.h"
 
 int foo(int a) {
   if (a % 2)
     return 4 * a + 1;
   return bar(a);
 }
+
+int unused(int a) { return a * a; }

compiler-rt/test/profile/Inputs/instrprof-debug-info-correlate-main.cpp

 #include "instrprof-debug-info-correlate-bar.h"
 
 typedef int (*FP)(int);
-FP Fps[2] = {foo, bar};
+FP Fps[3] = {foo, bar, unused};
 
 int main() {
   for (int i = 0; i < 5; i++)
     Fps[i % 2](i);
   return 0;
 }

compiler-rt/test/profile/Linux/instrprof-debug-info-correlate.c

 // Value profiling is currently not supported in lightweight mode.
 // RUN: %clang_pgogen -o %t.normal -mllvm --disable-vp=true %S/../Inputs/instrprof-debug-info-correlate-main.cpp %S/../Inputs/instrprof-debug-info-correlate-foo.cpp
 // RUN: env LLVM_PROFILE_FILE=%t.profraw %run %t.normal
 // RUN: llvm-profdata merge -o %t.normal.profdata %t.profraw
 
 // RUN: %clang_pgogen -o %t.d4 -g -gdwarf-4 -mllvm --debug-info-correlate -mllvm --disable-vp=true %S/../Inputs/instrprof-debug-info-correlate-main.cpp %S/../Inputs/instrprof-debug-info-correlate-foo.cpp
 // RUN: env LLVM_PROFILE_FILE=%t.d4.proflite %run %t.d4
 // RUN: llvm-profdata merge -o %t.d4.profdata --debug-info=%t.d4 %t.d4.proflite
 
 // RUN: diff %t.normal.profdata %t.d4.profdata
 
 // RUN: %clang_pgogen -o %t -g -mllvm --debug-info-correlate -mllvm --disable-vp=true %S/../Inputs/instrprof-debug-info-correlate-main.cpp %S/../Inputs/instrprof-debug-info-correlate-foo.cpp
 // RUN: env LLVM_PROFILE_FILE=%t.proflite %run %t
 // RUN: llvm-profdata merge -o %t.profdata --debug-info=%t %t.proflite
 
 // RUN: diff %t.normal.profdata %t.profdata
+
+// RUN: %clang_pgogen -o %t.cov -g -mllvm --debug-info-correlate -mllvm -pgo-function-entry-coverage -mllvm --disable-vp=true %S/../Inputs/instrprof-debug-info-correlate-main.cpp %S/../Inputs/instrprof-debug-info-correlate-foo.cpp
+// RUN: env LLVM_PROFILE_FILE=%t.cov.proflite %run %t.cov
+// RUN: llvm-profdata merge -o %t.cov.profdata --debug-info=%t.cov %t.cov.proflite
+
+// RUN: %clang_pgogen -o %t.cov.normal -mllvm --pgo-function-entry-coverage -mllvm --disable-vp=true %S/../Inputs/instrprof-debug-info-correlate-main.cpp %S/../Inputs/instrprof-debug-info-correlate-foo.cpp
+// RUN: env LLVM_PROFILE_FILE=%t.cov.profraw %run %t.cov.normal
+// RUN: llvm-profdata merge -o %t.cov.normal.profdata %t.cov.profraw
+
+// RUN: diff %t.cov.normal.profdata %t.cov.profdata

compiler-rt/test/profile/instrprof-coverage.c

+// RUN: %clang_pgogen -mllvm -pgo-function-entry-coverage %s -o %t.out
+// RUN: env LLVM_PROFILE_FILE=%t.profraw %run %t.out
+// RUN: llvm-profdata merge -o %t.profdata %t.profraw
+// RUN: llvm-profdata show --covered %t.profdata | FileCheck %s --check-prefix CHECK --implicit-check-not goo
+
+int foo(int i) { return 4 * i + 1; }
+int bar(int i) { return 4 * i + 2; }
+int goo(int i) { return 4 * i + 3; }
+
+int main(int argc, char *argv[]) {
+  foo(5);
+  argc ? bar(6) : goo(7);
+  return 0;
+}
+
+// CHECK: main
+// CHECK: foo
+// CHECK: bar

compiler-rt/test/profile/instrprof-merge-entry-cover.c

+// RUN: %clang_pgogen -O2 -mllvm -pgo-function-entry-coverage -o %t %s
+// RUN: %run %t %t.profraw 1 1
+// RUN: llvm-profdata show --all-functions --counts %t.profraw  | FileCheck %s
+
+// FIXME: llvm-profdata exits with "Malformed instrumentation profile data"
+// XFAIL: msvc
+
+#include "profile_test.h"
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+
+int __llvm_profile_runtime = 0;
+uint64_t __llvm_profile_get_size_for_buffer(void);
+int __llvm_profile_write_buffer(char *);
+void __llvm_profile_reset_counters(void);
+int __llvm_profile_merge_from_buffer(const char *, uint64_t);
+
+__attribute__((noinline)) int dumpBuffer(const char *FileN, const char *Buffer,
+                                         uint64_t Size) {
+  FILE *File = fopen(FileN, "w");
+  if (!File)
+    return 1;
+  if (fwrite(Buffer, 1, Size, File) != Size)
+    return 1;
+  return fclose(File);
+}
+
+int g = 0;
+__attribute__((noinline)) void foo(char c) {
+  if (c == '1')
+    g++;
+  else
+    g--;
+}
+
+/* This function is not profiled */
+__attribute__((noinline)) void bar(int M) { g += M; }
+
+int main(int argc, const char *argv[]) {
+  int i;
+  if (argc < 4)
+    return 1;
+
+  const uint64_t MaxSize = 10000;
+  static ALIGNED(sizeof(uint64_t)) char Buffer[MaxSize];
+
+  uint64_t Size = __llvm_profile_get_size_for_buffer();
+  if (Size > MaxSize)
+    return 1;
+
+  /* Start profiling. */
+  __llvm_profile_reset_counters();
+  foo(argv[2][0]);
+  /* End profiling by freezing counters. */
+  if (__llvm_profile_write_buffer(Buffer))
+    return 1;
+
+  /* Its profile will be discarded. */
+  for (i = 0; i < 10; i++)
+    bar(1);
+
+  /* Start profiling again and merge in previously
+     saved counters in buffer. */
+  __llvm_profile_reset_counters();
+  __llvm_profile_merge_from_buffer(Buffer, Size);
+  foo(argv[3][0]);
+  /* End profiling */
+  if (__llvm_profile_write_buffer(Buffer))
+    return 1;
+
+  /* Its profile will be discarded. */
+  bar(2);
+
+  /* Now it is time to dump the profile to file.  */
+  return dumpBuffer(argv[1], Buffer, Size);
+}
+
+// CHECK-LABEL: dumpBuffer:
+// CHECK:        Counters: 1
+// CHECK-NEXT:    Block counts: [0]
+
+// CHECK-LABEL:  foo:
+// CHECK:         Counters: 1
+// CHECK-NEXT:    Block counts: [1]
+
+// CHECK-LABEL:  bar:
+// CHECK:         Counters: 1
+// CHECK-NEXT:    Block counts: [0]
+
+// CHECK-LABEL:  main:
+// CHECK:         Counters: 1
+// CHECK-NEXT:    Block counts: [0]

llvm/docs/LangRef.rst

 intrinsic.
 
 The last argument specifies the value of the increment of the counter variable.
 
 Semantics:
 """"""""""
 See description of '``llvm.instrprof.increment``' intrinsic.
 
+'``llvm.instrprof.cover``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare void @llvm.instrprof.cover(i8* <name>, i64 <hash>,
+                                         i32 <num-counters>, i32 <index>)
+
+Overview:
+"""""""""
+
+The '``llvm.instrprof.cover``' intrinsic is used to implement coverage
+instrumentation.
+
+Arguments:
+""""""""""
+The arguments are the same as the first four arguments of
+'``llvm.instrprof.increment``'.
+
+Semantics:
+""""""""""
+Similar to the '``llvm.instrprof.increment``' intrinsic, but it stores zero to
+the profiling variable to signify that the function has been covered. We store
+zero because this is more efficient on some targets.
 
 '``llvm.instrprof.value.profile``' Intrinsic
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Syntax:
 """""""
 
 ::

llvm/include/llvm/IR/IntrinsicInst.h

ConstantInt *getHash() const {
     return cast<ConstantInt>(const_cast<Value *>(getArgOperand(1)));
   }
   // The number of counters for the instrumented function.
   ConstantInt *getNumCounters() const;
   // The index of the counter that this instruction acts on.
   ConstantInt *getIndex() const;
 };
 
+/// This represents the llvm.instrprof.cover intrinsic.
+class InstrProfCoverInst : public InstrProfInstBase {
+public:
+  static bool classof(const IntrinsicInst *I) {
+    return I->getIntrinsicID() == Intrinsic::instrprof_cover;
+  }
+  static bool classof(const Value *V) {
+    return isa<IntrinsicInst>(V) && classof(cast<IntrinsicInst>(V));
+  }
+};
+
 /// This represents the llvm.instrprof.increment intrinsic.
 class InstrProfIncrementInst : public InstrProfInstBase {
 public:
   static bool classof(const IntrinsicInst *I) {
     return I->getIntrinsicID() == Intrinsic::instrprof_increment;
   }
   static bool classof(const Value *V) {
     return isa<IntrinsicInst>(V) && classof(cast<IntrinsicInst>(V));

 class InstrProfValueProfileInst : public InstrProfInstBase {
 public:
   static bool classof(const IntrinsicInst *I) {
     return I->getIntrinsicID() == Intrinsic::instrprof_value_profile;
   }
   static bool classof(const Value *V) {
     return isa<IntrinsicInst>(V) && classof(cast<IntrinsicInst>(V));
   }
 
   Value *getTargetValue() const {
     return cast<Value>(const_cast<Value *>(getArgOperand(2)));
   }
 
   ConstantInt *getValueKind() const {
     return cast<ConstantInt>(const_cast<Value *>(getArgOperand(3)));
   }
 
   // Returns the value site index.

llvm/include/llvm/IR/Intrinsics.td

def int_experimental_noalias_scope_decl
     : DefaultAttrsIntrinsic<[], [llvm_metadata_ty],
         [IntrInaccessibleMemOnly]>; // blocks LICM and some more
 
 // Stack Protector Intrinsic - The stackprotector intrinsic writes the stack
 // guard to the correct place on the stack frame.
 def int_stackprotector : DefaultAttrsIntrinsic<[], [llvm_ptr_ty, llvm_ptrptr_ty], []>;
 def int_stackguard : DefaultAttrsIntrinsic<[llvm_ptr_ty], [], []>;
 
+// A cover for instrumentation based profiling.
+def int_instrprof_cover : Intrinsic<[], [llvm_ptr_ty, llvm_i64_ty,
+                                         llvm_i32_ty, llvm_i32_ty]>;
+
 // A counter increment for instrumentation based profiling.
 def int_instrprof_increment : Intrinsic<[],
                                         [llvm_ptr_ty, llvm_i64_ty,
                                          llvm_i32_ty, llvm_i32_ty]>;
 
 // A counter increment with step for instrumentation based profiling.
 def int_instrprof_increment_step : Intrinsic<[],
                                         [llvm_ptr_ty, llvm_i64_ty,

llvm/include/llvm/ProfileData/InstrProf.h

 
 /// An enum describing the attributes of an instrumented profile.
 enum class InstrProfKind {
   Unknown = 0x0,
   FE = 0x1, // A frontend clang profile, incompatible with other attrs.
   IR = 0x2, // An IR-level profile (default when -fprofile-generate is used).
   BB = 0x4, // A profile with entry basic block instrumentation.
   CS = 0x8, // A context sensitive IR-level profile.
-  LLVM_MARK_AS_BITMASK_ENUM(/*LargestValue=*/CS)
+  SingleByteCoverage = 0x10, // Use single byte probes for coverage.
+  FunctionEntryOnly = 0x20,  // Only instrument the function entry basic block.
+  LLVM_MARK_AS_BITMASK_ENUM(/*LargestValue=*/FunctionEntryOnly)
 };
 
 const std::error_category &instrprof_category();
 
 enum class instrprof_error {
   success = 0,
   eof,
   unrecognized_format,

 // Parse MemOP Size range option.
 void getMemOPSizeRangeFromOption(StringRef Str, int64_t &RangeStart,
                                  int64_t &RangeLast);
 
 // Create a COMDAT variable INSTR_PROF_RAW_VERSION_VAR to make the runtime
 // aware this is an ir_level profile so it can set the version flag.
 GlobalVariable *createIRLevelProfileFlagVar(Module &M, bool IsCS,
                                             bool InstrEntryBBEnabled,
-                                            bool DebugInfoCorrelate);
+                                            bool DebugInfoCorrelate,
+                                            bool PGOFunctionEntryCoverage);
 
 // Create the variable for the profile file name.
 void createProfileFileNameVar(Module &M, StringRef InstrProfileOutput);
 
 // Whether to compress function names in profile records, and filenames in
 // code coverage mappings. Used by the Instrumentation library and unit tests.
 extern cl::opt<bool> DoInstrProfNameCompression;
 
 } // end namespace llvm
 #endif // LLVM_PROFILEDATA_INSTRPROF_H

llvm/include/llvm/ProfileData/InstrProfData.inc

 #define INSTR_PROF_COVMAP_VERSION 5
 
 /* Profile version is always of type uint64_t. Reserve the upper 8 bits in the
  * version for other variants of profile. We set the lowest bit of the upper 8
  * bits (i.e. bit 56) to 1 to indicate if this is an IR-level instrumentation
  * generated profile, and 0 if this is a Clang FE generated profile.
  * 1 in bit 57 indicates there are context-sensitive records in the profile.
  * The 59th bit indicates whether to use debug info to correlate profiles.
+ * The 60th bit indicates single byte coverage instrumentation.
+ * The 61st bit indicates function entry instrumentation only.
  */
 #define VARIANT_MASKS_ALL 0xff00000000000000ULL
 #define GET_VERSION(V) ((V) & ~VARIANT_MASKS_ALL)
 #define VARIANT_MASK_IR_PROF (0x1ULL << 56)
 #define VARIANT_MASK_CSIR_PROF (0x1ULL << 57)
 #define VARIANT_MASK_INSTR_ENTRY (0x1ULL << 58)
 #define VARIANT_MASK_DBG_CORRELATE (0x1ULL << 59)
+#define VARIANT_MASK_BYTE_COVERAGE (0x1ULL << 60)
+#define VARIANT_MASK_FUNCTION_ENTRY_ONLY (0x1ULL << 61)
 #define INSTR_PROF_RAW_VERSION_VAR __llvm_profile_raw_version
 #define INSTR_PROF_PROFILE_RUNTIME_VAR __llvm_profile_runtime
 #define INSTR_PROF_PROFILE_COUNTER_BIAS_VAR __llvm_profile_counter_bias
 
 /* The variable that holds the name of the profile data
  * specified via command line. */
 #define INSTR_PROF_PROFILE_NAME_VAR __llvm_profile_filename
 

llvm/include/llvm/ProfileData/InstrProfReader.h

public:
 
   virtual bool hasCSIRLevelProfile() const = 0;
 
   virtual bool instrEntryBBEnabled() const = 0;
 
   /// Return true if we must provide debug info to create PGO profiles.
   virtual bool useDebugInfoCorrelate() const { return false; }
 
+  /// Return true if the profile has single byte counters representing coverage.
+  virtual bool hasSingleByteCoverage() const = 0;
+
+  /// Return true if the profile only instruments function entries.
+  virtual bool functionEntryOnly() const = 0;
+
   /// Returns a BitsetEnum describing the attributes of the profile. To check
   /// individual attributes prefer using the helpers above.
   virtual InstrProfKind getProfileKind() const = 0;
 
   /// Return the PGO symtab. There are three different readers:
   /// Raw, Text, and Indexed profile readers. The first two types
   /// of readers are used only by llvm-profdata tool, while the indexed
   /// profile reader is also used by llvm-cov tool and the compiler (
public:
   bool hasCSIRLevelProfile() const override {
     return static_cast<bool>(ProfileKind & InstrProfKind::CS);
   }
 
   bool instrEntryBBEnabled() const override {
     return static_cast<bool>(ProfileKind & InstrProfKind::BB);
   }
 
+  bool hasSingleByteCoverage() const override {
+    return static_cast<bool>(ProfileKind & InstrProfKind::SingleByteCoverage);
+  }
+
+  bool functionEntryOnly() const override {
+    return static_cast<bool>(ProfileKind & InstrProfKind::FunctionEntryOnly);
+  }
+
   InstrProfKind getProfileKind() const override { return ProfileKind; }
 
   /// Read the header.
   Error readHeader() override;
 
   /// Read a single record.
   Error readNextRecord(NamedInstrProfRecord &Record) override;
 
public:
   bool instrEntryBBEnabled() const override {
     return (Version & VARIANT_MASK_INSTR_ENTRY) != 0;
   }
 
   bool useDebugInfoCorrelate() const override {
     return (Version & VARIANT_MASK_DBG_CORRELATE) != 0;
   }
 
+  bool hasSingleByteCoverage() const override {
+    return (Version & VARIANT_MASK_BYTE_COVERAGE) != 0;
+  }
+
+  bool functionEntryOnly() const override {
+    return (Version & VARIANT_MASK_FUNCTION_ENTRY_ONLY) != 0;
+  }
+
   /// Returns a BitsetEnum describing the attributes of the raw instr profile.
   InstrProfKind getProfileKind() const override {
     InstrProfKind ProfileKind = InstrProfKind::Unknown;
     if (Version & VARIANT_MASK_IR_PROF) {
       ProfileKind |= InstrProfKind::IR;
     }
     if (Version & VARIANT_MASK_CSIR_PROF) {
       ProfileKind |= InstrProfKind::CS;
     }
     if (Version & VARIANT_MASK_INSTR_ENTRY) {
       ProfileKind |= InstrProfKind::BB;
     }
+    if (Version & VARIANT_MASK_BYTE_COVERAGE) {
+      ProfileKind |= InstrProfKind::SingleByteCoverage;
+    }
+    if (Version & VARIANT_MASK_FUNCTION_ENTRY_ONLY) {
+      ProfileKind |= InstrProfKind::FunctionEntryOnly;
+    }
     return ProfileKind;
   }
 
   InstrProfSymtab &getSymtab() override {
     assert(Symtab.get());
     return *Symtab.get();
   }
 
const char *getNextHeaderPos() const {
       assert(atEnd());
       return (const char *)ValueDataStart;
   }
 
   StringRef getName(uint64_t NameRef) const {
     return Symtab->getFuncName(swap(NameRef));
   }
 
-  int getCounterTypeSize() const { return sizeof(uint64_t); }
+  int getCounterTypeSize() const {
+    return hasSingleByteCoverage() ? sizeof(uint8_t) : sizeof(uint64_t);
+  }
 };
 
 using RawInstrProfReader32 = RawInstrProfReader<uint32_t>;
 using RawInstrProfReader64 = RawInstrProfReader<uint64_t>;
 
 namespace IndexedInstrProf {
 
 enum class HashT : uint32_t;
virtual Error getRecords(StringRef FuncName,
                                      ArrayRef<NamedInstrProfRecord> &Data) = 0;
   virtual void advanceToNextKey() = 0;
   virtual bool atEnd() const = 0;
   virtual void setValueProfDataEndianness(support::endianness Endianness) = 0;
   virtual uint64_t getVersion() const = 0;
   virtual bool isIRLevelProfile() const = 0;
   virtual bool hasCSIRLevelProfile() const = 0;
   virtual bool instrEntryBBEnabled() const = 0;
+  virtual bool hasSingleByteCoverage() const = 0;
+  virtual bool functionEntryOnly() const = 0;
   virtual InstrProfKind getProfileKind() const = 0;
   virtual Error populateSymtab(InstrProfSymtab &) = 0;
 };
 
 using OnDiskHashTableImplV3 =
     OnDiskIterableChainedHashTable<InstrProfLookupTrait>;
 
 template <typename HashTableImpl>
public:
   bool hasCSIRLevelProfile() const override {
     return (FormatVersion & VARIANT_MASK_CSIR_PROF) != 0;
   }
 
   bool instrEntryBBEnabled() const override {
     return (FormatVersion & VARIANT_MASK_INSTR_ENTRY) != 0;
   }
 
+  bool hasSingleByteCoverage() const override {
+    return (FormatVersion & VARIANT_MASK_BYTE_COVERAGE) != 0;
+  }
+
+  bool functionEntryOnly() const override {
+    return (FormatVersion & VARIANT_MASK_FUNCTION_ENTRY_ONLY) != 0;
+  }
+
   InstrProfKind getProfileKind() const override {
     InstrProfKind ProfileKind = InstrProfKind::Unknown;
     if (FormatVersion & VARIANT_MASK_IR_PROF) {
       ProfileKind |= InstrProfKind::IR;
     }
     if (FormatVersion & VARIANT_MASK_CSIR_PROF) {
       ProfileKind |= InstrProfKind::CS;
     }
     if (FormatVersion & VARIANT_MASK_INSTR_ENTRY) {
       ProfileKind |= InstrProfKind::BB;
     }
+    if (FormatVersion & VARIANT_MASK_BYTE_COVERAGE) {
+      ProfileKind |= InstrProfKind::SingleByteCoverage;
+    }
+    if (FormatVersion & VARIANT_MASK_FUNCTION_ENTRY_ONLY) {
+      ProfileKind |= InstrProfKind::FunctionEntryOnly;
+    }
     return ProfileKind;
   }
 
   Error populateSymtab(InstrProfSymtab &Symtab) override {
     return Symtab.create(HashTable->keys());
   }
 };
 
public:
   bool hasCSIRLevelProfile() const override {
     return Index->hasCSIRLevelProfile();
   }
 
   bool instrEntryBBEnabled() const override {
     return Index->instrEntryBBEnabled();
   }
 
+  bool hasSingleByteCoverage() const override {
+    return Index->hasSingleByteCoverage();
+  }
+
+  bool functionEntryOnly() const override { return Index->functionEntryOnly(); }
+
   /// Returns a BitsetEnum describing the attributes of the indexed instr
   /// profile.
   InstrProfKind getProfileKind() const override {
     return Index->getProfileKind();
   }
 
   /// Return true if the given buffer is in an indexed instrprof format.
   static bool hasFormat(const MemoryBuffer &DataBuffer);

llvm/include/llvm/ProfileData/InstrProfWriter.h

public:
   Error mergeProfileKind(const InstrProfKind Other) {
     // If the kind is unset, this is the first profile we are merging so just
     // set it to the given type.
     if (ProfileKind == InstrProfKind::Unknown) {
       ProfileKind = Other;
       return Error::success();
     }
 
+    // Returns true if merging is should fail assuming A and B are incompatible.
+    auto testIncompatible = [&](InstrProfKind A, InstrProfKind B) {
+      return (static_cast<bool>(ProfileKind & A) &&
+              static_cast<bool>(Other & B)) ||
+             (static_cast<bool>(ProfileKind & B) &&
+              static_cast<bool>(Other & A));
+    };
+
     // Check if the profiles are in-compatible. Clang frontend profiles can't be
     // merged with other profile types.
     if (static_cast<bool>((ProfileKind & InstrProfKind::FE) ^
                           (Other & InstrProfKind::FE))) {
       return make_error<InstrProfError>(instrprof_error::unsupported_version);
     }
+    if (testIncompatible(InstrProfKind::FunctionEntryOnly, InstrProfKind::BB)) {
+      return make_error<InstrProfError>(
+          instrprof_error::unsupported_version,
+          "cannot merge FunctionEntryOnly profiles and BB profiles together");
+    }
 
     // Now we update the profile type with the bits that are set.
     ProfileKind |= Other;
     return Error::success();
   }
 
   // Internal interface for testing purpose only.
   void setValueProfDataEndianness(support::endianness Endianness);

llvm/include/llvm/Transforms/Instrumentation/InstrProfiling.h

private:
   bool isRuntimeCounterRelocationEnabled() const;
 
   /// Returns true if profile counter update register promotion is enabled.
   bool isCounterPromotionEnabled() const;
 
   /// Count the number of instrumented value sites for the function.
   void computeNumValueSiteCounts(InstrProfValueProfileInst *Ins);
 
-  /// Replace instrprof_value_profile with a call to runtime library.
+  /// Replace instrprof.value.profile with a call to runtime library.
   void lowerValueProfileInst(InstrProfValueProfileInst *Ins);
 
-  /// Replace instrprof_increment with an increment of the appropriate value.
+  /// Replace instrprof.cover with a store instruction to the coverage byte.
+  void lowerCover(InstrProfCoverInst *Inc);
+
+  /// Replace instrprof.increment with an increment of the appropriate value.
   void lowerIncrement(InstrProfIncrementInst *Inc);
 
   /// Force emitting of name vars for unused functions.
   void lowerCoverageData(GlobalVariable *CoverageNamesVar);
 
+  /// Compute the address of the counter value that this profiling instruction
+  /// acts on.
+  Value *getCounterAddress(InstrProfInstBase *I);
+
   /// Get the region counters for an increment, creating them if necessary.
   ///
   /// If the counter array doesn't yet exist, the profile data variables
   /// referring to them will also be created.
   GlobalVariable *getOrCreateRegionCounters(InstrProfInstBase *Inc);
 
+  /// Create the region counters.
+  GlobalVariable *createRegionCounters(InstrProfInstBase *Inc, StringRef Name,
+                                       GlobalValue::LinkageTypes Linkage);
+
   /// Emit the section with compressed function names.
   void emitNameData();
 
   /// Emit value nodes section for value profiling.
   void emitVNodes();
 
   /// Emit runtime registration functions for each profile data variable.
   void emitRegistration();

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

case Intrinsic::experimental_gc_statepoint:
     LowerStatepoint(cast<GCStatepointInst>(I));
     return;
   case Intrinsic::experimental_gc_result:
     visitGCResult(cast<GCResultInst>(I));
     return;
   case Intrinsic::experimental_gc_relocate:
     visitGCRelocate(cast<GCRelocateInst>(I));
     return;
+  case Intrinsic::instrprof_cover:
+    llvm_unreachable("instrprof failed to lower a cover");
   case Intrinsic::instrprof_increment:
     llvm_unreachable("instrprof failed to lower an increment");
   case Intrinsic::instrprof_value_profile:
     llvm_unreachable("instrprof failed to lower a value profiling call");
   case Intrinsic::localescape: {
     MachineFunction &MF = DAG.getMachineFunction();
     const TargetInstrInfo *TII = DAG.getSubtarget().getInstrInfo();
 

llvm/lib/ProfileData/InstrProf.cpp

bool canRenameComdatFunc(const Function &F, bool CheckAddressTaken) {
   }
   return true;
 }
 
 // Create a COMDAT variable INSTR_PROF_RAW_VERSION_VAR to make the runtime
 // aware this is an ir_level profile so it can set the version flag.
 GlobalVariable *createIRLevelProfileFlagVar(Module &M, bool IsCS,
                                             bool InstrEntryBBEnabled,
-                                            bool DebugInfoCorrelate) {
+                                            bool DebugInfoCorrelate,
+                                            bool PGOFunctionEntryCoverage) {
   const StringRef VarName(INSTR_PROF_QUOTE(INSTR_PROF_RAW_VERSION_VAR));
   Type *IntTy64 = Type::getInt64Ty(M.getContext());
   uint64_t ProfileVersion = (INSTR_PROF_RAW_VERSION | VARIANT_MASK_IR_PROF);
   if (IsCS)
     ProfileVersion |= VARIANT_MASK_CSIR_PROF;
   if (InstrEntryBBEnabled)
     ProfileVersion |= VARIANT_MASK_INSTR_ENTRY;
   if (DebugInfoCorrelate)
     ProfileVersion |= VARIANT_MASK_DBG_CORRELATE;
+  if (PGOFunctionEntryCoverage)
+    ProfileVersion |=
+        VARIANT_MASK_BYTE_COVERAGE | VARIANT_MASK_FUNCTION_ENTRY_ONLY;
   auto IRLevelVersionVariable = new GlobalVariable(
       M, IntTy64, true, GlobalValue::WeakAnyLinkage,
       Constant::getIntegerValue(IntTy64, APInt(64, ProfileVersion)), VarName);
   IRLevelVersionVariable->setVisibility(GlobalValue::DefaultVisibility);
   Triple TT(M.getTargetTriple());
   if (TT.supportsCOMDAT()) {
     IRLevelVersionVariable->setLinkage(GlobalValue::ExternalLinkage);
     IRLevelVersionVariable->setComdat(M.getOrInsertComdat(VarName));

llvm/lib/ProfileData/InstrProfReader.cpp

return error(instrprof_error::malformed,
                  ("number of counters " + Twine(NumCounters) +
                   " is greater than the maximum number of counters " +
                   Twine(MaxNumCounters))
                      .str());
 
   Record.Counts.clear();
   Record.Counts.reserve(NumCounters);
   for (uint32_t I = 0; I < NumCounters; I++) {
-    const auto *CounterValue = reinterpret_cast<const uint64_t *>(
-        CountersStart + CounterBaseOffset + I * getCounterTypeSize());
+    const char *Ptr =
+        CountersStart + CounterBaseOffset + I * getCounterTypeSize();
+    if (hasSingleByteCoverage()) {
+      // A value of zero signifies the block is covered.
+      Record.Counts.push_back(*Ptr == 0 ? 1 : 0);
+    } else {
+      const auto *CounterValue = reinterpret_cast<const uint64_t *>(Ptr);
       Record.Counts.push_back(swap(*CounterValue));
     }
+  }
 
   return success();
 }
 
 template <class IntPtrT>
 Error RawInstrProfReader<IntPtrT>::readValueProfilingData(
     InstrProfRecord &Record) {
   Record.clearValueData();

llvm/lib/ProfileData/InstrProfWriter.cpp

Error InstrProfWriter::writeImpl(ProfOStream &OS) {
   Header.Magic = IndexedInstrProf::Magic;
   Header.Version = IndexedInstrProf::ProfVersion::CurrentVersion;
   if (static_cast<bool>(ProfileKind & InstrProfKind::IR))
     Header.Version |= VARIANT_MASK_IR_PROF;
   if (static_cast<bool>(ProfileKind & InstrProfKind::CS))
     Header.Version |= VARIANT_MASK_CSIR_PROF;
   if (static_cast<bool>(ProfileKind & InstrProfKind::BB))
     Header.Version |= VARIANT_MASK_INSTR_ENTRY;
+  if (static_cast<bool>(ProfileKind & InstrProfKind::SingleByteCoverage))
+    Header.Version |= VARIANT_MASK_BYTE_COVERAGE;
+  if (static_cast<bool>(ProfileKind & InstrProfKind::FunctionEntryOnly))
+    Header.Version |= VARIANT_MASK_FUNCTION_ENTRY_ONLY;
 
   Header.Unused = 0;
   Header.HashType = static_cast<uint64_t>(IndexedInstrProf::HashType);
   Header.HashOffset = 0;
   int N = sizeof(IndexedInstrProf::Header) / sizeof(uint64_t);
 
   // Only write out all the fields except 'HashOffset'. We need
   // to remember the offset of that field to allow back patching

llvm/lib/Transforms/Instrumentation/InstrProfiling.cpp

bool InstrProfiling::lowerIntrinsics(Function *F) {
   for (BasicBlock &BB : *F) {
     for (Instruction &Instr : llvm::make_early_inc_range(BB)) {
       if (auto *IPIS = dyn_cast<InstrProfIncrementInstStep>(&Instr)) {
         lowerIncrement(IPIS);
         MadeChange = true;
       } else if (auto *IPI = dyn_cast<InstrProfIncrementInst>(&Instr)) {
         lowerIncrement(IPI);
         MadeChange = true;
+      } else if (auto *IPC = dyn_cast<InstrProfCoverInst>(&Instr)) {
+        lowerCover(IPC);
+        MadeChange = true;
       } else if (auto *IPVP = dyn_cast<InstrProfValueProfileInst>(&Instr)) {
         lowerValueProfileInst(IPVP);
         MadeChange = true;
       }
     }
   }
 
   if (!MadeChange)

 
 /// Check if the module contains uses of any profiling intrinsics.
 static bool containsProfilingIntrinsics(Module &M) {
   auto containsIntrinsic = [&](int ID) {
     if (auto *F = M.getFunction(Intrinsic::getName(ID)))
       return !F->use_empty();
     return false;
   };
-  return containsIntrinsic(llvm::Intrinsic::instrprof_increment) ||
+  return containsIntrinsic(llvm::Intrinsic::instrprof_cover) ||
+         containsIntrinsic(llvm::Intrinsic::instrprof_increment) ||
          containsIntrinsic(llvm::Intrinsic::instrprof_increment_step) ||
          containsIntrinsic(llvm::Intrinsic::instrprof_value_profile);
 }
 
 bool InstrProfiling::run(
     Module &M, std::function<const TargetLibraryInfo &(Function &F)> GetTLI) {
   this->M = &M;
   this->GetTLI = std::move(GetTLI);
Call = Builder.CreateCall(
         Args, OpBundles);
   }
   if (auto AK = TLI->getExtAttrForI32Param(false))
     Call->addParamAttr(2, AK);
   Ind->replaceAllUsesWith(Call);
   Ind->eraseFromParent();
 }
 
-void InstrProfiling::lowerIncrement(InstrProfIncrementInst *Inc) {
-  GlobalVariable *Counters = getOrCreateRegionCounters(Inc);
+Value *InstrProfiling::getCounterAddress(InstrProfInstBase *I) {
+  auto *Counters = getOrCreateRegionCounters(I);
+  IRBuilder<> Builder(I);
 
-  IRBuilder<> Builder(Inc);
-  uint64_t Index = Inc->getIndex()->getZExtValue();
-  Value *Addr = Builder.CreateConstInBoundsGEP2_32(Counters->getValueType(),
-                                                   Counters, 0, Index);
+  auto *Addr = Builder.CreateConstInBoundsGEP2_32(
+      Counters->getValueType(), Counters, 0, I->getIndex()->getZExtValue());
+
+  if (!isRuntimeCounterRelocationEnabled())
+    return Addr;
 
-  if (isRuntimeCounterRelocationEnabled()) {
-    Type *Int64Ty = Type::getInt64Ty(M->getContext());
-    Type *Int64PtrTy = Type::getInt64PtrTy(M->getContext());
-    Function *Fn = Inc->getParent()->getParent();
-    Instruction &I = Fn->getEntryBlock().front();
-    LoadInst *LI = dyn_cast<LoadInst>(&I);
-    if (!LI) {
-      IRBuilder<> Builder(&I);
-      GlobalVariable *Bias =
-          M->getGlobalVariable(getInstrProfCounterBiasVarName());
-      if (!Bias) {
-        // Compiler must define this variable when runtime counter relocation
-        // is being used. Runtime has a weak external reference that is used
-        // to check whether that's the case or not.
-        Bias = new GlobalVariable(
-            *M, Int64Ty, false, GlobalValue::LinkOnceODRLinkage,
-            Constant::getNullValue(Int64Ty), getInstrProfCounterBiasVarName());
-        Bias->setVisibility(GlobalVariable::HiddenVisibility);
-        // A definition that's weak (linkonce_odr) without being in a COMDAT
-        // section wouldn't lead to link errors, but it would lead to a dead
-        // data word from every TU but one. Putting it in COMDAT ensures there
-        // will be exactly one data slot in the link.
-        if (TT.supportsCOMDAT())
-          Bias->setComdat(M->getOrInsertComdat(Bias->getName()));
-      }
-      LI = Builder.CreateLoad(Int64Ty, Bias);
-    }
-    auto *Add = Builder.CreateAdd(Builder.CreatePtrToInt(Addr, Int64Ty), LI);
-    Addr = Builder.CreateIntToPtr(Add, Int64PtrTy);
-  }
+  Function *Fn = I->getParent()->getParent();
+  Instruction &EntryI = Fn->getEntryBlock().front();
+  LoadInst *LI = dyn_cast<LoadInst>(&EntryI);
+    IRBuilder<> EntryBuilder(&EntryI);
+    auto *Bias = M->getGlobalVariable(getInstrProfCounterBiasVarName());
+    LI = EntryBuilder.CreateLoad(Int64Ty, Bias);
+  return Builder.CreateIntToPtr(Add, Addr->getType());
 
+void InstrProfiling::lowerCover(InstrProfCoverInst *CoverInstruction) {
+  auto *Addr = getCounterAddress(CoverInstruction);
+  IRBuilder<> Builder(CoverInstruction);
+  // We store zero to represent that this block is covered.
+  Builder.CreateStore(Builder.getInt8(0), Addr);
+  CoverInstruction->eraseFromParent();
+}
+
+void InstrProfiling::lowerIncrement(InstrProfIncrementInst *Inc) {
+  auto *Addr = getCounterAddress(Inc);
+
+  IRBuilder<> Builder(Inc);
   if (Options.Atomic || AtomicCounterUpdateAll ||
-      (Index == 0 && AtomicFirstCounter)) {
+      (Inc->getIndex()->isZeroValue() && AtomicFirstCounter)) {
     Builder.CreateAtomicRMW(AtomicRMWInst::Add, Addr, Inc->getStep(),
                             MaybeAlign(), AtomicOrdering::Monotonic);
   } else {
     Value *IncStep = Inc->getStep();
     Value *Load = Builder.CreateLoad(IncStep->getType(), Addr, "pgocount");
     auto *Count = Builder.CreateAdd(Load, Inc->getStep());
     auto *Store = Builder.CreateStore(Count, Addr);
     if (isCounterPromotionEnabled())
static bool needsRuntimeRegistrationOfSectionRange(const Triple &TT) {
   if (TT.isOSLinux() || TT.isOSFreeBSD() || TT.isOSNetBSD() ||
       TT.isOSSolaris() || TT.isOSFuchsia() || TT.isPS4CPU() || TT.isOSWindows())
     return false;
 
   return true;
 }
 
 GlobalVariable *
+InstrProfiling::createRegionCounters(InstrProfInstBase *Inc, StringRef Name,
+                                     GlobalValue::LinkageTypes Linkage) {
+  uint64_t NumCounters = Inc->getNumCounters()->getZExtValue();
+  auto &Ctx = M->getContext();
+  GlobalVariable *GV;
+  if (isa<InstrProfCoverInst>(Inc)) {
+    auto *CounterTy = Type::getInt8Ty(Ctx);
+    auto *CounterArrTy = ArrayType::get(CounterTy, NumCounters);
+    // TODO: `Constant::getAllOnesValue()` does not yet accept an array type.
+    std::vector<Constant *> InitialValues(NumCounters,
+                                          Constant::getAllOnesValue(CounterTy));
+    GV = new GlobalVariable(*M, CounterArrTy, false, Linkage,
+                            ConstantArray::get(CounterArrTy, InitialValues),
+                            Name);
+    GV->setAlignment(Align(1));
+  } else {
+    auto *CounterTy = ArrayType::get(Type::getInt64Ty(Ctx), NumCounters);
+    GV = new GlobalVariable(*M, CounterTy, false, Linkage,
+                            Constant::getNullValue(CounterTy), Name);
+    GV->setAlignment(Align(8));
+  }
+  return GV;
+}
+
+GlobalVariable *
 InstrProfiling::getOrCreateRegionCounters(InstrProfInstBase *Inc) {
   GlobalVariable *NamePtr = Inc->getName();
   auto &PD = ProfileDataMap[NamePtr];
   if (PD.RegionCounters)
     return PD.RegionCounters;
 
   // Match the linkage and visibility of the name global.
   Function *Fn = Inc->getParent()->getParent();
if (UseComdat) {
       if (!NeedComdat)
         C->setSelectionKind(Comdat::NoDeduplicate);
       GV->setComdat(C);
     }
   };
 
   uint64_t NumCounters = Inc->getNumCounters()->getZExtValue();
   LLVMContext &Ctx = M->getContext();
-  ArrayType *CounterTy = ArrayType::get(Type::getInt64Ty(Ctx), NumCounters);
 
-  // Create the counters variable.
-  auto *CounterPtr =
-      new GlobalVariable(*M, CounterTy, false, Linkage,
-                         Constant::getNullValue(CounterTy), CntsVarName);
+  auto *CounterPtr = createRegionCounters(Inc, CntsVarName, Linkage);
   CounterPtr->setVisibility(Visibility);
   CounterPtr->setSection(
       getInstrProfSectionName(IPSK_cnts, TT.getObjectFormat()));
-  CounterPtr->setAlignment(Align(8));
   MaybeSetComdat(CounterPtr);
   CounterPtr->setLinkage(Linkage);
   PD.RegionCounters = CounterPtr;
   if (DebugInfoCorrelate) {
     if (auto *SP = Fn->getSubprogram()) {
       DIBuilder DB(*M, true, SP->getUnit());
       Metadata *FunctionNameAnnotation[] = {
           MDString::get(Ctx, InstrProfCorrelator::FunctionNameAttributeName),
           MDString::get(Ctx, getPGOFuncNameVarInitializer(NamePtr)),
       };
       Metadata *CFGHashAnnotation[] = {
ValuesVar->setSection(
         getInstrProfSectionName(IPSK_vals, TT.getObjectFormat()));
     ValuesVar->setAlignment(Align(8));
     MaybeSetComdat(ValuesVar);
     ValuesPtrExpr =
         ConstantExpr::getBitCast(ValuesVar, Type::getInt8PtrTy(Ctx));
   }
 
   if (DebugInfoCorrelate) {
     // Mark the counter variable as used so that it isn't optimized out.
     CompilerUsedVars.push_back(PD.RegionCounters);
     return PD.RegionCounters;
   }
 
   // Create data variable.
   auto *IntPtrTy = M->getDataLayout().getIntPtrType(M->getContext());
   auto *Int16Ty = Type::getInt16Ty(Ctx);
   auto *Int16ArrayTy = ArrayType::get(Int16Ty, IPVK_Last + 1);

llvm/lib/Transforms/Instrumentation/PGOInstrumentation.cpp

EmitBranchProbability("pgo-emit-branch-prob", cl::init(false), cl::Hidden,
                                    "branch probability will be emitted as "
                                    "optimization remarks: -{Rpass|"
                                    "pass-remarks}=pgo-instrumentation"));
 
 static cl::opt<bool> PGOInstrumentEntry(
     "pgo-instrument-entry", cl::init(false), cl::Hidden,
     cl::desc("Force to instrument function entry basicblock."));
 
+static cl::opt<bool> PGOFunctionEntryCoverage(
+    "pgo-function-entry-coverage", cl::init(false), cl::Hidden, cl::ZeroOrMore,
+    cl::desc(
+        "Use this option to enable function entry coverage instrumentation."));
+
 static cl::opt<bool>
     PGOFixEntryCount("pgo-fix-entry-count", cl::init(true), cl::Hidden,
                      cl::desc("Fix function entry count in profile use."));
 
 static cl::opt<bool> PGOVerifyHotBFI(
     "pgo-verify-hot-bfi", cl::init(false), cl::Hidden,
     cl::desc("Print out the non-match BFI count if a hot raw profile count "
              "becomes non-hot, or a cold raw profile count becomes hot. "
initializePGOInstrumentationGenCreateVarLegacyPassPass(
         *PassRegistry::getPassRegistry());
   }
 
 private:
   bool runOnModule(Module &M) override {
     createProfileFileNameVar(M, InstrProfileOutput);
     // The variable in a comdat may be discarded by LTO. Ensure the
     // declaration will be retained.
-    appendToCompilerUsed(M, createIRLevelProfileFlagVar(M, /*IsCS=*/true,
-                                                        PGOInstrumentEntry,
-                                                        DebugInfoCorrelate));
+    appendToCompilerUsed(M, createIRLevelProfileFlagVar(
+                                M, /*IsCS=*/true, PGOInstrumentEntry,
+                                DebugInfoCorrelate, PGOFunctionEntryCoverage));
     return false;
   }
   std::string InstrProfileOutput;
 };
 
 } // end anonymous namespace
 
 char PGOInstrumentationGenLegacyPass::ID = 0;
static void instrumentOneFunc(
     std::unordered_multimap<Comdat *, GlobalValue *> &ComdatMembers,
     bool IsCS) {
   // Split indirectbr critical edges here before computing the MST rather than
   // later in getInstrBB() to avoid invalidating it.
   SplitIndirectBrCriticalEdges(F, BPI, BFI);
 
   FuncPGOInstrumentation<PGOEdge, BBInfo> FuncInfo(
       F, TLI, ComdatMembers, true, BPI, BFI, IsCS, PGOInstrumentEntry);
+
+  Type *I8PtrTy = Type::getInt8PtrTy(M->getContext());
+  auto Name = ConstantExpr::getBitCast(FuncInfo.FuncNameVar, I8PtrTy);
+  auto CFGHash = ConstantInt::get(Type::getInt64Ty(M->getContext()),
+                                  FuncInfo.FunctionHash);
+  if (PGOFunctionEntryCoverage) {
+    assert(!IsCS &&
+           "entry coverge does not support context-sensitive instrumentation");
+    auto &EntryBB = F.getEntryBlock();
+    IRBuilder<> Builder(&EntryBB, EntryBB.getFirstInsertionPt());
+    // llvm.instrprof.cover(i8* <name>, i64 <hash>, i32 <num-counters>,
+    //                      i32 <index>)
+    Builder.CreateCall(
+        Intrinsic::getDeclaration(M, Intrinsic::instrprof_cover),
+        {Name, CFGHash, Builder.getInt32(1), Builder.getInt32(0)});
+    return;
+  }
+
   std::vector<BasicBlock *> InstrumentBBs;
   FuncInfo.getInstrumentBBs(InstrumentBBs);
   unsigned NumCounters =
       InstrumentBBs.size() + FuncInfo.SIVisitor.getNumOfSelectInsts();
 
   uint32_t I = 0;
-  Type *I8PtrTy = Type::getInt8PtrTy(M->getContext());
   for (auto *InstrBB : InstrumentBBs) {
     IRBuilder<> Builder(InstrBB, InstrBB->getFirstInsertionPt());
     assert(Builder.GetInsertPoint() != InstrBB->end() &&
            "Cannot get the Instrumentation point");
+    // llvm.instrprof.increment(i8* <name>, i64 <hash>, i32 <num-counters>,
+    //                          i32 <index>)
     Builder.CreateCall(
         Intrinsic::getDeclaration(M, Intrinsic::instrprof_increment),
-        {ConstantExpr::getBitCast(FuncInfo.FuncNameVar, I8PtrTy),
-         Builder.getInt64(FuncInfo.FunctionHash), Builder.getInt32(NumCounters),
-         Builder.getInt32(I++)});
+        {Name, CFGHash, Builder.getInt32(NumCounters), Builder.getInt32(I++)});
   }
 
   // Now instrument select instructions:
   FuncInfo.SIVisitor.instrumentSelects(F, &I, NumCounters, FuncInfo.FuncNameVar,
                                        FuncInfo.FunctionHash);
   assert(I == NumCounters);
 
   if (DisableValueProfiling)
if (BFI->isIrrLoopHeader(&BB) || isIndirectBrTarget(&BB)) {
       Instruction *TI = BB.getTerminator();
       const UseBBInfo &BBCountInfo = getBBInfo(&BB);
       setIrrLoopHeaderMetadata(M, TI, BBCountInfo.CountValue);
     }
   }
 }
 
 void SelectInstVisitor::instrumentOneSelectInst(SelectInst &SI) {
+  if (PGOFunctionEntryCoverage)
+    return;
   Module *M = F.getParent();
   IRBuilder<> Builder(&SI);
   Type *Int64Ty = Builder.getInt64Ty();
   Type *I8PtrTy = Builder.getInt8PtrTy();
   auto *Step = Builder.CreateZExt(SI.getCondition(), Int64Ty);
   Builder.CreateCall(
       Intrinsic::getDeclaration(M, Intrinsic::instrprof_increment_step),
       {ConstantExpr::getBitCast(FuncNameVar, I8PtrTy),

 static bool InstrumentAllFunctions(
     Module &M, function_ref<TargetLibraryInfo &(Function &)> LookupTLI,
     function_ref<BranchProbabilityInfo *(Function &)> LookupBPI,
     function_ref<BlockFrequencyInfo *(Function &)> LookupBFI, bool IsCS) {
   // For the context-sensitve instrumentation, we should have a separated pass
   // (before LTO/ThinLTO linking) to create these variables.
   if (!IsCS)
     createIRLevelProfileFlagVar(M, /*IsCS=*/false, PGOInstrumentEntry,
-                                DebugInfoCorrelate);
+                                DebugInfoCorrelate, PGOFunctionEntryCoverage);
   std::unordered_multimap<Comdat *, GlobalValue *> ComdatMembers;
   collectComdatMembers(M, ComdatMembers);
 
   for (auto &F : M) {
     if (F.isDeclaration())
       continue;
     if (F.hasFnAttribute(llvm::Attribute::NoProfile))
       continue;
     auto &TLI = LookupTLI(F);
     auto *BPI = LookupBPI(F);
     auto *BFI = LookupBFI(F);
     instrumentOneFunc(F, &M, TLI, BPI, BFI, ComdatMembers, IsCS);
   }
   return true;
 }
 
 PreservedAnalyses
 PGOInstrumentationGenCreateVar::run(Module &M, ModuleAnalysisManager &AM) {
   createProfileFileNameVar(M, CSInstrName);
   // The variable in a comdat may be discarded by LTO. Ensure the declaration
   // will be retained.
-  appendToCompilerUsed(M, createIRLevelProfileFlagVar(M, /*IsCS=*/true,
-                                                      PGOInstrumentEntry,
-                                                      DebugInfoCorrelate));
+  appendToCompilerUsed(M, createIRLevelProfileFlagVar(
+                              M, /*IsCS=*/true, PGOInstrumentEntry,
+                              DebugInfoCorrelate, PGOFunctionEntryCoverage));
   return PreservedAnalyses::all();
 }
 
 bool PGOInstrumentationGenLegacyPass::runOnModule(Module &M) {
   if (skipModule(M))
     return false;
 
   auto LookupTLI = [this](Function &F) -> TargetLibraryInfo & {
if (!PGOReader->hasCSIRLevelProfile() && IsCS)
     return false;
 
   // TODO: might need to change the warning once the clang option is finalized.
   if (!PGOReader->isIRLevelProfile()) {
     Ctx.diagnose(DiagnosticInfoPGOProfile(
         ProfileFileName.data(), "Not an IR level instrumentation profile"));
     return false;
   }
+  if (PGOReader->hasSingleByteCoverage()) {
+    Ctx.diagnose(DiagnosticInfoPGOProfile(
+        ProfileFileName.data(),
+        "Cannot use coverage profiles for optimization"));
+    return false;
+  }
+  if (PGOReader->functionEntryOnly()) {
+    Ctx.diagnose(DiagnosticInfoPGOProfile(
+        ProfileFileName.data(),
+        "Function entry profiles are not yet supported for optimization"));
+    return false;
+  }
 
   // Add the profile summary (read from the header of the indexed summary) here
   // so that we can use it below when reading counters (which checks if the
   // function should be marked with a cold or inlinehint attribute).
   M.setProfileSummary(PGOReader->getSummary(IsCS).getMD(M.getContext()),
                       IsCS ? ProfileSummary::PSK_CSInstr
                            : ProfileSummary::PSK_Instr);
   PSI->refresh();

llvm/test/Instrumentation/InstrProfiling/coverage.ll

+; RUN: opt < %s -instrprof -S | FileCheck %s
+
+target triple = "aarch64-unknown-linux-gnu"
+
+@__profn_foo = private constant [3 x i8] c"foo"
+; CHECK: @__profc_foo = private global [1 x i8] c"\FF", section "__llvm_prf_cnts", comdat, align 1
+@__profn_bar = private constant [3 x i8] c"bar"
+; CHECK: @__profc_bar = private global [1 x i8] c"\FF", section "__llvm_prf_cnts", comdat, align 1
+
+define void @_Z3foov() {
+  call void @llvm.instrprof.cover(i8* getelementptr inbounds ([3 x i8], [3 x i8]* @__profn_foo, i32 0, i32 0), i64 12345678, i32 1, i32 0)
+  ; CHECK: store i8 0, i8* getelementptr inbounds ([1 x i8], [1 x i8]* @__profc_foo, i32 0, i32 0), align 1
+  ret void
+}
+
+%class.A = type { i32 (...)** }
+define dso_local void @_Z3barv(%class.A* nocapture nonnull align 8 %0) unnamed_addr #0 align 2 {
+  call void @llvm.instrprof.cover(i8* getelementptr inbounds ([3 x i8], [3 x i8]* @__profn_bar, i32 0, i32 0), i64 87654321, i32 1, i32 0)
+  ; CHECK: store i8 0, i8* getelementptr inbounds ([1 x i8], [1 x i8]* @__profc_bar, i32 0, i32 0), align 1
+  ret void
+}
+
+declare void @llvm.instrprof.cover(i8*, i64, i32, i32)

llvm/test/Instrumentation/InstrProfiling/debug-info-correlate-coverage.ll

+; RUN: opt < %s -instrprof -debug-info-correlate -S | opt --O2 -S | FileCheck %s
+
+@__profn_foo = private constant [3 x i8] c"foo"
+; CHECK:      @__profc_foo
+
+define  void @_Z3foov() !dbg !12 {
+  call void @llvm.instrprof.cover(i8* getelementptr inbounds ([3 x i8], [3 x i8]* @__profn_foo, i32 0, i32 0), i64 12345678, i32 1, i32 0)
+  ; CHECK: store i8 0, i8* getelementptr inbounds ([1 x i8], [1 x i8]* @__profc_foo
+  ret void
+}
+
+declare void @llvm.instrprof.cover(i8*, i64, i32, i32)
+
+!llvm.dbg.cu = !{!0}
+!llvm.module.flags = !{!2, !3, !4, !5, !6, !7, !8, !9, !10}
+!llvm.ident = !{!11}
+
+!0 = distinct !DICompileUnit(language: DW_LANG_C_plus_plus_14, file: !1, producer: "clang version 14.0.0", isOptimized: false, runtimeVersion: 0, emissionKind: FullDebug, splitDebugInlining: false, nameTableKind: None)
+!1 = !DIFile(filename: "debug-info-correlate-coverage.cpp", directory: "")
+!2 = !{i32 7, !"Dwarf Version", i32 4}
+!3 = !{i32 2, !"Debug Info Version", i32 3}
+!4 = !{i32 1, !"wchar_size", i32 4}
+!5 = !{i32 1, !"branch-target-enforcement", i32 0}
+!6 = !{i32 1, !"sign-return-address", i32 0}
+!7 = !{i32 1, !"sign-return-address-all", i32 0}
+!8 = !{i32 1, !"sign-return-address-with-bkey", i32 0}
+!9 = !{i32 7, !"uwtable", i32 1}
+!10 = !{i32 7, !"frame-pointer", i32 1}
+!11 = !{!"clang version 14.0.0"}
+!12 = distinct !DISubprogram(name: "foo", linkageName: "_Z3foov", scope: !13, file: !13, line: 1, type: !14, scopeLine: 1, flags: DIFlagPrototyped, spFlags: DISPFlagDefinition, unit: !0, retainedNodes: !16)
+!13 = !DIFile(filename: "debug-info-correlate-coverage.cpp", directory: "")
+!14 = !DISubroutineType(types: !15)
+!15 = !{null}
+!16 = !{}

llvm/test/Transforms/PGOProfile/coverage.ll

+; RUN: opt < %s -passes=pgo-instr-gen -pgo-function-entry-coverage -S | FileCheck %s
+target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+define i32 @foo(i32 %i) {
+entry:
+  ; CHECK: call void @llvm.instrprof.cover({{.*}})
+  %cmp = icmp sgt i32 %i, 0
+  br i1 %cmp, label %if.then, label %if.else
+
+if.then:
+  ; CHECK-NOT: llvm.instrprof.cover(
+  %add = add nsw i32 %i, 2
+  %s = select i1 %cmp, i32 %add, i32 0
+  br label %if.end
+
+if.else:
+  %sub = sub nsw i32 %i, 2
+  br label %if.end
+
+if.end:
+  %retv = phi i32 [ %add, %if.then ], [ %sub, %if.else ]
+  ret i32 %retv
+}
+
+; CHECK: declare void @llvm.instrprof.cover(

llvm/test/tools/llvm-profdata/show-covered.test

+// RUN: llvm-profdata show --covered %S/Inputs/function-entry-coverage.profdata | FileCheck %s
+
+// CHECK: main
+// CHECK: foo
+// CHECK: bar

llvm/tools/llvm-profdata/llvm-profdata.cpp

static void showValueSitesStats(raw_fd_ostream &OS, uint32_t VK,
 
   OS << "  Value sites histogram:\n\tNumTargets, SiteCount\n";
   for (unsigned I = 0; I < Stats.ValueSitesHistogram.size(); I++) {
     if (Stats.ValueSitesHistogram[I] > 0)
       OS << "\t" << I + 1 << ", " << Stats.ValueSitesHistogram[I] << "\n";
   }
 }
 
 static int showInstrProfile(const std::string &Filename, bool ShowCounts,
                             uint32_t TopN, bool ShowIndirectCallTargets,
                             bool ShowMemOPSizes, bool ShowDetailedSummary,
                             std::vector<uint32_t> DetailedSummaryCutoffs,
                             bool ShowAllFunctions, bool ShowCS,
                             uint64_t ValueCutoff, bool OnlyListBelow,
                             const std::string &ShowFunction, bool TextFormat,
-                            bool ShowBinaryIds, raw_fd_ostream &OS) {
+                            bool ShowBinaryIds, bool ShowCovered,
+                            raw_fd_ostream &OS) {
   auto ReaderOrErr = InstrProfReader::create(Filename);
   std::vector<uint32_t> Cutoffs = std::move(DetailedSummaryCutoffs);
   if (ShowDetailedSummary && Cutoffs.empty()) {
     Cutoffs = {800000, 900000, 950000, 990000, 999000, 999900, 999990};
   }
   InstrProfSummaryBuilder Builder(std::move(Cutoffs));
   if (Error E = ReaderOrErr.takeError())
     exitWithError(std::move(E), Filename);
if (doTextFormatDump) {
       InstrProfWriter::writeRecordInText(Func.Name, Func.Hash, Func, Symtab,
                                          OS);
       continue;
     }
 
     assert(Func.Counts.size() > 0 && "function missing entry counter");
     Builder.addRecord(Func);
 
+    if (ShowCovered) {
+      if (std::any_of(Func.Counts.begin(), Func.Counts.end(),
+                      [](uint64_t C) { return C; }))
+        OS << Func.Name << "\n";
+      continue;
+    }
+
     uint64_t FuncMax = 0;
     uint64_t FuncSum = 0;
     for (size_t I = 0, E = Func.Counts.size(); I < E; ++I) {
       if (Func.Counts[I] == (uint64_t)-1)
         continue;
       FuncMax = std::max(FuncMax, Func.Counts[I]);
       FuncSum += Func.Counts[I];
     }
if (Show) {
         traverseAllValueSites(Func, IPVK_MemOPSize, VPStats[IPVK_MemOPSize], OS,
                               nullptr);
       }
     }
   }
   if (Reader->hasError())
     exitWithError(Reader->getError(), Filename);
 
-  if (TextFormat)
+  if (TextFormat || ShowCovered)
     return 0;
   std::unique_ptr<ProfileSummary> PS(Builder.getSummary());
   bool IsIR = Reader->isIRLevelProfile();
   OS << "Instrumentation level: " << (IsIR ? "IR" : "Front-end");
   if (IsIR)
     OS << "  entry_first = " << Reader->instrEntryBBEnabled();
   OS << "\n";
   if (ShowAllFunctions || !ShowFunction.empty())
cl::opt<bool> ShowSectionInfoOnly(
                "The flag is only usable when the sample profile is in "
                "extbinary format"));
   cl::opt<bool> ShowBinaryIds("binary-ids", cl::init(false),
                               cl::desc("Show binary ids in the profile. "));
   cl::opt<std::string> DebugInfoFilename(
       "debug-info", cl::init(""),
       cl::desc("Read and extract profile metadata from debug info and show "
                "the functions it found."));
+  cl::opt<bool> ShowCovered(
+      "covered", cl::init(false),
+      cl::desc("Show only the functions that have been executed."));
 
   cl::ParseCommandLineOptions(argc, argv, "LLVM profile data summary\n");
 
   if (Filename.empty() && DebugInfoFilename.empty())
     exitWithError(
         "the positional argument '<profdata-file>' is required unless '--" +
         DebugInfoFilename.ArgStr + "' is provided");
 
if (!DebugInfoFilename.empty())
     return showDebugInfoCorrelation(DebugInfoFilename, ShowDetailedSummary,
                                     ShowProfileSymbolList, OS);
 
   if (ProfileKind == instr)
     return showInstrProfile(
         Filename, ShowCounts, TopNFunctions, ShowIndirectCallTargets,
         ShowMemOPSizes, ShowDetailedSummary, DetailedSummaryCutoffs,
         ShowAllFunctions, ShowCS, ValueCutoff, OnlyListBelow, ShowFunction,
-        TextFormat, ShowBinaryIds, OS);
+        TextFormat, ShowBinaryIds, ShowCovered, OS);
   if (ProfileKind == sample)
     return showSampleProfile(Filename, ShowCounts, TopNFunctions,
                              ShowAllFunctions, ShowDetailedSummary,
                              ShowFunction, ShowProfileSymbolList,
                              ShowSectionInfoOnly, ShowHotFuncList, OS);
   return showMemProfProfile(Filename, OS);
 }