[PGO] Instrument function entry BB by default in IR PGO

[xur]

Summary

This patch enables the instrumentation function entry BB by default in
IR PGO. This will slow down the instrumentation for some cases.
One can use the internal option "-mllvm -pgo-instrument-entry=false"
to disable it.

For single threaded programs, the slowdown should be very minimum.

The benefit of always instrumenting function entry BB is that we
can know the function entry count by form the profile and this often
facilitates some profile offline processing.

This patch also bumps up the index profile version, from 6 to 7.
Strictly speaking, this is not needed -- the instrumentation order depends
on the implementation and there is no guarantee of the instrumentation order.
We bump this up just to prevent some potential breakages (performance wise)
for people using some stored (old) profiles.

This patch will not change the behavior if the profile version is lower
than the new version (7) -- unless the internal option is explicitly specified.

For the index profile with version 7 or above, This patch assumes that the
function entry BB by default. Again the internal option
"-mllvm -pgo-instrument-entry" takes precedence over the index profile version.

[davidxl]

The version number probably can be kept unchanged -- however there needs to be a new variant bit to indicate the choice.

Changing version number does not work either -- suppose after the flip, user select the instr-entry=false, but the version number will still be 7 -- this will lead to problem at profile-use time.

[davidxl]

Here version is overloaded with different meanings: 1) to indicate format change; 2) to indicate instrumentation compiler version that is capable of producing this format. These two purposes can be contradicting to each other as the user can use option to force the old format with the new compiler.

I am not sure how compiler can relax the check at the profile use time.

[wmi]

In D83024#2132133, @davidxl wrote:

Here version is overloaded with different meanings: 1) to indicate format change; 2) to indicate instrumentation compiler version that is capable of producing this format. These two purposes can be contradicting to each other as the user can use option to force the old format with the new compiler.

For the case new compiler + -pgo-instrument-entry=false, if -pgo-instrument-entry=false is used in both profile-gen and profile-use explicitly, we won't have a problem. Do we have a senario which it doesn't work?

I am not sure how compiler can relax the check at the profile use time.

I just checked IndexedInstrProfReader::readHeader and the version check part has already allowed the case with profile version being smaller than compiler current format version so the relax I am talking about is already there -- Rong's patch has already covered my proposed solution.

[davidxl]

If we require profile-use also to use the option, it will work, but I think it is better and more convenient to change variant bit (I believe there are plenty). Bumping version can potentially complicate things in the future.

We should probably also add a new directive in text format for the variant, something like

:entry_first

When the directive is lacking, it means the old variant.

[xur]

Changed the implementation based on review comments.
Now we will not change the version of the index profile.
We will use bit 58 in the profile header to indicate if
we always instrument the entry block. This applies
to both raw and index format.

For the text formation, we add two new directives
:entry_first
:not_entry_first
When omitted, it's the same as :not_entry_first.

Also changed llvm-profdata to dump related information.

[davidxl]

Rong, please split the patch that handles variant and text dump support without changing the default first.

[xur]

Update the patch after committing the support in a separated patch:
commit 50da55a58534e9207d8d5e31c8b4b5cf0c624175

This patch will turn -pgo-instrument-entry to true by default.

Many tests are already submitted with the supporting patch.
This patch will update the rest of tests.

I'm not confident that the test changes are complete as I only tested X86 Linux platforms.

clang/test/CodeGenCXX/Inputs/profile-remap.proftext

 :ir
+:entry_first
 _ZN3Foo8functionENS_1XE
 29667547796
 2
-10
+100
 90
 

clang/test/Profile/Inputs/gcc-flag-compatibility_IR.proftext

 # IR level Instrumentation Flag
 :ir
+:entry_first
 main
 # Func Hash:
 34137660316
 # Num Counters:
 2
 # Counter Values:
-100
 1
+100
 

compiler-rt/test/profile/Linux/counter_promo_for.c

 // RUN: rm -fr %t.promo.prof
 // RUN: rm -fr %t.nopromo.prof
-// RUN: %clang_pgogen=%t.promo.prof/ -o %t.promo.gen -O2 %s
-// RUN: %clang_pgogen=%t.promo.prof/ -o %t.promo.gen.ll -emit-llvm -S -O2 %s
+// RUN: %clang_pgogen=%t.promo.prof/ -mllvm -pgo-instrument-entry=false -o %t.promo.gen -O2 %s
+// RUN: %clang_pgogen=%t.promo.prof/ -mllvm -pgo-instrument-entry=false -o %t.promo.gen.ll -emit-llvm -S -O2 %s
 // RUN: cat %t.promo.gen.ll | FileCheck --check-prefix=PROMO %s
 // RUN: %run %t.promo.gen
 // RUN: llvm-profdata merge -o %t.promo.profdata %t.promo.prof/
 // RUN: llvm-profdata show --counts --all-functions %t.promo.profdata  > %t.promo.dump
-// RUN: %clang_pgogen=%t.nopromo.prof/ -mllvm -do-counter-promotion=false -mllvm -simplifycfg-sink-common=false -o %t.nopromo.gen -O2 %s
-// RUN: %clang_pgogen=%t.nopromo.prof/ -mllvm -do-counter-promotion=false -mllvm -simplifycfg-sink-common=false -o %t.nopromo.gen.ll -emit-llvm -S -O2 %s
+// RUN: %clang_pgogen=%t.nopromo.prof/ -mllvm -do-counter-promotion=false -mllvm -pgo-instrument-entry=false -mllvm -simplifycfg-sink-common=false -o %t.nopromo.gen -O2 %s
+// RUN: %clang_pgogen=%t.nopromo.prof/ -mllvm -do-counter-promotion=false -mllvm -pgo-instrument-entry=false -mllvm -simplifycfg-sink-common=false -o %t.nopromo.gen.ll -emit-llvm -S -O2 %s
 // RUN: cat %t.nopromo.gen.ll | FileCheck --check-prefix=NOPROMO %s
 // RUN: %run %t.nopromo.gen
 // RUN: llvm-profdata merge -o %t.nopromo.profdata %t.nopromo.prof/
 // RUN: llvm-profdata show --counts --all-functions %t.nopromo.profdata  > %t.nopromo.dump
 // RUN: diff %t.promo.profdata %t.nopromo.profdata
 
 int g;
 __attribute__((noinline)) void bar(int i) { g += i; }

compiler-rt/test/profile/Linux/counter_promo_while.c

 // RUN: rm -fr %t.promo.prof
 // RUN: rm -fr %t.nopromo.prof
-// RUN: %clang_pgogen=%t.promo.prof/ -o %t.promo.gen -O2 %s
-// RUN: %clang_pgogen=%t.promo.prof/ -o %t.promo.gen.ll -emit-llvm -S -O2 %s
+// RUN: %clang_pgogen=%t.promo.prof/ -mllvm -pgo-instrument-entry=false -o %t.promo.gen -O2 %s
+// RUN: %clang_pgogen=%t.promo.prof/ -mllvm -pgo-instrument-entry=false -o %t.promo.gen.ll -emit-llvm -S -O2 %s
 // RUN: cat %t.promo.gen.ll | FileCheck --check-prefix=PROMO %s
 // RUN: %run %t.promo.gen
 // RUN: llvm-profdata merge -o %t.promo.profdata %t.promo.prof/
 // RUN: llvm-profdata show --counts --all-functions %t.promo.profdata  > %t.promo.dump
-// RUN: %clang_pgogen=%t.nopromo.prof/ -mllvm -do-counter-promotion=false -mllvm -simplifycfg-sink-common=false -o %t.nopromo.gen -O2 %s
-// RUN: %clang_pgogen=%t.nopromo.prof/ -mllvm -do-counter-promotion=false -mllvm -simplifycfg-sink-common=false -o %t.nopromo.gen.ll -emit-llvm -S -O2 %s
+// RUN: %clang_pgogen=%t.nopromo.prof/ -mllvm -do-counter-promotion=false -mllvm -pgo-instrument-entry=false -mllvm -simplifycfg-sink-common=false -o %t.nopromo.gen -O2 %s
+// RUN: %clang_pgogen=%t.nopromo.prof/ -mllvm -do-counter-promotion=false -mllvm -pgo-instrument-entry=false -mllvm -simplifycfg-sink-common=false -o %t.nopromo.gen.ll -emit-llvm -S -O2 %s
 // RUN: cat %t.nopromo.gen.ll | FileCheck --check-prefix=NOPROMO %s
 // RUN: %run %t.nopromo.gen
 // RUN: llvm-profdata merge -o %t.nopromo.profdata %t.nopromo.prof/
 // RUN: llvm-profdata show --counts --all-functions %t.nopromo.profdata  > %t.nopromo.dump
 // RUN: diff %t.promo.profdata %t.nopromo.profdata
 int g;
 __attribute__((noinline)) void bar(int i) { g += i; }
 __attribute__((noinline)) void foo(int n, int N) {

compiler-rt/test/profile/Linux/instrprof-value-merge.c

+// RUN: %clang_pgogen -mllvm -pgo-instrument-entry=false -o %t -O3 %s
+// RUN: rm -rf %t.profdir
+// RUN: env LLVM_PROFILE_FILE=%t.profdir/default_%m.profraw %run %t
+// RUN: env LLVM_PROFILE_FILE=%t.profdir/default_%m.profraw %run %t
+// RUN: env LLVM_PROFILE_FILE=%t.profdir/default_%m.profraw %run %t
+// RUN: env LLVM_PROFILE_FILE=%t.profdir/default_%m.profraw %run %t 1
+// RUN: env LLVM_PROFILE_FILE=%t.profdir/default_%m.profraw %run %t 1
+// RUN: llvm-profdata show -counts -function=main -ic-targets -memop-sizes %t.profdir/default_*.profraw | FileCheck %s --check-prefixes=CHECK,NOENTRY
 // RUN: %clang_pgogen -o %t -O3 %s
 // RUN: rm -rf %t.profdir
 // RUN: env LLVM_PROFILE_FILE=%t.profdir/default_%m.profraw %run %t
 // RUN: env LLVM_PROFILE_FILE=%t.profdir/default_%m.profraw %run %t
 // RUN: env LLVM_PROFILE_FILE=%t.profdir/default_%m.profraw %run %t
 // RUN: env LLVM_PROFILE_FILE=%t.profdir/default_%m.profraw %run %t 1
 // RUN: env LLVM_PROFILE_FILE=%t.profdir/default_%m.profraw %run %t 1
-// RUN: llvm-profdata show -counts -function=main -ic-targets -memop-sizes %t.profdir/default_*.profraw | FileCheck %s
+// RUN: llvm-profdata show -counts -function=main -ic-targets -memop-sizes %t.profdir/default_*.profraw | FileCheck %s --check-prefixes=CHECK,ENTRY
 
 #include <string.h>
 
 void (*f0)();
 void (*f1)();
 void (*f2)();
 
 char dst[200];

 }
 
 // CHECK: Counters:
 // CHECK:   main:
 // CHECK:     Hash: 0x00030012a7ab6e87
 // CHECK:     Counters: 6
 // CHECK:     Indirect Call Site Count: 3
 // CHECK:     Number of Memory Intrinsics Calls: 3
-// CHECK:     Block counts: [27, 297, 12, 132, 3, 2]
+// NOTENTRY:     Block counts: [27, 297, 12, 132, 3, 2]
+// ENTRY:     Block counts: [5, 27, 297, 12, 132, 2]
 // CHECK:     Indirect Target Results:
 // CHECK:         [ 0, foo, 3 ]
 // CHECK:         [ 1, foo, 27 ]
 // CHECK:         [ 2, foo, 297 ]
 // CHECK:     Memory Intrinsic Size Results:
 // CHECK:         [ 0, 4, 2 ]
 // CHECK:         [ 1, 5, 12 ]
 // CHECK:         [ 2, 6, 132 ]

compiler-rt/test/profile/infinite_loop.c

-// RUN: %clang_pgogen  -O2 -o %t %s
+// RUN: %clang_pgogen  -O2 -mllvm -pgo-instrument-entry=false -o %t %s
 // RUN: env LLVM_PROFILE_FILE=%t.profraw %run %t
-// RUN: llvm-profdata show -function main -counts  %t.profraw| FileCheck  %s 
+// RUN: llvm-profdata show -function main -counts  %t.profraw| FileCheck  %s --check-prefixes=CHECK,NOENTRY
+// RUN: %clang_pgogen  -O2 -o %t %s
+// RUN: env LLVM_PROFILE_FILE=%t2.profraw %run %t
+// RUN: llvm-profdata show -function main -counts  %t2.profraw| FileCheck  %s --check-prefixes=CHECK,ENTRY
 
 void exit(int);
 int g;
 __attribute__((noinline)) void foo()
 {
   g++;
   if (g==1000)
     exit(0);
 }
 
 
 int main()
 {
   while (1) {
     foo();
   }
 
 }
 
 // CHECK: Counters:
 // CHECK-NEXT:  main:
 // CHECK-NEXT:    Hash: {{.*}}
 // CHECK-NEXT:    Counters: 2
-// CHECK-NEXT:    Block counts: [1000, 1]
-
-
-
+// NOTENTRY-NEXT:    Block counts: [1000, 1]
+// ENTRY-NEXT:    Block counts: [1, 1000]

compiler-rt/test/profile/runtime_infinite.c

 // The waiting loop never exits via the normal
 // path before the profile is dumped and the
 // program is terminated. This tests checks
 // that the entry of main is properly instrumented
 // and has non-zero count.
 
-// RUN: %clang_pgogen -mllvm -do-counter-promotion=false -O2 -o %t %s
+// RUN: %clang_pgogen -mllvm -do-counter-promotion=false -mllvm -pgo-instrument-entry=false -O2 -o %t %s
 // RUN: env LLVM_PROFILE_FILE=%t.profraw %run %t
-// RUN: llvm-profdata show -function main -counts  %t.profraw| FileCheck  %s
+// RUN: llvm-profdata show -function main -counts  %t.profraw| FileCheck  %s --check-prefixes=CHECK,NOTENTRY
+// RUN: %clang_pgogen -mllvm -do-counter-promotion=false -O2 -o %t %s
+// RUN: env LLVM_PROFILE_FILE=%t2.profraw %run %t
+// RUN: llvm-profdata show -function main -counts  %t2.profraw| FileCheck  %s --check-prefixes=CHECK,ENTRY
 void exit(int);
 
 int __llvm_profile_dump(void);
 void __llvm_profile_reset_counters(void);
 
 int g = 0;
 __attribute__((noinline)) void doSth() {
   g++;

 int main(int argc, const char *argv[]) {
   //  waiting_loop
   while (noerror()) {
     doSth();
   }
 }
 
 // CHECK-LABEL: main
-// CHECK: [10001, 1]
+// NOENTRY: [10001, 1]
+// ENTRY: [1, 10001]

llvm/lib/Transforms/Instrumentation/PGOInstrumentation.cpp

 static cl::opt<bool>
     EmitBranchProbability("pgo-emit-branch-prob", cl::init(false), cl::Hidden,
                           cl::desc("When this option is on, the annotated "
                                    "branch probability will be emitted as "
                                    "optimization remarks: -{Rpass|"
                                    "pass-remarks}=pgo-instrumentation"));
 
 static cl::opt<bool> PGOInstrumentEntry(
-    "pgo-instrument-entry", cl::init(false), cl::Hidden,
+    "pgo-instrument-entry", cl::init(true), cl::Hidden,
     cl::desc("Force to instrument function entry basicblock."));
 
 // Command line option to turn on CFG dot dump after profile annotation.
 // Defined in Analysis/BlockFrequencyInfo.cpp:  -pgo-view-counts
 extern cl::opt<PGOViewCountsType> PGOViewCounts;
 
 // Command line option to specify the name of the function for CFG dump
 // Defined in Analysis/BlockFrequencyInfo.cpp:  -view-bfi-func-name=

llvm/test/Transforms/PGOProfile/Inputs/PR41279.proftext

 :ir
+:entry_first
 foo
 60927483247
 4
-3
+8
 2
 3
 2
 

llvm/test/Transforms/PGOProfile/Inputs/PR41279_2.proftext

 :ir
+:entry_first
 f
 62077759478
 2
+5
 3
-2
 

llvm/test/Transforms/PGOProfile/PR41279.ll

; USE-SAME: !prof ![[FUNC_ENTRY_COUNT:[0-9]+]]
 
   %3 = alloca i8, align 1
   store i8 0, i8* %3, align 1
   %4 = call i64 @f1()
   %5 = icmp ult i64 %4, 32
   br i1 %5, label %7, label %13
 ; USE: br i1 %5, label %7, label %13
 ; USE-SAME: !prof ![[BW_ENTRY1:[0-9]+]]
+; GEN:  call void @llvm.instrprof.increment(i8* getelementptr inbounds ([3 x i8], [3 x i8]* @__profn_foo, i32 0, i32 0), i64 60927483247, i32 4, i32 0)
 
 6:
   cleanupret from %17 unwind to caller
 ; GEN: 6:
 ; GEN:  call void @llvm.instrprof.increment(i8* getelementptr inbounds ([3 x i8], [3 x i8]* @__profn_foo, i32 0, i32 0), i64 60927483247, i32 4, i32 2)
 
 7:
   store i8 1, i8* %3, align 1
12:
   store i8 0, i8* %3, align 1
   br label %14
 ; GEN: 12:
 ; GEN:  call void @llvm.instrprof.increment(i8* getelementptr inbounds ([3 x i8], [3 x i8]* @__profn_foo, i32 0, i32 0), i64 60927483247, i32 4, i32 1)
 
 13:
   call void @f3({ i8*, i64 }* %0, { i8*, i64 }* %1, i64 1)
   br label %14
-; GEN: 13:
-; GEN:  call void @llvm.instrprof.increment(i8* getelementptr inbounds ([3 x i8], [3 x i8]* @__profn_foo, i32 0, i32 0), i64 60927483247, i32 4, i32 0)
 
 14:
   ret void
 
 15:
   store i8 0, i8* %3, align 1
   br label %6
 ; GEN: 15:

llvm/test/Transforms/PGOProfile/PR41279_2.ll

 ; USE-DAG: {{![0-9]+}} = !{i32 1, !"ProfileSummary", {{![0-9]+}}}
 ; USE-DAG: {{![0-9]+}} = !{!"DetailedSummary", {{![0-9]+}}}
 ; USE-DAG: ![[FUNC_ENTRY_COUNT]] = !{!"function_entry_count", i64 5}
 entry:
   %__exception_code = alloca i32, align 4
   %__exception_code2 = alloca i32, align 4
   invoke void @f() #2
           to label %invoke.cont unwind label %catch.dispatch
+;GEN: entry:
+;GEN:  call void @llvm.instrprof.increment(i8* getelementptr inbounds ([1 x i8], [1 x i8]* @__profn_f, i32 0, i32 0), i64 62077759478, i32 2, i32 0)
 
 catch.dispatch:
   %0 = catchswitch within none [label %__except] unwind to caller
 
 __except:
   %1 = catchpad within %0 [i8* null]
   catchret from %1 to label %__except1
 
__except6:
   store i32 %5, i32* %__exception_code2, align 4
   br label %__try.cont7
 
 __try.cont7:
   ret void
 
 invoke.cont3:
   br label %__try.cont7
-;GEN: invoke.cont3:
-;GEN:  call void @llvm.instrprof.increment(i8* getelementptr inbounds ([1 x i8], [1 x i8]* @__profn_f, i32 0, i32 0), i64 62077759478, i32 2, i32 0)
-
 }
 
 declare dso_local i32 @__C_specific_handler(...)
 
 declare i32 @llvm.eh.exceptioncode(token)

llvm/test/Transforms/PGOProfile/counter_promo.ll

 ; RUN: opt < %s -pgo-instr-gen -instrprof -do-counter-promotion=true -skip-ret-exit-block=0 -S | FileCheck --check-prefix=PROMO --check-prefix=NONATOMIC_PROMO %s
 ; RUN: opt < %s --passes=pgo-instr-gen,instrprof -do-counter-promotion=true -skip-ret-exit-block=0 -S | FileCheck --check-prefix=PROMO --check-prefix=NONATOMIC_PROMO %s
 ; RUN: opt < %s -pgo-instr-gen -instrprof -do-counter-promotion=true -atomic-counter-update-promoted -skip-ret-exit-block=0 -S | FileCheck --check-prefix=PROMO --check-prefix=ATOMIC_PROMO %s
 ; RUN: opt < %s --passes=pgo-instr-gen,instrprof -do-counter-promotion=true -atomic-counter-update-promoted -skip-ret-exit-block=0 -S | FileCheck --check-prefix=PROMO --check-prefix=ATOMIC_PROMO %s
 
 define void @foo(i32 %n, i32 %N) {
 ; PROMO-LABEL: @foo
-; PROMO: {{.*}} = load {{.*}} @__profc_foo{{.*}} 3)
+; PROMO: {{.*}} = load {{.*}} @__profc_foo{{.*}} 0)
 ; PROMO-NEXT: add
-; PROMO-NEXT: store {{.*}}@__profc_foo{{.*}}3)
+; PROMO-NEXT: store {{.*}}@__profc_foo{{.*}}0)
 bb:
   %tmp = add nsw i32 %n, 1
   %tmp1 = add nsw i32 %n, -1
   br label %bb2
 
 bb2:                                              ; preds = %bb9, %bb
 ; PROMO: phi {{.*}}
 ; PROMO-NEXT: phi {{.*}}

 ; PROMO-NEXT: %[[LIVEOUT2:[a-z0-9]+]] = phi {{.*}}
 ; PROMO-NEXT: %[[LIVEOUT1:[a-z0-9]+]] = phi {{.*}}
   %tmp10 = add nsw i32 %i.0, 1
   %tmp11 = icmp slt i32 %tmp10, %N
   br i1 %tmp11, label %bb2, label %bb12
 
 bb12:                                             ; preds = %bb9
   ret void
-; NONATOMIC_PROMO: %[[PROMO1:[a-z0-9.]+]] = load {{.*}} @__profc_foo{{.*}} 0)
+; NONATOMIC_PROMO: %[[PROMO1:[a-z0-9.]+]] = load {{.*}} @__profc_foo{{.*}} 1)
 ; NONATOMIC_PROMO-NEXT: add {{.*}} %[[PROMO1]], %[[LIVEOUT1]] 
-; NONATOMIC_PROMO-NEXT: store {{.*}}@__profc_foo{{.*}}0)
-; NONATOMIC_PROMO-NEXT: %[[PROMO2:[a-z0-9.]+]] = load {{.*}} @__profc_foo{{.*}} 1)
-; NONATOMIC_PROMO-NEXT: add {{.*}} %[[PROMO2]], %[[LIVEOUT2]]
 ; NONATOMIC_PROMO-NEXT: store {{.*}}@__profc_foo{{.*}}1)
-; NONATOMIC_PROMO-NEXT: %[[PROMO3:[a-z0-9.]+]] = load {{.*}} @__profc_foo{{.*}} 2)
-; NONATOMIC_PROMO-NEXT: add {{.*}} %[[PROMO3]], %[[LIVEOUT3]]
+; NONATOMIC_PROMO-NEXT: %[[PROMO2:[a-z0-9.]+]] = load {{.*}} @__profc_foo{{.*}} 2)
+; NONATOMIC_PROMO-NEXT: add {{.*}} %[[PROMO2]], %[[LIVEOUT2]]
 ; NONATOMIC_PROMO-NEXT: store {{.*}}@__profc_foo{{.*}}2)
-; ATOMIC_PROMO: atomicrmw add {{.*}} @__profc_foo{{.*}}0), i64 %[[LIVEOUT1]] seq_cst
-; ATOMIC_PROMO-NEXT: atomicrmw add {{.*}} @__profc_foo{{.*}}1), i64 %[[LIVEOUT2]] seq_cst
-; ATOMIC_PROMO-NEXT: atomicrmw add {{.*}} @__profc_foo{{.*}}2), i64 %[[LIVEOUT3]] seq_cst
+; NONATOMIC_PROMO-NEXT: %[[PROMO3:[a-z0-9.]+]] = load {{.*}} @__profc_foo{{.*}} 3)
+; NONATOMIC_PROMO-NEXT: add {{.*}} %[[PROMO3]], %[[LIVEOUT3]]
+; NONATOMIC_PROMO-NEXT: store {{.*}}@__profc_foo{{.*}}3)
+; ATOMIC_PROMO: atomicrmw add {{.*}} @__profc_foo{{.*}}1), i64 %[[LIVEOUT1]] seq_cst
+; ATOMIC_PROMO-NEXT: atomicrmw add {{.*}} @__profc_foo{{.*}}2), i64 %[[LIVEOUT2]] seq_cst
+; ATOMIC_PROMO-NEXT: atomicrmw add {{.*}} @__profc_foo{{.*}}3), i64 %[[LIVEOUT3]] seq_cst
 ; PROMO-NOT: @__profc_foo
 
 
 }
 
 declare void @bar(i32)

llvm/test/Transforms/PGOProfile/counter_promo_mexits.ll

bb11:                                             ; preds = %bb10, %bb9, %bb4
   %tmp12 = add nuw nsw i32 %tmp2, 1
   %tmp13 = icmp slt i32 %tmp2, 99
   br i1 %tmp13, label %bb1, label %bb14
 
 bb14:                                             ; preds = %bb11
 ; PROMO-LABEL: bb14:
   tail call void @bar(i32 0)
   br label %bb15
-; PROMO:  %pgocount.promoted{{.*}} = load {{.*}} @__profc_foo{{.*}} 0)
-; PROMO-NEXT: add 
-; PROMO-NEXT: store {{.*}}@__profc_foo{{.*}}0)
-; PROMO-NEXT:  %pgocount.promoted{{.*}} = load {{.*}} @__profc_foo{{.*}} 1)
-; PROMO-NEXT: add 
+; PROMO:  %pgocount.promoted{{.*}} = load {{.*}} @__profc_foo{{.*}} 1)
+; PROMO-NEXT: add
 ; PROMO-NEXT: store {{.*}}@__profc_foo{{.*}}1)
 ; PROMO-NEXT:  %pgocount.promoted{{.*}} = load {{.*}} @__profc_foo{{.*}} 2)
-; PROMO-NEXT: add 
+; PROMO-NEXT: add
 ; PROMO-NEXT: store {{.*}}@__profc_foo{{.*}}2)
-; PROMO-NEXT:  %pgocount{{.*}} = load {{.*}} @__profc_foo{{.*}} 3)
-; PROMO-NEXT: add 
+; PROMO-NEXT:  %pgocount.promoted{{.*}} = load {{.*}} @__profc_foo{{.*}} 3)
+; PROMO-NEXT: add
 ; PROMO-NEXT: store {{.*}}@__profc_foo{{.*}}3)
 
 
 bb15_0:                                             ; preds = %bb11
 ; PROMO-LABEL: bb15_0:
   br label %bb15
-; PROMO:  %pgocount.promoted{{.*}} = load {{.*}} @__profc_foo{{.*}} 0)
-; PROMO-NEXT: add 
-; PROMO-NEXT: store {{.*}}@__profc_foo{{.*}}0)
-; PROMO-NEXT:  %pgocount.promoted{{.*}} = load {{.*}} @__profc_foo{{.*}} 1)
-; PROMO-NEXT: add 
+; PROMO:  %pgocount.promoted{{.*}} = load {{.*}} @__profc_foo{{.*}} 1)
+; PROMO-NEXT: add
 ; PROMO-NEXT: store {{.*}}@__profc_foo{{.*}}1)
 ; PROMO-NEXT:  %pgocount.promoted{{.*}} = load {{.*}} @__profc_foo{{.*}} 2)
-; PROMO-NEXT: add 
+; PROMO-NEXT: add
 ; PROMO-NEXT: store {{.*}}@__profc_foo{{.*}}2)
+; PROMO-NEXT:  %pgocount.promoted{{.*}} = load {{.*}} @__profc_foo{{.*}} 3)
+; PROMO-NEXT: add
+; PROMO-NEXT: store {{.*}}@__profc_foo{{.*}}3)
 ; PROMO-NEXT:  %pgocount{{.*}} = load {{.*}} @__profc_foo{{.*}} 4)
-; PROMO-NEXT: add 
+; PROMO-NEXT: add
 ; PROMO-NEXT: store {{.*}}@__profc_foo{{.*}}4)
 ; PROMO-NOT: @__profc_foo
 
 
 bb15:                                             ; preds = %bb14, %bb4
   tail call void @bar(i32 1)
   ret void
 }
 
 declare void @bar(i32) local_unnamed_addr