This is an archive of the discontinued LLVM Phabricator instance.

Differential D88609

Use uint64_t for branch weights instead of uint32_t
ClosedPublic

Authored by aeubanks on Sep 30 2020, 1:02 PM.

Details

Reviewers
davidxl
mtrofin
wmi
vsk
aartbik
Commits
rG10f2a0d662d8: Use uint64_t for branch weights instead of uint32_t
rGe5766f25c62c: Use uint64_t for branch weights instead of uint32_t
Summary

CallInst::updateProfWeight() creates branch_weights with i64 instead of i32.
To be more consistent everywhere and remove lots of casts from uint64_t
to uint32_t, use i64 for branch_weights.

Diff Detail

Unit TestsFailed

TimeTest
560 mslinux > HWAddressSanitizer-x86_64.TestCases::sizes.cpp
80 mslinux > LLVM.Transforms/SampleProfile::profile-format-compress.ll
90 mslinux > MLIR.Target::llvmir.mlir
400 mslinux > Profile-x86_64.Linux::comdat_rename.test
150 mslinux > Profile-x86_64.Linux::instrprof-basic.c
View Full Test Results (20 Failed)

Event Timeline

aeubanks created this revision.Sep 30 2020, 1:02 PM
Herald added a project: Restricted Project. · View Herald TranscriptSep 30 2020, 1:02 PM
Herald added subscribers: llvm-commits, wenlei, dexonsmith and 4 others. · View Herald Transcript
aeubanks requested review of this revision.Sep 30 2020, 1:02 PM
aeubanks added reviewers: davidxl, mtrofin, wmi.Sep 30 2020, 1:04 PM
davidxl added a reviewer: vsk.Sep 30 2020, 1:12 PM

I like the direction of the patch. There is one concern -- it makes the IR not backward compatible which may affect some users. +vsk.

Harbormaster completed remote builds in B73566: Diff 295383.Sep 30 2020, 1:13 PM
aeubanks added a comment.Oct 2 2020, 10:53 AM
In D88609#2304497, @davidxl wrote:

I like the direction of the patch. There is one concern -- it makes the IR not backward compatible which may affect some users. +vsk.

I'm not too familiar with this, but it looks like weights are read via C->getValue().getZExtValue(), which should work with i32 and i64.
Maybe some version of LLVM without this commit that reads IR generated by a later version of LLVM with this commit will incorrectly read weights larger than i32, but not sure if that's a concern or not.

davidxl added a comment.Oct 2 2020, 11:03 AM

Perhaps add a new test to demonstrate it (that i32 profile weight still works)?

aeubanks added a comment.Oct 2 2020, 11:49 AM

There are actually lots of tests I didn't touch that still have i32 branch_weights, for example llvm/test/Transforms/SampleProfile/remap.ll has !9 = !{!"branch_weights", i32 90, i32 10} and still properly prints out the same branch probabilities with opt -passes='print<branch-prob>'. Is that good enough?

davidxl added a comment.Oct 2 2020, 11:52 AM

Then why modifying existing tests at all? How about add an explicit test using i64 type?

aeubanks added a comment.Oct 2 2020, 11:59 AM

The test modifications are because now when a pass creates a branch_weight, the type is now i64, so the CHECK lines need to be updated in those cases. Otherwise just reading an existing branch_weight is ok.
Basically I ran check-llvm, then in the failing tests I replaced all branch_weights with i64 values. Tests that didn't explicitly check for i32 branch_weights (which is most tests) still passed.

aeubanks updated this revision to Diff 299766.Oct 21 2020, 11:41 AM

add test for i32 and i64

aeubanks added a comment.Oct 21 2020, 11:42 AM

I changed llvm/test/Analysis/BranchProbabilityInfo/basic.ll to use i64 branch_weights and made llvm/test/Analysis/BranchProbabilityInfo/basic_i32.ll which uses i32 branch_weights.

Harbormaster completed remote builds in B75918: Diff 299766.Oct 21 2020, 11:57 AM
vsk added a comment.Oct 21 2020, 11:59 AM

To my knowledge, we're not relying on the fact that branch_weights are 32-bit for bitcode compatibility purposes, or for PGO. We do support using older .profdata files with newer builds, but (afaict) this change to branch_weights does not affect that.

davidxl accepted this revision.Oct 21 2020, 12:45 PM

lgtm

This revision is now accepted and ready to land.Oct 21 2020, 12:45 PM
aeubanks updated this revision to Diff 300447.Oct 23 2020, 6:10 PM

check-clang revealed more failures, had to modify some clang tests and BranchProbabilityInfo.cpp

Herald added a project: Restricted Project. · View Herald TranscriptOct 23 2020, 6:10 PM
Herald added a subscriber: cfe-commits. · View Herald Transcript
Harbormaster completed remote builds in B76270: Diff 300447.Oct 23 2020, 6:56 PM
aeubanks updated this revision to Diff 300764.Oct 26 2020, 12:51 PM

fix more tests found by running check-all on a Linux machine

Herald added a project: Restricted Project. · View Herald TranscriptOct 26 2020, 12:51 PM
Herald added a subscriber: Restricted Project. · View Herald Transcript
Harbormaster completed remote builds in B76457: Diff 300764.Oct 26 2020, 1:38 PM
This revision was landed with ongoing or failed builds.Oct 26 2020, 8:35 PM
Closed by commit rGe5766f25c62c: Use uint64_t for branch weights instead of uint32_t (authored by aeubanks). · Explain Why
This revision was automatically updated to reflect the committed changes.
aeubanks added a commit: rGe5766f25c62c: Use uint64_t for branch weights instead of uint32_t.
kiranchandramohan mentioned this in rG0fc1aa22ee6a: Updating llvm.mlir test to match recent IR change.Oct 27 2020, 1:18 AM
kiranchandramohan added a subscriber: kiranchandramohan.Oct 27 2020, 1:19 AM

Updated mlir test also to include this change.
https://reviews.llvm.org/rG0fc1aa22ee6ac337a5d51fa5666c9cd61da61b07

thakis added a reverting change: rG2a4e704c92e8: Revert "Use uint64_t for branch weights instead of uint32_t".Oct 27 2020, 6:26 AM
kiranchandramohan added a subscriber: thakis.Oct 27 2020, 7:20 AM

@thakis Can you revert the testcase fix in MLIR also which was necessary after this patch? https://reviews.llvm.org/rG0fc1aa22ee6ac337a5d51fa5666c9cd61da61b07

aeubanks reopened this revision.Oct 28 2020, 1:16 PM
This revision is now accepted and ready to land.Oct 28 2020, 1:16 PM
aeubanks updated this revision to Diff 301409.Oct 28 2020, 1:16 PM

scale down weights by UINT32_MAX when sum of branch weights overflows uint64_t

Herald added a reviewer: aartbik. · View Herald TranscriptOct 28 2020, 1:16 PM
Herald added a project: Restricted Project. · View Herald Transcript
Herald added subscribers: rdzhabarov, tatianashp, msifontes and 15 others. · View Herald Transcript
aeubanks added a comment.Oct 28 2020, 1:17 PM

This was reverted due to crashes caused by adding branch weights overflowing uint64_t. Now in BranchProbabilityInfo.cpp, if the sum of weights overflows, scale down all weights by UINT32_MAX (See ScaleWeights()). Is that ok?

Harbormaster completed remote builds in B76802: Diff 301409.Oct 28 2020, 4:25 PM
This revision was landed with ongoing or failed builds.Oct 30 2020, 10:03 AM
Closed by commit rG10f2a0d662d8: Use uint64_t for branch weights instead of uint32_t (authored by aeubanks). · Explain Why
This revision was automatically updated to reflect the committed changes.
aeubanks added a commit: rG10f2a0d662d8: Use uint64_t for branch weights instead of uint32_t.
dmajor added a subscriber: dmajor.Oct 30 2020, 6:35 PM

After 10f2a0d662d8 our self-host PGO builds are hitting:
llvm::BranchProbability::getBranchProbability(uint64_t, uint64_t): Assertion `Numerator <= Denominator && "Probability cannot be bigger than 1!"' failed.

https://treeherder.mozilla.org/logviewer?job_id=320324631&repo=try&lineNumber=52597 has the log as well as the .cpp/.sh reproducers, but I assume you'd also need the merged.profdata file, and our bots haven't been taught to save those. I can try to add that ability next week if needed.

aeubanks added a comment.Oct 31 2020, 12:18 AM

The stack trace helped me come up with my own repro, reverting.
It's more uint64_t overflow issues. This might be the reason branch_weights were initially i32 with casts in a bunch of places around LLVM. Not sure if I should try and fix all usages or just keep branch_weights as i32.

aeubanks added a reverting change: rG5c31b8b94fdd: Revert "Use uint64_t for branch weights instead of uint32_t".Oct 31 2020, 12:26 AM
aeubanks mentioned this in D90539: Make CallInst::updateProfWeight emit i32 weights instead of i64.Nov 3 2020, 7:24 PM

Revision Contents

PathSize
clang/
lib/
CodeGen/
36 lines
test/
CodeGen/
2 lines
CodeGenCXX/
4 lines
4 lines
30 lines
4 lines
4 lines
4 lines
Profile/
10 lines
152 lines
12 lines
10 lines
8 lines
4 lines
14 lines
4 lines
6 lines
llvm/
include/
llvm/
IR/
2 lines
4 lines
Transforms/
Scalar/
4 lines
Utils/
2 lines
lib/
Analysis/
9 lines
IR/
10 lines
10 lines
Transforms/
IPO/
16 lines
Instrumentation/
6 lines
3 lines
Scalar/
4 lines
18 lines
Utils/
3 lines
7 lines
4 lines
44 lines
test/
Analysis/
BranchProbabilityInfo/
28 lines
basic_i32.ll
basic.ll
650 lines
Instrumentation/
AddressSanitizer/
2 lines
Transforms/
CodeExtractor/
6 lines
CorrelatedValuePropagation/
12 lines
JumpThreading/
6 lines
6 lines
6 lines
LoopUnroll/
14 lines
10 lines
6 lines
LoopVectorize/
14 lines
10 lines
LowerExpectIntrinsic/
8 lines
8 lines
4 lines
4 lines
4 lines
2 lines
PGOProfile/
4 lines
2 lines
16 lines
6 lines
8 lines
2 lines
2 lines
2 lines
2 lines
2 lines
4 lines
8 lines
2 lines
2 lines
2 lines
6 lines
2 lines
4 lines
6 lines
4 lines
4 lines
2 lines
2 lines
4 lines
2 lines
6 lines
2 lines
SCCP/
4 lines
SampleProfile/
1 line
8 lines
4 lines
6 lines
6 lines
SimpleLoopUnswitch/
6 lines
16 lines
SimplifyCFG/
4 lines
18 lines
72 lines
4 lines
Util/
2 lines
2 lines
2 lines
unittests/
Analysis/
2 lines

Diff 300447

clang/lib/CodeGen/CodeGenPGO.cpp

Show First 20 Lines • Show All 1,007 Lines • ▼ Show 20 Lines else if (IPE == llvm::instrprof_error::malformed)
CGM.getPGOStats().addMismatched(IsInMainFile); CGM.getPGOStats().addMismatched(IsInMainFile);
return; return;
} }
ProfRecord = ProfRecord =
std::make_unique<llvm::InstrProfRecord>(std::move(RecordExpected.get())); std::make_unique<llvm::InstrProfRecord>(std::move(RecordExpected.get()));
RegionCounts = ProfRecord->Counts; RegionCounts = ProfRecord->Counts;
} }
/// Calculate what to divide by to scale weights. /// Scale an individual branch weight (add 1).
///
/// Given the maximum weight, calculate a divisor that will scale all the
/// weights to strictly less than UINT32_MAX.
static uint64_t calculateWeightScale(uint64_t MaxWeight) {
return MaxWeight < UINT32_MAX ? 1 : MaxWeight / UINT32_MAX + 1;
}
/// Scale an individual branch weight (and add 1).
///
/// Scale a 64-bit weight down to 32-bits using \c Scale.
/// ///
/// According to Laplace's Rule of Succession, it is better to compute the /// According to Laplace's Rule of Succession, it is better to compute the
/// weight based on the count plus 1, so universally add 1 to the value. /// weight based on the count plus 1, so universally add 1 to the value.
/// static uint64_t scaleBranchWeight(uint64_t Weight) { return Weight + 1; }
/// \pre \c Scale was calculated by \a calculateWeightScale() with a weight no
/// greater than \c Weight.
static uint32_t scaleBranchWeight(uint64_t Weight, uint64_t Scale) {
assert(Scale && "scale by 0?");
uint64_t Scaled = Weight / Scale + 1;
assert(Scaled <= UINT32_MAX && "overflow 32-bits");
return Scaled;
}
llvm::MDNode *CodeGenFunction::createProfileWeights(uint64_t TrueCount, llvm::MDNode *CodeGenFunction::createProfileWeights(uint64_t TrueCount,
uint64_t FalseCount) const { uint64_t FalseCount) const {
// Check for empty weights. // Check for empty weights.
if (!TrueCount && !FalseCount) if (!TrueCount && !FalseCount)
return nullptr; return nullptr;
// Calculate how to scale down to 32-bits.
uint64_t Scale = calculateWeightScale(std::max(TrueCount, FalseCount));
llvm::MDBuilder MDHelper(CGM.getLLVMContext()); llvm::MDBuilder MDHelper(CGM.getLLVMContext());
return MDHelper.createBranchWeights(scaleBranchWeight(TrueCount, Scale), return MDHelper.createBranchWeights(scaleBranchWeight(TrueCount),
scaleBranchWeight(FalseCount, Scale)); scaleBranchWeight(FalseCount));
} }
llvm::MDNode * llvm::MDNode *
CodeGenFunction::createProfileWeights(ArrayRef<uint64_t> Weights) const { CodeGenFunction::createProfileWeights(ArrayRef<uint64_t> Weights) const {
// We need at least two elements to create meaningful weights. // We need at least two elements to create meaningful weights.
if (Weights.size() < 2) if (Weights.size() < 2)
return nullptr; return nullptr;
// Check for empty weights. // Check for empty weights.
uint64_t MaxWeight = *std::max_element(Weights.begin(), Weights.end()); uint64_t MaxWeight = *std::max_element(Weights.begin(), Weights.end());
if (MaxWeight == 0) if (MaxWeight == 0)
return nullptr; return nullptr;
// Calculate how to scale down to 32-bits. SmallVector<uint64_t, 16> ScaledWeights;
uint64_t Scale = calculateWeightScale(MaxWeight);
SmallVector<uint32_t, 16> ScaledWeights;
ScaledWeights.reserve(Weights.size()); ScaledWeights.reserve(Weights.size());
for (uint64_t W : Weights) for (uint64_t W : Weights)
ScaledWeights.push_back(scaleBranchWeight(W, Scale)); ScaledWeights.push_back(scaleBranchWeight(W));
llvm::MDBuilder MDHelper(CGM.getLLVMContext()); llvm::MDBuilder MDHelper(CGM.getLLVMContext());
return MDHelper.createBranchWeights(ScaledWeights); return MDHelper.createBranchWeights(ScaledWeights);
} }
llvm::MDNode * llvm::MDNode *
CodeGenFunction::createProfileWeightsForLoop(const Stmt *Cond, CodeGenFunction::createProfileWeightsForLoop(const Stmt *Cond,
uint64_t LoopCount) const { uint64_t LoopCount) const {
if (!PGO.haveRegionCounts()) if (!PGO.haveRegionCounts())
return nullptr; return nullptr;
Optional<uint64_t> CondCount = PGO.getStmtCount(Cond); Optional<uint64_t> CondCount = PGO.getStmtCount(Cond);
if (!CondCount || *CondCount == 0) if (!CondCount || *CondCount == 0)
return nullptr; return nullptr;
return createProfileWeights(LoopCount, return createProfileWeights(LoopCount,
std::max(*CondCount, LoopCount) - LoopCount); std::max(*CondCount, LoopCount) - LoopCount);
} }

clang/test/CodeGen/catch-undef-behavior.c

Show First 20 Lines • Show All 379 Lines • ▼ Show 20 Linesvoid call_nonnull_variadic(int a, int *b) {
// CHECK-UBSAN: call void @__ubsan_handle_nonnull_arg // CHECK-UBSAN: call void @__ubsan_handle_nonnull_arg
// CHECK-UBSAN-NOT: __ubsan_handle_nonnull_arg // CHECK-UBSAN-NOT: __ubsan_handle_nonnull_arg
// CHECK-COMMON: call void (i32, ...) @nonnull_variadic // CHECK-COMMON: call void (i32, ...) @nonnull_variadic
nonnull_variadic(a, b); nonnull_variadic(a, b);
} }
// CHECK-UBSAN: ![[WEIGHT_MD]] = !{!"branch_weights", i32 1048575, i32 1} // CHECK-UBSAN: ![[WEIGHT_MD]] = !{!"branch_weights", i64 1048575, i64 1}
// CHECK-TRAP: attributes [[NR_NUW]] = { noreturn nounwind } // CHECK-TRAP: attributes [[NR_NUW]] = { noreturn nounwind }

clang/test/CodeGenCXX/attr-likelihood-if-branch-weights.cpp

Show First 20 Lines • Show All 136 Lines • ▼ Show 20 Lines if (b)
switch (i) switch (i)
[[unlikely]] {} [[unlikely]] {}
else { else {
// Make sure the branches aren't optimized away. // Make sure the branches aren't optimized away.
b = true; b = true;
} }
} }
// CHECK: !7 = !{!"branch_weights", i32 [[UNLIKELY]], i32 [[LIKELY]]} // CHECK: !7 = !{!"branch_weights", i64 [[UNLIKELY]], i64 [[LIKELY]]}
// CHECK: !8 = !{!"branch_weights", i32 [[LIKELY]], i32 [[UNLIKELY]]} // CHECK: !8 = !{!"branch_weights", i64 [[LIKELY]], i64 [[UNLIKELY]]}

clang/test/CodeGenCXX/attr-likelihood-if-vs-builtin-expect.cpp

Show First 20 Lines • Show All 215 Lines • ▼ Show 20 Linesvoid tu2(int &i) {
// CHECK: br {{.*}} !prof !10 // CHECK: br {{.*}} !prof !10
if (a() ? b() : c()) [[unlikely]] { if (a() ? b() : c()) [[unlikely]] {
++i; ++i;
} else { } else {
--i; --i;
} }
} }
// CHECK: !6 = !{!"branch_weights", i32 2000, i32 1} // CHECK: !6 = !{!"branch_weights", i64 2000, i64 1}
// CHECK: !10 = !{!"branch_weights", i32 1, i32 2000} // CHECK: !10 = !{!"branch_weights", i64 1, i64 2000}

clang/test/CodeGenCXX/attr-likelihood-switch-branch-weights.cpp

Show First 20 Lines • Show All 171 Lines • ▼ Show 20 Linesvoid TwoCasesDefaultLNU() {
// CHECK: {{.*}} !prof !20 // CHECK: {{.*}} !prof !20
switch (i) { switch (i) {
[[likely]] case 1: break; [[likely]] case 1: break;
case 2: break; case 2: break;
[[unlikely]] default: break; [[unlikely]] default: break;
} }
} }
// CHECK: !6 = !{!"branch_weights", i32 357913942, i32 715827883} // CHECK: !6 = !{!"branch_weights", i64 357913942, i64 715827883}
// CHECK: !7 = !{!"branch_weights", i32 536870912, i32 1} // CHECK: !7 = !{!"branch_weights", i64 536870912, i64 1}
// CHECK: !8 = !{!"branch_weights", i32 238609295, i32 715827883, i32 238609295} // CHECK: !8 = !{!"branch_weights", i64 238609295, i64 715827883, i64 238609295}
// CHECK: !9 = !{!"branch_weights", i32 357913942, i32 1, i32 357913942} // CHECK: !9 = !{!"branch_weights", i64 357913942, i64 1, i64 357913942}
// CHECK: !10 = !{!"branch_weights", i32 357913942, i32 715827883, i32 1} // CHECK: !10 = !{!"branch_weights", i64 357913942, i64 715827883, i64 1}
// CHECK: !11 = !{!"branch_weights", i32 143165577, i32 143165577, i32 143165577, i32 715827883, i32 143165577} // CHECK: !11 = !{!"branch_weights", i64 143165577, i64 143165577, i64 143165577, i64 715827883, i64 143165577}
// CHECK: !12 = !{!"branch_weights", i32 214748365, i32 214748365, i32 214748365, i32 1, i32 214748365} // CHECK: !12 = !{!"branch_weights", i64 214748365, i64 214748365, i64 214748365, i64 1, i64 214748365}
// CHECK: !13 = !{!"branch_weights", i32 79536432, i32 79536432, i32 79536432, i32 79536432, i32 79536432, i32 79536432, i32 79536432, i32 715827883, i32 79536432} // CHECK: !13 = !{!"branch_weights", i64 79536432, i64 79536432, i64 79536432, i64 79536432, i64 79536432, i64 79536432, i64 79536432, i64 715827883, i64 79536432}
// CHECK: !14 = !{!"branch_weights", i32 119304648, i32 119304648, i32 119304648, i32 119304648, i32 119304648, i32 119304648, i32 119304648, i32 1, i32 119304648} // CHECK: !14 = !{!"branch_weights", i64 119304648, i64 119304648, i64 119304648, i64 119304648, i64 119304648, i64 119304648, i64 119304648, i64 1, i64 119304648}
// CHECK: !15 = !{!"branch_weights", i32 2000, i32 1} // CHECK: !15 = !{!"branch_weights", i64 2000, i64 1}
// CHECK: !16 = !{!"branch_weights", i32 1, i32 2000} // CHECK: !16 = !{!"branch_weights", i64 1, i64 2000}
// CHECK: !17 = !{!"branch_weights", i32 715827883, i32 357913942} // CHECK: !17 = !{!"branch_weights", i64 715827883, i64 357913942}
// CHECK: !18 = !{!"branch_weights", i32 1, i32 536870912} // CHECK: !18 = !{!"branch_weights", i64 1, i64 536870912}
// CHECK: !19 = !{!"branch_weights", i32 536870912, i32 536870912, i32 268435456} // CHECK: !19 = !{!"branch_weights", i64 536870912, i64 536870912, i64 268435456}
// CHECK: !20 = !{!"branch_weights", i32 1, i32 715827883, i32 357913942} // CHECK: !20 = !{!"branch_weights", i64 1, i64 715827883, i64 357913942}

clang/test/CodeGenCXX/microsoft-abi-thread-safe-statics.cpp

Show First 20 Lines • Show All 90 Lines • ▼ Show 20 Lines
// CHECK-LABEL: define dso_local i32 @"?g1@@YAHXZ"() // CHECK-LABEL: define dso_local i32 @"?g1@@YAHXZ"()
int f1(); int f1();
int g1() { int g1() {
static int i = f1(); static int i = f1();
return i; return i;
} }
// CHECK-DAG: ![[unlikely_threadlocal]] = !{!"branch_weights", i32 1, i32 1023} // CHECK-DAG: ![[unlikely_threadlocal]] = !{!"branch_weights", i64 1, i64 1023}
// CHECK-DAG: ![[unlikely_staticlocal]] = !{!"branch_weights", i32 1, i32 1048575} // CHECK-DAG: ![[unlikely_staticlocal]] = !{!"branch_weights", i64 1, i64 1048575}

clang/test/CodeGenCXX/profile-remap.cpp

Show All 18 Linesnamespace Foo {
} }
} }
// CHECK: define {{.*}} @_ZN3Foo8functionENS_1XE() {{.*}} !prof [[FUNC_ENTRY:![0-9]*]] // CHECK: define {{.*}} @_ZN3Foo8functionENS_1XE() {{.*}} !prof [[FUNC_ENTRY:![0-9]*]]
// CHECK: br i1 {{.*}} !prof [[BR_WEIGHTS:![0-9]*]] // CHECK: br i1 {{.*}} !prof [[BR_WEIGHTS:![0-9]*]]
// //
// FIXME: Laplace's rule of succession is applied to sample profiles... // FIXME: Laplace's rule of succession is applied to sample profiles...
// CHECK-SAMPLES-DAG: [[FUNC_ENTRY]] = !{!"function_entry_count", i64 1} // CHECK-SAMPLES-DAG: [[FUNC_ENTRY]] = !{!"function_entry_count", i64 1}
// CHECK-SAMPLES-DAG: [[BR_WEIGHTS]] = !{!"branch_weights", i32 11, i32 91} // CHECK-SAMPLES-DAG: [[BR_WEIGHTS]] = !{!"branch_weights", i64 11, i64 91}
// //
// ... but not to instruction profiles. // ... but not to instruction profiles.
// CHECK-INSTR-DAG: [[FUNC_ENTRY]] = !{!"function_entry_count", i64 100} // CHECK-INSTR-DAG: [[FUNC_ENTRY]] = !{!"function_entry_count", i64 100}
// CHECK-INSTR-DAG: [[BR_WEIGHTS]] = !{!"branch_weights", i32 10, i32 90} // CHECK-INSTR-DAG: [[BR_WEIGHTS]] = !{!"branch_weights", i64 10, i64 90}

clang/test/CodeGenCXX/static-initializer-branch-weights.cpp

Show First 20 Lines • Show All 116 Lines • ▼ Show 20 Lines use(B<int>::member, B<int>::inline_member, B<int>::thread_local_member,
B<int>::thread_local_inline_member); B<int>::thread_local_inline_member);
} }
// CHECK-LABEL: define {{.*}}tls_init() // CHECK-LABEL: define {{.*}}tls_init()
// CHECK: load i8, i8* @__tls_guard, align 1 // CHECK: load i8, i8* @__tls_guard, align 1
// CHECK: icmp eq i8 {{.*}}, 0 // CHECK: icmp eq i8 {{.*}}, 0
// CHECK: br i1 {{.*}}, !prof ![[WEIGHTS_THREAD_LOCAL]] // CHECK: br i1 {{.*}}, !prof ![[WEIGHTS_THREAD_LOCAL]]
// CHECK-DAG: ![[WEIGHTS_THREAD_LOCAL]] = !{!"branch_weights", i32 1, i32 1023} // CHECK-DAG: ![[WEIGHTS_THREAD_LOCAL]] = !{!"branch_weights", i64 1, i64 1023}
// CHECK-DAG: ![[WEIGHTS_LOCAL]] = !{!"branch_weights", i32 1, i32 1048575} // CHECK-DAG: ![[WEIGHTS_LOCAL]] = !{!"branch_weights", i64 1, i64 1048575}

clang/test/Profile/c-captured.c

Show First 20 Lines • Show All 41 Lines • ▼ Show 20 Lines #pragma clang __debug captured
// PGOUSE: br {{.*}} !prof ![[C12:[0-9]+]] // PGOUSE: br {{.*}} !prof ![[C12:[0-9]+]]
if (x) {} if (x) {}
// PGOALL: ret // PGOALL: ret
} }
if (x) {} // This is DC2. Checked above. if (x) {} // This is DC2. Checked above.
} }
// PGOUSE-DAG: ![[DC1]] = !{!"branch_weights", i32 2, i32 1} // PGOUSE-DAG: ![[DC1]] = !{!"branch_weights", i64 2, i64 1}
// PGOUSE-DAG: ![[DC2]] = !{!"branch_weights", i32 2, i32 1} // PGOUSE-DAG: ![[DC2]] = !{!"branch_weights", i64 2, i64 1}
// PGOUSE-DAG: ![[CS1]] = !{!"branch_weights", i32 2, i32 1} // PGOUSE-DAG: ![[CS1]] = !{!"branch_weights", i64 2, i64 1}
// PGOUSE-DAG: ![[C11]] = !{!"branch_weights", i32 11, i32 2} // PGOUSE-DAG: ![[C11]] = !{!"branch_weights", i64 11, i64 2}
// PGOUSE-DAG: ![[C12]] = !{!"branch_weights", i32 2, i32 1} // PGOUSE-DAG: ![[C12]] = !{!"branch_weights", i64 2, i64 1}
int main(int argc, const char *argv[]) { int main(int argc, const char *argv[]) {
debug_captured(); debug_captured();
return 0; return 0;
} }

clang/test/Profile/c-counter-overflows.c

  • This file was deleted.
// Test that big branch weights get scaled down to 32-bits, rather than just
// truncated.
// RUN: llvm-profdata merge %S/Inputs/c-counter-overflows.proftext -o %t.profdata
// RUN: %clang_cc1 -triple x86_64-apple-macosx10.9 -main-file-name c-counter-overflows.c %s -o - -emit-llvm -fprofile-instrument-use-path=%t.profdata | FileCheck %s
typedef unsigned long long uint64_t;
int main(int argc, const char *argv[]) {
// Need counts higher than 32-bits.
// CHECK: br {{.*}} !prof ![[FOR:[0-9]+]]
// max = 0xffffffff0
// scale = 0xffffffff0 / 0xffffffff + 1 = 17
// loop-body: 0xffffffff0 / 17 + 1 = 0xf0f0f0f0 + 1 = 4042322161 => -252645135
// loop-exit: 0x000000001 / 17 + 1 = 0x00000000 + 1 = 1 => 1
for (uint64_t I = 0; I < 0xffffffff0; ++I) {
// max = 0xffffffff * 15 = 0xefffffff1
// scale = 0xefffffff1 / 0xffffffff + 1 = 16
// CHECK: br {{.*}} !prof ![[IF:[0-9]+]]
if (I & 0xf) {
// 0xefffffff1 / 16 + 1 = 0xefffffff + 1 = 4026531840 => -268435456
} else {
// 0x0ffffffff / 16 + 1 = 0x0fffffff + 1 = 268435456 => 268435456
}
// max = 0xffffffff * 5 = 0x4fffffffb
// scale = 0x4fffffffb / 0xffffffff + 1 = 6
// CHECK: ], !prof ![[SWITCH:[0-9]+]]
switch ((I & 0xf) / 5) {
case 0:
// 0x4fffffffb / 6 = 0xd5555554 + 1 = 3579139413 => -715827883
break;
case 1:
// 0x4fffffffb / 6 = 0xd5555554 + 1 = 3579139413 => -715827883
break;
case 2:
// 0x4fffffffb / 6 = 0xd5555554 + 1 = 3579139413 => -715827883
break;
default:
// 0x0ffffffff / 6 = 0x2aaaaaaa + 1 = 715827883 => 715827883
break;
}
}
return 0;
}
// CHECK-DAG: ![[FOR]] = !{!"branch_weights", i32 -252645135, i32 1}
// CHECK-DAG: ![[IF]] = !{!"branch_weights", i32 -268435456, i32 268435456}
// CHECK-DAG: ![[SWITCH]] = !{!"branch_weights", i32 715827883, i32 -715827883, i32 -715827883, i32 -715827883}

clang/test/Profile/c-general.c

Show First 20 Lines • Show All 464 Lines • ▼ Show 20 Lines
// PGOGEN: store {{.*}} @[[STC]], i64 0, i64 0 // PGOGEN: store {{.*}} @[[STC]], i64 0, i64 0
static void static_func() { static void static_func() {
// PGOGEN: store {{.*}} @[[STC]], i64 0, i64 1 // PGOGEN: store {{.*}} @[[STC]], i64 0, i64 1
// PGOUSE: br {{.*}} !prof ![[ST1:[0-9]+]] // PGOUSE: br {{.*}} !prof ![[ST1:[0-9]+]]
for (int i = 0; i < 10; ++i) { for (int i = 0; i < 10; ++i) {
} }
} }
// PGOUSE-DAG: ![[SL1]] = !{!"branch_weights", i32 101, i32 2} // PGOUSE-DAG: ![[SL1]] = !{!"branch_weights", i64 101, i64 2}
// PGOUSE-DAG: ![[SL2]] = !{!"branch_weights", i32 101, i32 2} // PGOUSE-DAG: ![[SL2]] = !{!"branch_weights", i64 101, i64 2}
// PGOUSE-DAG: ![[SL3]] = !{!"branch_weights", i32 76, i32 2} // PGOUSE-DAG: ![[SL3]] = !{!"branch_weights", i64 76, i64 2}
// PGOUSE-DAG: ![[EE1]] = !{!"branch_weights", i32 1, i32 2} // PGOUSE-DAG: ![[EE1]] = !{!"branch_weights", i64 1, i64 2}
// PGOUSE-DAG: ![[EE2]] = !{!"branch_weights", i32 52, i32 1} // PGOUSE-DAG: ![[EE2]] = !{!"branch_weights", i64 52, i64 1}
// PGOUSE-DAG: ![[EE3]] = !{!"branch_weights", i32 2, i32 51} // PGOUSE-DAG: ![[EE3]] = !{!"branch_weights", i64 2, i64 51}
// PGOUSE-DAG: ![[EE4]] = !{!"branch_weights", i32 26, i32 26} // PGOUSE-DAG: ![[EE4]] = !{!"branch_weights", i64 26, i64 26}
// PGOUSE-DAG: ![[EE5]] = !{!"branch_weights", i32 2, i32 1} // PGOUSE-DAG: ![[EE5]] = !{!"branch_weights", i64 2, i64 1}
// PGOUSE-DAG: ![[EE6]] = !{!"branch_weights", i32 2, i32 26} // PGOUSE-DAG: ![[EE6]] = !{!"branch_weights", i64 2, i64 26}
// PGOUSE-DAG: ![[EE7]] = !{!"branch_weights", i32 26, i32 1} // PGOUSE-DAG: ![[EE7]] = !{!"branch_weights", i64 26, i64 1}
// PGOUSE-DAG: ![[IF1]] = !{!"branch_weights", i32 101, i32 2} // PGOUSE-DAG: ![[IF1]] = !{!"branch_weights", i64 101, i64 2}
// PGOUSE-DAG: ![[IF2]] = !{!"branch_weights", i32 51, i32 51} // PGOUSE-DAG: ![[IF2]] = !{!"branch_weights", i64 51, i64 51}
// PGOUSE-DAG: ![[IF3]] = !{!"branch_weights", i32 51, i32 1} // PGOUSE-DAG: ![[IF3]] = !{!"branch_weights", i64 51, i64 1}
// PGOUSE-DAG: ![[IF4]] = !{!"branch_weights", i32 34, i32 18} // PGOUSE-DAG: ![[IF4]] = !{!"branch_weights", i64 34, i64 18}
// PGOUSE-DAG: ![[IF5]] = !{!"branch_weights", i32 34, i32 1} // PGOUSE-DAG: ![[IF5]] = !{!"branch_weights", i64 34, i64 1}
// PGOUSE-DAG: ![[IF6]] = !{!"branch_weights", i32 17, i32 2} // PGOUSE-DAG: ![[IF6]] = !{!"branch_weights", i64 17, i64 2}
// PGOUSE-DAG: ![[IF7]] = !{!"branch_weights", i32 100, i32 2} // PGOUSE-DAG: ![[IF7]] = !{!"branch_weights", i64 100, i64 2}
// PGOUSE-DAG: ![[IF8]] = !{!"branch_weights", i32 100, i32 2} // PGOUSE-DAG: ![[IF8]] = !{!"branch_weights", i64 100, i64 2}
// PGOUSE-DAG: ![[JM1]] = !{!"branch_weights", i32 2, i32 1} // PGOUSE-DAG: ![[JM1]] = !{!"branch_weights", i64 2, i64 1}
// PGOUSE-DAG: ![[JM2]] = !{!"branch_weights", i32 1, i32 2} // PGOUSE-DAG: ![[JM2]] = !{!"branch_weights", i64 1, i64 2}
// PGOUSE-DAG: ![[JM3]] = !{!"branch_weights", i32 1, i32 2} // PGOUSE-DAG: ![[JM3]] = !{!"branch_weights", i64 1, i64 2}
// PGOUSE-DAG: ![[JM4]] = !{!"branch_weights", i32 1, i32 2} // PGOUSE-DAG: ![[JM4]] = !{!"branch_weights", i64 1, i64 2}
// PGOUSE-DAG: ![[JM5]] = !{!"branch_weights", i32 3, i32 2} // PGOUSE-DAG: ![[JM5]] = !{!"branch_weights", i64 3, i64 2}
// PGOUSE-DAG: ![[JM6]] = !{!"branch_weights", i32 1, i32 2} // PGOUSE-DAG: ![[JM6]] = !{!"branch_weights", i64 1, i64 2}
// PGOUSE-DAG: ![[JM7]] = !{!"branch_weights", i32 1, i32 2, i32 2, i32 2} // PGOUSE-DAG: ![[JM7]] = !{!"branch_weights", i64 1, i64 2, i64 2, i64 2}
// PGOUSE-DAG: ![[JM8]] = !{!"branch_weights", i32 11, i32 2} // PGOUSE-DAG: ![[JM8]] = !{!"branch_weights", i64 11, i64 2}
// PGOUSE-DAG: ![[JM9]] = !{!"branch_weights", i32 10, i32 2} // PGOUSE-DAG: ![[JM9]] = !{!"branch_weights", i64 10, i64 2}
// PGOUSE-DAG: ![[SW1]] = !{!"branch_weights", i32 16, i32 1} // PGOUSE-DAG: ![[SW1]] = !{!"branch_weights", i64 16, i64 1}
// PGOUSE-DAG: ![[SW2]] = !{!"branch_weights", i32 6, i32 2, i32 3, i32 4, i32 5} // PGOUSE-DAG: ![[SW2]] = !{!"branch_weights", i64 6, i64 2, i64 3, i64 4, i64 5}
// PGOUSE-DAG: ![[SW3]] = !{!"branch_weights", i32 1, i32 2} // PGOUSE-DAG: ![[SW3]] = !{!"branch_weights", i64 1, i64 2}
// PGOUSE-DAG: ![[SW4]] = !{!"branch_weights", i32 3, i32 2} // PGOUSE-DAG: ![[SW4]] = !{!"branch_weights", i64 3, i64 2}
// PGOUSE-DAG: ![[SW5]] = !{!"branch_weights", i32 4, i32 1} // PGOUSE-DAG: ![[SW5]] = !{!"branch_weights", i64 4, i64 1}
// PGOUSE-DAG: ![[SW6]] = !{!"branch_weights", i32 5, i32 1} // PGOUSE-DAG: ![[SW6]] = !{!"branch_weights", i64 5, i64 1}
// PGOUSE-DAG: ![[SW7]] = !{!"branch_weights", i32 1, i32 2, i32 2, i32 2, i32 2} // PGOUSE-DAG: ![[SW7]] = !{!"branch_weights", i64 1, i64 2, i64 2, i64 2, i64 2}
// PGOUSE-DAG: ![[SW8]] = !{!"branch_weights", i32 5, i32 1} // PGOUSE-DAG: ![[SW8]] = !{!"branch_weights", i64 5, i64 1}
// PGOUSE-DAG: ![[SW9]] = !{!"branch_weights", i32 2, i32 5} // PGOUSE-DAG: ![[SW9]] = !{!"branch_weights", i64 2, i64 5}
// PGOUSE-DAG: ![[BS1]] = !{!"branch_weights", i32 33, i32 2} // PGOUSE-DAG: ![[BS1]] = !{!"branch_weights", i64 33, i64 2}
// PGOUSE-DAG: ![[BS2]] = !{!"branch_weights", i32 29, i32 2, i32 2, i32 2, i32 2, i32 1} // PGOUSE-DAG: ![[BS2]] = !{!"branch_weights", i64 29, i64 2, i64 2, i64 2, i64 2, i64 1}
// PGOUSE-DAG: ![[BS3]] = !{!"branch_weights", i32 1, i32 2} // PGOUSE-DAG: ![[BS3]] = !{!"branch_weights", i64 1, i64 2}
// PGOUSE-DAG: ![[BS4]] = !{!"branch_weights", i32 2, i32 2} // PGOUSE-DAG: ![[BS4]] = !{!"branch_weights", i64 2, i64 2}
// PGOUSE-DAG: ![[BS5]] = !{!"branch_weights", i32 12, i32 1} // PGOUSE-DAG: ![[BS5]] = !{!"branch_weights", i64 12, i64 1}
// PGOUSE-DAG: ![[BS6]] = !{!"branch_weights", i32 12, i32 3} // PGOUSE-DAG: ![[BS6]] = !{!"branch_weights", i64 12, i64 3}
// PGOUSE-DAG: ![[BS7]] = !{!"branch_weights", i32 2, i32 1} // PGOUSE-DAG: ![[BS7]] = !{!"branch_weights", i64 2, i64 1}
// PGOUSE-DAG: ![[BS8]] = !{!"branch_weights", i32 16, i32 1} // PGOUSE-DAG: ![[BS8]] = !{!"branch_weights", i64 16, i64 1}
// PGOUSE-DAG: ![[BS9]] = !{!"branch_weights", i32 16, i32 14} // PGOUSE-DAG: ![[BS9]] = !{!"branch_weights", i64 16, i64 14}
// PGOUSE-DAG: ![[BS10]] = !{!"branch_weights", i32 2, i32 1} // PGOUSE-DAG: ![[BS10]] = !{!"branch_weights", i64 2, i64 1}
// PGOUSE-DAG: ![[BS11]] = !{!"branch_weights", i32 3, i32 1} // PGOUSE-DAG: ![[BS11]] = !{!"branch_weights", i64 3, i64 1}
// PGOUSE-DAG: ![[BO1]] = !{!"branch_weights", i32 101, i32 2} // PGOUSE-DAG: ![[BO1]] = !{!"branch_weights", i64 101, i64 2}
// PGOUSE-DAG: ![[BO2]] = !{!"branch_weights", i32 67, i32 35} // PGOUSE-DAG: ![[BO2]] = !{!"branch_weights", i64 67, i64 35}
// PGOUSE-DAG: ![[BO3]] = !{!"branch_weights", i32 67, i32 35} // PGOUSE-DAG: ![[BO3]] = !{!"branch_weights", i64 67, i64 35}
// PGOUSE-DAG: ![[BO4]] = !{!"branch_weights", i32 67, i32 35} // PGOUSE-DAG: ![[BO4]] = !{!"branch_weights", i64 67, i64 35}
// PGOUSE-DAG: ![[BO5]] = !{!"branch_weights", i32 18, i32 18} // PGOUSE-DAG: ![[BO5]] = !{!"branch_weights", i64 18, i64 18}
// PGOUSE-DAG: ![[BO6]] = !{!"branch_weights", i32 51, i32 51} // PGOUSE-DAG: ![[BO6]] = !{!"branch_weights", i64 51, i64 51}
// PGOUSE-DAG: ![[BO7]] = !{!"branch_weights", i32 34, i32 18} // PGOUSE-DAG: ![[BO7]] = !{!"branch_weights", i64 34, i64 18}
// PGOUSE-DAG: ![[BL1]] = !{!"branch_weights", i32 52, i32 1} // PGOUSE-DAG: ![[BL1]] = !{!"branch_weights", i64 52, i64 1}
// PGOUSE-DAG: ![[BL2]] = !{!"branch_weights", i32 51, i32 2} // PGOUSE-DAG: ![[BL2]] = !{!"branch_weights", i64 51, i64 2}
// PGOUSE-DAG: ![[BL3]] = !{!"branch_weights", i32 26, i32 27} // PGOUSE-DAG: ![[BL3]] = !{!"branch_weights", i64 26, i64 27}
// PGOUSE-DAG: ![[BL4]] = !{!"branch_weights", i32 51, i32 2} // PGOUSE-DAG: ![[BL4]] = !{!"branch_weights", i64 51, i64 2}
// PGOUSE-DAG: ![[BL5]] = !{!"branch_weights", i32 52, i32 1} // PGOUSE-DAG: ![[BL5]] = !{!"branch_weights", i64 52, i64 1}
// PGOUSE-DAG: ![[BL6]] = !{!"branch_weights", i32 51, i32 2} // PGOUSE-DAG: ![[BL6]] = !{!"branch_weights", i64 51, i64 2}
// PGOUSE-DAG: ![[BL7]] = !{!"branch_weights", i32 26, i32 27} // PGOUSE-DAG: ![[BL7]] = !{!"branch_weights", i64 26, i64 27}
// PGOUSE-DAG: ![[BL8]] = !{!"branch_weights", i32 51, i32 2} // PGOUSE-DAG: ![[BL8]] = !{!"branch_weights", i64 51, i64 2}
// PGOUSE-DAG: ![[CO1]] = !{!"branch_weights", i32 1, i32 2} // PGOUSE-DAG: ![[CO1]] = !{!"branch_weights", i64 1, i64 2}
// PGOUSE-DAG: ![[CO2]] = !{!"branch_weights", i32 2, i32 1} // PGOUSE-DAG: ![[CO2]] = !{!"branch_weights", i64 2, i64 1}
// PGOUSE-DAG: ![[DF1]] = !{!"branch_weights", i32 11, i32 2} // PGOUSE-DAG: ![[DF1]] = !{!"branch_weights", i64 11, i64 2}
// PGOUSE-DAG: ![[DF2]] = !{!"branch_weights", i32 3, i32 3} // PGOUSE-DAG: ![[DF2]] = !{!"branch_weights", i64 3, i64 3}
// PGOUSE-DAG: ![[DF3]] = !{!"branch_weights", i32 9, i32 5} // PGOUSE-DAG: ![[DF3]] = !{!"branch_weights", i64 9, i64 5}
// PGOUSE-DAG: ![[ST1]] = !{!"branch_weights", i32 11, i32 2} // PGOUSE-DAG: ![[ST1]] = !{!"branch_weights", i64 11, i64 2}
int main(int argc, const char *argv[]) { int main(int argc, const char *argv[]) {
simple_loops(); simple_loops();
conditionals(); conditionals();
early_exits(); early_exits();
jumps(); jumps();
switches(); switches();
big_switch(); big_switch();
boolean_operators(); boolean_operators();
boolop_loops(); boolop_loops();
conditional_operator(); conditional_operator();
do_fallthrough(); do_fallthrough();
static_func(); static_func();
return 0; return 0;
} }

clang/test/Profile/cxx-class.cpp

Show All 25 Linespublic:
explicit Simple(int Member) : Member(Member) { explicit Simple(int Member) : Member(Member) {
// CTRGEN: store {{.*}} @[[SCC]], i64 0, i64 1 // CTRGEN: store {{.*}} @[[SCC]], i64 0, i64 1
// CTRUSE: br {{.*}} !prof ![[SC1:[0-9]+]] // CTRUSE: br {{.*}} !prof ![[SC1:[0-9]+]]
if (Member) {} if (Member) {}
// CTRGEN-NOT: store {{.*}} @[[SCC]], // CTRGEN-NOT: store {{.*}} @[[SCC]],
// CTRUSE-NOT: br {{.*}} !prof ![0-9]+ // CTRUSE-NOT: br {{.*}} !prof ![0-9]+
// CTRUSE: ret // CTRUSE: ret
} }
// CTRUSE: ![[SC1]] = !{!"branch_weights", i32 100, i32 2} // CTRUSE: ![[SC1]] = !{!"branch_weights", i64 100, i64 2}
// DTRGEN-LABEL: define {{.*}} @_ZN6SimpleD2Ev( // DTRGEN-LABEL: define {{.*}} @_ZN6SimpleD2Ev(
// DTRUSE-LABEL: define {{.*}} @_ZN6SimpleD2Ev( // DTRUSE-LABEL: define {{.*}} @_ZN6SimpleD2Ev(
// DTRGEN: store {{.*}} @[[SDC:__profc__ZN6SimpleD2Ev]], i64 0, i64 0 // DTRGEN: store {{.*}} @[[SDC:__profc__ZN6SimpleD2Ev]], i64 0, i64 0
~Simple() { ~Simple() {
// DTRGEN: store {{.*}} @[[SDC]], i64 0, i64 1 // DTRGEN: store {{.*}} @[[SDC]], i64 0, i64 1
// DTRUSE: br {{.*}} !prof ![[SD1:[0-9]+]] // DTRUSE: br {{.*}} !prof ![[SD1:[0-9]+]]
if (Member) {} if (Member) {}
// DTRGEN-NOT: store {{.*}} @[[SDC]], // DTRGEN-NOT: store {{.*}} @[[SDC]],
// DTRUSE-NOT: br {{.*}} !prof ![0-9]+ // DTRUSE-NOT: br {{.*}} !prof ![0-9]+
// DTRUSE: ret // DTRUSE: ret
} }
// DTRUSE: ![[SD1]] = !{!"branch_weights", i32 100, i32 2} // DTRUSE: ![[SD1]] = !{!"branch_weights", i64 100, i64 2}
// MTHGEN-LABEL: define {{.*}} @_ZN6Simple6methodEv( // MTHGEN-LABEL: define {{.*}} @_ZN6Simple6methodEv(
// MTHUSE-LABEL: define {{.*}} @_ZN6Simple6methodEv( // MTHUSE-LABEL: define {{.*}} @_ZN6Simple6methodEv(
// MTHGEN: store {{.*}} @[[SMC:__profc__ZN6Simple6methodEv]], i64 0, i64 0 // MTHGEN: store {{.*}} @[[SMC:__profc__ZN6Simple6methodEv]], i64 0, i64 0
void method() { void method() {
// MTHGEN: store {{.*}} @[[SMC]], i64 0, i64 1 // MTHGEN: store {{.*}} @[[SMC]], i64 0, i64 1
// MTHUSE: br {{.*}} !prof ![[SM1:[0-9]+]] // MTHUSE: br {{.*}} !prof ![[SM1:[0-9]+]]
if (Member) {} if (Member) {}
// MTHGEN-NOT: store {{.*}} @[[SMC]], // MTHGEN-NOT: store {{.*}} @[[SMC]],
// MTHUSE-NOT: br {{.*}} !prof ![0-9]+ // MTHUSE-NOT: br {{.*}} !prof ![0-9]+
// MTHUSE: ret // MTHUSE: ret
} }
// MTHUSE: ![[SM1]] = !{!"branch_weights", i32 100, i32 2} // MTHUSE: ![[SM1]] = !{!"branch_weights", i64 100, i64 2}
}; };
class Derived : virtual public Simple { class Derived : virtual public Simple {
public: public:
// VCTRGEN-LABEL: define {{.*}} @_ZN7DerivedC1Ev( // VCTRGEN-LABEL: define {{.*}} @_ZN7DerivedC1Ev(
// VCTRUSE-LABEL: define {{.*}} @_ZN7DerivedC1Ev( // VCTRUSE-LABEL: define {{.*}} @_ZN7DerivedC1Ev(
// VCTRGEN: store {{.*}} @[[SCC:__profc__ZN7DerivedC1Ev]], i64 0, i64 0 // VCTRGEN: store {{.*}} @[[SCC:__profc__ZN7DerivedC1Ev]], i64 0, i64 0
Derived() : Simple(0) { Derived() : Simple(0) {
// VCTRGEN: store {{.*}} @[[SCC]], i64 0, i64 1 // VCTRGEN: store {{.*}} @[[SCC]], i64 0, i64 1
// VCTRUSE: br {{.*}} !prof ![[SC1:[0-9]+]] // VCTRUSE: br {{.*}} !prof ![[SC1:[0-9]+]]
if (Member) {} if (Member) {}
// VCTRGEN-NOT: store {{.*}} @[[SCC]], // VCTRGEN-NOT: store {{.*}} @[[SCC]],
// VCTRUSE-NOT: br {{.*}} !prof ![0-9]+ // VCTRUSE-NOT: br {{.*}} !prof ![0-9]+
// VCTRUSE: ret // VCTRUSE: ret
} }
// VCTRUSE: ![[SC1]] = !{!"branch_weights", i32 100, i32 2} // VCTRUSE: ![[SC1]] = !{!"branch_weights", i64 100, i64 2}
// VDTRGEN-LABEL: define {{.*}} @_ZN7DerivedD2Ev( // VDTRGEN-LABEL: define {{.*}} @_ZN7DerivedD2Ev(
// VDTRUSE-LABEL: define {{.*}} @_ZN7DerivedD2Ev( // VDTRUSE-LABEL: define {{.*}} @_ZN7DerivedD2Ev(
// VDTRGEN: store {{.*}} @[[SDC:__profc__ZN7DerivedD2Ev]], i64 0, i64 0 // VDTRGEN: store {{.*}} @[[SDC:__profc__ZN7DerivedD2Ev]], i64 0, i64 0
~Derived() { ~Derived() {
// VDTRGEN: store {{.*}} @[[SDC]], i64 0, i64 1 // VDTRGEN: store {{.*}} @[[SDC]], i64 0, i64 1
// VDTRUSE: br {{.*}} !prof ![[SD1:[0-9]+]] // VDTRUSE: br {{.*}} !prof ![[SD1:[0-9]+]]
if (Member) {} if (Member) {}
// VDTRGEN-NOT: store {{.*}} @[[SDC]], // VDTRGEN-NOT: store {{.*}} @[[SDC]],
// VDTRUSE-NOT: br {{.*}} !prof ![0-9]+ // VDTRUSE-NOT: br {{.*}} !prof ![0-9]+
// VDTRUSE: ret // VDTRUSE: ret
} }
// VDTRUSE: ![[SD1]] = !{!"branch_weights", i32 100, i32 2} // VDTRUSE: ![[SD1]] = !{!"branch_weights", i64 100, i64 2}
}; };
// WRPGEN-LABEL: define {{.*}} @_Z14simple_wrapperv( // WRPGEN-LABEL: define {{.*}} @_Z14simple_wrapperv(
// WRPUSE-LABEL: define {{.*}} @_Z14simple_wrapperv( // WRPUSE-LABEL: define {{.*}} @_Z14simple_wrapperv(
// WRPGEN: store {{.*}} @[[SWC:__profc__Z14simple_wrapperv]], i64 0, i64 0 // WRPGEN: store {{.*}} @[[SWC:__profc__Z14simple_wrapperv]], i64 0, i64 0
void simple_wrapper() { void simple_wrapper() {
// WRPGEN: store {{.*}} @[[SWC]], i64 0, i64 1 // WRPGEN: store {{.*}} @[[SWC]], i64 0, i64 1
// WRPUSE: br {{.*}} !prof ![[SW1:[0-9]+]] // WRPUSE: br {{.*}} !prof ![[SW1:[0-9]+]]
for (int I = 0; I < 100; ++I) { for (int I = 0; I < 100; ++I) {
Derived d; Derived d;
Simple S(I); Simple S(I);
S.method(); S.method();
} }
// WRPGEN-NOT: store {{.*}} @[[SWC]], // WRPGEN-NOT: store {{.*}} @[[SWC]],
// WRPUSE-NOT: br {{.*}} !prof ![0-9]+ // WRPUSE-NOT: br {{.*}} !prof ![0-9]+
// WRPUSE: ret // WRPUSE: ret
} }
// WRPUSE: ![[SW1]] = !{!"branch_weights", i32 101, i32 2} // WRPUSE: ![[SW1]] = !{!"branch_weights", i64 101, i64 2}
int main(int argc, const char *argv[]) { int main(int argc, const char *argv[]) {
simple_wrapper(); simple_wrapper();
return 0; return 0;
} }

clang/test/Profile/cxx-lambda.cpp

Show All 39 Linesvoid lambdas() {
for (i = 0; i < 10; ++i) for (i = 0; i < 10; ++i)
f(9 - i); f(9 - i);
// PGOGEN: store {{.*}} @[[LWC]], i64 0, i64 3 // PGOGEN: store {{.*}} @[[LWC]], i64 0, i64 3
// PGOUSE: br {{.*}} !prof ![[LW3:[0-9]+]] // PGOUSE: br {{.*}} !prof ![[LW3:[0-9]+]]
if (i) {} if (i) {}
} }
// PGOUSE-DAG: ![[LW1]] = !{!"branch_weights", i32 2, i32 1} // PGOUSE-DAG: ![[LW1]] = !{!"branch_weights", i64 2, i64 1}
// PGOUSE-DAG: ![[LW2]] = !{!"branch_weights", i32 11, i32 2} // PGOUSE-DAG: ![[LW2]] = !{!"branch_weights", i64 11, i64 2}
// PGOUSE-DAG: ![[LW3]] = !{!"branch_weights", i32 2, i32 1} // PGOUSE-DAG: ![[LW3]] = !{!"branch_weights", i64 2, i64 1}
// LMBUSE-DAG: ![[LF1]] = !{!"branch_weights", i32 10, i32 2} // LMBUSE-DAG: ![[LF1]] = !{!"branch_weights", i64 10, i64 2}
// LMBUSE-DAG: ![[LF2]] = !{!"branch_weights", i32 10, i32 2} // LMBUSE-DAG: ![[LF2]] = !{!"branch_weights", i64 10, i64 2}
int main(int argc, const char *argv[]) { int main(int argc, const char *argv[]) {
lambdas(); lambdas();
return 0; return 0;
} }

clang/test/Profile/cxx-rangefor.cpp

Show All 27 Lines if (sum >= 7)
break; break;
} }
// PGOGEN: store {{.*}} @[[RFC]], i64 0, i64 4 // PGOGEN: store {{.*}} @[[RFC]], i64 0, i64 4
// PGOUSE: br {{.*}} !prof ![[RF4:[0-9]+]] // PGOUSE: br {{.*}} !prof ![[RF4:[0-9]+]]
if (sum) {} if (sum) {}
} }
// PGOUSE-DAG: ![[RF1]] = !{!"branch_weights", i32 5, i32 1} // PGOUSE-DAG: ![[RF1]] = !{!"branch_weights", i64 5, i64 1}
// PGOUSE-DAG: ![[RF2]] = !{!"branch_weights", i32 2, i32 4} // PGOUSE-DAG: ![[RF2]] = !{!"branch_weights", i64 2, i64 4}
// PGOUSE-DAG: ![[RF3]] = !{!"branch_weights", i32 2, i32 3} // PGOUSE-DAG: ![[RF3]] = !{!"branch_weights", i64 2, i64 3}
// PGOUSE-DAG: ![[RF4]] = !{!"branch_weights", i32 2, i32 1} // PGOUSE-DAG: ![[RF4]] = !{!"branch_weights", i64 2, i64 1}
int main(int argc, const char *argv[]) { int main(int argc, const char *argv[]) {
range_for(); range_for();
return 0; return 0;
} }

clang/test/Profile/cxx-templates.cpp

Show All 26 Linestemplate <unsigned N> void loop() {
// T0USE: br {{.*}} !prof ![[T01:[0-9]+]] // T0USE: br {{.*}} !prof ![[T01:[0-9]+]]
// T100GEN: store {{.*}} @[[T100C]], i64 0, i64 1 // T100GEN: store {{.*}} @[[T100C]], i64 0, i64 1
// T100USE: br {{.*}} !prof ![[T1001:[0-9]+]] // T100USE: br {{.*}} !prof ![[T1001:[0-9]+]]
for (unsigned I = 0; I < N; ++I) {} for (unsigned I = 0; I < N; ++I) {}
// ALL: ret // ALL: ret
} }
// T0USE-DAG: ![[T01]] = !{!"branch_weights", i32 1, i32 2} // T0USE-DAG: ![[T01]] = !{!"branch_weights", i64 1, i64 2}
// T100USE-DAG: ![[T1001]] = !{!"branch_weights", i32 101, i32 2} // T100USE-DAG: ![[T1001]] = !{!"branch_weights", i64 101, i64 2}
int main(int argc, const char *argv[]) { int main(int argc, const char *argv[]) {
loop<0>(); loop<0>();
loop<100>(); loop<100>();
return 0; return 0;
} }

clang/test/Profile/cxx-throws.cpp

Show First 20 Lines • Show All 70 Lines • ▼ Show 20 Lines if (i)
throw i; throw i;
// PGOGEN: store {{.*}} @[[UNC]], i64 0, i64 2 // PGOGEN: store {{.*}} @[[UNC]], i64 0, i64 2
// Since we never reach here, the weights should all be zero (and skipped) // Since we never reach here, the weights should all be zero (and skipped)
// PGOUSE-NOT: br {{.*}} !prof !{{.*}} // PGOUSE-NOT: br {{.*}} !prof !{{.*}}
if (i) {} if (i) {}
} }
// PGOUSE-DAG: ![[TH1]] = !{!"branch_weights", i32 101, i32 2} // PGOUSE-DAG: ![[TH1]] = !{!"branch_weights", i64 101, i64 2}
// PGOUSE-DAG: ![[TH2]] = !{!"branch_weights", i32 67, i32 35} // PGOUSE-DAG: ![[TH2]] = !{!"branch_weights", i64 67, i64 35}
// PGOUSE-DAG: ![[TH3]] = !{!"branch_weights", i32 34, i32 34} // PGOUSE-DAG: ![[TH3]] = !{!"branch_weights", i64 34, i64 34}
// PGOUSE-DAG: ![[TH4]] = !{!"branch_weights", i32 18, i32 18} // PGOUSE-DAG: ![[TH4]] = !{!"branch_weights", i64 18, i64 18}
// PGOUSE-EXC: ![[TH5]] = !{!"branch_weights", i32 34, i32 18} // PGOUSE-EXC: ![[TH5]] = !{!"branch_weights", i64 34, i64 18}
// PGOUSE-DAG: ![[TH6]] = !{!"branch_weights", i32 101, i32 1} // PGOUSE-DAG: ![[TH6]] = !{!"branch_weights", i64 101, i64 1}
// PGOUSE-DAG: ![[UN1]] = !{!"branch_weights", i32 2, i32 1} // PGOUSE-DAG: ![[UN1]] = !{!"branch_weights", i64 2, i64 1}
int main(int argc, const char *argv[]) { int main(int argc, const char *argv[]) {
throws(); throws();
try { try {
unreachable(1); unreachable(1);
} catch (int) {} } catch (int) {}
return 0; return 0;
} }

clang/test/Profile/gcc-flag-compatibility.c

Show All 26 Lines
// Check that -fprofile-use=some/path/file.prof reads some/path/file.prof // Check that -fprofile-use=some/path/file.prof reads some/path/file.prof
// This uses Clang FE format profile. // This uses Clang FE format profile.
// RUN: rm -rf %t.dir // RUN: rm -rf %t.dir
// RUN: mkdir -p %t.dir/some/path // RUN: mkdir -p %t.dir/some/path
// RUN: llvm-profdata merge %S/Inputs/gcc-flag-compatibility.proftext -o %t.dir/some/path/file.prof // RUN: llvm-profdata merge %S/Inputs/gcc-flag-compatibility.proftext -o %t.dir/some/path/file.prof
// RUN: %clang %s -o - -Xclang -disable-llvm-passes -emit-llvm -S -fprofile-use=%t.dir/some/path/file.prof -fno-experimental-new-pass-manager | FileCheck -check-prefix=PROFILE-USE %s // RUN: %clang %s -o - -Xclang -disable-llvm-passes -emit-llvm -S -fprofile-use=%t.dir/some/path/file.prof -fno-experimental-new-pass-manager | FileCheck -check-prefix=PROFILE-USE %s
// RUN: %clang %s -o - -Xclang -disable-llvm-passes -emit-llvm -S -fprofile-use=%t.dir/some/path/file.prof -fexperimental-new-pass-manager | FileCheck -check-prefix=PROFILE-USE %s // RUN: %clang %s -o - -Xclang -disable-llvm-passes -emit-llvm -S -fprofile-use=%t.dir/some/path/file.prof -fexperimental-new-pass-manager | FileCheck -check-prefix=PROFILE-USE %s
// PROFILE-USE: = !{!"branch_weights", i32 101, i32 2} // PROFILE-USE: = !{!"branch_weights", i64 101, i64 2}
// Check that -fprofile-use=some/path reads some/path/default.profdata // Check that -fprofile-use=some/path reads some/path/default.profdata
// This uses LLVM IR format profile. // This uses LLVM IR format profile.
// RUN: rm -rf %t.dir // RUN: rm -rf %t.dir
// RUN: mkdir -p %t.dir/some/path // RUN: mkdir -p %t.dir/some/path
// RUN: llvm-profdata merge %S/Inputs/gcc-flag-compatibility_IR.proftext -o %t.dir/some/path/default.profdata // RUN: llvm-profdata merge %S/Inputs/gcc-flag-compatibility_IR.proftext -o %t.dir/some/path/default.profdata
// RUN: %clang %s -o - -emit-llvm -S -fprofile-use=%t.dir/some/path -fno-experimental-new-pass-manager | FileCheck -check-prefix=PROFILE-USE-IR %s // RUN: %clang %s -o - -emit-llvm -S -fprofile-use=%t.dir/some/path -fno-experimental-new-pass-manager | FileCheck -check-prefix=PROFILE-USE-IR %s
// RUN: %clang %s -o - -emit-llvm -S -fprofile-use=%t.dir/some/path -fexperimental-new-pass-manager | FileCheck -check-prefix=PROFILE-USE-IR %s // RUN: %clang %s -o - -emit-llvm -S -fprofile-use=%t.dir/some/path -fexperimental-new-pass-manager | FileCheck -check-prefix=PROFILE-USE-IR %s
// Check that -fprofile-use=some/path/file.prof reads some/path/file.prof // Check that -fprofile-use=some/path/file.prof reads some/path/file.prof
// This uses LLVM IR format profile. // This uses LLVM IR format profile.
// RUN: rm -rf %t.dir // RUN: rm -rf %t.dir
// RUN: mkdir -p %t.dir/some/path // RUN: mkdir -p %t.dir/some/path
// RUN: llvm-profdata merge %S/Inputs/gcc-flag-compatibility_IR.proftext -o %t.dir/some/path/file.prof // RUN: llvm-profdata merge %S/Inputs/gcc-flag-compatibility_IR.proftext -o %t.dir/some/path/file.prof
// RUN: %clang %s -o - -emit-llvm -S -fprofile-use=%t.dir/some/path/file.prof -fno-experimental-new-pass-manager | FileCheck -check-prefix=PROFILE-USE-IR %s // RUN: %clang %s -o - -emit-llvm -S -fprofile-use=%t.dir/some/path/file.prof -fno-experimental-new-pass-manager | FileCheck -check-prefix=PROFILE-USE-IR %s
// RUN: %clang %s -o - -emit-llvm -S -fprofile-use=%t.dir/some/path/file.prof -fexperimental-new-pass-manager | FileCheck -check-prefix=PROFILE-USE-IR %s // RUN: %clang %s -o - -emit-llvm -S -fprofile-use=%t.dir/some/path/file.prof -fexperimental-new-pass-manager | FileCheck -check-prefix=PROFILE-USE-IR %s
// //
// RUN: llvm-profdata merge %S/Inputs/gcc-flag-compatibility_IR_entry.proftext -o %t.dir/some/path/file.prof // RUN: llvm-profdata merge %S/Inputs/gcc-flag-compatibility_IR_entry.proftext -o %t.dir/some/path/file.prof
// RUN: %clang %s -o - -emit-llvm -S -fprofile-use=%t.dir/some/path/file.prof -fno-experimental-new-pass-manager | FileCheck -check-prefix=PROFILE-USE-IR %s // RUN: %clang %s -o - -emit-llvm -S -fprofile-use=%t.dir/some/path/file.prof -fno-experimental-new-pass-manager | FileCheck -check-prefix=PROFILE-USE-IR %s
// RUN: %clang %s -o - -emit-llvm -S -fprofile-use=%t.dir/some/path/file.prof -fexperimental-new-pass-manager | FileCheck -check-prefix=PROFILE-USE-IR %s // RUN: %clang %s -o - -emit-llvm -S -fprofile-use=%t.dir/some/path/file.prof -fexperimental-new-pass-manager | FileCheck -check-prefix=PROFILE-USE-IR %s
// PROFILE-USE-IR: = !{!"branch_weights", i32 100, i32 1} // PROFILE-USE-IR: = !{!"branch_weights", i64 100, i64 1}
int X = 0; int X = 0;
int main() { int main() {
int i; int i;
for (i = 0; i < 100; i++) for (i = 0; i < 100; i++)
X += i; X += i;
return 0; return 0;
} }

clang/test/Profile/objc-general.m

Show First 20 Lines • Show All 73 Lines • ▼ Show 20 Lines
void consecutive_objc_for_ranges(NSArray *arr) { void consecutive_objc_for_ranges(NSArray *arr) {
int x = 0; int x = 0;
for (id a in arr) {} for (id a in arr) {}
for (id b in arr) for (id b in arr)
++x; ++x;
} }
// PGOUSE-DAG: ![[FR1]] = !{!"branch_weights", i32 2, i32 3} // PGOUSE-DAG: ![[FR1]] = !{!"branch_weights", i64 2, i64 3}
// PGOUSE-DAG: ![[FR2]] = !{!"branch_weights", i32 3, i32 2} // PGOUSE-DAG: ![[FR2]] = !{!"branch_weights", i64 3, i64 2}
// PGOUSE-DAG: ![[BL1]] = !{!"branch_weights", i32 2, i32 2} // PGOUSE-DAG: ![[BL1]] = !{!"branch_weights", i64 2, i64 2}
int main(int argc, const char *argv[]) { int main(int argc, const char *argv[]) {
A *a = [[A alloc] init]; A *a = [[A alloc] init];
NSArray *array = [NSArray arrayWithObjects: @"0", @"1", (void*)0]; NSArray *array = [NSArray arrayWithObjects: @"0", @"1", (void*)0];
[A foreach: array]; [A foreach: array];
return 0; return 0;
} }

llvm/include/llvm/IR/Instructions.h

Show First 20 Lines • Show All 3,464 Lines • ▼ Show 20 Lines static bool classof(const Value *V) {
return isa<Instruction>(V) && classof(cast<Instruction>(V)); return isa<Instruction>(V) && classof(cast<Instruction>(V));
} }
}; };
/// A wrapper class to simplify modification of SwitchInst cases along with /// A wrapper class to simplify modification of SwitchInst cases along with
/// their prof branch_weights metadata. /// their prof branch_weights metadata.
class SwitchInstProfUpdateWrapper { class SwitchInstProfUpdateWrapper {
SwitchInst &SI; SwitchInst &SI;
Optional<SmallVector<uint32_t, 8> > Weights = None; Optional<SmallVector<uint64_t, 8>> Weights = None;
bool Changed = false; bool Changed = false;
protected: protected:
static MDNode *getProfBranchWeightsMD(const SwitchInst &SI); static MDNode *getProfBranchWeightsMD(const SwitchInst &SI);
MDNode *buildProfBranchWeightsMD(); MDNode *buildProfBranchWeightsMD();
void init(); void init();
▲ Show 20 Lines • Show All 1,817 Lines • Show Last 20 Lines

llvm/include/llvm/IR/MDBuilder.h

Show First 20 Lines • Show All 52 Lines • ▼ Show 20 Linespublic:
/// setting. /// setting.
MDNode *createFPMath(float Accuracy); MDNode *createFPMath(float Accuracy);
//===------------------------------------------------------------------===// //===------------------------------------------------------------------===//
// Prof metadata. // Prof metadata.
//===------------------------------------------------------------------===// //===------------------------------------------------------------------===//
/// Return metadata containing two branch weights. /// Return metadata containing two branch weights.
MDNode *createBranchWeights(uint32_t TrueWeight, uint32_t FalseWeight); MDNode *createBranchWeights(uint64_t TrueWeight, uint64_t FalseWeight);
/// Return metadata containing a number of branch weights. /// Return metadata containing a number of branch weights.
MDNode *createBranchWeights(ArrayRef<uint32_t> Weights); MDNode *createBranchWeights(ArrayRef<uint64_t> Weights);
/// Return metadata specifying that a branch or switch is unpredictable. /// Return metadata specifying that a branch or switch is unpredictable.
MDNode *createUnpredictable(); MDNode *createUnpredictable();
/// Return metadata containing the entry \p Count for a function, a boolean /// Return metadata containing the entry \p Count for a function, a boolean
/// \Synthetic indicating whether the counts were synthetized, and the /// \Synthetic indicating whether the counts were synthetized, and the
/// GUIDs stored in \p Imports that need to be imported for sample PGO, to /// GUIDs stored in \p Imports that need to be imported for sample PGO, to
/// enable the same inlines as the profiled optimized binary /// enable the same inlines as the profiled optimized binary
▲ Show 20 Lines • Show All 146 Lines • Show Last 20 Lines

llvm/include/llvm/Transforms/Scalar/LowerExpectIntrinsic.h

Show All 26 Linesstruct LowerExpectIntrinsicPass : PassInfoMixin<LowerExpectIntrinsicPass> {
/// This will lower all of the expect intrinsic calls in this function into /// This will lower all of the expect intrinsic calls in this function into
/// branch weight metadata. That metadata will subsequently feed the analysis /// branch weight metadata. That metadata will subsequently feed the analysis
/// of the probabilities and frequencies of the CFG. After running this pass, /// of the probabilities and frequencies of the CFG. After running this pass,
/// no more expect intrinsics remain, allowing the rest of the optimizer to /// no more expect intrinsics remain, allowing the rest of the optimizer to
/// ignore them. /// ignore them.
PreservedAnalyses run(Function &F, FunctionAnalysisManager &); PreservedAnalyses run(Function &F, FunctionAnalysisManager &);
}; };
extern cl::opt<uint32_t> LikelyBranchWeight; extern cl::opt<uint64_t> LikelyBranchWeight;
extern cl::opt<uint32_t> UnlikelyBranchWeight; extern cl::opt<uint64_t> UnlikelyBranchWeight;
} }
#endif #endif

llvm/include/llvm/Transforms/Utils/MisExpect.h

Show All 22 Lines
/// verifyMisExpect - compares PGO counters to the thresholds used for /// verifyMisExpect - compares PGO counters to the thresholds used for
/// llvm.expect and warns if the PGO counters are outside of the expected /// llvm.expect and warns if the PGO counters are outside of the expected
/// range. /// range.
/// \param I The Instruction being checked /// \param I The Instruction being checked
/// \param Weights A vector of profile weights for each target block /// \param Weights A vector of profile weights for each target block
/// \param Ctx The current LLVM context /// \param Ctx The current LLVM context
void verifyMisExpect(llvm::Instruction *I, void verifyMisExpect(llvm::Instruction *I,
const llvm::SmallVector<uint32_t, 4> &Weights, const llvm::SmallVector<uint64_t, 4> &Weights,
llvm::LLVMContext &Ctx); llvm::LLVMContext &Ctx);
/// checkClangInstrumentation - verify if llvm.expect matches PGO profile /// checkClangInstrumentation - verify if llvm.expect matches PGO profile
/// This function checks the frontend instrumentation in the backend when /// This function checks the frontend instrumentation in the backend when
/// lowering llvm.expect intrinsics. It checks for existing metadata, and /// lowering llvm.expect intrinsics. It checks for existing metadata, and
/// then validates the use of llvm.expect against the assigned branch weights. /// then validates the use of llvm.expect against the assigned branch weights.
// //
/// \param I the Instruction being checked /// \param I the Instruction being checked
void checkFrontendInstrumentation(Instruction &I); void checkFrontendInstrumentation(Instruction &I);
} // namespace misexpect } // namespace misexpect
} // namespace llvm } // namespace llvm

llvm/lib/Analysis/BranchProbabilityInfo.cpp

Show First 20 Lines • Show All 452 Lines • ▼ Show 20 Linesbool BranchProbabilityInfo::calcMetadataWeights(const BasicBlock *BB) {
assert(TI->getNumSuccessors() < UINT32_MAX && "Too many successors"); assert(TI->getNumSuccessors() < UINT32_MAX && "Too many successors");
// Ensure there are weights for all of the successors. Note that the first // Ensure there are weights for all of the successors. Note that the first
// operand to the metadata node is a name, not a weight. // operand to the metadata node is a name, not a weight.
if (WeightsNode->getNumOperands() != TI->getNumSuccessors() + 1) if (WeightsNode->getNumOperands() != TI->getNumSuccessors() + 1)
return false; return false;
// Build up the final weights that will be used in a temporary buffer. // Build up the final weights that will be used in a temporary buffer.
// Compute the sum of all weights to later decide whether they need to
// be scaled to fit in 32 bits.
uint64_t WeightSum = 0; uint64_t WeightSum = 0;
SmallVector<uint32_t, 2> Weights; SmallVector<uint64_t, 2> Weights;
SmallVector<unsigned, 2> UnreachableIdxs; SmallVector<unsigned, 2> UnreachableIdxs;
SmallVector<unsigned, 2> ReachableIdxs; SmallVector<unsigned, 2> ReachableIdxs;
Weights.reserve(TI->getNumSuccessors()); Weights.reserve(TI->getNumSuccessors());
for (unsigned I = 1, E = WeightsNode->getNumOperands(); I != E; ++I) { for (unsigned I = 1, E = WeightsNode->getNumOperands(); I != E; ++I) {
ConstantInt *Weight = ConstantInt *Weight =
mdconst::dyn_extract<ConstantInt>(WeightsNode->getOperand(I)); mdconst::dyn_extract<ConstantInt>(WeightsNode->getOperand(I));
if (!Weight) if (!Weight)
return false; return false;
assert(Weight->getValue().getActiveBits() <= 32 &&
"Too many bits for uint32_t");
Weights.push_back(Weight->getZExtValue()); Weights.push_back(Weight->getZExtValue());
WeightSum += Weights.back(); WeightSum += Weights.back();
if (PostDominatedByUnreachable.count(TI->getSuccessor(I - 1))) if (PostDominatedByUnreachable.count(TI->getSuccessor(I - 1)))
UnreachableIdxs.push_back(I - 1); UnreachableIdxs.push_back(I - 1);
else else
ReachableIdxs.push_back(I - 1); ReachableIdxs.push_back(I - 1);
} }
assert(Weights.size() == TI->getNumSuccessors() && "Checked above"); assert(Weights.size() == TI->getNumSuccessors() && "Checked above");
Show All 17 Lines if (WeightSum == 0 || ReachableIdxs.size() == 0) {
for (unsigned I = 0, E = TI->getNumSuccessors(); I != E; ++I) for (unsigned I = 0, E = TI->getNumSuccessors(); I != E; ++I)
Weights[I] = 1; Weights[I] = 1;
WeightSum = TI->getNumSuccessors(); WeightSum = TI->getNumSuccessors();
} }
// Set the probability. // Set the probability.
SmallVector<BranchProbability, 2> BP; SmallVector<BranchProbability, 2> BP;
for (unsigned I = 0, E = TI->getNumSuccessors(); I != E; ++I) for (unsigned I = 0, E = TI->getNumSuccessors(); I != E; ++I)
BP.push_back({ Weights[I], static_cast<uint32_t>(WeightSum) }); BP.push_back(
BranchProbability::getBranchProbability(Weights[I], WeightSum));
// Examine the metadata against unreachable heuristic. // Examine the metadata against unreachable heuristic.
// If the unreachable heuristic is more strong then we use it for this edge. // If the unreachable heuristic is more strong then we use it for this edge.
if (UnreachableIdxs.size() == 0 || ReachableIdxs.size() == 0) { if (UnreachableIdxs.size() == 0 || ReachableIdxs.size() == 0) {
setEdgeProbability(BB, BP); setEdgeProbability(BB, BP);
return true; return true;
} }
▲ Show 20 Lines • Show All 771 Lines • Show Last 20 Lines

llvm/lib/IR/Instructions.cpp

Show First 20 Lines • Show All 4,054 Lines • ▼ Show 20 LinesMDNode *SwitchInstProfUpdateWrapper::buildProfBranchWeightsMD() {
if (!Weights) if (!Weights)
return nullptr; return nullptr;
assert(SI.getNumSuccessors() == Weights->size() && assert(SI.getNumSuccessors() == Weights->size() &&
"num of prof branch_weights must accord with num of successors"); "num of prof branch_weights must accord with num of successors");
bool AllZeroes = bool AllZeroes =
all_of(Weights.getValue(), [](uint32_t W) { return W == 0; }); all_of(Weights.getValue(), [](uint64_t W) { return W == 0; });
if (AllZeroes || Weights.getValue().size() < 2) if (AllZeroes || Weights.getValue().size() < 2)
return nullptr; return nullptr;
return MDBuilder(SI.getParent()->getContext()).createBranchWeights(*Weights); return MDBuilder(SI.getParent()->getContext()).createBranchWeights(*Weights);
} }
void SwitchInstProfUpdateWrapper::init() { void SwitchInstProfUpdateWrapper::init() {
MDNode *ProfileData = getProfBranchWeightsMD(SI); MDNode *ProfileData = getProfBranchWeightsMD(SI);
if (!ProfileData) if (!ProfileData)
return; return;
if (ProfileData->getNumOperands() != SI.getNumSuccessors() + 1) { if (ProfileData->getNumOperands() != SI.getNumSuccessors() + 1) {
llvm_unreachable("number of prof branch_weights metadata operands does " llvm_unreachable("number of prof branch_weights metadata operands does "
"not correspond to number of succesors"); "not correspond to number of succesors");
} }
SmallVector<uint32_t, 8> Weights; SmallVector<uint64_t, 8> Weights;
for (unsigned CI = 1, CE = SI.getNumSuccessors(); CI <= CE; ++CI) { for (unsigned CI = 1, CE = SI.getNumSuccessors(); CI <= CE; ++CI) {
ConstantInt *C = mdconst::extract<ConstantInt>(ProfileData->getOperand(CI)); ConstantInt *C = mdconst::extract<ConstantInt>(ProfileData->getOperand(CI));
uint32_t CW = C->getValue().getZExtValue(); uint64_t CW = C->getValue().getZExtValue();
Weights.push_back(CW); Weights.push_back(CW);
} }
this->Weights = std::move(Weights); this->Weights = std::move(Weights);
} }
SwitchInst::CaseIt SwitchInst::CaseIt
SwitchInstProfUpdateWrapper::removeCase(SwitchInst::CaseIt I) { SwitchInstProfUpdateWrapper::removeCase(SwitchInst::CaseIt I) {
if (Weights) { if (Weights) {
Show All 11 Lines
void SwitchInstProfUpdateWrapper::addCase( void SwitchInstProfUpdateWrapper::addCase(
ConstantInt *OnVal, BasicBlock *Dest, ConstantInt *OnVal, BasicBlock *Dest,
SwitchInstProfUpdateWrapper::CaseWeightOpt W) { SwitchInstProfUpdateWrapper::CaseWeightOpt W) {
SI.addCase(OnVal, Dest); SI.addCase(OnVal, Dest);
if (!Weights && W && *W) { if (!Weights && W && *W) {
Changed = true; Changed = true;
Weights = SmallVector<uint32_t, 8>(SI.getNumSuccessors(), 0); Weights = SmallVector<uint64_t, 8>(SI.getNumSuccessors(), 0);
Weights.getValue()[SI.getNumSuccessors() - 1] = *W; Weights.getValue()[SI.getNumSuccessors() - 1] = *W;
} else if (Weights) { } else if (Weights) {
Changed = true; Changed = true;
Weights.getValue().push_back(W ? *W : 0); Weights.getValue().push_back(W ? *W : 0);
} }
if (Weights) if (Weights)
assert(SI.getNumSuccessors() == Weights->size() && assert(SI.getNumSuccessors() == Weights->size() &&
"num of prof branch_weights must accord with num of successors"); "num of prof branch_weights must accord with num of successors");
Show All 16 Lines
} }
void SwitchInstProfUpdateWrapper::setSuccessorWeight( void SwitchInstProfUpdateWrapper::setSuccessorWeight(
unsigned idx, SwitchInstProfUpdateWrapper::CaseWeightOpt W) { unsigned idx, SwitchInstProfUpdateWrapper::CaseWeightOpt W) {
if (!W) if (!W)
return; return;
if (!Weights && *W) if (!Weights && *W)
Weights = SmallVector<uint32_t, 8>(SI.getNumSuccessors(), 0); Weights = SmallVector<uint64_t, 8>(SI.getNumSuccessors(), 0);
if (Weights) { if (Weights) {
auto &OldW = Weights.getValue()[idx]; auto &OldW = Weights.getValue()[idx];
if (*W != OldW) { if (*W != OldW) {
Changed = true; Changed = true;
OldW = *W; OldW = *W;
} }
} }
▲ Show 20 Lines • Show All 326 Lines • Show Last 20 Lines

llvm/lib/IR/MDBuilder.cpp

Show All 28 LinesMDNode *MDBuilder::createFPMath(float Accuracy) {
if (Accuracy == 0.0) if (Accuracy == 0.0)
return nullptr; return nullptr;
assert(Accuracy > 0.0 && "Invalid fpmath accuracy!"); assert(Accuracy > 0.0 && "Invalid fpmath accuracy!");
auto *Op = auto *Op =
createConstant(ConstantFP::get(Type::getFloatTy(Context), Accuracy)); createConstant(ConstantFP::get(Type::getFloatTy(Context), Accuracy));
return MDNode::get(Context, Op); return MDNode::get(Context, Op);
} }
MDNode *MDBuilder::createBranchWeights(uint32_t TrueWeight, MDNode *MDBuilder::createBranchWeights(uint64_t TrueWeight,
uint32_t FalseWeight) { uint64_t FalseWeight) {
return createBranchWeights({TrueWeight, FalseWeight}); return createBranchWeights({TrueWeight, FalseWeight});
} }
MDNode *MDBuilder::createBranchWeights(ArrayRef<uint32_t> Weights) { MDNode *MDBuilder::createBranchWeights(ArrayRef<uint64_t> Weights) {
assert(Weights.size() >= 1 && "Need at least one branch weights!"); assert(Weights.size() >= 1 && "Need at least one branch weights!");
SmallVector<Metadata *, 4> Vals(Weights.size() + 1); SmallVector<Metadata *, 4> Vals(Weights.size() + 1);
Vals[0] = createString("branch_weights"); Vals[0] = createString("branch_weights");
Type *Int32Ty = Type::getInt32Ty(Context); Type *Int64Ty = Type::getInt64Ty(Context);
for (unsigned i = 0, e = Weights.size(); i != e; ++i) for (unsigned i = 0, e = Weights.size(); i != e; ++i)
Vals[i + 1] = createConstant(ConstantInt::get(Int32Ty, Weights[i])); Vals[i + 1] = createConstant(ConstantInt::get(Int64Ty, Weights[i]));
return MDNode::get(Context, Vals); return MDNode::get(Context, Vals);
} }
MDNode *MDBuilder::createUnpredictable() { MDNode *MDBuilder::createUnpredictable() {
return MDNode::get(Context, None); return MDNode::get(Context, None);
} }
▲ Show 20 Lines • Show All 265 Lines • Show Last 20 Lines

llvm/lib/Transforms/IPO/SampleProfile.cpp

Show First 20 Lines • Show All 1,581 Lines • ▼ Show 20 Lines if (BlockWeights[BB]) {
GetSortedValueDataFromCallTargets(T.get()); GetSortedValueDataFromCallTargets(T.get());
uint64_t Sum; uint64_t Sum;
findIndirectCallFunctionSamples(I, Sum); findIndirectCallFunctionSamples(I, Sum);
annotateValueSite(*I.getParent()->getParent()->getParent(), I, annotateValueSite(*I.getParent()->getParent()->getParent(), I,
SortedCallTargets, Sum, IPVK_IndirectCallTarget, SortedCallTargets, Sum, IPVK_IndirectCallTarget,
SortedCallTargets.size()); SortedCallTargets.size());
} else if (!isa<IntrinsicInst>(&I)) { } else if (!isa<IntrinsicInst>(&I)) {
I.setMetadata(LLVMContext::MD_prof, I.setMetadata(LLVMContext::MD_prof,
MDB.createBranchWeights( MDB.createBranchWeights({BlockWeights[BB]}));
{static_cast<uint32_t>(BlockWeights[BB])}));
} }
} }
} }
Instruction *TI = BB->getTerminator(); Instruction *TI = BB->getTerminator();
if (TI->getNumSuccessors() == 1) if (TI->getNumSuccessors() == 1)
continue; continue;
if (!isa<BranchInst>(TI) && !isa<SwitchInst>(TI)) if (!isa<BranchInst>(TI) && !isa<SwitchInst>(TI))
continue; continue;
DebugLoc BranchLoc = TI->getDebugLoc(); DebugLoc BranchLoc = TI->getDebugLoc();
LLVM_DEBUG(dbgs() << "\nGetting weights for branch at line " LLVM_DEBUG(dbgs() << "\nGetting weights for branch at line "
<< ((BranchLoc) ? Twine(BranchLoc.getLine()) << ((BranchLoc) ? Twine(BranchLoc.getLine())
: Twine("<UNKNOWN LOCATION>")) : Twine("<UNKNOWN LOCATION>"))
<< ".\n"); << ".\n");
SmallVector<uint32_t, 4> Weights; SmallVector<uint64_t, 4> Weights;
uint32_t MaxWeight = 0; uint64_t MaxWeight = 0;
Instruction *MaxDestInst; Instruction *MaxDestInst;
for (unsigned I = 0; I < TI->getNumSuccessors(); ++I) { for (unsigned I = 0; I < TI->getNumSuccessors(); ++I) {
BasicBlock *Succ = TI->getSuccessor(I); BasicBlock *Succ = TI->getSuccessor(I);
Edge E = std::make_pair(BB, Succ); Edge E = std::make_pair(BB, Succ);
uint64_t Weight = EdgeWeights[E]; uint64_t Weight = EdgeWeights[E];
LLVM_DEBUG(dbgs() << "\t"; printEdgeWeight(dbgs(), E)); LLVM_DEBUG(dbgs() << "\t"; printEdgeWeight(dbgs(), E));
// Use uint32_t saturated arithmetic to adjust the incoming weights,
// if needed. Sample counts in profiles are 64-bit unsigned values,
// but internally branch weights are expressed as 32-bit values.
if (Weight > std::numeric_limits<uint32_t>::max()) {
LLVM_DEBUG(dbgs() << " (saturated due to uint32_t overflow)");
Weight = std::numeric_limits<uint32_t>::max();
}
// Weight is added by one to avoid propagation errors introduced by // Weight is added by one to avoid propagation errors introduced by
// 0 weights. // 0 weights.
Weights.push_back(static_cast<uint32_t>(Weight + 1)); Weights.push_back(Weight + 1);
if (Weight != 0) { if (Weight != 0) {
if (Weight > MaxWeight) { if (Weight > MaxWeight) {
MaxWeight = Weight; MaxWeight = Weight;
MaxDestInst = Succ->getFirstNonPHIOrDbgOrLifetime(); MaxDestInst = Succ->getFirstNonPHIOrDbgOrLifetime();
} }
} }
} }
▲ Show 20 Lines • Show All 424 Lines • Show Last 20 Lines

llvm/lib/Transforms/Instrumentation/ControlHeightReduction.cpp

Show First 20 Lines • Show All 1,859 Lines • ▼ Show 20 Lines ORE.emit([&]() {
return OptimizationRemark(DEBUG_TYPE, return OptimizationRemark(DEBUG_TYPE,
"CHR", "CHR",
// Refer to the hot (original) path // Refer to the hot (original) path
MergedBR->getSuccessor(0)->getTerminator()) MergedBR->getSuccessor(0)->getTerminator())
<< "Merged " << ore::NV("NumCHRedBranches", NumCHRedBranches) << "Merged " << ore::NV("NumCHRedBranches", NumCHRedBranches)
<< " branches or selects"; << " branches or selects";
}); });
MergedBR->setCondition(MergedCondition); MergedBR->setCondition(MergedCondition);
uint32_t Weights[] = { uint64_t Weights[] = {
static_cast<uint32_t>(CHRBranchBias.scale(1000)), CHRBranchBias.scale(1000),
static_cast<uint32_t>(CHRBranchBias.getCompl().scale(1000)), CHRBranchBias.getCompl().scale(1000),
}; };
MDBuilder MDB(F.getContext()); MDBuilder MDB(F.getContext());
MergedBR->setMetadata(LLVMContext::MD_prof, MDB.createBranchWeights(Weights)); MergedBR->setMetadata(LLVMContext::MD_prof, MDB.createBranchWeights(Weights));
CHR_DEBUG(dbgs() << "CHR branch bias " << Weights[0] << ":" << Weights[1] CHR_DEBUG(dbgs() << "CHR branch bias " << Weights[0] << ":" << Weights[1]
<< "\n"); << "\n");
} }
// A helper for fixupBranchesAndSelects. Add to the combined branch condition // A helper for fixupBranchesAndSelects. Add to the combined branch condition
▲ Show 20 Lines • Show All 229 Lines • Show Last 20 Lines

llvm/lib/Transforms/Instrumentation/PGOInstrumentation.cpp

Show First 20 Lines • Show All 110 Lines • ▼ Show 20 Lines
#include "llvm/Transforms/Instrumentation.h" #include "llvm/Transforms/Instrumentation.h"
#include "llvm/Transforms/Utils/BasicBlockUtils.h" #include "llvm/Transforms/Utils/BasicBlockUtils.h"
#include "llvm/Transforms/Utils/MisExpect.h" #include "llvm/Transforms/Utils/MisExpect.h"
#include <algorithm> #include <algorithm>
#include <cassert> #include <cassert>
#include <cstdint> #include <cstdint>
#include <memory> #include <memory>
#include <numeric> #include <numeric>
#include <stdint.h>
#include <string> #include <string>
#include <unordered_map> #include <unordered_map>
#include <utility> #include <utility>
#include <vector> #include <vector>
using namespace llvm; using namespace llvm;
using ProfileCount = Function::ProfileCount; using ProfileCount = Function::ProfileCount;
using VPCandidateInfo = ValueProfileCollector::CandidateInfo; using VPCandidateInfo = ValueProfileCollector::CandidateInfo;
▲ Show 20 Lines • Show All 1,698 Lines • ▼ Show 20 Lines
} }
void llvm::setProfMetadata(Module *M, Instruction *TI, void llvm::setProfMetadata(Module *M, Instruction *TI,
ArrayRef<uint64_t> EdgeCounts, ArrayRef<uint64_t> EdgeCounts,
uint64_t MaxCount) { uint64_t MaxCount) {
MDBuilder MDB(M->getContext()); MDBuilder MDB(M->getContext());
assert(MaxCount > 0 && "Bad max count"); assert(MaxCount > 0 && "Bad max count");
uint64_t Scale = calculateCountScale(MaxCount); uint64_t Scale = calculateCountScale(MaxCount);
SmallVector<unsigned, 4> Weights; SmallVector<uint64_t, 4> Weights;
for (const auto &ECI : EdgeCounts) for (const auto &ECI : EdgeCounts)
Weights.push_back(scaleBranchCount(ECI, Scale)); Weights.push_back(scaleBranchCount(ECI, Scale));
LLVM_DEBUG(dbgs() << "Weight is: "; for (const auto &W LLVM_DEBUG(dbgs() << "Weight is: "; for (const auto &W
: Weights) { : Weights) {
dbgs() << W << " "; dbgs() << W << " ";
} dbgs() << "\n";); } dbgs() << "\n";);
▲ Show 20 Lines • Show All 104 Lines • Show Last 20 Lines

llvm/lib/Transforms/Scalar/JumpThreading.cpp

Show First 20 Lines • Show All 287 Lines • ▼ Show 20 Lines for (unsigned i = 0, e = PN->getNumIncomingValues(); i != e; ++i) {
if (PredBr->extractProfMetadata(PredTrueWeight, PredFalseWeight)) if (PredBr->extractProfMetadata(PredTrueWeight, PredFalseWeight))
continue; continue;
// We can not infer anything useful when BP >= 50%, because BP is the // We can not infer anything useful when BP >= 50%, because BP is the
// upper bound probability value. // upper bound probability value.
if (BP >= BranchProbability(50, 100)) if (BP >= BranchProbability(50, 100))
continue; continue;
SmallVector<uint32_t, 2> Weights; SmallVector<uint64_t, 2> Weights;
if (PredBr->getSuccessor(0) == PredOutEdge.second) { if (PredBr->getSuccessor(0) == PredOutEdge.second) {
Weights.push_back(BP.getNumerator()); Weights.push_back(BP.getNumerator());
Weights.push_back(BP.getCompl().getNumerator()); Weights.push_back(BP.getCompl().getNumerator());
} else { } else {
Weights.push_back(BP.getCompl().getNumerator()); Weights.push_back(BP.getCompl().getNumerator());
Weights.push_back(BP.getNumerator()); Weights.push_back(BP.getNumerator());
} }
PredBr->setMetadata(LLVMContext::MD_prof, PredBr->setMetadata(LLVMContext::MD_prof,
▲ Show 20 Lines • Show All 2,223 Lines • ▼ Show 20 Linesvoid JumpThreadingPass::UpdateBlockFreqAndEdgeWeight(BasicBlock *PredBB,
// becomes 0%. This is even worse if the edge whose probability becomes 0% is // becomes 0%. This is even worse if the edge whose probability becomes 0% is
// the loop exit edge. Then based solely on static estimation we would assume // the loop exit edge. Then based solely on static estimation we would assume
// the loop was extremely hot. // the loop was extremely hot.
// //
// FIXME this locally as well so that BPI and BFI are consistent as well. We // FIXME this locally as well so that BPI and BFI are consistent as well. We
// shouldn't make edges extremely likely or unlikely based solely on static // shouldn't make edges extremely likely or unlikely based solely on static
// estimation. // estimation.
if (BBSuccProbs.size() >= 2 && doesBlockHaveProfileData(BB)) { if (BBSuccProbs.size() >= 2 && doesBlockHaveProfileData(BB)) {
SmallVector<uint32_t, 4> Weights; SmallVector<uint64_t, 4> Weights;
for (auto Prob : BBSuccProbs) for (auto Prob : BBSuccProbs)
Weights.push_back(Prob.getNumerator()); Weights.push_back(Prob.getNumerator());
auto TI = BB->getTerminator(); auto TI = BB->getTerminator();
TI->setMetadata( TI->setMetadata(
LLVMContext::MD_prof, LLVMContext::MD_prof,
MDBuilder(TI->getParent()->getContext()).createBranchWeights(Weights)); MDBuilder(TI->getParent()->getContext()).createBranchWeights(Weights));
} }
▲ Show 20 Lines • Show All 471 Lines • Show Last 20 Lines

llvm/lib/Transforms/Scalar/LowerExpectIntrinsic.cpp

Show First 20 Lines • Show All 41 Lines • ▼ Show 20 Lines
// only be used in extreme cases, we could make this ratio higher. As it stands, // only be used in extreme cases, we could make this ratio higher. As it stands,
// programmers may be using __builtin_expect() / llvm.expect to annotate that a // programmers may be using __builtin_expect() / llvm.expect to annotate that a
// branch is likely or unlikely to be taken. // branch is likely or unlikely to be taken.
// //
// There is a known dependency on this ratio in CodeGenPrepare when transforming // There is a known dependency on this ratio in CodeGenPrepare when transforming
// 'select' instructions. It may be worthwhile to hoist these values to some // 'select' instructions. It may be worthwhile to hoist these values to some
// shared space, so they can be used directly by other passes. // shared space, so they can be used directly by other passes.
cl::opt<uint32_t> llvm::LikelyBranchWeight( cl::opt<uint64_t> llvm::LikelyBranchWeight(
"likely-branch-weight", cl::Hidden, cl::init(2000), "likely-branch-weight", cl::Hidden, cl::init(2000),
cl::desc("Weight of the branch likely to be taken (default = 2000)")); cl::desc("Weight of the branch likely to be taken (default = 2000)"));
cl::opt<uint32_t> llvm::UnlikelyBranchWeight( cl::opt<uint64_t> llvm::UnlikelyBranchWeight(
"unlikely-branch-weight", cl::Hidden, cl::init(1), "unlikely-branch-weight", cl::Hidden, cl::init(1),
cl::desc("Weight of the branch unlikely to be taken (default = 1)")); cl::desc("Weight of the branch unlikely to be taken (default = 1)"));
static std::tuple<uint32_t, uint32_t> static std::tuple<uint64_t, uint64_t>
getBranchWeight(Intrinsic::ID IntrinsicID, CallInst *CI, int BranchCount) { getBranchWeight(Intrinsic::ID IntrinsicID, CallInst *CI, int BranchCount) {
if (IntrinsicID == Intrinsic::expect) { if (IntrinsicID == Intrinsic::expect) {
// __builtin_expect // __builtin_expect
return std::make_tuple(LikelyBranchWeight.getValue(), return std::make_tuple(LikelyBranchWeight.getValue(),
UnlikelyBranchWeight.getValue()); UnlikelyBranchWeight.getValue());
} else { } else {
// __builtin_expect_with_probability // __builtin_expect_with_probability
assert(CI->getNumOperands() >= 3 && assert(CI->getNumOperands() >= 3 &&
"expect with probability must have 3 arguments"); "expect with probability must have 3 arguments");
ConstantFP *Confidence = dyn_cast<ConstantFP>(CI->getArgOperand(2)); ConstantFP *Confidence = dyn_cast<ConstantFP>(CI->getArgOperand(2));
double TrueProb = Confidence->getValueAPF().convertToDouble(); double TrueProb = Confidence->getValueAPF().convertToDouble();
assert((TrueProb >= 0.0 && TrueProb <= 1.0) && assert((TrueProb >= 0.0 && TrueProb <= 1.0) &&
"probability value must be in the range [0.0, 1.0]"); "probability value must be in the range [0.0, 1.0]");
double FalseProb = (1.0 - TrueProb) / (BranchCount - 1); double FalseProb = (1.0 - TrueProb) / (BranchCount - 1);
uint32_t LikelyBW = ceil((TrueProb * (double)(INT32_MAX - 1)) + 1.0); uint64_t LikelyBW = ceil((TrueProb * (double)(INT32_MAX - 1)) + 1.0);
uint32_t UnlikelyBW = ceil((FalseProb * (double)(INT32_MAX - 1)) + 1.0); uint64_t UnlikelyBW = ceil((FalseProb * (double)(INT32_MAX - 1)) + 1.0);
return std::make_tuple(LikelyBW, UnlikelyBW); return std::make_tuple(LikelyBW, UnlikelyBW);
} }
} }
static bool handleSwitchExpect(SwitchInst &SI) { static bool handleSwitchExpect(SwitchInst &SI) {
CallInst *CI = dyn_cast<CallInst>(SI.getCondition()); CallInst *CI = dyn_cast<CallInst>(SI.getCondition());
if (!CI) if (!CI)
return false; return false;
Function *Fn = CI->getCalledFunction(); Function *Fn = CI->getCalledFunction();
if (!Fn || (Fn->getIntrinsicID() != Intrinsic::expect && if (!Fn || (Fn->getIntrinsicID() != Intrinsic::expect &&
Fn->getIntrinsicID() != Intrinsic::expect_with_probability)) Fn->getIntrinsicID() != Intrinsic::expect_with_probability))
return false; return false;
Value *ArgValue = CI->getArgOperand(0); Value *ArgValue = CI->getArgOperand(0);
ConstantInt *ExpectedValue = dyn_cast<ConstantInt>(CI->getArgOperand(1)); ConstantInt *ExpectedValue = dyn_cast<ConstantInt>(CI->getArgOperand(1));
if (!ExpectedValue) if (!ExpectedValue)
return false; return false;
SwitchInst::CaseHandle Case = *SI.findCaseValue(ExpectedValue); SwitchInst::CaseHandle Case = *SI.findCaseValue(ExpectedValue);
unsigned n = SI.getNumCases(); // +1 for default case. unsigned n = SI.getNumCases(); // +1 for default case.
uint32_t LikelyBranchWeightVal, UnlikelyBranchWeightVal; uint64_t LikelyBranchWeightVal, UnlikelyBranchWeightVal;
std::tie(LikelyBranchWeightVal, UnlikelyBranchWeightVal) = std::tie(LikelyBranchWeightVal, UnlikelyBranchWeightVal) =
getBranchWeight(Fn->getIntrinsicID(), CI, n + 1); getBranchWeight(Fn->getIntrinsicID(), CI, n + 1);
SmallVector<uint32_t, 16> Weights(n + 1, UnlikelyBranchWeightVal); SmallVector<uint64_t, 16> Weights(n + 1, UnlikelyBranchWeightVal);
uint64_t Index = (Case == *SI.case_default()) ? 0 : Case.getCaseIndex() + 1; uint64_t Index = (Case == *SI.case_default()) ? 0 : Case.getCaseIndex() + 1;
Weights[Index] = LikelyBranchWeightVal; Weights[Index] = LikelyBranchWeightVal;
SI.setMetadata(LLVMContext::MD_misexpect, SI.setMetadata(LLVMContext::MD_misexpect,
MDBuilder(CI->getContext()) MDBuilder(CI->getContext())
.createMisExpect(Index, LikelyBranchWeightVal, .createMisExpect(Index, LikelyBranchWeightVal,
UnlikelyBranchWeightVal)); UnlikelyBranchWeightVal));
▲ Show 20 Lines • Show All 135 Lines • ▼ Show 20 Lines auto IsOpndComingFromSuccessor = [&](BasicBlock *Succ) {
return true; return true;
if (OpndIncomingBB == BI->getParent() && Succ == PhiDef->getParent()) if (OpndIncomingBB == BI->getParent() && Succ == PhiDef->getParent())
// Otherwise, if the edge is directly from the branch // Otherwise, if the edge is directly from the branch
// to the Phi, this successor is the one feeding this // to the Phi, this successor is the one feeding this
// Phi operand. // Phi operand.
return true; return true;
return false; return false;
}; };
uint32_t LikelyBranchWeightVal, UnlikelyBranchWeightVal; uint64_t LikelyBranchWeightVal, UnlikelyBranchWeightVal;
std::tie(LikelyBranchWeightVal, UnlikelyBranchWeightVal) = getBranchWeight( std::tie(LikelyBranchWeightVal, UnlikelyBranchWeightVal) = getBranchWeight(
Expect->getCalledFunction()->getIntrinsicID(), Expect, 2); Expect->getCalledFunction()->getIntrinsicID(), Expect, 2);
if (IsOpndComingFromSuccessor(BI->getSuccessor(1))) if (IsOpndComingFromSuccessor(BI->getSuccessor(1)))
BI->setMetadata(LLVMContext::MD_prof, BI->setMetadata(LLVMContext::MD_prof,
MDB.createBranchWeights(LikelyBranchWeightVal, MDB.createBranchWeights(LikelyBranchWeightVal,
UnlikelyBranchWeightVal)); UnlikelyBranchWeightVal));
else if (IsOpndComingFromSuccessor(BI->getSuccessor(0))) else if (IsOpndComingFromSuccessor(BI->getSuccessor(0)))
▲ Show 20 Lines • Show All 53 Lines • ▼ Show 20 Linestemplate <class BrSelInst> static bool handleBrSelExpect(BrSelInst &BSI) {
ConstantInt *ExpectedValue = dyn_cast<ConstantInt>(CI->getArgOperand(1)); ConstantInt *ExpectedValue = dyn_cast<ConstantInt>(CI->getArgOperand(1));
if (!ExpectedValue) if (!ExpectedValue)
return false; return false;
MDBuilder MDB(CI->getContext()); MDBuilder MDB(CI->getContext());
MDNode *Node; MDNode *Node;
MDNode *ExpNode; MDNode *ExpNode;
uint32_t LikelyBranchWeightVal, UnlikelyBranchWeightVal; uint64_t LikelyBranchWeightVal, UnlikelyBranchWeightVal;
std::tie(LikelyBranchWeightVal, UnlikelyBranchWeightVal) = std::tie(LikelyBranchWeightVal, UnlikelyBranchWeightVal) =
getBranchWeight(Fn->getIntrinsicID(), CI, 2); getBranchWeight(Fn->getIntrinsicID(), CI, 2);
if ((ExpectedValue->getZExtValue() == ValueComparedTo) == if ((ExpectedValue->getZExtValue() == ValueComparedTo) ==
(Predicate == CmpInst::ICMP_EQ)) { (Predicate == CmpInst::ICMP_EQ)) {
Node = Node =
MDB.createBranchWeights(LikelyBranchWeightVal, UnlikelyBranchWeightVal); MDB.createBranchWeights(LikelyBranchWeightVal, UnlikelyBranchWeightVal);
ExpNode = ExpNode =
▲ Show 20 Lines • Show All 107 Lines • Show Last 20 Lines

llvm/lib/Transforms/Utils/CodeExtractor.cpp

Show First 20 Lines • Show All 61 Lines • ▼ Show 20 Lines
#include "llvm/Support/raw_ostream.h" #include "llvm/Support/raw_ostream.h"
#include "llvm/Transforms/Utils/BasicBlockUtils.h" #include "llvm/Transforms/Utils/BasicBlockUtils.h"
#include "llvm/Transforms/Utils/Local.h" #include "llvm/Transforms/Utils/Local.h"
#include <cassert> #include <cassert>
#include <cstdint> #include <cstdint>
#include <iterator> #include <iterator>
#include <map> #include <map>
#include <set> #include <set>
#include <stdint.h>
#include <utility> #include <utility>
#include <vector> #include <vector>
using namespace llvm; using namespace llvm;
using namespace llvm::PatternMatch; using namespace llvm::PatternMatch;
using ProfileCount = Function::ProfileCount; using ProfileCount = Function::ProfileCount;
#define DEBUG_TYPE "code-extractor" #define DEBUG_TYPE "code-extractor"
▲ Show 20 Lines • Show All 1,280 Lines • ▼ Show 20 Linesvoid CodeExtractor::calculateNewCallTerminatorWeights(
BasicBlock *CodeReplacer, BasicBlock *CodeReplacer,
DenseMap<BasicBlock *, BlockFrequency> &ExitWeights, DenseMap<BasicBlock *, BlockFrequency> &ExitWeights,
BranchProbabilityInfo *BPI) { BranchProbabilityInfo *BPI) {
using Distribution = BlockFrequencyInfoImplBase::Distribution; using Distribution = BlockFrequencyInfoImplBase::Distribution;
using BlockNode = BlockFrequencyInfoImplBase::BlockNode; using BlockNode = BlockFrequencyInfoImplBase::BlockNode;
// Update the branch weights for the exit block. // Update the branch weights for the exit block.
Instruction *TI = CodeReplacer->getTerminator(); Instruction *TI = CodeReplacer->getTerminator();
SmallVector<unsigned, 8> BranchWeights(TI->getNumSuccessors(), 0); SmallVector<uint64_t, 8> BranchWeights(TI->getNumSuccessors(), 0);
// Block Frequency distribution with dummy node. // Block Frequency distribution with dummy node.
Distribution BranchDist; Distribution BranchDist;
SmallVector<BranchProbability, 4> EdgeProbabilities( SmallVector<BranchProbability, 4> EdgeProbabilities(
TI->getNumSuccessors(), BranchProbability::getUnknown()); TI->getNumSuccessors(), BranchProbability::getUnknown());
// Add each of the frequencies of the successors. // Add each of the frequencies of the successors.
▲ Show 20 Lines • Show All 394 Lines • Show Last 20 Lines

llvm/lib/Transforms/Utils/Local.cpp

Show First 20 Lines • Show All 201 Lines • ▼ Show 20 Lines for (auto i = SI->case_begin(), e = SI->case_end(); i != e;) {
// dest. If so, eliminate it as an explicit compare. // dest. If so, eliminate it as an explicit compare.
if (i->getCaseSuccessor() == DefaultDest) { if (i->getCaseSuccessor() == DefaultDest) {
MDNode *MD = SI->getMetadata(LLVMContext::MD_prof); MDNode *MD = SI->getMetadata(LLVMContext::MD_prof);
unsigned NCases = SI->getNumCases(); unsigned NCases = SI->getNumCases();
// Fold the case metadata into the default if there will be any branches // Fold the case metadata into the default if there will be any branches
// left, unless the metadata doesn't match the switch. // left, unless the metadata doesn't match the switch.
if (NCases > 1 && MD && MD->getNumOperands() == 2 + NCases) { if (NCases > 1 && MD && MD->getNumOperands() == 2 + NCases) {
// Collect branch weights into a vector. // Collect branch weights into a vector.
SmallVector<uint32_t, 8> Weights; SmallVector<uint64_t, 8> Weights;
for (unsigned MD_i = 1, MD_e = MD->getNumOperands(); MD_i < MD_e; for (unsigned MD_i = 1, MD_e = MD->getNumOperands(); MD_i < MD_e;
++MD_i) { ++MD_i) {
auto *CI = mdconst::extract<ConstantInt>(MD->getOperand(MD_i)); auto *CI = mdconst::extract<ConstantInt>(MD->getOperand(MD_i));
Weights.push_back(CI->getValue().getZExtValue()); Weights.push_back(CI->getValue().getZExtValue());
} }
// Merge weight of this case to the default weight. // Merge weight of this case to the default weight.
unsigned idx = i->getCaseIndex(); unsigned idx = i->getCaseIndex();
Weights[0] += Weights[idx+1]; Weights[0] += Weights[idx+1];
▲ Show 20 Lines • Show All 1,867 Lines • ▼ Show 20 LinesCallInst *llvm::createCallMatchingInvoke(InvokeInst *II) {
NewCall->setCallingConv(II->getCallingConv()); NewCall->setCallingConv(II->getCallingConv());
NewCall->setAttributes(II->getAttributes()); NewCall->setAttributes(II->getAttributes());
NewCall->setDebugLoc(II->getDebugLoc()); NewCall->setDebugLoc(II->getDebugLoc());
NewCall->copyMetadata(*II); NewCall->copyMetadata(*II);
// If the invoke had profile metadata, try converting them for CallInst. // If the invoke had profile metadata, try converting them for CallInst.
uint64_t TotalWeight; uint64_t TotalWeight;
if (NewCall->extractProfTotalWeight(TotalWeight)) { if (NewCall->extractProfTotalWeight(TotalWeight)) {
// Set the total weight if it fits into i32, otherwise reset.
MDBuilder MDB(NewCall->getContext()); MDBuilder MDB(NewCall->getContext());
auto NewWeights = uint32_t(TotalWeight) != TotalWeight auto NewWeights = MDB.createBranchWeights({TotalWeight});
Lint: Pre-merge checks
Inline Actions

clang-tidy: warning: 'auto NewWeights' can be declared as 'auto *NewWeights' [llvm-qualified-auto]
not useful

Lint: Pre-merge checks: clang-tidy: warning: 'auto NewWeights' can be declared as 'auto *NewWeights' [llvm-qualified…
? nullptr
: MDB.createBranchWeights({uint32_t(TotalWeight)});
NewCall->setMetadata(LLVMContext::MD_prof, NewWeights); NewCall->setMetadata(LLVMContext::MD_prof, NewWeights);
} }
return NewCall; return NewCall;
} }
/// changeToCall - Convert the specified invoke into a normal call. /// changeToCall - Convert the specified invoke into a normal call.
void llvm::changeToCall(InvokeInst *II, DomTreeUpdater *DTU) { void llvm::changeToCall(InvokeInst *II, DomTreeUpdater *DTU) {
▲ Show 20 Lines • Show All 1,129 Lines • Show Last 20 Lines

llvm/lib/Transforms/Utils/MisExpect.cpp

Show First 20 Lines • Show All 87 Lines • ▼ Show 20 Linesvoid emitMisexpectDiagnostic(Instruction *I, LLVMContext &Ctx,
ORE.emit(OptimizationRemark(DEBUG_TYPE, "misexpect", Cond) << RemStr.str()); ORE.emit(OptimizationRemark(DEBUG_TYPE, "misexpect", Cond) << RemStr.str());
} }
} // namespace } // namespace
namespace llvm { namespace llvm {
namespace misexpect { namespace misexpect {
void verifyMisExpect(Instruction *I, const SmallVector<uint32_t, 4> &Weights, void verifyMisExpect(Instruction *I, const SmallVector<uint64_t, 4> &Weights,
LLVMContext &Ctx) { LLVMContext &Ctx) {
if (auto *MisExpectData = I->getMetadata(LLVMContext::MD_misexpect)) { if (auto *MisExpectData = I->getMetadata(LLVMContext::MD_misexpect)) {
auto *MisExpectDataName = dyn_cast<MDString>(MisExpectData->getOperand(0)); auto *MisExpectDataName = dyn_cast<MDString>(MisExpectData->getOperand(0));
if (MisExpectDataName && if (MisExpectDataName &&
MisExpectDataName->getString().equals("misexpect")) { MisExpectDataName->getString().equals("misexpect")) {
LLVM_DEBUG(llvm::dbgs() << "------------------\n"); LLVM_DEBUG(llvm::dbgs() << "------------------\n");
LLVM_DEBUG(llvm::dbgs() LLVM_DEBUG(llvm::dbgs()
<< "Function: " << I->getFunction()->getName() << "\n"); << "Function: " << I->getFunction()->getName() << "\n");
▲ Show 20 Lines • Show All 51 Lines • ▼ Show 20 Lines if (auto *MD = I.getMetadata(LLVMContext::MD_prof)) {
// 3) completely deterministic // 3) completely deterministic
// In these cases we should not emit any diagnostic related to misexpect. // In these cases we should not emit any diagnostic related to misexpect.
if (NOps < 3) if (NOps < 3)
return; return;
// Operand 0 is a string tag "branch_weights" // Operand 0 is a string tag "branch_weights"
if (MDString *Tag = cast<MDString>(MD->getOperand(0))) { if (MDString *Tag = cast<MDString>(MD->getOperand(0))) {
if (Tag->getString().equals("branch_weights")) { if (Tag->getString().equals("branch_weights")) {
SmallVector<uint32_t, 4> RealWeights(NOps - 1); SmallVector<uint64_t, 4> RealWeights(NOps - 1);
for (unsigned i = 1; i < NOps; i++) { for (unsigned i = 1; i < NOps; i++) {
ConstantInt *Value = ConstantInt *Value =
mdconst::dyn_extract<ConstantInt>(MD->getOperand(i)); mdconst::dyn_extract<ConstantInt>(MD->getOperand(i));
RealWeights[i - 1] = Value->getZExtValue(); RealWeights[i - 1] = Value->getZExtValue();
} }
verifyMisExpect(&I, RealWeights, I.getContext()); verifyMisExpect(&I, RealWeights, I.getContext());
} }
} }
} }
} }
} // namespace misexpect } // namespace misexpect
} // namespace llvm } // namespace llvm
#undef DEBUG_TYPE #undef DEBUG_TYPE

llvm/lib/Transforms/Utils/SimplifyCFG.cpp

Show First 20 Lines • Show All 232 Lines • ▼ Show 20 Linesclass SimplifyCFGOpt {
bool tryToSimplifyUncondBranchWithICmpInIt(ICmpInst *ICI, bool tryToSimplifyUncondBranchWithICmpInIt(ICmpInst *ICI,
IRBuilder<> &Builder); IRBuilder<> &Builder);
bool HoistThenElseCodeToIf(BranchInst *BI, const TargetTransformInfo &TTI); bool HoistThenElseCodeToIf(BranchInst *BI, const TargetTransformInfo &TTI);
bool SpeculativelyExecuteBB(BranchInst *BI, BasicBlock *ThenBB, bool SpeculativelyExecuteBB(BranchInst *BI, BasicBlock *ThenBB,
const TargetTransformInfo &TTI); const TargetTransformInfo &TTI);
bool SimplifyTerminatorOnSelect(Instruction *OldTerm, Value *Cond, bool SimplifyTerminatorOnSelect(Instruction *OldTerm, Value *Cond,
BasicBlock *TrueBB, BasicBlock *FalseBB, BasicBlock *TrueBB, BasicBlock *FalseBB,
uint32_t TrueWeight, uint32_t FalseWeight); uint64_t TrueWeight, uint64_t FalseWeight);
bool SimplifyBranchOnICmpChain(BranchInst *BI, IRBuilder<> &Builder, bool SimplifyBranchOnICmpChain(BranchInst *BI, IRBuilder<> &Builder,
const DataLayout &DL); const DataLayout &DL);
bool SimplifySwitchOnSelect(SwitchInst *SI, SelectInst *Select); bool SimplifySwitchOnSelect(SwitchInst *SI, SelectInst *Select);
bool SimplifyIndirectBrOnSelect(IndirectBrInst *IBI, SelectInst *SI); bool SimplifyIndirectBrOnSelect(IndirectBrInst *IBI, SelectInst *SI);
bool TurnSwitchRangeIntoICmp(SwitchInst *SI, IRBuilder<> &Builder); bool TurnSwitchRangeIntoICmp(SwitchInst *SI, IRBuilder<> &Builder);
public: public:
SimplifyCFGOpt(const TargetTransformInfo &TTI, const DataLayout &DL, SimplifyCFGOpt(const TargetTransformInfo &TTI, const DataLayout &DL,
▲ Show 20 Lines • Show All 570 Lines • ▼ Show 20 Lines while (i1 != e1 && i2 != e2) {
else else
++i2; ++i2;
} }
return false; return false;
} }
// Set branch weights on SwitchInst. This sets the metadata if there is at // Set branch weights on SwitchInst. This sets the metadata if there is at
// least one non-zero weight. // least one non-zero weight.
static void setBranchWeights(SwitchInst *SI, ArrayRef<uint32_t> Weights) { static void setBranchWeights(SwitchInst *SI, ArrayRef<uint64_t> Weights) {
// Check that there is at least one non-zero weight. Otherwise, pass // Check that there is at least one non-zero weight. Otherwise, pass
// nullptr to setMetadata which will erase the existing metadata. // nullptr to setMetadata which will erase the existing metadata.
MDNode *N = nullptr; MDNode *N = nullptr;
if (llvm::any_of(Weights, [](uint32_t W) { return W != 0; })) if (llvm::any_of(Weights, [](uint64_t W) { return W != 0; }))
N = MDBuilder(SI->getParent()->getContext()).createBranchWeights(Weights); N = MDBuilder(SI->getParent()->getContext()).createBranchWeights(Weights);
SI->setMetadata(LLVMContext::MD_prof, N); SI->setMetadata(LLVMContext::MD_prof, N);
} }
// Similar to the above, but for branch and select instructions that take // Similar to the above, but for branch and select instructions that take
// exactly 2 weights. // exactly 2 weights.
static void setBranchWeights(Instruction *I, uint32_t TrueWeight, static void setBranchWeights(Instruction *I, uint64_t TrueWeight,
uint32_t FalseWeight) { uint64_t FalseWeight) {
assert(isa<BranchInst>(I) || isa<SelectInst>(I)); assert(isa<BranchInst>(I) || isa<SelectInst>(I));
// Check that there is at least one non-zero weight. Otherwise, pass // Check that there is at least one non-zero weight. Otherwise, pass
// nullptr to setMetadata which will erase the existing metadata. // nullptr to setMetadata which will erase the existing metadata.
MDNode *N = nullptr; MDNode *N = nullptr;
if (TrueWeight || FalseWeight) if (TrueWeight || FalseWeight)
N = MDBuilder(I->getParent()->getContext()) N = MDBuilder(I->getParent()->getContext())
.createBranchWeights(TrueWeight, FalseWeight); .createBranchWeights(TrueWeight, FalseWeight);
I->setMetadata(LLVMContext::MD_prof, N); I->setMetadata(LLVMContext::MD_prof, N);
▲ Show 20 Lines • Show All 171 Lines • ▼ Show 20 Linesstatic void GetBranchWeights(Instruction *TI,
if (BranchInst *BI = dyn_cast<BranchInst>(TI)) { if (BranchInst *BI = dyn_cast<BranchInst>(TI)) {
assert(Weights.size() == 2); assert(Weights.size() == 2);
ICmpInst *ICI = cast<ICmpInst>(BI->getCondition()); ICmpInst *ICI = cast<ICmpInst>(BI->getCondition());
if (ICI->getPredicate() == ICmpInst::ICMP_EQ) if (ICI->getPredicate() == ICmpInst::ICMP_EQ)
std::swap(Weights.front(), Weights.back()); std::swap(Weights.front(), Weights.back());
} }
} }
/// Keep halving the weights until all can fit in uint32_t.
static void FitWeights(MutableArrayRef<uint64_t> Weights) {
uint64_t Max = *std::max_element(Weights.begin(), Weights.end());
if (Max > UINT_MAX) {
unsigned Offset = 32 - countLeadingZeros(Max);
for (uint64_t &I : Weights)
I >>= Offset;
}
}
/// The specified terminator is a value equality comparison instruction /// The specified terminator is a value equality comparison instruction
/// (either a switch or a branch on "X == c"). /// (either a switch or a branch on "X == c").
/// See if any of the predecessors of the terminator block are value comparisons /// See if any of the predecessors of the terminator block are value comparisons
/// on the same value. If so, and if safe to do so, fold them together. /// on the same value. If so, and if safe to do so, fold them together.
bool SimplifyCFGOpt::FoldValueComparisonIntoPredecessors(Instruction *TI, bool SimplifyCFGOpt::FoldValueComparisonIntoPredecessors(Instruction *TI,
IRBuilder<> &Builder) { IRBuilder<> &Builder) {
BasicBlock *BB = TI->getParent(); BasicBlock *BB = TI->getParent();
Value *CV = isValueEqualityComparison(TI); // CondVal Value *CV = isValueEqualityComparison(TI); // CondVal
▲ Show 20 Lines • Show All 169 Lines • ▼ Show 20 Lines if (PCV == CV && TI != PTI) {
// Now that the successors are updated, create the new Switch instruction. // Now that the successors are updated, create the new Switch instruction.
SwitchInst *NewSI = SwitchInst *NewSI =
Builder.CreateSwitch(CV, PredDefault, PredCases.size()); Builder.CreateSwitch(CV, PredDefault, PredCases.size());
NewSI->setDebugLoc(PTI->getDebugLoc()); NewSI->setDebugLoc(PTI->getDebugLoc());
for (ValueEqualityComparisonCase &V : PredCases) for (ValueEqualityComparisonCase &V : PredCases)
NewSI->addCase(V.Value, V.Dest); NewSI->addCase(V.Value, V.Dest);
if (PredHasWeights || SuccHasWeights) { if (PredHasWeights || SuccHasWeights) {
// Halve the weights if any of them cannot fit in an uint32_t SmallVector<uint64_t, 8> MDWeights(Weights.begin(), Weights.end());
FitWeights(Weights);
SmallVector<uint32_t, 8> MDWeights(Weights.begin(), Weights.end());
setBranchWeights(NewSI, MDWeights); setBranchWeights(NewSI, MDWeights);
} }
EraseTerminatorAndDCECond(PTI); EraseTerminatorAndDCECond(PTI);
// Okay, last check. If BB is still a successor of PSI, then we must // Okay, last check. If BB is still a successor of PSI, then we must
// have an infinite loop case. If so, add an infinitely looping block // have an infinite loop case. If so, add an infinitely looping block
▲ Show 20 Lines • Show All 1,714 Lines • ▼ Show 20 Lines if (BI->isConditional()) {
PredFalseWeight * SuccTrueWeight); PredFalseWeight * SuccTrueWeight);
// FalseWeight is FalseWeight for PBI * FalseWeight for BI. // FalseWeight is FalseWeight for PBI * FalseWeight for BI.
NewWeights.push_back(PredFalseWeight * SuccFalseWeight); NewWeights.push_back(PredFalseWeight * SuccFalseWeight);
} }
AddPredecessorToBlock(FalseDest, PredBlock, BB, MSSAU); AddPredecessorToBlock(FalseDest, PredBlock, BB, MSSAU);
PBI->setSuccessor(1, FalseDest); PBI->setSuccessor(1, FalseDest);
} }
if (NewWeights.size() == 2) { if (NewWeights.size() == 2) {
// Halve the weights if any of them cannot fit in an uint32_t SmallVector<uint64_t, 8> MDWeights(NewWeights.begin(),
FitWeights(NewWeights);
SmallVector<uint32_t, 8> MDWeights(NewWeights.begin(),
NewWeights.end()); NewWeights.end());
setBranchWeights(PBI, MDWeights[0], MDWeights[1]); setBranchWeights(PBI, MDWeights[0], MDWeights[1]);
} else } else
PBI->setMetadata(LLVMContext::MD_prof, nullptr); PBI->setMetadata(LLVMContext::MD_prof, nullptr);
} else { } else {
// Update PHI nodes in the common successors. // Update PHI nodes in the common successors.
for (unsigned i = 0, e = PHIs.size(); i != e; ++i) { for (unsigned i = 0, e = PHIs.size(); i != e; ++i) {
ConstantInt *PBI_C = cast<ConstantInt>( ConstantInt *PBI_C = cast<ConstantInt>(
▲ Show 20 Lines • Show All 611 Lines • ▼ Show 20 Lines if (HasWeights) {
SuccCommon = BIOp ? SuccFalseWeight : SuccTrueWeight; SuccCommon = BIOp ? SuccFalseWeight : SuccTrueWeight;
SuccOther = BIOp ? SuccTrueWeight : SuccFalseWeight; SuccOther = BIOp ? SuccTrueWeight : SuccFalseWeight;
// The weight to CommonDest should be PredCommon * SuccTotal + // The weight to CommonDest should be PredCommon * SuccTotal +
// PredOther * SuccCommon. // PredOther * SuccCommon.
// The weight to OtherDest should be PredOther * SuccOther. // The weight to OtherDest should be PredOther * SuccOther.
uint64_t NewWeights[2] = {PredCommon * (SuccCommon + SuccOther) + uint64_t NewWeights[2] = {PredCommon * (SuccCommon + SuccOther) +
PredOther * SuccCommon, PredOther * SuccCommon,
PredOther * SuccOther}; PredOther * SuccOther};
// Halve the weights if any of them cannot fit in an uint32_t
FitWeights(NewWeights);
setBranchWeights(PBI, NewWeights[0], NewWeights[1]); setBranchWeights(PBI, NewWeights[0], NewWeights[1]);
} }
// OtherDest may have phi nodes. If so, add an entry from PBI's // OtherDest may have phi nodes. If so, add an entry from PBI's
// block that are identical to the entries for BI's block. // block that are identical to the entries for BI's block.
AddPredecessorToBlock(OtherDest, PBI->getParent(), BB); AddPredecessorToBlock(OtherDest, PBI->getParent(), BB);
Show All 19 Lines if (BIV != PBIV) {
uint64_t PredOther = PBIOp ? PredTrueWeight : PredFalseWeight; uint64_t PredOther = PBIOp ? PredTrueWeight : PredFalseWeight;
uint64_t SuccCommon = BIOp ? SuccFalseWeight : SuccTrueWeight; uint64_t SuccCommon = BIOp ? SuccFalseWeight : SuccTrueWeight;
uint64_t SuccOther = BIOp ? SuccTrueWeight : SuccFalseWeight; uint64_t SuccOther = BIOp ? SuccTrueWeight : SuccFalseWeight;
// The weight to PredCommonDest should be PredCommon * SuccTotal. // The weight to PredCommonDest should be PredCommon * SuccTotal.
// The weight to PredOtherDest should be PredOther * SuccCommon. // The weight to PredOtherDest should be PredOther * SuccCommon.
uint64_t NewWeights[2] = {PredCommon * (SuccCommon + SuccOther), uint64_t NewWeights[2] = {PredCommon * (SuccCommon + SuccOther),
PredOther * SuccCommon}; PredOther * SuccCommon};
FitWeights(NewWeights);
setBranchWeights(NV, NewWeights[0], NewWeights[1]); setBranchWeights(NV, NewWeights[0], NewWeights[1]);
} }
} }
} }
LLVM_DEBUG(dbgs() << "INTO: " << *PBI->getParent()); LLVM_DEBUG(dbgs() << "INTO: " << *PBI->getParent());
LLVM_DEBUG(dbgs() << *PBI->getParent()->getParent()); LLVM_DEBUG(dbgs() << *PBI->getParent()->getParent());
// This basic block is probably dead. We know it has at least // This basic block is probably dead. We know it has at least
// one fewer predecessor. // one fewer predecessor.
return true; return true;
} }
// Simplifies a terminator by replacing it with a branch to TrueBB if Cond is // Simplifies a terminator by replacing it with a branch to TrueBB if Cond is
// true or to FalseBB if Cond is false. // true or to FalseBB if Cond is false.
// Takes care of updating the successors and removing the old terminator. // Takes care of updating the successors and removing the old terminator.
// Also makes sure not to introduce new successors by assuming that edges to // Also makes sure not to introduce new successors by assuming that edges to
// non-successor TrueBBs and FalseBBs aren't reachable. // non-successor TrueBBs and FalseBBs aren't reachable.
bool SimplifyCFGOpt::SimplifyTerminatorOnSelect(Instruction *OldTerm, bool SimplifyCFGOpt::SimplifyTerminatorOnSelect(Instruction *OldTerm,
Value *Cond, BasicBlock *TrueBB, Value *Cond, BasicBlock *TrueBB,
BasicBlock *FalseBB, BasicBlock *FalseBB,
uint32_t TrueWeight, uint64_t TrueWeight,
uint32_t FalseWeight) { uint64_t FalseWeight) {
// Remove any superfluous successor edges from the CFG. // Remove any superfluous successor edges from the CFG.
// First, figure out which successors to preserve. // First, figure out which successors to preserve.
// If TrueBB and FalseBB are equal, only try to preserve one copy of that // If TrueBB and FalseBB are equal, only try to preserve one copy of that
// successor. // successor.
BasicBlock *KeepEdge1 = TrueBB; BasicBlock *KeepEdge1 = TrueBB;
BasicBlock *KeepEdge2 = TrueBB != FalseBB ? FalseBB : nullptr; BasicBlock *KeepEdge2 = TrueBB != FalseBB ? FalseBB : nullptr;
// Then remove the rest. // Then remove the rest.
▲ Show 20 Lines • Show All 57 Lines • ▼ Show 20 Lines if (!TrueVal || !FalseVal)
return false; return false;
// Find the relevant condition and destinations. // Find the relevant condition and destinations.
Value *Condition = Select->getCondition(); Value *Condition = Select->getCondition();
BasicBlock *TrueBB = SI->findCaseValue(TrueVal)->getCaseSuccessor(); BasicBlock *TrueBB = SI->findCaseValue(TrueVal)->getCaseSuccessor();
BasicBlock *FalseBB = SI->findCaseValue(FalseVal)->getCaseSuccessor(); BasicBlock *FalseBB = SI->findCaseValue(FalseVal)->getCaseSuccessor();
// Get weight for TrueBB and FalseBB. // Get weight for TrueBB and FalseBB.
uint32_t TrueWeight = 0, FalseWeight = 0; uint64_t TrueWeight = 0, FalseWeight = 0;
SmallVector<uint64_t, 8> Weights; SmallVector<uint64_t, 8> Weights;
bool HasWeights = HasBranchWeights(SI); bool HasWeights = HasBranchWeights(SI);
if (HasWeights) { if (HasWeights) {
GetBranchWeights(SI, Weights); GetBranchWeights(SI, Weights);
if (Weights.size() == 1 + SI->getNumCases()) { if (Weights.size() == 1 + SI->getNumCases()) {
TrueWeight = TrueWeight =
(uint32_t)Weights[SI->findCaseValue(TrueVal)->getSuccessorIndex()]; (uint64_t)Weights[SI->findCaseValue(TrueVal)->getSuccessorIndex()];
FalseWeight = FalseWeight =
(uint32_t)Weights[SI->findCaseValue(FalseVal)->getSuccessorIndex()]; (uint64_t)Weights[SI->findCaseValue(FalseVal)->getSuccessorIndex()];
} }
} }
// Perform the actual simplification. // Perform the actual simplification.
return SimplifyTerminatorOnSelect(SI, Condition, TrueBB, FalseBB, TrueWeight, return SimplifyTerminatorOnSelect(SI, Condition, TrueBB, FalseBB, TrueWeight,
FalseWeight); FalseWeight);
} }
▲ Show 20 Lines • Show All 2,591 Lines • Show Last 20 Lines

llvm/test/Analysis/BranchProbabilityInfo/basic.ll

  • This file was copied to llvm/test/Analysis/BranchProbabilityInfo/basic_i32.ll.
Show All 39 Lineselse:
br label %exit br label %exit
; CHECK: edge else -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge else -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge]
exit: exit:
%result = phi i32 [ %a, %then ], [ %b, %else ] %result = phi i32 [ %a, %then ], [ %b, %else ]
ret i32 %result ret i32 %result
} }
!0 = !{!"branch_weights", i32 64, i32 4} !0 = !{!"branch_weights", i64 64, i64 4}
define i32 @test3(i32 %i, i32 %a, i32 %b, i32 %c, i32 %d, i32 %e) { define i32 @test3(i32 %i, i32 %a, i32 %b, i32 %c, i32 %d, i32 %e) {
; CHECK: Printing analysis {{.*}} for function 'test3' ; CHECK: Printing analysis {{.*}} for function 'test3'
entry: entry:
switch i32 %i, label %case_a [ i32 1, label %case_b switch i32 %i, label %case_a [ i32 1, label %case_b
i32 2, label %case_c i32 2, label %case_c
i32 3, label %case_d i32 3, label %case_d
i32 4, label %case_e ], !prof !1 i32 4, label %case_e ], !prof !1
Show All 27 Linesexit:
%result = phi i32 [ %a, %case_a ], %result = phi i32 [ %a, %case_a ],
[ %b, %case_b ], [ %b, %case_b ],
[ %c, %case_c ], [ %c, %case_c ],
[ %d, %case_d ], [ %d, %case_d ],
[ %e, %case_e ] [ %e, %case_e ]
ret i32 %result ret i32 %result
} }
!1 = !{!"branch_weights", i32 4, i32 4, i32 64, i32 4, i32 4} !1 = !{!"branch_weights", i64 4, i64 4, i64 64, i64 4, i64 4}
define i32 @test4(i32 %x) nounwind uwtable readnone ssp { define i32 @test4(i32 %x) nounwind uwtable readnone ssp {
; CHECK: Printing analysis {{.*}} for function 'test4' ; CHECK: Printing analysis {{.*}} for function 'test4'
entry: entry:
%conv = sext i32 %x to i64 %conv = sext i32 %x to i64
switch i64 %conv, label %return [ switch i64 %conv, label %return [
i64 0, label %sw.bb i64 0, label %sw.bb
i64 1, label %sw.bb i64 1, label %sw.bb
Show All 10 Lines
sw.bb1: sw.bb1:
br label %return br label %return
return: return:
%retval.0 = phi i32 [ 5, %sw.bb1 ], [ 1, %sw.bb ], [ 0, %entry ] %retval.0 = phi i32 [ 5, %sw.bb1 ], [ 1, %sw.bb ], [ 0, %entry ]
ret i32 %retval.0 ret i32 %retval.0
} }
!2 = !{!"branch_weights", i32 7, i32 6, i32 4, i32 4, i32 64} !2 = !{!"branch_weights", i64 7, i64 6, i64 4, i64 4, i64 64}
declare void @coldfunc() cold declare void @coldfunc() cold
define i32 @test5(i32 %a, i32 %b, i1 %flag) { define i32 @test5(i32 %a, i32 %b, i1 %flag) {
; CHECK: Printing analysis {{.*}} for function 'test5' ; CHECK: Printing analysis {{.*}} for function 'test5'
entry: entry:
br i1 %flag, label %then, label %else br i1 %flag, label %then, label %else
; CHECK: edge entry -> then probability is 0x07878788 / 0x80000000 = 5.88% ; CHECK: edge entry -> then probability is 0x07878788 / 0x80000000 = 5.88%
▲ Show 20 Lines • Show All 63 Lines • ▼ Show 20 Lineselse:
br label %exit br label %exit
; CHECK: edge else -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge else -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge]
exit: exit:
%result = phi i32 [ %joinresult, %join ], [ %b, %else ] %result = phi i32 [ %joinresult, %join ], [ %b, %else ]
ret i32 %result ret i32 %result
} }
!3 = !{!"branch_weights", i32 100, i32 1} !3 = !{!"branch_weights", i64 100, i64 1}
define i32 @test_cold_call_sites(i32* %a) { define i32 @test_cold_call_sites(i32* %a) {
; Test that edges to blocks post-dominated by cold call sites ; Test that edges to blocks post-dominated by cold call sites
; are marked as not expected to be taken. ; are marked as not expected to be taken.
; TODO(dnovillo) The calls to regular_function should not be merged, but ; TODO(dnovillo) The calls to regular_function should not be merged, but
; they are currently being merged. Convert this into a code generation test ; they are currently being merged. Convert this into a code generation test
; after that is fixed. ; after that is fixed.
▲ Show 20 Lines • Show All 115 Lines • ▼ Show 20 Lines
; CHECK: edge entry -> invoke.to0 probability is 0x7ffff800 / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge entry -> invoke.to0 probability is 0x7ffff800 / 0x80000000 = 100.00% [HOT edge]
; CHECK: edge entry -> lpad probability is 0x00000800 / 0x80000000 = 0.00% ; CHECK: edge entry -> lpad probability is 0x00000800 / 0x80000000 = 0.00%
invoke i32 @InvokeCall() to label %invoke.to0 unwind label %lpad invoke i32 @InvokeCall() to label %invoke.to0 unwind label %lpad
invoke.to0: invoke.to0:
; CHECK: edge invoke.to0 -> invoke.to1 probability is 0x7ffff800 / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge invoke.to0 -> invoke.to1 probability is 0x7ffff800 / 0x80000000 = 100.00% [HOT edge]
; CHECK: edge invoke.to0 -> lpad probability is 0x00000800 / 0x80000000 = 0.00% ; CHECK: edge invoke.to0 -> lpad probability is 0x00000800 / 0x80000000 = 0.00%
invoke i32 @InvokeCall() to label %invoke.to1 unwind label %lpad, invoke i32 @InvokeCall() to label %invoke.to1 unwind label %lpad,
!prof !{!"branch_weights", i32 444} !prof !{!"branch_weights", i64 444}
invoke.to1: invoke.to1:
; CHECK: invoke.to1 -> invoke.to2 probability is 0x55555555 / 0x80000000 = 66.67% ; CHECK: invoke.to1 -> invoke.to2 probability is 0x55555555 / 0x80000000 = 66.67%
; CHECK: invoke.to1 -> lpad probability is 0x2aaaaaab / 0x80000000 = 33.33% ; CHECK: invoke.to1 -> lpad probability is 0x2aaaaaab / 0x80000000 = 33.33%
invoke i32 @InvokeCall() to label %invoke.to2 unwind label %lpad, invoke i32 @InvokeCall() to label %invoke.to2 unwind label %lpad,
!prof !{!"branch_weights", i32 222, i32 111} !prof !{!"branch_weights", i64 222, i64 111}
ret void ret void
invoke.to2: invoke.to2:
ret void ret void
lpad: lpad:
%ll = landingpad { i8*, i32 } %ll = landingpad { i8*, i32 }
cleanup cleanup
▲ Show 20 Lines • Show All 84 Lines • ▼ Show 20 Lines
unr: unr:
unreachable unreachable
exit: exit:
ret i32 %b ret i32 %b
} }
!4 = !{!"branch_weights", i32 0, i32 1} !4 = !{!"branch_weights", i64 0, i64 1}
define i32 @test_unreachable_with_prof_equal(i32 %a, i32 %b) { define i32 @test_unreachable_with_prof_equal(i32 %a, i32 %b) {
; CHECK: Printing analysis {{.*}} for function 'test_unreachable_with_prof_equal' ; CHECK: Printing analysis {{.*}} for function 'test_unreachable_with_prof_equal'
entry: entry:
%cond = icmp eq i32 %a, 42 %cond = icmp eq i32 %a, 42
br i1 %cond, label %exit, label %unr, !prof !5 br i1 %cond, label %exit, label %unr, !prof !5
; CHECK: edge entry -> exit probability is 0x7fffffff / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge entry -> exit probability is 0x7fffffff / 0x80000000 = 100.00% [HOT edge]
; CHECK: edge entry -> unr probability is 0x00000001 / 0x80000000 = 0.00% ; CHECK: edge entry -> unr probability is 0x00000001 / 0x80000000 = 0.00%
unr: unr:
unreachable unreachable
exit: exit:
ret i32 %b ret i32 %b
} }
!5 = !{!"branch_weights", i32 2147483647, i32 1} !5 = !{!"branch_weights", i64 2147483647, i64 1}
define i32 @test_unreachable_with_prof_zero(i32 %a, i32 %b) { define i32 @test_unreachable_with_prof_zero(i32 %a, i32 %b) {
; CHECK: Printing analysis {{.*}} for function 'test_unreachable_with_prof_zero' ; CHECK: Printing analysis {{.*}} for function 'test_unreachable_with_prof_zero'
entry: entry:
%cond = icmp eq i32 %a, 42 %cond = icmp eq i32 %a, 42
br i1 %cond, label %exit, label %unr, !prof !6 br i1 %cond, label %exit, label %unr, !prof !6
; CHECK: edge entry -> exit probability is 0x7fffffff / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge entry -> exit probability is 0x7fffffff / 0x80000000 = 100.00% [HOT edge]
; CHECK: edge entry -> unr probability is 0x00000001 / 0x80000000 = 0.00% ; CHECK: edge entry -> unr probability is 0x00000001 / 0x80000000 = 0.00%
unr: unr:
unreachable unreachable
exit: exit:
ret i32 %b ret i32 %b
} }
!6 = !{!"branch_weights", i32 0, i32 0} !6 = !{!"branch_weights", i64 0, i64 0}
define i32 @test_unreachable_with_prof_less(i32 %a, i32 %b) { define i32 @test_unreachable_with_prof_less(i32 %a, i32 %b) {
; CHECK: Printing analysis {{.*}} for function 'test_unreachable_with_prof_less' ; CHECK: Printing analysis {{.*}} for function 'test_unreachable_with_prof_less'
entry: entry:
%cond = icmp eq i32 %a, 42 %cond = icmp eq i32 %a, 42
br i1 %cond, label %exit, label %unr, !prof !7 br i1 %cond, label %exit, label %unr, !prof !7
; CHECK: edge entry -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge entry -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge]
; CHECK: edge entry -> unr probability is 0x00000000 / 0x80000000 = 0.00% ; CHECK: edge entry -> unr probability is 0x00000000 / 0x80000000 = 0.00%
unr: unr:
unreachable unreachable
exit: exit:
ret i32 %b ret i32 %b
} }
!7 = !{!"branch_weights", i32 1, i32 0} !7 = !{!"branch_weights", i64 1, i64 0}
define i32 @test_unreachable_with_switch_prof1(i32 %i, i32 %a, i32 %b, i32 %c, i32 %d, i32 %e) { define i32 @test_unreachable_with_switch_prof1(i32 %i, i32 %a, i32 %b, i32 %c, i32 %d, i32 %e) {
; CHECK: Printing analysis {{.*}} for function 'test_unreachable_with_switch_prof1' ; CHECK: Printing analysis {{.*}} for function 'test_unreachable_with_switch_prof1'
entry: entry:
switch i32 %i, label %case_a [ i32 1, label %case_b switch i32 %i, label %case_a [ i32 1, label %case_b
i32 2, label %case_c i32 2, label %case_c
i32 3, label %case_d i32 3, label %case_d
i32 4, label %case_e ], !prof !8 i32 4, label %case_e ], !prof !8
Show All 26 Lines
exit: exit:
%result = phi i32 [ %b, %case_b ], %result = phi i32 [ %b, %case_b ],
[ %c, %case_c ], [ %c, %case_c ],
[ %d, %case_d ], [ %d, %case_d ],
[ %e, %case_e ] [ %e, %case_e ]
ret i32 %result ret i32 %result
} }
!8 = !{!"branch_weights", i32 4, i32 4, i32 64, i32 4, i32 4} !8 = !{!"branch_weights", i64 4, i64 4, i64 64, i64 4, i64 4}
define i32 @test_unreachable_with_switch_prof2(i32 %i, i32 %a, i32 %b, i32 %c, i32 %d, i32 %e) { define i32 @test_unreachable_with_switch_prof2(i32 %i, i32 %a, i32 %b, i32 %c, i32 %d, i32 %e) {
; CHECK: Printing analysis {{.*}} for function 'test_unreachable_with_switch_prof2' ; CHECK: Printing analysis {{.*}} for function 'test_unreachable_with_switch_prof2'
entry: entry:
switch i32 %i, label %case_a [ i32 1, label %case_b switch i32 %i, label %case_a [ i32 1, label %case_b
i32 2, label %case_c i32 2, label %case_c
i32 3, label %case_d i32 3, label %case_d
i32 4, label %case_e ], !prof !9 i32 4, label %case_e ], !prof !9
Show All 25 Lines
exit: exit:
%result = phi i32 [ %c, %case_c ], %result = phi i32 [ %c, %case_c ],
[ %d, %case_d ], [ %d, %case_d ],
[ %e, %case_e ] [ %e, %case_e ]
ret i32 %result ret i32 %result
} }
!9 = !{!"branch_weights", i32 4, i32 4, i32 64, i32 4, i32 4} !9 = !{!"branch_weights", i64 4, i64 4, i64 64, i64 4, i64 4}
define i32 @test_unreachable_with_switch_prof3(i32 %i, i32 %a, i32 %b, i32 %c, i32 %d, i32 %e) { define i32 @test_unreachable_with_switch_prof3(i32 %i, i32 %a, i32 %b, i32 %c, i32 %d, i32 %e) {
; CHECK: Printing analysis {{.*}} for function 'test_unreachable_with_switch_prof3' ; CHECK: Printing analysis {{.*}} for function 'test_unreachable_with_switch_prof3'
entry: entry:
switch i32 %i, label %case_a [ i32 1, label %case_b switch i32 %i, label %case_a [ i32 1, label %case_b
i32 2, label %case_c i32 2, label %case_c
i32 3, label %case_d i32 3, label %case_d
i32 4, label %case_e ], !prof !10 i32 4, label %case_e ], !prof !10
Show All 24 Lines
exit: exit:
%result = phi i32 [ %c, %case_c ], %result = phi i32 [ %c, %case_c ],
[ %d, %case_d ], [ %d, %case_d ],
[ %e, %case_e ] [ %e, %case_e ]
ret i32 %result ret i32 %result
} }
!10 = !{!"branch_weights", i32 0, i32 4, i32 64, i32 4, i32 4} !10 = !{!"branch_weights", i64 0, i64 4, i64 64, i64 4, i64 4}
define i32 @test_unreachable_with_switch_prof4(i32 %i, i32 %a, i32 %b, i32 %c, i32 %d, i32 %e) { define i32 @test_unreachable_with_switch_prof4(i32 %i, i32 %a, i32 %b, i32 %c, i32 %d, i32 %e) {
; CHECK: Printing analysis {{.*}} for function 'test_unreachable_with_switch_prof4' ; CHECK: Printing analysis {{.*}} for function 'test_unreachable_with_switch_prof4'
entry: entry:
switch i32 %i, label %case_a [ i32 1, label %case_b switch i32 %i, label %case_a [ i32 1, label %case_b
i32 2, label %case_c i32 2, label %case_c
i32 3, label %case_d i32 3, label %case_d
i32 4, label %case_e ], !prof !11 i32 4, label %case_e ], !prof !11
Show All 15 Lines
case_d: case_d:
unreachable unreachable
case_e: case_e:
unreachable unreachable
} }
!11 = !{!"branch_weights", i32 0, i32 4, i32 64, i32 4, i32 4} !11 = !{!"branch_weights", i64 0, i64 4, i64 64, i64 4, i64 4}

llvm/test/Analysis/BranchProbabilityInfo/basic_i32.ll

  • This file was copied from llvm/test/Analysis/BranchProbabilityInfo/basic.ll.
; RUN: opt < %s -analyze -branch-prob -enable-new-pm=0 | FileCheck %s ; RUN: opt < %s -analyze -branch-prob -enable-new-pm=0 | FileCheck %s
; RUN: opt < %s -analyze -lazy-branch-prob -enable-new-pm=0 | FileCheck %s ; RUN: opt < %s -analyze -lazy-branch-prob -enable-new-pm=0 | FileCheck %s
; RUN: opt < %s -passes='print<branch-prob>' -disable-output 2>&1 | FileCheck %s ; RUN: opt < %s -passes='print<branch-prob>' -disable-output 2>&1 | FileCheck %s
define i32 @test1(i32 %i, i32* %a) { define i32 @test1(i32 %i, i32* %a) {
; CHECK: Printing analysis {{.*}} for function 'test1' ; CHECK: Printing analysis {{.*}} for function 'test1'
entry: entry:
br label %body br label %body
; CHECK: edge entry -> body probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge entry -> body probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge]
body: body:
%iv = phi i32 [ 0, %entry ], [ %next, %body ] %iv = phi i32 [ 0, %entry ], [ %next, %body ]
%base = phi i32 [ 0, %entry ], [ %sum, %body ] %base = phi i32 [ 0, %entry ], [ %sum, %body ]
%arrayidx = getelementptr inbounds i32, i32* %a, i32 %iv %arrayidx = getelementptr inbounds i32, i32* %a, i32 %iv
%0 = load i32, i32* %arrayidx %0 = load i32, i32* %arrayidx
%sum = add nsw i32 %0, %base %sum = add nsw i32 %0, %base
%next = add i32 %iv, 1 %next = add i32 %iv, 1
%exitcond = icmp eq i32 %next, %i %exitcond = icmp eq i32 %next, %i
br i1 %exitcond, label %exit, label %body br i1 %exitcond, label %exit, label %body
; CHECK: edge body -> exit probability is 0x04000000 / 0x80000000 = 3.12% ; CHECK: edge body -> exit probability is 0x04000000 / 0x80000000 = 3.12%
; CHECK: edge body -> body probability is 0x7c000000 / 0x80000000 = 96.88% [HOT edge] ; CHECK: edge body -> body probability is 0x7c000000 / 0x80000000 = 96.88% [HOT edge]
exit: exit:
ret i32 %sum ret i32 %sum
} }
define i32 @test2(i32 %i, i32 %a, i32 %b) { define i32 @test2(i32 %i, i32 %a, i32 %b) {
; CHECK: Printing analysis {{.*}} for function 'test2' ; CHECK: Printing analysis {{.*}} for function 'test2'
entry: entry:
%cond = icmp ult i32 %i, 42 %cond = icmp ult i32 %i, 42
br i1 %cond, label %then, label %else, !prof !0 br i1 %cond, label %then, label %else, !prof !0
; CHECK: edge entry -> then probability is 0x78787878 / 0x80000000 = 94.12% [HOT edge] ; CHECK: edge entry -> then probability is 0x78787878 / 0x80000000 = 94.12% [HOT edge]
; CHECK: edge entry -> else probability is 0x07878788 / 0x80000000 = 5.88% ; CHECK: edge entry -> else probability is 0x07878788 / 0x80000000 = 5.88%
then: then:
br label %exit br label %exit
; CHECK: edge then -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge then -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge]
else: else:
br label %exit br label %exit
; CHECK: edge else -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge else -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge]
exit: exit:
%result = phi i32 [ %a, %then ], [ %b, %else ] %result = phi i32 [ %a, %then ], [ %b, %else ]
ret i32 %result ret i32 %result
} }
!0 = !{!"branch_weights", i32 64, i32 4} !0 = !{!"branch_weights", i32 64, i32 4}
define i32 @test3(i32 %i, i32 %a, i32 %b, i32 %c, i32 %d, i32 %e) { define i32 @test3(i32 %i, i32 %a, i32 %b, i32 %c, i32 %d, i32 %e) {
; CHECK: Printing analysis {{.*}} for function 'test3' ; CHECK: Printing analysis {{.*}} for function 'test3'
entry: entry:
switch i32 %i, label %case_a [ i32 1, label %case_b switch i32 %i, label %case_a [ i32 1, label %case_b
i32 2, label %case_c i32 2, label %case_c
i32 3, label %case_d i32 3, label %case_d
i32 4, label %case_e ], !prof !1 i32 4, label %case_e ], !prof !1
; CHECK: edge entry -> case_a probability is 0x06666666 / 0x80000000 = 5.00% ; CHECK: edge entry -> case_a probability is 0x06666666 / 0x80000000 = 5.00%
; CHECK: edge entry -> case_b probability is 0x06666666 / 0x80000000 = 5.00% ; CHECK: edge entry -> case_b probability is 0x06666666 / 0x80000000 = 5.00%
; CHECK: edge entry -> case_c probability is 0x66666666 / 0x80000000 = 80.00% ; CHECK: edge entry -> case_c probability is 0x66666666 / 0x80000000 = 80.00%
; CHECK: edge entry -> case_d probability is 0x06666666 / 0x80000000 = 5.00% ; CHECK: edge entry -> case_d probability is 0x06666666 / 0x80000000 = 5.00%
; CHECK: edge entry -> case_e probability is 0x06666666 / 0x80000000 = 5.00% ; CHECK: edge entry -> case_e probability is 0x06666666 / 0x80000000 = 5.00%
case_a: case_a:
br label %exit br label %exit
; CHECK: edge case_a -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge case_a -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge]
case_b: case_b:
br label %exit br label %exit
; CHECK: edge case_b -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge case_b -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge]
case_c: case_c:
br label %exit br label %exit
; CHECK: edge case_c -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge case_c -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge]
case_d: case_d:
br label %exit br label %exit
; CHECK: edge case_d -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge case_d -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge]
case_e: case_e:
br label %exit br label %exit
; CHECK: edge case_e -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge case_e -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge]
exit: exit:
%result = phi i32 [ %a, %case_a ], %result = phi i32 [ %a, %case_a ],
[ %b, %case_b ], [ %b, %case_b ],
[ %c, %case_c ], [ %c, %case_c ],
[ %d, %case_d ], [ %d, %case_d ],
[ %e, %case_e ] [ %e, %case_e ]
ret i32 %result ret i32 %result
} }
!1 = !{!"branch_weights", i32 4, i32 4, i32 64, i32 4, i32 4} !1 = !{!"branch_weights", i32 4, i32 4, i32 64, i32 4, i32 4}
define i32 @test4(i32 %x) nounwind uwtable readnone ssp { define i32 @test4(i32 %x) nounwind uwtable readnone ssp {
; CHECK: Printing analysis {{.*}} for function 'test4' ; CHECK: Printing analysis {{.*}} for function 'test4'
entry: entry:
%conv = sext i32 %x to i64 %conv = sext i32 %x to i64
switch i64 %conv, label %return [ switch i64 %conv, label %return [
i64 0, label %sw.bb i64 0, label %sw.bb
i64 1, label %sw.bb i64 1, label %sw.bb
i64 2, label %sw.bb i64 2, label %sw.bb
i64 5, label %sw.bb1 i64 5, label %sw.bb1
], !prof !2 ], !prof !2
; CHECK: edge entry -> return probability is 0x0a8a8a8b / 0x80000000 = 8.24% ; CHECK: edge entry -> return probability is 0x0a8a8a8b / 0x80000000 = 8.24%
; CHECK: edge entry -> sw.bb probability is 0x15151515 / 0x80000000 = 16.47% ; CHECK: edge entry -> sw.bb probability is 0x15151515 / 0x80000000 = 16.47%
; CHECK: edge entry -> sw.bb1 probability is 0x60606060 / 0x80000000 = 75.29% ; CHECK: edge entry -> sw.bb1 probability is 0x60606060 / 0x80000000 = 75.29%
sw.bb: sw.bb:
br label %return br label %return
sw.bb1: sw.bb1:
br label %return br label %return
return: return:
%retval.0 = phi i32 [ 5, %sw.bb1 ], [ 1, %sw.bb ], [ 0, %entry ] %retval.0 = phi i32 [ 5, %sw.bb1 ], [ 1, %sw.bb ], [ 0, %entry ]
ret i32 %retval.0 ret i32 %retval.0
} }
!2 = !{!"branch_weights", i32 7, i32 6, i32 4, i32 4, i32 64} !2 = !{!"branch_weights", i32 7, i32 6, i32 4, i32 4, i32 64}
declare void @coldfunc() cold declare void @coldfunc() cold
define i32 @test5(i32 %a, i32 %b, i1 %flag) { define i32 @test5(i32 %a, i32 %b, i1 %flag) {
; CHECK: Printing analysis {{.*}} for function 'test5' ; CHECK: Printing analysis {{.*}} for function 'test5'
entry: entry:
br i1 %flag, label %then, label %else br i1 %flag, label %then, label %else
; CHECK: edge entry -> then probability is 0x07878788 / 0x80000000 = 5.88% ; CHECK: edge entry -> then probability is 0x07878788 / 0x80000000 = 5.88%
; CHECK: edge entry -> else probability is 0x78787878 / 0x80000000 = 94.12% [HOT edge] ; CHECK: edge entry -> else probability is 0x78787878 / 0x80000000 = 94.12% [HOT edge]
then: then:
call void @coldfunc() call void @coldfunc()
br label %exit br label %exit
; CHECK: edge then -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge then -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge]
else: else:
br label %exit br label %exit
; CHECK: edge else -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge else -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge]
exit: exit:
%result = phi i32 [ %a, %then ], [ %b, %else ] %result = phi i32 [ %a, %then ], [ %b, %else ]
ret i32 %result ret i32 %result
} }
define i32 @test_cold_loop(i32 %a, i32 %b) { define i32 @test_cold_loop(i32 %a, i32 %b) {
entry: entry:
%cond1 = icmp eq i32 %a, 42 %cond1 = icmp eq i32 %a, 42
br i1 %cond1, label %header, label %exit br i1 %cond1, label %header, label %exit
header: header:
br label %body br label %body
body: body:
%cond2 = icmp eq i32 %b, 42 %cond2 = icmp eq i32 %b, 42
br i1 %cond2, label %header, label %exit br i1 %cond2, label %header, label %exit
; CHECK: edge body -> header probability is 0x40000000 / 0x80000000 = 50.00% ; CHECK: edge body -> header probability is 0x40000000 / 0x80000000 = 50.00%
exit: exit:
call void @coldfunc() call void @coldfunc()
ret i32 %b ret i32 %b
} }
declare i32 @regular_function(i32 %i) declare i32 @regular_function(i32 %i)
define i32 @test_cold_call_sites_with_prof(i32 %a, i32 %b, i1 %flag, i1 %flag2) { define i32 @test_cold_call_sites_with_prof(i32 %a, i32 %b, i1 %flag, i1 %flag2) {
; CHECK: Printing analysis {{.*}} for function 'test_cold_call_sites_with_prof' ; CHECK: Printing analysis {{.*}} for function 'test_cold_call_sites_with_prof'
entry: entry:
br i1 %flag, label %then, label %else br i1 %flag, label %then, label %else
; CHECK: edge entry -> then probability is 0x07878788 / 0x80000000 = 5.88% ; CHECK: edge entry -> then probability is 0x07878788 / 0x80000000 = 5.88%
; CHECK: edge entry -> else probability is 0x78787878 / 0x80000000 = 94.12% [HOT edge] ; CHECK: edge entry -> else probability is 0x78787878 / 0x80000000 = 94.12% [HOT edge]
then: then:
br i1 %flag2, label %then2, label %else2, !prof !3 br i1 %flag2, label %then2, label %else2, !prof !3
; CHECK: edge then -> then2 probability is 0x7ebb907a / 0x80000000 = 99.01% [HOT edge] ; CHECK: edge then -> then2 probability is 0x7ebb907a / 0x80000000 = 99.01% [HOT edge]
; CHECK: edge then -> else2 probability is 0x01446f86 / 0x80000000 = 0.99% ; CHECK: edge then -> else2 probability is 0x01446f86 / 0x80000000 = 0.99%
then2: then2:
br label %join br label %join
; CHECK: edge then2 -> join probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge then2 -> join probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge]
else2: else2:
br label %join br label %join
; CHECK: edge else2 -> join probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge else2 -> join probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge]
join: join:
%joinresult = phi i32 [ %a, %then2 ], [ %b, %else2 ] %joinresult = phi i32 [ %a, %then2 ], [ %b, %else2 ]
call void @coldfunc() call void @coldfunc()
br label %exit br label %exit
; CHECK: edge join -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge join -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge]
else: else:
br label %exit br label %exit
; CHECK: edge else -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge else -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge]
exit: exit:
%result = phi i32 [ %joinresult, %join ], [ %b, %else ] %result = phi i32 [ %joinresult, %join ], [ %b, %else ]
ret i32 %result ret i32 %result
} }
!3 = !{!"branch_weights", i32 100, i32 1} !3 = !{!"branch_weights", i32 100, i32 1}
define i32 @test_cold_call_sites(i32* %a) { define i32 @test_cold_call_sites(i32* %a) {
; Test that edges to blocks post-dominated by cold call sites ; Test that edges to blocks post-dominated by cold call sites
; are marked as not expected to be taken. ; are marked as not expected to be taken.
; TODO(dnovillo) The calls to regular_function should not be merged, but ; TODO(dnovillo) The calls to regular_function should not be merged, but
; they are currently being merged. Convert this into a code generation test ; they are currently being merged. Convert this into a code generation test
; after that is fixed. ; after that is fixed.
; CHECK: Printing analysis {{.*}} for function 'test_cold_call_sites' ; CHECK: Printing analysis {{.*}} for function 'test_cold_call_sites'
; CHECK: edge entry -> then probability is 0x07878788 / 0x80000000 = 5.88% ; CHECK: edge entry -> then probability is 0x07878788 / 0x80000000 = 5.88%
; CHECK: edge entry -> else probability is 0x78787878 / 0x80000000 = 94.12% [HOT edge] ; CHECK: edge entry -> else probability is 0x78787878 / 0x80000000 = 94.12% [HOT edge]
entry: entry:
%gep1 = getelementptr i32, i32* %a, i32 1 %gep1 = getelementptr i32, i32* %a, i32 1
%val1 = load i32, i32* %gep1 %val1 = load i32, i32* %gep1
%cond1 = icmp ugt i32 %val1, 1 %cond1 = icmp ugt i32 %val1, 1
br i1 %cond1, label %then, label %else br i1 %cond1, label %then, label %else
then: then:
; This function is not declared cold, but this call site is. ; This function is not declared cold, but this call site is.
%val4 = call i32 @regular_function(i32 %val1) cold %val4 = call i32 @regular_function(i32 %val1) cold
br label %exit br label %exit
else: else:
%gep2 = getelementptr i32, i32* %a, i32 2 %gep2 = getelementptr i32, i32* %a, i32 2
%val2 = load i32, i32* %gep2 %val2 = load i32, i32* %gep2
%val3 = call i32 @regular_function(i32 %val2) %val3 = call i32 @regular_function(i32 %val2)
br label %exit br label %exit
exit: exit:
%ret = phi i32 [ %val4, %then ], [ %val3, %else ] %ret = phi i32 [ %val4, %then ], [ %val3, %else ]
ret i32 %ret ret i32 %ret
} }
; CHECK-LABEL: test_invoke_code_callsite1 ; CHECK-LABEL: test_invoke_code_callsite1
define i32 @test_invoke_code_callsite1(i1 %c) personality i8* bitcast (i32 (...)* @__gxx_personality_v0 to i8*) { define i32 @test_invoke_code_callsite1(i1 %c) personality i8* bitcast (i32 (...)* @__gxx_personality_v0 to i8*) {
entry: entry:
br i1 %c, label %if.then, label %if.end br i1 %c, label %if.then, label %if.end
; Edge "entry->if.end" should have higher probability based on the cold call ; Edge "entry->if.end" should have higher probability based on the cold call
; heuristic which treat %if.then as a cold block because the normal destination ; heuristic which treat %if.then as a cold block because the normal destination
; of the invoke instruction in %if.then is post-dominated by ColdFunc(). ; of the invoke instruction in %if.then is post-dominated by ColdFunc().
; CHECK: edge entry -> if.then probability is 0x07878788 / 0x80000000 = 5.88% ; CHECK: edge entry -> if.then probability is 0x07878788 / 0x80000000 = 5.88%
; CHECK: edge entry -> if.end probability is 0x78787878 / 0x80000000 = 94.12% [HOT edge] ; CHECK: edge entry -> if.end probability is 0x78787878 / 0x80000000 = 94.12% [HOT edge]
if.then: if.then:
invoke i32 @InvokeCall() invoke i32 @InvokeCall()
to label %invoke.cont unwind label %lpad to label %invoke.cont unwind label %lpad
; CHECK: edge if.then -> invoke.cont probability is 0x7ffff800 / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge if.then -> invoke.cont probability is 0x7ffff800 / 0x80000000 = 100.00% [HOT edge]
; CHECK: edge if.then -> lpad probability is 0x00000800 / 0x80000000 = 0.00% ; CHECK: edge if.then -> lpad probability is 0x00000800 / 0x80000000 = 0.00%
invoke.cont: invoke.cont:
call void @ColdFunc() #0 call void @ColdFunc() #0
br label %if.end br label %if.end
lpad: lpad:
%ll = landingpad { i8*, i32 } %ll = landingpad { i8*, i32 }
cleanup cleanup
br label %if.end br label %if.end
if.end: if.end:
ret i32 0 ret i32 0
} }
; CHECK-LABEL: test_invoke_code_callsite2 ; CHECK-LABEL: test_invoke_code_callsite2
define i32 @test_invoke_code_callsite2(i1 %c) personality i8* bitcast (i32 (...)* @__gxx_personality_v0 to i8*) { define i32 @test_invoke_code_callsite2(i1 %c) personality i8* bitcast (i32 (...)* @__gxx_personality_v0 to i8*) {
entry: entry:
br i1 %c, label %if.then, label %if.end br i1 %c, label %if.then, label %if.end
; CHECK: edge entry -> if.then probability is 0x40000000 / 0x80000000 = 50.00% ; CHECK: edge entry -> if.then probability is 0x40000000 / 0x80000000 = 50.00%
; CHECK: edge entry -> if.end probability is 0x40000000 / 0x80000000 = 50.00% ; CHECK: edge entry -> if.end probability is 0x40000000 / 0x80000000 = 50.00%
if.then: if.then:
invoke i32 @InvokeCall() invoke i32 @InvokeCall()
to label %invoke.cont unwind label %lpad to label %invoke.cont unwind label %lpad
; The cold call heuristic should not kick in when the cold callsite is in EH path. ; The cold call heuristic should not kick in when the cold callsite is in EH path.
; CHECK: edge if.then -> invoke.cont probability is 0x7ffff800 / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge if.then -> invoke.cont probability is 0x7ffff800 / 0x80000000 = 100.00% [HOT edge]
; CHECK: edge if.then -> lpad probability is 0x00000800 / 0x80000000 = 0.00% ; CHECK: edge if.then -> lpad probability is 0x00000800 / 0x80000000 = 0.00%
invoke.cont: invoke.cont:
br label %if.end br label %if.end
lpad: lpad:
%ll = landingpad { i8*, i32 } %ll = landingpad { i8*, i32 }
cleanup cleanup
call void @ColdFunc() #0 call void @ColdFunc() #0
br label %if.end br label %if.end
if.end: if.end:
ret i32 0 ret i32 0
} }
; CHECK-LABEL: test_invoke_code_callsite3 ; CHECK-LABEL: test_invoke_code_callsite3
define i32 @test_invoke_code_callsite3(i1 %c) personality i8* bitcast (i32 (...)* @__gxx_personality_v0 to i8*) { define i32 @test_invoke_code_callsite3(i1 %c) personality i8* bitcast (i32 (...)* @__gxx_personality_v0 to i8*) {
entry: entry:
br i1 %c, label %if.then, label %if.end br i1 %c, label %if.then, label %if.end
; CHECK: edge entry -> if.then probability is 0x07878788 / 0x80000000 = 5.88% ; CHECK: edge entry -> if.then probability is 0x07878788 / 0x80000000 = 5.88%
; CHECK: edge entry -> if.end probability is 0x78787878 / 0x80000000 = 94.12% [HOT edge] ; CHECK: edge entry -> if.end probability is 0x78787878 / 0x80000000 = 94.12% [HOT edge]
if.then: if.then:
invoke i32 @InvokeCall() invoke i32 @InvokeCall()
to label %invoke.cont unwind label %lpad to label %invoke.cont unwind label %lpad
; Regardless of cold calls, edge weights from a invoke instruction should be ; Regardless of cold calls, edge weights from a invoke instruction should be
; determined by the invoke heuristic. ; determined by the invoke heuristic.
; CHECK: edge if.then -> invoke.cont probability is 0x7ffff800 / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge if.then -> invoke.cont probability is 0x7ffff800 / 0x80000000 = 100.00% [HOT edge]
; CHECK: edge if.then -> lpad probability is 0x00000800 / 0x80000000 = 0.00% ; CHECK: edge if.then -> lpad probability is 0x00000800 / 0x80000000 = 0.00%
invoke.cont: invoke.cont:
call void @ColdFunc() #0 call void @ColdFunc() #0
br label %if.end br label %if.end
lpad: lpad:
%ll = landingpad { i8*, i32 } %ll = landingpad { i8*, i32 }
cleanup cleanup
call void @ColdFunc() #0 call void @ColdFunc() #0
br label %if.end br label %if.end
if.end: if.end:
ret i32 0 ret i32 0
} }
; CHECK-LABEL: test_invoke_code_profiled ; CHECK-LABEL: test_invoke_code_profiled
define void @test_invoke_code_profiled(i1 %c) personality i8* bitcast (i32 (...)* @__gxx_personality_v0 to i8*) { define void @test_invoke_code_profiled(i1 %c) personality i8* bitcast (i32 (...)* @__gxx_personality_v0 to i8*) {
entry: entry:
; CHECK: edge entry -> invoke.to0 probability is 0x7ffff800 / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge entry -> invoke.to0 probability is 0x7ffff800 / 0x80000000 = 100.00% [HOT edge]
; CHECK: edge entry -> lpad probability is 0x00000800 / 0x80000000 = 0.00% ; CHECK: edge entry -> lpad probability is 0x00000800 / 0x80000000 = 0.00%
invoke i32 @InvokeCall() to label %invoke.to0 unwind label %lpad invoke i32 @InvokeCall() to label %invoke.to0 unwind label %lpad
invoke.to0: invoke.to0:
; CHECK: edge invoke.to0 -> invoke.to1 probability is 0x7ffff800 / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge invoke.to0 -> invoke.to1 probability is 0x7ffff800 / 0x80000000 = 100.00% [HOT edge]
; CHECK: edge invoke.to0 -> lpad probability is 0x00000800 / 0x80000000 = 0.00% ; CHECK: edge invoke.to0 -> lpad probability is 0x00000800 / 0x80000000 = 0.00%
invoke i32 @InvokeCall() to label %invoke.to1 unwind label %lpad, invoke i32 @InvokeCall() to label %invoke.to1 unwind label %lpad,
!prof !{!"branch_weights", i32 444} !prof !{!"branch_weights", i32 444}
invoke.to1: invoke.to1:
; CHECK: invoke.to1 -> invoke.to2 probability is 0x55555555 / 0x80000000 = 66.67% ; CHECK: invoke.to1 -> invoke.to2 probability is 0x55555555 / 0x80000000 = 66.67%
; CHECK: invoke.to1 -> lpad probability is 0x2aaaaaab / 0x80000000 = 33.33% ; CHECK: invoke.to1 -> lpad probability is 0x2aaaaaab / 0x80000000 = 33.33%
invoke i32 @InvokeCall() to label %invoke.to2 unwind label %lpad, invoke i32 @InvokeCall() to label %invoke.to2 unwind label %lpad,
!prof !{!"branch_weights", i32 222, i32 111} !prof !{!"branch_weights", i32 222, i32 111}
ret void ret void
invoke.to2: invoke.to2:
ret void ret void
lpad: lpad:
%ll = landingpad { i8*, i32 } %ll = landingpad { i8*, i32 }
cleanup cleanup
ret void ret void
} }
declare i32 @__gxx_personality_v0(...) declare i32 @__gxx_personality_v0(...)
declare void @ColdFunc() declare void @ColdFunc()
declare i32 @InvokeCall() declare i32 @InvokeCall()
attributes #0 = { cold } attributes #0 = { cold }
define i32 @zero1(i32 %i, i32 %a, i32 %b) { define i32 @zero1(i32 %i, i32 %a, i32 %b) {
; CHECK: Printing analysis {{.*}} for function 'zero1' ; CHECK: Printing analysis {{.*}} for function 'zero1'
entry: entry:
%cond = icmp eq i32 %i, 0 %cond = icmp eq i32 %i, 0
br i1 %cond, label %then, label %else br i1 %cond, label %then, label %else
; CHECK: edge entry -> then probability is 0x30000000 / 0x80000000 = 37.50% ; CHECK: edge entry -> then probability is 0x30000000 / 0x80000000 = 37.50%
; CHECK: edge entry -> else probability is 0x50000000 / 0x80000000 = 62.50% ; CHECK: edge entry -> else probability is 0x50000000 / 0x80000000 = 62.50%
then: then:
br label %exit br label %exit
else: else:
br label %exit br label %exit
exit: exit:
%result = phi i32 [ %a, %then ], [ %b, %else ] %result = phi i32 [ %a, %then ], [ %b, %else ]
ret i32 %result ret i32 %result
} }
define i32 @zero2(i32 %i, i32 %a, i32 %b) { define i32 @zero2(i32 %i, i32 %a, i32 %b) {
; CHECK: Printing analysis {{.*}} for function 'zero2' ; CHECK: Printing analysis {{.*}} for function 'zero2'
entry: entry:
%cond = icmp ne i32 %i, -1 %cond = icmp ne i32 %i, -1
br i1 %cond, label %then, label %else br i1 %cond, label %then, label %else
; CHECK: edge entry -> then probability is 0x50000000 / 0x80000000 = 62.50% ; CHECK: edge entry -> then probability is 0x50000000 / 0x80000000 = 62.50%
; CHECK: edge entry -> else probability is 0x30000000 / 0x80000000 = 37.50% ; CHECK: edge entry -> else probability is 0x30000000 / 0x80000000 = 37.50%
then: then:
br label %exit br label %exit
else: else:
br label %exit br label %exit
exit: exit:
%result = phi i32 [ %a, %then ], [ %b, %else ] %result = phi i32 [ %a, %then ], [ %b, %else ]
ret i32 %result ret i32 %result
} }
define i32 @zero3(i32 %i, i32 %a, i32 %b) { define i32 @zero3(i32 %i, i32 %a, i32 %b) {
; CHECK: Printing analysis {{.*}} for function 'zero3' ; CHECK: Printing analysis {{.*}} for function 'zero3'
entry: entry:
; AND'ing with a single bit bitmask essentially leads to a bool comparison, ; AND'ing with a single bit bitmask essentially leads to a bool comparison,
; meaning we don't have probability information. ; meaning we don't have probability information.
%and = and i32 %i, 2 %and = and i32 %i, 2
%tobool = icmp eq i32 %and, 0 %tobool = icmp eq i32 %and, 0
br i1 %tobool, label %then, label %else br i1 %tobool, label %then, label %else
; CHECK: edge entry -> then probability is 0x40000000 / 0x80000000 = 50.00% ; CHECK: edge entry -> then probability is 0x40000000 / 0x80000000 = 50.00%
; CHECK: edge entry -> else probability is 0x40000000 / 0x80000000 = 50.00% ; CHECK: edge entry -> else probability is 0x40000000 / 0x80000000 = 50.00%
then: then:
; AND'ing with other bitmask might be something else, so we still assume the ; AND'ing with other bitmask might be something else, so we still assume the
; usual probabilities. ; usual probabilities.
%and2 = and i32 %i, 5 %and2 = and i32 %i, 5
%tobool2 = icmp eq i32 %and2, 0 %tobool2 = icmp eq i32 %and2, 0
br i1 %tobool2, label %else, label %exit br i1 %tobool2, label %else, label %exit
; CHECK: edge then -> else probability is 0x30000000 / 0x80000000 = 37.50% ; CHECK: edge then -> else probability is 0x30000000 / 0x80000000 = 37.50%
; CHECK: edge then -> exit probability is 0x50000000 / 0x80000000 = 62.50% ; CHECK: edge then -> exit probability is 0x50000000 / 0x80000000 = 62.50%
else: else:
br label %exit br label %exit
exit: exit:
%result = phi i32 [ %a, %then ], [ %b, %else ] %result = phi i32 [ %a, %then ], [ %b, %else ]
ret i32 %result ret i32 %result
} }
define i32 @test_unreachable_with_prof_greater(i32 %a, i32 %b) { define i32 @test_unreachable_with_prof_greater(i32 %a, i32 %b) {
; CHECK: Printing analysis {{.*}} for function 'test_unreachable_with_prof_greater' ; CHECK: Printing analysis {{.*}} for function 'test_unreachable_with_prof_greater'
entry: entry:
%cond = icmp eq i32 %a, 42 %cond = icmp eq i32 %a, 42
br i1 %cond, label %exit, label %unr, !prof !4 br i1 %cond, label %exit, label %unr, !prof !4
; CHECK: edge entry -> exit probability is 0x7fffffff / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge entry -> exit probability is 0x7fffffff / 0x80000000 = 100.00% [HOT edge]
; CHECK: edge entry -> unr probability is 0x00000001 / 0x80000000 = 0.00% ; CHECK: edge entry -> unr probability is 0x00000001 / 0x80000000 = 0.00%
unr: unr:
unreachable unreachable
exit: exit:
ret i32 %b ret i32 %b
} }
!4 = !{!"branch_weights", i32 0, i32 1} !4 = !{!"branch_weights", i32 0, i32 1}
define i32 @test_unreachable_with_prof_equal(i32 %a, i32 %b) { define i32 @test_unreachable_with_prof_equal(i32 %a, i32 %b) {
; CHECK: Printing analysis {{.*}} for function 'test_unreachable_with_prof_equal' ; CHECK: Printing analysis {{.*}} for function 'test_unreachable_with_prof_equal'
entry: entry:
%cond = icmp eq i32 %a, 42 %cond = icmp eq i32 %a, 42
br i1 %cond, label %exit, label %unr, !prof !5 br i1 %cond, label %exit, label %unr, !prof !5
; CHECK: edge entry -> exit probability is 0x7fffffff / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge entry -> exit probability is 0x7fffffff / 0x80000000 = 100.00% [HOT edge]
; CHECK: edge entry -> unr probability is 0x00000001 / 0x80000000 = 0.00% ; CHECK: edge entry -> unr probability is 0x00000001 / 0x80000000 = 0.00%
unr: unr:
unreachable unreachable
exit: exit:
ret i32 %b ret i32 %b
} }
!5 = !{!"branch_weights", i32 2147483647, i32 1} !5 = !{!"branch_weights", i32 2147483647, i32 1}
define i32 @test_unreachable_with_prof_zero(i32 %a, i32 %b) { define i32 @test_unreachable_with_prof_zero(i32 %a, i32 %b) {
; CHECK: Printing analysis {{.*}} for function 'test_unreachable_with_prof_zero' ; CHECK: Printing analysis {{.*}} for function 'test_unreachable_with_prof_zero'
entry: entry:
%cond = icmp eq i32 %a, 42 %cond = icmp eq i32 %a, 42
br i1 %cond, label %exit, label %unr, !prof !6 br i1 %cond, label %exit, label %unr, !prof !6
; CHECK: edge entry -> exit probability is 0x7fffffff / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge entry -> exit probability is 0x7fffffff / 0x80000000 = 100.00% [HOT edge]
; CHECK: edge entry -> unr probability is 0x00000001 / 0x80000000 = 0.00% ; CHECK: edge entry -> unr probability is 0x00000001 / 0x80000000 = 0.00%
unr: unr:
unreachable unreachable
exit: exit:
ret i32 %b ret i32 %b
} }
!6 = !{!"branch_weights", i32 0, i32 0} !6 = !{!"branch_weights", i32 0, i32 0}
define i32 @test_unreachable_with_prof_less(i32 %a, i32 %b) { define i32 @test_unreachable_with_prof_less(i32 %a, i32 %b) {
; CHECK: Printing analysis {{.*}} for function 'test_unreachable_with_prof_less' ; CHECK: Printing analysis {{.*}} for function 'test_unreachable_with_prof_less'
entry: entry:
%cond = icmp eq i32 %a, 42 %cond = icmp eq i32 %a, 42
br i1 %cond, label %exit, label %unr, !prof !7 br i1 %cond, label %exit, label %unr, !prof !7
; CHECK: edge entry -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge entry -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge]
; CHECK: edge entry -> unr probability is 0x00000000 / 0x80000000 = 0.00% ; CHECK: edge entry -> unr probability is 0x00000000 / 0x80000000 = 0.00%
unr: unr:
unreachable unreachable
exit: exit:
ret i32 %b ret i32 %b
} }
!7 = !{!"branch_weights", i32 1, i32 0} !7 = !{!"branch_weights", i32 1, i32 0}
define i32 @test_unreachable_with_switch_prof1(i32 %i, i32 %a, i32 %b, i32 %c, i32 %d, i32 %e) { define i32 @test_unreachable_with_switch_prof1(i32 %i, i32 %a, i32 %b, i32 %c, i32 %d, i32 %e) {
; CHECK: Printing analysis {{.*}} for function 'test_unreachable_with_switch_prof1' ; CHECK: Printing analysis {{.*}} for function 'test_unreachable_with_switch_prof1'
entry: entry:
switch i32 %i, label %case_a [ i32 1, label %case_b switch i32 %i, label %case_a [ i32 1, label %case_b
i32 2, label %case_c i32 2, label %case_c
i32 3, label %case_d i32 3, label %case_d
i32 4, label %case_e ], !prof !8 i32 4, label %case_e ], !prof !8
; Reachable probabilities keep their relation: 4/64/4/4 = 5.26% / 84.21% / 5.26% / 5.26%. ; Reachable probabilities keep their relation: 4/64/4/4 = 5.26% / 84.21% / 5.26% / 5.26%.
; CHECK: edge entry -> case_a probability is 0x00000001 / 0x80000000 = 0.00% ; CHECK: edge entry -> case_a probability is 0x00000001 / 0x80000000 = 0.00%
; CHECK: edge entry -> case_b probability is 0x06bca1af / 0x80000000 = 5.26% ; CHECK: edge entry -> case_b probability is 0x06bca1af / 0x80000000 = 5.26%
; CHECK: edge entry -> case_c probability is 0x6bca1af3 / 0x80000000 = 84.21% [HOT edge] ; CHECK: edge entry -> case_c probability is 0x6bca1af3 / 0x80000000 = 84.21% [HOT edge]
; CHECK: edge entry -> case_d probability is 0x06bca1af / 0x80000000 = 5.26% ; CHECK: edge entry -> case_d probability is 0x06bca1af / 0x80000000 = 5.26%
; CHECK: edge entry -> case_e probability is 0x06bca1af / 0x80000000 = 5.26% ; CHECK: edge entry -> case_e probability is 0x06bca1af / 0x80000000 = 5.26%
case_a: case_a:
unreachable unreachable
case_b: case_b:
br label %exit br label %exit
; CHECK: edge case_b -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge case_b -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge]
case_c: case_c:
br label %exit br label %exit
; CHECK: edge case_c -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge case_c -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge]
case_d: case_d:
br label %exit br label %exit
; CHECK: edge case_d -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge case_d -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge]
case_e: case_e:
br label %exit br label %exit
; CHECK: edge case_e -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge case_e -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge]
exit: exit:
%result = phi i32 [ %b, %case_b ], %result = phi i32 [ %b, %case_b ],
[ %c, %case_c ], [ %c, %case_c ],
[ %d, %case_d ], [ %d, %case_d ],
[ %e, %case_e ] [ %e, %case_e ]
ret i32 %result ret i32 %result
} }
!8 = !{!"branch_weights", i32 4, i32 4, i32 64, i32 4, i32 4} !8 = !{!"branch_weights", i32 4, i32 4, i32 64, i32 4, i32 4}
define i32 @test_unreachable_with_switch_prof2(i32 %i, i32 %a, i32 %b, i32 %c, i32 %d, i32 %e) { define i32 @test_unreachable_with_switch_prof2(i32 %i, i32 %a, i32 %b, i32 %c, i32 %d, i32 %e) {
; CHECK: Printing analysis {{.*}} for function 'test_unreachable_with_switch_prof2' ; CHECK: Printing analysis {{.*}} for function 'test_unreachable_with_switch_prof2'
entry: entry:
switch i32 %i, label %case_a [ i32 1, label %case_b switch i32 %i, label %case_a [ i32 1, label %case_b
i32 2, label %case_c i32 2, label %case_c
i32 3, label %case_d i32 3, label %case_d
i32 4, label %case_e ], !prof !9 i32 4, label %case_e ], !prof !9
; Reachable probabilities keep their relation: 64/4/4 = 88.89% / 5.56% / 5.56%. ; Reachable probabilities keep their relation: 64/4/4 = 88.89% / 5.56% / 5.56%.
; CHECK: edge entry -> case_a probability is 0x00000001 / 0x80000000 = 0.00% ; CHECK: edge entry -> case_a probability is 0x00000001 / 0x80000000 = 0.00%
; CHECK: edge entry -> case_b probability is 0x00000001 / 0x80000000 = 0.00% ; CHECK: edge entry -> case_b probability is 0x00000001 / 0x80000000 = 0.00%
; CHECK: edge entry -> case_c probability is 0x71c71c71 / 0x80000000 = 88.89% [HOT edge] ; CHECK: edge entry -> case_c probability is 0x71c71c71 / 0x80000000 = 88.89% [HOT edge]
; CHECK: edge entry -> case_d probability is 0x071c71c7 / 0x80000000 = 5.56% ; CHECK: edge entry -> case_d probability is 0x071c71c7 / 0x80000000 = 5.56%
; CHECK: edge entry -> case_e probability is 0x071c71c7 / 0x80000000 = 5.56% ; CHECK: edge entry -> case_e probability is 0x071c71c7 / 0x80000000 = 5.56%
case_a: case_a:
unreachable unreachable
case_b: case_b:
unreachable unreachable
case_c: case_c:
br label %exit br label %exit
; CHECK: edge case_c -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge case_c -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge]
case_d: case_d:
br label %exit br label %exit
; CHECK: edge case_d -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge case_d -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge]
case_e: case_e:
br label %exit br label %exit
; CHECK: edge case_e -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge case_e -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge]
exit: exit:
%result = phi i32 [ %c, %case_c ], %result = phi i32 [ %c, %case_c ],
[ %d, %case_d ], [ %d, %case_d ],
[ %e, %case_e ] [ %e, %case_e ]
ret i32 %result ret i32 %result
} }
!9 = !{!"branch_weights", i32 4, i32 4, i32 64, i32 4, i32 4} !9 = !{!"branch_weights", i32 4, i32 4, i32 64, i32 4, i32 4}
define i32 @test_unreachable_with_switch_prof3(i32 %i, i32 %a, i32 %b, i32 %c, i32 %d, i32 %e) { define i32 @test_unreachable_with_switch_prof3(i32 %i, i32 %a, i32 %b, i32 %c, i32 %d, i32 %e) {
; CHECK: Printing analysis {{.*}} for function 'test_unreachable_with_switch_prof3' ; CHECK: Printing analysis {{.*}} for function 'test_unreachable_with_switch_prof3'
entry: entry:
switch i32 %i, label %case_a [ i32 1, label %case_b switch i32 %i, label %case_a [ i32 1, label %case_b
i32 2, label %case_c i32 2, label %case_c
i32 3, label %case_d i32 3, label %case_d
i32 4, label %case_e ], !prof !10 i32 4, label %case_e ], !prof !10
; Reachable probabilities keep their relation: 64/4/4 = 88.89% / 5.56% / 5.56%. ; Reachable probabilities keep their relation: 64/4/4 = 88.89% / 5.56% / 5.56%.
; CHECK: edge entry -> case_a probability is 0x00000000 / 0x80000000 = 0.00% ; CHECK: edge entry -> case_a probability is 0x00000000 / 0x80000000 = 0.00%
; CHECK: edge entry -> case_b probability is 0x00000001 / 0x80000000 = 0.00% ; CHECK: edge entry -> case_b probability is 0x00000001 / 0x80000000 = 0.00%
; CHECK: edge entry -> case_c probability is 0x71c71c71 / 0x80000000 = 88.89% [HOT edge] ; CHECK: edge entry -> case_c probability is 0x71c71c71 / 0x80000000 = 88.89% [HOT edge]
; CHECK: edge entry -> case_d probability is 0x071c71c7 / 0x80000000 = 5.56% ; CHECK: edge entry -> case_d probability is 0x071c71c7 / 0x80000000 = 5.56%
; CHECK: edge entry -> case_e probability is 0x071c71c7 / 0x80000000 = 5.56% ; CHECK: edge entry -> case_e probability is 0x071c71c7 / 0x80000000 = 5.56%
case_a: case_a:
unreachable unreachable
case_b: case_b:
unreachable unreachable
case_c: case_c:
br label %exit br label %exit
; CHECK: edge case_c -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge case_c -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge]
case_d: case_d:
br label %exit br label %exit
; CHECK: edge case_d -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge case_d -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge]
case_e: case_e:
br label %exit br label %exit
; CHECK: edge case_e -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge] ; CHECK: edge case_e -> exit probability is 0x80000000 / 0x80000000 = 100.00% [HOT edge]
exit: exit:
%result = phi i32 [ %c, %case_c ], %result = phi i32 [ %c, %case_c ],
[ %d, %case_d ], [ %d, %case_d ],
[ %e, %case_e ] [ %e, %case_e ]
ret i32 %result ret i32 %result
} }
!10 = !{!"branch_weights", i32 0, i32 4, i32 64, i32 4, i32 4} !10 = !{!"branch_weights", i32 0, i32 4, i32 64, i32 4, i32 4}
define i32 @test_unreachable_with_switch_prof4(i32 %i, i32 %a, i32 %b, i32 %c, i32 %d, i32 %e) { define i32 @test_unreachable_with_switch_prof4(i32 %i, i32 %a, i32 %b, i32 %c, i32 %d, i32 %e) {
; CHECK: Printing analysis {{.*}} for function 'test_unreachable_with_switch_prof4' ; CHECK: Printing analysis {{.*}} for function 'test_unreachable_with_switch_prof4'
entry: entry:
switch i32 %i, label %case_a [ i32 1, label %case_b switch i32 %i, label %case_a [ i32 1, label %case_b
i32 2, label %case_c i32 2, label %case_c
i32 3, label %case_d i32 3, label %case_d
i32 4, label %case_e ], !prof !11 i32 4, label %case_e ], !prof !11
; CHECK: edge entry -> case_a probability is 0x1999999a / 0x80000000 = 20.00% ; CHECK: edge entry -> case_a probability is 0x1999999a / 0x80000000 = 20.00%
; CHECK: edge entry -> case_b probability is 0x1999999a / 0x80000000 = 20.00% ; CHECK: edge entry -> case_b probability is 0x1999999a / 0x80000000 = 20.00%
; CHECK: edge entry -> case_c probability is 0x1999999a / 0x80000000 = 20.00% ; CHECK: edge entry -> case_c probability is 0x1999999a / 0x80000000 = 20.00%
; CHECK: edge entry -> case_d probability is 0x1999999a / 0x80000000 = 20.00% ; CHECK: edge entry -> case_d probability is 0x1999999a / 0x80000000 = 20.00%
; CHECK: edge entry -> case_e probability is 0x1999999a / 0x80000000 = 20.00% ; CHECK: edge entry -> case_e probability is 0x1999999a / 0x80000000 = 20.00%
case_a: case_a:
unreachable unreachable
case_b: case_b:
unreachable unreachable
case_c: case_c:
unreachable unreachable
case_d: case_d:
unreachable unreachable
case_e: case_e:
unreachable unreachable
} }
!11 = !{!"branch_weights", i32 0, i32 4, i32 64, i32 4, i32 4} !11 = !{!"branch_weights", i32 0, i32 4, i32 64, i32 4, i32 4}

llvm/test/Instrumentation/AddressSanitizer/basic.ll

Show First 20 Lines • Show All 223 Lines • ▼ Show 20 Linesdefine void @test_swifterror_3() sanitize_address {
call void @test_swifterror_2(i8** swifterror %swifterror_addr) call void @test_swifterror_2(i8** swifterror %swifterror_addr)
ret void ret void
} }
; CHECK: define internal void @asan.module_ctor() ; CHECK: define internal void @asan.module_ctor()
; CHECK: call void @__asan_init() ; CHECK: call void @__asan_init()
; PROF ; PROF
; CHECK: ![[PROF]] = !{!"branch_weights", i32 1, i32 100000} ; CHECK: ![[PROF]] = !{!"branch_weights", i64 1, i64 100000}

llvm/test/Transforms/CodeExtractor/MultipleExitBranchProb.ll

Show All 22 Lines
; CHECK-LABEL: @dummyCaller ; CHECK-LABEL: @dummyCaller
; CHECK: call ; CHECK: call
; CHECK-NEXT: br i1 {{.*}}return.i{{.*}}return.2{{.*}}!prof [[COUNT1:![0-9]+]] ; CHECK-NEXT: br i1 {{.*}}return.i{{.*}}return.2{{.*}}!prof [[COUNT1:![0-9]+]]
} }
!llvm.module.flags = !{!0} !llvm.module.flags = !{!0}
!0 = !{i32 1, !"MaxFunctionCount", i32 10000} !0 = !{i32 1, !"MaxFunctionCount", i32 10000}
!1 = !{!"function_entry_count", i64 10000} !1 = !{!"function_entry_count", i64 10000}
!2 = !{!"branch_weights", i32 5, i32 5} !2 = !{!"branch_weights", i64 5, i64 5}
!3 = !{!"branch_weights", i32 4, i32 1} !3 = !{!"branch_weights", i64 4, i64 1}
; CHECK: [[COUNT1]] = !{!"branch_weights", i32 31, i32 8} ; CHECK: [[COUNT1]] = !{!"branch_weights", i64 31, i64 8}

llvm/test/Transforms/CorrelatedValuePropagation/profmd.ll

Show All 18 Lines; CHECK-NEXT: i32 -1, label [[NEXT]]
switch i32 %s, label %out [ switch i32 %s, label %out [
i32 0, label %out i32 0, label %out
i32 1, label %out i32 1, label %out
i32 -1, label %next i32 -1, label %next
i32 -2, label %next i32 -2, label %next
i32 2, label %out i32 2, label %out
i32 3, label %out i32 3, label %out
; CHECK-NEXT: !prof ![[MD0:[0-9]+]] ; CHECK-NEXT: !prof ![[MD0:[0-9]+]]
], !prof !{!"branch_weights", i32 99, i32 1, i32 2, i32 3, i32 4, i32 5, i32 6} ], !prof !{!"branch_weights", i64 99, i64 1, i64 2, i64 3, i64 4, i64 5, i64 6}
out: out:
%p = phi i32 [ 1, %entry ], [ -1, %negative ], [ -1, %negative ], [ -1, %negative ], [ -1, %negative ], [ -1, %negative ] %p = phi i32 [ 1, %entry ], [ -1, %negative ], [ -1, %negative ], [ -1, %negative ], [ -1, %negative ], [ -1, %negative ]
ret i32 %p ret i32 %p
next: next:
%q = phi i32 [ 0, %negative ], [ 0, %negative ] %q = phi i32 [ 0, %negative ], [ 0, %negative ]
ret i32 %q ret i32 %q
Show All 10 Linesentry:
%cmp = icmp sgt i32 %s, 0 %cmp = icmp sgt i32 %s, 0
br i1 %cmp, label %positive, label %out br i1 %cmp, label %positive, label %out
positive: positive:
switch i32 %s, label %out [ switch i32 %s, label %out [
i32 0, label %out i32 0, label %out
i32 -1, label %next i32 -1, label %next
i32 -2, label %next i32 -2, label %next
], !prof !{!"branch_weights", i32 99, i32 1, i32 2, i32 3} ], !prof !{!"branch_weights", i64 99, i64 1, i64 2, i64 3}
out: out:
%p = phi i32 [ -1, %entry ], [ 1, %positive ], [ 1, %positive ] %p = phi i32 [ -1, %entry ], [ 1, %positive ], [ 1, %positive ]
ret i32 %p ret i32 %p
next: next:
%q = phi i32 [ 0, %positive ], [ 0, %positive ] %q = phi i32 [ 0, %positive ], [ 0, %positive ]
ret i32 %q ret i32 %q
Show All 10 Lines
positive: positive:
; CHECK: positive: ; CHECK: positive:
; CHECK-NEXT: [[CMP:%.*]] = icmp eq i32 %s, 1 ; CHECK-NEXT: [[CMP:%.*]] = icmp eq i32 %s, 1
; CHECK-NEXT: br i1 [[CMP]], label [[NEXT:%.*]], label [[OUT:%.*]], !prof ![[MD1:[0-9]+]] ; CHECK-NEXT: br i1 [[CMP]], label [[NEXT:%.*]], label [[OUT:%.*]], !prof ![[MD1:[0-9]+]]
switch i32 %s, label %out [ switch i32 %s, label %out [
i32 1, label %next i32 1, label %next
i32 -1, label %next i32 -1, label %next
i32 -2, label %next i32 -2, label %next
], !prof !{!"branch_weights", i32 99, i32 1, i32 2, i32 3} ], !prof !{!"branch_weights", i64 99, i64 1, i64 2, i64 3}
out: out:
%p = phi i32 [ -1, %entry ], [ 1, %positive ] %p = phi i32 [ -1, %entry ], [ 1, %positive ]
ret i32 %p ret i32 %p
next: next:
%q = phi i32 [ 0, %positive ], [ 0, %positive ], [ 0, %positive ] %q = phi i32 [ 0, %positive ], [ 0, %positive ], [ 0, %positive ]
ret i32 %q ret i32 %q
Show All 13 Lines
negative: negative:
; CHECK: negative: ; CHECK: negative:
; CHECK-NEXT: br label %out ; CHECK-NEXT: br label %out
switch i32 %s, label %out [ switch i32 %s, label %out [
i32 0, label %out i32 0, label %out
i32 1, label %out i32 1, label %out
i32 2, label %out i32 2, label %out
i32 3, label %out i32 3, label %out
], !prof !{!"branch_weights", i32 99, i32 1, i32 2, i32 3, i32 4} ], !prof !{!"branch_weights", i64 99, i64 1, i64 2, i64 3, i64 4}
out: out:
%p = phi i32 [ 1, %entry ], [ -1, %negative ], [ -1, %negative ], [ -1, %negative ], [ -1, %negative ], [ -1, %negative ] %p = phi i32 [ 1, %entry ], [ -1, %negative ], [ -1, %negative ], [ -1, %negative ], [ -1, %negative ], [ -1, %negative ]
ret i32 %p ret i32 %p
} }
; CHECK: ![[MD0]] = !{!"branch_weights", i32 99, i32 4, i32 3} ; CHECK: ![[MD0]] = !{!"branch_weights", i64 99, i64 4, i64 3}
; CHECK: ![[MD1]] = !{!"branch_weights", i32 1, i32 99} ; CHECK: ![[MD1]] = !{!"branch_weights", i64 1, i64 99}

llvm/test/Transforms/JumpThreading/threading_prof1.ll

Show First 20 Lines • Show All 88 Lines • ▼ Show 20 Lines
} }
declare void @bar() declare void @bar()
declare i32 @a() declare i32 @a()
declare i32 @b() declare i32 @b()
!0 = !{!"branch_weights", i32 2146410443, i32 1073205} !0 = !{!"branch_weights", i64 2146410443, i64 1073205}
;CHECK: ![[PROF1]] = !{!"branch_weights", i32 1073205, i32 2146410443} ;CHECK: ![[PROF1]] = !{!"branch_weights", i64 1073205, i64 2146410443}
;CHECK: ![[PROF2]] = !{!"branch_weights", i32 2146410443, i32 1073205} ;CHECK: ![[PROF2]] = !{!"branch_weights", i64 2146410443, i64 1073205}

llvm/test/Transforms/JumpThreading/threading_prof2.ll

Show All 31 Lines
declare void @bar() declare void @bar()
declare void @foo() declare void @foo()
declare i32 @a() declare i32 @a()
declare i32 @b() declare i32 @b()
!0 = !{!"branch_weights", i32 2146410443, i32 1073205} !0 = !{!"branch_weights", i64 2146410443, i64 1073205}
;CHECK: ![[PROF1]] = !{!"branch_weights", i32 1073205, i32 2146410443} ;CHECK: ![[PROF1]] = !{!"branch_weights", i64 1073205, i64 2146410443}
;CHECK: ![[PROF2]] = !{!"branch_weights", i32 2146410443, i32 1073205} ;CHECK: ![[PROF2]] = !{!"branch_weights", i64 2146410443, i64 1073205}

llvm/test/Transforms/JumpThreading/update-edge-weight.ll

; RUN: opt -S -jump-threading %s | FileCheck %s ; RUN: opt -S -jump-threading %s | FileCheck %s
; RUN: opt -S -passes=jump-threading %s | FileCheck %s ; RUN: opt -S -passes=jump-threading %s | FileCheck %s
; Test if edge weights are properly updated after jump threading. ; Test if edge weights are properly updated after jump threading.
; CHECK: !2 = !{!"branch_weights", i32 1629125526, i32 518358122} ; CHECK: !2 = !{!"branch_weights", i64 1629125526, i64 518358122}
define void @foo(i32 %n) !prof !0 { define void @foo(i32 %n) !prof !0 {
entry: entry:
%cmp = icmp sgt i32 %n, 10 %cmp = icmp sgt i32 %n, 10
br i1 %cmp, label %if.then.1, label %if.else.1, !prof !1 br i1 %cmp, label %if.then.1, label %if.else.1, !prof !1
if.then.1: if.then.1:
tail call void @a() tail call void @a()
Show All 20 Lines
} }
declare void @a() declare void @a()
declare void @b() declare void @b()
declare void @c() declare void @c()
declare void @d() declare void @d()
!0 = !{!"function_entry_count", i64 1} !0 = !{!"function_entry_count", i64 1}
!1 = !{!"branch_weights", i32 10, i32 5} !1 = !{!"branch_weights", i64 10, i64 5}
!2 = !{!"branch_weights", i32 10, i32 1} !2 = !{!"branch_weights", i64 10, i64 1}

llvm/test/Transforms/LoopUnroll/peel-loop-pgo-deopt.ll

Show First 20 Lines • Show All 71 Lines • ▼ Show 20 Lines
!8 = !{!"NumCounts", i64 2} !8 = !{!"NumCounts", i64 2}
!9 = !{!"NumFunctions", i64 2} !9 = !{!"NumFunctions", i64 2}
!10 = !{!"DetailedSummary", !11} !10 = !{!"DetailedSummary", !11}
!11 = !{!12, !13, !14} !11 = !{!12, !13, !14}
!12 = !{i32 10000, i64 3, i32 2} !12 = !{i32 10000, i64 3, i32 2}
!13 = !{i32 999000, i64 1, i32 10} !13 = !{i32 999000, i64 1, i32 10}
!14 = !{i32 999999, i64 1, i32 10} !14 = !{i32 999999, i64 1, i32 10}
!15 = !{!"function_entry_count", i64 1} !15 = !{!"function_entry_count", i64 1}
!16 = !{!"branch_weights", i32 3001, i32 1001} !16 = !{!"branch_weights", i64 3001, i64 1001}
!17 = !{!"branch_weights", i32 1, i32 0} !17 = !{!"branch_weights", i64 1, i64 0}
; This is a weights of deopt side-exit. ; This is a weights of deopt side-exit.
;CHECK: !15 = !{!"branch_weights", i32 1, i32 0} ;CHECK: !15 = !{!"branch_weights", i64 1, i64 0}
; This is a weights of latch and its copies. ; This is a weights of latch and its copies.
;CHECK: !16 = !{!"branch_weights", i32 3001, i32 1001} ;CHECK: !16 = !{!"branch_weights", i64 3001, i64 1001}
;CHECK: !17 = !{!"branch_weights", i32 2000, i32 1001} ;CHECK: !17 = !{!"branch_weights", i64 2000, i64 1001}
;CHECK: !18 = !{!"branch_weights", i32 999, i32 1001} ;CHECK: !18 = !{!"branch_weights", i64 999, i64 1001}
;CHECK: !19 = !{!"branch_weights", i32 1, i32 1001} ;CHECK: !19 = !{!"branch_weights", i64 1, i64 1001}

llvm/test/Transforms/LoopUnroll/peel-loop-pgo.ll

Show First 20 Lines • Show All 95 Lines • ▼ Show 20 Lines
!8 = !{!"NumCounts", i64 2} !8 = !{!"NumCounts", i64 2}
!9 = !{!"NumFunctions", i64 2} !9 = !{!"NumFunctions", i64 2}
!10 = !{!"DetailedSummary", !11} !10 = !{!"DetailedSummary", !11}
!11 = !{!12, !13, !14} !11 = !{!12, !13, !14}
!12 = !{i32 10000, i64 3, i32 2} !12 = !{i32 10000, i64 3, i32 2}
!13 = !{i32 999000, i64 1, i32 10} !13 = !{i32 999000, i64 1, i32 10}
!14 = !{i32 999999, i64 1, i32 10} !14 = !{i32 999999, i64 1, i32 10}
!15 = !{!"function_entry_count", i64 1} !15 = !{!"function_entry_count", i64 1}
!16 = !{!"branch_weights", i32 3001, i32 1001} !16 = !{!"branch_weights", i64 3001, i64 1001}
;CHECK: !15 = !{!"branch_weights", i32 3001, i32 1001} ;CHECK: !15 = !{!"branch_weights", i64 3001, i64 1001}
;CHECK: !16 = !{!"branch_weights", i32 2000, i32 1001} ;CHECK: !16 = !{!"branch_weights", i64 2000, i64 1001}
;CHECK: !17 = !{!"branch_weights", i32 999, i32 1001} ;CHECK: !17 = !{!"branch_weights", i64 999, i64 1001}
;CHECK: !18 = !{!"branch_weights", i32 1, i32 1001} ;CHECK: !18 = !{!"branch_weights", i64 1, i64 1001}

llvm/test/Transforms/LoopUnroll/runtime-loop-branchweight.ll

; RUN: opt < %s -S -loop-unroll -unroll-runtime=true -unroll-count=4 | FileCheck %s ; RUN: opt < %s -S -loop-unroll -unroll-runtime=true -unroll-count=4 | FileCheck %s
; RUN: opt < %s -S -passes=loop-unroll -unroll-runtime=true -unroll-count=4 | FileCheck %s ; RUN: opt < %s -S -passes=loop-unroll -unroll-runtime=true -unroll-count=4 | FileCheck %s
;; Check that the remainder loop is properly assigned a branch weight for its latch branch. ;; Check that the remainder loop is properly assigned a branch weight for its latch branch.
; CHECK-LABEL: @test( ; CHECK-LABEL: @test(
; CHECK-LABEL: for.body: ; CHECK-LABEL: for.body:
; CHECK: br i1 [[COND1:%.*]], label %for.end.loopexit.unr-lcssa.loopexit, label %for.body, !prof ![[#PROF:]], !llvm.loop ![[#LOOP:]] ; CHECK: br i1 [[COND1:%.*]], label %for.end.loopexit.unr-lcssa.loopexit, label %for.body, !prof ![[#PROF:]], !llvm.loop ![[#LOOP:]]
; CHECK-LABEL: for.body.epil: ; CHECK-LABEL: for.body.epil:
; CHECK: br i1 [[COND2:%.*]], label %for.body.epil, label %for.end.loopexit.epilog-lcssa, !prof ![[#PROF2:]], !llvm.loop ![[#LOOP2:]] ; CHECK: br i1 [[COND2:%.*]], label %for.body.epil, label %for.end.loopexit.epilog-lcssa, !prof ![[#PROF2:]], !llvm.loop ![[#LOOP2:]]
; CHECK: ![[#PROF]] = !{!"branch_weights", i32 1, i32 9999} ; CHECK: ![[#PROF]] = !{!"branch_weights", i64 1, i64 9999}
; CHECK: ![[#PROF2]] = !{!"branch_weights", i32 3, i32 1} ; CHECK: ![[#PROF2]] = !{!"branch_weights", i64 3, i64 1}
define i3 @test(i3* %a, i3 %n) { define i3 @test(i3* %a, i3 %n) {
entry: entry:
%cmp1 = icmp eq i3 %n, 0 %cmp1 = icmp eq i3 %n, 0
br i1 %cmp1, label %for.end, label %for.body br i1 %cmp1, label %for.end, label %for.body
for.body: for.body:
%indvars.iv = phi i64 [ %indvars.iv.next, %for.body ], [ 0, %entry ] %indvars.iv = phi i64 [ %indvars.iv.next, %for.body ], [ 0, %entry ]
%sum.02 = phi i3 [ %add, %for.body ], [ 0, %entry ] %sum.02 = phi i3 [ %add, %for.body ], [ 0, %entry ]
%arrayidx = getelementptr inbounds i3, i3* %a, i64 %indvars.iv %arrayidx = getelementptr inbounds i3, i3* %a, i64 %indvars.iv
%0 = load i3, i3* %arrayidx %0 = load i3, i3* %arrayidx
%add = add nsw i3 %0, %sum.02 %add = add nsw i3 %0, %sum.02
%indvars.iv.next = add i64 %indvars.iv, 1 %indvars.iv.next = add i64 %indvars.iv, 1
%lftr.wideiv = trunc i64 %indvars.iv.next to i3 %lftr.wideiv = trunc i64 %indvars.iv.next to i3
%exitcond = icmp eq i3 %lftr.wideiv, %n %exitcond = icmp eq i3 %lftr.wideiv, %n
br i1 %exitcond, label %for.end, label %for.body, !prof !0 br i1 %exitcond, label %for.end, label %for.body, !prof !0
for.end: for.end:
%sum.0.lcssa = phi i3 [ 0, %entry ], [ %add, %for.body ] %sum.0.lcssa = phi i3 [ 0, %entry ], [ %add, %for.body ]
ret i3 %sum.0.lcssa ret i3 %sum.0.lcssa
} }
!0 = !{!"branch_weights", i32 1, i32 9999} !0 = !{!"branch_weights", i64 1, i64 9999}

llvm/test/Transforms/LoopVectorize/check-prof-info.ll

Show First 20 Lines • Show All 74 Lines • ▼ Show 20 Linesfor.body: ; preds = %for.body, %entry
br i1 %exitcond, label %for.cond.cleanup, label %for.body, !prof !7 br i1 %exitcond, label %for.cond.cleanup, label %for.body, !prof !7
} }
attributes #0 = { "use-soft-float"="false" } attributes #0 = { "use-soft-float"="false" }
!llvm.module.flags = !{!0} !llvm.module.flags = !{!0}
!llvm.ident = !{!1} !llvm.ident = !{!1}
; CHECK: [[LP1_255]] = !{!"branch_weights", i32 1, i32 255} ; CHECK: [[LP1_255]] = !{!"branch_weights", i64 1, i64 255}
; CHECK: [[LP0_0]] = !{!"branch_weights", i32 0, i32 0} ; CHECK: [[LP0_0]] = !{!"branch_weights", i64 0, i64 0}
; CHECK-MASKED: [[LP1_63]] = !{!"branch_weights", i32 1, i32 63} ; CHECK-MASKED: [[LP1_63]] = !{!"branch_weights", i64 1, i64 63}
; CHECK-MASKED: [[LP0_0]] = !{!"branch_weights", i32 0, i32 0} ; CHECK-MASKED: [[LP0_0]] = !{!"branch_weights", i64 0, i64 0}
; CHECK: [[LP1_2]] = !{!"branch_weights", i32 1, i32 2} ; CHECK: [[LP1_2]] = !{!"branch_weights", i64 1, i64 2}
!0 = !{i32 1, !"wchar_size", i32 4} !0 = !{i32 1, !"wchar_size", i32 4}
!1 = !{!"clang version 10.0.0 (https://github.com/llvm/llvm-project c292b5b5e059e6ce3e6449e6827ef7e1037c21c4)"} !1 = !{!"clang version 10.0.0 (https://github.com/llvm/llvm-project c292b5b5e059e6ce3e6449e6827ef7e1037c21c4)"}
!2 = !{!3, !3, i64 0} !2 = !{!3, !3, i64 0}
!3 = !{!"int", !4, i64 0} !3 = !{!"int", !4, i64 0}
!4 = !{!"omnipotent char", !5, i64 0} !4 = !{!"omnipotent char", !5, i64 0}
!5 = !{!"Simple C++ TBAA"} !5 = !{!"Simple C++ TBAA"}
!6 = !{!"branch_weights", i32 1, i32 1023} !6 = !{!"branch_weights", i64 1, i64 1023}
!7 = !{!"branch_weights", i32 1, i32 1026} !7 = !{!"branch_weights", i64 1, i64 1026}

llvm/test/Transforms/LoopVectorize/tripcount.ll

Show First 20 Lines • Show All 201 Lines • ▼ Show 20 Linesfor.body: ; preds = %for.body, %entry
%inc = add nsw i32 %i.08, 1 %inc = add nsw i32 %i.08, 1
%exitcond = icmp slt i32 %i.08, 1000 %exitcond = icmp slt i32 %i.08, 1000
br i1 %exitcond, label %for.body, label %for.end, !prof !1 br i1 %exitcond, label %for.body, label %for.end, !prof !1
for.end: ; preds = %for.body for.end: ; preds = %for.body
ret i32 0 ret i32 0
} }
; CHECK: [[LP3]] = !{!"branch_weights", i32 10, i32 2490} ; CHECK: [[LP3]] = !{!"branch_weights", i64 10, i64 2490}
; CHECK: [[LP6]] = !{!"branch_weights", i32 10, i32 0} ; CHECK: [[LP6]] = !{!"branch_weights", i64 10, i64 0}
; original loop has latchExitWeight=10 and backedgeTakenWeight=10,000, ; original loop has latchExitWeight=10 and backedgeTakenWeight=10,000,
; therefore estimatedBackedgeTakenCount=1,000 and estimatedTripCount=1,001. ; therefore estimatedBackedgeTakenCount=1,000 and estimatedTripCount=1,001.
; Vectorizing by 4 produces estimatedTripCounts of 1,001/4=250 and 1,001%4=1 ; Vectorizing by 4 produces estimatedTripCounts of 1,001/4=250 and 1,001%4=1
; for vectorized and remainder loops, respectively, therefore their ; for vectorized and remainder loops, respectively, therefore their
; estimatedBackedgeTakenCounts are 249 and 0, and so the weights recorded with ; estimatedBackedgeTakenCounts are 249 and 0, and so the weights recorded with
; loop invocation weights of 10 are the above {10, 2490} and {10, 0}. ; loop invocation weights of 10 are the above {10, 2490} and {10, 0}.
!0 = !{!"function_entry_count", i64 100} !0 = !{!"function_entry_count", i64 100}
!1 = !{!"branch_weights", i32 100, i32 0} !1 = !{!"branch_weights", i64 100, i64 0}
!2 = !{!"branch_weights", i32 10, i32 90} !2 = !{!"branch_weights", i64 10, i64 90}
!3 = !{!"branch_weights", i32 10, i32 10000} !3 = !{!"branch_weights", i64 10, i64 10000}

llvm/test/Transforms/LowerExpectIntrinsic/basic.ll

Show First 20 Lines • Show All 279 Lines • ▼ Show 20 Linesdefine i32 @test10(i64 %t6) {
%t9 = select i1 %t8, i32 1, i32 2 %t9 = select i1 %t8, i32 1, i32 2
; CHECK: select{{.*}}, !prof !2, !misexpect !3 ; CHECK: select{{.*}}, !prof !2, !misexpect !3
ret i32 %t9 ret i32 %t9
} }
declare i1 @llvm.expect.i1(i1, i1) nounwind readnone declare i1 @llvm.expect.i1(i1, i1) nounwind readnone
; CHECK: !0 = !{!"branch_weights", i32 2000, i32 1} ; CHECK: !0 = !{!"branch_weights", i64 2000, i64 1}
; CHECK: !1 = !{!"misexpect", i64 0, i64 2000, i64 1} ; CHECK: !1 = !{!"misexpect", i64 0, i64 2000, i64 1}
; CHECK: !2 = !{!"branch_weights", i32 1, i32 2000} ; CHECK: !2 = !{!"branch_weights", i64 1, i64 2000}
; CHECK: !3 = !{!"misexpect", i64 1, i64 2000, i64 1} ; CHECK: !3 = !{!"misexpect", i64 1, i64 2000, i64 1}
; CHECK: !4 = !{!"branch_weights", i32 1, i32 1, i32 2000} ; CHECK: !4 = !{!"branch_weights", i64 1, i64 1, i64 2000}
; CHECK: !5 = !{!"misexpect", i64 2, i64 2000, i64 1} ; CHECK: !5 = !{!"misexpect", i64 2, i64 2000, i64 1}
; CHECK: !6 = !{!"branch_weights", i32 2000, i32 1, i32 1} ; CHECK: !6 = !{!"branch_weights", i64 2000, i64 1, i64 1}

llvm/test/Transforms/LowerExpectIntrinsic/expect-with-probability.ll

Show First 20 Lines • Show All 279 Lines • ▼ Show 20 Linesdefine i32 @test10(i64 %t6) {
%t9 = select i1 %t8, i32 1, i32 2 %t9 = select i1 %t8, i32 1, i32 2
; CHECK: select{{.*}}, !prof !2, !misexpect !3 ; CHECK: select{{.*}}, !prof !2, !misexpect !3
ret i32 %t9 ret i32 %t9
} }
declare i1 @llvm.expect.with.probability.i1(i1, i1, double) nounwind readnone declare i1 @llvm.expect.with.probability.i1(i1, i1, double) nounwind readnone
; CHECK: !0 = !{!"branch_weights", i32 1717986918, i32 429496731} ; CHECK: !0 = !{!"branch_weights", i64 1717986918, i64 429496731}
; CHECK: !1 = !{!"misexpect", i64 0, i64 1717986918, i64 429496731} ; CHECK: !1 = !{!"misexpect", i64 0, i64 1717986918, i64 429496731}
; CHECK: !2 = !{!"branch_weights", i32 429496731, i32 1717986918} ; CHECK: !2 = !{!"branch_weights", i64 429496731, i64 1717986918}
; CHECK: !3 = !{!"misexpect", i64 1, i64 1717986918, i64 429496731} ; CHECK: !3 = !{!"misexpect", i64 1, i64 1717986918, i64 429496731}
; CHECK: !4 = !{!"branch_weights", i32 214748366, i32 214748366, i32 1717986918} ; CHECK: !4 = !{!"branch_weights", i64 214748366, i64 214748366, i64 1717986918}
; CHECK: !5 = !{!"misexpect", i64 2, i64 1717986918, i64 214748366} ; CHECK: !5 = !{!"misexpect", i64 2, i64 1717986918, i64 214748366}
; CHECK: !6 = !{!"branch_weights", i32 1717986918, i32 214748366, i32 214748366} ; CHECK: !6 = !{!"branch_weights", i64 1717986918, i64 214748366, i64 214748366}
; CHECK: !7 = !{!"misexpect", i64 0, i64 1717986918, i64 214748366} ; CHECK: !7 = !{!"misexpect", i64 0, i64 1717986918, i64 214748366}

llvm/test/Transforms/LowerExpectIntrinsic/expect_nonboolean.ll

Show First 20 Lines • Show All 93 Lines • ▼ Show 20 Lines
attributes #0 = { nounwind uwtable } attributes #0 = { nounwind uwtable }
attributes #1 = { nounwind readnone } attributes #1 = { nounwind readnone }
!llvm.module.flags = !{!0} !llvm.module.flags = !{!0}
!llvm.ident = !{!1} !llvm.ident = !{!1}
!0 = !{i32 1, !"wchar_size", i32 4} !0 = !{i32 1, !"wchar_size", i32 4}
!1 = !{!"clang version 5.0.0 (trunk 304373)"} !1 = !{!"clang version 5.0.0 (trunk 304373)"}
; CHECK: [[LIKELY]] = !{!"branch_weights", i32 2000, i32 1} ; CHECK: [[LIKELY]] = !{!"branch_weights", i64 2000, i64 1}
; CHECK: [[UNLIKELY]] = !{!"branch_weights", i32 1, i32 2000} ; CHECK: [[UNLIKELY]] = !{!"branch_weights", i64 1, i64 2000}

llvm/test/Transforms/LowerExpectIntrinsic/phi_merge.ll

Show First 20 Lines • Show All 346 Lines • ▼ Show 20 Lines
declare i8 @too8() declare i8 @too8()
; Function Attrs: nounwind readnone ; Function Attrs: nounwind readnone
declare i64 @llvm.expect.i64(i64, i64) declare i64 @llvm.expect.i64(i64, i64)
!llvm.ident = !{!0} !llvm.ident = !{!0}
!0 = !{!"clang version 5.0.0 (trunk 302965)"} !0 = !{!"clang version 5.0.0 (trunk 302965)"}
; CHECK: [[WEIGHT]] = !{!"branch_weights", i32 2000, i32 1} ; CHECK: [[WEIGHT]] = !{!"branch_weights", i64 2000, i64 1}
; CHECK: [[WEIGHT2]] = !{!"branch_weights", i32 1, i32 2000} ; CHECK: [[WEIGHT2]] = !{!"branch_weights", i64 1, i64 2000}

llvm/test/Transforms/LowerExpectIntrinsic/phi_or.ll

Show First 20 Lines • Show All 93 Lines • ▼ Show 20 Lines
declare i32 @hoo() declare i32 @hoo()
declare i64 @llvm.expect.i64(i64, i64) declare i64 @llvm.expect.i64(i64, i64)
!llvm.ident = !{!0} !llvm.ident = !{!0}
!0 = !{!"clang version 5.0.0 (trunk 302965)"} !0 = !{!"clang version 5.0.0 (trunk 302965)"}
; CHECK: [[WEIGHT]] = !{!"branch_weights", i32 2000, i32 1} ; CHECK: [[WEIGHT]] = !{!"branch_weights", i64 2000, i64 1}
; CHECK: [[WEIGHT2]] = !{!"branch_weights", i32 1, i32 2000} ; CHECK: [[WEIGHT2]] = !{!"branch_weights", i64 1, i64 2000}

llvm/test/Transforms/LowerExpectIntrinsic/phi_tern.ll

Show First 20 Lines • Show All 47 Lines • ▼ Show 20 Lines
declare i64 @llvm.expect.i64(i64, i64) declare i64 @llvm.expect.i64(i64, i64)
!llvm.ident = !{!0} !llvm.ident = !{!0}
!0 = !{!"clang version 5.0.0 (trunk 302965)"} !0 = !{!"clang version 5.0.0 (trunk 302965)"}
; CHECK: [[WEIGHT]] = !{!"branch_weights", i32 1, i32 2000} ; CHECK: [[WEIGHT]] = !{!"branch_weights", i64 1, i64 2000}

llvm/test/Transforms/PGOProfile/branch1.ll

Show All 31 Lines
; USE-SAME: !prof ![[FUNC_ENTRY_COUNT:[0-9]+]] ; USE-SAME: !prof ![[FUNC_ENTRY_COUNT:[0-9]+]]
entry: entry:
; GEN: entry: ; GEN: entry:
; GEN: call void @llvm.instrprof.increment(i8* getelementptr inbounds ([9 x i8], [9 x i8]* @__profn_test_br_1, i32 0, i32 0), i64 {{[0-9]+}}, i32 2, i32 0) ; GEN: call void @llvm.instrprof.increment(i8* getelementptr inbounds ([9 x i8], [9 x i8]* @__profn_test_br_1, i32 0, i32 0), i64 {{[0-9]+}}, i32 2, i32 0)
%cmp = icmp sgt i32 %i, 0 %cmp = icmp sgt i32 %i, 0
br i1 %cmp, label %if.then, label %if.end br i1 %cmp, label %if.then, label %if.end
; USE: br i1 %cmp, label %if.then, label %if.end ; USE: br i1 %cmp, label %if.then, label %if.end
; USE-SAME: !prof ![[BW_ENTRY:[0-9]+]] ; USE-SAME: !prof ![[BW_ENTRY:[0-9]+]]
; USE-DAG: ![[BW_ENTRY]] = !{!"branch_weights", i32 2, i32 1} ; USE-DAG: ![[BW_ENTRY]] = !{!"branch_weights", i64 2, i64 1}
; USE-LARGE: br i1 %cmp, label %if.then, label %if.end ; USE-LARGE: br i1 %cmp, label %if.then, label %if.end
; USE-LARGE-SAME: !prof ![[BW_L_ENTRY:[0-9]+]] ; USE-LARGE-SAME: !prof ![[BW_L_ENTRY:[0-9]+]]
; USE-LARGE-DAG: ![[BW_L_ENTRY]] = !{!"branch_weights", i32 -1431655766, i32 1431655765} ; USE-LARGE-DAG: ![[BW_L_ENTRY]] = !{!"branch_weights", i64 2863311530, i64 1431655765}
if.then: if.then:
; GEN: if.then: ; GEN: if.then:
; GEN: call void @llvm.instrprof.increment(i8* getelementptr inbounds ([9 x i8], [9 x i8]* @__profn_test_br_1, i32 0, i32 0), i64 {{[0-9]+}}, i32 2, i32 1) ; GEN: call void @llvm.instrprof.increment(i8* getelementptr inbounds ([9 x i8], [9 x i8]* @__profn_test_br_1, i32 0, i32 0), i64 {{[0-9]+}}, i32 2, i32 1)
%add = add nsw i32 %i, 2 %add = add nsw i32 %i, 2
br label %if.end br label %if.end
if.end: if.end:
Show All 11 Lines

llvm/test/Transforms/PGOProfile/branch2.ll

Show All 19 Lines
entry: entry:
; GEN: entry: ; GEN: entry:
; NOTENTRY-NOT: llvm.instrprof.increment ; NOTENTRY-NOT: llvm.instrprof.increment
; ENTRY: call void @llvm.instrprof.increment(i8* getelementptr inbounds ([9 x i8], [9 x i8]* @__profn_test_br_2, i32 0, i32 0), i64 {{[0-9]+}}, i32 2, i32 0) ; ENTRY: call void @llvm.instrprof.increment(i8* getelementptr inbounds ([9 x i8], [9 x i8]* @__profn_test_br_2, i32 0, i32 0), i64 {{[0-9]+}}, i32 2, i32 0)
%cmp = icmp sgt i32 %i, 0 %cmp = icmp sgt i32 %i, 0
br i1 %cmp, label %if.then, label %if.else br i1 %cmp, label %if.then, label %if.else
; USE: br i1 %cmp, label %if.then, label %if.else ; USE: br i1 %cmp, label %if.then, label %if.else
; USE-SAME: !prof ![[BW_ENTRY:[0-9]+]] ; USE-SAME: !prof ![[BW_ENTRY:[0-9]+]]
; USE: ![[BW_ENTRY]] = !{!"branch_weights", i32 1, i32 1} ; USE: ![[BW_ENTRY]] = !{!"branch_weights", i64 1, i64 1}
if.then: if.then:
; GEN: if.then: ; GEN: if.then:
; NOTENTRY: call void @llvm.instrprof.increment(i8* getelementptr inbounds ([9 x i8], [9 x i8]* @__profn_test_br_2, i32 0, i32 0), i64 {{[0-9]+}}, i32 2, i32 0) ; NOTENTRY: call void @llvm.instrprof.increment(i8* getelementptr inbounds ([9 x i8], [9 x i8]* @__profn_test_br_2, i32 0, i32 0), i64 {{[0-9]+}}, i32 2, i32 0)
; ENTRY-NOT: llvm.instrprof.increment ; ENTRY-NOT: llvm.instrprof.increment
%add = add nsw i32 %i, 2 %add = add nsw i32 %i, 2
br label %if.end br label %if.end
Show All 13 Lines

llvm/test/Transforms/PGOProfile/chr.ll

Show First 20 Lines • Show All 2,537 Lines • ▼ Show 20 Lines
!8 = !{!"NumFunctions", i64 3} !8 = !{!"NumFunctions", i64 3}
!9 = !{!"DetailedSummary", !10} !9 = !{!"DetailedSummary", !10}
!10 = !{!11, !12, !13} !10 = !{!11, !12, !13}
!11 = !{i32 10000, i64 100, i32 1} !11 = !{i32 10000, i64 100, i32 1}
!12 = !{i32 999000, i64 100, i32 1} !12 = !{i32 999000, i64 100, i32 1}
!13 = !{i32 999999, i64 1, i32 2} !13 = !{i32 999999, i64 1, i32 2}
!14 = !{!"function_entry_count", i64 100} !14 = !{!"function_entry_count", i64 100}
!15 = !{!"branch_weights", i32 0, i32 1} !15 = !{!"branch_weights", i64 0, i64 1}
!16 = !{!"branch_weights", i32 1, i32 1} !16 = !{!"branch_weights", i64 1, i64 1}
!17 = !{!"branch_weights", i32 0, i32 0} !17 = !{!"branch_weights", i64 0, i64 0}
; CHECK: !15 = !{!"branch_weights", i32 1000, i32 0} ; CHECK: !15 = !{!"branch_weights", i64 1000, i64 0}
; CHECK: !16 = !{!"branch_weights", i32 0, i32 1} ; CHECK: !16 = !{!"branch_weights", i64 0, i64 1}
; CHECK: !17 = !{!"branch_weights", i32 1, i32 1} ; CHECK: !17 = !{!"branch_weights", i64 1, i64 1}
; CHECK: !18 = !{!"branch_weights", i32 1, i32 0} ; CHECK: !18 = !{!"branch_weights", i64 1, i64 0}
; CHECK: !19 = !{!"branch_weights", i32 0, i32 1000} ; CHECK: !19 = !{!"branch_weights", i64 0, i64 1000}

llvm/test/Transforms/PGOProfile/criticaledge.ll

Show First 20 Lines • Show All 111 Lines • ▼ Show 20 Lines
} }
define internal i32 @bar(i32 %i) { define internal i32 @bar(i32 %i) {
entry: entry:
; GEN: call void @llvm.instrprof.increment(i8* getelementptr inbounds ([11 x i8], [11 x i8]* @__profn__stdin__bar, i32 0, i32 0), i64 {{[0-9]+}}, i32 1, i32 0) ; GEN: call void @llvm.instrprof.increment(i8* getelementptr inbounds ([11 x i8], [11 x i8]* @__profn__stdin__bar, i32 0, i32 0), i64 {{[0-9]+}}, i32 1, i32 0)
ret i32 %i ret i32 %i
} }
; USE: ![[BW_SWITCH]] = !{!"branch_weights", i32 2, i32 1, i32 0, i32 2, i32 1, i32 1} ; USE: ![[BW_SWITCH]] = !{!"branch_weights", i64 2, i64 1, i64 0, i64 2, i64 1, i64 1}
; USE: ![[BW_SW_BB2]] = !{!"branch_weights", i32 2, i32 2} ; USE: ![[BW_SW_BB2]] = !{!"branch_weights", i64 2, i64 2}
; USE: ![[BW_SW_DEFAULT]] = !{!"branch_weights", i32 1, i32 1} ; USE: ![[BW_SW_DEFAULT]] = !{!"branch_weights", i64 1, i64 1}

llvm/test/Transforms/PGOProfile/cspgo_profile_summary.ll

Show First 20 Lines • Show All 126 Lines • ▼ Show 20 Lines
; PGOSUMMARY: {{![0-9]+}} = !{i32 1, !"ProfileSummary", !{{[0-9]+}}} ; PGOSUMMARY: {{![0-9]+}} = !{i32 1, !"ProfileSummary", !{{[0-9]+}}}
; PGOSUMMARY: {{![0-9]+}} = !{!"ProfileFormat", !"InstrProf"} ; PGOSUMMARY: {{![0-9]+}} = !{!"ProfileFormat", !"InstrProf"}
; PGOSUMMARY: {{![0-9]+}} = !{!"TotalCount", i64 2100001} ; PGOSUMMARY: {{![0-9]+}} = !{!"TotalCount", i64 2100001}
; PGOSUMMARY: {{![0-9]+}} = !{!"MaxCount", i64 800000} ; PGOSUMMARY: {{![0-9]+}} = !{!"MaxCount", i64 800000}
; PGOSUMMARY: {{![0-9]+}} = !{!"MaxInternalCount", i64 399999} ; PGOSUMMARY: {{![0-9]+}} = !{!"MaxInternalCount", i64 399999}
; PGOSUMMARY: {{![0-9]+}} = !{!"MaxFunctionCount", i64 800000} ; PGOSUMMARY: {{![0-9]+}} = !{!"MaxFunctionCount", i64 800000}
; PGOSUMMARY: {{![0-9]+}} = !{!"NumCounts", i64 14} ; PGOSUMMARY: {{![0-9]+}} = !{!"NumCounts", i64 14}
; PGOSUMMARY: {{![0-9]+}} = !{!"NumFunctions", i64 8} ; PGOSUMMARY: {{![0-9]+}} = !{!"NumFunctions", i64 8}
; PGOSUMMARY-DAG: ![[BW_PGO_BAR]] = !{!"branch_weights", i32 100000, i32 100000} ; PGOSUMMARY-DAG: ![[BW_PGO_BAR]] = !{!"branch_weights", i64 100000, i64 100000}
; CSPGOSUMMARY: {{![0-9]+}} = !{i32 1, !"ProfileSummary", !1} ; CSPGOSUMMARY: {{![0-9]+}} = !{i32 1, !"ProfileSummary", !1}
; CSPGOSUMMARY: {{![0-9]+}} = !{!"ProfileFormat", !"InstrProf"} ; CSPGOSUMMARY: {{![0-9]+}} = !{!"ProfileFormat", !"InstrProf"}
; CSPGOSUMMARY: {{![0-9]+}} = !{!"TotalCount", i64 2100001} ; CSPGOSUMMARY: {{![0-9]+}} = !{!"TotalCount", i64 2100001}
; CSPGOSUMMARY: {{![0-9]+}} = !{!"MaxCount", i64 800000} ; CSPGOSUMMARY: {{![0-9]+}} = !{!"MaxCount", i64 800000}
; CSPGOSUMMARY: {{![0-9]+}} = !{!"MaxInternalCount", i64 399999} ; CSPGOSUMMARY: {{![0-9]+}} = !{!"MaxInternalCount", i64 399999}
; CSPGOSUMMARY: {{![0-9]+}} = !{!"MaxFunctionCount", i64 800000} ; CSPGOSUMMARY: {{![0-9]+}} = !{!"MaxFunctionCount", i64 800000}
; CSPGOSUMMARY: {{![0-9]+}} = !{!"NumCounts", i64 14} ; CSPGOSUMMARY: {{![0-9]+}} = !{!"NumCounts", i64 14}
; CSPGOSUMMARY: {{![0-9]+}} = !{!"NumFunctions", i64 8} ; CSPGOSUMMARY: {{![0-9]+}} = !{!"NumFunctions", i64 8}
; CSPGOSUMMARY: {{![0-9]+}} = !{!"DetailedSummary", !{{[0-9]+}}} ; CSPGOSUMMARY: {{![0-9]+}} = !{!"DetailedSummary", !{{[0-9]+}}}
; CSPGOSUMMARY: {{![0-9]+}} = !{i32 1, !"CSProfileSummary", !{{[0-9]+}}} ; CSPGOSUMMARY: {{![0-9]+}} = !{i32 1, !"CSProfileSummary", !{{[0-9]+}}}
; CSPGOSUMMARY: {{![0-9]+}} = !{!"ProfileFormat", !"CSInstrProf"} ; CSPGOSUMMARY: {{![0-9]+}} = !{!"ProfileFormat", !"CSInstrProf"}
; CSPGOSUMMARY: {{![0-9]+}} = !{!"TotalCount", i64 1299950} ; CSPGOSUMMARY: {{![0-9]+}} = !{!"TotalCount", i64 1299950}
; CSPGOSUMMARY: {{![0-9]+}} = !{!"MaxCount", i64 200000} ; CSPGOSUMMARY: {{![0-9]+}} = !{!"MaxCount", i64 200000}
; CSPGOSUMMARY: {{![0-9]+}} = !{!"MaxInternalCount", i64 100000} ; CSPGOSUMMARY: {{![0-9]+}} = !{!"MaxInternalCount", i64 100000}
; CSPGOSUMMARY: {{![0-9]+}} = !{!"MaxFunctionCount", i64 200000} ; CSPGOSUMMARY: {{![0-9]+}} = !{!"MaxFunctionCount", i64 200000}
; CSPGOSUMMARY: {{![0-9]+}} = !{!"NumCounts", i64 23} ; CSPGOSUMMARY: {{![0-9]+}} = !{!"NumCounts", i64 23}
; CSPGOSUMMARY-DAG: ![[BW_CSPGO_BAR]] = !{!"branch_weights", i32 100000, i32 100000} ; CSPGOSUMMARY-DAG: ![[BW_CSPGO_BAR]] = !{!"branch_weights", i64 100000, i64 100000}
; CSPGOSUMMARY-DAG: ![[BW1_CSPGO_FOO]] = !{!"branch_weights", i32 100000, i32 0} ; CSPGOSUMMARY-DAG: ![[BW1_CSPGO_FOO]] = !{!"branch_weights", i64 100000, i64 0}
; CSPGOSUMMARY-DAG: ![[BW2_CSPGO_FOO]] = !{!"branch_weights", i32 0, i32 100000} ; CSPGOSUMMARY-DAG: ![[BW2_CSPGO_FOO]] = !{!"branch_weights", i64 0, i64 100000}

llvm/test/Transforms/PGOProfile/fix_entry_count.ll

Show All 25 Lines
for.inc: for.inc:
%inc1 = add nsw i32 %i, 1 %inc1 = add nsw i32 %i, 1
br label %for.cond br label %for.cond
for.end: for.end:
ret i32 %sum ret i32 %sum
} }
; USE: ![[ENTRY_COUNT]] = !{!"function_entry_count", i64 1} ; USE: ![[ENTRY_COUNT]] = !{!"function_entry_count", i64 1}
; USE: ![[BW_FOR_COND]] = !{!"branch_weights", i32 96, i32 1} ; USE: ![[BW_FOR_COND]] = !{!"branch_weights", i64 96, i64 1}

llvm/test/Transforms/PGOProfile/icp_covariant_call_return.ll

Show All 34 Lines
; ICALL-PROM:if.end.icp: ; ICALL-PROM:if.end.icp:
; ICALL-PROM: [[PHI_RET:%[0-9]+]] = phi %struct.Base* [ %call1, %if.false.orig_indirect ], [ [[DIRCALL_RET_CAST]], %if.true.direct_targ ] ; ICALL-PROM: [[PHI_RET:%[0-9]+]] = phi %struct.Base* [ %call1, %if.false.orig_indirect ], [ [[DIRCALL_RET_CAST]], %if.true.direct_targ ]
%call1 = call %struct.Base* %tmp3(%struct.B* %tmp1), !prof !1 %call1 = call %struct.Base* %tmp3(%struct.B* %tmp1), !prof !1
ret %struct.Base* %call1 ret %struct.Base* %call1
} }
!1 = !{!"VP", i32 0, i64 12345, i64 -3913987384944532146, i64 12345} !1 = !{!"VP", i32 0, i64 12345, i64 -3913987384944532146, i64 12345}
; ICALL-PROM-NOT: !1 = !{!"VP", i32 0, i64 12345, i64 -3913987384944532146, i64 12345} ; ICALL-PROM-NOT: !1 = !{!"VP", i32 0, i64 12345, i64 -3913987384944532146, i64 12345}
; ICALL-PROM: [[BRANCH_WEIGHT]] = !{!"branch_weights", i32 12345, i32 0} ; ICALL-PROM: [[BRANCH_WEIGHT]] = !{!"branch_weights", i64 12345, i64 0}
; ICALL-PROM-NOT: !1 = !{!"VP", i32 0, i64 12345, i64 -3913987384944532146, i64 12345} ; ICALL-PROM-NOT: !1 = !{!"VP", i32 0, i64 12345, i64 -3913987384944532146, i64 12345}

llvm/test/Transforms/PGOProfile/icp_covariant_invoke_return.ll

Show First 20 Lines • Show All 102 Lines • ▼ Show 20 Lines
eh.resume: eh.resume:
%lpad.val = insertvalue { i8*, i32 } undef, i8* %tmp8, 0 %lpad.val = insertvalue { i8*, i32 } undef, i8* %tmp8, 0
%lpad.val11 = insertvalue { i8*, i32 } %lpad.val, i32 %tmp9, 1 %lpad.val11 = insertvalue { i8*, i32 } %lpad.val, i32 %tmp9, 1
resume { i8*, i32 } %lpad.val11 resume { i8*, i32 } %lpad.val11
} }
!1 = !{!"VP", i32 0, i64 12345, i64 -3913987384944532146, i64 12345} !1 = !{!"VP", i32 0, i64 12345, i64 -3913987384944532146, i64 12345}
; ICALL-PROM-NOT: !1 = !{!"VP", i32 0, i64 12345, i64 -3913987384944532146, i64 12345} ; ICALL-PROM-NOT: !1 = !{!"VP", i32 0, i64 12345, i64 -3913987384944532146, i64 12345}
; ICALL-PROM: [[BRANCH_WEIGHT]] = !{!"branch_weights", i32 12345, i32 0} ; ICALL-PROM: [[BRANCH_WEIGHT]] = !{!"branch_weights", i64 12345, i64 0}
; ICALL-PROM-NOT: !1 = !{!"VP", i32 0, i64 12345, i64 -3913987384944532146, i64 12345} ; ICALL-PROM-NOT: !1 = !{!"VP", i32 0, i64 12345, i64 -3913987384944532146, i64 12345}

llvm/test/Transforms/PGOProfile/icp_invoke.ll

Show First 20 Lines • Show All 97 Lines • ▼ Show 20 Lines
declare void @__cxa_end_catch() declare void @__cxa_end_catch()
!0 = !{!"invoke.ll:_ZL4bar1v"} !0 = !{!"invoke.ll:_ZL4bar1v"}
!1 = !{!"invoke.ll:_ZL4bar2v"} !1 = !{!"invoke.ll:_ZL4bar2v"}
!2 = !{!"VP", i32 0, i64 1, i64 -2732222848796217051, i64 1} !2 = !{!"VP", i32 0, i64 1, i64 -2732222848796217051, i64 1}
!3 = !{!"VP", i32 0, i64 1, i64 -6116256810522035449, i64 1} !3 = !{!"VP", i32 0, i64 1, i64 -6116256810522035449, i64 1}
; ICP-NOT: !3 = !{!"VP", i32 0, i64 1, i64 -2732222848796217051, i64 1} ; ICP-NOT: !3 = !{!"VP", i32 0, i64 1, i64 -2732222848796217051, i64 1}
; ICP-NOT: !4 = !{!"VP", i32 0, i64 1, i64 -6116256810522035449, i64 1} ; ICP-NOT: !4 = !{!"VP", i32 0, i64 1, i64 -6116256810522035449, i64 1}
; ICP: [[BRANCH_WEIGHT]] = !{!"branch_weights", i32 1, i32 0} ; ICP: [[BRANCH_WEIGHT]] = !{!"branch_weights", i64 1, i64 0}

llvm/test/Transforms/PGOProfile/icp_vararg.ll

Show All 24 Lines; ICALL-PROM: br label %if.end.icp
ret i32 %call ret i32 %call
; ICALL-PROM:if.end.icp: ; ICALL-PROM:if.end.icp:
; ICALL-PROM: [[PHI_RET:%[0-9]+]] = phi i32 [ %call, %if.false.orig_indirect ], [ [[DIRCALL_RET]], %if.true.direct_targ ] ; ICALL-PROM: [[PHI_RET:%[0-9]+]] = phi i32 [ %call, %if.false.orig_indirect ], [ [[DIRCALL_RET]], %if.true.direct_targ ]
; ICALL-PROM: ret i32 [[PHI_RET]] ; ICALL-PROM: ret i32 [[PHI_RET]]
} }
!1 = !{!"VP", i32 0, i64 12345, i64 989055279648259519, i64 12345} !1 = !{!"VP", i32 0, i64 12345, i64 989055279648259519, i64 12345}
; ICALL-PROM: [[BRANCH_WEIGHT]] = !{!"branch_weights", i32 12345, i32 0} ; ICALL-PROM: [[BRANCH_WEIGHT]] = !{!"branch_weights", i64 12345, i64 0}

llvm/test/Transforms/PGOProfile/indirect_call_promotion.ll

Show First 20 Lines • Show All 54 Lines • ▼ Show 20 Lines; ICALL-PROM: %call = call i32 %tmp(), !prof [[NEW_VP_METADATA:![0-9]+]]
ret i32 %call ret i32 %call
; ICALL-PROM: if.end.icp: ; ICALL-PROM: if.end.icp:
; ICALL-PROM: [[PHI_RET:%[0-9]+]] = phi i32 [ %call, %if.false.orig_indirect ], [ [[DIRCALL_RET]], %if.true.direct_targ ] ; ICALL-PROM: [[PHI_RET:%[0-9]+]] = phi i32 [ %call, %if.false.orig_indirect ], [ [[DIRCALL_RET]], %if.true.direct_targ ]
; ICALL-PROM: ret i32 [[PHI_RET]] ; ICALL-PROM: ret i32 [[PHI_RET]]
} }
!1 = !{!"VP", i32 0, i64 1600, i64 7651369219802541373, i64 1030, i64 -4377547752858689819, i64 410, i64 -6929281286627296573, i64 150, i64 -2545542355363006406, i64 10} !1 = !{!"VP", i32 0, i64 1600, i64 7651369219802541373, i64 1030, i64 -4377547752858689819, i64 410, i64 -6929281286627296573, i64 150, i64 -2545542355363006406, i64 10}
; ICALL-PROM: [[BRANCH_WEIGHT]] = !{!"branch_weights", i32 1030, i32 570} ; ICALL-PROM: [[BRANCH_WEIGHT]] = !{!"branch_weights", i64 1030, i64 570}
; ICALL-PROM: [[NEW_VP_METADATA]] = !{!"VP", i32 0, i64 570, i64 -4377547752858689819, i64 410} ; ICALL-PROM: [[NEW_VP_METADATA]] = !{!"VP", i32 0, i64 570, i64 -4377547752858689819, i64 410}
; ICALL-PROM-SAMPLEPGO: [[CALL_METADATA]] = !{!"branch_weights", i32 1030} ; ICALL-PROM-SAMPLEPGO: [[CALL_METADATA]] = !{!"branch_weights", i64 1030}

llvm/test/Transforms/PGOProfile/indirect_call_promotion_musttail.ll

Show First 20 Lines • Show All 56 Lines • ▼ Show 20 Lines; ICALL-PROM: ret i64* %rv
%call = musttail call i32* %tmp(), !prof !2 %call = musttail call i32* %tmp(), !prof !2
%rv = bitcast i32* %call to i64* %rv = bitcast i32* %call to i64*
ret i64* %rv ret i64* %rv
} }
!1 = !{!"VP", i32 0, i64 1600, i64 7651369219802541373, i64 1030, i64 -4377547752858689819, i64 410, i64 -6929281286627296573, i64 150, i64 -2545542355363006406, i64 10} !1 = !{!"VP", i32 0, i64 1600, i64 7651369219802541373, i64 1030, i64 -4377547752858689819, i64 410, i64 -6929281286627296573, i64 150, i64 -2545542355363006406, i64 10}
!2 = !{!"VP", i32 0, i64 100, i64 7651369219802541373, i64 100} !2 = !{!"VP", i32 0, i64 100, i64 7651369219802541373, i64 100}
; ICALL-PROM: [[BRANCH_WEIGHT1]] = !{!"branch_weights", i32 1030, i32 570} ; ICALL-PROM: [[BRANCH_WEIGHT1]] = !{!"branch_weights", i64 1030, i64 570}
; ICALL-PROM: [[BRANCH_WEIGHT2]] = !{!"branch_weights", i32 410, i32 160} ; ICALL-PROM: [[BRANCH_WEIGHT2]] = !{!"branch_weights", i64 410, i64 160}
; ICALL-PROM: [[BRANCH_WEIGHT3]] = !{!"branch_weights", i32 150, i32 10} ; ICALL-PROM: [[BRANCH_WEIGHT3]] = !{!"branch_weights", i64 150, i64 10}
; ICALL-PROM: [[BRANCH_WEIGHT4]] = !{!"branch_weights", i32 100, i32 0} ; ICALL-PROM: [[BRANCH_WEIGHT4]] = !{!"branch_weights", i64 100, i64 0}

llvm/test/Transforms/PGOProfile/indirect_call_promotion_vla.ll

Show All 18 Lines
; CHECK: br i1 [[CMP]], label %if.true.direct_targ, label %if.false.orig_indirect, !prof [[BRANCH_WEIGHT:![0-9]+]] ; CHECK: br i1 [[CMP]], label %if.true.direct_targ, label %if.false.orig_indirect, !prof [[BRANCH_WEIGHT:![0-9]+]]
; CHECK: if.true.direct_targ: ; CHECK: if.true.direct_targ:
; CHECK: [[DIRCALL_RET:%[0-9]+]] = call i32 (%struct.B*, ...) @func1 ; CHECK: [[DIRCALL_RET:%[0-9]+]] = call i32 (%struct.B*, ...) @func1
; CHECK: br label %if.end.icp ; CHECK: br label %if.end.icp
%call = call i32 (%struct.A*, ...) %tmp(%struct.A* %x, i32 0), !prof !1 %call = call i32 (%struct.A*, ...) %tmp(%struct.A* %x, i32 0), !prof !1
ret i32 %call ret i32 %call
} }
; CHECK: [[BRANCH_WEIGHT]] = !{!"branch_weights", i32 1500, i32 100} ; CHECK: [[BRANCH_WEIGHT]] = !{!"branch_weights", i64 1500, i64 100}
!1 = !{!"VP", i32 0, i64 1600, i64 -2545542355363006406, i64 1500} !1 = !{!"VP", i32 0, i64 1600, i64 -2545542355363006406, i64 1500}

llvm/test/Transforms/PGOProfile/indirectbr.ll

Show All 20 Lines
if.then: if.then:
%idxprom = zext i32 %i to i64 %idxprom = zext i32 %i to i64
%arrayidx = getelementptr inbounds [3 x i8*], [3 x i8*]* @foo.table, i64 0, i64 %idxprom %arrayidx = getelementptr inbounds [3 x i8*], [3 x i8*]* @foo.table, i64 0, i64 %idxprom
%0 = load i8*, i8** %arrayidx, align 8 %0 = load i8*, i8** %arrayidx, align 8
indirectbr i8* %0, [label %return, label %label2, label %label3] indirectbr i8* %0, [label %return, label %label2, label %label3]
; USE: indirectbr i8* %0, [label %return, label %label2, label %label3] ; USE: indirectbr i8* %0, [label %return, label %label2, label %label3]
; USE-SAME: !prof ![[BW_INDBR:[0-9]+]] ; USE-SAME: !prof ![[BW_INDBR:[0-9]+]]
; USE: ![[BW_INDBR]] = !{!"branch_weights", i32 63, i32 20, i32 5} ; USE: ![[BW_INDBR]] = !{!"branch_weights", i64 63, i64 20, i64 5}
label2: label2:
br label %return br label %return
label3: label3:
br label %return br label %return
return: return:
Show All 18 Lines

llvm/test/Transforms/PGOProfile/instr_entry_bb.ll

Show All 15 Lines
define i32 @test_br_2(i32 %i) { define i32 @test_br_2(i32 %i) {
entry: entry:
; GEN: entry: ; GEN: entry:
; GEN: call void @llvm.instrprof.increment(i8* getelementptr inbounds ([9 x i8], [9 x i8]* @__profn_test_br_2, i32 0, i32 0), i64 {{[0-9]+}}, i32 2, i32 0) ; GEN: call void @llvm.instrprof.increment(i8* getelementptr inbounds ([9 x i8], [9 x i8]* @__profn_test_br_2, i32 0, i32 0), i64 {{[0-9]+}}, i32 2, i32 0)
; GENA: entry: ; GENA: entry:
; GENA: %{{[0-9+]}} = atomicrmw add i64* getelementptr inbounds ([2 x i64], [2 x i64]* @__profc_test_br_2, i64 0, i64 0), i64 1 monotonic ; GENA: %{{[0-9+]}} = atomicrmw add i64* getelementptr inbounds ([2 x i64], [2 x i64]* @__profc_test_br_2, i64 0, i64 0), i64 1 monotonic
; USE: br i1 %cmp, label %if.then, label %if.else ; USE: br i1 %cmp, label %if.then, label %if.else
; USE-SAME: !prof ![[BW_ENTRY:[0-9]+]] ; USE-SAME: !prof ![[BW_ENTRY:[0-9]+]]
; USE: ![[BW_ENTRY]] = !{!"branch_weights", i32 0, i32 1} ; USE: ![[BW_ENTRY]] = !{!"branch_weights", i64 0, i64 1}
%cmp = icmp sgt i32 %i, 0 %cmp = icmp sgt i32 %i, 0
br i1 %cmp, label %if.then, label %if.else br i1 %cmp, label %if.then, label %if.else
if.then: if.then:
; GEN: if.then: ; GEN: if.then:
; GEN-NOT: llvm.instrprof.increment ; GEN-NOT: llvm.instrprof.increment
%add = add nsw i32 %i, 2 %add = add nsw i32 %i, 2
br label %if.end br label %if.end
Show All 18 Lines

llvm/test/Transforms/PGOProfile/landingpad.ll

Show First 20 Lines • Show All 126 Lines • ▼ Show 20 Lines
declare i32 @__gxx_personality_v0(...) declare i32 @__gxx_personality_v0(...)
declare i32 @llvm.eh.typeid.for(i8*) declare i32 @llvm.eh.typeid.for(i8*)
declare i8* @__cxa_begin_catch(i8*) declare i8* @__cxa_begin_catch(i8*)
declare void @__cxa_end_catch() declare void @__cxa_end_catch()
; USE: ![[BW_BAR_ENTRY]] = !{!"branch_weights", i32 2, i32 1} ; USE: ![[BW_BAR_ENTRY]] = !{!"branch_weights", i64 2, i64 1}
; USE: ![[BW_FOO_ENTRY]] = !{!"branch_weights", i32 3, i32 2} ; USE: ![[BW_FOO_ENTRY]] = !{!"branch_weights", i64 3, i64 2}
; USE: ![[BW_CATCH_DISPATCH]] = !{!"branch_weights", i32 2, i32 0} ; USE: ![[BW_CATCH_DISPATCH]] = !{!"branch_weights", i64 2, i64 0}

llvm/test/Transforms/PGOProfile/loop1.ll

Show All 26 Lines
; GEN: for.cond: ; GEN: for.cond:
; GEN-NOT: call void @llvm.instrprof.increment ; GEN-NOT: call void @llvm.instrprof.increment
%i = phi i32 [ 0, %entry ], [ %inc1, %for.inc ] %i = phi i32 [ 0, %entry ], [ %inc1, %for.inc ]
%sum = phi i32 [ 1, %entry ], [ %inc, %for.inc ] %sum = phi i32 [ 1, %entry ], [ %inc, %for.inc ]
%cmp = icmp slt i32 %i, %n %cmp = icmp slt i32 %i, %n
br i1 %cmp, label %for.body, label %for.end br i1 %cmp, label %for.body, label %for.end
; USE: br i1 %cmp, label %for.body, label %for.end ; USE: br i1 %cmp, label %for.body, label %for.end
; USE-SAME: !prof ![[BW_FOR_COND:[0-9]+]] ; USE-SAME: !prof ![[BW_FOR_COND:[0-9]+]]
; USE: ![[BW_FOR_COND]] = !{!"branch_weights", i32 96, i32 4} ; USE: ![[BW_FOR_COND]] = !{!"branch_weights", i64 96, i64 4}
for.body: for.body:
; GEN: for.body: ; GEN: for.body:
; GEN-NOT: call void @llvm.instrprof.increment ; GEN-NOT: call void @llvm.instrprof.increment
%inc = add nsw i32 %sum, 1 %inc = add nsw i32 %sum, 1
br label %for.inc br label %for.inc
for.inc: for.inc:
Show All 12 Lines

llvm/test/Transforms/PGOProfile/loop2.ll

Show First 20 Lines • Show All 72 Lines • ▼ Show 20 Lines
for.end.outer: for.end.outer:
; GEN: for.end.outer: ; GEN: for.end.outer:
; GEN-NOT: call void @llvm.instrprof.increment ; GEN-NOT: call void @llvm.instrprof.increment
; GEN: ret i32 ; GEN: ret i32
ret i32 %sum.0 ret i32 %sum.0
} }
; USE-DAG: ![[BW_FOR_COND_OUTER]] = !{!"branch_weights", i32 10, i32 6} ; USE-DAG: ![[BW_FOR_COND_OUTER]] = !{!"branch_weights", i64 10, i64 6}
; USE-DAG: ![[BW_FOR_COND_INNER]] = !{!"branch_weights", i32 33, i32 10} ; USE-DAG: ![[BW_FOR_COND_INNER]] = !{!"branch_weights", i64 33, i64 10}

llvm/test/Transforms/PGOProfile/memop_size_opt.ll

Show First 20 Lines • Show All 137 Lines • ▼ Show 20 Lines
for.inc4: for.inc4:
%inc5 = add nsw i32 %i.0, 1 %inc5 = add nsw i32 %i.0, 1
br label %for.cond br label %for.cond
for.end6: for.end6:
ret void ret void
} }
; MEMOP_OPT: [[SWITCH_BW]] = !{!"branch_weights", i32 457, i32 99} ; MEMOP_OPT: [[SWITCH_BW]] = !{!"branch_weights", i64 457, i64 99}
; Should be 457 total left (original total count 556, minus 99 from specialized ; Should be 457 total left (original total count 556, minus 99 from specialized
; value 1, which is removed from VP array. Also, we only end up with 5 total ; value 1, which is removed from VP array. Also, we only end up with 5 total
; values, since the default max number of promotions is 5 and therefore ; values, since the default max number of promotions is 5 and therefore
; the rest of the values are ignored when extracting the VP metadata. ; the rest of the values are ignored when extracting the VP metadata.
; MEMOP_OPT: [[NEWVP]] = !{!"VP", i32 1, i64 457, i64 2, i64 88, i64 3, i64 77, i64 9, i64 72, i64 4, i64 66} ; MEMOP_OPT: [[NEWVP]] = !{!"VP", i32 1, i64 457, i64 2, i64 88, i64 3, i64 77, i64 9, i64 72, i64 4, i64 66}
!llvm.module.flags = !{!0} !llvm.module.flags = !{!0}
Show All 20 Lines
!20 = !{i32 900000, i64 556, i32 1} !20 = !{i32 900000, i64 556, i32 1}
!21 = !{i32 950000, i64 556, i32 1} !21 = !{i32 950000, i64 556, i32 1}
!22 = !{i32 990000, i64 20, i32 2} !22 = !{i32 990000, i64 20, i32 2}
!23 = !{i32 999000, i64 1, i32 5} !23 = !{i32 999000, i64 1, i32 5}
!24 = !{i32 999900, i64 1, i32 5} !24 = !{i32 999900, i64 1, i32 5}
!25 = !{i32 999990, i64 1, i32 5} !25 = !{i32 999990, i64 1, i32 5}
!26 = !{i32 999999, i64 1, i32 5} !26 = !{i32 999999, i64 1, i32 5}
!27 = !{!"function_entry_count", i64 1} !27 = !{!"function_entry_count", i64 1}
!28 = !{!"branch_weights", i32 20, i32 1} !28 = !{!"branch_weights", i64 20, i64 1}
!29 = !{!"branch_weights", i32 556, i32 20} !29 = !{!"branch_weights", i64 556, i64 20}
!30 = !{!"VP", i32 1, i64 556, i64 1, i64 99, i64 2, i64 88, i64 3, i64 77, i64 9, i64 72, i64 4, i64 66, i64 5, i64 55, i64 6, i64 44, i64 7, i64 33, i64 8, i64 22} !30 = !{!"VP", i32 1, i64 556, i64 1, i64 99, i64 2, i64 88, i64 3, i64 77, i64 9, i64 72, i64 4, i64 66, i64 5, i64 55, i64 6, i64 44, i64 7, i64 33, i64 8, i64 22}
!31 = !{!"VP", i32 1, i64 556, i64 1, i64 99, i64 2, i64 88, i64 3, i64 77, i64 9, i64 72, i64 4, i64 66, i64 5, i64 55, i64 6, i64 44, i64 7, i64 33, i64 8, i64 22} !31 = !{!"VP", i32 1, i64 556, i64 1, i64 99, i64 2, i64 88, i64 3, i64 77, i64 9, i64 72, i64 4, i64 66, i64 5, i64 55, i64 6, i64 44, i64 7, i64 33, i64 8, i64 22}
declare void @llvm.lifetime.start(i64, i8* nocapture) declare void @llvm.lifetime.start(i64, i8* nocapture)
declare void @llvm.memcpy.p0i8.p0i8.i64(i8* nocapture writeonly, i8* nocapture readonly, i64, i1) declare void @llvm.memcpy.p0i8.p0i8.i64(i8* nocapture writeonly, i8* nocapture readonly, i64, i1)
declare i32 @memcmp(i8*, i8*, i64) declare i32 @memcmp(i8*, i8*, i64)
▲ Show 20 Lines • Show All 68 Lines • Show Last 20 Lines

llvm/test/Transforms/PGOProfile/multiple_hash_profile.ll

Show All 24 Lines
} }
define linkonce_odr i32 @_Z3fooi(i32 %i) comdat { define linkonce_odr i32 @_Z3fooi(i32 %i) comdat {
entry: entry:
%cmp.i = icmp sgt i32 %i, 2 %cmp.i = icmp sgt i32 %i, 2
%mul.i = select i1 %cmp.i, i32 1, i32 %i %mul.i = select i1 %cmp.i, i32 1, i32 %i
; CHECK: %mul.i = select i1 %cmp.i, i32 1, i32 %i ; CHECK: %mul.i = select i1 %cmp.i, i32 1, i32 %i
; CHECK-SAME: !prof ![[BW:[0-9]+]] ; CHECK-SAME: !prof ![[BW:[0-9]+]]
; CHECK: ![[BW]] = !{!"branch_weights", i32 12, i32 6} ; CHECK: ![[BW]] = !{!"branch_weights", i64 12, i64 6}
; CHECKOLDHASH: %mul.i = select i1 %cmp.i, i32 1, i32 %i ; CHECKOLDHASH: %mul.i = select i1 %cmp.i, i32 1, i32 %i
; CHECKOLDHASH-SAME: !prof ![[BW:[0-9]+]] ; CHECKOLDHASH-SAME: !prof ![[BW:[0-9]+]]
; CHECKOLDHASH: ![[BW]] = !{!"branch_weights", i32 6, i32 12} ; CHECKOLDHASH: ![[BW]] = !{!"branch_weights", i64 6, i64 12}
%retval.0.i = mul nsw i32 %mul.i, %i %retval.0.i = mul nsw i32 %mul.i, %i
ret i32 %retval.0.i ret i32 %retval.0.i
} }

llvm/test/Transforms/PGOProfile/noreturncall.ll

Show All 35 Lines
if.then4: if.then4:
%call5 = call i32 @bar2(i32 undef) %call5 = call i32 @bar2(i32 undef)
br label %if.end7 br label %if.end7
if.end7: if.end7:
%mul = mul nsw i32 %ret.0, %ret.0 %mul = mul nsw i32 %ret.0, %ret.0
ret i32 %mul ret i32 %mul
} }
; USE: ![[BW_ENTRY]] = !{!"branch_weights", i32 21, i32 0} ; USE: ![[BW_ENTRY]] = !{!"branch_weights", i64 21, i64 0}
; USE: ![[BW_IF]] = !{!"branch_weights", i32 0, i32 21} ; USE: ![[BW_IF]] = !{!"branch_weights", i64 0, i64 21}

llvm/test/Transforms/PGOProfile/remap.ll

Show All 19 Lines
if.end: if.end:
%retv = phi i32 [ %add, %if.then ], [ %i, %entry ] %retv = phi i32 [ %add, %if.then ], [ %i, %entry ]
ret i32 %retv ret i32 %retv
} }
; USE-DAG: {{![0-9]+}} = !{i32 1, !"ProfileSummary", {{![0-9]+}}} ; USE-DAG: {{![0-9]+}} = !{i32 1, !"ProfileSummary", {{![0-9]+}}}
; USE-DAG: {{![0-9]+}} = !{!"DetailedSummary", {{![0-9]+}}} ; USE-DAG: {{![0-9]+}} = !{!"DetailedSummary", {{![0-9]+}}}
; USE-DAG: ![[FUNC_ENTRY_COUNT]] = !{!"function_entry_count", i64 3} ; USE-DAG: ![[FUNC_ENTRY_COUNT]] = !{!"function_entry_count", i64 3}
; USE-DAG: ![[BW_ENTRY]] = !{!"branch_weights", i32 2, i32 1} ; USE-DAG: ![[BW_ENTRY]] = !{!"branch_weights", i64 2, i64 1}

llvm/test/Transforms/PGOProfile/select1.ll

Show All 14 Lines
if.then: if.then:
%add = add nsw i32 %i, 2 %add = add nsw i32 %i, 2
;GEN: %[[STEP:[0-9]+]] = zext i1 %cmp to i64 ;GEN: %[[STEP:[0-9]+]] = zext i1 %cmp to i64
;GEN: call void @llvm.instrprof.increment.step({{.*}} i32 3, i32 2, i64 %[[STEP]]) ;GEN: call void @llvm.instrprof.increment.step({{.*}} i32 3, i32 2, i64 %[[STEP]])
;NOSELECT-NOT: call void @llvm.instrprof.increment.step ;NOSELECT-NOT: call void @llvm.instrprof.increment.step
%s = select i1 %cmp, i32 %add, i32 0 %s = select i1 %cmp, i32 %add, i32 0
;USE: select i1 %cmp{{.*}}, !prof ![[BW_ENTRY:[0-9]+]] ;USE: select i1 %cmp{{.*}}, !prof ![[BW_ENTRY:[0-9]+]]
;USE: ![[BW_ENTRY]] = !{!"branch_weights", i32 1, i32 3} ;USE: ![[BW_ENTRY]] = !{!"branch_weights", i64 1, i64 3}
br label %if.end br label %if.end
if.else: if.else:
%sub = sub nsw i32 %i, 2 %sub = sub nsw i32 %i, 2
br label %if.end br label %if.end
if.end: if.end:
%retv = phi i32 [ %add, %if.then ], [ %sub, %if.else ] %retv = phi i32 [ %add, %if.then ], [ %sub, %if.else ]
ret i32 %retv ret i32 %retv
} }

llvm/test/Transforms/PGOProfile/select2.ll

Show All 27 Linesfor.inc:
%inc = add nsw i32 %i.0, 1 %inc = add nsw i32 %i.0, 1
br label %for.cond br label %for.cond
for.end: for.end:
ret i32 %sum.0 ret i32 %sum.0
} }
;USE: ![[ENTRY_COUNT]] = !{!"function_entry_count", i64 3} ;USE: ![[ENTRY_COUNT]] = !{!"function_entry_count", i64 3}
;USE: ![[BW_FOR_BR]] = !{!"branch_weights", i32 800, i32 3} ;USE: ![[BW_FOR_BR]] = !{!"branch_weights", i64 800, i64 3}
;USE: ![[BW_FOR_SELECT]] = !{!"branch_weights", i32 300, i32 500} ;USE: ![[BW_FOR_SELECT]] = !{!"branch_weights", i64 300, i64 500}

llvm/test/Transforms/PGOProfile/switch.ll

Show All 21 Lines
; ENTRY: call void @llvm.instrprof.increment(i8* getelementptr inbounds ([11 x i8], [11 x i8]* @__profn_test_switch, i32 0, i32 0), i64 {{[0-9]+}}, i32 4, i32 0) ; ENTRY: call void @llvm.instrprof.increment(i8* getelementptr inbounds ([11 x i8], [11 x i8]* @__profn_test_switch, i32 0, i32 0), i64 {{[0-9]+}}, i32 4, i32 0)
switch i32 %i, label %sw.default [ switch i32 %i, label %sw.default [
i32 1, label %sw.bb i32 1, label %sw.bb
i32 2, label %sw.bb1 i32 2, label %sw.bb1
i32 3, label %sw.bb2 i32 3, label %sw.bb2
] ]
; USE: ] ; USE: ]
; USE-SAME: !prof ![[BW_SWITCH:[0-9]+]] ; USE-SAME: !prof ![[BW_SWITCH:[0-9]+]]
; USE: ![[BW_SWITCH]] = !{!"branch_weights", i32 3, i32 2, i32 0, i32 5} ; USE: ![[BW_SWITCH]] = !{!"branch_weights", i64 3, i64 2, i64 0, i64 5}
sw.bb: sw.bb:
; GEN: sw.bb: ; GEN: sw.bb:
; GEN: call void @llvm.instrprof.increment(i8* getelementptr inbounds ([11 x i8], [11 x i8]* @__profn_test_switch, i32 0, i32 0), i64 {{[0-9]+}}, i32 4, i32 2) ; GEN: call void @llvm.instrprof.increment(i8* getelementptr inbounds ([11 x i8], [11 x i8]* @__profn_test_switch, i32 0, i32 0), i64 {{[0-9]+}}, i32 4, i32 2)
br label %sw.epilog br label %sw.epilog
sw.bb1: sw.bb1:
; GEN: sw.bb1: ; GEN: sw.bb1:
Show All 20 Lines

llvm/test/Transforms/PGOProfile/thinlto_cspgo_use.ll

; REQUIRES: x86-registered-target ; REQUIRES: x86-registered-target
; RUN: opt -module-summary %s -o %t1.bc ; RUN: opt -module-summary %s -o %t1.bc
; RUN: opt -module-summary %S/Inputs/thinlto_cspgo_bar_use.ll -o %t2.bc ; RUN: opt -module-summary %S/Inputs/thinlto_cspgo_bar_use.ll -o %t2.bc
; RUN: llvm-profdata merge %S/Inputs/thinlto_cs.proftext -o %t3.profdata ; RUN: llvm-profdata merge %S/Inputs/thinlto_cs.proftext -o %t3.profdata
; RUN: llvm-lto2 run -lto-cspgo-profile-file=%t3.profdata -pgo-instrument-entry=false -save-temps -o %t %t1.bc %t2.bc \ ; RUN: llvm-lto2 run -lto-cspgo-profile-file=%t3.profdata -pgo-instrument-entry=false -save-temps -o %t %t1.bc %t2.bc \
; RUN: -r=%t1.bc,foo,pl \ ; RUN: -r=%t1.bc,foo,pl \
; RUN: -r=%t1.bc,bar,l \ ; RUN: -r=%t1.bc,bar,l \
; RUN: -r=%t1.bc,main,plx \ ; RUN: -r=%t1.bc,main,plx \
; RUN: -r=%t2.bc,bar,pl \ ; RUN: -r=%t2.bc,bar,pl \
; RUN: -r=%t2.bc,odd,pl \ ; RUN: -r=%t2.bc,odd,pl \
; RUN: -r=%t2.bc,even,pl ; RUN: -r=%t2.bc,even,pl
; RUN: llvm-dis %t.1.4.opt.bc -o - | FileCheck %s --check-prefix=CSUSE ; RUN: llvm-dis %t.1.4.opt.bc -o - | FileCheck %s --check-prefix=CSUSE
; CSUSE: {{![0-9]+}} = !{i32 1, !"ProfileSummary", {{![0-9]+}}} ; CSUSE: {{![0-9]+}} = !{i32 1, !"ProfileSummary", {{![0-9]+}}}
; CSUSE: {{![0-9]+}} = !{i32 1, !"CSProfileSummary", {{![0-9]+}}} ; CSUSE: {{![0-9]+}} = !{i32 1, !"CSProfileSummary", {{![0-9]+}}}
; CSUSE-DAG: {{![0-9]+}} = !{!"branch_weights", i32 100000, i32 0} ; CSUSE-DAG: {{![0-9]+}} = !{!"branch_weights", i64 100000, i64 0}
; CSUSE-DAG: {{![0-9]+}} = !{!"branch_weights", i32 0, i32 100000} ; CSUSE-DAG: {{![0-9]+}} = !{!"branch_weights", i64 0, i64 100000}
source_filename = "cspgo.c" source_filename = "cspgo.c"
target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128" target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
target triple = "x86_64-unknown-linux-gnu" target triple = "x86_64-unknown-linux-gnu"
define dso_local void @foo() #0 !prof !29 { define dso_local void @foo() #0 !prof !29 {
entry: entry:
br label %for.body br label %for.body
▲ Show 20 Lines • Show All 47 Lines • ▼ Show 20 Lines
!21 = !{i32 900000, i64 100000, i32 4} !21 = !{i32 900000, i64 100000, i32 4}
!22 = !{i32 950000, i64 100000, i32 4} !22 = !{i32 950000, i64 100000, i32 4}
!23 = !{i32 990000, i64 100000, i32 4} !23 = !{i32 990000, i64 100000, i32 4}
!24 = !{i32 999000, i64 100000, i32 4} !24 = !{i32 999000, i64 100000, i32 4}
!25 = !{i32 999900, i64 100000, i32 4} !25 = !{i32 999900, i64 100000, i32 4}
!26 = !{i32 999990, i64 100000, i32 4} !26 = !{i32 999990, i64 100000, i32 4}
!27 = !{i32 999999, i64 1, i32 6} !27 = !{i32 999999, i64 1, i32 6}
!29 = !{!"function_entry_count", i64 1} !29 = !{!"function_entry_count", i64 1}
!30 = !{!"branch_weights", i32 100000, i32 1} !30 = !{!"branch_weights", i64 100000, i64 1}

llvm/test/Transforms/PGOProfile/thinlto_indirect_call_promotion.ll

Show All 29 Lines
} }
!1 = !{!"VP", i32 0, i64 1, i64 -6289574019528802036, i64 1} !1 = !{!"VP", i32 0, i64 1, i64 -6289574019528802036, i64 1}
!2 = !{!"VP", i32 0, i64 1, i64 591260329866125152, i64 1} !2 = !{!"VP", i32 0, i64 1, i64 591260329866125152, i64 1}
; Should not have a VP annotation on new indirect call (check before and after ; Should not have a VP annotation on new indirect call (check before and after
; branch_weights annotation). ; branch_weights annotation).
; ICALL-PROM-NOT: !"VP" ; ICALL-PROM-NOT: !"VP"
; ICALL-PROM: [[BRANCH_WEIGHT]] = !{!"branch_weights", i32 1, i32 0} ; ICALL-PROM: [[BRANCH_WEIGHT]] = !{!"branch_weights", i64 1, i64 0}
; ICALL-PROM-NOT: !"VP" ; ICALL-PROM-NOT: !"VP"

llvm/test/Transforms/SCCP/switch.ll

Show First 20 Lines • Show All 214 Lines • ▼ Show 20 Lines
; CHECK: switch.3: ; CHECK: switch.3:
; CHECK-NEXT: ret i32 3 ; CHECK-NEXT: ret i32 3
; ;
switch i32 %x, label %switch.default [ switch i32 %x, label %switch.default [
i32 0, label %switch.0 i32 0, label %switch.0
i32 1, label %switch.1 i32 1, label %switch.1
i32 2, label %switch.2 i32 2, label %switch.2
i32 3, label %switch.3 i32 3, label %switch.3
], !prof !{!"branch_weights", i32 1, i32 2, i32 3, i32 4, i32 5} ], !prof !{!"branch_weights", i64 1, i64 2, i64 3, i64 4, i64 5}
switch.default: switch.default:
ret i32 -1 ret i32 -1
switch.0: switch.0:
ret i32 0 ret i32 0
switch.1: switch.1:
Show All 14 Lines
; ;
call i32 @test_ip_range(i32 1) call i32 @test_ip_range(i32 1)
call i32 @test_ip_range(i32 3) call i32 @test_ip_range(i32 3)
ret void ret void
} }
declare void @llvm.assume(i1) declare void @llvm.assume(i1)
; CHECK: !1 = !{!"branch_weights", i32 1, i32 5, i32 3, i32 4} ; CHECK: !1 = !{!"branch_weights", i64 1, i64 5, i64 3, i64 4}

llvm/test/Transforms/SampleProfile/entry_counts_cold.ll

; RUN: opt < %s -passes=sample-profile -sample-profile-file=%S/Inputs/entry_counts_cold.prof -S | FileCheck %s
; RUN: opt < %s -sample-profile -sample-profile-file=%S/Inputs/entry_counts_cold.prof -S | FileCheck %s ; RUN: opt < %s -sample-profile -sample-profile-file=%S/Inputs/entry_counts_cold.prof -S | FileCheck %s
; ModuleID = 'temp.bc' ; ModuleID = 'temp.bc'
source_filename = "temp.c" source_filename = "temp.c"
target datalayout = "e-m:o-i64:64-f80:128-n8:16:32:64-S128" target datalayout = "e-m:o-i64:64-f80:128-n8:16:32:64-S128"
target triple = "x86_64-apple-macosx10.14.0" target triple = "x86_64-apple-macosx10.14.0"
; Function Attrs: nounwind ssp uwtable ; Function Attrs: nounwind ssp uwtable
; CHECK: define i32 @top({{.*}} !prof [[TOP:![0-9]+]] ; CHECK: define i32 @top({{.*}} !prof [[TOP:![0-9]+]]
▲ Show 20 Lines • Show All 162 Lines • Show Last 20 Lines

llvm/test/Transforms/SampleProfile/indirect-call.ll

Show First 20 Lines • Show All 117 Lines • ▼ Show 20 Linesthen:
ret i32* %3 ret i32* %3
else: else:
ret i32* null ret i32* null
} }
; CHECK-LABEL: @branch_prof_valid ; CHECK-LABEL: @branch_prof_valid
; Check the conditional branch generated by indirect call promotion won't ; Check the conditional branch generated by indirect call promotion won't
; have invalid profile like !{!"branch_weights", i32 0, i32 0}. ; have invalid profile like !{!"branch_weights", i64 0, i64 0}.
define void @branch_prof_valid(void ()* %t0) #0 !dbg !33 { define void @branch_prof_valid(void ()* %t0) #0 !dbg !33 {
%t1 = alloca void ()* %t1 = alloca void ()*
store void ()* %t0, void ()** %t1 store void ()* %t0, void ()** %t1
%t2 = load void ()*, void ()** %t1 %t2 = load void ()*, void ()** %t1
; CHECK-NOT: call {{.*}} ; CHECK-NOT: call {{.*}}
; CHECK: br i1 {{.*}}, label %if.true.direct_targ, label %if.false.orig_indirect, {{.*}}, !prof ![[BR3:[0-9]+]] ; CHECK: br i1 {{.*}}, label %if.true.direct_targ, label %if.false.orig_indirect, {{.*}}, !prof ![[BR3:[0-9]+]]
call void %t2(), !dbg !34 call void %t2(), !dbg !34
ret void ret void
▲ Show 20 Lines • Show All 60 Lines • ▼ Show 20 Lines
!0 = distinct !DICompileUnit(language: DW_LANG_C_plus_plus, file: !1) !0 = distinct !DICompileUnit(language: DW_LANG_C_plus_plus, file: !1)
!1 = !DIFile(filename: "test.cc", directory: "/") !1 = !DIFile(filename: "test.cc", directory: "/")
!2 = !{i32 2, !"Debug Info Version", i32 3} !2 = !{i32 2, !"Debug Info Version", i32 3}
!3 = distinct !DISubprogram(name: "test", scope: !1, file: !1, line: 3, unit: !0) !3 = distinct !DISubprogram(name: "test", scope: !1, file: !1, line: 3, unit: !0)
!4 = !DILocation(line: 4, scope: !3) !4 = !DILocation(line: 4, scope: !3)
!5 = !DILocation(line: 6, scope: !3) !5 = !DILocation(line: 6, scope: !3)
; CHECK: ![[PROF]] = !{!"VP", i32 0, i64 3457, i64 9191153033785521275, i64 2059, i64 -1069303473483922844, i64 1398} ; CHECK: ![[PROF]] = !{!"VP", i32 0, i64 3457, i64 9191153033785521275, i64 2059, i64 -1069303473483922844, i64 1398}
; CHECK: ![[BR1]] = !{!"branch_weights", i32 4000, i32 4000} ; CHECK: ![[BR1]] = !{!"branch_weights", i64 4000, i64 4000}
; CHECK: ![[BR2]] = !{!"branch_weights", i32 3000, i32 1000} ; CHECK: ![[BR2]] = !{!"branch_weights", i64 3000, i64 1000}
; CHECK: ![[VP]] = !{!"VP", i32 0, i64 8000, i64 -6391416044382067764, i64 1000} ; CHECK: ![[VP]] = !{!"VP", i32 0, i64 8000, i64 -6391416044382067764, i64 1000}
; CHECK: ![[BR3]] = !{!"branch_weights", i32 1, i32 0} ; CHECK: ![[BR3]] = !{!"branch_weights", i64 1, i64 0}
!6 = distinct !DISubprogram(name: "test_inline", scope: !1, file: !1, line: 6, unit: !0) !6 = distinct !DISubprogram(name: "test_inline", scope: !1, file: !1, line: 6, unit: !0)
!7 = !DILocation(line: 7, scope: !6) !7 = !DILocation(line: 7, scope: !6)
!8 = distinct !DISubprogram(name: "test_inline_strip", scope: !1, file: !1, line: 8, unit: !0) !8 = distinct !DISubprogram(name: "test_inline_strip", scope: !1, file: !1, line: 8, unit: !0)
!9 = !DILocation(line: 9, scope: !8) !9 = !DILocation(line: 9, scope: !8)
!10 = distinct !DISubprogram(name: "test_inline_strip_conflict", scope: !1, file: !1, line: 10, unit: !0) !10 = distinct !DISubprogram(name: "test_inline_strip_conflict", scope: !1, file: !1, line: 10, unit: !0)
!11 = !DILocation(line: 11, scope: !10) !11 = !DILocation(line: 11, scope: !10)
!12 = distinct !DISubprogram(name: "test_noinline", scope: !1, file: !1, line: 12, unit: !0) !12 = distinct !DISubprogram(name: "test_noinline", scope: !1, file: !1, line: 12, unit: !0)
!13 = !DILocation(line: 13, scope: !12) !13 = !DILocation(line: 13, scope: !12)
Show All 22 Lines

llvm/test/Transforms/SampleProfile/inline-mergeprof-dup.ll

Show First 20 Lines • Show All 70 Lines • ▼ Show 20 Lines
!13 = !DILocation(line: 4, scope: !12) !13 = !DILocation(line: 4, scope: !12)
!14 = !DILocation(line: 5, scope: !12) !14 = !DILocation(line: 5, scope: !12)
!15 = !DILocation(line: 6, scope: !12) !15 = !DILocation(line: 6, scope: !12)
;; Check the profile of funciton sum is only merged once though the original callsite is replicted. ;; Check the profile of funciton sum is only merged once though the original callsite is replicted.
; CHECK: name: "sum" ; CHECK: name: "sum"
; CHECK-NEXT: {!"function_entry_count", i64 46} ; CHECK-NEXT: {!"function_entry_count", i64 46}
; CHECK: !{!"branch_weights", i32 11, i32 37} ; CHECK: !{!"branch_weights", i64 11, i64 37}
; CHECK: !{!"branch_weights", i32 11, i32 1} ; CHECK: !{!"branch_weights", i64 11, i64 1}

llvm/test/Transforms/SampleProfile/inline-mergeprof.ll

Show First 20 Lines • Show All 84 Lines • ▼ Show 20 Lines
!14 = !DILocation(line: 5, scope: !12) !14 = !DILocation(line: 5, scope: !12)
!15 = !DILocation(line: 6, scope: !12) !15 = !DILocation(line: 6, scope: !12)
!16 = distinct !DISubprogram(name: "sub", scope: !1, file: !1, line: 20, type: !7, scopeLine: 20, virtualIndex: 6, flags: DIFlagPrototyped, spFlags: DISPFlagDefinition, unit: !0, retainedNodes: !2) !16 = distinct !DISubprogram(name: "sub", scope: !1, file: !1, line: 20, type: !7, scopeLine: 20, virtualIndex: 6, flags: DIFlagPrototyped, spFlags: DISPFlagDefinition, unit: !0, retainedNodes: !2)
!17 = !DILocation(line: 20, scope: !16) !17 = !DILocation(line: 20, scope: !16)
!18 = !DILocation(line: 21, scope: !16) !18 = !DILocation(line: 21, scope: !16)
; SCALE: name: "sum" ; SCALE: name: "sum"
; SCALE-NEXT: {!"function_entry_count", i64 46} ; SCALE-NEXT: {!"function_entry_count", i64 46}
; SCALE: !{!"branch_weights", i32 11, i32 2} ; SCALE: !{!"branch_weights", i64 11, i64 2}
; SCALE: !{!"branch_weights", i64 20} ; SCALE: !{!"branch_weights", i64 20}
; SCALE: name: "sub" ; SCALE: name: "sub"
; SCALE-NEXT: {!"function_entry_count", i64 -1} ; SCALE-NEXT: {!"function_entry_count", i64 -1}
; MERGE: name: "sum" ; MERGE: name: "sum"
; MERGE-NEXT: {!"function_entry_count", i64 46} ; MERGE-NEXT: {!"function_entry_count", i64 46}
; MERGE: !{!"branch_weights", i32 11, i32 23} ; MERGE: !{!"branch_weights", i64 11, i64 23}
; MERGE: !{!"branch_weights", i32 10} ; MERGE: !{!"branch_weights", i64 10}
; MERGE: name: "sub" ; MERGE: name: "sub"
; MERGE-NEXT: {!"function_entry_count", i64 3} ; MERGE-NEXT: {!"function_entry_count", i64 3}

llvm/test/Transforms/SampleProfile/profile-format.ll

Show All 26 Lines
; Check sample-profile phase using compactbinary or extbinary format profile ; Check sample-profile phase using compactbinary or extbinary format profile
; will annotate the IR with exactly the same result as using text format. ; will annotate the IR with exactly the same result as using text format.
; CHECK: br i1 %cmp, label %while.body, label %while.end{{.*}} !prof ![[IDX1:[0-9]*]] ; CHECK: br i1 %cmp, label %while.body, label %while.end{{.*}} !prof ![[IDX1:[0-9]*]]
; CHECK: br i1 %cmp1, label %if.then, label %if.else{{.*}} !prof ![[IDX2:[0-9]*]] ; CHECK: br i1 %cmp1, label %if.then, label %if.else{{.*}} !prof ![[IDX2:[0-9]*]]
; CHECK: call i32 (i8*, ...) @printf{{.*}} !prof ![[IDX3:[0-9]*]] ; CHECK: call i32 (i8*, ...) @printf{{.*}} !prof ![[IDX3:[0-9]*]]
; CHECK: = !{!"TotalCount", i64 26781} ; CHECK: = !{!"TotalCount", i64 26781}
; CHECK: = !{!"MaxCount", i64 5553} ; CHECK: = !{!"MaxCount", i64 5553}
; CHECK: ![[IDX1]] = !{!"branch_weights", i32 5392, i32 163} ; CHECK: ![[IDX1]] = !{!"branch_weights", i64 5392, i64 163}
; CHECK: ![[IDX2]] = !{!"branch_weights", i32 5280, i32 113} ; CHECK: ![[IDX2]] = !{!"branch_weights", i64 5280, i64 113}
; CHECK: ![[IDX3]] = !{!"branch_weights", i32 1} ; CHECK: ![[IDX3]] = !{!"branch_weights", i64 1}
; Function Attrs: nounwind uwtable ; Function Attrs: nounwind uwtable
define i32 @_Z3sumii(i32 %x, i32 %y) #0 !dbg !4 { define i32 @_Z3sumii(i32 %x, i32 %y) #0 !dbg !4 {
entry: entry:
%x.addr = alloca i32, align 4 %x.addr = alloca i32, align 4
%y.addr = alloca i32, align 4 %y.addr = alloca i32, align 4
store i32 %x, i32* %x.addr, align 4 store i32 %x, i32* %x.addr, align 4
store i32 %y, i32* %y.addr, align 4 store i32 %y, i32* %y.addr, align 4
▲ Show 20 Lines • Show All 82 Lines • Show Last 20 Lines

llvm/test/Transforms/SimpleLoopUnswitch/basictest-profmd.ll

Show All 10 Lines
; CHECK: !prof ![[MD0:[0-9]+]] ; CHECK: !prof ![[MD0:[0-9]+]]
; CHECK: loop_begin: ; CHECK: loop_begin:
; CHECK: !prof ![[MD1:[0-9]+]] ; CHECK: !prof ![[MD1:[0-9]+]]
loop_begin: loop_begin:
switch i32 %c, label %default [ switch i32 %c, label %default [
i32 1, label %inc i32 1, label %inc
i32 2, label %dec i32 2, label %dec
], !prof !{!"branch_weights", i32 99, i32 1, i32 2} ], !prof !{!"branch_weights", i64 99, i64 1, i64 2}
inc: inc:
call void @incf() call void @incf()
br label %loop_begin br label %loop_begin
dec: dec:
call void @decf() call void @decf()
br label %loop_begin br label %loop_begin
default: default:
ret i32 0 ret i32 0
} }
; CHECK: ![[MD0]] = !{!"branch_weights", i32 99, i32 1, i32 2} ; CHECK: ![[MD0]] = !{!"branch_weights", i64 99, i64 1, i64 2}
; CHECK: ![[MD1]] = !{!"branch_weights", i32 2, i32 1} ; CHECK: ![[MD1]] = !{!"branch_weights", i64 2, i64 1}

llvm/test/Transforms/SimpleLoopUnswitch/trivial-unswitch-profmd.ll

Show First 20 Lines • Show All 41 Lines • ▼ Show 20 Linesentry:
br label %loop_begin br label %loop_begin
loop_begin: loop_begin:
%var_val = load i32, i32* %var %var_val = load i32, i32* %var
switch i32 %cond2, label %loop2 [ switch i32 %cond2, label %loop2 [
i32 0, label %loop0 i32 0, label %loop0
i32 1, label %loop1 i32 1, label %loop1
i32 2, label %loop_exit2 i32 2, label %loop_exit2
], !prof !{!"branch_weights", i32 99, i32 100, i32 101, i32 102} ], !prof !{!"branch_weights", i64 99, i64 100, i64 101, i64 102}
loop0: loop0:
call void @some_func() call void @some_func()
br label %loop_latch br label %loop_latch
loop1: loop1:
call void @some_func() call void @some_func()
br label %loop_latch br label %loop_latch
▲ Show 20 Lines • Show All 54 Lines • ▼ Show 20 Linesentry:
br label %loop_begin br label %loop_begin
loop_begin: loop_begin:
%var_val = load i32, i32* %var %var_val = load i32, i32* %var
switch i32 %cond2, label %loop_exit2 [ switch i32 %cond2, label %loop_exit2 [
i32 0, label %loop0 i32 0, label %loop0
i32 1, label %loop1 i32 1, label %loop1
i32 2, label %loop2 i32 2, label %loop2
], !prof !{!"branch_weights", i32 99, i32 100, i32 101, i32 102} ], !prof !{!"branch_weights", i64 99, i64 100, i64 101, i64 102}
loop0: loop0:
call void @some_func() call void @some_func()
br label %loop_latch br label %loop_latch
loop1: loop1:
call void @some_func() call void @some_func()
br label %loop_latch br label %loop_latch
▲ Show 20 Lines • Show All 59 Lines • ▼ Show 20 Lines
loop_begin: loop_begin:
%var_val = load i32, i32* %var %var_val = load i32, i32* %var
switch i32 %cond2, label %loop_exit2 [ switch i32 %cond2, label %loop_exit2 [
i32 0, label %loop0 i32 0, label %loop0
i32 1, label %loop1 i32 1, label %loop1
i32 13, label %loop_exit1 i32 13, label %loop_exit1
i32 2, label %loop2 i32 2, label %loop2
i32 42, label %loop_exit3 i32 42, label %loop_exit3
], !prof !{!"branch_weights", i32 99, i32 100, i32 101, i32 113, i32 102, i32 142} ], !prof !{!"branch_weights", i64 99, i64 100, i64 101, i64 113, i64 102, i64 142}
loop0: loop0:
call void @some_func() call void @some_func()
br label %loop_latch br label %loop_latch
loop1: loop1:
call void @some_func() call void @some_func()
br label %loop_latch br label %loop_latch
Show All 10 Lines
loop_exit2: loop_exit2:
ret i32 0 ret i32 0
loop_exit3: loop_exit3:
ret i32 0 ret i32 0
} }
; CHECK: ![[MD0]] = !{!"branch_weights", i32 300, i32 102} ; CHECK: ![[MD0]] = !{!"branch_weights", i64 300, i64 102}
; CHECK: ![[MD1]] = !{!"branch_weights", i32 99, i32 100, i32 101} ; CHECK: ![[MD1]] = !{!"branch_weights", i64 99, i64 100, i64 101}
; CHECK: ![[MD2]] = !{!"branch_weights", i32 99, i32 100, i32 101, i32 102} ; CHECK: ![[MD2]] = !{!"branch_weights", i64 99, i64 100, i64 101, i64 102}
; CHECK: ![[MD3]] = !{!"branch_weights", i32 102, i32 100, i32 101} ; CHECK: ![[MD3]] = !{!"branch_weights", i64 102, i64 100, i64 101}
; CHECK: ![[MD4]] = !{!"branch_weights", i32 99, i32 113, i32 142, i32 100, i32 101, i32 102} ; CHECK: ![[MD4]] = !{!"branch_weights", i64 99, i64 113, i64 142, i64 100, i64 101, i64 102}

llvm/test/Transforms/SimplifyCFG/invoke_unwind.ll

Show All 11 Lines; CHECK-NEXT: ret i32 0
to label %1 unwind label %Rethrow, !prof !0 to label %1 unwind label %Rethrow, !prof !0
ret i32 0 ret i32 0
Rethrow: Rethrow:
%exn = landingpad {i8*, i32} %exn = landingpad {i8*, i32}
catch i8* null catch i8* null
resume { i8*, i32 } %exn resume { i8*, i32 } %exn
} }
!0 = !{!"branch_weights", i32 369, i32 2} !0 = !{!"branch_weights", i64 369, i64 2}
define i32 @test2() personality i32 (...)* @__gxx_personality_v0 { define i32 @test2() personality i32 (...)* @__gxx_personality_v0 {
; CHECK-LABEL: @test2( ; CHECK-LABEL: @test2(
; CHECK-NEXT: call void @bar() [ "foo"(i32 100) ] ; CHECK-NEXT: call void @bar() [ "foo"(i32 100) ]
; CHECK-NEXT: ret i32 0 ; CHECK-NEXT: ret i32 0
invoke void @bar( ) [ "foo"(i32 100) ] invoke void @bar( ) [ "foo"(i32 100) ]
to label %1 unwind label %Rethrow to label %1 unwind label %Rethrow
ret i32 0 ret i32 0
▲ Show 20 Lines • Show All 42 Lines • ▼ Show 20 Lines
lpad2: lpad2:
%1 = landingpad { i8*, i32 } %1 = landingpad { i8*, i32 }
cleanup cleanup
br label %rethrow br label %rethrow
} }
declare i32 @__gxx_personality_v0(...) declare i32 @__gxx_personality_v0(...)
; CHECK: ![[PROF]] = !{!"branch_weights", i32 371} ; CHECK: ![[PROF]] = !{!"branch_weights", i64 371}

llvm/test/Transforms/SimplifyCFG/preserve-branchweights-switch-create.ll

Show First 20 Lines • Show All 123 Lines • ▼ Show 20 Lines
sw.bb5: sw.bb5:
call void @func8(i32 %N) nounwind call void @func8(i32 %N) nounwind
br label %sw.epilog br label %sw.epilog
sw.epilog: sw.epilog:
ret void ret void
} }
!0 = !{!"branch_weights", i32 64, i32 4} !0 = !{!"branch_weights", i64 64, i64 4}
!1 = !{!"branch_weights", i32 4, i32 64} !1 = !{!"branch_weights", i64 4, i64 64}
; CHECK: !0 = !{!"branch_weights", i32 256, i32 4352, i32 16} ; CHECK: !0 = !{!"branch_weights", i64 256, i64 4352, i64 16}
!2 = !{!"branch_weights", i32 4, i32 4, i32 8} !2 = !{!"branch_weights", i64 4, i64 4, i64 8}
!3 = !{!"branch_weights", i32 8, i32 8, i32 4} !3 = !{!"branch_weights", i64 8, i64 8, i64 4}
; CHECK: !1 = !{!"branch_weights", i32 32, i32 48, i32 96, i32 16} ; CHECK: !1 = !{!"branch_weights", i64 32, i64 48, i64 96, i64 16}
!4 = !{!"branch_weights", i32 7, i32 6, i32 4, i32 3} !4 = !{!"branch_weights", i64 7, i64 6, i64 4, i64 3}
!5 = !{!"branch_weights", i32 17, i32 13, i32 9} !5 = !{!"branch_weights", i64 17, i64 13, i64 9}
; CHECK: !3 = !{!"branch_weights", i32 7, i32 3, i32 4, i32 6} ; CHECK: !3 = !{!"branch_weights", i64 7, i64 3, i64 4, i64 6}

llvm/test/Transforms/SimplifyCFG/preserve-branchweights.ll

Show First 20 Lines • Show All 630 Lines • ▼ Show 20 Linesblock3:
%cowval = phi i32 [ 2, %block2 ], [ 0, %block1 ] %cowval = phi i32 [ 2, %block2 ], [ 0, %block1 ]
br label %exit br label %exit
exit: exit:
%outval = phi i32 [ %cowval, %block3 ], [ 1, %block2 ] %outval = phi i32 [ %cowval, %block3 ], [ 1, %block2 ]
ret i32 %outval ret i32 %outval
} }
!0 = !{!"branch_weights", i32 3, i32 5} !0 = !{!"branch_weights", i64 3, i64 5}
!1 = !{!"branch_weights", i32 1, i32 1} !1 = !{!"branch_weights", i64 1, i64 1}
!2 = !{!"branch_weights", i32 1, i32 2} !2 = !{!"branch_weights", i64 1, i64 2}
!3 = !{!"branch_weights", i32 4, i32 3, i32 2, i32 1} !3 = !{!"branch_weights", i64 4, i64 3, i64 2, i64 1}
!4 = !{!"branch_weights", i32 4, i32 3, i32 2, i32 1} !4 = !{!"branch_weights", i64 4, i64 3, i64 2, i64 1}
!5 = !{!"branch_weights", i32 7, i32 6, i32 5} !5 = !{!"branch_weights", i64 7, i64 6, i64 5}
!6 = !{!"branch_weights", i32 1, i32 3} !6 = !{!"branch_weights", i64 1, i64 3}
!7 = !{!"branch_weights", i32 33, i32 9, i32 8, i32 7} !7 = !{!"branch_weights", i64 33, i64 9, i64 8, i64 7}
!8 = !{!"branch_weights", i32 33, i32 9, i32 8} !8 = !{!"branch_weights", i64 33, i64 9, i64 8}
!9 = !{!"branch_weights", i32 7, i32 6} !9 = !{!"branch_weights", i64 7, i64 6}
!10 = !{!"branch_weights", i32 672646, i32 21604207} !10 = !{!"branch_weights", i64 672646, i64 21604207}
!11 = !{!"branch_weights", i32 6960, i32 21597248} !11 = !{!"branch_weights", i64 6960, i64 21597248}
!12 = !{!"these_are_not_the_branch_weights_you_are_looking_for", i32 3, i32 5} !12 = !{!"these_are_not_the_branch_weights_you_are_looking_for", i64 3, i64 5}
!13 = !{!"branch_weights", i32 2, i32 3} !13 = !{!"branch_weights", i64 2, i64 3}
!14 = !{!"branch_weights", i32 4, i32 7} !14 = !{!"branch_weights", i64 4, i64 7}
; CHECK: !0 = !{!"branch_weights", i32 5, i32 11} ; CHECK: !0 = !{!"branch_weights", i64 5, i64 11}
; CHECK: !1 = !{!"branch_weights", i32 1, i32 3} ; CHECK: !1 = !{!"branch_weights", i64 1, i64 3}
; CHECK: !2 = !{!"branch_weights", i32 1, i32 5} ; CHECK: !2 = !{!"branch_weights", i64 1, i64 5}
; CHECK: !3 = !{!"branch_weights", i32 7, i32 1, i32 2} ; CHECK: !3 = !{!"branch_weights", i64 7, i64 1, i64 2}
; CHECK: !4 = !{!"branch_weights", i32 49, i32 12, i32 24, i32 35} ; CHECK: !4 = !{!"branch_weights", i64 49, i64 12, i64 24, i64 35}
; CHECK: !5 = !{!"branch_weights", i32 11, i32 5} ; CHECK: !5 = !{!"branch_weights", i64 11, i64 5}
; CHECK: !6 = !{!"branch_weights", i32 17, i32 15} ; CHECK: !6 = !{!"branch_weights", i64 17, i64 15}
; CHECK: !7 = !{!"branch_weights", i32 9, i32 7} ; CHECK: !7 = !{!"branch_weights", i64 9, i64 7}
; CHECK: !8 = !{!"branch_weights", i32 17, i32 9, i32 8, i32 7, i32 17} ; CHECK: !8 = !{!"branch_weights", i64 17, i64 9, i64 8, i64 7, i64 17}
; CHECK: !9 = !{!"branch_weights", i32 24, i32 33} ; CHECK: !9 = !{!"branch_weights", i64 24, i64 33}
; CHECK: !10 = !{!"branch_weights", i32 8, i32 33} ; CHECK: !10 = !{!"branch_weights", i64 8, i64 33}
;; The false weight prints out as a negative integer here, but inside llvm, we ; CHECK: !11 = !{!"branch_weights", i64 14682349375088, i64 466591416422336}
;; treat the weight as an unsigned integer. ; CHECK: !12 = !{!"branch_weights", i64 3, i64 5}
; CHECK: !11 = !{!"branch_weights", i32 112017436, i32 -735157296} ; CHECK: !13 = !{!"branch_weights", i64 22, i64 12}
; CHECK: !12 = !{!"branch_weights", i32 3, i32 5} ; CHECK: !14 = !{!"branch_weights", i64 34, i64 21}
; CHECK: !13 = !{!"branch_weights", i32 22, i32 12} ; CHECK: !15 = !{!"branch_weights", i64 33, i64 14}
; CHECK: !14 = !{!"branch_weights", i32 34, i32 21} ; CHECK: !16 = !{!"branch_weights", i64 47, i64 8}
; CHECK: !15 = !{!"branch_weights", i32 33, i32 14} ; CHECK: !17 = !{!"branch_weights", i64 6, i64 2}
; CHECK: !16 = !{!"branch_weights", i32 47, i32 8} ; CHECK: !18 = !{!"branch_weights", i64 8, i64 2}
; CHECK: !17 = !{!"branch_weights", i32 6, i32 2}
; CHECK: !18 = !{!"branch_weights", i32 8, i32 2}

llvm/test/Transforms/SimplifyCFG/switch-profmd.ll

; RUN: opt -S -simplifycfg < %s | FileCheck %s ; RUN: opt -S -simplifycfg < %s | FileCheck %s
declare i32 @f(i32 %val) declare i32 @f(i32 %val)
; Check that eliminating cases with unreachable branches keeps ; Check that eliminating cases with unreachable branches keeps
; prof branch_weights metadata consistent with switch instruction. ; prof branch_weights metadata consistent with switch instruction.
define i32 @test_switch_to_unreachable(i32 %val) { define i32 @test_switch_to_unreachable(i32 %val) {
; CHECK-LABEL: test_switch_to_unreachable ; CHECK-LABEL: test_switch_to_unreachable
; CHECK: switch ; CHECK: switch
; CHECK-NOT: i32 0, label %on0 ; CHECK-NOT: i32 0, label %on0
switch i32 %val, label %otherwise [ switch i32 %val, label %otherwise [
i32 0, label %on0 i32 0, label %on0
i32 1, label %on1 i32 1, label %on1
i32 2, label %on2 i32 2, label %on2
], !prof !{!"branch_weights", i32 99, i32 0, i32 1, i32 2} ], !prof !{!"branch_weights", i64 99, i64 0, i64 1, i64 2}
; CHECK: !prof ![[MD0:[0-9]+]] ; CHECK: !prof ![[MD0:[0-9]+]]
otherwise: otherwise:
%result = call i32 @f(i32 -1) %result = call i32 @f(i32 -1)
ret i32 %result ret i32 %result
on0: on0:
unreachable unreachable
ret i32 125 ret i32 125
on1: on1:
%result1 = call i32 @f(i32 -2) %result1 = call i32 @f(i32 -2)
ret i32 %result1 ret i32 %result1
on2: on2:
%result2 = call i32 @f(i32 -3) %result2 = call i32 @f(i32 -3)
ret i32 %result2 ret i32 %result2
} }
; CHECK: ![[MD0]] = !{!"branch_weights", i32 99, i32 2, i32 1} ; CHECK: ![[MD0]] = !{!"branch_weights", i64 99, i64 2, i64 1}

llvm/test/Transforms/Util/libcalls-shrinkwrap-double.ll

Show First 20 Lines • Show All 232 Lines • ▼ Show 20 Lines
; CHECK-NEXT: br label %[[END_LABEL]] ; CHECK-NEXT: br label %[[END_LABEL]]
; CHECK: [[END_LABEL]]: ; CHECK: [[END_LABEL]]:
ret void ret void
} }
declare double @pow(double, double) declare double @pow(double, double)
; CHECK: ![[BRANCH_WEIGHT]] = !{!"branch_weights", i32 1, i32 2000} ; CHECK: ![[BRANCH_WEIGHT]] = !{!"branch_weights", i64 1, i64 2000}

llvm/test/Transforms/Util/libcalls-shrinkwrap-float.ll

Show First 20 Lines • Show All 182 Lines • ▼ Show 20 Lines
declare float @sqrtf(float) declare float @sqrtf(float)
declare float @atanhf(float) declare float @atanhf(float)
declare float @logf(float) declare float @logf(float)
declare float @log10f(float) declare float @log10f(float)
declare float @log2f(float) declare float @log2f(float)
declare float @logbf(float) declare float @logbf(float)
declare float @log1pf(float) declare float @log1pf(float)
; CHECK: ![[BRANCH_WEIGHT]] = !{!"branch_weights", i32 1, i32 2000} ; CHECK: ![[BRANCH_WEIGHT]] = !{!"branch_weights", i64 1, i64 2000}

llvm/test/Transforms/Util/libcalls-shrinkwrap-long-double.ll

Show First 20 Lines • Show All 183 Lines • ▼ Show 20 Lines
declare x86_fp80 @sqrtl(x86_fp80) declare x86_fp80 @sqrtl(x86_fp80)
declare x86_fp80 @atanhl(x86_fp80) declare x86_fp80 @atanhl(x86_fp80)
declare x86_fp80 @logl(x86_fp80) declare x86_fp80 @logl(x86_fp80)
declare x86_fp80 @log10l(x86_fp80) declare x86_fp80 @log10l(x86_fp80)
declare x86_fp80 @log2l(x86_fp80) declare x86_fp80 @log2l(x86_fp80)
declare x86_fp80 @logbl(x86_fp80) declare x86_fp80 @logbl(x86_fp80)
declare x86_fp80 @log1pl(x86_fp80) declare x86_fp80 @log1pl(x86_fp80)
; CHECK: ![[BRANCH_WEIGHT]] = !{!"branch_weights", i32 1, i32 2000} ; CHECK: ![[BRANCH_WEIGHT]] = !{!"branch_weights", i64 1, i64 2000}

llvm/unittests/Analysis/ProfileSummaryInfoTest.cpp

Show First 20 Lines • Show All 333 Lines • ▼ Show 20 LinesTEST_F(ProfileSummaryInfoTest, SampleProf) {
EXPECT_TRUE(PSI.isColdBlockNthPercentile(10000, &BB0, &BFI)); EXPECT_TRUE(PSI.isColdBlockNthPercentile(10000, &BB0, &BFI));
EXPECT_TRUE(PSI.isColdBlockNthPercentile(10000, BB1, &BFI)); EXPECT_TRUE(PSI.isColdBlockNthPercentile(10000, BB1, &BFI));
EXPECT_TRUE(PSI.isColdBlockNthPercentile(10000, BB2, &BFI)); EXPECT_TRUE(PSI.isColdBlockNthPercentile(10000, BB2, &BFI));
EXPECT_TRUE(PSI.isColdBlockNthPercentile(10000, BB3, &BFI)); EXPECT_TRUE(PSI.isColdBlockNthPercentile(10000, BB3, &BFI));
CallBase &CS1 = cast<CallBase>(*BB1->getFirstNonPHI()); CallBase &CS1 = cast<CallBase>(*BB1->getFirstNonPHI());
auto *CI2 = BB2->getFirstNonPHI(); auto *CI2 = BB2->getFirstNonPHI();
// Manually attach branch weights metadata to the call instruction. // Manually attach branch weights metadata to the call instruction.
SmallVector<uint32_t, 1> Weights; SmallVector<uint64_t, 1> Weights;
Weights.push_back(1000); Weights.push_back(1000);
MDBuilder MDB(M->getContext()); MDBuilder MDB(M->getContext());
CI2->setMetadata(LLVMContext::MD_prof, MDB.createBranchWeights(Weights)); CI2->setMetadata(LLVMContext::MD_prof, MDB.createBranchWeights(Weights));
CallBase &CS2 = cast<CallBase>(*CI2); CallBase &CS2 = cast<CallBase>(*CI2);
EXPECT_FALSE(PSI.isHotCallSite(CS1, &BFI)); EXPECT_FALSE(PSI.isHotCallSite(CS1, &BFI));
EXPECT_TRUE(PSI.isHotCallSite(CS2, &BFI)); EXPECT_TRUE(PSI.isHotCallSite(CS2, &BFI));
▲ Show 20 Lines • Show All 83 Lines • Show Last 20 Lines
llvm/test/Analysis/BranchProbabilityInfo/basic.ll