This is an archive of the discontinued LLVM Phabricator instance.

Differential D156722

[CSSPGO] Support stale profile matching for LTO
ClosedPublic

Authored by wlei on Jul 31 2023, 11:01 AM.

Details

Reviewers
hoy
wenlei
Commits
rG062af2e76366: [CSSPGO] Support stale profile matching for LTO
Summary

As in per-link time, callsites could be optimized out by inlining, we don't have those original call targets in the IR in LTO time. Additionally, the inlined code doesn't actually belong to the original function, the IR locations or pseudo probe parsed from it are incorrect and could mislead the matching later.

This change adds the support to extract the original IR location info from the inlined code, specifically, it make sure to skip all the inlined code that doesn't belong the original function, but before that, it processes the inline frames of the debug info to extract the base frame and recover its callsite and callee target(name).

Measured on some stale profile instances, all showed some perf improvements.

Diff Detail

Event Timeline

wlei created this revision.Jul 31 2023, 11:01 AM
Herald added a project: Restricted Project. · View Herald TranscriptJul 31 2023, 11:01 AM
Herald added subscribers: ormris, hoy, wenlei and 3 others. · View Herald Transcript
wlei requested review of this revision.Jul 31 2023, 11:01 AM
Herald added a project: Restricted Project. · View Herald TranscriptJul 31 2023, 11:01 AM
Herald added a subscriber: llvm-commits. · View Herald Transcript
wlei edited the summary of this revision. (Show Details)Jul 31 2023, 11:19 AM
wlei added reviewers: hoy, wenlei.
wlei added a parent revision: D156725: [CSSPGO] Skip reporting staleness metrics for imported functions.Jul 31 2023, 12:29 PM
wlei added a child revision: D156715: [CSSPGO] Fix issues with post-link function checksum check.
wlei removed a child revision: D156715: [CSSPGO] Fix issues with post-link function checksum check.
wlei added a parent revision: D156715: [CSSPGO] Fix issues with post-link function checksum check.
wlei removed a parent revision: D156725: [CSSPGO] Skip reporting staleness metrics for imported functions.
wlei added a child revision: D156725: [CSSPGO] Skip reporting staleness metrics for imported functions.Jul 31 2023, 12:31 PM
Harbormaster completed remote builds in B249276: Diff 545743.Jul 31 2023, 1:17 PM
hoy added inline comments.Aug 3 2023, 1:24 PM
llvm/lib/Transforms/IPO/SampleProfile.cpp
2143–2146

nit: use PrevDIL->getSubprogramLinkageName() instead.

2150

Is the continue needed since instructions in the same BB may not from a single callsite.

wlei updated this revision to Diff 547276.Aug 4 2023, 10:50 AM

addressing feedback

llvm/lib/Transforms/IPO/SampleProfile.cpp
2143–2146

Done, thanks!

2150

We need the continue, this is needed for all the inlined code, because it's not part of the original function.
otherwise the below code will extract the probe and callsite from a different function.

Note that this is under this condition, so it won't affect the original code.

if (DIL->getInlinedAt()) { 
   ....
   continue;
  }
Harbormaster completed remote builds in B250376: Diff 547276.Aug 4 2023, 12:37 PM
hoy added inline comments.Aug 10 2023, 10:42 AM
llvm/lib/Transforms/IPO/SampleProfile.cpp
2127

Might be good to mention IR flattening in the comment somewhere which is needed as the flattened profile is always used for matching.

2134

Do we also want to flatten for callsite probes?

hoy added inline comments.Aug 10 2023, 10:47 AM
llvm/lib/Transforms/IPO/SampleProfile.cpp
2134

Also should this be done under FlattenProfileForMatching?

wlei added inline comments.Aug 10 2023, 11:57 AM
llvm/lib/Transforms/IPO/SampleProfile.cpp
2134

Do we also want to flatten for callsite probes?

Block probe should be sufficient. for every callsite, there should be a block probe in the same BB, so their debug info inline stack is the same.

Also should this be done under FlattenProfileForMatching?

Not sure if this will works with FlattenProfileForMatching off. Maybe we even don't need this `FlattenProfileForMatching switch, I'm thinking FlattenIR or FlattenProfile should be always beneficial to the matching. Or maybe we need another switch FlattenIRForMatching

wlei updated this revision to Diff 549130.Aug 10 2023, 12:09 PM

update the commment.

hoy added inline comments.Aug 10 2023, 1:12 PM
llvm/lib/Transforms/IPO/SampleProfile.cpp
2134

Block probe should be sufficient. for every callsite, there should be a block probe in the same BB, so their debug info inline stack is the same.

Sounds good, as long as the inlinee has one block probe left, the original callsite should be recoverd.

Not sure if this will works with FlattenProfileForMatching off. Maybe we even don't need this `FlattenProfileForMatching switch, I'm thinking FlattenIR or FlattenProfile should be always beneficial to the matching. Or maybe we need another switch FlattenIRForMatching

I mean as long as the IR flattening and profile flattening are synchronized it should be fine. Maybe we should just retire the FlattenProfileForMatching switch as we always use flattened profile.

Harbormaster completed remote builds in B251756: Diff 549130.Aug 10 2023, 6:30 PM
wlei added inline comments.Aug 11 2023, 9:41 AM
llvm/lib/Transforms/IPO/SampleProfile.cpp
2134

Sounds good to retire FlattenProfileForMatching , I will do it in https://reviews.llvm.org/D156725, which already made some changes to FlattenProfileForMatching.

wenlei added inline comments.Aug 24 2023, 10:44 AM
llvm/lib/Transforms/IPO/SampleProfile.cpp
2121

nit on function naming: it not only populates IRLocations, but also MatchedCallsiteLocs.

2138

Given we tested isa<PseudoProbeInst>(&I) above, is extractProbe ever going to fail? Remove the if check or replace it with an assertion?

2139

nit: "base inline" is not a canonical term and can be confusing. You mean top-level inliner frame?

2156

nit: instead of spraying TODOs for line-number-based AutoFDO, we can have a top-level top for AutoFDO, and also explain 1) the current behavior for AutoFDO, 2) what actually needs to be done for AutoFDO.

2157–2160

For readability, would it make sense to tweak the code a bit so it's explicit that we're handling not-inlined probes. Initially I got confused since it didn't pay attention to the continue above..

if (isa<PseudoProbeInst>(&I)) {
  get DIL to check for inlining..
  if (Inlined case)... 

  else { // not inlined case

  }
}
2162

I know this is old code, but what intrinsics are we skipping here, is that just for skipping probe intrinsics or something else? If it's for skipping probes, maybe we can consider this?

if (probe inst) {
  record empty string in IRLocations
else if (call inst) {
  record actual call site in IRLocations
}

or

if (probe inst) {
  record empty string in IRLocations
else if (not call inst) {
  continue
}
record actual call site in IRLocations
2389–2390

As I read the code I noticed that "non-direct-call site" can be confusing as it can mean call site that is not direct call. But we really mean to say is 1) not a call site, 2) indirect call site?

wenlei added inline comments.Aug 24 2023, 10:59 AM
llvm/lib/Transforms/IPO/SampleProfile.cpp
2121

More like findAnchorLocations?

wlei updated this revision to Diff 553373.Aug 24 2023, 11:30 PM

addressing feedback

llvm/lib/Transforms/IPO/SampleProfile.cpp
2121

Sounds good to move MatchedCallsiteLocs out of populateIRLocations, I also saw you mentioned to break the functions in other patch, I will do the refactoring in a new diff.

Regarding the name findAnchorLocations, we use the term "Anchor" for both IR and Profile, see in the runStaleProfileMatching, we use the ProfileAnchors. So maybe still keep populateIRlocations or findIRLocations? after the refactoring, it should only be related with IR.

2138

Sounds good!

2139

changed to top-level frame.

2156

Thanks for the suggestion, refine the comments.

2157–2160

Sounds good!

2162

IntrinsicInst may not only be used for probe intrinsics. I can't remember clearly, but I vaguely remembered there were a lot of dbg intrinsics that messed up the matching, and the fix was to add this isa<IntrinsicInst>(&I) check.

wlei mentioned this in D158817: [CSSPGO] Refactoring SampleProfileMatcher::runOnFunction.Aug 24 2023, 11:42 PM
wlei added a child revision: D158817: [CSSPGO] Refactoring SampleProfileMatcher::runOnFunction.Aug 24 2023, 11:46 PM
wlei removed a child revision: D156725: [CSSPGO] Skip reporting staleness metrics for imported functions.
Harbormaster completed remote builds in B254804: Diff 553373.Aug 25 2023, 1:12 AM
wenlei added inline comments.Aug 25 2023, 9:35 AM
llvm/lib/Transforms/IPO/SampleProfile.cpp
2121

Assuming the part that deals with profile will be moved out of this function ,how about let's name it as findIRAnchors?

In runStaleProfileMatching, we find ProfileAnchors and match them with IRAnchors established here.

2389–2390

Can you update the comments to avoid ambiguity of "non-direct-call site"?

wlei updated this revision to Diff 553597.Aug 25 2023, 1:31 PM

rebase on https://reviews.llvm.org/D158817

Harbormaster completed remote builds in B254979: Diff 553597.Aug 25 2023, 1:33 PM
wlei removed a child revision: D158817: [CSSPGO] Refactoring SampleProfileMatcher::runOnFunction.Aug 25 2023, 1:35 PM
wlei edited parent revisions, added: D158817: [CSSPGO] Refactoring SampleProfileMatcher::runOnFunction; removed: D156715: [CSSPGO] Fix issues with post-link function checksum check.
wlei added a child revision: D158891: [CSSPGO] Retire FlattenProfileForMatching.Aug 25 2023, 2:21 PM
wenlei accepted this revision.Aug 25 2023, 3:36 PM

lgtm, thanks.

This revision is now accepted and ready to land.Aug 25 2023, 3:36 PM
hoy accepted this revision.Aug 29 2023, 5:02 PM
This revision was landed with ongoing or failed builds.Aug 30 2023, 6:01 PM
Closed by commit rG062af2e76366: [CSSPGO] Support stale profile matching for LTO (authored by wlei). · Explain Why
This revision was automatically updated to reflect the committed changes.
wlei added a commit: rG062af2e76366: [CSSPGO] Support stale profile matching for LTO.

Revision Contents

PathSize
llvm/
lib/
Transforms/
IPO/
30 lines
test/
Transforms/
SampleProfile/
Inputs/
26 lines
252 lines

Diff 554876

llvm/lib/Transforms/IPO/SampleProfile.cpp

Show First 20 Lines • Show All 2,112 Lines • ▼ Show 20 Lines if (ReportProfileStaleness || PersistProfileStaleness ||
SalvageStaleProfile) { SalvageStaleProfile) {
MatchingManager = MatchingManager =
std::make_unique<SampleProfileMatcher>(M, *Reader, ProbeManager.get()); std::make_unique<SampleProfileMatcher>(M, *Reader, ProbeManager.get());
} }
return true; return true;
} }
void SampleProfileMatcher::findIRAnchors( void SampleProfileMatcher::findIRAnchors(
wenleiUnsubmitted
Not Done
ReplyInline Actions

nit on function naming: it not only populates IRLocations, but also MatchedCallsiteLocs.

wenlei: nit on function naming: it not only populates `IRLocations`, but also `MatchedCallsiteLocs`.
wenleiUnsubmitted
Not Done
ReplyInline Actions

More like findAnchorLocations?

wenlei: More like `findAnchorLocations`?
wleiAuthorUnsubmitted
Done
ReplyInline Actions

Sounds good to move MatchedCallsiteLocs out of populateIRLocations, I also saw you mentioned to break the functions in other patch, I will do the refactoring in a new diff.

Regarding the name findAnchorLocations, we use the term "Anchor" for both IR and Profile, see in the runStaleProfileMatching, we use the ProfileAnchors. So maybe still keep populateIRlocations or findIRLocations? after the refactoring, it should only be related with IR.

wlei: Sounds good to move `MatchedCallsiteLocs ` out of `populateIRLocations`, I also saw you…
wenleiUnsubmitted
Not Done
ReplyInline Actions

Assuming the part that deals with profile will be moved out of this function ,how about let's name it as findIRAnchors?

In runStaleProfileMatching, we find ProfileAnchors and match them with IRAnchors established here.

wenlei: Assuming the part that deals with profile will be moved out of this function ,how about let's…
const Function &F, std::map<LineLocation, StringRef> &IRAnchors) { const Function &F, std::map<LineLocation, StringRef> &IRAnchors) {
// Extract profile matching anchors in the IR. // Extract profile matching anchors in the IR.
for (auto &BB : F) { for (auto &BB : F) {
for (auto &I : BB) { for (auto &I : BB) {
// TODO: Support line-number based location(AutoFDO). // TODO: To support AutoFDO, we need to parse all the non-call
// instructions to extract the line-number based locations. For
hoyUnsubmitted
Not Done
ReplyInline Actions

Might be good to mention IR flattening in the comment somewhere which is needed as the flattened profile is always used for matching.

hoy: Might be good to mention IR flattening in the comment somewhere which is needed as the…
// pseudo-probe mode, since each block is instrumented with one probe
// inst, parsing probe inst is enough.
if (FunctionSamples::ProfileIsProbeBased && isa<PseudoProbeInst>(&I)) { if (FunctionSamples::ProfileIsProbeBased && isa<PseudoProbeInst>(&I)) {
if (std::optional<PseudoProbe> Probe = extractProbe(I)) std::optional<PseudoProbe> Probe = extractProbe(I);
assert(Probe &&
"Probe should not be null for pseudo-probe instruction");
// Flatten inlined IR for the matching. Recover the original callsite
hoyUnsubmitted
Not Done
ReplyInline Actions

Do we also want to flatten for callsite probes?

hoy: Do we also want to flatten for callsite probes?
hoyUnsubmitted
Not Done
ReplyInline Actions

Also should this be done under FlattenProfileForMatching?

hoy: Also should this be done under `FlattenProfileForMatching`?
wleiAuthorUnsubmitted
Done
ReplyInline Actions

Do we also want to flatten for callsite probes?

Block probe should be sufficient. for every callsite, there should be a block probe in the same BB, so their debug info inline stack is the same.

Also should this be done under FlattenProfileForMatching?

Not sure if this will works with FlattenProfileForMatching off. Maybe we even don't need this `FlattenProfileForMatching switch, I'm thinking FlattenIR or FlattenProfile should be always beneficial to the matching. Or maybe we need another switch FlattenIRForMatching

wlei: > Do we also want to flatten for callsite probes? Block probe should be sufficient. for every…
hoyUnsubmitted
Not Done
ReplyInline Actions

Block probe should be sufficient. for every callsite, there should be a block probe in the same BB, so their debug info inline stack is the same.

Sounds good, as long as the inlinee has one block probe left, the original callsite should be recoverd.

Not sure if this will works with FlattenProfileForMatching off. Maybe we even don't need this `FlattenProfileForMatching switch, I'm thinking FlattenIR or FlattenProfile should be always beneficial to the matching. Or maybe we need another switch FlattenIRForMatching

I mean as long as the IR flattening and profile flattening are synchronized it should be fine. Maybe we should just retire the FlattenProfileForMatching switch as we always use flattened profile.

hoy: > Block probe should be sufficient. for every callsite, there should be a block probe in the…
wleiAuthorUnsubmitted
Done
ReplyInline Actions

Sounds good to retire FlattenProfileForMatching , I will do it in https://reviews.llvm.org/D156725, which already made some changes to FlattenProfileForMatching.

wlei: Sounds good to retire `FlattenProfileForMatching `, I will do it in https://reviews.llvm.
// and call target by analyzing the inline frames from the debug info.
if (DILocation *DIL = I.getDebugLoc()) {
if (DIL->getInlinedAt()) {
// Find the top-level inline frame.
wenleiUnsubmitted
Not Done
ReplyInline Actions

Given we tested isa<PseudoProbeInst>(&I) above, is extractProbe ever going to fail? Remove the if check or replace it with an assertion?

wenlei: Given we tested `isa<PseudoProbeInst>(&I)` above, is `extractProbe` ever going to fail? Remove…
wleiAuthorUnsubmitted
Done
ReplyInline Actions

Sounds good!

wlei: Sounds good!
const DILocation *PrevDIL = DIL;
wenleiUnsubmitted
Not Done
ReplyInline Actions

nit: "base inline" is not a canonical term and can be confusing. You mean top-level inliner frame?

wenlei: nit: "base inline" is not a canonical term and can be confusing. You mean top-level inliner…
wleiAuthorUnsubmitted
Done
ReplyInline Actions

changed to top-level frame.

wlei: changed to top-level frame.
for (; DIL->getInlinedAt(); DIL = DIL->getInlinedAt())
PrevDIL = DIL;
LineLocation Callsite = FunctionSamples::getCallSiteIdentifier(DIL);
StringRef CalleeName = PrevDIL->getSubprogramLinkageName();
IRAnchors.emplace(Callsite, CalleeName);
} else
hoyUnsubmitted
Not Done
ReplyInline Actions

nit: use PrevDIL->getSubprogramLinkageName() instead.

hoy: nit: use `PrevDIL->getSubprogramLinkageName()` instead.
wleiAuthorUnsubmitted
Done
ReplyInline Actions

Done, thanks!

wlei: Done, thanks!
IRAnchors.emplace(LineLocation(Probe->Id, 0), StringRef()); IRAnchors.emplace(LineLocation(Probe->Id, 0), StringRef());
} }
}
hoyUnsubmitted
Not Done
ReplyInline Actions

Is the continue needed since instructions in the same BB may not from a single callsite.

hoy: Is the `continue` needed since instructions in the same BB may not from a single callsite.
wleiAuthorUnsubmitted
Done
ReplyInline Actions

We need the continue, this is needed for all the inlined code, because it's not part of the original function.
otherwise the below code will extract the probe and callsite from a different function.

Note that this is under this condition, so it won't affect the original code.

if (DIL->getInlinedAt()) { 
   ....
   continue;
  }
wlei: We need the `continue`, this is needed for all the inlined code, because it's not part of the…
if (!isa<CallBase>(&I) || isa<IntrinsicInst>(&I)) if (!isa<CallBase>(&I) || isa<IntrinsicInst>(&I))
continue; continue;
const auto *CB = dyn_cast<CallBase>(&I); const auto *CB = dyn_cast<CallBase>(&I);
if (auto &DLoc = I.getDebugLoc()) { if (auto &DLoc = I.getDebugLoc()) {
// Skip the inlined callsite. For pseudo probe mode, extracting inline
wenleiUnsubmitted
Not Done
ReplyInline Actions

nit: instead of spraying TODOs for line-number-based AutoFDO, we can have a top-level top for AutoFDO, and also explain 1) the current behavior for AutoFDO, 2) what actually needs to be done for AutoFDO.

wenlei: nit: instead of spraying TODOs for line-number-based AutoFDO, we can have a top-level top for…
wleiAuthorUnsubmitted
Done
ReplyInline Actions

Thanks for the suggestion, refine the comments.

wlei: Thanks for the suggestion, refine the comments.
// info from the probe inst is enough.
if (DLoc.getInlinedAt())
continue;
wenleiUnsubmitted
Not Done
ReplyInline Actions

For readability, would it make sense to tweak the code a bit so it's explicit that we're handling not-inlined probes. Initially I got confused since it didn't pay attention to the continue above..

if (isa<PseudoProbeInst>(&I)) {
  get DIL to check for inlining..
  if (Inlined case)... 

  else { // not inlined case

  }
}
wenlei: For readability, would it make sense to tweak the code a bit so it's explicit that we're…
wleiAuthorUnsubmitted
Done
ReplyInline Actions

Sounds good!

wlei: Sounds good!
LineLocation IRCallsite = FunctionSamples::getCallSiteIdentifier(DLoc); LineLocation IRCallsite = FunctionSamples::getCallSiteIdentifier(DLoc);
StringRef CalleeName = UnknownIndirectCallee; StringRef CalleeName = UnknownIndirectCallee;
wenleiUnsubmitted
Not Done
ReplyInline Actions

I know this is old code, but what intrinsics are we skipping here, is that just for skipping probe intrinsics or something else? If it's for skipping probes, maybe we can consider this?

if (probe inst) {
  record empty string in IRLocations
else if (call inst) {
  record actual call site in IRLocations
}

or

if (probe inst) {
  record empty string in IRLocations
else if (not call inst) {
  continue
}
record actual call site in IRLocations
wenlei: I know this is old code, but what intrinsics are we skipping here, is that just for skipping…
wleiAuthorUnsubmitted
Done
ReplyInline Actions

IntrinsicInst may not only be used for probe intrinsics. I can't remember clearly, but I vaguely remembered there were a lot of dbg intrinsics that messed up the matching, and the fix was to add this isa<IntrinsicInst>(&I) check.

wlei: `IntrinsicInst` may not only be used for probe intrinsics. I can't remember clearly, but I…
if (Function *Callee = CB->getCalledFunction()) if (Function *Callee = CB->getCalledFunction())
CalleeName = FunctionSamples::getCanonicalFnName(Callee->getName()); CalleeName = FunctionSamples::getCanonicalFnName(Callee->getName());
// Force to overwrite the callee name in case any non-call location was // Force to overwrite the callee name in case any non-call location was
// written before. // written before.
auto R = IRAnchors.emplace(IRCallsite, CalleeName); auto R = IRAnchors.emplace(IRCallsite, CalleeName);
R.first->second = CalleeName; R.first->second = CalleeName;
assert((!FunctionSamples::ProfileIsProbeBased || R.second || assert((!FunctionSamples::ProfileIsProbeBased || R.second ||
▲ Show 20 Lines • Show All 210 Lines • ▼ Show 20 Lines if (!IsMatchedAnchor) {
LLVM_DEBUG(dbgs() << "Location is matched from " << Loc << " to " LLVM_DEBUG(dbgs() << "Location is matched from " << Loc << " to "
<< Candidate << "\n"); << Candidate << "\n");
LastMatchedNonAnchors.emplace_back(Loc); LastMatchedNonAnchors.emplace_back(Loc);
} }
} }
} }
void SampleProfileMatcher::runOnFunction(const Function &F, void SampleProfileMatcher::runOnFunction(const Function &F,
const FunctionSamples &FS) { const FunctionSamples &FS) {
// Anchors for IR. It's a map from IR location to callee name, callee name is // Anchors for IR. It's a map from IR location to callee name, callee name is
wenleiUnsubmitted
Not Done
ReplyInline Actions

As I read the code I noticed that "non-direct-call site" can be confusing as it can mean call site that is not direct call. But we really mean to say is 1) not a call site, 2) indirect call site?

wenlei: As I read the code I noticed that "non-direct-call site" can be confusing as it can mean call…
wenleiUnsubmitted
Not Done
ReplyInline Actions

Can you update the comments to avoid ambiguity of "non-direct-call site"?

wenlei: Can you update the comments to avoid ambiguity of "non-direct-call site"?
// empty for non-call instruction and use a dummy name(UnknownIndirectCallee) // empty for non-call instruction and use a dummy name(UnknownIndirectCallee)
// for unknown indrect callee name. // for unknown indrect callee name.
std::map<LineLocation, StringRef> IRAnchors; std::map<LineLocation, StringRef> IRAnchors;
findIRAnchors(F, IRAnchors); findIRAnchors(F, IRAnchors);
// Anchors for profile. It's a map from callsite location to a set of callee // Anchors for profile. It's a map from callsite location to a set of callee
// name. // name.
std::map<LineLocation, StringSet<>> ProfileAnchors; std::map<LineLocation, StringSet<>> ProfileAnchors;
findProfileAnchors(FS, ProfileAnchors); findProfileAnchors(FS, ProfileAnchors);
▲ Show 20 Lines • Show All 286 Lines • Show Last 20 Lines

llvm/test/Transforms/SampleProfile/Inputs/pseudo-probe-stale-profile-matching-lto.prof

  • This file was added.
main:6822:0
1: 0
2: 614
3: 614
4: 603
5: 0
7: 631 bar:631
9: 636 bar:636
6: foo:1900
1: 614
2: 554
3: 59
4: 614
6: 59 bar:59
!CFGChecksum: 563022570642068
8: foo:1859
1: 589
2: 526
3: 63
4: 618
6: 63 bar:65
!CFGChecksum: 563022570642068
!CFGChecksum: 1125988587804525
bar:2401:2401
1: 2401
!CFGChecksum: 4294967295

llvm/test/Transforms/SampleProfile/pseudo-probe-stale-profile-matching-lto.ll

  • This file was added.
; REQUIRES: x86_64-linux
; REQUIRES: asserts
; RUN: opt < %s -passes=sample-profile -sample-profile-file=%S/Inputs/pseudo-probe-stale-profile-matching-lto.prof --salvage-stale-profile -S --debug-only=sample-profile,sample-profile-impl 2>&1 | FileCheck %s
; CHECK: Run stale profile matching for main
; CHECK: Location is matched from 1 to 1
; CHECK: Location is matched from 2 to 2
; CHECK: Location is matched from 4 to 4
; CHECK: Location is matched from 6 to 6
; CHECK: Location is matched from 7 to 7
; CHECK: Location is matched from 8 to 8
; CHECK: Location is matched from 10 to 10
; CHECK: Callsite with callee:foo is matched from 12 to 6
; CHECK: Location is rematched backwards from 7 to 1
; CHECK: Location is rematched backwards from 8 to 2
; CHECK: Location is rematched backwards from 10 to 4
; CHECK: Callsite with callee:bar is matched from 13 to 7
; CHECK: Callsite with callee:foo is matched from 14 to 8
; CHECK: Callsite with callee:bar is matched from 15 to 9
target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
target triple = "x86_64-unknown-linux-gnu"
@x = internal global i32 1, align 4, !dbg !0
; Function Attrs: nounwind uwtable
define dso_local i32 @main() local_unnamed_addr #0 !dbg !49 !prof !55 {
call void @llvm.pseudoprobe(i64 -2624081020897602054, i64 1, i32 0, i64 -1), !dbg !56
call void @llvm.dbg.value(metadata i32 0, metadata !53, metadata !DIExpression()), !dbg !57
call void @llvm.pseudoprobe(i64 -2624081020897602054, i64 2, i32 0, i64 6148914874488455168), !dbg !58
br label %1, !dbg !61
1: ; preds = %17, %0
%2 = phi i32 [ 0, %0 ], [ %28, %17 ]
call void @llvm.dbg.value(metadata i32 %2, metadata !53, metadata !DIExpression()), !dbg !57
call void @llvm.pseudoprobe(i64 -2624081020897602054, i64 4, i32 0, i64 -1), !dbg !63
%3 = load volatile i32, ptr @x, align 4, !dbg !65, !tbaa !66
call void @llvm.dbg.value(metadata i32 %2, metadata !70, metadata !DIExpression()), !dbg !76
call void @llvm.dbg.value(metadata i32 %3, metadata !75, metadata !DIExpression()), !dbg !76
call void @llvm.pseudoprobe(i64 6699318081062747564, i64 1, i32 0, i64 -1), !dbg !79
%4 = urem i32 %2, 10, !dbg !81
%5 = icmp eq i32 %4, 0, !dbg !81
%6 = zext i1 %5 to i32, !dbg !82
%7 = add nsw i32 %3, %6, !dbg !82
%8 = call i32 @bar(i32 noundef %7) #4, !dbg !83, !prof !84
call void @llvm.pseudoprobe(i64 6699318081062747564, i64 4, i32 0, i64 -1), !dbg !85
%9 = load volatile i32, ptr @x, align 4, !dbg !86, !tbaa !66
%10 = add nsw i32 %9, %8, !dbg !86
store volatile i32 %10, ptr @x, align 4, !dbg !86, !tbaa !66
%11 = load volatile i32, ptr @x, align 4, !dbg !87, !tbaa !66
%12 = call i32 @bar(i32 noundef %11) #4, !dbg !88, !prof !90
%13 = load volatile i32, ptr @x, align 4, !dbg !91, !tbaa !66
%14 = add nsw i32 %13, %12, !dbg !91
store volatile i32 %14, ptr @x, align 4, !dbg !91, !tbaa !66
%15 = load volatile i32, ptr @x, align 4, !dbg !92, !tbaa !66
%16 = icmp slt i32 %15, 0, !dbg !94
br i1 %16, label %30, label %17, !dbg !95, !prof !96
17: ; preds = %1
call void @llvm.pseudoprobe(i64 -2624081020897602054, i64 6, i32 0, i64 -1), !dbg !97
%18 = load volatile i32, ptr @x, align 4, !dbg !98, !tbaa !66
call void @llvm.dbg.value(metadata i32 %2, metadata !70, metadata !DIExpression()), !dbg !99
call void @llvm.dbg.value(metadata i32 %18, metadata !75, metadata !DIExpression()), !dbg !99
call void @llvm.pseudoprobe(i64 6699318081062747564, i64 1, i32 0, i64 -1), !dbg !102
%19 = zext i1 %5 to i32, !dbg !103
%20 = add nsw i32 %18, %19, !dbg !103
%21 = call i32 @bar(i32 noundef %20) #4, !dbg !104, !prof !84
call void @llvm.pseudoprobe(i64 6699318081062747564, i64 4, i32 0, i64 -1), !dbg !105
%22 = load volatile i32, ptr @x, align 4, !dbg !106, !tbaa !66
%23 = add nsw i32 %22, %21, !dbg !106
store volatile i32 %23, ptr @x, align 4, !dbg !106, !tbaa !66
%24 = load volatile i32, ptr @x, align 4, !dbg !107, !tbaa !66
%25 = call i32 @bar(i32 noundef %24) #4, !dbg !108, !prof !90
%26 = load volatile i32, ptr @x, align 4, !dbg !110, !tbaa !66
%27 = add nsw i32 %26, %25, !dbg !110
store volatile i32 %27, ptr @x, align 4, !dbg !110, !tbaa !66
call void @llvm.pseudoprobe(i64 -2624081020897602054, i64 7, i32 0, i64 -1), !dbg !111
%28 = add nuw nsw i32 %2, 1, !dbg !112
call void @llvm.dbg.value(metadata i32 %28, metadata !53, metadata !DIExpression()), !dbg !57
call void @llvm.pseudoprobe(i64 -2624081020897602054, i64 2, i32 0, i64 -6148914324732641280), !dbg !58
%29 = icmp eq i32 %28, 1000000, !dbg !114
br i1 %29, label %30, label %1, !dbg !61, !prof !116, !llvm.loop !117
30: ; preds = %17, %1
%31 = phi i32 [ 1, %1 ], [ 0, %17 ]
call void @llvm.pseudoprobe(i64 -2624081020897602054, i64 8, i32 0, i64 -1), !dbg !121
call void @llvm.pseudoprobe(i64 -2624081020897602054, i64 10, i32 0, i64 -1), !dbg !122
ret i32 %31, !dbg !122
}
; Function Attrs: nocallback nofree nosync nounwind willreturn memory(inaccessiblemem: readwrite)
declare void @llvm.pseudoprobe(i64, i64, i32, i64) #1
; Function Attrs: nocallback nofree nosync nounwind speculatable willreturn memory(none)
declare void @llvm.dbg.value(metadata, metadata, metadata) #2
; Function Attrs: mustprogress nofree norecurse nosync nounwind willreturn memory(none) uwtable
define available_externally dso_local i32 @bar(i32 noundef %0) local_unnamed_addr #3 !dbg !123 !prof !128 {
call void @llvm.dbg.value(metadata i32 %0, metadata !127, metadata !DIExpression()), !dbg !129
call void @llvm.pseudoprobe(i64 -2012135647395072713, i64 1, i32 0, i64 -1), !dbg !130
%2 = add nsw i32 %0, 1, !dbg !131
ret i32 %2, !dbg !132
}
attributes #0 = { nounwind uwtable "disable-tail-calls"="true" "frame-pointer"="all" "min-legal-vector-width"="0" "no-trapping-math"="true" "stack-protector-buffer-size"="8" "target-cpu"="x86-64" "target-features"="+cmov,+cx8,+fxsr,+mmx,+sse,+sse2,+x87" "tune-cpu"="generic" "use-sample-profile" }
attributes #1 = { nocallback nofree nosync nounwind willreturn memory(inaccessiblemem: readwrite) }
attributes #2 = { nocallback nofree nosync nounwind speculatable willreturn memory(none) }
attributes #3 = { mustprogress nofree norecurse nosync nounwind willreturn memory(none) uwtable "disable-tail-calls"="true" "frame-pointer"="all" "min-legal-vector-width"="0" "no-trapping-math"="true" "stack-protector-buffer-size"="8" "target-cpu"="x86-64" "target-features"="+cmov,+cx8,+fxsr,+mmx,+sse,+sse2,+x87" "tune-cpu"="generic" "use-sample-profile" }
attributes #4 = { nounwind }
!llvm.dbg.cu = !{!2, !7}
!llvm.module.flags = !{!9, !10, !11, !12, !13, !14, !15, !16, !17}
!llvm.ident = !{!46, !46}
!llvm.pseudo_probe_desc = !{!47, !48}
!0 = !DIGlobalVariableExpression(var: !1, expr: !DIExpression())
!1 = distinct !DIGlobalVariable(name: "x", scope: !2, file: !3, line: 2, type: !5, isLocal: false, isDefinition: true)
!2 = distinct !DICompileUnit(language: DW_LANG_C11, file: !3, producer: "clang version 18.0.0", isOptimized: true, runtimeVersion: 0, emissionKind: FullDebug, globals: !4, splitDebugInlining: false, debugInfoForProfiling: true, nameTableKind: None)
!3 = !DIFile(filename: "file1.c", directory: "/home/", checksumkind: CSK_MD5, checksum: "03dfbda098b1285e8c9837f184747483")
!4 = !{!0}
!5 = !DIDerivedType(tag: DW_TAG_volatile_type, baseType: !6)
!6 = !DIBasicType(name: "int", size: 32, encoding: DW_ATE_signed)
!7 = distinct !DICompileUnit(language: DW_LANG_C11, file: !8, producer: "clang version 18.0.0 ", isOptimized: true, runtimeVersion: 0, emissionKind: FullDebug, splitDebugInlining: false, debugInfoForProfiling: true, nameTableKind: None)
!8 = !DIFile(filename: "file2.c", directory: "/home", checksumkind: CSK_MD5, checksum: "bd15f56a5a8604ec94485c3dbf9d0cdd")
!9 = !{i32 7, !"Dwarf Version", i32 5}
!10 = !{i32 2, !"Debug Info Version", i32 3}
!11 = !{i32 1, !"wchar_size", i32 4}
!12 = !{i32 8, !"PIC Level", i32 2}
!13 = !{i32 7, !"PIE Level", i32 2}
!14 = !{i32 7, !"uwtable", i32 2}
!15 = !{i32 7, !"frame-pointer", i32 2}
!16 = !{i32 1, !"EnableSplitLTOUnit", i32 0}
!17 = !{i32 1, !"ProfileSummary", !18}
!18 = !{!19, !20, !21, !22, !23, !24, !25, !26, !27, !28}
!19 = !{!"ProfileFormat", !"SampleProfile"}
!20 = !{!"TotalCount", i64 9258}
!21 = !{!"MaxCount", i64 2401}
!22 = !{!"MaxInternalCount", i64 0}
!23 = !{!"MaxFunctionCount", i64 2401}
!24 = !{!"NumCounts", i64 18}
!25 = !{!"NumFunctions", i64 2}
!26 = !{!"IsPartialProfile", i64 0}
!27 = !{!"PartialProfileRatio", double 0.000000e+00}
!28 = !{!"DetailedSummary", !29}
!29 = !{!30, !31, !32, !33, !34, !35, !36, !37, !38, !39, !40, !41, !42, !43, !44, !45}
!30 = !{i32 10000, i64 2401, i32 1}
!31 = !{i32 100000, i64 2401, i32 1}
!32 = !{i32 200000, i64 2401, i32 1}
!33 = !{i32 300000, i64 636, i32 2}
!34 = !{i32 400000, i64 618, i32 4}
!35 = !{i32 500000, i64 614, i32 8}
!36 = !{i32 600000, i64 614, i32 8}
!37 = !{i32 700000, i64 614, i32 8}
!38 = !{i32 800000, i64 589, i32 10}
!39 = !{i32 900000, i64 554, i32 11}
!40 = !{i32 950000, i64 526, i32 12}
!41 = !{i32 990000, i64 59, i32 16}
!42 = !{i32 999000, i64 59, i32 16}
!43 = !{i32 999900, i64 59, i32 16}
!44 = !{i32 999990, i64 59, i32 16}
!45 = !{i32 999999, i64 59, i32 16}
!46 = !{!"clang version 18.0.0"}
!47 = !{i64 6699318081062747564, i64 563022570642068, !"foo"}
!48 = !{i64 -2624081020897602054, i64 1126124211449298, !"main"}
!49 = distinct !DISubprogram(name: "main", scope: !3, file: !3, line: 11, type: !50, scopeLine: 12, flags: DIFlagAllCallsDescribed, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !2, retainedNodes: !52)
!50 = !DISubroutineType(types: !51)
!51 = !{!6}
!52 = !{!53}
!53 = !DILocalVariable(name: "i", scope: !54, file: !3, line: 13, type: !6)
!54 = distinct !DILexicalBlock(scope: !49, file: !3, line: 13, column: 3)
!55 = !{!"function_entry_count", i64 608}
!56 = !DILocation(line: 13, column: 11, scope: !54)
!57 = !DILocation(line: 0, scope: !54)
!58 = !DILocation(line: 13, column: 18, scope: !59)
!59 = !DILexicalBlockFile(scope: !60, file: !3, discriminator: 0)
!60 = distinct !DILexicalBlock(scope: !54, file: !3, line: 13, column: 3)
!61 = !DILocation(line: 13, column: 3, scope: !62)
!62 = !DILexicalBlockFile(scope: !54, file: !3, discriminator: 2)
!63 = !DILocation(line: 14, column: 15, scope: !64)
!64 = distinct !DILexicalBlock(scope: !60, file: !3, line: 13, column: 40)
!65 = !DILocation(line: 14, column: 18, scope: !64)
!66 = !{!67, !67, i64 0}
!67 = !{!"int", !68, i64 0}
!68 = !{!"omnipotent char", !69, i64 0}
!69 = !{!"Simple C/C++ TBAA"}
!70 = !DILocalVariable(name: "i", arg: 1, scope: !71, file: !3, line: 6, type: !6)
!71 = distinct !DISubprogram(name: "foo", scope: !3, file: !3, line: 6, type: !72, scopeLine: 6, flags: DIFlagPrototyped | DIFlagAllCallsDescribed, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !2, retainedNodes: !74)
!72 = !DISubroutineType(types: !73)
!73 = !{!6, !6, !6}
!74 = !{!70, !75}
!75 = !DILocalVariable(name: "p", arg: 2, scope: !71, file: !3, line: 6, type: !6)
!76 = !DILocation(line: 0, scope: !71, inlinedAt: !77)
!77 = distinct !DILocation(line: 14, column: 11, scope: !78)
!78 = !DILexicalBlockFile(scope: !64, file: !3, discriminator: 186646631)
!79 = !DILocation(line: 7, column: 6, scope: !80, inlinedAt: !77)
!80 = distinct !DILexicalBlock(scope: !71, file: !3, line: 7, column: 6)
!81 = !DILocation(line: 7, column: 8, scope: !80, inlinedAt: !77)
!82 = !DILocation(line: 7, column: 6, scope: !71, inlinedAt: !77)
!83 = !DILocation(line: 0, scope: !80, inlinedAt: !77)
!84 = !{!"branch_weights", i64 614}
!85 = !DILocation(line: 9, column: 1, scope: !71, inlinedAt: !77)
!86 = !DILocation(line: 14, column: 8, scope: !64)
!87 = !DILocation(line: 15, column: 15, scope: !64)
!88 = !DILocation(line: 15, column: 11, scope: !89)
!89 = !DILexicalBlockFile(scope: !64, file: !3, discriminator: 186646639)
!90 = !{!"branch_weights", i32 614}
!91 = !DILocation(line: 15, column: 8, scope: !64)
!92 = !DILocation(line: 16, column: 9, scope: !93)
!93 = distinct !DILexicalBlock(scope: !64, file: !3, line: 16, column: 9)
!94 = !DILocation(line: 16, column: 11, scope: !93)
!95 = !DILocation(line: 16, column: 9, scope: !64)
!96 = !{!"branch_weights", i32 0, i32 614}
!97 = !DILocation(line: 18, column: 15, scope: !64)
!98 = !DILocation(line: 18, column: 18, scope: !64)
!99 = !DILocation(line: 0, scope: !71, inlinedAt: !100)
!100 = distinct !DILocation(line: 18, column: 11, scope: !101)
!101 = !DILexicalBlockFile(scope: !64, file: !3, discriminator: 186646647)
!102 = !DILocation(line: 7, column: 6, scope: !80, inlinedAt: !100)
!103 = !DILocation(line: 7, column: 6, scope: !71, inlinedAt: !100)
!104 = !DILocation(line: 0, scope: !80, inlinedAt: !100)
!105 = !DILocation(line: 9, column: 1, scope: !71, inlinedAt: !100)
!106 = !DILocation(line: 18, column: 8, scope: !64)
!107 = !DILocation(line: 19, column: 15, scope: !64)
!108 = !DILocation(line: 19, column: 11, scope: !109)
!109 = !DILexicalBlockFile(scope: !64, file: !3, discriminator: 186646655)
!110 = !DILocation(line: 19, column: 8, scope: !64)
!111 = !DILocation(line: 13, column: 36, scope: !59)
!112 = !DILocation(line: 13, column: 36, scope: !113)
!113 = !DILexicalBlockFile(scope: !60, file: !3, discriminator: 6)
!114 = !DILocation(line: 13, column: 20, scope: !115)
!115 = !DILexicalBlockFile(scope: !60, file: !3, discriminator: 2)
!116 = !{!"branch_weights", i32 608, i32 614}
!117 = distinct !{!117, !118, !119, !120}
!118 = !DILocation(line: 13, column: 3, scope: !54)
!119 = !DILocation(line: 20, column: 3, scope: !54)
!120 = !{!"llvm.loop.mustprogress"}
!121 = !DILocation(line: 0, scope: !49)
!122 = !DILocation(line: 22, column: 1, scope: !49)
!123 = distinct !DISubprogram(name: "bar", scope: !8, file: !8, line: 1, type: !124, scopeLine: 1, flags: DIFlagPrototyped | DIFlagAllCallsDescribed, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !7, retainedNodes: !126)
!124 = !DISubroutineType(types: !125)
!125 = !{!6, !6}
!126 = !{!127}
!127 = !DILocalVariable(name: "x", arg: 1, scope: !123, file: !8, line: 1, type: !6)
!128 = !{!"function_entry_count", i64 2401}
!129 = !DILocation(line: 0, scope: !123)
!130 = !DILocation(line: 5, column: 10, scope: !123)
!131 = !DILocation(line: 5, column: 12, scope: !123)
!132 = !DILocation(line: 5, column: 3, scope: !123)