LGTM, thanks.

LGTM, thanks.

LGTM.

LGTM.

Address Teresa's comment.

LGTM.

LGTM.

The performance test result is neutral, so I think we can enable UseProfiledCallGraph by default.

Thanks for the change. Could you add a test for it?

Seems there is still build error:

In D99351#2654236, @hoy wrote:

In D99351#2654048, @wmi wrote:

Just find ProfiledCallGraph.h is not included in the patch so the build failed after applying the patch.

Thanks for trying it. You’ll also need to apply Wenlei’s patch D99146.

Just find ProfiledCallGraph.h is not included in the patch so the build failed after applying the patch.

LGTM. Thanks.

The change is an enhancement to https://reviews.llvm.org/D95988 so it is better to mention D95988 in the description/commit log.

LGTM.

In D99146#2649599, @wenlei wrote:

In D99146#2649557, @davidxl wrote:

ThinLTO is known to have issues related to profile update (cross module), so we were thinking something similar in ThinLink phase.

This is the exact problem we are trying to mitigate. We also considered doing this in ThinLink but adjusting profiles for thin-backends and communicating inline decisions to thin-backends would add quite a bit of complexity, which could also slow down ThinLink. With CSSPGO, doing it in profile generation and use adjusted profile to convey inline estimation/suggestion is much simpler and cheaper.

One of the issues is that the pre-inlining needs to make similar decisions as the compiler. How well is the preinliner doing in this regard?

Yes, this is a challenge. We don't have data yet, but I hope with some tuning we can get them to be close. One problem with doing pre-inlining is we don't have a lot of information that compiler can see from IR, though if needed some of that can be embedded into binary (some metadata in probe descriptor, etc.) for preinliner. I hope a more accurate view on machine code byte size for inline cost can offset some of the disadvantages due to lack of IR.

LGTM.

Address David's comment.

LGTM. Thanks.

LGTM. Thanks!

In D98389#2621398, @mstorsjo wrote:

In D98389#2620729, @wmi wrote:

In D98389#2620644, @mstorsjo wrote:

FWIW, I'm seeing errors like "Malformed instrumentation profile data" when building code for a i686-w64-mingw32 target after this commit. I've yet to reduce the actual trigger for it, but I thought I'd let you know...

Thanks Martin for reporting the error. I think the error is thrown in the call of addFuncName(OtherFuncName) here:
https://github.com/llvm/llvm-project/blob/b552adf8b388a4fbdaa6fb46bdedc83fc738fc2b/llvm/lib/ProfileData/InstrProf.cpp#L376

Could you help to get the "PGOFuncName" string before addFuncName throws the error? Then we can know what is wrong.

Ok, sure. Here's a full reproducer:

$ cat ../repro.c 
static void func(void) { }
void (*funcptr)(void) = func;
$ clang -target x86_64-linux-gnu -c ../repro.c -O2
PGOFuncName "../repro.c:func"
OtherFuncName ""
Program aborted due to an unhandled Error:
Malformed instrumentation profile data

In D98389#2620644, @mstorsjo wrote:

FWIW, I'm seeing errors like "Malformed instrumentation profile data" when building code for a i686-w64-mingw32 target after this commit. I've yet to reduce the actual trigger for it, but I thought I'd let you know...

In D98187#2618612, @wmi wrote:

In D98187#2618605, @wenlei wrote:

@wmi any concern with landing this change? I don't expect any issues, but just to be prudent do you want to experiment before this is landed?

That is a nice catch! I will definitely experiment it and expect some improvement from it. Will have the data tomorrow and get back.

Address reviewers' comments.

In D98187#2618605, @wenlei wrote:

@wmi any concern with landing this change? I don't expect any issues, but just to be prudent do you want to experiment before this is landed?

LGTM.

Address Hongtao's comment.

Address Hongtao's comment.

Address David, Wenlei and Hongtao's comments.

In D97350#2601716, @hoy wrote:

Thank for the fix. I've been seeing an assert failure related to ICP. Do you think that can be fixed by this patch?

llvm/lib/Analysis/IndirectCallPromotionAnalysis.cpp:78: uint32_t llvm::ICallPromotionAnalysis::getProfitablePromotionCandidates(const llvm::Instruction*, uint32_t, uint64_t): Assertion `Count <= RemainingCount' failed.

Address David's comment. By using -1 to represent targets which have been promoted in the value profile, we only have to sort the values in count descending order, and choose the top MaxNumPromotions values whenever we store or load the values from value profile.

LGTM.

LGTM.

Address Teresa's comment.

Fixed a typo.

Address Wenlei's comment. Also change the default elide strategy to "selected" because I think that is the most commonly used strategy.

A general question, do we care about where we should put dangling probe? Currently dangling probes are scattered in some predecessor/successor blocks they don't belong to. Looks like they can be anywhere. Wondering whether it is better to collect all of them to some place (entry block?). I am not sure about it either. Just want to know what you think.

That's a very good question. We don't care about where a dangling probe is placed technically. But in reality placing dangling probes around the original place they are from helps IR readability. It can help reason about why they are dangled.

A general question, do we care about where we should put dangling probe? Currently dangling probes are scattered in some predecessor/successor blocks they don't belong to. Looks like they can be anywhere. Wondering whether it is better to collect all of them to some place (entry block?). I am not sure about it either. Just want to know what you think.

LGTM.

Thanks for the detailed description. That really helps for better understanding the patch.

In D96932#2586378, @wenlei wrote:

Actually we are not canonicalizing the names in profile from compiler side, right? We are only canonicalizing names from IR to match names from profile.

Sorry I missed this patch. I will look at it soon.

In D96932#2585530, @hoy wrote:

In D96932#2585513, @wmi wrote:

In D96932#2584061, @hoy wrote:

BTW, in case of duplicated functions due to multi-versioning, the profile generator may not be able to merge them as expected without properly canonicalizing function names, the compiler might be able to achieve that though. As of now, it seems that the last profile for a given canonical name always supersedes all others in the compiler reader. This may need a change for a "merge" instead.

Right. Do context string need to be canonicalized? If it is needed, the current canonicalization needs some enhancement to support that.

A couple places in samplecontexttracker will need call getCanonicalFnName. What enhancement is needed in addition to that?

In D96932#2584061, @hoy wrote:

BTW, in case of duplicated functions due to multi-versioning, the profile generator may not be able to merge them as expected without properly canonicalizing function names, the compiler might be able to achieve that though. As of now, it seems that the last profile for a given canonical name always supersedes all others in the compiler reader. This may need a change for a "merge" instead.

In D96932#2583927, @hoy wrote:

Are the test failures related?

In D97350#2583817, @davidxl wrote:

The logic seems complicated. Is it better to use (uint64)-1 as the special value to mark promoted candidates?
( I posted this comment on the wrong thread).

Address Hongtao's comment.

LGTM.