This is an archive of the discontinued LLVM Phabricator instance.

[CSSPGO][llvm-profdata] Support trimming cold context when merging profiles
ClosedPublic

Authored by wenlei on Apr 14 2021, 10:58 PM.

Details

Summary

The change adds support for triming and merging cold context when mergine CSSPGO profiles using llvm-profdata. This is similar to the context profile trimming in llvm-profgen, however the flexibility to trim cold context after profile is generated can be useful.

A bug fix is also included to make sure PSI thresholds are always calcuated before query.

Diff Detail

Event Timeline

wenlei created this revision.Apr 14 2021, 10:58 PM
wenlei requested review of this revision.Apr 14 2021, 10:58 PM
Herald added a project: Restricted Project. · View Herald TranscriptApr 14 2021, 10:58 PM
wenlei added inline comments.Apr 14 2021, 11:02 PM
llvm/tools/llvm-profdata/CMakeLists.txt
2 ↗(On Diff #337638)

This is because we now use ProfileSummaryInfo.cpp, so we can get all consistent hot/cold threshold computation and related switches, though pulling in entire Analysis lib is less than ideal. I'm thinking about moving ProfileSummaryInfo into ProfileData/ProfileSummaryBuildder.cpo from Analysis/ProfileSummaryInfo.cpp. But I'd like to get feedback before I move things around.

wmi added inline comments.Apr 18 2021, 9:08 PM
llvm/tools/llvm-profdata/CMakeLists.txt
2 ↗(On Diff #337638)

Extract ProfileSummaryInfo from Analysis/ directory sounds reasonable to me. How about move it to Core -- lib/IR/ProfileSummary.cpp for example? The reason is ProfileSummaryInfo interface is used widely. If someday we want to use the interface in Core or Support which ProfileData depends on, it will introduce cyclic dependence. Putting it in Core will clear the problem.

wenlei added inline comments.Apr 19 2021, 10:10 AM
llvm/tools/llvm-profdata/CMakeLists.txt
2 ↗(On Diff #337638)

Sounds good. Thanks for the suggestion.

wenlei added inline comments.Apr 21 2021, 11:21 AM
llvm/tools/llvm-profdata/CMakeLists.txt
2 ↗(On Diff #337638)

Looks like extracting ProfileSummaryInfo to Core isn't a trivial move. ProfileSummaryInfo has a bunch of interfaces taking BFI as parameter, and also offers invalidation, so it depends on Analysis. It's fine when ProfileSummaryInfo itself is part of Analysis, but if we move it to Core, we will have cyclic dependency. For the same reason, we can't move it to ProfileData either.

If we want to move it, we will need to break it up, so the part that does the threshold computation can be moved, and the part that does the query (relies on Analysis) stays.

Now what I'm thinking about is only moving the threshold/cutoff switches in ProfileSummaryInfo.cpp along with part of ProfileSummaryInfo::computeThresholds into ProfileData/ProfileSummaryBuilder. That won't change dependencies as ProfileSummaryInfo uses ProfileSummaryBuilder already. But it lets tools that only uses ProfileData access consistent threshold computation too.

wenlei updated this revision to Diff 339359.Apr 21 2021, 1:02 PM

Move threshold knobs into ProfileSummaryBuilder, avoid pulling Analysis lib into tools.

wenlei updated this revision to Diff 339360.Apr 21 2021, 1:06 PM

Cleanup.

wmi added inline comments.Apr 21 2021, 3:18 PM
llvm/tools/llvm-profdata/CMakeLists.txt
2 ↗(On Diff #337638)

Ok, that sounds good to me. Thanks for trying to find a good solution.

llvm/tools/llvm-profdata/llvm-profdata.cpp
887–891

What will happen if these two flags are enabled for non-CSSPGO sample profile?

llvm/tools/llvm-profgen/CMakeLists.txt
6 ↗(On Diff #337638)

After the PSI switch and threshold are moved to ProfData, no need to depend on Analysis anymore?

The dependency on Analysis should be removed in the latest version.

llvm/tools/llvm-profdata/llvm-profdata.cpp
887–891

Good catch, they should be ignored. Added a check.

wenlei updated this revision to Diff 339409.Apr 21 2021, 3:31 PM

Ignore trim/merge flag for non-CS profile.

wmi accepted this revision.Apr 21 2021, 5:23 PM

LGTM, thanks.

This revision is now accepted and ready to land.Apr 21 2021, 5:23 PM
This revision was landed with ongoing or failed builds.Apr 22 2021, 12:43 AM
This revision was automatically updated to reflect the committed changes.