Page MenuHomePhabricator
Feed Advanced Search

Wed, Apr 21

wmi accepted D100528: [CSSPGO][llvm-profdata] Support trimming cold context when merging profiles.

LGTM, thanks.

Wed, Apr 21, 5:23 PM · Restricted Project
wmi added inline comments to D100528: [CSSPGO][llvm-profdata] Support trimming cold context when merging profiles.
Wed, Apr 21, 3:18 PM · Restricted Project

Mon, Apr 19

wmi accepted D100332: [CSSPGO] Flip SkipPseudoOp to true for MIR APIs..

LGTM, thanks.

Mon, Apr 19, 3:05 PM · Restricted Project

Sun, Apr 18

wmi accepted D100334: [CSSPGO] Exclude pseudo probes from slot index.

LGTM.

Sun, Apr 18, 10:11 PM · Restricted Project
wmi added inline comments to D100528: [CSSPGO][llvm-profdata] Support trimming cold context when merging profiles.
Sun, Apr 18, 9:08 PM · Restricted Project
wmi added inline comments to D100332: [CSSPGO] Flip SkipPseudoOp to true for MIR APIs..
Sun, Apr 18, 1:11 PM · Restricted Project

Wed, Apr 7

wmi accepted D100002: [CSSPGO] Move pseudo probes to the beginning of a block to unblock SelectionDAG combine..

LGTM.

Wed, Apr 7, 8:32 PM · Restricted Project
wmi added inline comments to D100002: [CSSPGO] Move pseudo probes to the beginning of a block to unblock SelectionDAG combine..
Wed, Apr 7, 5:10 PM · Restricted Project

Tue, Mar 30

wmi committed rGd535a05ca1a6: [ThinLTO] During module importing, close one source module before open (authored by wmi).
[ThinLTO] During module importing, close one source module before open
Tue, Mar 30, 2:42 PM
wmi closed D99554: [ThinLTO] During module importing, close one source module before open another one for distributed mode..
Tue, Mar 30, 2:42 PM · Restricted Project, Restricted Project
wmi added inline comments to D99554: [ThinLTO] During module importing, close one source module before open another one for distributed mode..
Tue, Mar 30, 11:38 AM · Restricted Project, Restricted Project
wmi updated the diff for D99554: [ThinLTO] During module importing, close one source module before open another one for distributed mode..

Address Teresa's comment.

Tue, Mar 30, 11:37 AM · Restricted Project, Restricted Project
wmi accepted D99550: keeping prof metadata for IndirectBrInst.

LGTM.

Tue, Mar 30, 10:32 AM · Restricted Project
wmi accepted D99351: [CSSPGO] Top-down processing order based on full profile..

LGTM.

Tue, Mar 30, 9:31 AM · Restricted Project

Mon, Mar 29

wmi added inline comments to D99351: [CSSPGO] Top-down processing order based on full profile..
Mon, Mar 29, 8:35 PM · Restricted Project
wmi added a comment to D99351: [CSSPGO] Top-down processing order based on full profile..

The performance test result is neutral, so I think we can enable UseProfiledCallGraph by default.

Mon, Mar 29, 5:25 PM · Restricted Project
wmi requested review of D99554: [ThinLTO] During module importing, close one source module before open another one for distributed mode..
Mon, Mar 29, 5:13 PM · Restricted Project, Restricted Project
wmi added a comment to D99550: keeping prof metadata for IndirectBrInst.

Thanks for the change. Could you add a test for it?

Mon, Mar 29, 4:26 PM · Restricted Project
wmi committed rG3cbf44190b59: [SampleFDO] Do not scale the magic number NOMORE_ICP_MAGICNUM in value profile (authored by wmi).
[SampleFDO] Do not scale the magic number NOMORE_ICP_MAGICNUM in value profile
Mon, Mar 29, 9:35 AM
wmi closed D99394: [SampleFDO] Do not scale the magic number NOMORE_ICP_MAGICNUM in value profile during profile update..
Mon, Mar 29, 9:34 AM · Restricted Project

Sun, Mar 28

wmi added a comment to D99351: [CSSPGO] Top-down processing order based on full profile..

Seems there is still build error:

Sun, Mar 28, 11:01 PM · Restricted Project
wmi added a comment to D99351: [CSSPGO] Top-down processing order based on full profile..
In D99351#2654236, @hoy wrote:
In D99351#2654048, @wmi wrote:

Just find ProfiledCallGraph.h is not included in the patch so the build failed after applying the patch.

Thanks for trying it. You’ll also need to apply Wenlei’s patch D99146.

Sun, Mar 28, 2:45 PM · Restricted Project

Fri, Mar 26

wmi added a comment to D99351: [CSSPGO] Top-down processing order based on full profile..

Just find ProfiledCallGraph.h is not included in the patch so the build failed after applying the patch.

Fri, Mar 26, 11:30 PM · Restricted Project
wmi accepted D99146: [CSSPGO][llvm-profgen] Context-sensitive global pre-inliner.

LGTM. Thanks.

Fri, Mar 26, 5:08 PM · Restricted Project
wmi added a comment to D99351: [CSSPGO] Top-down processing order based on full profile..

The change is an enhancement to https://reviews.llvm.org/D95988 so it is better to mention D95988 in the description/commit log.

Fri, Mar 26, 1:54 PM · Restricted Project
wmi added inline comments to D99146: [CSSPGO][llvm-profgen] Context-sensitive global pre-inliner.
Fri, Mar 26, 12:28 PM · Restricted Project
wmi added inline comments to D99146: [CSSPGO][llvm-profgen] Context-sensitive global pre-inliner.
Fri, Mar 26, 10:56 AM · Restricted Project
wmi added inline comments to D99146: [CSSPGO][llvm-profgen] Context-sensitive global pre-inliner.
Fri, Mar 26, 10:04 AM · Restricted Project

Thu, Mar 25

wmi accepted D99370: [CSSPGO] Minor tweak for inline candidate priority tie breaker.

LGTM.

Thu, Mar 25, 5:39 PM · Restricted Project
wmi requested review of D99394: [SampleFDO] Do not scale the magic number NOMORE_ICP_MAGICNUM in value profile during profile update..
Thu, Mar 25, 5:30 PM · Restricted Project

Wed, Mar 24

wmi added a comment to D99146: [CSSPGO][llvm-profgen] Context-sensitive global pre-inliner.

ThinLTO is known to have issues related to profile update (cross module), so we were thinking something similar in ThinLink phase.

This is the exact problem we are trying to mitigate. We also considered doing this in ThinLink but adjusting profiles for thin-backends and communicating inline decisions to thin-backends would add quite a bit of complexity, which could also slow down ThinLink. With CSSPGO, doing it in profile generation and use adjusted profile to convey inline estimation/suggestion is much simpler and cheaper.

One of the issues is that the pre-inlining needs to make similar decisions as the compiler. How well is the preinliner doing in this regard?

Yes, this is a challenge. We don't have data yet, but I hope with some tuning we can get them to be close. One problem with doing pre-inlining is we don't have a lot of information that compiler can see from IR, though if needed some of that can be embedded into binary (some metadata in probe descriptor, etc.) for preinliner. I hope a more accurate view on machine code byte size for inline cost can offset some of the disadvantages due to lack of IR.

Wed, Mar 24, 11:00 PM · Restricted Project

Mar 21 2021

wmi accepted D98921: [CSSPGO][llvm-profgen] Use profile summary based threshold for context trimming and merging.

LGTM.

Mar 21 2021, 10:29 PM · Restricted Project

Mar 19 2021

wmi added inline comments to D98921: [CSSPGO][llvm-profgen] Use profile summary based threshold for context trimming and merging.
Mar 19 2021, 1:53 PM · Restricted Project

Mar 18 2021

wmi added inline comments to D98835: [SampleFDO] Don't mix up the existing indirect call value profile with the new value profile annotated after inlining..
Mar 18 2021, 11:53 AM · Restricted Project
wmi committed rG14756b70eeba: [SampleFDO] Don't mix up the existing indirect call value profile with the new (authored by wmi).
[SampleFDO] Don't mix up the existing indirect call value profile with the new
Mar 18 2021, 9:55 AM
wmi closed D98835: [SampleFDO] Don't mix up the existing indirect call value profile with the new value profile annotated after inlining..
Mar 18 2021, 9:55 AM · Restricted Project

Mar 17 2021

wmi updated the diff for D98835: [SampleFDO] Don't mix up the existing indirect call value profile with the new value profile annotated after inlining..

Address David's comment.

Mar 17 2021, 10:47 PM · Restricted Project
wmi added inline comments to D98835: [SampleFDO] Don't mix up the existing indirect call value profile with the new value profile annotated after inlining..
Mar 17 2021, 10:47 PM · Restricted Project
wmi requested review of D98835: [SampleFDO] Don't mix up the existing indirect call value profile with the new value profile annotated after inlining..
Mar 17 2021, 6:26 PM · Restricted Project

Mar 15 2021

wmi accepted D98226: [CSSPGO][llvm-profgen] Fix getCanonicalFnName usage in llvm-profgen.

LGTM. Thanks.

Mar 15 2021, 5:20 PM · Restricted Project
wmi accepted D98590: [CSSPGO] Load context profile for external functions in PreLink and populate ThinLTO import list.

LGTM. Thanks!

Mar 15 2021, 11:56 AM · Restricted Project

Mar 14 2021

wmi added inline comments to D98226: [CSSPGO][llvm-profgen] Fix getCanonicalFnName usage in llvm-profgen.
Mar 14 2021, 10:40 PM · Restricted Project
wmi added inline comments to D98590: [CSSPGO] Load context profile for external functions in PreLink and populate ThinLTO import list.
Mar 14 2021, 12:57 PM · Restricted Project

Mar 12 2021

wmi added a comment to D98389: [IndirectCallPromotion] Don't strip ".__uniq." suffix when it strips ".llvm." suffix..
In D98389#2620729, @wmi wrote:

FWIW, I'm seeing errors like "Malformed instrumentation profile data" when building code for a i686-w64-mingw32 target after this commit. I've yet to reduce the actual trigger for it, but I thought I'd let you know...

Thanks Martin for reporting the error. I think the error is thrown in the call of addFuncName(OtherFuncName) here:
https://github.com/llvm/llvm-project/blob/b552adf8b388a4fbdaa6fb46bdedc83fc738fc2b/llvm/lib/ProfileData/InstrProf.cpp#L376

Could you help to get the "PGOFuncName" string before addFuncName throws the error? Then we can know what is wrong.

Ok, sure. Here's a full reproducer:

$ cat ../repro.c 
static void func(void) { }
void (*funcptr)(void) = func;
$ clang -target x86_64-linux-gnu -c ../repro.c -O2
PGOFuncName "../repro.c:func"
OtherFuncName ""
Program aborted due to an unhandled Error:
Malformed instrumentation profile data
Mar 12 2021, 1:55 PM · Restricted Project
wmi committed rGef9d7db72362: [IndirectCallPromotion] Recommit "Don't strip ".__uniq." suffix when it strips (authored by wmi).
[IndirectCallPromotion] Recommit "Don't strip ".__uniq." suffix when it strips
Mar 12 2021, 1:48 PM

Mar 11 2021

wmi added a comment to D98389: [IndirectCallPromotion] Don't strip ".__uniq." suffix when it strips ".llvm." suffix..

FWIW, I'm seeing errors like "Malformed instrumentation profile data" when building code for a i686-w64-mingw32 target after this commit. I've yet to reduce the actual trigger for it, but I thought I'd let you know...

Mar 11 2021, 2:58 PM · Restricted Project
wmi committed rG90dfbeef5982: [IndirectCallPromotion] Don't strip ".__uniq." suffix when it strips (authored by wmi).
[IndirectCallPromotion] Don't strip ".__uniq." suffix when it strips
Mar 11 2021, 11:09 AM
wmi closed D98389: [IndirectCallPromotion] Don't strip ".__uniq." suffix when it strips ".llvm." suffix..
Mar 11 2021, 11:09 AM · Restricted Project
wmi accepted D98187: [SamplePGO] Skip inlinee profile scaling for sample loader inlining.
In D98187#2618612, @wmi wrote:

@wmi any concern with landing this change? I don't expect any issues, but just to be prudent do you want to experiment before this is landed?

That is a nice catch! I will definitely experiment it and expect some improvement from it. Will have the data tomorrow and get back.

Mar 11 2021, 9:31 AM · Restricted Project

Mar 10 2021

wmi updated the diff for D98389: [IndirectCallPromotion] Don't strip ".__uniq." suffix when it strips ".llvm." suffix..

Address reviewers' comments.

Mar 10 2021, 11:00 PM · Restricted Project
wmi added a comment to D98187: [SamplePGO] Skip inlinee profile scaling for sample loader inlining.

@wmi any concern with landing this change? I don't expect any issues, but just to be prudent do you want to experiment before this is landed?

Mar 10 2021, 9:59 PM · Restricted Project
wmi added inline comments to D98389: [IndirectCallPromotion] Don't strip ".__uniq." suffix when it strips ".llvm." suffix..
Mar 10 2021, 9:43 PM · Restricted Project
wmi added inline comments to D98389: [IndirectCallPromotion] Don't strip ".__uniq." suffix when it strips ".llvm." suffix..
Mar 10 2021, 9:23 PM · Restricted Project
wmi requested review of D98389: [IndirectCallPromotion] Don't strip ".__uniq." suffix when it strips ".llvm." suffix..
Mar 10 2021, 8:47 PM · Restricted Project

Mar 9 2021

wmi committed rGee35784a909b: [SampleFDO] Support enabling -funique-internal-linkage-name. (authored by wmi).
[SampleFDO] Support enabling -funique-internal-linkage-name.
Mar 9 2021, 9:42 PM
wmi closed D96932: [SampleFDO] Support enabling -funique-internal-linkage-name.
Mar 9 2021, 9:42 PM · Restricted Project

Mar 5 2021

wmi accepted D97998: [CSSPGO] llvm-profdata support for CS profile..

LGTM.

Mar 5 2021, 6:15 PM · Restricted Project
wmi added inline comments to D97998: [CSSPGO] llvm-profdata support for CS profile..
Mar 5 2021, 3:31 PM · Restricted Project

Mar 4 2021

wmi committed rG2357d29335f2: [SampleFDO] Another fix to prevent repeated indirect call promotion in (authored by wmi).
[SampleFDO] Another fix to prevent repeated indirect call promotion in
Mar 4 2021, 6:44 PM
wmi closed D97350: [SampleFDO] Another fix to prevent repeated indirect call promotion in sample loader pass.
Mar 4 2021, 6:44 PM · Restricted Project
wmi updated the diff for D97350: [SampleFDO] Another fix to prevent repeated indirect call promotion in sample loader pass.

Address Hongtao's comment.

Mar 4 2021, 1:24 PM · Restricted Project
wmi added inline comments to D97350: [SampleFDO] Another fix to prevent repeated indirect call promotion in sample loader pass.
Mar 4 2021, 1:23 PM · Restricted Project

Mar 3 2021

wmi updated the diff for D97350: [SampleFDO] Another fix to prevent repeated indirect call promotion in sample loader pass.

Address Hongtao's comment.

Mar 3 2021, 8:18 PM · Restricted Project
wmi added inline comments to D97350: [SampleFDO] Another fix to prevent repeated indirect call promotion in sample loader pass.
Mar 3 2021, 7:34 PM · Restricted Project
wmi updated the diff for D97350: [SampleFDO] Another fix to prevent repeated indirect call promotion in sample loader pass.

Address David, Wenlei and Hongtao's comments.

Mar 3 2021, 5:23 PM · Restricted Project
wmi added inline comments to D97350: [SampleFDO] Another fix to prevent repeated indirect call promotion in sample loader pass.
Mar 3 2021, 5:21 PM · Restricted Project
wmi added a comment to D97350: [SampleFDO] Another fix to prevent repeated indirect call promotion in sample loader pass.
In D97350#2601716, @hoy wrote:

Thank for the fix. I've been seeing an assert failure related to ICP. Do you think that can be fixed by this patch?

llvm/lib/Analysis/IndirectCallPromotionAnalysis.cpp:78: uint32_t llvm::ICallPromotionAnalysis::getProfitablePromotionCandidates(const llvm::Instruction*, uint32_t, uint64_t): Assertion `Count <= RemainingCount' failed.

Mar 3 2021, 4:39 PM · Restricted Project
wmi updated the diff for D97350: [SampleFDO] Another fix to prevent repeated indirect call promotion in sample loader pass.

Address David's comment. By using -1 to represent targets which have been promoted in the value profile, we only have to sort the values in count descending order, and choose the top MaxNumPromotions values whenever we store or load the values from value profile.

Mar 3 2021, 3:05 PM · Restricted Project

Mar 2 2021

wmi accepted D95962: [CSSPGO] Introducing dangling pseudo probes..

LGTM.

Mar 2 2021, 11:12 PM · Restricted Project
wmi added inline comments to D95962: [CSSPGO] Introducing dangling pseudo probes..
Mar 2 2021, 10:56 PM · Restricted Project
wmi added inline comments to D95962: [CSSPGO] Introducing dangling pseudo probes..
Mar 2 2021, 8:08 PM · Restricted Project

Mar 1 2021

wmi accepted D97481: [CSSPGO] Unblocking optimizations by dangling pseudo probes..

LGTM.

Mar 1 2021, 8:43 AM · Restricted Project

Feb 28 2021

wmi added inline comments to D97481: [CSSPGO] Unblocking optimizations by dangling pseudo probes..
Feb 28 2021, 10:22 PM · Restricted Project

Feb 27 2021

wmi committed rG7fb400112f6c: [SampleFDO] Add a cutoff flag to control how many symbols will be included (authored by wmi).
[SampleFDO] Add a cutoff flag to control how many symbols will be included
Feb 27 2021, 11:16 PM
wmi closed D97623: [SampleFDO] Add a cutoff flag to control how many symbols will be included into profile symbol list..
Feb 27 2021, 11:16 PM · Restricted Project
wmi updated the diff for D97623: [SampleFDO] Add a cutoff flag to control how many symbols will be included into profile symbol list..

Address Teresa's comment.

Feb 27 2021, 11:02 PM · Restricted Project
wmi added inline comments to D97623: [SampleFDO] Add a cutoff flag to control how many symbols will be included into profile symbol list..
Feb 27 2021, 11:01 PM · Restricted Project
wmi requested review of D97623: [SampleFDO] Add a cutoff flag to control how many symbols will be included into profile symbol list..
Feb 27 2021, 9:56 PM · Restricted Project

Feb 26 2021

wmi updated the diff for D96932: [SampleFDO] Support enabling -funique-internal-linkage-name.

Fixed a typo.

Feb 26 2021, 6:11 PM · Restricted Project
wmi updated the diff for D96932: [SampleFDO] Support enabling -funique-internal-linkage-name.

Address Wenlei's comment. Also change the default elide strategy to "selected" because I think that is the most commonly used strategy.

Feb 26 2021, 6:08 PM · Restricted Project
wmi added inline comments to D97481: [CSSPGO] Unblocking optimizations by dangling pseudo probes..
Feb 26 2021, 3:58 PM · Restricted Project
wmi added a comment to D97481: [CSSPGO] Unblocking optimizations by dangling pseudo probes..

A general question, do we care about where we should put dangling probe? Currently dangling probes are scattered in some predecessor/successor blocks they don't belong to. Looks like they can be anywhere. Wondering whether it is better to collect all of them to some place (entry block?). I am not sure about it either. Just want to know what you think.

That's a very good question. We don't care about where a dangling probe is placed technically. But in reality placing dangling probes around the original place they are from helps IR readability. It can help reason about why they are dangled.

Feb 26 2021, 3:42 PM · Restricted Project

Feb 25 2021

wmi added a comment to D97481: [CSSPGO] Unblocking optimizations by dangling pseudo probes..

A general question, do we care about where we should put dangling probe? Currently dangling probes are scattered in some predecessor/successor blocks they don't belong to. Looks like they can be anywhere. Wondering whether it is better to collect all of them to some place (entry block?). I am not sure about it either. Just want to know what you think.

Feb 25 2021, 6:22 PM · Restricted Project
wmi accepted D97482: [CSSPGO] Deduplicating dangling pseudo probes..

LGTM.

Feb 25 2021, 5:08 PM · Restricted Project
wmi added inline comments to D97482: [CSSPGO] Deduplicating dangling pseudo probes..
Feb 25 2021, 4:28 PM · Restricted Project
wmi added inline comments to D97482: [CSSPGO] Deduplicating dangling pseudo probes..
Feb 25 2021, 3:49 PM · Restricted Project
wmi added inline comments to D96932: [SampleFDO] Support enabling -funique-internal-linkage-name.
Feb 25 2021, 11:00 AM · Restricted Project

Feb 24 2021

wmi added a comment to D95962: [CSSPGO] Introducing dangling pseudo probes..

Thanks for the detailed description. That really helps for better understanding the patch.

Feb 24 2021, 11:32 PM · Restricted Project
wmi added a comment to D96932: [SampleFDO] Support enabling -funique-internal-linkage-name.

Actually we are not canonicalizing the names in profile from compiler side, right? We are only canonicalizing names from IR to match names from profile.

Feb 24 2021, 11:02 PM · Restricted Project
wmi added a comment to D95962: [CSSPGO] Introducing dangling pseudo probes..

Sorry I missed this patch. I will look at it soon.

Feb 24 2021, 4:31 PM · Restricted Project
wmi added a comment to D96932: [SampleFDO] Support enabling -funique-internal-linkage-name.
In D96932#2585530, @hoy wrote:
In D96932#2585513, @wmi wrote:
In D96932#2584061, @hoy wrote:

BTW, in case of duplicated functions due to multi-versioning, the profile generator may not be able to merge them as expected without properly canonicalizing function names, the compiler might be able to achieve that though. As of now, it seems that the last profile for a given canonical name always supersedes all others in the compiler reader. This may need a change for a "merge" instead.

Right. Do context string need to be canonicalized? If it is needed, the current canonicalization needs some enhancement to support that.

A couple places in samplecontexttracker will need call getCanonicalFnName. What enhancement is needed in addition to that?

Feb 24 2021, 11:11 AM · Restricted Project
wmi added a comment to D96932: [SampleFDO] Support enabling -funique-internal-linkage-name.
In D96932#2584061, @hoy wrote:

BTW, in case of duplicated functions due to multi-versioning, the profile generator may not be able to merge them as expected without properly canonicalizing function names, the compiler might be able to achieve that though. As of now, it seems that the last profile for a given canonical name always supersedes all others in the compiler reader. This may need a change for a "merge" instead.

Feb 24 2021, 10:57 AM · Restricted Project

Feb 23 2021

wmi added a comment to D96932: [SampleFDO] Support enabling -funique-internal-linkage-name.
In D96932#2583927, @hoy wrote:

Are the test failures related?

Feb 23 2021, 10:41 PM · Restricted Project
wmi added a comment to D97350: [SampleFDO] Another fix to prevent repeated indirect call promotion in sample loader pass.

The logic seems complicated. Is it better to use (uint64)-1 as the special value to mark promoted candidates?
( I posted this comment on the wrong thread).

Feb 23 2021, 8:53 PM · Restricted Project
wmi added inline comments to D96932: [SampleFDO] Support enabling -funique-internal-linkage-name.
Feb 23 2021, 8:46 PM · Restricted Project
wmi added inline comments to D96932: [SampleFDO] Support enabling -funique-internal-linkage-name.
Feb 23 2021, 6:03 PM · Restricted Project
wmi requested review of D97350: [SampleFDO] Another fix to prevent repeated indirect call promotion in sample loader pass.
Feb 23 2021, 5:45 PM · Restricted Project
wmi updated the diff for D96932: [SampleFDO] Support enabling -funique-internal-linkage-name.

Address Hongtao's comment.

Feb 23 2021, 5:03 PM · Restricted Project
wmi added inline comments to D96932: [SampleFDO] Support enabling -funique-internal-linkage-name.
Feb 23 2021, 4:36 PM · Restricted Project

Feb 22 2021

wmi accepted D96981: [SampleFDO][NFC] Refactor: make SampleProfileLoaderBaseImpl a template class.

LGTM.

Feb 22 2021, 10:20 PM · Restricted Project