This is an archive of the discontinued LLVM Phabricator instance.

Differential D118203

[CSSPGO] Adjust SampleContextFrameVector to be a shorter SmallVector
ClosedPublic

Authored by hoy on Jan 25 2022, 4:13 PM.

Details

Reviewers
wenlei
wlei
Commits
rGcc4beda039ba: [CSSPGO] Adjust SampleContextFrameVector to be a shorter SmallVector
Summary

It appears that some memory saving can be archived by tweaking the existing SampleContextFrameVector setup. Below are memory usage for two benchmarks before and after this change.

  1. Medium benchmark:
    • Before:
note: After computeSizeForProfiledFunctions
note: VM: 12.50 GB   RSS: 11.02 GB
note: After generateProbeBasedProfile
note: **VM: 30.56 GB   RSS: 29.08 GB**
note: After postProcessProfiles
note: VM: 31.91 GB   RSS: 30.43 GB
  • After:
note: After computeSizeForProfiledFunctions
note: VM: 12.26 GB   RSS: 10.80 GB
note: After generateProbeBasedProfile
note: **VM: 28.76 GB   RSS: 27.29 GB**
note: After postProcessProfiles
note: VM: 30.12 GB   RSS: 28.66 GB
  1. Large benchmark:
    • Before:
note: After computeSizeForProfiledFunctions
note: VM: 106.28 GB   RSS: 91.25 GB
note: After generateProbeBasedProfile
note: **VM: 245.19 GB   RSS: 224.30 GB**
note: After postProcessProfiles
note: VM: 254.96 GB   RSS: 221.58 GB
  • After:
note: After computeSizeForProfiledFunctions
note: VM: 105.95 GB   RSS: 91.04 GB
note: After generateProbeBasedProfile
note: VM: **234.29 GB   RSS: 215.39 GB**
note: After postProcessProfiles
note: VM: 244.03 GB   RSS: 213.45 GB

Diff Detail

Event Timeline

hoy created this revision.Jan 25 2022, 4:13 PM
Herald added subscribers: modimo, wenlei. · View Herald TranscriptJan 25 2022, 4:13 PM
hoy requested review of this revision.Jan 25 2022, 4:13 PM
Herald added a project: Restricted Project. · View Herald TranscriptJan 25 2022, 4:13 PM
Herald added a subscriber: llvm-commits. · View Herald Transcript
hoy edited the summary of this revision. (Show Details)Jan 25 2022, 4:37 PM
hoy added reviewers: wenlei, wlei.
hoy edited the summary of this revision. (Show Details)Jan 25 2022, 4:39 PM
wenlei added a comment.Jan 25 2022, 4:58 PM

Does this lengthen process time?

hoy added a comment.Jan 25 2022, 5:06 PM
In D118203#3271143, @wenlei wrote:

Does this lengthen process time?

Not necessarily if reallocaton happens frequently.

Before:

note: After computeSizeForProfiledFunctions
note: Tue Jan 25 10:38:10 2022
note: VM: 106.28 GB   RSS: 91.25 GB
note: After generateProbeBasedProfile
note: Tue Jan 25 11:06:02 2022
note: VM: 245.19 GB   RSS: 224.30 GB

After:

note: After computeSizeForProfiledFunctions
note: Tue Jan 25 15:31:13 2022
note: VM: 105.95 GB   RSS: 91.04 GB
note: After generateProbeBasedProfile
note: Tue Jan 25 15:58:03 2022
note: VM: 234.29 GB   RSS: 215.39 GB
wenlei accepted this revision.Jan 25 2022, 5:09 PM
In D118203#3271163, @hoy wrote:
In D118203#3271143, @wenlei wrote:

Does this lengthen process time?

Not necessarily if reallocaton happens frequently.

Before:

note: After computeSizeForProfiledFunctions
note: Tue Jan 25 10:38:10 2022
note: VM: 106.28 GB   RSS: 91.25 GB
note: After generateProbeBasedProfile
note: Tue Jan 25 11:06:02 2022
note: VM: 245.19 GB   RSS: 224.30 GB

After:

note: After computeSizeForProfiledFunctions
note: Tue Jan 25 15:31:13 2022
note: VM: 105.95 GB   RSS: 91.04 GB
note: After generateProbeBasedProfile
note: Tue Jan 25 15:58:03 2022
note: VM: 234.29 GB   RSS: 215.39 GB

Thanks for checking. Nice improvements!

This revision is now accepted and ready to land.Jan 25 2022, 5:09 PM
This revision was landed with ongoing or failed builds.Jan 25 2022, 9:37 PM
Closed by commit rGcc4beda039ba: [CSSPGO] Adjust SampleContextFrameVector to be a shorter SmallVector (authored by hoy). · Explain Why
This revision was automatically updated to reflect the committed changes.
hoy added a commit: rGcc4beda039ba: [CSSPGO] Adjust SampleContextFrameVector to be a shorter SmallVector.
Harbormaster completed remote builds in B145626: Diff 403067.Jan 26 2022, 10:22 PM

Revision Contents

PathSize
llvm/
include/
llvm/
ProfileData/
2 lines

Diff 403132

llvm/include/llvm/ProfileData/SampleProf.h

Show First 20 Lines • Show All 444 Lines • ▼ Show 20 Linesstruct SampleContextFrame {
} }
}; };
static inline hash_code hash_value(const SampleContextFrame &arg) { static inline hash_code hash_value(const SampleContextFrame &arg) {
return hash_combine(arg.FuncName, arg.Location.LineOffset, return hash_combine(arg.FuncName, arg.Location.LineOffset,
arg.Location.Discriminator); arg.Location.Discriminator);
} }
using SampleContextFrameVector = SmallVector<SampleContextFrame, 10>; using SampleContextFrameVector = SmallVector<SampleContextFrame, 1>;
using SampleContextFrames = ArrayRef<SampleContextFrame>; using SampleContextFrames = ArrayRef<SampleContextFrame>;
struct SampleContextFrameHash { struct SampleContextFrameHash {
uint64_t operator()(const SampleContextFrameVector &S) const { uint64_t operator()(const SampleContextFrameVector &S) const {
return hash_combine_range(S.begin(), S.end()); return hash_combine_range(S.begin(), S.end());
} }
}; };
▲ Show 20 Lines • Show All 818 Lines • Show Last 20 Lines