This is an archive of the discontinued LLVM Phabricator instance.

[lld] Add a new output section ".text.unknown" for funtions with unknown hotness info especially in sampleFDO
ClosedPublic

Authored by wmi on May 7 2020, 10:39 AM.

Details

Summary

For sampleFDO, because the optimized build uses profile generated from previous release, often we couldn't tell a function without profile was truely cold or just newly created so we had to treat them conservatively and put them in .text section instead of .text.unlikely. The result was when we persue the best performance by locking .text.hot and .text in memory, we wasted a lot of memory to keep cold functions inside. This problem has been largely solved for regular sampleFDO using profile-symbol-list (https://reviews.llvm.org/D66374), but for the case when we use partial profile, we still waste a lot of memory because of it.

In https://reviews.llvm.org/D62540, we propose to save functions with unknown hotness information in a special section called ".text.unknown", so that compiler will treat those functions as luck-warm, but runtime can choose not to mlock the special section in memory or use other strategy to save memory. That will solve most of the memory problem even if we use a partial profile.

The patch adds the support in lld for the special section.

Diff Detail

Repository
rL LLVM

Event Timeline

wmi created this revision.May 7 2020, 10:39 AM

Add a new section ".text.unknown"

Add a new *output* section

lld/ELF/Writer.cpp
131

a function

132

a new function

wmi retitled this revision from [lld] Add a new section ".text.unknown" for funtions with unknown hotness info especially in sampleFDO to [lld] Add a new output section ".text.unknown" for funtions with unknown hotness info especially in sampleFDO .May 7 2020, 12:25 PM
wmi marked 2 inline comments as done.

Thanks for looking at it.

lld/ELF/Writer.cpp
131

Fixed.

132

Fixed.

wmi updated this revision to Diff 262732.May 7 2020, 12:38 PM

Address MaskRay's comment.

tmsriram added inline comments.May 7 2020, 12:39 PM
lld/ELF/Writer.cpp
136

The new prefix looks good to me.

Just curious about input sections that are named just ".text.hot" or ".text.unknown" without the traling period. This could happen with -fno-unique-section-names I think? In that case, this handling seems insufficient. This is related to -keep-text-section-prefix in general. I can take a look at this.

wmi marked an inline comment as done.May 7 2020, 1:22 PM
wmi added inline comments.
lld/ELF/Writer.cpp
136

isSectionPrefix seems considering the section name without trailing period (prefix.drop_back())?

static bool isSectionPrefix(StringRef prefix, StringRef name) {
  return name.startswith(prefix) || name == prefix.drop_back();
}
tmsriram added inline comments.May 7 2020, 1:27 PM
lld/ELF/Writer.cpp
136

Ah, thanks!

tmsriram accepted this revision.May 7 2020, 1:31 PM

LGTM. You still need an approval from a LLD owner.

This revision is now accepted and ready to land.May 7 2020, 1:31 PM
MaskRay accepted this revision.May 7 2020, 2:24 PM

Looks great!

lld/ELF/Writer.cpp
136

I know what you are discussing... (And I mentioned this problem internally previously)

To prevent a linker change (the GNU ld and gold approach has an small annoying behavior), I created https://reviews.llvm.org/D79600

wmi marked an inline comment as done.May 7 2020, 2:59 PM
wmi added inline comments.
lld/ELF/Writer.cpp
136

I am not familiar with linker so I don't know the behavior of GNU ld and gold, but lld seems recognize ".text.hot" and return ".text.hot." for it, so I don't fully understand the intention of D79600, could you explain it a little bit? Thanks.

MaskRay added inline comments.May 7 2020, 3:31 PM
lld/ELF/Writer.cpp
136

I updated the description of D79600 a bit. Hope it is clear now... I can clarify if you still think that unclear.

In lld, I hope we only recognize .text.exit.* but not .text.exit, so that the function exit in a function section will not be ordered strangely.

To achieve that goal, we can rename .text.exit (without function sections, or with -fno-unique-section-names) to .text.exit.

wmi marked an inline comment as done.May 7 2020, 3:59 PM
wmi added inline comments.
lld/ELF/Writer.cpp
136

Thanks, now I see what you and Sri mean. Although ".text.hot" is mapped to ".text.hot." when outputing the section, it is not recognized during section ordering or other phases in linker. The function isSectionPrefix is only called by getOutputSectionName so only section outputing phase recognize ".text.hot"

Then D79600 makes sense to me. Thanks for clarifying!

tmsriram added inline comments.May 7 2020, 4:34 PM
lld/ELF/Writer.cpp
136

Isn't handling it in the linker the right solution?

Prefixes like ".text.unikely", ".text.hot", etc. are produced by all compilers. That also allows interoperability of various compilers and linkers. LLD can be fixed to handle the prefixes without the trailing period right? I dont have a strong opinion, just my 2 cents.

MaskRay added inline comments.May 7 2020, 4:46 PM
lld/ELF/Writer.cpp
136

By all compilers, do you mean GCC? I think we can patch GCC as well (if we still care about the use case) to avoid the aforementioned ambiguity.
I prefer the current way LLD handles things to avoid ambiguity.

MaskRay added inline comments.May 7 2020, 6:16 PM
lld/ELF/Writer.cpp
136

Actually, GCC doesn't support -fno-unique-section-names. This requires a GNU as feature which is fairly new addition http://sourceware.org/PR25380 (I raised these feature requests but somehow I forgot this point when making the previous comment...)

wmi added a comment.May 8 2020, 9:39 AM

D79600 is a followup and the discussion about it won't affect this change, so I will go ahead and submit the change.

This revision was automatically updated to reflect the committed changes.
Herald added a project: Restricted Project. · View Herald TranscriptMay 8 2020, 11:15 AM