This is an archive of the discontinued LLVM Phabricator instance.

[HotColdSplit] Move splitting earlier in the pipeline
ClosedPublic

Authored by vsk on Jan 22 2019, 8:12 PM.

Details

Summary

Performing splitting early has several advantages:

  • Inhibiting inlining of cold code early improves code size. Compared to scheduling splitting at the end of the pipeline, this cuts code size growth in half within the iOS shared cache (0.69% to 0.34%).
  • Inhibiting inlining of cold code improves compile time. There's no need to inline split cold functions, or to inline as much *within* those split functions as they are marked minsize.
  • During LTO, extra work is only done in the pre-link step. Less code must be inlined during cross-module inlining.
  • The most common cold regions identified by the static/conservative splitting heuristic can (a) be found before inlining and (b) do not grow after inlining. E.g. __assert_fail, os_log_error.

The disadvantages are:

  • Some opportunities for splitting out cold code may be missed. This gap can potentially be narrowed by adding a worklist algorithm to the splitting pass.
  • Some opportunities to reduce code size may be lost (e.g. store sinking, when one side of the CFG diamond is split). This does not outweigh the code size benefits of splitting earlier.

On net, splitting early in the pipeline has substantial code size
benefits, and no major effects on memory locality or performance. We
measured memory locality using ktrace data, and consistently found that
10% fewer pages were needed to capture 95% of text page faults in key
iOS benchmarks. We measured performance on frequency-stabilized iOS
devices using LNT+externals.

This reverses course on the decision made to schedule splitting late in
r344869 (D53437).

Diff Detail

Repository
rL LLVM

Event Timeline

vsk created this revision.Jan 22 2019, 8:12 PM
tejohnson added inline comments.Jan 23 2019, 7:03 AM
llvm/lib/Passes/PassBuilder.cpp
661 ↗(On Diff #183030)

Probably should have similar comment about why here (like you added in old PM).

llvm/lib/Transforms/IPO/PassManagerBuilder.cpp
521 ↗(On Diff #183030)

I think you have the wrong LTO guards. This is the opposite of the ThinLTO guard you have in the new PM. Here you are preventing this from running during the ThinLTO (and regular LTO) pre-link steps. The following are equivalent:

PrepareForThinLTO (oldPM) == ThinLTOPhase::PreLink (newPM)
PerformThinLTO (oldPM) == ThinLTOPhase::PostLink (newPM)

Note that there is no PerformLTO, since in the old PM this function is not even called during the regular LTO post link (which has a different, specialized pipeline). So in a nutshell, I believe this should be:

if (EnableHotColdSplit && !PerformThinLTO)

This will give you splitting during a non-LTO compile, and during the pre-link *LTO compiles.

vsk updated this revision to Diff 183143.Jan 23 2019, 11:46 AM
vsk marked 3 inline comments as done.
  • Fix the pipeline issue in the old PM pointed out by Teresa. Add a test for it.
llvm/lib/Transforms/IPO/PassManagerBuilder.cpp
521 ↗(On Diff #183030)

Thanks for explaining this.

tejohnson accepted this revision.Jan 23 2019, 11:54 AM

LGTM with suggestion about cutting down test.

llvm/test/Other/opt-hot-cold-split.ll
30 ↗(On Diff #183143)

Suggest cutting the expected output here and in the pre-link cases down to just look for Hot Cold Splitting. Otherwise this test will need to be changed for unrelated opt pipeline changes. And I don't think we need to check the whole pipeline?

This revision is now accepted and ready to land.Jan 23 2019, 11:54 AM
vsk updated this revision to Diff 183159.Jan 23 2019, 12:47 PM
vsk marked an inline comment as done.
vsk added inline comments.
llvm/test/Other/opt-hot-cold-split.ll
30 ↗(On Diff #183143)

I'll trim this down, and just check the higher-level invariants described earlier (i.e., after mem2reg, before function simplification passes), if that's all right.

tejohnson added inline comments.Jan 23 2019, 12:53 PM
llvm/test/Other/opt-hot-cold-split.ll
30 ↗(On Diff #183143)

Sure.

This revision was automatically updated to reflect the committed changes.
thakis added a subscriber: thakis.Jan 24 2019, 2:56 PM

Test fails on Windows: https://logs.chromium.org/logs/chromium/bb/tryserver.chromium.win/win_upload_clang/467/+/recipes/steps/package_clang/0/stdout

FAIL: LLVM :: Other/opt-hot-cold-split.ll (36787 of 46358)
******************** TEST 'LLVM :: Other/opt-hot-cold-split.ll' FAILED ********************
Script:
--
: 'RUN: at line 1';   C:\b\rr\tmpgtkggu\w\src\third_party\llvm-bootstrap\bin\opt.EXE -mtriple=x86_64-- -Os -hot-cold-split=true -debug-pass=Structure < C:\b\rr\tmpgtkggu\w\src\third_party\llvm\test\Other\opt-hot-cold-split.ll -o /dev/null 2>&1 | C:\b\rr\tmpgtkggu\w\src\third_party\llvm-bootstrap\bin\FileCheck.EXE C:\b\rr\tmpgtkggu\w\src\third_party\llvm\test\Other\opt-hot-cold-split.ll -check-prefix=DEFAULT-Os
: 'RUN: at line 2';   C:\b\rr\tmpgtkggu\w\src\third_party\llvm-bootstrap\bin\opt.EXE -mtriple=x86_64-- -Os -hot-cold-split=true -passes='lto-pre-link<Os>' -debug-pass-manager < C:\b\rr\tmpgtkggu\w\src\third_party\llvm\test\Other\opt-hot-cold-split.ll -o /dev/null 2>&1 | C:\b\rr\tmpgtkggu\w\src\third_party\llvm-bootstrap\bin\FileCheck.EXE C:\b\rr\tmpgtkggu\w\src\third_party\llvm\test\Other\opt-hot-cold-split.ll -check-prefix=LTO-PRELINK-Os
: 'RUN: at line 3';   C:\b\rr\tmpgtkggu\w\src\third_party\llvm-bootstrap\bin\opt.EXE -mtriple=x86_64-- -Os -hot-cold-split=true -passes='thinlto-pre-link<Os>' -debug-pass-manager < C:\b\rr\tmpgtkggu\w\src\third_party\llvm\test\Other\opt-hot-cold-split.ll -o /dev/null 2>&1 | C:\b\rr\tmpgtkggu\w\src\third_party\llvm-bootstrap\bin\FileCheck.EXE C:\b\rr\tmpgtkggu\w\src\third_party\llvm\test\Other\opt-hot-cold-split.ll -check-prefix=THINLTO-PRELINK-Os
: 'RUN: at line 4';   C:\b\rr\tmpgtkggu\w\src\third_party\llvm-bootstrap\bin\opt.EXE -mtriple=x86_64-- -Os -hot-cold-split=true -passes='thinlto<Os>' -debug-pass-manager < C:\b\rr\tmpgtkggu\w\src\third_party\llvm\test\Other\opt-hot-cold-split.ll -o /dev/null 2>&1 | C:\b\rr\tmpgtkggu\w\src\third_party\llvm-bootstrap\bin\FileCheck.EXE C:\b\rr\tmpgtkggu\w\src\third_party\llvm\test\Other\opt-hot-cold-split.ll -check-prefix=THINLTO-POSTLINK-Os
--
Exit Code: 1

Command Output (stdout):
--
$ ":" "RUN: at line 1"
$ "C:\b\rr\tmpgtkggu\w\src\third_party\llvm-bootstrap\bin\opt.EXE" "-mtriple=x86_64--" "-Os" "-hot-cold-split=true" "-debug-pass=Structure" "-o" "/dev/null"
$ "C:\b\rr\tmpgtkggu\w\src\third_party\llvm-bootstrap\bin\FileCheck.EXE" "C:\b\rr\tmpgtkggu\w\src\third_party\llvm\test\Other\opt-hot-cold-split.ll" "-check-prefix=DEFAULT-Os"
$ ":" "RUN: at line 2"
$ "C:\b\rr\tmpgtkggu\w\src\third_party\llvm-bootstrap\bin\opt.EXE" "-mtriple=x86_64--" "-Os" "-hot-cold-split=true" "-passes=lto-pre-link<Os>" "-debug-pass-manager" "-o" "/dev/null"
$ "C:\b\rr\tmpgtkggu\w\src\third_party\llvm-bootstrap\bin\FileCheck.EXE" "C:\b\rr\tmpgtkggu\w\src\third_party\llvm\test\Other\opt-hot-cold-split.ll" "-check-prefix=LTO-PRELINK-Os"
# command stderr:
C:\b\rr\tmpgtkggu\w\src\third_party\llvm\test\Other\opt-hot-cold-split.ll:15:19: error: LTO-PRELINK-Os: expected string not found in input
; LTO-PRELINK-Os: Running pass: ModuleToFunctionPassAdaptor<llvm::PromotePass>
                  ^
<stdin>:3:1: note: scanning from here
Running pass: VerifierPass on <stdin>
^
<stdin>:17:1: note: possible intended match here
Running pass: ModuleToFunctionPassAdaptor<class llvm::PromotePass> on <stdin>

(note additional "class ")

vsk added a comment.Jan 24 2019, 4:07 PM

Sorry about that, taking a look now

vsk added a comment.Jan 24 2019, 4:20 PM

I relaxed the test in r352138, and think that should address the issue.