This is an archive of the discontinued LLVM Phabricator instance.

Single function compilation mode.
Changes PlannedPublic

Authored by hoy on Dec 7 2020, 6:35 PM.

Details

Summary

As I worked on analyzing hot spots for large applications I found that iterating on a particular function was painful primarily because recompilation took a long time. For example, compiling SPEC2017/blender in LTO mode takes 30 minutes. Also a debug-only dump is huge since that has to be done for all functions. In this change we are introducing a single method compilation mode where only a function and optionally all callee functions reachable from the specified function will be compiled. This aims at compiling the minimal part of the program to replay the compilation of the user-specified function.

It might be never possible to replay a method compilation accurately without compiling the whole program. For example, a top-down interprocedural pass may propagate a value down from the root entry to the target function. A callee function on the path from entry to the target may return a value that contributes to the target compilation. That said, this change can still help if the compilation it triggers generates the same code with the original full compilation.

To minimized callee function exploding, a profile is used to narrow down the potential indirect call targets. Without a profile, alias analysis or type analysis can be used instead. This is not done in this change.

The mode works with non-LTO, thinLTO and fullLTO modes. More specifically, the following switches are introduced. All switches can be specified multiple times in one invocation with input aggregated.

  1. A clang switch -fsingle-func=Name for non-LTO.
  2. A lld switch --lto-single-func=Name for LTO.
  3. An opt switch -single-func=Name.

Diff Detail

Event Timeline

hoy created this revision.Dec 7 2020, 6:35 PM
hoy requested review of this revision.Dec 7 2020, 6:35 PM
Herald added a project: Restricted Project. · View Herald TranscriptDec 7 2020, 6:35 PM
hoy edited the summary of this revision. (Show Details)Dec 7 2020, 6:38 PM
hoy added a reviewer: wenlei.
zino added inline comments.
llvm/test/Other/extract-func.ll
2 ↗(On Diff #310070)

Can you add a test to test multiple functions? and testing fullLTO for example

lxfind added a subscriber: lxfind.Dec 8 2020, 1:50 PM
lxfind added inline comments.
llvm/lib/Transforms/IPO/ExtractGV.cpp
25

We might need a better name to reflect on the fact that there can be multiple functions.
Something like "only-compile-funcs"?

hoy updated this revision to Diff 310380.Dec 8 2020, 4:05 PM
hoy marked an inline comment as done.

Addressing feedbacks.

hoy added inline comments.Dec 8 2020, 4:06 PM
llvm/lib/Transforms/IPO/ExtractGV.cpp
25

How about -compile-funcs=?

hoy updated this revision to Diff 310606.Dec 9 2020, 12:13 PM

Updating D92806: Single function compilation mode.

hoy added a subscriber: wmi.Dec 9 2020, 12:14 PM

@wmi Can you please recommend a reviewer for this? Thanks!

wmi added a comment.Dec 9 2020, 4:13 PM
In D92806#2443435, @hoy wrote:

@wmi Can you please recommend a reviewer for this? Thanks!

Maybe Chandler (chandlerc@google.com) or Hal (hfinkel@anl.gov)?

hoy updated this revision to Diff 313221.Dec 21 2020, 5:06 PM

Adding LTO, linker and clang supports.

hoy edited the summary of this revision. (Show Details)Dec 21 2020, 5:11 PM
hoy added a reviewer: tejohnson.
hoy edited the summary of this revision. (Show Details)Jan 7 2021, 6:30 PM
hoy added a reviewer: aeubanks.

WDYT about adding optnone to all functions except the ones you're interested in? Would that cut down the compile times?
We could reuse ForceFunctionAttrsPass rather than adding so much plumbing. This is too much plumbing for my liking.

And since this is a debug thing, instead of creating new Clang flags, we can just pass flags like -Xclang -mllvm?

hoy planned changes to this revision.Sep 28 2021, 2:48 PM