Page MenuHomePhabricator

Load balancing for LTO
Needs ReviewPublic

Authored by david2050 on Apr 9 2019, 7:32 PM.

Details

Summary

Use bitcode size as an estimated for compilation time and sort
compilation tasks in order decreasing sizee to provide better
load balance when there are large number of taks.

Diff Detail

Event Timeline

david2050 created this revision.Apr 9 2019, 7:32 PM
Herald added a project: Restricted Project. · View Herald TranscriptApr 9 2019, 7:32 PM
fhahn added a subscriber: fhahn.Apr 10 2019, 5:04 AM
mehdi_amini added inline comments.Apr 11 2019, 5:48 PM
lib/LTO/LTO.cpp
1301

The ThinLink has access to much fine grain data: like number of function and number of instructions. Have you looked into using this?

1312

FYI: https://github.com/llvm/llvm-project/blob/master/llvm/lib/LTO/ThinLTOCodeGenerator.cpp#L923

I used a separate index vector in ThinLTOCodeGenerator to do the same, it may make the code a bit easier to track (less pair<> and .first->second kind of thing).

david2050 planned changes to this revision.Apr 11 2019, 6:54 PM
david2050 marked 2 inline comments as done.
david2050 added inline comments.
lib/LTO/LTO.cpp
1301

I did not explore alternatives. This was effective for the workloads that motivated it so I thought I would share.

1312

Your implementation is cleaner. I may not have time to retest.

david2050 updated this revision to Diff 194799.Apr 11 2019, 7:41 PM

copy & paste & modify :-) from Mehdi pointerwd

ormris added a subscriber: ormris.Apr 12 2019, 9:36 AM