This is an archive of the discontinued LLVM Phabricator instance.

[find-all-symbols] Parallelize the merge step.
ClosedPublic

Authored by bkramer on Apr 29 2016, 5:05 AM.

Details

Summary

There is still more parallelism to get here because we synchonize on the
actual uniquing but just doing YAML parsing in parallel already gives a
significant speedup.

Merging all symbols in LLVM+clang+compiler-rt+lld+libc++, 48 cores.
before: 201.55s user 1.47s system 99% cpu 3:23.04 total
after: 279.13s user 7.53s system 929% cpu 30.838 total

Diff Detail

Repository
rL LLVM

Event Timeline

bkramer updated this revision to Diff 55575.Apr 29 2016, 5:05 AM
bkramer retitled this revision from to [find-all-symbols] Parallelize the merge step..
bkramer updated this object.
bkramer added reviewers: hokein, djasper.
bkramer added a subscriber: cfe-commits.
bkramer updated this revision to Diff 55577.Apr 29 2016, 5:09 AM

Drain the pool, fix typo.

hokein added inline comments.Apr 29 2016, 5:14 AM
include-fixer/find-all-symbols/tool/FindAllSymbolsMain.cpp
107 ↗(On Diff #55575)

Don't we need to all 'Pool.wait()' to wait for all working threads complete?

hokein accepted this revision.Apr 29 2016, 5:15 AM
hokein edited edge metadata.

Just see you updated code. LGTM.

This revision is now accepted and ready to land.Apr 29 2016, 5:15 AM
This revision was automatically updated to reflect the committed changes.