This is an archive of the discontinued LLVM Phabricator instance.

[ThinLTO] Efficiency improvement when writing module path string table
ClosedPublic

Authored by tejohnson on Jun 1 2017, 5:24 PM.

Details

Summary

When writing the combined index, we are walking the entire module
path StringMap in the full index, and checking whether each one should be
included in the index being written. For distributed backends, where we
write an individual combined index for each file, each with only a few
module paths, this is incredibly inefficient. Add a method that takes
a callback and hides the details of whether we are writing the full
combined index, or just a slice, and in the latter case it walks the set
of modules to include instead of the entire index.

For a huge application with around 23K files (i.e. where we were iterating
through the 23K-entry modulePath StringMap 23K times), this change improved
the thin link time by a whopping 48%.

Event Timeline

tejohnson created this revision.Jun 1 2017, 5:24 PM
pcc accepted this revision.Jun 1 2017, 6:00 PM

LGTM

This revision is now accepted and ready to land.Jun 1 2017, 6:00 PM
This revision was automatically updated to reflect the committed changes.