This is an archive of the discontinued LLVM Phabricator instance.

[ThinLTOCodeGenerator] Avoid Rehash StringMap in ThreadPool
ClosedPublic

Authored by steven_wu on Sep 13 2018, 11:06 AM.

Details

Summary

During threaded thinLTO, it is possible that the entry for current
module doesn't exist in StringMaps (like ExportLists, ResolvedODR,
etc.). Using operator[] might trigger a rehash for the StringMap, which
might happen on multiple threads at the same time.

rdar://problem/43846199

Diff Detail

Repository
rL LLVM

Event Timeline

steven_wu created this revision.Sep 13 2018, 11:06 AM

I can't find a super clean solution for this. Here is one possible solution, which is to iterate and insert the entry before entering thread pool

Alternatives are:

  • Using lookup, which might copy construct std::map
  • Using find, together with some default constructed value.
  • Pre-allocate a larger StringMap, and hope rehash doesn't happen.

Let me know which one is preferred or there are better ones.

Fix up my previous patch.

It seems we have already done similar stuff but ModuleToDefinedGVSummaries
doesn't have all the modules so it cannot avoid rehashing.

Also a bit clarification why ModuleToDefinedGVSummaries can have missing entries, because there can be an empty module, or a module has no GV in it. And this happens during thinLTO WebKit.
For example, in WebCore MacOS build, UnifiedSource26-mm.o has empty text and data section. Its object file only has debug info.

steven_wu updated this revision to Diff 165399.Sep 13 2018, 4:23 PM

Update patch

dang added a subscriber: dang.Sep 13 2018, 4:28 PM
dang added a comment.Sep 13 2018, 4:52 PM
This comment was removed by dang.
dang added a comment.EditedSep 13 2018, 5:14 PM

LGTM I think both find and this are not the nicest, but I can't come up with anything better.

tejohnson accepted this revision.Sep 14 2018, 9:41 AM

LGTM with nit.

lib/LTO/ThinLTOCodeGenerator.cpp
954 ↗(On Diff #165399)

update comment

This revision is now accepted and ready to land.Sep 14 2018, 9:41 AM

Update comments

This revision was automatically updated to reflect the committed changes.
steven_wu marked an inline comment as done.