[ThinLTOCodeGenerator] Avoid Rehash StringMap in ThreadPool

[steven_wu]

Summary

During threaded thinLTO, it is possible that the entry for current
module doesn't exist in StringMaps (like ExportLists, ResolvedODR,
etc.). Using operator[] might trigger a rehash for the StringMap, which
might happen on multiple threads at the same time.

rdar://problem/43846199

[steven_wu]

I can't find a super clean solution for this. Here is one possible solution, which is to iterate and insert the entry before entering thread pool

Alternatives are:

• Using lookup, which might copy construct std::map
• Using find, together with some default constructed value.
• Pre-allocate a larger StringMap, and hope rehash doesn't happen.

Let me know which one is preferred or there are better ones.

[steven_wu]

Fix up my previous patch.

It seems we have already done similar stuff but ModuleToDefinedGVSummaries
doesn't have all the modules so it cannot avoid rehashing.

[steven_wu]

Also a bit clarification why ModuleToDefinedGVSummaries can have missing entries, because there can be an empty module, or a module has no GV in it. And this happens during thinLTO WebKit.
For example, in WebCore MacOS build, UnifiedSource26-mm.o has empty text and data section. Its object file only has debug info.

[steven_wu]

Update patch

[dang]

This comment was removed by dang.

[dang]

LGTM I think both find and this are not the nicest, but I can't come up with anything better.

[tejohnson]

LGTM with nit.

lib/LTO/ThinLTOCodeGenerator.cpp

954 ↗

(On Diff #165399)

update comment

[steven_wu]

Update comments

llvm/trunk/lib/LTO/ThinLTOCodeGenerator.cpp

void ThinLTOCodeGenerator::run() {
   // impacts the caching.
   resolveWeakForLinkerInIndex(*Index, ResolvedODR);
 
   // Use global summary-based analysis to identify symbols that can be
   // internalized (because they aren't exported or preserved as per callback).
   // Changes are made in the index, consumed in the ThinLTO backends.
   internalizeAndPromoteInIndex(ExportLists, GUIDPreservedSymbols, *Index);
 
-  // Make sure that every module has an entry in the ExportLists and
-  // ResolvedODR maps to enable threaded access to these maps below.
-  for (auto &DefinedGVSummaries : ModuleToDefinedGVSummaries) {
-    ExportLists[DefinedGVSummaries.first()];
-    ResolvedODR[DefinedGVSummaries.first()];
+  // Make sure that every module has an entry in the ExportLists, ImportList,
+  // GVSummary and ResolvedODR maps to enable threaded access to these maps
+  // below.
+  for (auto &Module : Modules) {
+    auto ModuleIdentifier = Module.getBufferIdentifier();
+    ExportLists[ModuleIdentifier];
+    ImportLists[ModuleIdentifier];
+    ResolvedODR[ModuleIdentifier];
+    ModuleToDefinedGVSummaries[ModuleIdentifier];
   }
 
   // Compute the ordering we will process the inputs: the rough heuristic here
   // is to sort them per size so that the largest module get schedule as soon as
   // possible. This is purely a compile-time optimization.
   std::vector<int> ModulesOrdering;
   ModulesOrdering.resize(Modules.size());
   std::iota(ModulesOrdering.begin(), ModulesOrdering.end(), 0);