Correctly parse LateParsedTemplates in case of multiple dependent modules

[gargvaibhav64]

Summary

While parsing LateParsedTemplates, Clang assumes that the Global DeclID matches with the Local DeclID of a Decl. This is not the case when we have multiple dependent modules , each having their own LateParsedTemplate section. In such a case, a Local/Global DeclID confusion occurs which leads to improper casting of FunctionDecl's.

This commit creates a Vector to map the LateParsedTemplate section of each Module with their module file and therefore resolving the Global/Local DeclID confusion.

[gargvaibhav64]

Resolve a typo.

[rsmith]

clang/include/clang/Serialization/ASTReader.h

902–906

Given the changes below, we shouldn't need this any more, and can instead use this name for the container that carries the ModuleFile*.

908–909

We don't need a map here; we never need to look up the vector from the module file.

clang/lib/Serialization/ASTReader.cpp

3725–3728

We can directly initialize the vector in the container rather than initializing LateParsedTemplates and moving it. I think this also removes the only remaining use of the old LateParsedTemplates.

8394–8400

8396

No need to copy the vector here, and we can avoid hardcoding the preallocated SmallVector size.

[gargvaibhav64]

Thanks. I incorporated all the changes.

[rsmith]

Thanks!

clang/lib/Serialization/ASTReader.cpp

8398

Phabricator claims there's a tab here and below. Please double-check you're indenting with spaces before commit.

clang/include/clang/Serialization/ASTReader.h

private:
   SmallVector<uint64_t, 4> KnownNamespaces;
 
   /// A list of undefined decls with internal linkage followed by the
   /// SourceLocation of a matching ODR-use.
   SmallVector<uint64_t, 8> UndefinedButUsed;
 
   /// Delete expressions to analyze at the end of translation unit.
   SmallVector<uint64_t, 8> DelayedDeleteExprs;
 
-  // A list of late parsed template function data.
-  SmallVector<uint64_t, 1> LateParsedTemplates;
+  // A list of late parsed template function data with their module files.
+  SmallVector<std::pair<ModuleFile *, SmallVector<uint64_t, 1>>, 4>
+      LateParsedTemplates;
 
   /// The IDs of all decls to be checked for deferred diags.
   ///
   /// Sema tracks these to emit deferred diags.
   SmallVector<uint64_t, 4> DeclsToCheckForDeferredDiags;
 
 
 public:
   struct ImportedSubmodule {
     serialization::SubmoduleID ID;
     SourceLocation ImportLoc;
 

clang/lib/Serialization/ASTReader.cpp

case MACRO_OFFSET: {
                          F.BaseMacroID - LocalBaseMacroID));
 
         MacrosLoaded.resize(MacrosLoaded.size() + F.LocalNumMacros);
       }
       break;
     }
 
     case LATE_PARSED_TEMPLATE:
-      LateParsedTemplates.append(Record.begin(), Record.end());
+      LateParsedTemplates.emplace_back(
+          std::piecewise_construct, std::forward_as_tuple(&F),
+          std::forward_as_tuple(Record.begin(), Record.end()));
       break;
 
     case OPTIMIZE_PRAGMA_OPTIONS:
       if (Record.size() != 1) {
         Error("invalid pragma optimize record");
         return Failure;
       }
       OptimizeOffPragmaLocation = ReadSourceLocation(F, Record[0]);
       break;
for (unsigned Idx = 0, N = PendingInstantiations.size(); Idx < N;) {
     Pending.push_back(std::make_pair(D, Loc));
   }
   PendingInstantiations.clear();
 }
 
 void ASTReader::ReadLateParsedTemplates(
     llvm::MapVector<const FunctionDecl *, std::unique_ptr<LateParsedTemplate>>
         &LPTMap) {
-  for (unsigned Idx = 0, N = LateParsedTemplates.size(); Idx < N;
+  for (auto &LPT : LateParsedTemplates) {
+    ModuleFile *FMod = LPT.first;
+    RecordDataImpl &LateParsed = LPT.second;
+    for (unsigned Idx = 0, N = LateParsed.size(); Idx < N;
          /* In loop */) {
-    FunctionDecl *FD = cast<FunctionDecl>(GetDecl(LateParsedTemplates[Idx++]));
+      FunctionDecl *FD =
+          cast<FunctionDecl>(GetLocalDecl(*FMod, LateParsed[Idx++]));
 
       auto LT = std::make_unique<LateParsedTemplate>();
-    LT->D = GetDecl(LateParsedTemplates[Idx++]);
+      LT->D = GetLocalDecl(*FMod, LateParsed[Idx++]);
 
       ModuleFile *F = getOwningModuleFile(LT->D);
       assert(F && "No module");
 
-    unsigned TokN = LateParsedTemplates[Idx++];
+      unsigned TokN = LateParsed[Idx++];
       LT->Toks.reserve(TokN);
       for (unsigned T = 0; T < TokN; ++T)
-      LT->Toks.push_back(ReadToken(*F, LateParsedTemplates, Idx));
+        LT->Toks.push_back(ReadToken(*F, LateParsed, Idx));
 
       LPTMap.insert(std::make_pair(FD, std::move(LT)));
     }
-
-  LateParsedTemplates.clear();
+  }
 }
 
 void ASTReader::LoadSelector(Selector Sel) {
   // It would be complicated to avoid reading the methods anyway. So don't.
   ReadMethodPool(Sel);
 }
 
 void ASTReader::SetIdentifierInfo(IdentifierID ID, IdentifierInfo *II) {