[lldb][WIP] Use forward decls with redeclared definitions instead of minimal import for records

[teemperor]

Summary

This patch is rewriting the way we move Clang types between different ASTS in Clang.

The motivation is that the current approach for 'completing types' in LLDB just doesn't work. We have all kinds of situations where we either have Clang crash due to for example having a Record without DefinitionData but with FieldDecls in it.

The reason for that is that at the moment we create record types ([CXX]RecordDecls and ObjCInterfaceDecls) and we always pretend that a type potentially has a definition that somehow could be lazily pulled in. However, Clang doesn't have any interface (at least none that is consistently called) that turns a RecordDecl without DefinitionData into a RecordDecl with DefinitionData. The only relevant API in the ExternalASTSource is CompleteType is suffering from the fact that it's essentially not used by generic Clang code.

The part of the ExternalASTSource API that is consistently called to pull in a potential definition is CompleteRedeclChain (which is for example automatically called when calling getDefinition on a RecordDecl). The problem with CompleteRedeclChain however is that it's not supposed to add definition data to the Decl its called on, but instead it should provide a redeclaration that is defining the record. That's a very different system than what we currently have and we can't just swap it out under the hood.

To make it short: We probably need to rewrite that part of LLDB.

So the solution here is essentially rewriting all our completion code to do this:

1. Instead of creating these weirdly defined records that have fields but maybe not definition and so on, we only create forward decls at the start.
2. We then bump the GenerationCounter of the ExternalASTSource of the current AST to force rebuilding of the redeclaration chain.
3. When Clang asks for the definition of the record, it will ask the ExternalASTSource to complete the redeclaration chain. At this point we build the definition and add it as a second decl.

The ASTImporter can now also stop using the minimal mode for records as there is no reason anymore. It just imports first the forward declaration and then the definition when needed.

[teemperor]

• Add a second DIEtoX map from DIEs to records (and their definitions once we have them).
• Avoid some calls that try to find the 'interesting' TagDecl redeclaration.

[martong]

This is a great initiative! Finally, the CTU analysis code and LLDB could share a simplified ASTImporter. In the past years we've seen that the minimal import mode caused many corner cases in the ASTImporter and complicated the code pretty much.

lldb/source/Plugins/ExpressionParser/Clang/ClangASTImporter.h

265

[teemperor]

Just to clarify because Jan abandoned the linked ASTImporter patch: I don't have a timeline for this. If I find the time to work on this then this might land quite soon or maybe it will only land in a few months. Or I'll run into something serious that just won't work with this approach and we have to abandon this.

[teemperor]

• All C tests are passing.
• Nearly all Obj-C tests are passing.
• about 70% of the C++ tests are passing now.

[teemperor]

Just moving this out of the review queue. Also as there are a bunch of Clang changes (that probably trigger a ton of Herald rules), I keep pushing new updates to https://github.com/teemperor/llvm-project/tree/Rewrite until I can split all those changes out into their own smaller patches.

[dblaikie]

Just want to say this sounds like a really great direction - putting lldb in a codepath that's much more robustly tested (by lazy loading modules during normal compilation) than the "let's create interesting incomplete types in ways Clang Sema never expects, and then try to patch up all the holes where Clang assumes the type must be complete". Really looking forward to this & hope it provides the sort of robustness improvements it sounds like it should.

[teemperor]

• Update from git

[martong]

@JDevlieghere @labath Do you know if someone took it over from @teemperor (since AFAIK he is no longer working on lldb) ? Are there any plans to continue this effort? I am interested as a clang::ASTImporter maintainer, we could have so many simplifications in clang::ASTImporter if this patch will have been landed sometime.

[JDevlieghere]

In D101950#3418751, @martong wrote:

@JDevlieghere @labath Do you know if someone took it over from @teemperor (since AFAIK he is no longer working on lldb) ? Are there any plans to continue this effort? I am interested as a clang::ASTImporter maintainer, we could have so many simplifications in clang::ASTImporter if this patch will have been landed sometime.

Yup. We've hired someone new to work on the expression evaluator in LLDB but that person hasn't started yet and will need some time to ramp up. I'll sync with @aprantl and see if this is something we can prioritize.

clang/include/clang/AST/ExternalASTSource.h

protected:
   SetExternalVisibleDeclsForName(const DeclContext *DC,
                                  DeclarationName Name,
                                  ArrayRef<NamedDecl*> Decls);
 
   static DeclContextLookupResult
   SetNoExternalVisibleDeclsForName(const DeclContext *DC,
                                    DeclarationName Name);
 
+public:
   /// Increment the current generation.
   uint32_t incrementGeneration(ASTContext &C);
 };
 
 /// A lazy pointer to an AST node (of base type T) that resides
 /// within an external AST source.
 ///
 /// The AST node is identified within the external AST source by a

clang/include/clang/AST/Redeclarable.h

protected:
   /// of this same Decl. If isFirst() is true, this is the first
   /// declaration and Link points to the latest declaration. For example:
   ///
   ///  #1 int f(int x, int y = 1); // <pointer to #3, true>
   ///  #2 int f(int x = 0, int y); // <pointer to #1, false>
   ///  #3 int f(int x, int y) { return x + y; } // <pointer to #2, false>
   ///
   /// If there is only one declaration, it is <pointer to self, true>
+public:
   DeclLink RedeclLink;
+protected:
 
   decl_type *First;
 
   decl_type *getNextRedeclaration() const {
     return RedeclLink.getPrevious(static_cast<const decl_type *>(this));
   }
 
 public:
   friend class ASTDeclReader;

lldb/source/Plugins/ExpressionParser/Clang/ClangASTImporter.h

 
 #include "clang/AST/ASTImporter.h"
 #include "clang/AST/CharUnits.h"
 #include "clang/AST/Decl.h"
 #include "clang/AST/DeclCXX.h"
 #include "clang/Basic/FileManager.h"
 #include "clang/Basic/FileSystemOptions.h"
 
 #include "lldb/Host/FileSystem.h"
 #include "lldb/Symbol/CompilerDeclContext.h"
 #include "lldb/Utility/LLDBAssert.h"
 #include "lldb/lldb-types.h"
 
 #include "Plugins/ExpressionParser/Clang/CxxModuleHandler.h"
 
 #include "llvm/ADT/DenseMap.h"
 
public:
   ///         Note: Does *not* return false if an original type was found but
   ///               no information was imported over.
   ///
   /// \see ClangASTImporter::Import
   bool Import(const CompilerType &type);
 
   bool CompleteType(const CompilerType &compiler_type);
 
-  bool CompleteTagDecl(clang::TagDecl *decl);
+  bool CompleteTagDecl(const clang::TagDecl *decl);
 
   bool CompleteTagDeclWithOrigin(clang::TagDecl *decl, clang::TagDecl *origin);
 
   bool CompleteObjCInterfaceDecl(clang::ObjCInterfaceDecl *interface_decl);
 
   bool CompleteAndFetchChildren(clang::QualType type);
 
   bool RequireCompleteType(clang::QualType type);
public:
   ///
   /// When intercepting a declaration import, the ASTImporterDelegate uses the
   /// CxxModuleHandler to replace any missing or malformed declarations with
   /// their counterpart from a C++ module.
   struct ASTImporterDelegate : public clang::ASTImporter {
     ASTImporterDelegate(ClangASTImporter &master, clang::ASTContext *target_ctx,
                         clang::ASTContext *source_ctx)
         : clang::ASTImporter(*target_ctx, master.m_file_manager, *source_ctx,
-                             master.m_file_manager, true /*minimal*/),
+                             master.m_file_manager, false /*minimal*/),
           m_master(master), m_source_ctx(source_ctx) {
       // Target and source ASTContext shouldn't be identical. Importing AST
       // nodes within the same AST doesn't make any sense as the whole idea
       // is to import them to a different AST.
       lldbassert(target_ctx != source_ctx && "Can't import into itself");
       // This is always doing a minimal import of any declarations. This means
       // that there has to be an ExternalASTSource in the target ASTContext
       // (that should implement the callbacks that complete any declarations

lldb/source/Plugins/ExpressionParser/Clang/ClangASTImporter.cpp

 //===-- ClangASTImporter.cpp ----------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "lldb/Core/Module.h"
 #include "lldb/Utility/LLDBAssert.h"
 #include "lldb/Utility/Log.h"
 #include "clang/AST/Decl.h"
 #include "clang/AST/DeclCXX.h"
 #include "clang/AST/DeclObjC.h"
 #include "clang/Sema/Lookup.h"
 #include "clang/Sema/Sema.h"
 #include "llvm/Support/raw_ostream.h"
while (!m_decls_to_complete.empty()) {
       TypeSystemClang::GetCompleteDecl(m_src_ctx, original_decl);
       if (auto *tag_decl = dyn_cast<TagDecl>(decl)) {
         if (auto *original_tag_decl = dyn_cast<TagDecl>(original_decl)) {
           if (original_tag_decl->isCompleteDefinition()) {
             m_delegate->ImportDefinitionTo(tag_decl, original_tag_decl);
             tag_decl->setCompleteDefinition(true);
           }
         }
-
-        tag_decl->setHasExternalLexicalStorage(false);
-        tag_decl->setHasExternalVisibleStorage(false);
-      } else if (auto *container_decl = dyn_cast<ObjCContainerDecl>(decl)) {
-        container_decl->setHasExternalLexicalStorage(false);
-        container_decl->setHasExternalVisibleStorage(false);
       }
 
       to_context_md->removeOrigin(decl);
     }
 
     // Stop listening to imported decls. We do this after clearing the
     // Decls we needed to import to catch all Decls they might have pulled in.
     m_delegate->RemoveImportListener();
bool ClangASTImporter::Import(const CompilerType &type) {
   }
   return false;
 }
 
 bool ClangASTImporter::CompleteType(const CompilerType &compiler_type) {
   if (!CanImport(compiler_type))
     return false;
 
-  if (Import(compiler_type)) {
-    TypeSystemClang::CompleteTagDeclarationDefinition(compiler_type);
-    return true;
-  }
-
-  TypeSystemClang::SetHasExternalStorage(compiler_type.GetOpaqueQualType(),
-                                         false);
-  return false;
+  return Import(compiler_type);
 }
 
 bool ClangASTImporter::LayoutRecordType(
     const clang::RecordDecl *record_decl, uint64_t &bit_size,
     uint64_t &alignment,
     llvm::DenseMap<const clang::FieldDecl *, uint64_t> &field_offsets,
     llvm::DenseMap<const clang::CXXRecordDecl *, clang::CharUnits>
         &base_offsets,
     llvm::DenseMap<const clang::CXXRecordDecl *, clang::CharUnits>
         &vbase_offsets) {
+  record_decl = record_decl->getDefinition();
   RecordDeclToLayoutMap::iterator pos =
       m_record_decl_to_layout_map.find(record_decl);
   bool success = false;
   base_offsets.clear();
   vbase_offsets.clear();
   if (pos != m_record_decl_to_layout_map.end()) {
     bit_size = pos->second.bit_size;
     alignment = pos->second.alignment;
     field_offsets.swap(pos->second.field_offsets);
     base_offsets.swap(pos->second.base_offsets);
     vbase_offsets.swap(pos->second.vbase_offsets);
-    m_record_decl_to_layout_map.erase(pos);
     success = true;
   } else {
     bit_size = 0;
     alignment = 0;
     field_offsets.clear();
   }
   return success;
 }
 
 void ClangASTImporter::SetRecordLayout(clang::RecordDecl *decl,
                                         const LayoutInfo &layout) {
   m_record_decl_to_layout_map.insert(std::make_pair(decl, layout));
 }
 
-bool ClangASTImporter::CompleteTagDecl(clang::TagDecl *decl) {
+bool ClangASTImporter::CompleteTagDecl(const TagDecl *decl) {
   DeclOrigin decl_origin = GetDeclOrigin(decl);
 
   if (!decl_origin.Valid())
     return false;
 
   if (!TypeSystemClang::GetCompleteDecl(decl_origin.ctx, decl_origin.decl))
     return false;
 
   ImporterDelegateSP delegate_sp(
       GetDelegate(&decl->getASTContext(), decl_origin.ctx));
 
   ASTImporterDelegate::CxxModuleScope std_scope(*delegate_sp,
                                                 &decl->getASTContext());
-  if (delegate_sp)
-    delegate_sp->ImportDefinitionTo(decl, decl_origin.decl);
+  if (!delegate_sp)
+    return false;
+
+  clang::TagDecl *origin_decl = llvm::cast<TagDecl>(decl_origin.decl);
 
+  origin_decl = origin_decl->getDefinition();
+  if (!origin_decl)
+    return false;
+
+  llvm::Expected<Decl *> result = delegate_sp->Import(origin_decl);
+  if (result)
     return true;
+  llvm::handleAllErrors(result.takeError(), [](clang::ImportError &e){
+      abort();
+  });
+
+  return false;
 }
 
 bool ClangASTImporter::CompleteTagDeclWithOrigin(clang::TagDecl *decl,
                                                  clang::TagDecl *origin_decl) {
-  clang::ASTContext *origin_ast_ctx = &origin_decl->getASTContext();
-
-  if (!TypeSystemClang::GetCompleteDecl(origin_ast_ctx, origin_decl))
+  origin_decl = origin_decl->getDefinition();
+  if (!origin_decl)
     return false;
 
-  ImporterDelegateSP delegate_sp(
-      GetDelegate(&decl->getASTContext(), origin_ast_ctx));
-
-  if (delegate_sp)
-    delegate_sp->ImportDefinitionTo(decl, origin_decl);
-
+  clang::ASTContext *origin_ast_ctx = &origin_decl->getASTContext();
   ASTContextMetadataSP context_md = GetContextMetadata(&decl->getASTContext());
-
   context_md->setOrigin(decl, DeclOrigin(origin_ast_ctx, origin_decl));
-  return true;
+
+  return CompleteTagDecl(decl);
 }
 
 bool ClangASTImporter::CompleteObjCInterfaceDecl(
     clang::ObjCInterfaceDecl *interface_decl) {
   DeclOrigin decl_origin = GetDeclOrigin(interface_decl);
 
   if (!decl_origin.Valid())
     return false;
ClangASTMetadata *ClangASTImporter::GetDeclMetadata(const clang::Decl *decl) {
   }
   TypeSystemClang *ast = TypeSystemClang::GetASTContext(&decl->getASTContext());
   return ast->GetMetadata(decl);
 }
 
 ClangASTImporter::DeclOrigin
 ClangASTImporter::GetDeclOrigin(const clang::Decl *decl) {
   ASTContextMetadataSP context_md = GetContextMetadata(&decl->getASTContext());
-
-  return context_md->getOrigin(decl);
+  DeclOrigin o = context_md->getOrigin(decl);
+  if (o.Valid())
+    return o;
+  if (const clang::TagDecl *td = dyn_cast<const TagDecl>(decl)) {
+    if (const clang::TagDecl *def = td->getDefinition()) {
+      o = context_md->getOrigin(def);
+    }
+  }
+  return o;
 }
 
 void ClangASTImporter::SetDeclOrigin(const clang::Decl *decl,
                                      clang::Decl *original_decl) {
   ASTContextMetadataSP context_md = GetContextMetadata(&decl->getASTContext());
   context_md->setOrigin(
       decl, DeclOrigin(&original_decl->getASTContext(), original_decl));
 }
ClangASTImporter::ASTImporterDelegate::ImportImpl(Decl *From) {
   if (origin.Valid()) {
     auto R = m_master.CopyDecl(&getToContext(), origin.decl);
     if (R) {
       RegisterImportedDecl(From, R);
       return R;
     }
   }
 
+  if (isa<TagDecl>(From))
+    if (clang::ExternalASTSource *s = getToContext().getExternalSource())
+      s->incrementGeneration(getToContext());
+
   // If we have a forcefully completed type, try to find an actual definition
   // for it in other modules.
   const ClangASTMetadata *md = m_master.GetDeclMetadata(From);
   auto *td = dyn_cast<TagDecl>(From);
   if (td && md && md->IsForcefullyCompleted()) {
     Log *log = GetLogIfAllCategoriesSet(LIBLLDB_LOG_EXPRESSIONS);
     LLDB_LOG(log,
              "[ClangASTImporter] Searching for a complete definition of {0} in "
if (from_context_md) {
     to_context_md->setOrigin(to, DeclOrigin(m_source_ctx, from));
 
     LLDB_LOG(log,
              "    [ClangASTImporter] Sourced origin "
              "(Decl*){0}/(ASTContext*){1} into (ASTContext*){2}",
              from, m_source_ctx, &to->getASTContext());
   }
 
-  if (auto *to_tag_decl = dyn_cast<TagDecl>(to)) {
-    to_tag_decl->setHasExternalLexicalStorage();
-    to_tag_decl->getPrimaryContext()->setMustBuildLookupTable();
-    auto from_tag_decl = cast<TagDecl>(from);
-
-    LLDB_LOG(
-        log,
-        "    [ClangASTImporter] To is a TagDecl - attributes {0}{1} [{2}->{3}]",
-        (to_tag_decl->hasExternalLexicalStorage() ? " Lexical" : ""),
-        (to_tag_decl->hasExternalVisibleStorage() ? " Visible" : ""),
-        (from_tag_decl->isCompleteDefinition() ? "complete" : "incomplete"),
-        (to_tag_decl->isCompleteDefinition() ? "complete" : "incomplete"));
-  }
-
   if (auto *to_namespace_decl = dyn_cast<NamespaceDecl>(to)) {
     m_master.BuildNamespaceMap(to_namespace_decl);
     to_namespace_decl->setHasExternalVisibleStorage();
   }
 
   if (auto *to_container_decl = dyn_cast<ObjCContainerDecl>(to)) {
-    to_container_decl->setHasExternalLexicalStorage();
-    to_container_decl->setHasExternalVisibleStorage();
-
     if (log) {
       if (ObjCInterfaceDecl *to_interface_decl =
               llvm::dyn_cast<ObjCInterfaceDecl>(to_container_decl)) {
         LLDB_LOG(
             log,
             "    [ClangASTImporter] To is an ObjCInterfaceDecl - attributes "
-            "{0}{1}{2}",
-            (to_interface_decl->hasExternalLexicalStorage() ? " Lexical" : ""),
-            (to_interface_decl->hasExternalVisibleStorage() ? " Visible" : ""),
+            "{0}",
             (to_interface_decl->hasDefinition() ? " HasDefinition" : ""));
       } else {
         LLDB_LOG(
-            log, "    [ClangASTImporter] To is an {0}Decl - attributes {1}{2}",
-            ((Decl *)to_container_decl)->getDeclKindName(),
-            (to_container_decl->hasExternalLexicalStorage() ? " Lexical" : ""),
-            (to_container_decl->hasExternalVisibleStorage() ? " Visible" : ""));
+            log, "    [ClangASTImporter] To is an {0}Decl",
+            ((Decl *)to_container_decl)->getDeclKindName());
       }
     }
   }
 
   if (clang::CXXMethodDecl *to_method = dyn_cast<CXXMethodDecl>(to))
     MaybeCompleteReturnType(m_master, to_method);
 }
 
 clang::Decl *
 ClangASTImporter::ASTImporterDelegate::GetOriginalDecl(clang::Decl *To) {
   return m_master.GetDeclOrigin(To).decl;
 }

lldb/source/Plugins/ExpressionParser/Clang/ClangASTSource.h

 //===-- ClangASTSource.h ----------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLDB_SOURCE_PLUGINS_EXPRESSIONPARSER_CLANG_CLANGASTSOURCE_H
 #define LLDB_SOURCE_PLUGINS_EXPRESSIONPARSER_CLANG_CLANGASTSOURCE_H
 
 #include <set>
 
 #include "Plugins/ExpressionParser/Clang/ClangASTImporter.h"
 #include "Plugins/ExpressionParser/Clang/NameSearchContext.h"
 #include "lldb/Symbol/CompilerType.h"
 #include "lldb/Target/Target.h"
 #include "clang/AST/ExternalASTSource.h"
 #include "clang/Basic/IdentifierTable.h"
 
 #include "llvm/ADT/SmallSet.h"
 
public:
   void CompleteType(clang::TagDecl *Tag) override;
 
   /// Complete an ObjCInterfaceDecl.
   ///
   /// \param[in] Class
   ///     The Decl to be completed in place.
   void CompleteType(clang::ObjCInterfaceDecl *Class) override;
 
+  void CompleteRedeclChain(const clang::Decl *D) override;
+
   /// Called on entering a translation unit.  Tells Clang by calling
   /// setHasExternalVisibleStorage() and setHasExternalLexicalStorage() that
   /// this object has something to say about undefined names.
   ///
   /// \param[in] Consumer
   ///     Unused.
   void StartTranslationUnit(clang::ASTConsumer *Consumer) override;
 
public:
     void CompleteType(clang::TagDecl *Tag) override {
       return m_original.CompleteType(Tag);
     }
 
     void CompleteType(clang::ObjCInterfaceDecl *Class) override {
       return m_original.CompleteType(Class);
     }
 
+    void CompleteRedeclChain(const clang::Decl *D) override {
+      return m_original.CompleteRedeclChain(D);
+    }
+
+
     bool layoutRecordType(
         const clang::RecordDecl *Record, uint64_t &Size, uint64_t &Alignment,
         llvm::DenseMap<const clang::FieldDecl *, uint64_t> &FieldOffsets,
         llvm::DenseMap<const clang::CXXRecordDecl *, clang::CharUnits>
             &BaseOffsets,
         llvm::DenseMap<const clang::CXXRecordDecl *, clang::CharUnits>
             &VirtualBaseOffsets) override {
       return m_original.layoutRecordType(Record, Size, Alignment, FieldOffsets,

lldb/source/Plugins/ExpressionParser/Clang/ClangASTSource.cpp

void ClangASTSource::CompleteType(clang::ObjCInterfaceDecl *interface_decl) {
   if (interface_decl->getSuperClass() &&
       interface_decl->getSuperClass() != interface_decl)
     CompleteType(interface_decl->getSuperClass());
 
   LLDB_LOG(log, "      [COID] After:");
   LLDB_LOG(log, "      [COID] {0}", ClangUtil::DumpDecl(interface_decl));
 }
 
+void ClangASTSource::CompleteRedeclChain(const Decl *d) {
+  if (const clang::TagDecl *td = llvm::dyn_cast<TagDecl>(d)) {
+    m_ast_importer_sp->CompleteTagDecl(td);
+  }
+}
+
 clang::ObjCInterfaceDecl *ClangASTSource::GetCompleteObjCInterface(
     const clang::ObjCInterfaceDecl *interface_decl) {
   lldb::ProcessSP process(m_target->GetProcessSP());
 
   if (!process)
     return nullptr;
 
   ObjCLanguageRuntime *language_runtime(ObjCLanguageRuntime::Get(*process));

lldb/source/Plugins/ExpressionParser/Clang/ClangExternalASTSourceCallbacks.h

 //===-- ClangExternalASTSourceCallbacks.h -----------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLDB_SOURCE_PLUGINS_EXPRESSIONPARSER_CLANG_CLANGEXTERNALASTSOURCECALLBACKS_H
 #define LLDB_SOURCE_PLUGINS_EXPRESSIONPARSER_CLANG_CLANGEXTERNALASTSOURCECALLBACKS_H
 
 #include "Plugins/TypeSystem/Clang/TypeSystemClang.h"
 #include "clang/Basic/Module.h"
 
 namespace lldb_private {
 
 class ClangExternalASTSourceCallbacks : public clang::ExternalASTSource {
   /// LLVM RTTI support.
   static char ID;
 
public:
 
   bool FindExternalVisibleDeclsByName(const clang::DeclContext *DC,
                                       clang::DeclarationName Name) override;
 
   void CompleteType(clang::TagDecl *tag_decl) override;
 
   void CompleteType(clang::ObjCInterfaceDecl *objc_decl) override;
 
+  void CompleteRedeclChain(const clang::Decl *D) override;
+
   bool layoutRecordType(
       const clang::RecordDecl *Record, uint64_t &Size, uint64_t &Alignment,
       llvm::DenseMap<const clang::FieldDecl *, uint64_t> &FieldOffsets,
       llvm::DenseMap<const clang::CXXRecordDecl *, clang::CharUnits>
           &BaseOffsets,
       llvm::DenseMap<const clang::CXXRecordDecl *, clang::CharUnits>
           &VirtualBaseOffsets) override;
 
   TypeSystemClang &GetTypeSystem() const { return m_ast; }
 
+  void bumpGeneration() {
+    incrementGeneration(m_ast.getASTContext());
+  }
+
   /// Module-related methods.
   /// \{
   llvm::Optional<clang::ASTSourceDescriptor>
   getSourceDescriptor(unsigned ID) override;
   clang::Module *getModule(unsigned ID) override;
   OptionalClangModuleID RegisterModule(clang::Module *module);
   OptionalClangModuleID GetIDForModule(clang::Module *module);
   /// \}

lldb/source/Plugins/ExpressionParser/Clang/ClangExternalASTSourceCallbacks.cpp

 //===-- ClangExternalASTSourceCallbacks.cpp -------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "Plugins/ExpressionParser/Clang/ClangExternalASTSourceCallbacks.h"
 #include "Plugins/TypeSystem/Clang/TypeSystemClang.h"
 
 #include "clang/AST/Decl.h"
 #include "clang/AST/DeclObjC.h"
 
 using namespace lldb_private;
 
 char ClangExternalASTSourceCallbacks::ID;
 
 void ClangExternalASTSourceCallbacks::CompleteType(clang::TagDecl *tag_decl) {
-  m_ast.CompleteTagDecl(tag_decl);
+  //m_ast.CompleteTagDecl(tag_decl);
 }
 
 void ClangExternalASTSourceCallbacks::CompleteType(
     clang::ObjCInterfaceDecl *objc_decl) {
   m_ast.CompleteObjCInterfaceDecl(objc_decl);
 }
 
+void ClangExternalASTSourceCallbacks::CompleteRedeclChain(const clang::Decl *d) {
+  if (const clang::TagDecl *td = llvm::dyn_cast<clang::TagDecl>(d))
+    m_ast.CompleteTagDecl(td);
+}
+
 bool ClangExternalASTSourceCallbacks::layoutRecordType(
     const clang::RecordDecl *Record, uint64_t &Size, uint64_t &Alignment,
     llvm::DenseMap<const clang::FieldDecl *, uint64_t> &FieldOffsets,
     llvm::DenseMap<const clang::CXXRecordDecl *, clang::CharUnits> &BaseOffsets,
     llvm::DenseMap<const clang::CXXRecordDecl *, clang::CharUnits>
         &VirtualBaseOffsets) {
   return m_ast.LayoutRecordType(Record, Size, Alignment, FieldOffsets,
                                 BaseOffsets, VirtualBaseOffsets);

lldb/source/Plugins/SymbolFile/DWARF/DWARFASTParserClang.h

 //===-- DWARFASTParserClang.h -----------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLDB_SOURCE_PLUGINS_SYMBOLFILE_DWARF_DWARFASTPARSERCLANG_H
 #define LLDB_SOURCE_PLUGINS_SYMBOLFILE_DWARF_DWARFASTPARSERCLANG_H
 
 #include "clang/AST/CharUnits.h"
 #include "llvm/ADT/DenseMap.h"
 #include "llvm/ADT/SmallPtrSet.h"
 #include "llvm/ADT/SmallVector.h"
 
 #include "DWARFASTParser.h"
 #include "DWARFDIE.h"
 #include "DWARFDefines.h"
 #include "DWARFFormValue.h"
protected:
   /// Protected typedefs and members.
   /// @{
   class DelayedAddObjCClassProperty;
   typedef std::vector<DelayedAddObjCClassProperty> DelayedPropertyList;
 
   typedef llvm::SmallPtrSet<const DWARFDebugInfoEntry *, 4> DIEPointerSet;
   typedef llvm::DenseMap<const DWARFDebugInfoEntry *, clang::DeclContext *>
       DIEToDeclContextMap;
+  typedef llvm::DenseMap<const DWARFDebugInfoEntry *, clang::TagDecl *>
+      DIEToRecordMap;
   typedef std::multimap<const clang::DeclContext *, const DWARFDIE>
       DeclContextToDIEMap;
   typedef llvm::DenseMap<const DWARFDebugInfoEntry *,
                          lldb_private::OptionalClangModuleID>
       DIEToModuleMap;
   typedef llvm::DenseMap<const DWARFDebugInfoEntry *, clang::Decl *>
       DIEToDeclMap;
   typedef llvm::DenseMap<const clang::Decl *, DIEPointerSet> DeclToDIEMap;
 
   lldb_private::TypeSystemClang &m_ast;
   DIEToDeclMap m_die_to_decl;
   DeclToDIEMap m_decl_to_die;
   DIEToDeclContextMap m_die_to_decl_ctx;
   DeclContextToDIEMap m_decl_ctx_to_die;
+  DIEToRecordMap m_die_to_record_map;
   DIEToModuleMap m_die_to_module;
   std::unique_ptr<lldb_private::ClangASTImporter> m_clang_ast_importer_up;
   /// @}
 
   clang::DeclContext *GetDeclContextForBlock(const DWARFDIE &die);
 
   clang::BlockDecl *ResolveBlockDIE(const DWARFDIE &die);
 
ParseSingleMember(const DWARFDIE &die, const DWARFDIE &parent_die,
                     const lldb_private::CompilerType &class_clang_type,
                     std::vector<int> &member_accessibilities,
                     lldb::AccessType default_accessibility,
                     DelayedPropertyList &delayed_properties,
                     lldb_private::ClangASTImporter::LayoutInfo &layout_info,
                     FieldInfo &last_field_info);
 
   bool CompleteRecordType(const DWARFDIE &die, lldb_private::Type *type,
-                          lldb_private::CompilerType &clang_type);
+                          lldb_private::CompilerType clang_type);
   bool CompleteEnumType(const DWARFDIE &die, lldb_private::Type *type,
                         lldb_private::CompilerType &clang_type);
 
   lldb::TypeSP ParseTypeModifier(const lldb_private::SymbolContext &sc,
                                  const DWARFDIE &die,
                                  ParsedDWARFTypeAttributes &attrs);
   lldb::TypeSP ParseEnum(const lldb_private::SymbolContext &sc,
                          const DWARFDIE &die, ParsedDWARFTypeAttributes &attrs);

lldb/source/Plugins/SymbolFile/DWARF/DWARFASTParserClang.cpp

if (attrs.name.GetStringRef().contains('<')) {
 
         m_ast.SetMetadata(class_template_decl, metadata);
         m_ast.SetMetadata(class_specialization_decl, metadata);
       }
     }
 
     if (!clang_type_was_created) {
       clang_type_was_created = true;
-      clang_type = m_ast.CreateRecordType(
+      auto created_type = m_ast.CreateRecordDecl(
           decl_ctx, GetOwningClangModule(die), attrs.accessibility,
           attrs.name.GetCString(), tag_decl_kind, attrs.class_language,
           &metadata, attrs.exports_symbols);
+      if (clang::TagDecl *td = llvm::dyn_cast<clang::TagDecl>(created_type)) {
+        m_die_to_record_map[die.GetDIE()] = td;
+      }
+      clang_type = m_ast.GetTypeForDecl(created_type);
     }
   }
 
   // Store a forward declaration to this class type in case any
   // parameters in any class methods need it for the clang types for
   // function prototypes.
-  LinkDeclContextToDIE(m_ast.GetDeclContextForType(clang_type), die);
+  LinkDeclContextToDIE(GetClangDeclContextContainingDIE(die, nullptr), die);
   type_sp = std::make_shared<Type>(
       die.GetID(), dwarf, attrs.name, attrs.byte_size, nullptr,
       LLDB_INVALID_UID, Type::eEncodingIsUID, &attrs.decl, clang_type,
       Type::ResolveState::Forward,
       TypePayloadClang(OptionalClangModuleID(), attrs.is_complete_objc_class));
 
   // Add our type to the unique type map so we don't end up creating many
   // copies of the same type over and over in the ASTContext for our
if (!die.HasChildren()) {
             TypeSystemClang::GetAsRecordDecl(clang_type);
 
         if (record_decl) {
           GetClangASTImporter().SetRecordLayout(
               record_decl, ClangASTImporter::LayoutInfo());
         }
       }
     } else if (clang_type_was_created) {
-      // Start the definition if the class is not objective C since the
-      // underlying decls respond to isCompleteDefinition(). Objective
-      // C decls don't respond to isCompleteDefinition() so we can't
-      // start the declaration definition right away. For C++
-      // class/union/structs we want to start the definition in case the
-      // class is needed as the declaration context for a contained class
-      // or type without the need to complete that type..
-
-      if (attrs.class_language != eLanguageTypeObjC &&
-          attrs.class_language != eLanguageTypeObjC_plus_plus)
-        TypeSystemClang::StartTagDeclarationDefinition(clang_type);
-
       // Leave this as a forward declaration until we need to know the
       // details of the type. lldb_private::Type will automatically call
       // the SymbolFile virtual function
       // "SymbolFileDWARF::CompleteType(Type *)" When the definition
       // needs to be defined.
       assert(!dwarf->GetForwardDeclClangTypeToDie().count(
                  ClangUtil::RemoveFastQualifiers(clang_type)
                      .GetOpaqueQualType()) &&
              "Type already in the forward declaration map!");
       // Can't assume m_ast.GetSymbolFile() is actually a
       // SymbolFileDWARF, it can be a SymbolFileDWARFDebugMap for Apple
       // binaries.
       dwarf->GetForwardDeclDieToClangType()[die.GetDIE()] =
           clang_type.GetOpaqueQualType();
       dwarf->GetForwardDeclClangTypeToDie().try_emplace(
           ClangUtil::RemoveFastQualifiers(clang_type).GetOpaqueQualType(),
           *die.GetDIERef());
-      m_ast.SetHasExternalStorage(clang_type.GetOpaqueQualType(), true);
     }
   }
 
   // If we made a clang type, set the trivial abi if applicable: We only
   // do this for pass by value - which implies the Trivial ABI. There
   // isn't a way to assert that something that would normally be pass by
   // value is pass by reference, so we ignore that attribute if set.
-  if (attrs.calling_convention == llvm::dwarf::DW_CC_pass_by_value) {
+  /*if (attrs.calling_convention == llvm::dwarf::DW_CC_pass_by_value) {
     clang::CXXRecordDecl *record_decl =
         m_ast.GetAsCXXRecordDecl(clang_type.GetOpaqueQualType());
     if (record_decl && record_decl->getDefinition()) {
       record_decl->setHasTrivialSpecialMemberForCall();
     }
   }
 
   if (attrs.calling_convention == llvm::dwarf::DW_CC_pass_by_reference) {
     clang::CXXRecordDecl *record_decl =
         m_ast.GetAsCXXRecordDecl(clang_type.GetOpaqueQualType());
     if (record_decl)
       record_decl->setArgPassingRestrictions(
           clang::RecordDecl::APK_CannotPassInRegs);
-  }
+  } */
   return type_sp;
 }
 
 // DWARF parsing functions
 
 class DWARFASTParserClang::DelayedAddObjCClassProperty {
 public:
   DelayedAddObjCClassProperty(
default:
       break;
     }
   }
   return template_param_infos.args.size() == template_param_infos.names.size();
 }
 
 bool DWARFASTParserClang::CompleteRecordType(const DWARFDIE &die,
                                              lldb_private::Type *type,
-                                             CompilerType &clang_type) {
+                                             CompilerType clang_type) {
   const dw_tag_t tag = die.Tag();
   SymbolFileDWARF *dwarf = die.GetDWARF();
 
   ClangASTImporter::LayoutInfo layout_info;
 
   if (die.HasChildren()) {
-    const bool type_is_objc_object_or_interface =
-        TypeSystemClang::IsObjCObjectOrInterfaceType(clang_type);
-    if (type_is_objc_object_or_interface) {
-      // For objective C we don't start the definition when the class is
-      // created.
-      TypeSystemClang::StartTagDeclarationDefinition(clang_type);
-    }
-
+    clang_type = m_ast.RedeclTagDecl(clang_type);
+    m_die_to_record_map[die.GetDIE()] = ClangUtil::GetAsTagDecl(clang_type);
     int tag_decl_kind = -1;
     AccessType default_accessibility = eAccessNone;
     if (tag == DW_TAG_structure_type) {
       tag_decl_kind = clang::TTK_Struct;
       default_accessibility = eAccessPublic;
     } else if (tag == DW_TAG_union_type) {
       tag_decl_kind = clang::TTK_Union;
       default_accessibility = eAccessPublic;
if (die.HasChildren()) {
     ParseChildMembers(die, clang_type, bases, member_accessibilities,
                       member_function_dies, delayed_properties,
                       default_accessibility, is_a_class, layout_info);
 
     // Now parse any methods if there were any...
     for (const DWARFDIE &die : member_function_dies)
       dwarf->ResolveType(die);
 
+    const bool type_is_objc_object_or_interface =
+        TypeSystemClang::IsObjCObjectOrInterfaceType(clang_type);
     if (type_is_objc_object_or_interface) {
       ConstString class_name(clang_type.GetTypeName());
       if (class_name) {
         dwarf->GetObjCMethods(class_name, [&](DWARFDIE method_die) {
           method_die.ResolveType();
           return true;
         });
 
if (layout_info.bit_size == 0)
       layout_info.bit_size =
           die.GetAttributeValueAsUnsigned(DW_AT_byte_size, 0) * 8;
 
     clang::CXXRecordDecl *record_decl =
         m_ast.GetAsCXXRecordDecl(clang_type.GetOpaqueQualType());
     if (record_decl)
       GetClangASTImporter().SetRecordLayout(record_decl, layout_info);
   }
-
   return (bool)clang_type;
 }
 
 bool DWARFASTParserClang::CompleteEnumType(const DWARFDIE &die,
                                            lldb_private::Type *type,
                                            CompilerType &clang_type) {
   if (TypeSystemClang::StartTagDeclarationDefinition(clang_type)) {
     if (die.HasChildren()) {
       bool is_signed = false;
       clang_type.IsIntegerType(is_signed);
       ParseChildEnumerators(clang_type, is_signed,
                             type->GetByteSize(nullptr).getValueOr(0), die);
     }
     TypeSystemClang::CompleteTagDeclarationDefinition(clang_type);
   }
+
   return (bool)clang_type;
 }
 
 bool DWARFASTParserClang::CompleteTypeFromDWARF(const DWARFDIE &die,
                                                 lldb_private::Type *type,
                                                 CompilerType &clang_type) {
   SymbolFileDWARF *dwarf = die.GetDWARF();
 
   std::lock_guard<std::recursive_mutex> guard(
       dwarf->GetObjectFile()->GetModule()->GetMutex());
 
-  // Disable external storage for this type so we don't get anymore
-  // clang::ExternalASTSource queries for this type.
-  m_ast.SetHasExternalStorage(clang_type.GetOpaqueQualType(), false);
-
   if (!die)
     return false;
 
   const dw_tag_t tag = die.Tag();
 
   Log *log =
       nullptr; // (LogChannelDWARF::GetLogIfAny(DWARF_LOG_DEBUG_INFO|DWARF_LOG_TYPE_COMPLETION));
   if (log)
if (clang_decl_ctx)
       return clang_decl_ctx;
   }
   return m_ast.GetTranslationUnitDecl();
 }
 
 clang::DeclContext *
 DWARFASTParserClang::GetCachedClangDeclContextForDIE(const DWARFDIE &die) {
   if (die) {
+    DIEToRecordMap::iterator pos2 = m_die_to_record_map.find(die.GetDIE());
+    if (pos2 != m_die_to_record_map.end())
+      return pos2->second;
     DIEToDeclContextMap::iterator pos = m_die_to_decl_ctx.find(die.GetDIE());
     if (pos != m_die_to_decl_ctx.end())
       return pos->second;
   }
   return nullptr;
 }
 
 void DWARFASTParserClang::LinkDeclContextToDIE(clang::DeclContext *decl_ctx,

lldb/source/Plugins/TypeSystem/Clang/TypeSystemClang.h

 
 #include "clang/AST/ASTContext.h"
 #include "clang/AST/ASTFwd.h"
 #include "clang/AST/TemplateBase.h"
 #include "clang/Basic/TargetInfo.h"
 #include "llvm/ADT/APSInt.h"
 #include "llvm/ADT/SmallVector.h"
 
 #include "Plugins/ExpressionParser/Clang/ClangPersistentVariables.h"
 #include "lldb/Expression/ExpressionVariable.h"
 #include "lldb/Symbol/CompilerType.h"
 #include "lldb/Symbol/TypeSystem.h"
 #include "lldb/Target/Target.h"
 #include "lldb/Utility/ConstString.h"
 #include "lldb/Utility/Flags.h"
 #include "lldb/Utility/Log.h"
 #include "lldb/Utility/Logging.h"
 #include "lldb/lldb-enumerations.h"
 
 class DWARFASTParserClang;
 class PDBASTParser;
 
 namespace clang {
 class FileManager;
 class HeaderSearch;
 class ModuleMap;
 } // namespace clang
 
 namespace lldb_private {
 
+class ClangExternalASTSourceCallbacks;
 class ClangASTMetadata;
 class ClangASTSource;
 class Declaration;
 
 /// A Clang module ID.
 class OptionalClangModuleID {
   unsigned m_id = 0;
 
public:
   /// \param qt The QualType for a type that belongs to the ASTContext of this
   ///           TypeSystemClang.
   /// \return The CompilerType representing the given QualType. If the
   ///         QualType's type pointer is a nullptr then the function returns an
   ///         invalid CompilerType.
   CompilerType GetType(clang::QualType qt) {
     if (qt.getTypePtrOrNull() == nullptr)
       return CompilerType();
-    // Check that the type actually belongs to this TypeSystemClang.
-    assert(qt->getAsTagDecl() == nullptr ||
-           &qt->getAsTagDecl()->getASTContext() == &getASTContext());
     return CompilerType(this, qt.getAsOpaquePtr());
   }
 
   CompilerType GetTypeForDecl(clang::NamedDecl *decl);
 
   CompilerType GetTypeForDecl(clang::TagDecl *decl);
 
   CompilerType GetTypeForDecl(clang::ObjCInterfaceDecl *objc_decl);
OptionalClangModuleID GetOrCreateClangModule(llvm::StringRef name,
                                                bool is_explicit = false);
 
   CompilerType CreateRecordType(clang::DeclContext *decl_ctx,
                                 OptionalClangModuleID owning_module,
                                 lldb::AccessType access_type,
                                 llvm::StringRef name, int kind,
                                 lldb::LanguageType language,
                                 ClangASTMetadata *metadata = nullptr,
+                                bool exports_symbols = false) {
+    clang::NamedDecl *d = CreateRecordDecl(decl_ctx, owning_module, access_type,
+                                      name, kind, language, metadata,
+                                      exports_symbols);
+    return GetTypeForDecl(d);
+  }
+
+  clang::NamedDecl *CreateRecordDecl(clang::DeclContext *decl_ctx,
+                                OptionalClangModuleID owning_module,
+                                lldb::AccessType access_type,
+                                llvm::StringRef name, int kind,
+                                lldb::LanguageType language,
+                                ClangASTMetadata *metadata = nullptr,
                                 bool exports_symbols = false);
 
   class TemplateParameterInfos {
   public:
     bool IsValid() const {
       // Having a pack name but no packed args doesn't make sense, so mark
       // these template parameters as invalid.
       if (pack_name && !packed_args)
public:
   bool FieldIsBitfield(clang::FieldDecl *field, uint32_t &bitfield_bit_size);
 
   static bool RecordHasFields(const clang::RecordDecl *record_decl);
 
   CompilerType CreateObjCClass(llvm::StringRef name,
                                clang::DeclContext *decl_ctx,
                                OptionalClangModuleID owning_module,
                                bool isForwardDecl, bool isInternal,
+                               ClangASTMetadata *metadata = nullptr) {
+    clang::ObjCInterfaceDecl *d = CreateObjCDecl(name, decl_ctx, owning_module,
+                                                 isForwardDecl, isInternal,
+                                                 metadata);
+    return GetTypeForDecl(d);
+  }
+
+  clang::ObjCInterfaceDecl *CreateObjCDecl(llvm::StringRef name,
+                               clang::DeclContext *decl_ctx,
+                               OptionalClangModuleID owning_module,
+                               bool isForwardDecl, bool isInternal,
                                ClangASTMetadata *metadata = nullptr);
 
   bool SetTagTypeKind(clang::QualType type, int kind) const;
 
   bool SetDefaultAccessForRecordFields(clang::RecordDecl *record_decl,
                                        int default_accessibility,
                                        int *assigned_accessibilities,
                                        size_t num_assigned_accessibilities);
public:
   static CompilerType GetFloatTypeFromBitSize(clang::ASTContext *ast,
                                               size_t bit_size);
 
   // TypeSystem methods
   DWARFASTParser *GetDWARFParser() override;
   PDBASTParser *GetPDBParser() override;
 
   // TypeSystemClang callbacks for external source lookups.
-  void CompleteTagDecl(clang::TagDecl *);
+  void CompleteTagDecl(const clang::TagDecl *);
 
   void CompleteObjCInterfaceDecl(clang::ObjCInterfaceDecl *);
 
+  CompilerType RedeclTagDecl(CompilerType ct);
+
   bool LayoutRecordType(
       const clang::RecordDecl *record_decl, uint64_t &size, uint64_t &alignment,
       llvm::DenseMap<const clang::FieldDecl *, uint64_t> &field_offsets,
       llvm::DenseMap<const clang::CXXRecordDecl *, clang::CharUnits>
           &base_offsets,
       llvm::DenseMap<const clang::CXXRecordDecl *, clang::CharUnits>
           &vbase_offsets);
 
private:
   /// Returns the internal type name for the given NamedDecl using the
   /// type printing policy.
   std::string GetTypeNameForDecl(const clang::NamedDecl *named_decl);
 
   const clang::ClassTemplateSpecializationDecl *
   GetAsTemplateSpecialization(lldb::opaque_compiler_type_t type);
 
   // Classes that inherit from TypeSystemClang can see and modify these
+  ClangExternalASTSourceCallbacks *m_ast_callbacks = nullptr;
   std::string m_target_triple;
   std::unique_ptr<clang::ASTContext> m_ast_up;
   std::unique_ptr<clang::LangOptions> m_language_options_up;
   std::unique_ptr<clang::FileManager> m_file_manager_up;
   std::unique_ptr<clang::SourceManager> m_source_manager_up;
   std::unique_ptr<clang::DiagnosticsEngine> m_diagnostics_engine_up;
   std::unique_ptr<clang::DiagnosticConsumer> m_diagnostic_consumer_up;
   std::shared_ptr<clang::TargetOptions> m_target_options_rp;

lldb/source/Plugins/TypeSystem/Clang/TypeSystemClang.cpp

 #include "clang/Basic/SourceManager.h"
 #include "clang/Basic/TargetInfo.h"
 #include "clang/Basic/TargetOptions.h"
 #include "clang/Frontend/FrontendOptions.h"
 #include "clang/Lex/HeaderSearch.h"
 #include "clang/Lex/HeaderSearchOptions.h"
 #include "clang/Lex/ModuleMap.h"
 #include "clang/Sema/Sema.h"
 
 #include "llvm/Support/Signals.h"
 #include "llvm/Support/Threading.h"
+#include "llvm/ADT/ScopeExit.h"
 
 #include "Plugins/ExpressionParser/Clang/ClangASTImporter.h"
 #include "Plugins/ExpressionParser/Clang/ClangASTMetadata.h"
 #include "Plugins/ExpressionParser/Clang/ClangExternalASTSourceCallbacks.h"
 #include "Plugins/ExpressionParser/Clang/ClangFunctionCaller.h"
 #include "Plugins/ExpressionParser/Clang/ClangPersistentVariables.h"
 #include "Plugins/ExpressionParser/Clang/ClangUserExpression.h"
 #include "Plugins/ExpressionParser/Clang/ClangUtil.h"
void TypeSystemClang::CreateASTContext() {
   // built
   TargetInfo *target_info = getTargetInfo();
   if (target_info)
     m_ast_up->InitBuiltinTypes(*target_info);
 
   GetASTMap().Insert(m_ast_up.get(), this);
 
   llvm::IntrusiveRefCntPtr<clang::ExternalASTSource> ast_source_up(
-      new ClangExternalASTSourceCallbacks(*this));
+      m_ast_callbacks = new ClangExternalASTSourceCallbacks(*this));
   SetExternalSource(ast_source_up);
 }
 
 TypeSystemClang *TypeSystemClang::GetASTContext(clang::ASTContext *ast) {
   TypeSystemClang *clang_ast = GetASTMap().Lookup(ast);
   return clang_ast;
 }
 
std::tie(module, created) = m_module_map_up->findOrCreateModule(
       name, parent_desc ? parent_desc->getModuleOrNull() : nullptr,
       is_framework, is_explicit);
   if (!created)
     return ast_source->GetIDForModule(module);
 
   return ast_source->RegisterModule(module);
 }
 
-CompilerType TypeSystemClang::CreateRecordType(
-    clang::DeclContext *decl_ctx, OptionalClangModuleID owning_module,
+clang::NamedDecl *TypeSystemClang::CreateRecordDecl(clang::DeclContext *decl_ctx, OptionalClangModuleID owning_module,
     AccessType access_type, llvm::StringRef name, int kind,
     LanguageType language, ClangASTMetadata *metadata, bool exports_symbols) {
   ASTContext &ast = getASTContext();
 
   if (decl_ctx == nullptr)
     decl_ctx = ast.getTranslationUnitDecl();
 
+  auto bumper = llvm::make_scope_exit([this](){
+    if (m_ast_up)
+      m_ast_up->getExternalSource()->incrementGeneration(*m_ast_up);
+  });
+
   if (language == eLanguageTypeObjC ||
       language == eLanguageTypeObjC_plus_plus) {
     bool isForwardDecl = true;
     bool isInternal = false;
-    return CreateObjCClass(name, decl_ctx, owning_module, isForwardDecl,
+    return CreateObjCDecl(name, decl_ctx, owning_module, isForwardDecl,
                            isInternal, metadata);
   }
 
   // NOTE: Eventually CXXRecordDecl will be merged back into RecordDecl and
   // we will need to update this code. I was told to currently always use the
   // CXXRecordDecl class since we often don't know from debug information if
   // something is struct or a class, so we default to always use the more
   // complete definition just in case.
 
if (metadata)
       SetMetadata(decl, *metadata);
 
     if (access_type != eAccessNone)
       decl->setAccess(ConvertAccessTypeToAccessSpecifier(access_type));
 
     if (decl_ctx)
       decl_ctx->addDecl(decl);
 
-    return GetType(ast.getTagDeclType(decl));
+    return decl;
   }
-  return CompilerType();
+  return nullptr;
 }
 
 namespace {
   bool IsValueParam(const clang::TemplateArgument &argument) {
     return argument.getKind() == TemplateArgument::Integral;
   }
 }
 
for (base_class = cxx_record_decl->bases_begin(),
         return true;
     }
   }
   return false;
 }
 
 #pragma mark Objective-C Classes
 
-CompilerType TypeSystemClang::CreateObjCClass(
+clang::ObjCInterfaceDecl *TypeSystemClang::CreateObjCDecl(
     llvm::StringRef name, clang::DeclContext *decl_ctx,
     OptionalClangModuleID owning_module, bool isForwardDecl, bool isInternal,
     ClangASTMetadata *metadata) {
   ASTContext &ast = getASTContext();
   assert(!name.empty());
   if (!decl_ctx)
     decl_ctx = ast.getTranslationUnitDecl();
 
   ObjCInterfaceDecl *decl = ObjCInterfaceDecl::CreateDeserialized(ast, 0);
   decl->setDeclContext(decl_ctx);
   decl->setDeclName(&ast.Idents.get(name));
   /*isForwardDecl,*/
   decl->setImplicit(isInternal);
   SetOwningModule(decl, owning_module);
 
   if (decl && metadata)
     SetMetadata(decl, *metadata);
 
-  return GetType(ast.getObjCInterfaceType(decl));
+  return decl;
 }
 
 static inline bool BaseSpecifierIsEmpty(const CXXBaseSpecifier *b) {
   return !TypeSystemClang::RecordHasFields(b->getType()->getAsCXXRecordDecl());
 }
 
 uint32_t
 TypeSystemClang::GetNumBaseClasses(const CXXRecordDecl *cxx_record_decl,
bool TypeSystemClang::GetCompleteDecl(clang::ASTContext *ast,
 
   if (!ast_source)
     return false;
 
   if (clang::TagDecl *tag_decl = llvm::dyn_cast<clang::TagDecl>(decl)) {
     if (tag_decl->isCompleteDefinition())
       return true;
 
-    if (!tag_decl->hasExternalLexicalStorage())
-      return false;
-
     ast_source->CompleteType(tag_decl);
 
-    return !tag_decl->getTypeForDecl()->isIncompleteType();
+    tag_decl = tag_decl->getDefinition();
+
+    return tag_decl && !tag_decl->getTypeForDecl()->isIncompleteType();
   } else if (clang::ObjCInterfaceDecl *objc_interface_decl =
                  llvm::dyn_cast<clang::ObjCInterfaceDecl>(decl)) {
     if (objc_interface_decl->getDefinition())
       return true;
 
-    if (!objc_interface_decl->hasExternalLexicalStorage())
-      return false;
-
     ast_source->CompleteType(objc_interface_decl);
 
-    return !objc_interface_decl->getTypeForDecl()->isIncompleteType();
+    objc_interface_decl = objc_interface_decl->getDefinition();
+
+    return objc_interface_decl && !objc_interface_decl->getTypeForDecl()->isIncompleteType();
   } else {
     return false;
   }
 }
 
 void TypeSystemClang::SetMetadataAsUserID(const clang::Decl *decl,
                                           user_id_t user_id) {
   ClangASTMetadata meta_data;
case clang::Type::Enum:
     return llvm::cast<clang::EnumType>(qual_type)->getDecl();
   default:
     break;
   }
   // No DeclContext in this type...
   return nullptr;
 }
 
-static bool GetCompleteQualType(clang::ASTContext *ast,
+static QualType GetCompleteQualType(clang::ASTContext *ast,
                                 clang::QualType qual_type,
                                 bool allow_completion = true) {
   qual_type = RemoveWrappingTypes(qual_type);
   const clang::Type::TypeClass type_class = qual_type->getTypeClass();
   switch (type_class) {
   case clang::Type::ConstantArray:
   case clang::Type::IncompleteArray:
   case clang::Type::VariableArray: {
     const clang::ArrayType *array_type =
         llvm::dyn_cast<clang::ArrayType>(qual_type.getTypePtr());
 
     if (array_type)
       return GetCompleteQualType(ast, array_type->getElementType(),
                                  allow_completion);
   } break;
   case clang::Type::Record: {
     clang::CXXRecordDecl *cxx_record_decl = qual_type->getAsCXXRecordDecl();
-    if (cxx_record_decl) {
-      if (cxx_record_decl->hasExternalLexicalStorage()) {
-        const bool is_complete = cxx_record_decl->isCompleteDefinition();
-        const bool fields_loaded =
-            cxx_record_decl->hasLoadedFieldsFromExternalStorage();
-        if (is_complete && fields_loaded)
-          return true;
-
-        if (!allow_completion)
-          return false;
-
-        // Call the field_begin() accessor to for it to use the external source
-        // to load the fields...
-        clang::ExternalASTSource *external_ast_source =
-            ast->getExternalSource();
-        if (external_ast_source) {
-          external_ast_source->CompleteType(cxx_record_decl);
-          if (cxx_record_decl->isCompleteDefinition()) {
-            cxx_record_decl->field_begin();
-            cxx_record_decl->setHasLoadedFieldsFromExternalStorage(true);
-          }
-        }
-      }
-    }
-    const clang::TagType *tag_type =
-        llvm::cast<clang::TagType>(qual_type.getTypePtr());
-    return !tag_type->isIncompleteType();
+    cxx_record_decl = cxx_record_decl->getDefinition();
+    if (!cxx_record_decl)
+      return QualType();
+    return QualType(cxx_record_decl->getTypeForDecl(), 0);
   } break;
 
   case clang::Type::Enum: {
     const clang::TagType *tag_type =
         llvm::dyn_cast<clang::TagType>(qual_type.getTypePtr());
     if (tag_type) {
       clang::TagDecl *tag_decl = tag_type->getDecl();
       if (tag_decl) {
         if (tag_decl->getDefinition())
-          return true;
-
-        if (!allow_completion)
-          return false;
+          return QualType(tag_decl->getTypeForDecl(), 0);
 
         if (tag_decl->hasExternalLexicalStorage()) {
           if (ast) {
             clang::ExternalASTSource *external_ast_source =
                 ast->getExternalSource();
             if (external_ast_source) {
               external_ast_source->CompleteType(tag_decl);
-              return !tag_type->isIncompleteType();
+              return QualType(tag_decl->getTypeForDecl(), 0);
             }
           }
         }
-        return false;
+        return QualType(tag_decl->getTypeForDecl(), 0);
       }
     }
 
   } break;
   case clang::Type::ObjCObject:
   case clang::Type::ObjCInterface: {
     const clang::ObjCObjectType *objc_class_type =
         llvm::dyn_cast<clang::ObjCObjectType>(qual_type);
     if (objc_class_type) {
       clang::ObjCInterfaceDecl *class_interface_decl =
           objc_class_type->getInterface();
-      // We currently can't complete objective C types through the newly added
-      // ASTContext because it only supports TagDecl objects right now...
-      if (class_interface_decl) {
-        if (class_interface_decl->getDefinition())
-          return true;
-
-        if (!allow_completion)
-          return false;
-
-        if (class_interface_decl->hasExternalLexicalStorage()) {
-          if (ast) {
-            clang::ExternalASTSource *external_ast_source =
-                ast->getExternalSource();
-            if (external_ast_source) {
-              external_ast_source->CompleteType(class_interface_decl);
-              return !objc_class_type->isIncompleteType();
-            }
-          }
-        }
-        return false;
-      }
+      class_interface_decl = class_interface_decl->getDefinition();
+      return QualType(class_interface_decl->getTypeForDecl(), 0);
     }
   } break;
 
   case clang::Type::Attributed:
     return GetCompleteQualType(
         ast, llvm::cast<clang::AttributedType>(qual_type)->getModifiedType(),
         allow_completion);
 
   default:
     break;
   }
 
-  return true;
+  return qual_type;
 }
 
 static clang::ObjCIvarDecl::AccessControl
 ConvertAccessTypeToObjCIvarAccessControl(AccessType access) {
   switch (access) {
   case eAccessNone:
     return clang::ObjCIvarDecl::None;
   case eAccessPublic:

 }
 
 bool TypeSystemClang::IsCharType(lldb::opaque_compiler_type_t type) {
   return GetQualType(type).getUnqualifiedType()->isCharType();
 }
 
 bool TypeSystemClang::IsCompleteType(lldb::opaque_compiler_type_t type) {
   const bool allow_completion = false;
-  return GetCompleteQualType(&getASTContext(), GetQualType(type),
-                             allow_completion);
+  return !GetCompleteQualType(&getASTContext(), GetQualType(type),
+                             allow_completion).isNull();
 }
 
 bool TypeSystemClang::IsConst(lldb::opaque_compiler_type_t type) {
   return GetQualType(type).isConstQualified();
 }
 
 bool TypeSystemClang::IsCStringType(lldb::opaque_compiler_type_t type,
                                     uint32_t &length) {

 }
 
 // Type Completion
 
 bool TypeSystemClang::GetCompleteType(lldb::opaque_compiler_type_t type) {
   if (!type)
     return false;
   const bool allow_completion = true;
-  return GetCompleteQualType(&getASTContext(), GetQualType(type),
-                             allow_completion);
+  return !GetCompleteQualType(&getASTContext(), GetQualType(type),
+                             allow_completion).isNull();
 }
 
 ConstString TypeSystemClang::GetTypeName(lldb::opaque_compiler_type_t type) {
   if (!type)
     return ConstString();
 
   clang::QualType qual_type(GetQualType(type));
 

 
 size_t
 TypeSystemClang::GetNumMemberFunctions(lldb::opaque_compiler_type_t type) {
   size_t num_functions = 0;
   if (type) {
     clang::QualType qual_type = RemoveWrappingTypes(GetCanonicalQualType(type));
     switch (qual_type->getTypeClass()) {
     case clang::Type::Record:
-      if (GetCompleteQualType(&getASTContext(), qual_type)) {
+      qual_type = GetCompleteQualType(&getASTContext(), qual_type);
+      if (!qual_type.isNull()) {
         const clang::RecordType *record_type =
             llvm::cast<clang::RecordType>(qual_type.getTypePtr());
         const clang::RecordDecl *record_decl = record_type->getDecl();
         assert(record_decl);
         const clang::CXXRecordDecl *cxx_record_decl =
             llvm::dyn_cast<clang::CXXRecordDecl>(record_decl);
         if (cxx_record_decl)
           num_functions = std::distance(cxx_record_decl->method_begin(),
TypeSystemClang::GetMemberFunctionAtIndex(lldb::opaque_compiler_type_t type,
   std::string name;
   MemberFunctionKind kind(MemberFunctionKind::eMemberFunctionKindUnknown);
   CompilerType clang_type;
   CompilerDecl clang_decl;
   if (type) {
     clang::QualType qual_type = RemoveWrappingTypes(GetCanonicalQualType(type));
     switch (qual_type->getTypeClass()) {
     case clang::Type::Record:
-      if (GetCompleteQualType(&getASTContext(), qual_type)) {
+      qual_type = GetCompleteQualType(&getASTContext(), qual_type);
+      if (!qual_type.isNull()) {
         const clang::RecordType *record_type =
             llvm::cast<clang::RecordType>(qual_type.getTypePtr());
         const clang::RecordDecl *record_decl = record_type->getDecl();
         assert(record_decl);
         const clang::CXXRecordDecl *cxx_record_decl =
             llvm::dyn_cast<clang::CXXRecordDecl>(record_decl);
         if (cxx_record_decl) {
           auto method_iter = cxx_record_decl->method_begin();
case clang::Type::Builtin:
     default:
       break;
     }
     break;
 
   case clang::Type::Complex:
     return 0;
   case clang::Type::Record:
-    if (GetCompleteQualType(&getASTContext(), qual_type)) {
+      qual_type = GetCompleteQualType(&getASTContext(), qual_type);
+      if (!qual_type.isNull()) {
       const clang::RecordType *record_type =
           llvm::cast<clang::RecordType>(qual_type.getTypePtr());
       const clang::RecordDecl *record_decl = record_type->getDecl();
       assert(record_decl);
       const clang::CXXRecordDecl *cxx_record_decl =
           llvm::dyn_cast<clang::CXXRecordDecl>(record_decl);
       if (cxx_record_decl) {
         if (omit_empty_base_classes) {
case clang::Type::Record:
           field_end = record_decl->field_end();
            field != field_end; ++field)
         ++num_children;
     }
     break;
 
   case clang::Type::ObjCObject:
   case clang::Type::ObjCInterface:
-    if (GetCompleteQualType(&getASTContext(), qual_type)) {
+      qual_type = GetCompleteQualType(&getASTContext(), qual_type);
+      if (!qual_type.isNull()) {
       const clang::ObjCObjectType *objc_class_type =
           llvm::dyn_cast<clang::ObjCObjectType>(qual_type.getTypePtr());
       assert(objc_class_type);
       if (objc_class_type) {
         clang::ObjCInterfaceDecl *class_interface_decl =
             objc_class_type->getInterface();
 
         if (class_interface_decl) {
if (auto *cxx_record_decl = llvm::dyn_cast<CXXRecordDecl>(tag_decl)) {
         if (cxx_record_decl->needsImplicitCopyConstructor())
           cxx_record_decl->setImplicitCopyConstructorIsDeleted();
         if (cxx_record_decl->needsImplicitCopyAssignment())
           cxx_record_decl->setImplicitCopyAssignmentIsDeleted();
       }
 
       if (!cxx_record_decl->isCompleteDefinition())
         cxx_record_decl->completeDefinition();
-      cxx_record_decl->setHasLoadedFieldsFromExternalStorage(true);
-      cxx_record_decl->setHasExternalLexicalStorage(false);
-      cxx_record_decl->setHasExternalVisibleStorage(false);
       return true;
     }
   }
 
   const clang::EnumType *enutype = qual_type->getAs<clang::EnumType>();
 
   if (!enutype)
     return false;
if (template_param_infos.IsValid()) {
 
     return CreateClassTemplateDecl(decl_ctx, owning_module, access_type,
                                    template_basename.c_str(), tag_decl_kind,
                                    template_param_infos);
   }
   return nullptr;
 }
 
-void TypeSystemClang::CompleteTagDecl(clang::TagDecl *decl) {
+void TypeSystemClang::CompleteTagDecl(const clang::TagDecl *decl) {
   SymbolFile *sym_file = GetSymbolFile();
   if (sym_file) {
-    CompilerType clang_type = GetTypeForDecl(decl);
+    CompilerType clang_type = GetTypeForDecl(const_cast<clang::TagDecl *>(decl));
     if (clang_type)
       sym_file->CompleteType(clang_type);
   }
 }
 
 void TypeSystemClang::CompleteObjCInterfaceDecl(
     clang::ObjCInterfaceDecl *decl) {
   SymbolFile *sym_file = GetSymbolFile();
   if (sym_file) {
     CompilerType clang_type = GetTypeForDecl(decl);
     if (clang_type)
       sym_file->CompleteType(clang_type);
   }
 }
 
+CompilerType TypeSystemClang::RedeclTagDecl(CompilerType ct) {
+  if (clang::TagDecl *d = ClangUtil::GetAsTagDecl(ct)) {
+    CompilerType res = CreateRecordType(d->getDeclContext()->getRedeclContext(), OptionalClangModuleID(),
+                            lldb::eAccessPublic, d->getName(), d->getTagKind(),
+                            eLanguageTypeC_plus_plus, nullptr);
+    clang::TagDecl *td = ClangUtil::GetAsTagDecl(res);
+    d->RedeclLink.setLatest(td);
+    return res;
+  }
+  return ct;
+}
+
 DWARFASTParser *TypeSystemClang::GetDWARFParser() {
   if (!m_dwarf_ast_parser_up)
     m_dwarf_ast_parser_up = std::make_unique<DWARFASTParserClang>(*this);
   return m_dwarf_ast_parser_up.get();
 }
 
 PDBASTParser *TypeSystemClang::GetPDBParser() {
   if (!m_pdb_ast_parser_up)