Preserve the owning module information from DWARF in the synthesized AST

[aprantl]

Summary

Types that came from a Clang module are nested in DW_TAG_module tags in DWARF. This patch recreates the Clang module hierarchy in LLDB and sets the owning module information accordingly. My primary motivation is to facilitate looking up per-module APINotes for individual declarations, but this likely also has other applications.

The fact that Clang modules are orthogonal to DeclContexts made the implementation in LLDB challenging. In the end I extended lldb_private::CompilerDeclContext to hold a pair of clang::DeclContext * and clang::Module * and made sure to use CompilerDeclContext everywhere in TypeSystemClang. To save memory for all non-Clang typesystems, the extra data for the owning module ID is hidden in the CompilerDeclContext's opaque pointer: If it isn't a raw clang::DeclContext pointer, it is an index into an array of Module ID + DeclContexts.

[aprantl]

Raphael just pointed out to me that since each DeclContext is also a Decl I don't need the machinery to hide the owning module ID in CompilerDeclContext and can just rely on the owning module ID stored before the Decl. That should make this patch a lot shorter.

[aprantl]

Here is an updated version the makes do without the CompilerDeclContext trickery. New is the m_payload field in lldb::Type. This is needed to convey the ownership information to typedefs, which are completed asynchronously.

[aprantl]

Now with a more comprehensive testcase and support for deferred completion via DWARFASTParserClang::ParseTypeFromClangModule().

This is good to review in earnest now!

[aprantl]

Factored out NFC changes into separate reviews.

[shafik]

lldb/include/lldb/Symbol/CompilerType.h

236 ↗

(On Diff #247948)

Why not convert this comment style?

lldb/source/Plugins/TypeSystem/Clang/TypeSystemClang.cpp

6997

Do we also need to make sure setModuleOwnershipKind(...) is consistent?

lldb/source/Plugins/TypeSystem/Clang/TypeSystemClang.h

72

So we at least expect the last bit to be available, do we expect to add more bits like ObjCClassBit, how many do we expect to have available?

I guess conversely what range do we expect id to fit in?

Does it make sense to document this?

[aprantl]

Add C++ tests.

[aprantl]

Address feedback from Shafik.

[labath]

This sounds like it could be useful though I can't say I really no much about modules.

I appreciate the effort in splitting this patch up, but I am wondering if we couldn't do this is one more step. Would it be possible to split the TypeSystem changes into a patch of its own (with some TypeSystemClang unit tests), and then have separate a patch where SymbolFileDWARF starts making use of this functionality? I think that would make easier to follow what's going in the two subsystems/plugins as well as reduce some of the noise caused by the interface changes.

lldb/source/Plugins/SymbolFile/DWARF/DWARFASTParserClang.cpp

223

Maybe spell out the type here? I have no idea what's the type of this..

lldb/source/Plugins/TypeSystem/Clang/TypeSystemClang.h

69–94

Maybe it would be simpler to have a struct full of bitfields and then just memcpy it to/from the uint32_t in the constructor/destructor?

lldb/test/Shell/SymbolFile/DWARF/Inputs/ModuleOwnership/A.h

25

How about a case where A.h defines a type which references a type from B.h (e.g. contains it as a member variable) ?

lldb/test/Shell/SymbolFile/DWARF/module-ownership.mm

2

I think this test will fail on windows. At least it did fail for me when I manually forced a windows target here (linux was fine though). I think the problem is that lldb does not know how to open an unlinked COFF file.

I think it would be better to hardcode a triple here instead of using %clang_host.

[aprantl]

Address feedback from Pavel.

[aprantl]

Hardcode triple in test

[aprantl]

lldb/test/Shell/SymbolFile/DWARF/module-ownership.mm

26

Note that the *Field* decl still comes from A, but its type (see above) doesn't.

[labath]

Thanks for splitting this up. This does make things much easier to understand.

I don't have any real objections to this, but I have some "thoughts" in inline comments.

lldb/source/Plugins/SymbolFile/DWARF/DWARFASTParserClang.cpp

214–215

Did you deliberately not include the payload as an argument to the Type constructor? I can understand not wanting to add a extra argument to that constructor, but OTOH, having this in the constructor would make it harder to forget setting the module id when creating a type elsewhere. (And it's always nice to have immutable members if it is possible.)

220–236

I don't know how feasible this is, but it has occurred to me that this is basically repeating the same structure iteration that would be done as a part of the import a couple of lines above. If would be nice if the importer could somehow set this property automatically.

[teemperor]

lldb/include/lldb/Symbol/Type.h

198 ↗

(On Diff #247948)

Remove reference?

lldb/source/Plugins/SymbolFile/DWARF/DWARFASTParserClang.h

92

Maybe I'm missing something but this is DIEToModuleMap but it seems the value in this map is actually a module ID and not a module map?

[aprantl]

lldb/source/Plugins/SymbolFile/DWARF/DWARFASTParserClang.cpp

223

Yeah, this should either be a Visitor or in our ASTImporterDelegate in lldb.

[aprantl]

lldb/source/Plugins/SymbolFile/DWARF/DWARFASTParserClang.cpp

223

Another comment from an in-person discussion with @teemperor: We probably should not attach owning module information in the scratch/expression context.

[aprantl]

Address review feedback from Raphael.

[aprantl]

Rolled https://reviews.llvm.org/D75626 back into this patch, addressed more review feedback.

[teemperor]

LGTM

lldb/source/Plugins/TypeSystem/Clang/TypeSystemClang.h

319

This probably deserves a hint that this can only be called on an TypeSystemClang that has created its own clang::ASTContext (otherwise the ASTSource could be anything and this asserts) as all other functions in TypeSystemClang otherwise work in both modes. Should not be called when TypeSystemClang adopted an existing clang::ASTContext or something like that.

[aprantl]

lldb/source/Plugins/TypeSystem/Clang/TypeSystemClang.cpp

1254

@teemperor This is the place where we need Module * instead of a const Module *.

[teemperor]

Sorry for resetting this back from accepted :)

lldb/source/Plugins/TypeSystem/Clang/TypeSystemClang.cpp

1260

Why is that done? The module should already have the correct name when it is returned from findOrCreateModule (where we pass the same name and then create the new module with that name).

If this line is removed, do we still need D75561?

[aprantl]

(Made const_cast explicit to illustrate mismatch.)

However, I still need to de-constify in order to create the module.

[aprantl]

lldb/source/Plugins/TypeSystem/Clang/TypeSystemClang.cpp

1256

here ^^

1260

Good catch! This is a leftover from my first version of the patch where I created the Module object manually without instantiating a clang::ModuleMap.

[teemperor]

Ok, no more complaints from my side. LGTM

lldb/source/Plugins/TypeSystem/Clang/TypeSystemClang.cpp

1256

True, it seems we can't fix that. Also the ExternalASTSource uses non-const modules, so I guess that's fine then.

lldb/source/Plugins/ExpressionParser/Clang/ClangASTImporter.cpp

 #include "clang/AST/DeclCXX.h"
 #include "clang/AST/DeclObjC.h"
 #include "clang/Sema/Lookup.h"
 #include "clang/Sema/Sema.h"
 #include "llvm/Support/raw_ostream.h"
 
 #include "Plugins/ExpressionParser/Clang/ClangASTImporter.h"
 #include "Plugins/ExpressionParser/Clang/ClangASTMetadata.h"
+#include "Plugins/ExpressionParser/Clang/ClangExternalASTSourceCallbacks.h"
 #include "Plugins/ExpressionParser/Clang/ClangUtil.h"
 #include "Plugins/TypeSystem/Clang/TypeSystemClang.h"
 
 #include <memory>
 
 using namespace lldb_private;
 using namespace clang;
 
static void MaybeCompleteReturnType(ClangASTImporter &importer,
 
   clang::RecordDecl *rd = return_type->getPointeeType()->getAsRecordDecl();
   if (!rd)
     return;
 
   importer.CompleteTagDecl(rd);
 }
 
+/// Recreate a module with its parents in \p to_source and return its id.
+static ModuleID RemapModule(ModuleID from_id,
+                            ClangExternalASTSourceCallbacks &from_source,
+                            ClangExternalASTSourceCallbacks &to_source) {
+  if (!from_id.HasValue())
+    return {};
+  clang::Module *module = from_source.getModule(*from_id.get());
+  ModuleID parent = RemapModule(from_source.GetIDForModule(module->Parent),
+                                from_source, to_source);
+  TypeSystemClang &to_ts = to_source.GetTypeSystem();
+  return to_ts.GetOrCreateClangModule(module->Name, parent, module->IsFramework,
+                                      module->IsExplicit);
+}
+
 void ClangASTImporter::ASTImporterDelegate::Imported(clang::Decl *from,
                                                      clang::Decl *to) {
   Log *log(lldb_private::GetLogIfAllCategoriesSet(LIBLLDB_LOG_EXPRESSIONS));
 
   // Some decls shouldn't be tracked here because they were not created by
   // copying 'from' to 'to'. Just exit early for those.
   if (m_decls_to_ignore.find(to) != m_decls_to_ignore.end())
     return clang::ASTImporter::Imported(from, to);
 
+  // Transfer module ownership information.
+  auto *from_source = llvm::dyn_cast_or_null<ClangExternalASTSourceCallbacks>(
+      getFromContext().getExternalSource());
+  // Can also be a ClangASTSourceProxy.
+  auto *to_source = llvm::dyn_cast_or_null<ClangExternalASTSourceCallbacks>(
+      getToContext().getExternalSource());
+  if (from_source && to_source) {
+    ModuleID from_id(from->getOwningModuleID());
+    ModuleID to_id = RemapModule(from_id, *from_source, *to_source);
+    TypeSystemClang &to_ts = to_source->GetTypeSystem();
+    to_ts.SetOwningModule(to, to_id);
+  }
+  
   lldb::user_id_t user_id = LLDB_INVALID_UID;
   ClangASTMetadata *metadata = m_master.GetDeclMetadata(from);
   if (metadata)
     user_id = metadata->GetUserID();
 
   if (log) {
     if (NamedDecl *from_named_decl = dyn_cast<clang::NamedDecl>(from)) {
       std::string name_string;

lldb/source/Plugins/ExpressionParser/Clang/ClangExternalASTSourceCallbacks.h

 //===-- ClangExternalASTSourceCallbacks.h -----------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLDB_SOURCE_PLUGINS_EXPRESSIONPARSER_CLANG_CLANGEXTERNALASTSOURCECALLBACKS_H
 #define LLDB_SOURCE_PLUGINS_EXPRESSIONPARSER_CLANG_CLANGEXTERNALASTSOURCECALLBACKS_H
 
 #include "Plugins/TypeSystem/Clang/TypeSystemClang.h"
 #include "clang/Basic/Module.h"
 
 namespace lldb_private {
 
-class TypeSystemClang;
-
 class ClangExternalASTSourceCallbacks : public clang::ExternalASTSource {
   /// LLVM RTTI support.
   static char ID;
 
 public:
   /// LLVM RTTI support.
   bool isA(const void *ClassID) const override { return ClassID == &ID; }
   static bool classof(const clang::ExternalASTSource *s) { return s->isA(&ID); }
public:
 
   TypeSystemClang &GetTypeSystem() const { return m_ast; }
 
   /// Module-related methods.
   /// \{
   llvm::Optional<clang::ASTSourceDescriptor>
   getSourceDescriptor(unsigned ID) override;
   clang::Module *getModule(unsigned ID) override;
-  unsigned RegisterModule(clang::Module *module);
-  unsigned GetIDForModule(clang::Module *module);
+  ModuleID RegisterModule(clang::Module *module);
+  ModuleID GetIDForModule(clang::Module *module);
   /// \}
 private:
   TypeSystemClang &m_ast;
   std::vector<clang::Module *> m_modules;
   llvm::DenseMap<clang::Module *, unsigned> m_ids;
 };
 
 } // namespace lldb_private
 
 #endif // LLDB_SOURCE_PLUGINS_EXPRESSIONPARSER_CLANG_CLANGEXTERNALASTSOURCECALLBACKS_H

lldb/source/Plugins/ExpressionParser/Clang/ClangExternalASTSourceCallbacks.cpp

void ClangExternalASTSourceCallbacks::FindExternalLexicalDecls(
   if (decl_ctx) {
     clang::TagDecl *tag_decl = llvm::dyn_cast<clang::TagDecl>(
         const_cast<clang::DeclContext *>(decl_ctx));
     if (tag_decl)
       CompleteType(tag_decl);
   }
 }
 
-unsigned ClangExternalASTSourceCallbacks::RegisterModule(clang::Module *module) {
+ModuleID ClangExternalASTSourceCallbacks::RegisterModule(clang::Module *module) {
   m_modules.push_back(module);
   unsigned id = m_modules.size();
   m_ids.insert({module, id});
-  return id;
+  return ModuleID(id);
 }
 
 llvm::Optional<clang::ASTSourceDescriptor>
 ClangExternalASTSourceCallbacks::getSourceDescriptor(unsigned id) {
   if (clang::Module *module = getModule(id))
     return {*module};
   return {};
 }
 
 clang::Module *ClangExternalASTSourceCallbacks::getModule(unsigned id) {
   if (id && id <= m_modules.size())
     return m_modules[id - 1];
   return nullptr;
 }
 
-unsigned ClangExternalASTSourceCallbacks::GetIDForModule(clang::Module *module) {
-  return m_ids[module];
+ModuleID ClangExternalASTSourceCallbacks::GetIDForModule(clang::Module *module) {
+  return ModuleID(m_ids[module]);
 }

lldb/source/Plugins/SymbolFile/DWARF/DWARFASTParserClang.h

protected:
   class DelayedAddObjCClassProperty;
   typedef std::vector<DelayedAddObjCClassProperty> DelayedPropertyList;
 
   typedef llvm::SmallPtrSet<const DWARFDebugInfoEntry *, 4> DIEPointerSet;
   typedef llvm::DenseMap<const DWARFDebugInfoEntry *, clang::DeclContext *>
       DIEToDeclContextMap;
   typedef std::multimap<const clang::DeclContext *, const DWARFDIE>
       DeclContextToDIEMap;
+  typedef llvm::DenseMap<const DWARFDebugInfoEntry *, lldb_private::ModuleID>
+      DIEToModuleMap;
   typedef llvm::DenseMap<const DWARFDebugInfoEntry *, clang::Decl *>
       DIEToDeclMap;
   typedef llvm::DenseMap<const clang::Decl *, DIEPointerSet> DeclToDIEMap;
 
   lldb_private::TypeSystemClang &m_ast;
   DIEToDeclMap m_die_to_decl;
   DeclToDIEMap m_decl_to_die;
   DIEToDeclContextMap m_die_to_decl_ctx;
   DeclContextToDIEMap m_decl_ctx_to_die;
+  DIEToModuleMap m_die_to_module;
   std::unique_ptr<lldb_private::ClangASTImporter> m_clang_ast_importer_up;
   /// @}
 
   clang::DeclContext *GetDeclContextForBlock(const DWARFDIE &die);
 
   clang::BlockDecl *ResolveBlockDIE(const DWARFDIE &die);
 
   clang::NamespaceDecl *ResolveNamespaceDIE(const DWARFDIE &die);
protected:
   lldb_private::Type *GetTypeForDIE(const DWARFDIE &die);
 
   clang::Decl *GetClangDeclForDIE(const DWARFDIE &die);
 
   clang::DeclContext *GetClangDeclContextForDIE(const DWARFDIE &die);
 
   clang::DeclContext *GetClangDeclContextContainingDIE(const DWARFDIE &die,
                                                        DWARFDIE *decl_ctx_die);
+  lldb_private::ModuleID GetOwningModuleID(const DWARFDIE &die);
 
   bool CopyUniqueClassMethodTypes(const DWARFDIE &src_class_die,
                                   const DWARFDIE &dst_class_die,
                                   lldb_private::Type *class_type,
                                   std::vector<DWARFDIE> &failures);
 
   clang::DeclContext *GetCachedClangDeclContextForDIE(const DWARFDIE &die);
 

lldb/source/Plugins/SymbolFile/DWARF/DWARFASTParserClang.cpp

TypeSP DWARFASTParserClang::ParseTypeFromClangModule(const SymbolContext &sc,
   // in only the forward declaration to be visible.
   if (pcm_type.IsDefined())
     GetClangASTImporter().RequireCompleteType(ClangUtil::GetQualType(type));
 
   SymbolFileDWARF *dwarf = die.GetDWARF();
   TypeSP type_sp(new Type(
       die.GetID(), dwarf, pcm_type_sp->GetName(), pcm_type_sp->GetByteSize(),
       nullptr, LLDB_INVALID_UID, Type::eEncodingInvalid,
-      &pcm_type_sp->GetDeclaration(), type, Type::ResolveState::Forward));
+      &pcm_type_sp->GetDeclaration(), type, Type::ResolveState::Forward,
+      TypePayloadClang(GetOwningModuleID(die))));
 
   dwarf->GetTypeList().Insert(type_sp);
   dwarf->GetDIEToType()[die.GetDIE()] = type_sp.get();
   clang::TagDecl *tag_decl = TypeSystemClang::GetAsTagDecl(type);
-  if (tag_decl)
+  if (tag_decl) {
     LinkDeclContextToDIE(tag_decl, die);
-  else {
+  } else {
     clang::DeclContext *defn_decl_ctx = GetCachedClangDeclContextForDIE(die);
     if (defn_decl_ctx)
       LinkDeclContextToDIE(defn_decl_ctx, die);
   }
 
   return type_sp;
 }
 
 static void CompleteExternalTagDeclType(TypeSystemClang &ast,
                                         ClangASTImporter &ast_importer,
                                         clang::DeclContext *decl_ctx,
                                         DWARFDIE die,
                                         const char *type_name_cstr) {
   auto *tag_decl_ctx = clang::dyn_cast<clang::TagDecl>(decl_ctx);
   if (!tag_decl_ctx)
     return;
 
   // If this type was not imported from an external AST, there's nothing to do.
   CompilerType type = ast.GetTypeForDecl(tag_decl_ctx);
   if (!type || !ast_importer.CanImport(type))
     return;
 
   auto qual_type = ClangUtil::GetQualType(type);
if (cu_language == eLanguageTypeObjC ||
         }
       }
     }
   }
 
   type_sp = std::make_shared<Type>(
       die.GetID(), dwarf, attrs.name, attrs.byte_size, nullptr,
       dwarf->GetUID(attrs.type.Reference()), encoding_data_type, &attrs.decl,
-      clang_type, resolve_state);
+      clang_type, resolve_state, TypePayloadClang(GetOwningModuleID(die)));
 
   dwarf->GetDIEToType()[die.GetDIE()] = type_sp.get();
   return type_sp;
 }
 
 TypeSP DWARFASTParserClang::ParseEnum(const SymbolContext &sc,
                                       const DWARFDIE &die,
                                       ParsedDWARFTypeAttributes &attrs) {
if (!enumerator_clang_type) {
             "", DW_ATE_signed, *attrs.byte_size * 8);
       } else {
         enumerator_clang_type = m_ast.GetBasicType(eBasicTypeInt);
       }
     }
 
     clang_type = m_ast.CreateEnumerationType(
         attrs.name.GetCString(), GetClangDeclContextContainingDIE(die, nullptr),
-        ModuleID(), attrs.decl, enumerator_clang_type, attrs.is_scoped_enum);
+        GetOwningModuleID(die), attrs.decl, enumerator_clang_type,
+        attrs.is_scoped_enum);
   } else {
     enumerator_clang_type = m_ast.GetEnumerationIntegerType(clang_type);
   }
 
   LinkDeclContextToDIE(TypeSystemClang::GetDeclContextForType(clang_type), die);
 
   type_sp = std::make_shared<Type>(
       die.GetID(), dwarf, attrs.name, attrs.byte_size, nullptr,
       dwarf->GetUID(attrs.type.Reference()), Type::eEncodingIsUID, &attrs.decl,
-      clang_type, Type::ResolveState::Forward);
+      clang_type, Type::ResolveState::Forward,
+      TypePayloadClang(GetOwningModuleID(die)));
 
   if (TypeSystemClang::StartTagDeclarationDefinition(clang_type)) {
     if (die.HasChildren()) {
       bool is_signed = false;
       enumerator_clang_type.IsIntegerType(is_signed);
       ParseChildEnumerators(clang_type, is_signed,
                             type_sp->GetByteSize().getValueOr(0), die);
     }
if (!type_handled) {
         }
       }
 
       if (!function_decl) {
         // We just have a function that isn't part of a class
         function_decl = m_ast.CreateFunctionDeclaration(
             ignore_containing_context ? m_ast.GetTranslationUnitDecl()
                                       : containing_decl_ctx,
-            ModuleID(), attrs.name.GetCString(), clang_type, attrs.storage,
+            GetOwningModuleID(die),
+            attrs.name.GetCString(), clang_type, attrs.storage,
             attrs.is_inline);
 
         if (has_template_params) {
           TypeSystemClang::TemplateParameterInfos template_param_infos;
           ParseTemplateParameterInfos(die, template_param_infos);
           template_function_decl = m_ast.CreateFunctionDeclaration(
               ignore_containing_context ? m_ast.GetTranslationUnitDecl()
                                         : containing_decl_ctx,
-              ModuleID(), attrs.name.GetCString(), clang_type, attrs.storage,
-              attrs.is_inline);
+              GetOwningModuleID(die), attrs.name.GetCString(), clang_type,
+              attrs.storage, attrs.is_inline);
           clang::FunctionTemplateDecl *func_template_decl =
               m_ast.CreateFunctionTemplateDecl(
-                  containing_decl_ctx, ModuleID(), template_function_decl,
-                  attrs.name.GetCString(), template_param_infos);
+                  containing_decl_ctx, GetOwningModuleID(die),
+                  template_function_decl, attrs.name.GetCString(),
+                  template_param_infos);
           m_ast.CreateFunctionTemplateSpecializationInfo(
               function_decl, func_template_decl, template_param_infos);
         }
 
         lldbassert(function_decl);
 
         if (function_decl) {
           LinkDeclContextToDIE(function_decl, die);
if (!clang_type) {
     metadata.SetUserID(die.GetID());
     metadata.SetIsDynamicCXXType(dwarf->ClassOrStructIsVirtual(die));
 
     if (attrs.name.GetStringRef().contains('<')) {
       TypeSystemClang::TemplateParameterInfos template_param_infos;
       if (ParseTemplateParameterInfos(die, template_param_infos)) {
         clang::ClassTemplateDecl *class_template_decl =
             m_ast.ParseClassTemplateDecl(
-                decl_ctx, ModuleID(), attrs.accessibility,
+                decl_ctx, GetOwningModuleID(die), attrs.accessibility,
                 attrs.name.GetCString(), tag_decl_kind, template_param_infos);
         if (!class_template_decl) {
           if (log) {
             dwarf->GetObjectFile()->GetModule()->LogMessage(
                 log,
                 "SymbolFileDWARF(%p) - 0x%8.8x: %s type \"%s\" "
                 "clang::ClassTemplateDecl failed to return a decl.",
                 static_cast<void *>(this), die.GetOffset(),
                 DW_TAG_value_to_name(tag), attrs.name.GetCString());
           }
           return TypeSP();
         }
 
         clang::ClassTemplateSpecializationDecl *class_specialization_decl =
             m_ast.CreateClassTemplateSpecializationDecl(
-                decl_ctx, ModuleID(), class_template_decl, tag_decl_kind,
-                template_param_infos);
+                decl_ctx, GetOwningModuleID(die), class_template_decl,
+                tag_decl_kind, template_param_infos);
         clang_type = m_ast.CreateClassTemplateSpecializationType(
             class_specialization_decl);
         clang_type_was_created = true;
 
         m_ast.SetMetadata(class_template_decl, metadata);
         m_ast.SetMetadata(class_specialization_decl, metadata);
       }
     }
 
     if (!clang_type_was_created) {
       clang_type_was_created = true;
       clang_type = m_ast.CreateRecordType(
-          decl_ctx, ModuleID(), attrs.accessibility, attrs.name.GetCString(),
-          tag_decl_kind, attrs.class_language, &metadata,
-          attrs.exports_symbols);
+          decl_ctx, GetOwningModuleID(die), attrs.accessibility,
+          attrs.name.GetCString(), tag_decl_kind, attrs.class_language,
+          &metadata, attrs.exports_symbols);
     }
   }
 
   // Store a forward declaration to this class type in case any
   // parameters in any class methods need it for the clang types for
   // function prototypes.
   LinkDeclContextToDIE(m_ast.GetDeclContextForType(clang_type), die);
   type_sp = std::make_shared<Type>(
       die.GetID(), dwarf, attrs.name, attrs.byte_size, nullptr,
       LLDB_INVALID_UID, Type::eEncodingIsUID, &attrs.decl, clang_type,
       Type::ResolveState::Forward,
-      TypePayloadClang(attrs.is_complete_objc_class));
+      TypePayloadClang(ModuleID(), attrs.is_complete_objc_class));
 
   // Add our type to the unique type map so we don't end up creating many
   // copies of the same type over and over in the ASTContext for our
   // module
   unique_ast_entry_up->m_type_sp = type_sp;
   unique_ast_entry_up->m_die = die;
   unique_ast_entry_up->m_declaration = unique_decl;
   unique_ast_entry_up->m_byte_size = attrs.byte_size.getValueOr(0);
case DW_TAG_formal_parameter: {
 
         if (!skip) {
           Type *type = die.ResolveTypeUID(param_type_die_form.Reference());
           if (type) {
             function_param_types.push_back(type->GetForwardCompilerType());
 
             clang::ParmVarDecl *param_var_decl =
                 m_ast.CreateParameterDeclaration(
-                    containing_decl_ctx, ModuleID(), name,
+                    containing_decl_ctx, GetOwningModuleID(die), name,
                     type->GetForwardCompilerType(), storage);
             assert(param_var_decl);
             function_param_decls.push_back(param_var_decl);
 
             m_ast.SetMetadataAsUserID(param_var_decl, die.GetID());
           }
         }
       }
case DW_TAG_formal_parameter: {
     SymbolFileDWARF *dwarf = die.GetDWARF();
     Type *type = GetTypeForDIE(die);
     if (dwarf && type) {
       const char *name = die.GetName();
       clang::DeclContext *decl_context =
           TypeSystemClang::DeclContextGetAsDeclContext(
               dwarf->GetDeclContextContainingUID(die.GetID()));
       decl = m_ast.CreateVariableDeclaration(
-          decl_context, ModuleID(), name,
+          decl_context, GetOwningModuleID(die), name,
           ClangUtil::GetQualType(type->GetForwardCompilerType()));
     }
     break;
   }
   case DW_TAG_imported_declaration: {
     SymbolFileDWARF *dwarf = die.GetDWARF();
     DWARFDIE imported_uid = die.GetAttributeValueAsReferenceDIE(DW_AT_import);
     if (imported_uid) {
if (die) {
     if (decl_ctx) {
       LinkDeclContextToDIE(decl_ctx, die);
       return decl_ctx;
     }
   }
   return nullptr;
 }
 
+ModuleID DWARFASTParserClang::GetOwningModuleID(const DWARFDIE &die) {
+  if (!die.IsValid())
+    return {};
+
+  for (DWARFDIE parent = die.GetParent(); parent.IsValid();
+       parent = parent.GetParent()) {
+    const dw_tag_t tag = parent.Tag();
+    if (tag == DW_TAG_module) {
+      DWARFDIE module_die = parent;
+      auto it = m_die_to_module.find(module_die.GetDIE());
+      if (it != m_die_to_module.end())
+        return it->second;
+      const char *name = module_die.GetAttributeValueAsString(DW_AT_name, 0);
+      if (!name)
+        return {};
+
+      ModuleID id =
+          m_ast.GetOrCreateClangModule(name, GetOwningModuleID(module_die));
+      m_die_to_module.insert({module_die.GetDIE(), id});
+      return id;
+    }
+  }
+  return {};
+}
+
 static bool IsSubroutine(const DWARFDIE &die) {
   switch (die.Tag()) {
   case DW_TAG_subprogram:
   case DW_TAG_inlined_subroutine:
     return true;
   default:
     return false;
   }
clang::BlockDecl *DWARFASTParserClang::ResolveBlockDIE(const DWARFDIE &die) {
   if (die && die.Tag() == DW_TAG_lexical_block) {
     clang::BlockDecl *decl =
         llvm::cast_or_null<clang::BlockDecl>(m_die_to_decl_ctx[die.GetDIE()]);
 
     if (!decl) {
       DWARFDIE decl_context_die;
       clang::DeclContext *decl_context =
           GetClangDeclContextContainingDIE(die, &decl_context_die);
-      decl = m_ast.CreateBlockDeclaration(decl_context, ModuleID());
+      decl = m_ast.CreateBlockDeclaration(decl_context, GetOwningModuleID(die));
 
       if (decl)
         LinkDeclContextToDIE((clang::DeclContext *)decl, die);
     }
 
     return decl;
   }
   return nullptr;

lldb/source/Plugins/TypeSystem/Clang/TypeSystemClang.h

struct ModuleID {
   explicit ModuleID(unsigned id) : id(id) {}
   bool HasValue() const { return id; }
   llvm::Optional<unsigned> get() const {
     return id ? llvm::Optional<unsigned>(id) : llvm::None;
   }
 };
   
 /// The implementation of lldb::Type's m_payload field for TypeSystemClang.
 class TypePayloadClang {
   /// The Layout is as follows:
   /// \verbatim
   /// bit 0..30 ... Owning Module ID.
   /// bit 31 ...... IsCompleteObjCClass.
   /// \endverbatim
   uint32_t m_payload;
 public:
   TypePayloadClang() = default;
-  explicit TypePayloadClang(bool is_complete_objc_class, ModuleID owning_module);
+  explicit TypePayloadClang(ModuleID owning_module,
+                            bool is_complete_objc_class = false);
   explicit TypePayloadClang(uint32_t opaque_payload) : m_payload(opaque_payload) {}
   operator uint32_t() { return m_payload; }
 
   static constexpr unsigned ObjCClassBit = 1 << 31;
   bool IsCompleteObjCClass() { return Flags(m_payload).Test(ObjCClassBit); }
   void SetIsCompleteObjCClass(bool is_complete_objc_class) {
     m_payload = is_complete_objc_class ? Flags(m_payload).Set(ObjCClassBit)
                                        : Flags(m_payload).Clear(ObjCClassBit);
   }
   ModuleID GetOwningModuleID() {
     return ModuleID(Flags(m_payload).Clear(ObjCClassBit));
   }
   void SetOwningModuleID(ModuleID id);
   /// \}
 };
   
 /// A TypeSystem implementation based on Clang.
 ///
 /// This class uses a single clang::ASTContext as the backend for storing
 /// its types and declarations. Every clang::ASTContext should also just have
 /// a single associated TypeSystemClang instance that manages it.
 ///
 /// The clang::ASTContext instance can either be created by TypeSystemClang
public:
   ConvertAccessTypeToAccessSpecifier(lldb::AccessType access);
 
   static clang::AccessSpecifier
   UnifyAccessSpecifiers(clang::AccessSpecifier lhs, clang::AccessSpecifier rhs);
 
   static uint32_t GetNumBaseClasses(const clang::CXXRecordDecl *cxx_record_decl,
                                     bool omit_empty_base_classes);
 
-  /// Synthesize a clang::Module and return its ID or 0.
-  unsigned GetOrCreateClangModule(llvm::StringRef name, unsigned parent,
+  /// Synthesize a clang::Module and return its ID or a default-constructed ID.
+  ModuleID GetOrCreateClangModule(llvm::StringRef name, ModuleID parent,
                                   bool is_framework = false,
                                   bool is_explicit = false);
 
   CompilerType CreateRecordType(clang::DeclContext *decl_ctx,
                                 ModuleID owning_module,
                                 lldb::AccessType access_type,
                                 llvm::StringRef name, int kind,
                                 lldb::LanguageType language,

lldb/source/Plugins/TypeSystem/Clang/TypeSystemClang.cpp

static ClangASTMap &GetASTMap() {
   static ClangASTMap *g_map_ptr = nullptr;
   static llvm::once_flag g_once_flag;
   llvm::call_once(g_once_flag, []() {
     g_map_ptr = new ClangASTMap(); // leaked on purpose to avoid spins
   });
   return *g_map_ptr;
 }
 
-TypePayloadClang::TypePayloadClang(bool is_complete_objc_class,
-                                   ModuleID owning_module)
+TypePayloadClang::TypePayloadClang(ModuleID owning_module,
+                                   bool is_complete_objc_class)
     : m_payload(owning_module.id) {
   SetIsCompleteObjCClass(is_complete_objc_class);
 }
 
 void TypePayloadClang::SetOwningModuleID(ModuleID id) {
   assert(id.id < ObjCClassBit);
   bool is_complete = IsCompleteObjCClass();
   m_payload = id.id;
void TypeSystemClang::SetOwningModule(clang::Decl *decl, ModuleID owning_module) {
   if (!decl || !owning_module.HasValue())
     return;
 
   decl->setFromASTFile();
   decl->setOwningModuleID(*owning_module.get());
   decl->setModuleOwnershipKind(clang::Decl::ModuleOwnershipKind::Visible);
 }
 
-unsigned TypeSystemClang::GetOrCreateClangModule(llvm::StringRef name,
-                                                 unsigned parent,
+ModuleID TypeSystemClang::GetOrCreateClangModule(llvm::StringRef name,
+                                                 ModuleID parent,
                                                  bool is_framework,
                                                  bool is_explicit) {
   // Get the external AST source which holds the modules.
   auto *ast_source = llvm::dyn_cast_or_null<ClangExternalASTSourceCallbacks>(
       getASTContext().getExternalSource());
   assert(ast_source && "external ast source was lost");
   if (!ast_source)
-    return 0;
+    return {};
 
   // Lazily initialize the module map.
   if (!m_header_search_up) {
     auto HSOpts = std::make_shared<clang::HeaderSearchOptions>();
     m_header_search_up = std::make_unique<clang::HeaderSearch>(
         HSOpts, *m_source_manager_up, *m_diagnostics_engine_up,
         *m_language_options_up, m_target_info_up.get());
     m_module_map_up = std::make_unique<clang::ModuleMap>(
         *m_source_manager_up, *m_diagnostics_engine_up, *m_language_options_up,
         m_target_info_up.get(), *m_header_search_up);
   }
 
   // Get or create the module context.
   bool created;
   clang::Module *module;
-  auto parent_desc = ast_source->getSourceDescriptor(parent);
+  auto parent_desc = ast_source->getSourceDescriptor(parent.id);
   std::tie(module, created) = m_module_map_up->findOrCreateModule(
       name, parent_desc ? parent_desc->getModuleOrNull() : nullptr,
       is_framework, is_explicit);
   if (!created)
     return ast_source->GetIDForModule(module);
 
   module->Name = name.str();
   return ast_source->RegisterModule(module);
 }
 
 CompilerType TypeSystemClang::CreateRecordType(clang::DeclContext *decl_ctx,
                                                ModuleID owning_module,
                                                AccessType access_type,
                                                llvm::StringRef name, int kind,
                                                LanguageType language,
TypeSystemClang::GetAsObjCInterfaceDecl(const CompilerType &type) {
   return nullptr;
 }
 
 clang::FieldDecl *TypeSystemClang::AddFieldToRecordType(
     const CompilerType &type, llvm::StringRef name,
     const CompilerType &field_clang_type, AccessType access,
     uint32_t bitfield_bit_size) {
   if (!type.IsValid() || !field_clang_type.IsValid())
     return nullptr;
   TypeSystemClang *ast =
       llvm::dyn_cast_or_null<TypeSystemClang>(type.GetTypeSystem());
   if (!ast)
     return nullptr;
   clang::ASTContext &clang_ast = ast->getASTContext();
   clang::IdentifierInfo *ident = nullptr;
   if (!name.empty())
     ident = &clang_ast.Idents.get(name);

lldb/test/Shell/SymbolFile/DWARF/Inputs/ModuleOwnership/A.h

+#include "B.h"                                                 // -*- ObjC -*-
+
+typedef int Typedef;
+
+struct TopLevelStruct {
+  int a;
+};
+
+typedef struct Struct_s {
+  int a;
+} Struct;
+
+struct Nested {
+  StructB fromb;
+};
+
+typedef enum Enum_e {
+  a = 0
+} Enum;
+
+@interface SomeClass {}
+@end
+
+template<typename T> struct Template { T field; };
+extern template struct Template<int>;
+
+namespace Namespace {
+template<typename T> struct InNamespace { T field; };
+extern template struct InNamespace<int>;
+}

lldb/test/Shell/SymbolFile/DWARF/Inputs/ModuleOwnership/B.h

+typedef struct {
+  int b;
+} StructB;
+
+namespace Namespace {
+template<typename T> struct AlsoInNamespace { T field; };
+ extern template struct AlsoInNamespace<int>;
+}

lldb/test/Shell/SymbolFile/DWARF/Inputs/ModuleOwnership/module.modulemap

+module A {
+  header "A.h"
+  module B {
+    header "B.h"
+  }
+}

lldb/test/Shell/SymbolFile/DWARF/lit.local.cfg

-config.suffixes = ['.cpp', '.m', '.s', '.test', '.ll']
+config.suffixes = ['.cpp', '.m', '.mm', '.s', '.test', '.ll']

lldb/test/Shell/SymbolFile/DWARF/module-ownership.mm

+// RUN: %clang --target=x86_64-apple-macosx -g -gmodules \
+// RUN:    -fmodules -fmodules-cache-path=%t.cache \
+// RUN:    -c -o %t.o %s -I%S/Inputs
+// RUN: lldb-test symbols -dump-clang-ast %t.o | FileCheck %s
+// Verify that the owning module information from DWARF is preserved in the AST. 
+
+@import A;
+
+Typedef t1;
+// CHECK-DAG: TypedefDecl {{.*}} imported in A Typedef
+
+TopLevelStruct s1;
+// CHECK-DAG: CXXRecordDecl {{.*}} imported in A struct TopLevelStruct
+// CHECK-DAG: -FieldDecl {{.*}} in A a 'int'
+
+Struct s2;
+// CHECK-DAG: CXXRecordDecl {{.*}} imported in A struct
+
+StructB s3;
+// CHECK-DAG: CXXRecordDecl {{.*}} imported in A.B struct
+// CHECK-DAG: -FieldDecl {{.*}} in A.B b 'int'
+
+Nested s4;
+// CHECK-DAG: CXXRecordDecl {{.*}} imported in A struct Nested
+// CHECK-DAG: -FieldDecl {{.*}} in A fromb 'StructB'
+
+Enum e1;
+// CHECK-DAG: EnumDecl {{.*}} imported in A {{.*}} Enum_e
+// FIXME: : -EnumConstantDecl {{.*}} imported in A a
+
+SomeClass *obj1;
+// CHECK-DAG: ObjCInterfaceDecl {{.*}} imported in A SomeClass
+
+Template<int> t2;
+// CHECK-DAG: ClassTemplateSpecializationDecl {{.*}} imported in A struct Template
+
+Namespace::InNamespace<int> t3;
+// CHECK-DAG: ClassTemplateSpecializationDecl {{.*}} imported in A struct InNamespace
+
+Namespace::AlsoInNamespace<int> t4;
+// CHECK-DAG: ClassTemplateSpecializationDecl {{.*}} imported in A.B struct AlsoInNamespace