DWZ 03/06: Main functionality

[jankratochvil]

Summary

This is the kitchen sink. It also covers separate DWZ common files.

Patch is using (it contains the implementation of) DWARFDataExtractor::OffsetData from D32167 which is not yet accepted and where I agree with the review by Pavel Labath.

All DWZ patches are also applied in: git clone -b dwz git://git.jankratochvil.net/lldb

[labath]

I am only commenting on the c++14 stuff. I think all of the things you need here are available to us already through various llvm utilities. See inline comments for details.

For the "meat" of this patchset, I think clayborg is the only one who can do a proper review of that.

source/Plugins/SymbolFile/DWARF/DWARFDebugInfo.h

74–75

Is llvm::sys::RWMutex what you need here?

source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.h

468

Have you looked at llvm::DenseSet? It already supports heterogenous lookup( find_as(...)). It can also be more efficient than unordered_set.

486

Will std::atomic<size_t> not work?

[jankratochvil]

Going to update the patch.

source/Plugins/SymbolFile/DWARF/DWARFDebugInfo.h

74–75

Yes, llvm::sys::RWMutex should do the trick, thanks.

source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.h

468

llvm::DenseSet should do the trick, thanks.

486

Yes, std::atomic_size_t would work but it currently has no benefit (rather a needless overhead) there without read/write locks, the atomicity makes sense if it is read-locked only. With llvm::sys::RWMutex now I will use the atomic type.

[jankratochvil]

source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.h

468

"The iterators in a DenseMap are invalidated whenever an insertion occurs, unlike map." - there is now std::unordered_set<DWZCommonFile, ...> and a pointer to it by DWZCommonFile *m_dwz_common_file;. So thanks for the hint but I do not find DenseSet to be applicable here.

[jankratochvil]

ping: Are there more comments how to make this whole patchset accepted upstream? Thanks.

[clayborg]

Too many changes to give the whole thing the OK to checkin. I will worry about each one as an individual patch. 02/11 next.

[labath]

I think the most controversial issue here is the testing... davide may have some thoughts on that.

source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.cpp

3931

the references are superfluous here (this just caught my eye because it's at the bottom of the patch).

[jankratochvil]

It is now reworked without FileOffset and the remapping to unique DIE offsets for each DW_TAG_partial_unit inclusion by DW_TAG_imported_unit, as discussed in: https://lists.llvm.org/pipermail/lldb-commits/Week-of-Mon-20180409/040324.html

[jankratochvil]

Replaced by D96240.

include/lldb/Utility/ConstString.h

public:
   /// any other values as a byte size for the entire string pool.
   ///
   /// @return
   ///     The number of bytes that the global string pool occupies
   ///     in memory.
   //------------------------------------------------------------------
   static size_t StaticMemorySize();
 
+  class Hasher {
+  public:
+    size_t operator()(const ConstString &key) const {
+      // https://stackoverflow.com/questions/7666509/hash-function-for-string
+      // C++17: std::string_view
+      size_t hash = 5381;
+      for (const char *p = key.m_string; *p; ++p)
+        hash = hash * 33 + static_cast<uint8_t>(*p);
+      return hash;
+    }
+  };
+
 protected:
   //------------------------------------------------------------------
   // Member variables
   //------------------------------------------------------------------
   const char *m_string;
 };
 
 //------------------------------------------------------------------

include/lldb/Utility/FileSpec.h

static void EnumerateDirectory(llvm::StringRef dir_path,
                                  bool find_other,
                                  EnumerateDirectoryCallbackType callback,
                                  void *callback_baton);
 
   typedef std::function<EnumerateDirectoryResult(
       llvm::sys::fs::file_type file_type, const FileSpec &spec)>
       DirectoryCallback;
 
+  class Hasher {
+  public:
+    size_t operator()(const FileSpec &key) const {
+      return (ConstString::Hasher()(key.m_directory) << 16)
+          ^ ConstString::Hasher()(key.m_filename) ^ key.m_is_resolved;
+    }
+  };
+
 protected:
   //------------------------------------------------------------------
   // Convenience method for setting the file without changing the style.
   //------------------------------------------------------------------
   void SetFile(llvm::StringRef path, bool resolve_path);
 
   //------------------------------------------------------------------
   // Member variables

source/Plugins/SymbolFile/DWARF/DWARFDataExtractor.h

public:
   uint64_t GetDWARFInitialLength(lldb::offset_t *offset_ptr) const;
 
   dw_offset_t GetDWARFOffset(lldb::offset_t *offset_ptr) const;
 
   size_t GetDWARFSizeofInitialLength() const { return m_is_dwarf64 ? 12 : 4; }
   size_t GetDWARFSizeOfOffset() const { return m_is_dwarf64 ? 8 : 4; }
   bool IsDWARF64() const { return m_is_dwarf64; }
 
+  //------------------------------------------------------------------
+  /// Slide the data in the buffer so that access to the data will
+  /// start at offset \a offset.
+  ///
+  /// This is currently used to provide access to the .debug_types
+  /// section and pretend it starts at at offset of the size of the
+  /// .debug_info section. This allows a minimally invasive change to
+  /// add support for .debug_types by allowing all DIEs to have unique
+  /// offsets and thus allowing no code to change in the DWARF parser.
+  /// Modifying the offsets in the .debug_types doesn't affect
+  /// anything because since all info in the .debug_types is type unit
+  /// relative and no types within a type unit can refer to any DIEs
+  /// outside of the type unit without using DW_AT_signature. It also
+  /// sets us up to move to DWARF5 where there is no .debug_types
+  /// section as compile units and type units are in the .debug_info.
+  //------------------------------------------------------------------
+  void OffsetData(lldb::offset_t offset)
+  {
+    if (GetByteSize())
+      m_start -= offset;
+  }
+
 protected:
   mutable bool m_is_dwarf64;
 };
 }
 
 #endif // liblldb_DWARFDataExtractor_h_

source/Plugins/SymbolFile/DWARF/DWARFDebugAranges.h

 public:
   typedef RangeToDIE::Entry Range;
   typedef std::vector<RangeToDIE::Entry> RangeColl;
 
   DWARFDebugAranges();
 
   void Clear() { m_aranges.Clear(); }
 
-  bool Extract(const lldb_private::DWARFDataExtractor &debug_aranges_data);
+  bool Extract(const lldb_private::DWARFDataExtractor &debug_aranges_data,
+      dw_offset_t dwz_offset);
 
   bool Generate(SymbolFileDWARF *dwarf2Data);
 
   // Use append range multiple times and then call sort
   void AppendRange(dw_offset_t cu_offset, dw_addr_t low_pc, dw_addr_t high_pc);
 
   void Sort(bool minimize);
 

source/Plugins/SymbolFile/DWARF/DWARFDebugAranges.cpp

void operator()(const DWARFDebugArangeSet &set) {
     count += set.NumDescriptors();
   }
   uint32_t &count;
 };
 
 //----------------------------------------------------------------------
 // Extract
 //----------------------------------------------------------------------
-bool DWARFDebugAranges::Extract(const DWARFDataExtractor &debug_aranges_data) {
+bool DWARFDebugAranges::Extract(
+    const DWARFDataExtractor &debug_aranges_data, dw_offset_t dwz_offset) {
   if (debug_aranges_data.ValidOffset(0)) {
     lldb::offset_t offset = 0;
 
     DWARFDebugArangeSet set;
     Range range;
     while (set.Extract(debug_aranges_data, &offset)) {
       const uint32_t num_descriptors = set.NumDescriptors();
       if (num_descriptors > 0) {
-        const dw_offset_t cu_offset = set.GetCompileUnitDIEOffset();
+        const dw_offset_t cu_offset = set.GetCompileUnitDIEOffset()
+            + dwz_offset;
 
         for (uint32_t i = 0; i < num_descriptors; ++i) {
           const DWARFDebugArangeSet::Descriptor &descriptor =
               set.GetDescriptorRef(i);
           m_aranges.Append(RangeToDIE::Entry(descriptor.address,
                                              descriptor.length, cu_offset));
         }
       }

source/Plugins/SymbolFile/DWARF/DWARFDebugInfo.h

public:
   DWARFDebugInfo();
   void SetDwarfData(SymbolFileDWARF *dwarf2Data);
 
   size_t GetNumCompileUnits();
   bool ContainsCompileUnit(const DWARFUnit *cu) const;
   DWARFUnit *GetCompileUnitAtIndex(uint32_t idx);
   DWARFUnit *GetCompileUnit(dw_offset_t cu_offset, uint32_t *idx_ptr = NULL);
   DWARFUnit *GetCompileUnitContainingDIEOffset(dw_offset_t die_offset);
+  // Used only by DWARFFormValue::Reference().
+  DWARFUnit *GetCompileUnitContainingDIEOffsetInFile(dw_offset_t die_offset);
   DWARFUnit *GetCompileUnit(const DIERef &die_ref);
   DWARFDIE GetDIEForDIEOffset(dw_offset_t die_offset);
   DWARFDIE GetDIE(const DIERef &die_ref);
 
   enum {
     eDumpFlag_Verbose = (1 << 0),  // Verbose dumping
     eDumpFlag_ShowForm = (1 << 1), // Show the DW_form type
     eDumpFlag_ShowAncestors =
         (1 << 2) // Show all parent DIEs when dumping single DIEs
   };
 
   DWARFDebugAranges &GetCompileUnitAranges();
 
 protected:
   static bool OffsetLessThanCompileUnitOffset(dw_offset_t offset,
                                               const DWARFUnitSP &cu_sp);
 
   typedef std::vector<DWARFUnitSP> CompileUnitColl;
 
+  DWARFDebugInfo *DWZRedirect(dw_offset_t offset);
+  DWARFUnit *GetCompileUnitInFile(dw_offset_t cu_offset, uint32_t *idx_ptr);
+
   //----------------------------------------------------------------------
   // Member variables
   //----------------------------------------------------------------------
   SymbolFileDWARF *m_dwarf2Data;
   CompileUnitColl m_compile_units;
   std::unique_ptr<DWARFDebugAranges>
       m_cu_aranges_ap; // A quick address to compile unit table
 
 private:
   // All parsing needs to be done partially any managed by this class as
   // accessors are called.
   void ParseCompileUnitHeadersIfNeeded();
 
   DISALLOW_COPY_AND_ASSIGN(DWARFDebugInfo);
 };
 
 #endif // SymbolFileDWARF_DWARFDebugInfo_h_

source/Plugins/SymbolFile/DWARF/DWARFDebugInfo.cpp

 //===-- DWARFDebugInfo.cpp --------------------------------------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #include "SymbolFileDWARF.h"
 
 #include <algorithm>
 #include <set>
 
+#include "lldb/Core/Module.h"
 #include "lldb/Host/PosixApi.h"
 #include "lldb/Symbol/ObjectFile.h"
+#include "lldb/Utility/LLDBAssert.h"
 #include "lldb/Utility/RegularExpression.h"
 #include "lldb/Utility/Stream.h"
 #include "lldb/Core/Module.h"
 
 #include "DWARFCompileUnit.h"
 #include "DWARFDebugAranges.h"
 #include "DWARFDebugAranges.h"
 #include "DWARFDebugInfo.h"
if (m_cu_aranges_ap.get() == NULL && m_dwarf2Data) {
     const DWARFDataExtractor &debug_aranges_data =
         m_dwarf2Data->get_debug_aranges_data();
     if (debug_aranges_data.GetByteSize() > 0) {
       if (log)
         log->Printf(
             "DWARFDebugInfo::GetCompileUnitAranges() for \"%s\" from "
             ".debug_aranges",
             m_dwarf2Data->GetObjectFile()->GetFileSpec().GetPath().c_str());
-      m_cu_aranges_ap->Extract(debug_aranges_data);
+      dw_offset_t dwz_offset = 0;
+      lldbassert(!m_dwarf2Data->GetIsDWZ());
+      SymbolFileDWARF *dwz_symbol_file = m_dwarf2Data->GetDWZSymbolFile();
+      if (dwz_symbol_file) {
+        const DWARFDataExtractor &dwz_debug_info_data =
+            dwz_symbol_file->get_debug_info_data();
+        auto dwz_debug_info_size = dwz_debug_info_data.GetByteSize();
+        dwz_offset += dwz_debug_info_size;
+      }
+      m_cu_aranges_ap->Extract(debug_aranges_data, dwz_offset);
     }
 
     // Make a list of all CUs represented by the arange data in the file.
     std::set<dw_offset_t> cus_with_data;
     for (size_t n = 0; n < m_cu_aranges_ap.get()->GetNumRanges(); n++) {
       dw_offset_t offset = m_cu_aranges_ap.get()->OffsetAtIndex(n);
       if (offset != DW_INVALID_OFFSET)
         cus_with_data.insert(offset);
DWARFDebugAranges &DWARFDebugInfo::GetCompileUnitAranges() {
   }
   return *m_cu_aranges_ap.get();
 }
 
 void DWARFDebugInfo::ParseCompileUnitHeadersIfNeeded() {
   if (m_compile_units.empty()) {
     if (m_dwarf2Data != NULL) {
       lldb::offset_t offset = 0;
+      // All CUs in a base symbol file are shifted after its DWZ file (if exists).
+      SymbolFileDWARF *dwz_symbol_file = m_dwarf2Data->GetDWZSymbolFile();
+      if (dwz_symbol_file) {
+        const DWARFDataExtractor &dwz_debug_info_data =
+            dwz_symbol_file->get_debug_info_data();
+        offset += dwz_debug_info_data.GetByteSize();
+      }
+
       DWARFUnitSP cu_sp;
       const auto &debug_info_data = m_dwarf2Data->get_debug_info_data();
       while ((cu_sp = DWARFCompileUnit::Extract(m_dwarf2Data, debug_info_data,
                                                 &offset))) {
         m_compile_units.push_back(cu_sp);
 
         offset = cu_sp->GetNextCompileUnitOffset();
       }
bool DWARFDebugInfo::ContainsCompileUnit(const DWARFUnit *cu) const {
   return false;
 }
 
 bool DWARFDebugInfo::OffsetLessThanCompileUnitOffset(
     dw_offset_t offset, const DWARFUnitSP &cu_sp) {
   return offset < cu_sp->GetOffset();
 }
 
+DWARFDebugInfo *DWARFDebugInfo::DWZRedirect(dw_offset_t offset) {
+  if (m_dwarf2Data->GetIsDWZ()) {
+      m_dwarf2Data->GetObjectFile()->GetModule()->ReportError(
+          "Internal LLDB error - unable to resolve offset 0x%8.8" PRIx32
+          " as DWZ common file has been passed, change the callers to use:"
+          " DWARFDIE::GetDWARF() -> DWARFDIE::GetMainDWARF()",
+          offset);
+    return this;
+  }
+  SymbolFileDWARF *dwz_symbol_file = m_dwarf2Data->GetDWZSymbolFile();
+  if (!dwz_symbol_file)
+    return this;
+  const DWARFDataExtractor &dwz_debug_info_data =
+      dwz_symbol_file->get_debug_info_data();
+  auto dwz_debug_info_size = dwz_debug_info_data.GetByteSize();
+  if (offset < dwz_debug_info_size)
+    return dwz_symbol_file->DebugInfo();
+  return this;
+}
+
 DWARFUnit *DWARFDebugInfo::GetCompileUnit(dw_offset_t cu_offset,
-                                                 uint32_t *idx_ptr) {
+  return DWZRedirect(cu_offset)->GetCompileUnitInFile(cu_offset, idx_ptr);
+}
+
+DWARFUnit *DWARFDebugInfo::GetCompileUnitInFile(dw_offset_t cu_offset,
+                                                uint32_t *idx_ptr) {
   DWARFUnitSP cu_sp;
   uint32_t cu_idx = DW_INVALID_INDEX;
   if (cu_offset != DW_INVALID_OFFSET) {
     ParseCompileUnitHeadersIfNeeded();
 
     // Watch out for single compile unit executable as they are pretty common
     const size_t num_cus = m_compile_units.size();
     if (num_cus == 1) {
DWARFUnit *DWARFDebugInfo::GetCompileUnit(const DIERef &die_ref) {
   if (die_ref.cu_offset == DW_INVALID_OFFSET)
     return GetCompileUnitContainingDIEOffset(die_ref.die_offset);
   else
     return GetCompileUnit(die_ref.cu_offset);
 }
 
 DWARFUnit *
 DWARFDebugInfo::GetCompileUnitContainingDIEOffset(dw_offset_t die_offset) {
+  return DWZRedirect(die_offset)->GetCompileUnitContainingDIEOffsetInFile(
+      die_offset);
+}
+
+DWARFUnit *
+DWARFDebugInfo::GetCompileUnitContainingDIEOffsetInFile(
+    dw_offset_t die_offset) {
   ParseCompileUnitHeadersIfNeeded();
 
   DWARFUnitSP cu_sp;
 
   // Watch out for single compile unit executable as they are pretty common
   const size_t num_cus = m_compile_units.size();
   if (num_cus == 1) {
     if (m_compile_units[0]->ContainsDIEOffset(die_offset))

source/Plugins/SymbolFile/DWARF/DWARFDebugInfoEntry.cpp

for (i = 0; i < numAttributes; ++i) {
             form_size = cu->GetAddressByteSize();
             break;
           case DW_FORM_ref_addr:
             if (cu->GetVersion() <= 2)
               form_size = cu->GetAddressByteSize();
             else
               form_size = cu->IsDWARF64() ? 8 : 4;
             break;
+          case DW_FORM_GNU_ref_alt:
+            form_size = cu->IsDWARF64() ? 8 : 4;
+            break;
 
           // 0 sized form
           case DW_FORM_flag_present:
             form_size = 0;
             break;
 
           // 1 byte values
           case DW_FORM_data1:
for (i = 0; i < numAttributes; ++i) {
             break;
 
           case DW_FORM_indirect:
             form_is_indirect = true;
             form = debug_info_data.GetULEB128(&offset);
             break;
 
           case DW_FORM_strp:
+          case DW_FORM_GNU_strp_alt:
           case DW_FORM_sec_offset:
             if (cu->IsDWARF64())
               debug_info_data.GetU64(&offset);
             else
               debug_info_data.GetU32(&offset);
             break;
 
           default:
if (abbr_idx) {
                 form_size = cu->GetAddressByteSize();
                 break;
               case DW_FORM_ref_addr:
                 if (cu->GetVersion() <= 2)
                   form_size = cu->GetAddressByteSize();
                 else
                   form_size = cu->IsDWARF64() ? 8 : 4;
                 break;
+              case DW_FORM_GNU_ref_alt:
+                form_size = cu->IsDWARF64() ? 8 : 4;
+                break;
 
               // 0 sized form
               case DW_FORM_flag_present:
                 form_size = 0;
                 break;
 
               // 1 byte values
               case DW_FORM_data1:
if (abbr_idx) {
                 break;
 
               case DW_FORM_indirect:
                 form = debug_info_data.GetULEB128(&offset);
                 form_is_indirect = true;
                 break;
 
               case DW_FORM_strp:
+              case DW_FORM_GNU_strp_alt:
               case DW_FORM_sec_offset:
                 if (cu->IsDWARF64())
                   debug_info_data.GetU64(&offset);
                 else
                   debug_info_data.GetU32(&offset);
                 break;
 
               default:

source/Plugins/SymbolFile/DWARF/DWARFFormValue.cpp

 //===-- DWARFFormValue.cpp --------------------------------------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #include <assert.h>
 
 #include "lldb/Core/dwarf.h"
+#include "lldb/Utility/LLDBAssert.h"
 #include "lldb/Utility/Stream.h"
+#include "lldb/Symbol/ObjectFile.h"
+#include "lldb/Core/Module.h"
 
 #include "DWARFUnit.h"
 #include "DWARFFormValue.h"
+#include "DWARFDebugInfo.h"
 
 class DWARFUnit;
 
 using namespace lldb_private;
 
 static uint8_t g_form_sizes_addr4[] = {
     0, // 0x00 unused
     4, // 0x01 DW_FORM_addr
case DW_FORM_data1:
       break;
     case DW_FORM_flag:
       m_value.value.uval = data.GetU8(offset_ptr);
       break;
     case DW_FORM_sdata:
       m_value.value.sval = data.GetSLEB128(offset_ptr);
       break;
     case DW_FORM_strp:
+    case DW_FORM_GNU_strp_alt:
       assert(m_cu);
       m_value.value.uval =
           data.GetMaxU64(offset_ptr, DWARFUnit::IsDWARF64(m_cu) ? 8 : 4);
       break;
     //  case DW_FORM_APPLE_db_str:
     case DW_FORM_udata:
       m_value.value.uval = data.GetULEB128(offset_ptr);
       break;
case DW_FORM_ref4:
       m_value.value.uval = data.GetU32(offset_ptr);
       break;
     case DW_FORM_ref8:
       m_value.value.uval = data.GetU64(offset_ptr);
       break;
     case DW_FORM_ref_udata:
       m_value.value.uval = data.GetULEB128(offset_ptr);
       break;
+    case DW_FORM_GNU_ref_alt:
+      assert(m_cu);
+      ref_addr_size = m_cu->IsDWARF64() ? 8 : 4;
+      m_value.value.uval = data.GetMaxU64(offset_ptr, ref_addr_size);
+      break;
     case DW_FORM_indirect:
       m_form = data.GetULEB128(offset_ptr);
       indirect = true;
       break;
 
     case DW_FORM_sec_offset:
       assert(m_cu);
       m_value.value.uval =
assert(cu); // CU must be valid for DW_FORM_ref_addr objects or we will get
                 // this wrong
     if (cu->GetVersion() <= 2)
       ref_addr_size = cu->GetAddressByteSize();
     else
       ref_addr_size = cu->IsDWARF64() ? 8 : 4;
     *offset_ptr += ref_addr_size;
     return true;
 
+  case DW_FORM_GNU_ref_alt:
+    assert(cu); // CU must be valid for DW_FORM_GNU_ref_alt objects or we will get
+                // this wrong
+    ref_addr_size = cu->IsDWARF64() ? 8 : 4;
+    *offset_ptr += ref_addr_size;
+    return true;
+
   // 0 bytes values (implied from DW_FORM)
   case DW_FORM_flag_present:
     return true;
 
   // 1 byte values
   case DW_FORM_data1:
   case DW_FORM_flag:
   case DW_FORM_ref1:
     *offset_ptr += 1;
     return true;
 
   // 2 byte values
   case DW_FORM_data2:
   case DW_FORM_ref2:
     *offset_ptr += 2;
     return true;
 
   // 32 bit for DWARF 32, 64 for DWARF 64
   case DW_FORM_sec_offset:
   case DW_FORM_strp:
+  case DW_FORM_GNU_strp_alt:
     assert(cu);
     *offset_ptr += (cu->IsDWARF64() ? 8 : 4);
     return true;
 
   // 4 byte values
   case DW_FORM_data4:
   case DW_FORM_ref4:
     *offset_ptr += 4;
case DW_FORM_ref4:
     cu_relative_offset = true;
     break;
   case DW_FORM_ref8:
     cu_relative_offset = true;
     break;
   case DW_FORM_ref_udata:
     cu_relative_offset = true;
     break;
+  case DW_FORM_GNU_ref_alt: {
+    assert(m_cu); // CU must be valid for DW_FORM_GNU_ref_alt objects or we will
+                  // get this wrong
+    s.Address(uvalue, 4 * 2); // 4 for DWARF32, 8 for DWARF64, but we don't
+                              // support DWARF64 yet
+    break;
+  }
 
   // All DW_FORM_indirect attributes should be resolved prior to calling this
   // function
   case DW_FORM_indirect:
     s.PutCString("DW_FORM_indirect");
     break;
   case DW_FORM_flag_present:
     break;
if (!symbol_file)
       return nullptr;
 
     uint32_t index_size = m_cu->IsDWARF64() ? 8 : 4;
     lldb::offset_t offset = m_value.value.uval * index_size;
     dw_offset_t str_offset =
         symbol_file->get_debug_str_offsets_data().GetMaxU64(&offset,
                                                             index_size);
     return symbol_file->get_debug_str_data().PeekCStr(str_offset);
+  } else if (m_form == DW_FORM_GNU_strp_alt) {
+    SymbolFileDWARF *dwz_symbol_file = symbol_file->GetDWZSymbolFile();
+    if (!dwz_symbol_file) {
+      symbol_file->GetObjectFile()->GetModule()->ReportError(
+         "String reference to DWZ offset 0x%8.8" PRIx64
+         " is in a file without associated DWZ common file",
+         m_value.value.uval);
+      return nullptr;
+    }
+    return dwz_symbol_file->get_debug_str_data().PeekCStr(m_value.value.uval);
   }
   return nullptr;
 }
 
 dw_addr_t DWARFFormValue::Address() const {
   SymbolFileDWARF *symbol_file = m_cu->GetSymbolFileDWARF();
 
   if (m_form == DW_FORM_addr)
     return Unsigned();
 
   assert(m_cu);
   assert(m_form == DW_FORM_GNU_addr_index);
 
   if (!symbol_file)
     return 0;
 
   uint32_t index_size = m_cu->GetAddressByteSize();
   dw_offset_t addr_base = m_cu->GetAddrBase();
   lldb::offset_t offset = addr_base + m_value.value.uval * index_size;
   return symbol_file->get_debug_addr_data().GetMaxU64(&offset, index_size);
 }
 
 uint64_t DWARFFormValue::Reference() const {
   uint64_t die_offset = m_value.value.uval;
+  assert(m_cu);
+  SymbolFileDWARF *cu_symbol_file = m_cu->GetSymbolFileDWARF();
+  bool is_dwz = cu_symbol_file->GetIsDWZ();
   switch (m_form) {
   case DW_FORM_ref1:
   case DW_FORM_ref2:
   case DW_FORM_ref4:
   case DW_FORM_ref8:
   case DW_FORM_ref_udata:
-    assert(m_cu); // CU must be valid for DW_FORM_ref forms that are compile
-                  // unit relative or we will get this wrong
     die_offset += m_cu->GetOffset();
+    // The offset adjustment is already appropriate inside this CU.
+    return die_offset;
+
+  case DW_FORM_GNU_ref_alt:
+    if (is_dwz)
+      cu_symbol_file->GetObjectFile()->GetModule()->ReportError(
+         "DW_FORM_GNU_ref_alt to DWZ offset 0x%8.8" PRIx64
+         " is in a DWZ common file itself",
+         die_offset);
+    else if (!cu_symbol_file->GetDWZSymbolFile()) {
+      cu_symbol_file->GetObjectFile()->GetModule()->ReportError(
+         "DW_FORM_GNU_ref_alt to DWZ offset 0x%8.8" PRIx64
+         " has no DWZ common file",
+         die_offset);
+      return DW_INVALID_OFFSET;
+    }
+    is_dwz = true;
     break;
 
+  case DW_FORM_ref_addr:
   default:
     break;
   }
 
+  if (!is_dwz) {
+    lldbassert(!cu_symbol_file->GetIsDWZ());
+    SymbolFileDWARF *dwz_symbol_file = cu_symbol_file->GetDWZSymbolFile();
+    if (dwz_symbol_file) {
+      // All CUs in a base symbol file are shifted after its DWZ file.
+      const DWARFDataExtractor &dwz_debug_info_data =
+          dwz_symbol_file->get_debug_info_data();
+      die_offset += dwz_debug_info_data.GetByteSize();
+    }
+  }
+
+  if (!m_cu->ContainsDIEOffset(die_offset)) {
+    SymbolFileDWARF *target_symbol_file = cu_symbol_file;
+    if (is_dwz && !target_symbol_file->GetIsDWZ()) {
+      target_symbol_file = cu_symbol_file->GetDWZSymbolFile();
+      if (!target_symbol_file) {
+        cu_symbol_file->GetObjectFile()->GetModule()->ReportError(
+           "There is Reference to DWZ offset 0x%8.8" PRIx64
+           " but the DWZ common file is missing",
+           die_offset);
+        target_symbol_file = cu_symbol_file;
+      }
+    }
+    lldbassert(is_dwz == target_symbol_file->GetIsDWZ());
+    DWARFDebugInfo *target_debuginfo = target_symbol_file->DebugInfo();
+    // FIXME: This is expensive for files containing no DWZ.
+    // But there is no way to easily find in advance if this file has been
+    // DWZ-processed (in the case of DWARF <=4).
+    DWARFUnit *target_cu =
+      target_debuginfo->GetCompileUnitContainingDIEOffsetInFile(die_offset);
+    if (target_cu)
+      target_cu->SetMainCU(const_cast<DWARFUnit *>(m_cu->GetMainCU()));
+  }
+
   return die_offset;
 }
 
 uint64_t DWARFFormValue::Reference(dw_offset_t base_offset) const {
   uint64_t die_offset = m_value.value.uval;
   switch (m_form) {
   case DW_FORM_ref1:
   case DW_FORM_ref2:
case DW_FORM_exprloc: {
     return memcmp(a_value.BlockData(), b_value.BlockData(), a_value.Unsigned());
   } break;
 
   case DW_FORM_ref1:
   case DW_FORM_ref2:
   case DW_FORM_ref4:
   case DW_FORM_ref8:
   case DW_FORM_ref_udata: {
+  case DW_FORM_GNU_ref_alt:
     uint64_t a = a_value.Reference();
     uint64_t b = b_value.Reference();
     if (a < b)
       return -1;
     if (a > b)
       return 1;
     return 0;
   }
switch (form) {
     case DW_FORM_ref_udata:
     case DW_FORM_indirect:
     case DW_FORM_sec_offset:
     case DW_FORM_exprloc:
     case DW_FORM_flag_present:
     case DW_FORM_ref_sig8:
     case DW_FORM_GNU_str_index:
     case DW_FORM_GNU_addr_index:
+    case DW_FORM_GNU_ref_alt:
+    case DW_FORM_GNU_strp_alt:
       return true;
     default:
       break;
   }
   return false;
 }

source/Plugins/SymbolFile/DWARF/DWARFUnit.h

public:
   dw_offset_t GetBaseObjOffset() const;
 
   die_iterator_range dies() {
     ExtractDIEsIfNeeded();
     return die_iterator_range(m_die_array.begin(), m_die_array.end());
   }
 
   // DW_TAG_compile_unit with DW_TAG_imported_unit for this DW_TAG_partial_unit.
-  DWARFUnit *GetMainCU() const {
-    return const_cast<DWARFUnit *>(this);
-  }
+  DWARFUnit *GetMainCU() { return m_main_cu; }
+  const DWARFUnit *GetMainCU() const { return m_main_cu; }
+  void SetMainCU(DWARFUnit *new_main_cu);
+  void DWZDeleteSymbolFile(SymbolFileDWARF *symbol_file);
 
 protected:
   DWARFUnit(SymbolFileDWARF *dwarf);
 
   SymbolFileDWARF *m_dwarf = nullptr;
   std::unique_ptr<SymbolFileDWARFDwo> m_dwo_symbol_file;
   const DWARFAbbreviationDeclarationSet *m_abbrevs = nullptr;
   void *m_user_data = nullptr;
protected:
   lldb::LanguageType m_language_type = lldb::eLanguageTypeUnknown;
   bool m_is_dwarf64 = false;
   lldb_private::LazyBool m_is_optimized = lldb_private::eLazyBoolCalculate;
   dw_addr_t m_addr_base = 0;   // Value of DW_AT_addr_base
   dw_addr_t m_ranges_base = 0; // Value of DW_AT_ranges_base
   // If this is a dwo compile unit this is the offset of the base compile unit
   // in the main object file
   dw_offset_t m_base_obj_offset = DW_INVALID_OFFSET;
+  // DW_TAG_compile_unit with DW_TAG_imported_unit for this DW_TAG_partial_unit.
+  std::atomic<DWARFUnit *> m_main_cu;
+  // It is used for restoring valid m_main_cu if its current SymbolFileDWARF
+  // gets removed. Then choose any DWARFUnit from any other file which also
+  // has DW_TAG_imported_unit for this DW_TAG_partial_unit.
+  std::unordered_map<SymbolFileDWARF *, DWARFUnit *> m_main_cu_map;
+  llvm::sys::RWMutex m_main_cu_map_mutex;
 
   // Offset of the initial length field.
   dw_offset_t m_offset;
 
 private:
   void ParseProducerInfo();
   void ExtractDIEsRWLocked();
   void ClearDIEsRWLocked();

source/Plugins/SymbolFile/DWARF/DWARFUnit.cpp

 
 using namespace lldb;
 using namespace lldb_private;
 using namespace std;
 
 extern int g_verbose;
 
 DWARFUnit::DWARFUnit(SymbolFileDWARF *dwarf)
-    : m_dwarf(dwarf), m_cancel_scopes(false) {}
+    : m_dwarf(dwarf), m_cancel_scopes(false), m_main_cu(this) {}
 
 DWARFUnit::~DWARFUnit() {}
 
 //----------------------------------------------------------------------
 // Parses first DIE of a compile unit.
 //----------------------------------------------------------------------
 void DWARFUnit::ExtractUnitDIEIfNeeded() {
   {
void DWARFUnit::BuildAddressRangeTable(SymbolFileDWARF *dwarf,
   // order to produce a compile unit level set of address ranges that is
   // accurate.
 
   size_t num_debug_aranges = debug_aranges->GetNumRanges();
 
   // First get the compile unit DIE only and check if it has a DW_AT_ranges
   const DWARFDebugInfoEntry *die = GetUnitDIEPtrOnly();
 
+  // DWZ partial units never contain PC.
+  if (die && die->Tag() == DW_TAG_partial_unit)
+    return;
+
   const dw_offset_t cu_offset = GetOffset();
   if (die) {
     DWARFRangeList ranges;
     const size_t num_ranges =
         die->GetAttributeAddressRanges(dwarf, this, ranges, false);
     if (num_ranges > 0) {
       // This compile unit has DW_AT_ranges, assume this is correct if it is
       // present since clang no longer makes .debug_aranges by default and it
if (ContainsDIEOffset(die_offset)) {
       if (pos != end) {
         if (die_offset == (*pos).GetOffset())
           return DWARFDIE(this, &(*pos));
       }
     } else {
       // Don't specify the compile unit offset as we don't know it because the
       // DIE belongs to
       // a different compile unit in the same symbol file.
-      return GetMainCU()->m_dwarf->DebugInfo()->GetDIEForDIEOffset(die_offset);
+      DWARFDIE die =
+          GetMainCU()->m_dwarf->DebugInfo()->GetDIEForDIEOffset(die_offset);
+      if (die)
+        die.GetCU()->SetMainCU(GetMainCU());
+      return die;
     }
   }
   m_dwarf->GetObjectFile()->GetModule()->ReportError(
      "CU 0x%8.8" PRIx32 " does not contain DIE 0x%8.8" PRIx32,
      m_offset, die_offset);
   return DWARFDIE(); // Not found
 }
 

 }
 
 SymbolFileDWARFDwo *DWARFUnit::GetDwoSymbolFile() const {
   return m_dwo_symbol_file.get();
 }
 
 dw_offset_t DWARFUnit::GetBaseObjOffset() const { return m_base_obj_offset; }
 
+void DWARFUnit::SetMainCU(DWARFUnit *new_main_cu) {
+  if (new_main_cu == this)
+    return;
+  if (m_main_cu == new_main_cu)
+    return;
+  if (m_main_cu != this
+      && (*m_main_cu).GetSymbolFileDWARF() == new_main_cu->GetSymbolFileDWARF())
+    return;
+  const DWARFDebugInfoEntry *die = GetUnitDIEPtrOnly();
+  if (!die || die->Tag() != DW_TAG_partial_unit)
+    return;
+  bool found;
+  {
+    llvm::sys::ScopedReader lock(m_main_cu_map_mutex);
+    found = m_main_cu_map.find(new_main_cu->GetSymbolFileDWARF())
+        != m_main_cu_map.cend();
+  }
+  if (found)
+    lldbassert(m_main_cu != this);
+  else {
+    llvm::sys::ScopedWriter lock(m_main_cu_map_mutex);
+    DWARFUnit *&ref =
+        m_main_cu_map[new_main_cu->GetSymbolFileDWARF()];
+    if (ref)
+      lldbassert(m_main_cu != this);
+    else {
+      ref = new_main_cu;
+      if (m_main_cu == this)
+        m_main_cu = new_main_cu;
+    }
+  }
+}
+
+void DWARFUnit::DWZDeleteSymbolFile(SymbolFileDWARF *symbol_file) {
+  llvm::sys::ScopedWriter lock(m_main_cu_map_mutex);
+  if (!m_main_cu_map.erase(symbol_file))
+    return;
+  if ((*m_main_cu).GetSymbolFileDWARF() != symbol_file)
+    return;
+  if (!m_main_cu_map.empty())
+    m_main_cu = m_main_cu_map.cbegin()->second;
+  else
+    m_main_cu = this;
+}
+
 const DWARFDebugAranges &DWARFUnit::GetFunctionAranges() {
   if (m_func_aranges_ap.get() == NULL) {
     m_func_aranges_ap.reset(new DWARFDebugAranges());
     Log *log(LogChannelDWARF::GetLogIfAll(DWARF_LOG_DEBUG_ARANGES));
 
     if (log) {
       m_dwarf->GetObjectFile()->GetModule()->LogMessage(
           log,

source/Plugins/SymbolFile/DWARF/ManualDWARFIndex.h

struct IndexSet {
     NameToDIE objc_class_selectors;
     NameToDIE globals;
     NameToDIE types;
     NameToDIE namespaces;
   };
   void Index();
   void IndexUnit(DWARFUnit &unit, IndexSet &set);
 
-  static void
+  void
   IndexUnitImpl(DWARFUnit &unit, const lldb::LanguageType cu_language,
                 const DWARFFormValue::FixedFormSizes &fixed_form_sizes,
                 const dw_offset_t cu_offset, IndexSet &set);
 
   /// Non-null value means we haven't built the index yet.
   DWARFDebugInfo *m_debug_info;
   /// Which dwarf units should we skip while building the index.
   llvm::DenseSet<dw_offset_t> m_units_to_avoid;
 
   IndexSet m_set;
 };
 } // namespace lldb_private
 
 #endif // LLDB_MANUALDWARFINDEX_H

source/Plugins/SymbolFile/DWARF/ManualDWARFIndex.cpp

TaskPool::RunTasks([&]() { finalize_fn(&IndexSet::function_basenames); },
                      [&]() { finalize_fn(&IndexSet::function_selectors); },
                      [&]() { finalize_fn(&IndexSet::objc_class_selectors); },
                      [&]() { finalize_fn(&IndexSet::globals); },
                      [&]() { finalize_fn(&IndexSet::types); },
                      [&]() { finalize_fn(&IndexSet::namespaces); });
 }
 
 void ManualDWARFIndex::IndexUnit(DWARFUnit &unit, IndexSet &set) {
+  // Are we called for DW_TAG_compile_unit (contrary to DW_TAG_partial_unit)?
+  if (unit.GetMainCU() == &unit) {
+    // DWZ DW_TAG_partial_unit will get indexed by DW_AT_import
+    // from its DW_TAG_compile_unit (possibly transitively).
+    // GetUnitDIEPtrOnly() is too expensive.
+    if (unit.GetSymbolFileDWARF()->GetIsDWZ())
+      return;
+    if (unit.GetUnitDIEOnly().Tag() == DW_TAG_partial_unit)
+      return;
+  }
+
   Log *log = LogChannelDWARF::GetLogIfAll(DWARF_LOG_LOOKUPS);
 
   if (log) {
     m_module.LogMessage(
         log, "ManualDWARFIndex::IndexUnit for compile unit at .debug_info[0x%8.8x]",
         unit.GetOffset());
   }
 
for (const DWARFDebugInfoEntry &die : unit.dies()) {
     case DW_TAG_string_type:
     case DW_TAG_structure_type:
     case DW_TAG_subprogram:
     case DW_TAG_subroutine_type:
     case DW_TAG_typedef:
     case DW_TAG_union_type:
     case DW_TAG_unspecified_type:
     case DW_TAG_variable:
+    case DW_TAG_imported_unit:
       break;
 
     default:
       continue;
     }
 
     DWARFAttributes attributes;
     const char *name = NULL;
     const char *mangled_cstr = NULL;
     bool is_declaration = false;
     // bool is_artificial = false;
     bool has_address = false;
     bool has_location_or_const_value = false;
     bool is_global_or_static_variable = false;
 
-    DWARFFormValue specification_die_form;
+    DWARFFormValue specification_die_form, import_die_form;
     const size_t num_attributes =
         die.GetAttributes(&unit, fixed_form_sizes, attributes);
     if (num_attributes > 0) {
       for (uint32_t i = 0; i < num_attributes; ++i) {
         dw_attr_t attr = attributes.AttributeAtIndex(i);
         DWARFFormValue form_value;
         switch (attr) {
         case DW_AT_name:
if (num_attributes > 0) {
             }
           }
           break;
 
         case DW_AT_specification:
           if (attributes.ExtractFormValueAtIndex(i, form_value))
             specification_die_form = form_value;
           break;
+
+        case DW_AT_import:
+          if (attributes.ExtractFormValueAtIndex(i, form_value))
+            import_die_form = form_value;
+          break;
         }
       }
     }
 
     switch (tag) {
     case DW_TAG_inlined_subroutine:
     case DW_TAG_subprogram:
       if (has_address) {
case DW_TAG_variable:
         if (mangled_cstr && name != mangled_cstr &&
             ((mangled_cstr[0] == '_') || (::strcmp(name, mangled_cstr) != 0))) {
           set.globals.Insert(ConstString(mangled_cstr),
                              DIERef(cu_offset, die.GetOffset()));
         }
       }
       break;
 
+    case DW_TAG_imported_unit:
+      if (import_die_form.IsValid()) {
+        // DWZ is using DW_TAG_imported_unit only at the top level of CUs.
+        // Therefore GetParent() of any top level DIE in partial unit is always
+        // DW_TAG_compile_unit (possibly after multiple import levels).
+        const DWARFDebugInfoEntry *parent_die = die.GetParent();
+        if (!parent_die || (parent_die->Tag() != DW_TAG_compile_unit
+            && parent_die->Tag() != DW_TAG_partial_unit)) {
+          unit.GetSymbolFileDWARF()->GetObjectFile()->GetModule()
+              ->ReportError(
+                  "CU 0x%8.8" PRIx32 " DIE 0x%8.8" PRIx32
+                  " DW_TAG_imported_unit does not have DW_TAG_compile_unit"
+                  " as its parent",
+                  unit.GetOffset(), die.GetOffset());
+          break;
+        }
+        dw_offset_t import_offset = import_die_form.Reference();
+        // main_unit may be different from our callers as GetMainCU() returns
+        // only the first unit which DW_TAG_imported this unit.
+        DWARFUnit &main_unit = *unit.GetMainCU();
+        DWARFDebugInfo *main_debuginfo =
+            main_unit.GetSymbolFileDWARF()->DebugInfo();
+        DWARFUnit *import_cu =
+            main_debuginfo->GetCompileUnitContainingDIEOffset(import_offset);
+        if (!import_cu)
+          break;
+        dw_offset_t import_cu_firstdie_offset = import_cu->GetFirstDIEOffset();
+        if (import_offset != import_cu_firstdie_offset) {
+          unit.GetSymbolFileDWARF()->GetObjectFile()->GetModule()
+              ->ReportError(
+                  "CU 0x%8.8" PRIx32 " DIE 0x%8.8" PRIx32
+                  " DW_TAG_imported_unit of DIE 0x%16.16" PRIx32
+                  " is not the target CU first DIE 0x%8.8" PRIx32,
+                  unit.GetOffset(), die.GetOffset(),
+                  import_offset, import_cu_firstdie_offset);
+          break;
+        }
+        import_cu->SetMainCU(&main_unit);
+        IndexUnit(*import_cu,set);
+      }
+      break;
+
     default:
       continue;
     }
   }
 }
 
 void ManualDWARFIndex::GetGlobalVariables(ConstString basename, DIEArray &offsets) {
   Index();

source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.h

 // C Includes
 // C++ Includes
 #include <list>
 #include <map>
 #include <mutex>
 #include <set>
 #include <unordered_map>
 #include <vector>
+#include <unordered_set>
 
 // Other libraries and framework includes
 #include "llvm/ADT/DenseMap.h"
 #include "llvm/Support/Threading.h"
+#include "llvm/Support/RWMutex.h"
 
 #include "lldb/Utility/Flags.h"
 
 #include "lldb/Core/RangeMap.h"
 #include "lldb/Core/UniqueCStringMap.h"
 #include "lldb/Core/dwarf.h"
 #include "lldb/Expression/DWARFExpression.h"
 #include "lldb/Symbol/DebugMacros.h"
public:
   virtual DWARFUnit *GetBaseCompileUnit();
 
   static bool
   DIEInDeclContext(const lldb_private::CompilerDeclContext *parent_decl_ctx,
                    const DWARFDIE &die);
 
   void Dump(lldb_private::Stream &s) override;
 
+  SymbolFileDWARF *GetDWZSymbolFile() const {
+    if (!m_dwz_common_file)
+      return nullptr;
+    return m_dwz_common_file->SymbolFile();
+  }
+  bool GetIsDWZ() const { return m_is_dwz; }
+
 protected:
   typedef llvm::DenseMap<const DWARFDebugInfoEntry *, lldb_private::Type *>
       DIEToTypePtr;
   typedef llvm::DenseMap<const DWARFDebugInfoEntry *, lldb::VariableSP>
       DIEToVariableSP;
   typedef llvm::DenseMap<const DWARFDebugInfoEntry *,
                          lldb::opaque_compiler_type_t>
       DIEToClangType;
protected:
   }
 
   virtual ClangTypeToDIE &GetForwardDeclClangTypeToDie() {
     return m_forward_decl_clang_type_to_die;
   }
 
   SymbolFileDWARFDwp *GetDwpSymbolFile();
 
+  void InitializeDWZ();
+
+  class DWZCommonFile {
+  public:
+    // C++14: Use heterogenous lookup.
+    DWZCommonFile(const lldb_private::FileSpec &filespec_ref);
+    DWZCommonFile(std::unique_ptr<SymbolFileDWARF> symbol_file,
+        lldb::ObjectFileSP obj_file, lldb::ModuleSP module);
+    SymbolFileDWARF *SymbolFile() const { return m_symbol_file.get(); }
+
+    bool operator==(const DWZCommonFile &rhs) const {
+      return m_filespec_ref == rhs.m_filespec_ref;
+    }
+    bool operator!=(const DWZCommonFile &rhs) const { return !(*this == rhs); }
+    class Hasher {
+    public:
+      size_t operator()(const DWZCommonFile &key) const {
+        return lldb_private::FileSpec::Hasher()(key.m_filespec_ref);
+      }
+    };
+
+    size_t m_use_count = 0;
+
+  private:
+    std::unique_ptr<SymbolFileDWARF> m_symbol_file;
+    lldb::ObjectFileSP m_obj_file;
+    lldb::ModuleSP m_module;
+    const lldb_private::FileSpec &m_filespec_ref;
+
+    DISALLOW_COPY_AND_ASSIGN(DWZCommonFile);
+  };
+  DWZCommonFile *m_dwz_common_file = nullptr;
+  void DWZCommonFileSet(DWZCommonFile *dwz_common_file);
+  void DWZCommonFileClear();
+  static std::unordered_set<DWZCommonFile, DWZCommonFile::Hasher>
+      m_filespec_to_dwzcommonfile;
+  static llvm::sys::RWMutex m_filespec_to_dwzcommonfile_mutex;
+  bool m_is_dwz = false;
+
   lldb::ModuleWP m_debug_map_module_wp;
   SymbolFileDWARFDebugMap *m_debug_map_symfile;
 
   llvm::once_flag m_dwp_symfile_once_flag;
   std::unique_ptr<SymbolFileDWARFDwp> m_dwp_symfile;
 
   lldb_private::DWARFDataExtractor m_dwarf_data;
 

source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.cpp

: SymbolFile(objfile), UserID(uint64_t(DW_INVALID_OFFSET)
       m_data_debug_aranges(), m_data_debug_frame(), m_data_debug_info(),
       m_data_debug_line(), m_data_debug_macro(), m_data_debug_loc(),
       m_data_debug_ranges(), m_data_debug_str(), m_data_apple_names(),
       m_data_apple_types(), m_data_apple_namespaces(), m_abbr(), m_info(),
       m_line(), m_fetched_external_modules(false),
       m_supports_DW_AT_APPLE_objc_complete_type(eLazyBoolCalculate), m_ranges(),
       m_unique_ast_type_map() {}
 
-SymbolFileDWARF::~SymbolFileDWARF() {}
+SymbolFileDWARF::~SymbolFileDWARF() {
+  DWZCommonFileClear();
+}
 
 static const ConstString &GetDWARFMachOSegmentName() {
   static ConstString g_dwarf_section_name("__DWARF");
   return g_dwarf_section_name;
 }
 
 UniqueDWARFASTTypeMap &SymbolFileDWARF::GetUniqueDWARFASTTypeMap() {
   SymbolFileDWARFDebugMap *debug_map_symfile = GetDebugMapSymfile();
   if (debug_map_symfile)
     return debug_map_symfile->GetUniqueDWARFASTTypeMap();
   else
     return m_unique_ast_type_map;
 }
 
 TypeSystem *SymbolFileDWARF::GetTypeSystemForLanguage(LanguageType language) {
+  lldbassert(!GetIsDWZ());
   SymbolFileDWARFDebugMap *debug_map_symfile = GetDebugMapSymfile();
   TypeSystem *type_system;
   if (debug_map_symfile) {
     type_system = debug_map_symfile->GetTypeSystemForLanguage(language);
   } else {
     type_system = m_obj_file->GetModule()->GetTypeSystemForLanguage(language);
     if (type_system)
       type_system->SetSymbolFile(this);
   }
   return type_system;
 }
 
 void SymbolFileDWARF::InitializeObject() {
   Log *log = LogChannelDWARF::GetLogIfAll(DWARF_LOG_DEBUG_INFO);
   ModuleSP module_sp(m_obj_file->GetModule());
   if (module_sp) {
     const SectionList *section_list = module_sp->GetSectionList();
     Section *section =
         section_list->FindSectionByName(GetDWARFMachOSegmentName()).get();
 
     if (section)
       m_obj_file->ReadSectionData(section, m_dwarf_data);
   }
 
+  // DWZ needs to be already prepared for indexing below.
+  InitializeDWZ();
+
   if (!GetGlobalPluginProperties()->IgnoreFileIndexes()) {
     DWARFDataExtractor apple_names, apple_namespaces, apple_types, apple_objc;
     LoadSectionData(eSectionTypeDWARFAppleNames, apple_names);
     LoadSectionData(eSectionTypeDWARFAppleNamespaces, apple_namespaces);
     LoadSectionData(eSectionTypeDWARFAppleTypes, apple_types);
     LoadSectionData(eSectionTypeDWARFAppleObjC, apple_objc);
 
     m_index = AppleDWARFIndex::Create(
uint32_t SymbolFileDWARF::CalculateAbilities() {
   return abilities;
 }
 
 const DWARFDataExtractor &
 SymbolFileDWARF::GetCachedSectionData(lldb::SectionType sect_type,
                                       DWARFDataSegment &data_segment) {
   llvm::call_once(data_segment.m_flag, [this, sect_type, &data_segment] {
     this->LoadSectionData(sect_type, std::ref(data_segment.m_data));
+    if (sect_type == eSectionTypeDWARFDebugInfo && m_dwz_common_file) {
+      SymbolFileDWARF *dwz_symbol_file = m_dwz_common_file->SymbolFile();
+      const DWARFDataExtractor &dwz_debug_info_data =
+          dwz_symbol_file->get_debug_info_data();
+      lldbassert(dwz_debug_info_data.GetByteSize());
+      data_segment.m_data.OffsetData(dwz_debug_info_data.GetByteSize());
+    }
   });
   return data_segment.m_data;
 }
 
 void SymbolFileDWARF::LoadSectionData(lldb::SectionType sect_type,
                                       DWARFDataExtractor &data) {
   ModuleSP module_sp(m_obj_file->GetModule());
   const SectionList *section_list = module_sp->GetSectionList();
llvm::call_once(m_dwp_symfile_once_flag, [this]() {
     FileSpec dwp_filespec = Symbols::LocateExecutableSymbolFile(module_spec);
     if (dwp_filespec.Exists()) {
       m_dwp_symfile = SymbolFileDWARFDwp::Create(GetObjectFile()->GetModule(),
                                                  dwp_filespec);
     }
   });
   return m_dwp_symfile.get();
 }
+
+std::unordered_set<SymbolFileDWARF::DWZCommonFile,
+    SymbolFileDWARF::DWZCommonFile::Hasher>
+    SymbolFileDWARF::m_filespec_to_dwzcommonfile;
+llvm::sys::RWMutex SymbolFileDWARF::m_filespec_to_dwzcommonfile_mutex;
+
+void SymbolFileDWARF::InitializeDWZ() {
+  const DWARFDataExtractor &section_extractor(get_gnu_debugaltlink());
+  if (section_extractor.GetByteSize() == 0)
+    return; // .gnu_debugaltlink does not exist
+  if (GetIsDWZ()) {
+    GetObjectFile()->GetModule()->ReportWarning(
+        "Error reading DWZ common file - it does contain .gnu_debugaltlink");
+    return;
+  }
+  lldb::offset_t offset = 0;
+  const char *link_cstr(section_extractor.GetCStr(&offset));
+  if (!link_cstr) {
+    GetObjectFile()->GetModule()->ReportWarning(
+        "Cannot get DWZ common file - missing section .gnu_debugaltlink");
+    return;
+  }
+  lldb::offset_t uuid_bytes_size = section_extractor.BytesLeft(offset);
+  const void *uuid_bytes(section_extractor.GetData(&offset, uuid_bytes_size));
+  if (!uuid_bytes) {
+    GetObjectFile()->GetModule()->ReportWarning(
+        "Cannot get DWZ common file - missing build-id"
+        " in section .gnu_debugaltlink with string \"%s\"",
+        link_cstr);
+    return;
+  }
+  UUID link_uuid = UUID::fromOptionalData(uuid_bytes, uuid_bytes_size);
+  if (!link_uuid.IsValid()) {
+    GetObjectFile()->GetModule()->ReportWarning(
+        "Cannot get DWZ common file - invalid build-id size %" PRIu64
+        " in section .gnu_debugaltlink with string \"%s\"",
+        uuid_bytes_size, link_cstr);
+    return;
+  }
+  // For objfile "/usr/lib/debug/usr/bin/true.debug"
+  // link_cstr is "../../.dwz/coreutils-8.25-17.fc25.x86_64".
+  ModuleSpec dwz_module_spec;
+  FileSpec &dwz_fspec = dwz_module_spec.GetFileSpec();
+  dwz_fspec = FileSpec(link_cstr, false /* resolve_path */);
+  dwz_fspec.PrependPathComponent(
+      GetObjectFile()->GetFileSpec().CopyByRemovingLastPathComponent());
+  // C++14: Use heterogenous lookup.
+  DWZCommonFile dwz_fspec_lookup(dwz_fspec);
+  {
+    llvm::sys::ScopedReader lock(m_filespec_to_dwzcommonfile_mutex);
+    const auto it = m_filespec_to_dwzcommonfile.find(dwz_fspec_lookup);
+    if (it != m_filespec_to_dwzcommonfile.end()) {
+      DWZCommonFileSet(const_cast<DWZCommonFile *>(&*it));
+      return;
+    }
+  }
+  dwz_module_spec.GetArchitecture() =
+      GetObjectFile()->GetModule()->GetArchitecture();
+  ModuleSP dwz_module = std::make_shared<Module>(dwz_module_spec);
+  DataBufferSP dwz_file_data_sp;
+  lldb::offset_t dwz_file_data_offset = 0;
+  lldb::ObjectFileSP dwz_obj_file = ObjectFile::FindPlugin(
+      dwz_module, &dwz_fspec, 0, dwz_fspec.GetByteSize(),
+      dwz_file_data_sp, dwz_file_data_offset);
+  if (!dwz_obj_file) {
+    GetObjectFile()->GetModule()->ReportWarning(
+        "Cannot get DWZ common file - file \"%s\" cannot be opened",
+        dwz_fspec.GetCString());
+    return;
+  }
+  lldbassert(dwz_obj_file->GetFileSpec() == dwz_fspec);
+  UUID dwz_uuid;
+  if (!dwz_obj_file->GetUUID(&dwz_uuid)) {
+    GetObjectFile()->GetModule()->ReportWarning(
+        "Cannot get DWZ common file - file \"%s\" does not have build-id",
+        dwz_fspec.GetCString());
+    return;
+  }
+  if (link_uuid != dwz_uuid) {
+    GetObjectFile()->GetModule()->ReportWarning(
+        "Cannot get DWZ common file - expected build-id %s but file \"%s\""
+        " has build-id %s",
+        link_uuid.GetAsString().c_str(), dwz_fspec.GetCString(),
+        dwz_uuid.GetAsString().c_str());
+    return;
+  }
+  auto dwz_symbol_file =
+      llvm::make_unique<SymbolFileDWARF>(dwz_obj_file.get());
+  dwz_symbol_file->m_is_dwz = true;
+  dwz_symbol_file->InitializeObject();
+  // Call private DWARFDebugInfo::ParseCompileUnitHeadersIfNeeded() as
+  // otherwise DWARFCompileUnit::GetAbbreviations() would have no data.
+  dwz_symbol_file->DebugInfo()->GetNumCompileUnits();
+  {
+    llvm::sys::ScopedWriter lock(m_filespec_to_dwzcommonfile_mutex);
+    const auto it = m_filespec_to_dwzcommonfile.find(dwz_fspec_lookup);
+    if (it != m_filespec_to_dwzcommonfile.end())
+      DWZCommonFileSet(const_cast<DWZCommonFile *>(&*it));
+    else {
+      const auto insertpair = m_filespec_to_dwzcommonfile.emplace(
+          std::move(dwz_symbol_file), std::move(dwz_obj_file),
+          std::move(dwz_module));
+      lldbassert(insertpair.second);
+      DWZCommonFileSet(const_cast<DWZCommonFile *>(&*insertpair.first));
+      lldbassert(*m_dwz_common_file == dwz_fspec_lookup);
+    }
+  }
+}
+
+void SymbolFileDWARF::DWZCommonFileSet(DWZCommonFile *dwz_common_file) {
+  if (m_dwz_common_file == dwz_common_file)
+    return;
+  DWZCommonFileClear();
+  m_dwz_common_file = dwz_common_file;
+  ++m_dwz_common_file->m_use_count;
+
+  // GetCachedSectionData calls OffsetData as m_dwarf_data is not yet fetched.
+}
+
+void SymbolFileDWARF::DWZCommonFileClear() {
+  if (!m_dwz_common_file)
+    return;
+  SymbolFileDWARF *dwz_symbol_file = m_dwz_common_file->SymbolFile();
+
+  DWARFDebugInfo *dwz_debuginfo = dwz_symbol_file->DebugInfo();
+  if (dwz_debuginfo) {
+    const size_t dwz_num_cus = dwz_debuginfo->GetNumCompileUnits();
+    for (size_t dwz_cu_idx = 0; dwz_cu_idx < dwz_num_cus; ++dwz_cu_idx) {
+      DWARFUnit *dwz_cu = dwz_debuginfo->GetCompileUnitAtIndex(dwz_cu_idx);
+      if (dwz_cu)
+        dwz_cu->DWZDeleteSymbolFile(this);
+    }
+  }
+
+  const DWARFDataExtractor &dwz_debug_info_data =
+      dwz_symbol_file->get_debug_info_data();
+  lldbassert(dwz_debug_info_data.GetByteSize());
+  m_dwarf_data.OffsetData(-dwz_debug_info_data.GetByteSize());
+
+  llvm::sys::ScopedWriter lock(m_filespec_to_dwzcommonfile_mutex);
+  lldbassert(m_dwz_common_file->m_use_count);
+  if (--m_dwz_common_file->m_use_count)
+    return;
+  size_t erased = m_filespec_to_dwzcommonfile.erase(*m_dwz_common_file);
+  lldbassert(erased == 1);
+}
+
+SymbolFileDWARF::DWZCommonFile::DWZCommonFile(const FileSpec &filespec_ref)
+    : m_filespec_ref(filespec_ref) {}
+
+SymbolFileDWARF::DWZCommonFile::DWZCommonFile(
+    std::unique_ptr<SymbolFileDWARF> symbol_file,
+    lldb::ObjectFileSP obj_file, lldb::ModuleSP module)
+    : m_symbol_file(std::move(symbol_file)), m_obj_file(std::move(obj_file)),
+    m_module(std::move(module)), m_filespec_ref(m_obj_file->GetFileSpec()) {}

source/Utility/DataEncoder.cpp

 // that shared data. Else zero is returned.
 //------------------------------------------------------------------
 size_t DataEncoder::GetSharedDataOffset() const {
   if (m_start != nullptr) {
     const DataBuffer *data = m_data_sp.get();
     if (data != nullptr) {
       const uint8_t *data_bytes = data->GetBytes();
       if (data_bytes != nullptr) {
-        assert(m_start >= data_bytes);
+        // For DWARFDataExtractor::OffsetData we need to return negative value.
+        // assert(m_start >= data_bytes);
         return m_start - data_bytes;
       }
     }
   }
   return 0;
 }
 
 //----------------------------------------------------------------------

source/Utility/DataExtractor.cpp

 // that shared data. Else zero is returned.
 //------------------------------------------------------------------
 size_t DataExtractor::GetSharedDataOffset() const {
   if (m_start != nullptr) {
     const DataBuffer *data = m_data_sp.get();
     if (data != nullptr) {
       const uint8_t *data_bytes = data->GetBytes();
       if (data_bytes != nullptr) {
-        assert(m_start >= data_bytes);
+        // For DWARFDataExtractor::OffsetData we need to return negative value.
+        // assert(m_start >= data_bytes);
         return m_start - data_bytes;
       }
     }
   }
   return 0;
 }
 
 //----------------------------------------------------------------------