02/03: Contiguous sections (.debug_info+.debug_types) for D54670==D32167 (.debug_types)

[jankratochvil]

Summary

A different trivial approach is to use the mmapped memory range <.debug_info ... garbage ... .debug_types>, then no dispatching is needed and it has no performance impact.
With this patch one could revert the new GetData() from D46606 but the code may be cleaner with it anyway. It is just no longer needed to be virtual so this patch removes its virtuality.
The core is new SymbolFileDWARF::get_debug_info_data().

It has no regressions.

That new class and inheritance of SectionReader (ObjectFileELF::SectionReaderCompressed, ObjectFileJIT::SectionReaderJIT) copying inheritance of ObjectFile' is there because SectionReader` may have local state (such as Decompressor) which is needed for each Section. Additionally ObjectFileELF may contain both uncompressed and compressed sections so it can choose the proper class for instantiation.

[clayborg]

Overall I am ok with minimal regression in speed if a few percent is all that it is costing us. I am generally ok with this patch. A few questions below.

Is there a reason we need DWARFConcatenatingDataExtractor? Can we just put all functionality into DWARFDataExtractor? Then the only place we need to modify if the place that parses the old CU and the type units from .debug_types.

Also is there a reason we need to pass the block data in code such as:

const DWARFDataExtractor &debug_info_data = die.GetData().GetDWARFDataExtractor(form_value.BlockData());

[jankratochvil]

In D51578#1233518, @clayborg wrote:

Is there a reason we need DWARFConcatenatingDataExtractor? Can we just put all functionality into DWARFDataExtractor?

OK, I will try it that way.

Also is there a reason we need to pass the block data in code such as:

const DWARFDataExtractor &debug_info_data = die.GetData().GetDWARFDataExtractor(form_value.BlockData());

EDIT: This case cannot be solved transparently as DWARFConcatenatingDataExtractor (no matter how it is called) has discontiguous data. Existing code does not expect that:

uint32_t block_offset = form_value.BlockData() - debug_info_data.GetDataStart();

What should be GetDataStart() for a discontiguous set of memory data areas?
Still I will try to refactor it named DWARFDataExtractor but this specific change may need to remain there.

I am sorry I will be now for 2 weeks away.

[jankratochvil]

Patch concat = original posting with DWARFConcatenatingDataExtractor used only for .debug_info(+.debug_types) section. download
Patch concat2 = used DWARFDataExtractor with integrated concatenation for all DWARF sections. download

master  real=0.996s user=14.744s sys=1.050s
concat  real=0.994s user=15.524s sys=1.117s
concat2 real=1.001s user=15.580s sys=1.334s

concat vs. concat2 benchmark difference was mostly a measurement error. It is now 5% slowdown compared to trunk - even for former concat which I do not understand now, I will investigate it more as my original measured slowdown was just 1% before.
concat2 sometimes needs to cast DWARFDataExtractor to DataExtractor by .AsDataExtractor() as some functions expected a contiguous block of memory. That is a method applicable only for non-.debug_info sections (it would assert on .debug_info). It needs to be called as DWARFDataExtractor can no longer inherit DataExtractor. I did not use operator DataExtractor & as it could accidentally by applied to .debug_info which may be difficult to catch when not tested with -fdebug-types-section.
DataExtractorConcat is a separate class but it is never used separately without DWARFDataExtractor inheriting it.
Greg, do you mean it somehow way or do I miss something? Personally I may like a bit more concat but then the patch size is almost the same for both.

[jankratochvil]

I got an idea to try making the .debug_info+.debug_types memory contiguous. It cannot be mmapped then but LLDB does not seem to do that now anyway (except maybe MachO but despite some comment I haven't found a mmap() even there). Besides that there could be even .debug_info+gap+.debug_types layout.

[clayborg]

Just a few nits, but _very_ close.

source/Plugins/SymbolFile/DWARF/DWARFDebugInfo.h

65 ↗

(On Diff #172549)

Use llvm::DenseMap here?

source/Plugins/SymbolFile/DWARF/DWARFUnit.cpp

782

Should this just be "if (GetUnitDIEOnly().Tag() != DW_TAG_type_unit)"? Partial units and many others can be top level DIEs.

[jankratochvil]

There is a _dwarf -> _dwarf_type_units regression for: -f TestCppTypeLookup.test_namespace_only_dwarf_type_units

./bin/clang -o 1 ../llvm-git/tools/lldb/packages/Python/lldbsuite/test/lang/cpp/type_lookup/main.cpp -g -fdebug-types-section;./bin/lldb -o 'settings set target.experimental.inject-local-vars false' -o 'target create "./1"' -o 'b 66' -o r -o 'p *((in_contains_type *)&i)'

Expected:

(lldb) p *((in_contains_type *)&i)
error: use of undeclared identifier 'in_contains_type'
error: expected expression

Actual:

(lldb) p *((in_contains_type *)&i)
(in_contains_type) $0 = (aaa = 123)

I will check it more.

source/Plugins/SymbolFile/DWARF/DWARFDebugInfo.h

65 ↗

(On Diff #172549)

Yes, I agree that is a good idea - changed it (BTW this is your part of the patch).

source/Plugins/SymbolFile/DWARF/DWARFUnit.cpp

782

When you ask then I disagree. Currently GetAsCompileUnit() is used only for checking DW_TAG_partial_unit::DW_AT_name (from D11003) which is never present in files by DWZ tool. DWZ tool never puts DIEs referencing any code into DW_TAG_partial_unit so D11003 should not be needed; although DWZ intentionally sometimes uses DW_TAG_partial_unit::DW_AT_stmt_list but I haven't found when/how it could be useful.
> many others
Top level DIEs can be according to DWARF-5 7.5 just DW_TAG_compile_unit, DW_TAG_partial_unit or DW_TAG_type_unit.
So personally I would keep GetAsCompileUnit() to do what its name exactly says.

[clayborg]

source/Plugins/SymbolFile/DWARF/DWARFDebugInfo.h

65 ↗

(On Diff #173535)

yeah, I kept getting people commenting on my patches that told me to use DenseMap and StringMap!

[jankratochvil]

I have left here only the sections concatenation. The .debug_types itself by Greg Clayton is now updated on top of this patch in D32167.

[jankratochvil]

Updated against trunk due to new getSectionType from rL348936.

[jankratochvil]

This update fixes a regression after D52403, the code is simplified - eSectionTypeDWARFDebugInfo is no longer hardcoded in this patch.

OK for check-in with this new code?

[clayborg]

lldb/source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.cpp

693–707 ↗

(On Diff #184919)

So if we have 3GB of .debug_info and 1GB of .debug_types, we are expecting to allocate a heap based buffer and copy all the data into this? This will fail.

[jankratochvil]

So if we have 3GB of .debug_info and 1GB of .debug_types, we are expecting to allocate a heap based buffer and copy all the data into this? This will fail.

That fallback code path is used only when there is no way to access .debug_info and .debug_types from mmapped area. I was thinking that fallback would be used for example for Linux vDSO which needs to be read by Process::ReadModuleFromMemory() but that has only about 16KB so its size does not matter. But maybe on OSX (or for GDB JIT modules) the data can be bigger so in this patch I have implemented the most effective merging of non-mmapped sections.

[jankratochvil]

lldb/source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.cpp

693–707 ↗

(On Diff #184919)

replied by: https://reviews.llvm.org/D51578#1400641

[JDevlieghere]

Few comments inline.

lldb/include/lldb/Symbol/ObjectFile.h

781 ↗

(On Diff #187169)

Please clang-format the patch before landing this.

lldb/source/Plugins/ObjectFile/ELF/ObjectFileELF.cpp

3400 ↗

(On Diff #187169)

Rather than returning 0 when there's no decompressor, why not return llvm::Optional<uint64_t> to differentiate between with an actual size of zero?

3409 ↗

(On Diff #187169)

I don't think having an expected as a member makes sense. Maybe you want to use an optional? Is the goal to abstract over the decompression not being available? Or should the class assume the the decompressor is there (and itself be wrapped in an expected when that's not the case?). I'm not too familiar with this code but it seems a rather odd design.

3413 ↗

(On Diff #187169)

Trailing underscore?

3467 ↗

(On Diff #187169)

You could early return here (return 0;).

lldb/source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.cpp

684 ↗

(On Diff #187169)

Can we split this up? There's no way I remember the first two clauses by the time I'm reading the third.

include/lldb/lldb-forward.h

 class CompilerType;
 class CompileUnit;
 class Condition;
 class Connection;
 class ConnectionFileDescriptor;
 class ConstString;
 class CXXSyntheticChildren;
 class DWARFCallFrameInfo;
+class DWARFConcatenatingDataExtractor;
 class DWARFDataExtractor;
 class DWARFExpression;
 class DataBuffer;
 class DataEncoder;
 class DataExtractor;
 class Debugger;
 class Declaration;
 class DiagnosticManager;

source/Plugins/SymbolFile/DWARF/CMakeLists.txt

 add_lldb_library(lldbPluginSymbolFileDWARF PLUGIN
   AppleDWARFIndex.cpp
   DebugNamesDWARFIndex.cpp
   DIERef.cpp
   DWARFAbbreviationDeclaration.cpp
   DWARFASTParserClang.cpp
   DWARFASTParserGo.cpp
   DWARFASTParserJava.cpp
   DWARFASTParserOCaml.cpp
   DWARFAttribute.cpp
   DWARFBaseDIE.cpp
   DWARFCompileUnit.cpp
+  DWARFConcatenatingDataExtractor.cpp
   DWARFDataExtractor.cpp
   DWARFDebugAbbrev.cpp
   DWARFDebugAranges.cpp
   DWARFDebugArangeSet.cpp
   DWARFDebugInfo.cpp
   DWARFDebugInfoEntry.cpp
   DWARFDebugLine.cpp
   DWARFDebugMacro.cpp

source/Plugins/SymbolFile/DWARF/DWARFASTParserClang.cpp

case DW_TAG_APPLE_property: {
               break;
             case DW_AT_data_bit_offset:
               data_bit_offset = form_value.Unsigned();
               break;
             case DW_AT_data_member_location:
               if (form_value.BlockData()) {
                 Value initialValue(0);
                 Value memberOffset(0);
-                const DWARFDataExtractor &debug_info_data = die.GetData();
+                const DWARFDataExtractor &debug_info_data =
+                    die.GetData().GetDWARFDataExtractor(form_value.BlockData());
                 uint32_t block_length = form_value.Unsigned();
                 uint32_t block_offset =
                     form_value.BlockData() - debug_info_data.GetDataStart();
                 if (DWARFExpression::Evaluate(
                         nullptr, // ExecutionContext *
                         nullptr, // RegisterContext *
                         module_sp, debug_info_data, die.GetCU(), block_offset,
                         block_length, eRegisterKindDWARF, &initialValue,
case DW_TAG_inheritance: {
               break;
             case DW_AT_type:
               encoding_form = form_value;
               break;
             case DW_AT_data_member_location:
               if (form_value.BlockData()) {
                 Value initialValue(0);
                 Value memberOffset(0);
-                const DWARFDataExtractor &debug_info_data = die.GetData();
+                const DWARFDataExtractor &debug_info_data =
+                    die.GetData().GetDWARFDataExtractor(form_value.BlockData());
                 uint32_t block_length = form_value.Unsigned();
                 uint32_t block_offset =
                     form_value.BlockData() - debug_info_data.GetDataStart();
                 if (DWARFExpression::Evaluate(nullptr, nullptr, module_sp,
                                               debug_info_data, die.GetCU(),
                                               block_offset, block_length,
                                               eRegisterKindDWARF, &initialValue,
                                               nullptr, memberOffset, nullptr)) {

source/Plugins/SymbolFile/DWARF/DWARFASTParserGo.cpp

case DW_TAG_member: {
               break;
             case DW_AT_type:
               encoding_uid = form_value;
               break;
             case DW_AT_data_member_location:
               if (form_value.BlockData()) {
                 Value initialValue(0);
                 Value memberOffset(0);
-                const DWARFDataExtractor &debug_info_data = die.GetData();
+                const DWARFDataExtractor &debug_info_data =
+                    die.GetData().GetDWARFDataExtractor(form_value.BlockData());
                 uint32_t block_length = form_value.Unsigned();
                 uint32_t block_offset =
                     form_value.BlockData() - debug_info_data.GetDataStart();
                 if (DWARFExpression::Evaluate(
                         NULL, // ExecutionContext *
                         NULL, // RegisterContext *
                         module_sp, debug_info_data, die.GetCU(), block_offset,
                         block_length, eRegisterKindDWARF, &initialValue, NULL,

source/Plugins/SymbolFile/DWARF/DWARFBaseDIE.h

public:
   //----------------------------------------------------------------------
   // Get the data that contains the attribute values for this DIE. Support
   // for .debug_types means that any DIE can have its data either in the
   // .debug_info or the .debug_types section; this method will return the
   // correct section data.
   //
   // Clients must validate that this object is valid before calling this.
   //----------------------------------------------------------------------
-  const lldb_private::DWARFDataExtractor &GetData() const;
+  const lldb_private::DWARFConcatenatingDataExtractor &GetData() const;
 
   //----------------------------------------------------------------------
   // Accessing information about a DIE
   //----------------------------------------------------------------------
   dw_tag_t Tag() const;
 
   const char *GetTagAsCString() const;
 

source/Plugins/SymbolFile/DWARF/DWARFBaseDIE.cpp

 bool operator==(const DWARFBaseDIE &lhs, const DWARFBaseDIE &rhs) {
   return lhs.GetDIE() == rhs.GetDIE() && lhs.GetCU() == rhs.GetCU();
 }
 
 bool operator!=(const DWARFBaseDIE &lhs, const DWARFBaseDIE &rhs) {
   return !(lhs == rhs);
 }
 
-const DWARFDataExtractor &DWARFBaseDIE::GetData() const {
+const DWARFConcatenatingDataExtractor &DWARFBaseDIE::GetData() const {
   // Clients must check if this DIE is valid before calling this function.
   assert(IsValid());
   return m_cu->GetData();
 }

source/Plugins/SymbolFile/DWARF/DWARFCompileUnit.h

 class DWARFCompileUnit : public DWARFUnit {
 public:
   static DWARFCompileUnitSP Extract(SymbolFileDWARF *dwarf2Data,
       const lldb_private::DWARFDataExtractor &debug_info,
       lldb::offset_t *offset_ptr);
   void Dump(lldb_private::Stream *s) const override;
 
   //------------------------------------------------------------------
-  /// Get the data that contains the DIE information for this unit.
-  ///
-  /// @return
-  ///   The correct data (.debug_types for DWARF 4 and earlier, and
-  ///   .debug_info for DWARF 5 and later) for the DIE information in
-  ///   this unit.
-  //------------------------------------------------------------------
-  const lldb_private::DWARFDataExtractor &GetData() const override;
-
-  //------------------------------------------------------------------
   /// Get the size in bytes of the header.
   ///
   /// @return
   ///     Byte size of the compile unit header
   //------------------------------------------------------------------
   uint32_t GetHeaderByteSize() const override {
     return m_is_dwarf64 ? 23 : 11;
   }
 
 private:
   DWARFCompileUnit(SymbolFileDWARF *dwarf2Data);
   DISALLOW_COPY_AND_ASSIGN(DWARFCompileUnit);
 };
 
 #endif // SymbolFileDWARF_DWARFCompileUnit_h_

source/Plugins/SymbolFile/DWARF/DWARFCompileUnit.cpp

 
 void DWARFCompileUnit::Dump(Stream *s) const {
   s->Printf("0x%8.8x: Compile Unit: length = 0x%8.8x, version = 0x%4.4x, "
             "abbr_offset = 0x%8.8x, addr_size = 0x%2.2x (next CU at "
             "{0x%8.8x})\n",
             m_offset, m_length, m_version, GetAbbrevOffset(), m_addr_size,
             GetNextCompileUnitOffset());
 }
-
-
-const lldb_private::DWARFDataExtractor &DWARFCompileUnit::GetData() const {
-  return m_dwarf->get_debug_info_data();
-}

source/Plugins/SymbolFile/DWARF/DWARFConcatenatingDataExtractor.h

+//===-- DWARFConcatenatingDataExtractor.h -----------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef liblldb_DWARFConcatenatingDataExtractor_h_
+#define liblldb_DWARFConcatenatingDataExtractor_h_
+
+#include "DWARFDataExtractor.h"
+
+// Other libraries and framework includes.
+#include "lldb/Core/dwarf.h"
+
+namespace lldb_private {
+
+class DWARFConcatenatingDataExtractor {
+public:
+  DWARFConcatenatingDataExtractor(const DWARFDataExtractor &extractor)
+      : m_extractors({&extractor}) {}
+  DWARFConcatenatingDataExtractor() {}
+  void AppendDWARFDataExtractor(const DWARFDataExtractor &extractor) {
+    if (extractor.GetByteSize())
+      m_extractors.push_back(&extractor);
+  }
+  const DWARFDataExtractor &GetDWARFDataExtractor_slow(
+      const uint8_t *inner) const;
+  inline const DWARFDataExtractor &GetDWARFDataExtractor(
+      const uint8_t *inner) const;
+  inline const DWARFDataExtractor &GetDWARFDataExtractor(
+      lldb::offset_t offset) const {
+    return offset_to_extractor(offset, nullptr);
+  }
+  inline bool IsDWARF64(lldb::offset_t offset) const
+  { return GetDWARFDataExtractor(offset).IsDWARF64(); }
+  // forwarder-based implementation would assert on out of bound 'offset'.
+  bool ValidOffset(lldb::offset_t offset) const;
+
+#define FORWARDER(Extractor, FuncName, RetType, ParamListParen, ...) \
+    inline RetType FuncName ParamListParen const \
+    { return forwarder<RetType, Extractor>(&Extractor::FuncName, __VA_ARGS__); }
+  FORWARDER(DataExtractor, GetFloat, float,
+      (lldb::offset_t *offset_ptr), offset_ptr)
+  FORWARDER(DataExtractor, GetDouble, double,
+      (lldb::offset_t *offset_ptr), offset_ptr)
+  FORWARDER(DataExtractor, GetLongDouble, long double,
+      (lldb::offset_t *offset_ptr), offset_ptr)
+  FORWARDER(DataExtractor, GetMaxU32, uint32_t,
+      (lldb::offset_t *offset_ptr, size_t byte_size), offset_ptr, byte_size)
+  FORWARDER(DataExtractor, GetMaxU64, uint64_t,
+      (lldb::offset_t *offset_ptr, size_t byte_size), offset_ptr, byte_size)
+  FORWARDER(DataExtractor, GetMaxU64_unchecked, uint64_t,
+    (lldb::offset_t *offset_ptr, size_t byte_size), offset_ptr, byte_size)
+  FORWARDER(DataExtractor, GetMaxS64, int64_t,
+      (lldb::offset_t *offset_ptr, size_t byte_size), offset_ptr, byte_size)
+  FORWARDER(DataExtractor, GetMaxU64Bitfield, uint64_t,
+      (lldb::offset_t *offset_ptr, size_t size, uint32_t bitfield_bit_size,
+          uint32_t bitfield_bit_offset),
+      offset_ptr, size, bitfield_bit_size, bitfield_bit_offset)
+  FORWARDER(DataExtractor, GetMaxS64Bitfield, int64_t,
+      (lldb::offset_t *offset_ptr, size_t size, uint32_t bitfield_bit_size,
+          uint32_t bitfield_bit_offset),
+      offset_ptr, size, bitfield_bit_size, bitfield_bit_offset)
+  FORWARDER(DataExtractor, GetPointer, uint64_t,
+      (lldb::offset_t *offset_ptr), offset_ptr)
+  FORWARDER(DataExtractor, GetU8, uint8_t,
+      (lldb::offset_t *offset_ptr), offset_ptr)
+  FORWARDER(DataExtractor, GetU8_unchecked, uint8_t,
+      (lldb::offset_t *offset_ptr), offset_ptr)
+  FORWARDER(DataExtractor, GetU16_unchecked, uint16_t,
+      (lldb::offset_t *offset_ptr), offset_ptr)
+  FORWARDER(DataExtractor, GetU32_unchecked, uint32_t,
+      (lldb::offset_t *offset_ptr), offset_ptr)
+  FORWARDER(DataExtractor, GetU64_unchecked, uint64_t,
+      (lldb::offset_t *offset_ptr), offset_ptr)
+  FORWARDER(DataExtractor, GetU8, void *,
+      (lldb::offset_t *offset_ptr, void *dst, uint32_t count),
+      offset_ptr, dst, count)
+  FORWARDER(DataExtractor, GetU16, uint16_t,
+      (lldb::offset_t *offset_ptr), offset_ptr)
+  FORWARDER(DataExtractor, GetU16, void *,
+      (lldb::offset_t *offset_ptr, void *dst, uint32_t count),
+      offset_ptr, dst, count)
+  FORWARDER(DataExtractor, GetU32, uint32_t,
+      (lldb::offset_t *offset_ptr), offset_ptr)
+  FORWARDER(DataExtractor, GetU32, void *,
+      (lldb::offset_t *offset_ptr, void *dst, uint32_t count),
+      offset_ptr, dst, count)
+  FORWARDER(DataExtractor, GetU64, uint64_t,
+      (lldb::offset_t *offset_ptr), offset_ptr)
+  FORWARDER(DataExtractor, GetU64, void *,
+      (lldb::offset_t *offset_ptr, void *dst, uint32_t count),
+      offset_ptr, dst, count)
+  FORWARDER(DataExtractor, GetSLEB128, int64_t,
+      (lldb::offset_t *offset_ptr), offset_ptr)
+  FORWARDER(DataExtractor, GetULEB128, uint64_t,
+      (lldb::offset_t *offset_ptr), offset_ptr)
+  FORWARDER(DataExtractor, PeekData, const uint8_t *,
+      (lldb::offset_t offset, lldb::offset_t length), offset, length)
+  FORWARDER(DataExtractor, GetCStr, const char *,
+      (lldb::offset_t *offset_ptr), offset_ptr)
+  FORWARDER(DataExtractor, Skip_LEB128, uint32_t,
+      (lldb::offset_t *offset_ptr), offset_ptr)
+  FORWARDER(DataExtractor, ValidOffsetForDataOfSize, bool,
+      (lldb::offset_t offset, lldb::offset_t length), offset, length)
+  FORWARDER(DWARFDataExtractor, GetDWARFOffset, dw_offset_t,
+      (lldb::offset_t *offset_ptr), offset_ptr)
+  FORWARDER(DWARFDataExtractor, GetDWARFInitialLength, uint64_t,
+      (lldb::offset_t *offset_ptr), offset_ptr)
+#undef FORWARDER
+
+private:
+  const DWARFDataExtractor &offset_to_extractor_slow(lldb::offset_t offset,
+      lldb::offset_t *extractor_offsetp) const;
+  inline const DWARFDataExtractor &offset_to_extractor(lldb::offset_t offset,
+      lldb::offset_t *extractor_offsetp) const;
+  template<class RetType, class Extractor, class... Args>
+  inline RetType forwarder(
+      RetType (Extractor::*mptr)(lldb::offset_t *offset_ptr, Args...) const,
+      lldb::offset_t *offset_ptr, Args... args) const {
+    lldb::offset_t extractor_offset;
+    const Extractor &extractor =
+        offset_to_extractor(*offset_ptr, &extractor_offset);
+    *offset_ptr -= extractor_offset;
+    RetType retval = (extractor.*mptr)(offset_ptr, std::forward<Args>(args)...);
+    *offset_ptr += extractor_offset;
+    return retval;
+  }
+  template<class RetType, class Extractor, class... Args>
+  inline RetType forwarder(
+      RetType (Extractor::*mptr)(lldb::offset_t offset, Args...) const,
+      lldb::offset_t offset, Args... args) const {
+    lldb::offset_t extractor_offset;
+    const Extractor &extractor = offset_to_extractor(offset, &extractor_offset);
+    offset -= extractor_offset;
+    return (extractor.*mptr)(offset, std::forward<Args>(args)...);
+  }
+
+  llvm::SmallVector<const DWARFDataExtractor *, 2> m_extractors;
+};
+
+const DWARFDataExtractor &DWARFConcatenatingDataExtractor::offset_to_extractor(
+    lldb::offset_t offset, lldb::offset_t *extractor_offsetp) const {
+  if (extractor_offsetp)
+    *extractor_offsetp = 0;
+  if (m_extractors.size() == 1)
+    return *m_extractors[0];
+
+  return offset_to_extractor_slow(offset, extractor_offsetp);
+}
+
+const DWARFDataExtractor &
+    DWARFConcatenatingDataExtractor::GetDWARFDataExtractor(
+    const uint8_t *inner) const {
+  if (m_extractors.size() == 1)
+    return *m_extractors[0];
+
+  return GetDWARFDataExtractor_slow(inner);
+}
+
+}
+
+#endif // liblldb_DWARFConcatenatingDataExtractor_h_

source/Plugins/SymbolFile/DWARF/DWARFConcatenatingDataExtractor.cpp

+//===-- DWARFConcatenatingDataExtractor.cpp ---------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "DWARFConcatenatingDataExtractor.h"
+#include "lldb/Utility/LLDBAssert.h"
+
+namespace lldb_private {
+
+const DWARFDataExtractor &
+    DWARFConcatenatingDataExtractor::offset_to_extractor_slow(
+    lldb::offset_t offset, lldb::offset_t *extractor_offsetp) const {
+  for (const DWARFDataExtractor *extractorp : m_extractors) {
+    if (offset < extractorp->GetByteSize())
+      return *extractorp;
+    offset -= extractorp->GetByteSize();
+    if (extractor_offsetp)
+      *extractor_offsetp += extractorp->GetByteSize();
+  }
+  lldbassert(0);
+  return *m_extractors[0];
+}
+
+const DWARFDataExtractor &
+    DWARFConcatenatingDataExtractor::GetDWARFDataExtractor_slow(
+    const uint8_t *inner) const {
+  for (const DWARFDataExtractor *extractorp : m_extractors)
+    if (inner >= extractorp->GetDataStart() && inner < extractorp->GetDataEnd())
+      return *extractorp;
+  lldbassert(0);
+  return *m_extractors[0];
+}
+
+bool DWARFConcatenatingDataExtractor::ValidOffset(lldb::offset_t offset) const {
+  for (const DWARFDataExtractor *extractorp : m_extractors) {
+    if (offset < extractorp->GetByteSize())
+      return true;
+    offset -= extractorp->GetByteSize();
+  }
+  return false;
+}
+
+}

source/Plugins/SymbolFile/DWARF/DWARFDataExtractor.h

public:
   uint64_t GetDWARFInitialLength(lldb::offset_t *offset_ptr) const;
 
   dw_offset_t GetDWARFOffset(lldb::offset_t *offset_ptr) const;
 
   size_t GetDWARFSizeofInitialLength() const { return m_is_dwarf64 ? 12 : 4; }
   size_t GetDWARFSizeOfOffset() const { return m_is_dwarf64 ? 8 : 4; }
   bool IsDWARF64() const { return m_is_dwarf64; }
 
-  //------------------------------------------------------------------
-  /// Slide the data in the buffer so that access to the data will
-  /// start at offset \a offset.
-  ///
-  /// This is currently used to provide access to the .debug_types
-  /// section and pretend it starts at at offset of the size of the
-  /// .debug_info section. This allows a minimally invasive change to
-  /// add support for .debug_types by allowing all DIEs to have unique
-  /// offsets and thus allowing no code to change in the DWARF parser.
-  /// Modifying the offsets in the .debug_types doesn't affect
-  /// anything because since all info in the .debug_types is type unit
-  /// relative and no types within a type unit can refer to any DIEs
-  /// outside of the type unit without using DW_AT_signature. It also
-  /// sets us up to move to DWARF5 where there is no .debug_types
-  /// section as compile units and type units are in the .debug_info.
-  ///
-  /// @param[in] offset
-  ///   The amount to slide the data contents by.
-  //------------------------------------------------------------------
-  void OffsetData(lldb::offset_t offset) {
-    if (GetByteSize())
-      m_start -= offset;
-  }
-
 protected:
   mutable bool m_is_dwarf64;
 };
 }
 
 #endif // liblldb_DWARFDataExtractor_h_

source/Plugins/SymbolFile/DWARF/DWARFDebugInfo.cpp

void DWARFDebugInfo::ParseCompileUnitHeadersIfNeeded() {
   if (!m_compile_units.empty())
     return;
   
   if (m_dwarf2Data == nullptr)
     return;
 
   lldb::offset_t offset = 0;
   DWARFCompileUnitSP cu_sp;
-  const auto &debug_info_data = m_dwarf2Data->get_debug_info_data();
-  while ((cu_sp = DWARFCompileUnit::Extract(m_dwarf2Data, debug_info_data,
-                                            &offset))) {
+  const DWARFDataExtractor &debug_info_data =
+      m_dwarf2Data->get_debug_info_data();
+  while (offset < debug_info_data.GetByteSize()
+      && (cu_sp = DWARFCompileUnit::Extract(
+          m_dwarf2Data, debug_info_data, &offset))) {
     m_compile_units.push_back(cu_sp);
     offset = cu_sp->GetNextCompileUnitOffset();
   }
   
   // If we have a separate .debug_types section, it means we are using DWARF4
   // or earlier where type units are a separate section. All data in the
   // .debug_types section doesn't refer directly to any other DIEs, all
   // references are done via type signatures. With this in mind, we can support
   // parsing .debug_types by pretending that the data in .debug_types
   // immediately follows all data in the .debug_info section. This allows all
   // types in the .debug_types to have a unique DIE offset that is:
   //    offset = sizeof(.debug_info) + debug_type_offset
   //
-  // So below we set "offset" to be the size of the .debug_info section. We have
-  // modified the debug_types_data to know that its first byte starts at this
-  // offset.
-  auto debug_types_data = m_dwarf2Data->get_debug_types_data();
-  if (debug_types_data.GetByteSize() == 0)
-    return;
-  offset = debug_info_data.GetByteSize();
+  // We must use get_debug_concatenated_data() so that .debug_types start at
+  // size of the .debug_info section.
+  const DWARFConcatenatingDataExtractor &debug_concatenated_data =
+      m_dwarf2Data->get_debug_concatenated_data();
   DWARFTypeUnitSP tu_sp;
-  while ((tu_sp = DWARFTypeUnit::Extract(m_dwarf2Data, debug_types_data,
+  while ((tu_sp = DWARFTypeUnit::Extract(m_dwarf2Data, debug_concatenated_data,
                                          &offset))) {
     m_type_sig_to_cu_index[tu_sp->GetTypeSignature()] = m_compile_units.size();
     m_compile_units.push_back(tu_sp);
     offset = tu_sp->GetNextCompileUnitOffset();
   }
 }
 
 size_t DWARFDebugInfo::GetNumCompileUnits() {

source/Plugins/SymbolFile/DWARF/DWARFDebugInfoEntry.h

public:
   void BuildAddressRangeTable(SymbolFileDWARF *dwarf2Data,
                               const DWARFUnit *cu,
                               DWARFDebugAranges *debug_aranges) const;
 
   void BuildFunctionAddressRangeTable(SymbolFileDWARF *dwarf2Data,
                                       const DWARFUnit *cu,
                                       DWARFDebugAranges *debug_aranges) const;
 
-  bool FastExtract(const lldb_private::DWARFDataExtractor &debug_info_data,
-                   const DWARFUnit *cu,
-                   const DWARFFormValue::FixedFormSizes &fixed_form_sizes,
-                   lldb::offset_t *offset_ptr);
+  bool FastExtract(
+      const lldb_private::DWARFConcatenatingDataExtractor &debug_info_data,
 
   bool Extract(SymbolFileDWARF *dwarf2Data, const DWARFUnit *cu,
                lldb::offset_t *offset_ptr);
 
   bool LookupAddress(const dw_addr_t address, SymbolFileDWARF *dwarf2Data,
                      const DWARFUnit *cu,
                      DWARFDebugInfoEntry **function_die,
                      DWARFDebugInfoEntry **block_die);

source/Plugins/SymbolFile/DWARF/DWARFDebugInfoEntry.cpp

 #include "SymbolFileDWARF.h"
 #include "SymbolFileDWARFDwo.h"
 
 using namespace lldb_private;
 using namespace std;
 extern int g_verbose;
 
 bool DWARFDebugInfoEntry::FastExtract(
-    const DWARFDataExtractor &debug_info_data, const DWARFUnit *cu,
+    const DWARFConcatenatingDataExtractor &debug_info_data, const DWARFUnit *cu,
     const DWARFFormValue::FixedFormSizes &fixed_form_sizes,
     lldb::offset_t *offset_ptr) {
   m_offset = *offset_ptr;
   m_parent_idx = 0;
   m_sibling_idx = 0;
   m_empty_children = false;
   const uint64_t abbr_idx = debug_info_data.GetULEB128(offset_ptr);
   assert(abbr_idx < (1 << DIE_ABBR_IDX_BITSIZE));

 //
 // Extract a debug info entry for a given compile unit from the .debug_info and
 // .debug_abbrev data within the SymbolFileDWARF class starting at the given
 // offset
 //----------------------------------------------------------------------
 bool DWARFDebugInfoEntry::Extract(SymbolFileDWARF *dwarf2Data,
                                   const DWARFUnit *cu,
                                   lldb::offset_t *offset_ptr) {
-  const DWARFDataExtractor &debug_info_data = cu->GetData();
+  const DWARFConcatenatingDataExtractor &debug_info_data = cu->GetData();
   //    const DWARFDataExtractor& debug_str_data =
   //    dwarf2Data->get_debug_str_data();
   const uint32_t cu_end_offset = cu->GetNextCompileUnitOffset();
   lldb::offset_t offset = *offset_ptr;
   //  if (offset >= cu_end_offset)
   //      Log::Status("DIE at offset 0x%8.8x is beyond the end of the current
   //      compile unit (0x%8.8x)", m_offset, cu_end_offset);
   if ((offset < cu_end_offset) && debug_info_data.ValidOffset(offset)) {
bool DWARFDebugInfoEntry::GetDIENamesAndRanges(
 
   lldb::offset_t offset;
   const DWARFAbbreviationDeclaration *abbrevDecl =
       GetAbbreviationDeclarationPtr(dwarf2Data, cu, offset);
 
   lldb::ModuleSP module = dwarf2Data->GetObjectFile()->GetModule();
 
   if (abbrevDecl) {
-    const DWARFDataExtractor &debug_info_data = cu->GetData();
+    const DWARFConcatenatingDataExtractor &debug_info_data = cu->GetData();
 
     if (!debug_info_data.ValidOffset(offset))
       return false;
 
     const uint32_t numAttributes = abbrevDecl->NumAttributes();
     uint32_t i;
     dw_attr_t attr;
     dw_form_t form;
for (i = 0; i < numAttributes; ++i) {
         case DW_AT_call_column:
           if (call_column == 0)
             call_column = form_value.Unsigned();
           break;
 
         case DW_AT_frame_base:
           if (frame_base) {
             if (form_value.BlockData()) {
-              uint32_t block_offset =
-                  form_value.BlockData() - debug_info_data.GetDataStart();
+              const DWARFDataExtractor &block_extractor =
+                  debug_info_data.GetDWARFDataExtractor(form_value.BlockData());
+              uint32_t block_offset = form_value.BlockData()
+                  - block_extractor.GetDataStart();
               uint32_t block_length = form_value.Unsigned();
-              frame_base->SetOpcodeData(module, debug_info_data, block_offset,
+              frame_base->SetOpcodeData(module, block_extractor, block_offset,
                                         block_length);
             } else {
               const DWARFDataExtractor &debug_loc_data =
                   dwarf2Data->get_debug_loc_data();
               const dw_offset_t debug_loc_offset = form_value.Unsigned();
 
               size_t loc_list_length = DWARFExpression::LocationListSize(
                   cu, debug_loc_data, debug_loc_offset);

 // Dump
 //
 // Dumps a debug information entry and all of it's attributes to the specified
 // stream.
 //----------------------------------------------------------------------
 void DWARFDebugInfoEntry::Dump(SymbolFileDWARF *dwarf2Data,
                                const DWARFUnit *cu, Stream &s,
                                uint32_t recurse_depth) const {
-  const DWARFDataExtractor &debug_info_data = cu->GetData();
+  const DWARFConcatenatingDataExtractor &debug_info_data = cu->GetData();
   lldb::offset_t offset = m_offset;
 
   if (debug_info_data.ValidOffset(offset)) {
     dw_uleb128_t abbrCode = debug_info_data.GetULEB128(&offset);
 
     s.Printf("\n0x%8.8x: ", m_offset);
     s.Indent();
     if (abbrCode != m_abbr_idx) {
if (abbrCode != m_abbr_idx) {
         // Dump all data in the .debug_info for the attributes
         const uint32_t numAttributes = abbrevDecl->NumAttributes();
         uint32_t i;
         dw_attr_t attr;
         dw_form_t form;
         for (i = 0; i < numAttributes; ++i) {
           abbrevDecl->GetAttrAndFormByIndexUnchecked(i, attr, form);
 
-          DumpAttribute(dwarf2Data, cu, debug_info_data, &offset, s, attr,
-                        form);
+          DumpAttribute(dwarf2Data, cu,
+                        debug_info_data.GetDWARFDataExtractor(offset), &offset,
+                        s, attr, form);
         }
 
         const DWARFDebugInfoEntry *child = GetFirstChild();
         if (recurse_depth > 0 && child) {
           s.IndentMore();
 
           while (child) {
             child->Dump(dwarf2Data, cu, s, recurse_depth - 1);
if (m_tag != DW_TAG_compile_unit && m_tag != DW_TAG_partial_unit) {
                              fixed_form_sizes, attributes, curr_depth);
     }
 
     dwarf2Data = cu->GetSymbolFileDWARF();
     abbrevDecl = GetAbbreviationDeclarationPtr(dwarf2Data, cu, offset);
   }
 
   if (abbrevDecl) {
-    const DWARFDataExtractor &debug_info_data = cu->GetData();
+    const DWARFConcatenatingDataExtractor &debug_info_data = cu->GetData();
 
     if (fixed_form_sizes.Empty())
       fixed_form_sizes = DWARFFormValue::GetFixedFormSizesForAddressSize(
           cu->GetAddressByteSize(), cu->IsDWARF64());
 
     const uint32_t num_attributes = abbrevDecl->NumAttributes();
     uint32_t i;
     dw_attr_t attr;
dw_offset_t DWARFDebugInfoEntry::GetAttributeValue(
   lldb::offset_t offset;
   const DWARFAbbreviationDeclaration *abbrevDecl =
       GetAbbreviationDeclarationPtr(dwarf2Data, cu, offset);
 
   if (abbrevDecl) {
     uint32_t attr_idx = abbrevDecl->FindAttributeIndex(attr);
 
     if (attr_idx != DW_INVALID_INDEX) {
-      const DWARFDataExtractor &debug_info_data = cu->GetData();
+      const DWARFConcatenatingDataExtractor &debug_info_data = cu->GetData();
 
       uint32_t idx = 0;
       while (idx < attr_idx)
         DWARFFormValue::SkipValue(abbrevDecl->GetFormByIndex(idx++),
                                   debug_info_data, &offset, cu);
 
       const dw_offset_t attr_offset = offset;
       form_value.SetCompileUnit(cu);

source/Plugins/SymbolFile/DWARF/DWARFFormValue.h

 //===-- DWARFFormValue.h ----------------------------------------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef SymbolFileDWARF_DWARFFormValue_h_
 #define SymbolFileDWARF_DWARFFormValue_h_
 
 #include "DWARFDIE.h"
-#include "DWARFDataExtractor.h"
+#include "DWARFConcatenatingDataExtractor.h"
 #include <stddef.h> // for NULL
 
 class DWARFUnit;
 
 class DWARFFormValue {
 public:
   typedef struct ValueTypeTag {
     ValueTypeTag() : value(), data(NULL) { value.uval = 0; }
public:
   DWARFFormValue();
   DWARFFormValue(const DWARFUnit *cu, dw_form_t form);
   const DWARFUnit *GetCompileUnit() const { return m_cu; }
   void SetCompileUnit(const DWARFUnit *cu) { m_cu = cu; }
   dw_form_t Form() const { return m_form; }
   void SetForm(dw_form_t form) { m_form = form; }
   const ValueType &Value() const { return m_value; }
   void Dump(lldb_private::Stream &s) const;
-  bool ExtractValue(const lldb_private::DWARFDataExtractor &data,
+  bool ExtractValue(const lldb_private::DWARFConcatenatingDataExtractor &data,
                     lldb::offset_t *offset_ptr);
   const uint8_t *BlockData() const;
   uint64_t Reference() const;
   uint64_t Reference(dw_offset_t offset) const;
   DWARFDIE GetTypeSignatureDIE() const;
   bool Boolean() const { return m_value.value.uval != 0; }
   uint64_t Unsigned() const { return m_value.value.uval; }
   void SetUnsigned(uint64_t uval) { m_value.value.uval = uval; }
   int64_t Signed() const { return m_value.value.sval; }
   void SetSigned(int64_t sval) { m_value.value.sval = sval; }
   const char *AsCString() const;
   dw_addr_t Address() const;
   bool IsValid() const { return m_form != 0; }
-  bool SkipValue(const lldb_private::DWARFDataExtractor &debug_info_data,
-                 lldb::offset_t *offset_ptr) const;
-  static bool SkipValue(const dw_form_t form,
-                        const lldb_private::DWARFDataExtractor &debug_info_data,
-                        lldb::offset_t *offset_ptr, const DWARFUnit *cu);
+  bool SkipValue(
+      const lldb_private::DWARFConcatenatingDataExtractor &debug_info_data,
+  static bool SkipValue(
+      const dw_form_t form,
+      const lldb_private::DWARFConcatenatingDataExtractor &debug_info_data,
   static bool IsBlockForm(const dw_form_t form);
   static bool IsDataForm(const dw_form_t form);
   static FixedFormSizes GetFixedFormSizesForAddressSize(uint8_t addr_size,
                                                         bool is_dwarf64);
   static int Compare(const DWARFFormValue &a, const DWARFFormValue &b);
   void Clear();
   static bool FormIsSupported(dw_form_t form);
 
 protected:
   const DWARFUnit *m_cu;        // Compile unit for this form
   dw_form_t m_form;             // Form for this value
   ValueType m_value;            // Contains all data for the form
 };
 
 #endif // SymbolFileDWARF_DWARFFormValue_h_

source/Plugins/SymbolFile/DWARF/DWARFFormValue.cpp

DWARFFormValue::DWARFFormValue(const DWARFUnit *cu, dw_form_t form)
     : m_cu(cu), m_form(form), m_value() {}
 
 void DWARFFormValue::Clear() {
   m_cu = nullptr;
   m_form = 0;
   memset(&m_value, 0, sizeof(m_value));
 }
 
-bool DWARFFormValue::ExtractValue(const DWARFDataExtractor &data,
+bool DWARFFormValue::ExtractValue(const DWARFConcatenatingDataExtractor &data,
                                   lldb::offset_t *offset_ptr) {
   bool indirect = false;
   bool is_block = false;
   m_value.data = NULL;
   uint8_t ref_addr_size;
   // Read the value for the form into value and follow and DW_FORM_indirect
   // instances we run into
   do {
if (is_block) {
     if (m_value.data != NULL) {
       *offset_ptr += m_value.value.uval;
     }
   }
 
   return true;
 }
 
-bool DWARFFormValue::SkipValue(const DWARFDataExtractor &debug_info_data,
-                               lldb::offset_t *offset_ptr) const {
+bool DWARFFormValue::SkipValue(
+    const DWARFConcatenatingDataExtractor &debug_info_data,
   return DWARFFormValue::SkipValue(m_form, debug_info_data, offset_ptr, m_cu);
 }
 
-bool DWARFFormValue::SkipValue(dw_form_t form,
-                               const DWARFDataExtractor &debug_info_data,
-                               lldb::offset_t *offset_ptr,
-                               const DWARFUnit *cu) {
+bool DWARFFormValue::SkipValue(
+    dw_form_t form, const DWARFConcatenatingDataExtractor &debug_info_data,
+    lldb::offset_t *offset_ptr, const DWARFUnit *cu) {
   uint8_t ref_addr_size;
   switch (form) {
   // Blocks if inlined data that have a length field and the data bytes inlined
   // in the .debug_info
   case DW_FORM_exprloc:
   case DW_FORM_block: {
     dw_uleb128_t size = debug_info_data.GetULEB128(offset_ptr);
     *offset_ptr += size;

source/Plugins/SymbolFile/DWARF/DWARFTypeUnit.h

 #include "DWARFUnit.h"
 
 class DWARFTypeUnit : public DWARFUnit {
   friend class DWARFUnit;
 
 public:
   virtual ~DWARFTypeUnit();
 
-  static DWARFTypeUnitSP Extract(SymbolFileDWARF *dwarf2Data,
-                                 const lldb_private::DWARFDataExtractor &data,
-                                 lldb::offset_t *offset_ptr);
+  static DWARFTypeUnitSP Extract(
+      SymbolFileDWARF *dwarf2Data,
+      const lldb_private::DWARFConcatenatingDataExtractor &data,
 
   void Dump(lldb_private::Stream *s) const override;
 
   //------------------------------------------------------------------
-  /// Get the data that contains the DIE information for this unit.
-  ///
-  /// @return
-  ///   The correct data (.debug_types for DWARF 4 and earlier, and
-  ///   .debug_info for DWARF 5 and later) for the DIE information in
-  ///   this unit.
-  //------------------------------------------------------------------
-  const lldb_private::DWARFDataExtractor &GetData() const override;
-  
-
-  //------------------------------------------------------------------
   /// Get the size in bytes of the header.
   ///
   /// @return
   ///     Byte size of the type unit header
   //------------------------------------------------------------------
   uint32_t GetHeaderByteSize() const override {
     return m_is_dwarf64 ? 39 : 23;
   }

source/Plugins/SymbolFile/DWARF/DWARFTypeUnit.cpp

s->Printf("0x%8.8x: Type Unit: length = 0x%8.8x, version = 0x%4.4x, "
             "abbr_offset = 0x%8.8x, addr_size = 0x%2.2x, "
             "type_signature = 0x%16.16" PRIx64 ", type_offset = 0x%8.8x "
             "(next CU at {0x%8.8x})\n",
             m_offset, m_length, m_version, GetAbbrevOffset(), m_addr_size,
             m_type_signature, m_type_offset, GetNextCompileUnitOffset());
 }
 
 DWARFTypeUnitSP DWARFTypeUnit::Extract(SymbolFileDWARF *dwarf,
-    const lldb_private::DWARFDataExtractor &data, lldb::offset_t *offset_ptr) {
+    const DWARFConcatenatingDataExtractor &data, lldb::offset_t *offset_ptr) {
   // std::make_shared would require the ctor to be public.
   std::shared_ptr<DWARFTypeUnit> cu_sp(new DWARFTypeUnit(dwarf));
   // Out of memory?
   if (!cu_sp)
     return nullptr;
   if (cu_sp->ExtractHeader(dwarf, data, offset_ptr)) {
     cu_sp->m_type_signature = data.GetU64(offset_ptr);
     cu_sp->m_type_offset = data.GetDWARFOffset(offset_ptr);
     return cu_sp;
   }
   // reset the offset to where we tried to parse from if anything went wrong
   *offset_ptr = cu_sp->m_offset;
   return nullptr;
 }
 
 DWARFDIE DWARFTypeUnit::GetTypeUnitDIE() {
   // The type offset is compile unit relative, so we need to add the compile
   // unit offset to ensure we get the correct DIE.
   return GetDIE(GetTypeUnitDIEOffset());
 }
-
-const lldb_private::DWARFDataExtractor &DWARFTypeUnit::GetData() const {
-  // In DWARF 5, type units are in the .debug_info section. Prior to DWARF 5
-  // type units are in the .debug_types section.
-  if (GetVersion() < 5)
-    return m_dwarf->get_debug_types_data();
-  else
-    return m_dwarf->get_debug_info_data();
-}
-

source/Plugins/SymbolFile/DWARF/DWARFUnit.h

 class DWARFUnit {
   using die_iterator_range =
       llvm::iterator_range<DWARFDebugInfoEntry::collection::iterator>;
 
 public:
   virtual ~DWARFUnit();
 
   bool ExtractHeader(SymbolFileDWARF *dwarf,
-                     const lldb_private::DWARFDataExtractor &data,
+                     const lldb_private::DWARFConcatenatingDataExtractor &data,
                      lldb::offset_t *offset_ptr);
 
   DWARFTypeUnit *GetAsTypeUnit();
   DWARFCompileUnit *GetAsCompileUnit();
   void ExtractUnitDIEIfNeeded();
   void ExtractDIEsIfNeeded();
 
   class ScopedExtractDIEs {
public:
   ///
   /// This will return the correct bytes that contain the data for
   /// this DWARFUnit. It could be .debug_info or .debug_types
   /// depending on where the data for this unit originates.
   ///
   /// @return
   ///   The correct data for the DIE information in this unit.
   //------------------------------------------------------------------
-  virtual const lldb_private::DWARFDataExtractor &GetData() const = 0;
+  const lldb_private::DWARFConcatenatingDataExtractor &GetData() const;
   //------------------------------------------------------------------
   /// Get the size in bytes of the compile unit header.
   ///
   /// @return
   ///     Byte size of the compile unit header
   //------------------------------------------------------------------
   virtual uint32_t GetHeaderByteSize() const = 0;
   // Offset of the initial length field.

source/Plugins/SymbolFile/DWARF/DWARFUnit.cpp

Timer scoped_timer(
       func_cat, "%8.8x: DWARFUnit::ExtractUnitDIEIfNeeded()", m_offset);
 
   // Set the offset to that of the first DIE and calculate the start of the
   // next compilation unit header.
   lldb::offset_t offset = GetFirstDIEOffset();
 
   // We are in our compile unit, parse starting at the offset we were told to
   // parse
-  const DWARFDataExtractor &data = GetData();
+  const DWARFConcatenatingDataExtractor &data = GetData();
   DWARFFormValue::FixedFormSizes fixed_form_sizes =
       DWARFFormValue::GetFixedFormSizesForAddressSize(GetAddressByteSize(),
                                                       IsDWARF64());
   if (offset < GetNextCompileUnitOffset() &&
       m_first_die.FastExtract(data, this, fixed_form_sizes, &offset)) {
     AddUnitDIE(m_first_die);
     return;
   }
m_dwarf->GetObjectFile()->GetModule()->LogMessageVerboseBacktrace(
         "DWARFUnit::ExtractDIEsIfNeeded () for compile unit at "
         ".debug_info[0x%8.8x]",
         GetOffset());
   }
 
   uint32_t depth = 0;
   // We are in our compile unit, parse starting at the offset we were told to
   // parse
-  const DWARFDataExtractor &data = GetData();
+  const DWARFConcatenatingDataExtractor &data = GetData();
   std::vector<uint32_t> die_index_stack;
   die_index_stack.reserve(32);
   die_index_stack.push_back(0);
   bool prev_die_had_children = false;
   DWARFFormValue::FixedFormSizes fixed_form_sizes =
       DWARFFormValue::GetFixedFormSizesForAddressSize(GetAddressByteSize(),
                                                       IsDWARF64());
   while (offset < next_cu_offset &&

 
 DWARFTypeUnit *DWARFUnit::GetAsTypeUnit() {
   if (GetUnitDIEOnly().Tag() == DW_TAG_type_unit)
     return static_cast<DWARFTypeUnit *>(this);
   return nullptr;
 }
 
 DWARFCompileUnit *DWARFUnit::GetAsCompileUnit() {
   if (GetUnitDIEOnly().Tag() == DW_TAG_compile_unit)
     return static_cast<DWARFCompileUnit *>(this);
   return nullptr;
 }
 
 bool
 DWARFUnit::ExtractHeader(SymbolFileDWARF *dwarf,
-                         const lldb_private::DWARFDataExtractor &data,
+                         const DWARFConcatenatingDataExtractor &data,
                          lldb::offset_t *offset_ptr) {
   m_offset = *offset_ptr;
 
   if (data.ValidOffset(*offset_ptr)) {
     dw_offset_t abbr_offset;
     const DWARFDebugAbbrev *abbr = dwarf->DebugAbbrev();
     m_length = data.GetDWARFInitialLength(offset_ptr);
-    m_is_dwarf64 = data.IsDWARF64();
+    m_is_dwarf64 = data.IsDWARF64(*offset_ptr);
     m_version = data.GetU16(offset_ptr);
     abbr_offset = data.GetDWARFOffset(offset_ptr);
     m_addr_size = data.GetU8(offset_ptr);
 
     bool length_OK = data.ValidOffset(GetNextCompileUnitOffset() - 1);
     bool version_OK = SymbolFileDWARF::SupportedVersion(m_version);
     bool abbr_offset_OK =
         dwarf->get_debug_abbrev_data().ValidOffset(abbr_offset);
     bool addr_size_OK = (m_addr_size == 4) || (m_addr_size == 8);
 
     if (length_OK && version_OK && addr_size_OK && abbr_offset_OK && abbr) {
       m_abbrevs = abbr->GetAbbreviationDeclarationSet(abbr_offset);
       return m_abbrevs != nullptr;
     }
 
     // reset the offset to where we tried to parse from if anything went wrong
     *offset_ptr = m_offset;
   }
   return false;
 }
 
+const lldb_private::DWARFConcatenatingDataExtractor &DWARFUnit::GetData() const {
+  return m_dwarf->get_debug_concatenated_data();
+}

source/Plugins/SymbolFile/DWARF/DebugNamesDWARFIndex.h

 //===-- DebugNamesDWARFIndex.h ---------------------------------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLDB_DEBUGNAMESDWARFINDEX_H
 #define LLDB_DEBUGNAMESDWARFINDEX_H
 
+#include "Plugins/SymbolFile/DWARF/DWARFDataExtractor.h"
 #include "Plugins/SymbolFile/DWARF/DWARFIndex.h"
 #include "Plugins/SymbolFile/DWARF/LogChannelDWARF.h"
 #include "Plugins/SymbolFile/DWARF/ManualDWARFIndex.h"
 #include "lldb/Utility/ConstString.h"
 #include "llvm/DebugInfo/DWARF/DWARFAcceleratorTable.h"
 
 namespace lldb_private {
 class DebugNamesDWARFIndex : public DWARFIndex {

source/Plugins/SymbolFile/DWARF/HashedNameToDIE.h

 
 #include <vector>
 
 #include "lldb/Core/MappedHash.h"
 #include "lldb/Core/dwarf.h"
 #include "lldb/Utility/RegularExpression.h"
 #include "lldb/lldb-defines.h"
 
+#include "DWARFDataExtractor.h"
 #include "DWARFDefines.h"
 #include "DWARFFormValue.h"
 #include "NameToDIE.h"
 
 class DWARFMappedHash {
 public:
   enum AtomType : uint16_t {
     eAtomTypeNULL = 0u,

source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.h

 #include "lldb/Expression/DWARFExpression.h"
 #include "lldb/Symbol/DebugMacros.h"
 #include "lldb/Symbol/SymbolContext.h"
 #include "lldb/Symbol/SymbolFile.h"
 #include "lldb/Utility/ConstString.h"
 #include "lldb/lldb-private.h"
 
 // Project includes
+#include "DWARFConcatenatingDataExtractor.h"
 #include "DWARFDataExtractor.h"
 #include "DWARFDefines.h"
 #include "DWARFIndex.h"
 #include "UniqueDWARFASTType.h"
 
 //----------------------------------------------------------------------
 // Forward Declarations for this DWARF plugin
 //----------------------------------------------------------------------
public:
   const lldb_private::DWARFDataExtractor &get_debug_str_offsets_data();
   const lldb_private::DWARFDataExtractor &get_debug_types_data();
   const lldb_private::DWARFDataExtractor &get_apple_names_data();
   const lldb_private::DWARFDataExtractor &get_apple_types_data();
   const lldb_private::DWARFDataExtractor &get_apple_namespaces_data();
   const lldb_private::DWARFDataExtractor &get_apple_objc_data();
   const lldb_private::DWARFDataExtractor &get_gnu_debugaltlink();
 
+  const lldb_private::DWARFConcatenatingDataExtractor &
+      get_debug_concatenated_data();
+
   DWARFDebugAbbrev *DebugAbbrev();
 
   const DWARFDebugAbbrev *DebugAbbrev() const;
 
   DWARFDebugInfo *DebugInfo();
 
   const DWARFDebugInfo *DebugInfo() const;
 
protected:
 
   lldb::ModuleWP m_debug_map_module_wp;
   SymbolFileDWARFDebugMap *m_debug_map_symfile;
 
   llvm::once_flag m_dwp_symfile_once_flag;
   std::unique_ptr<SymbolFileDWARFDwp> m_dwp_symfile;
 
   lldb_private::DWARFDataExtractor m_dwarf_data;
+  llvm::once_flag m_concatenated_data_once;
+  lldb_private::DWARFConcatenatingDataExtractor m_concatenated_data;
 
   DWARFDataSegment m_data_debug_abbrev;
   DWARFDataSegment m_data_debug_addr;
   DWARFDataSegment m_data_debug_aranges;
   DWARFDataSegment m_data_debug_frame;
   DWARFDataSegment m_data_debug_info;
   DWARFDataSegment m_data_debug_line;
   DWARFDataSegment m_data_debug_macro;

source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.cpp

if (debug_line_file_size > 0)
       abilities |= LineTables;
   }
   return abilities;
 }
 
 const DWARFDataExtractor &
 SymbolFileDWARF::GetCachedSectionData(lldb::SectionType sect_type,
                                       DWARFDataSegment &data_segment) {
-  llvm::call_once(data_segment.m_flag, [&] {
+  llvm::call_once(data_segment.m_flag, [this, sect_type, &data_segment] {
     this->LoadSectionData(sect_type, std::ref(data_segment.m_data));
-    if (sect_type == eSectionTypeDWARFDebugTypes) {
-      // To add .debug_types support in DWARF 4 and earlier with minimally
-      // invasive changes to the current DWARF parsing code, we pretend that
-      // any DIEs in .debug_types start at the end of the .debug_info section.
-      // All info in .debug_types is relative and has no external DIE
-      // references unless thay are DW_AT_signature references, so the DIE
-      // offset for things in the .debug_types. If we do this, then we can
-      // just add the type units to the compile units collection and treat all
-      // information just as we do for all other information in the DWARF and
-      // everything just works. If we were to try to split this out, we would
-      // end up having to change a TON of code. Also DWARF 5 will have compile
-      // and type units in the .debug_info, so coding it this way will prepare
-      // use for an easy transition to DWARF 5.
-      uint64_t debug_info_size = get_debug_info_data().GetByteSize();
-      data_segment.m_data.OffsetData(debug_info_size);
-    }
   });
   return data_segment.m_data;
 }
 
 void SymbolFileDWARF::LoadSectionData(lldb::SectionType sect_type,
                                       DWARFDataExtractor &data) {
   ModuleSP module_sp(m_obj_file->GetModule());
   const SectionList *section_list = module_sp->GetSectionList();
const DWARFDataExtractor &SymbolFileDWARF::get_apple_objc_data() {
   return GetCachedSectionData(eSectionTypeDWARFAppleObjC, m_data_apple_objc);
 }
 
 const DWARFDataExtractor &SymbolFileDWARF::get_gnu_debugaltlink() {
   return GetCachedSectionData(eSectionTypeDWARFGNUDebugAltLink,
                               m_data_gnu_debugaltlink);
 }
 
+const DWARFConcatenatingDataExtractor &
+    SymbolFileDWARF::get_debug_concatenated_data() {
+  llvm::call_once(m_concatenated_data_once, [&] {
+    m_concatenated_data.AppendDWARFDataExtractor(get_debug_info_data());
+    m_concatenated_data.AppendDWARFDataExtractor(get_debug_types_data());
+  });
+  return m_concatenated_data;
+}
+
 DWARFDebugAbbrev *SymbolFileDWARF::DebugAbbrev() {
   if (m_abbr.get() == NULL) {
     const DWARFDataExtractor &debug_abbrev_data = get_debug_abbrev_data();
     if (debug_abbrev_data.GetByteSize() > 0) {
       m_abbr.reset(new DWARFDebugAbbrev());
       if (m_abbr.get())
         m_abbr->Parse(debug_abbrev_data);
     }
if (num_attributes > 0) {
             // attribute.
             if (!has_explicit_location) {
               location_is_const_value_data = true;
               // The constant value will be either a block, a data value or a
               // string.
               auto debug_info_data = die.GetData();
               if (DWARFFormValue::IsBlockForm(form_value.Form())) {
                 // Retrieve the value as a block expression.
+                const DWARFDataExtractor &block_extractor =
+                    debug_info_data.GetDWARFDataExtractor(form_value.BlockData());
                 uint32_t block_offset =
-                    form_value.BlockData() - debug_info_data.GetDataStart();
+                    form_value.BlockData() - block_extractor.GetDataStart();
                 uint32_t block_length = form_value.Unsigned();
-                location.CopyOpcodeData(module, debug_info_data, block_offset,
+                location.CopyOpcodeData(module, block_extractor, block_offset,
                                         block_length);
               } else if (DWARFFormValue::IsDataForm(form_value.Form())) {
                 // Retrieve the value as a data expression.
                 DWARFFormValue::FixedFormSizes fixed_form_sizes =
                     DWARFFormValue::GetFixedFormSizesForAddressSize(
                         attributes.CompileUnitAtIndex(i)->GetAddressByteSize(),
                         attributes.CompileUnitAtIndex(i)->IsDWARF64());
                 uint32_t data_offset = attributes.DIEOffsetAtIndex(i);
                 uint32_t data_length =
                     fixed_form_sizes.GetSize(form_value.Form());
                 if (data_length == 0) {
                   const uint8_t *data_pointer = form_value.BlockData();
                   if (data_pointer) {
                     form_value.Unsigned();
                   } else if (DWARFFormValue::IsDataForm(form_value.Form())) {
                     // we need to get the byte size of the type later after we
                     // create the variable
                     const_value = form_value;
                   }
-                } else
-                  location.CopyOpcodeData(module, debug_info_data, data_offset,
+                } else {
+                  const DWARFDataExtractor &block_extractor =
+                      debug_info_data.GetDWARFDataExtractor(data_offset);
+                  location.CopyOpcodeData(module, block_extractor, data_offset,
                                           data_length);
+                }
               } else {
                 // Retrieve the value as a string expression.
                 if (form_value.Form() == DW_FORM_strp) {
                   DWARFFormValue::FixedFormSizes fixed_form_sizes =
                       DWARFFormValue::GetFixedFormSizesForAddressSize(
                           attributes.CompileUnitAtIndex(i)
                               ->GetAddressByteSize(),
                           attributes.CompileUnitAtIndex(i)->IsDWARF64());
                   uint32_t data_offset = attributes.DIEOffsetAtIndex(i);
+                  const DWARFDataExtractor &block_extractor =
+                      debug_info_data.GetDWARFDataExtractor(data_offset);
                   uint32_t data_length =
                       fixed_form_sizes.GetSize(form_value.Form());
-                  location.CopyOpcodeData(module, debug_info_data, data_offset,
+                  location.CopyOpcodeData(module, block_extractor, data_offset,
                                           data_length);
                 } else {
                   const char *str = form_value.AsCString();
+                  const DWARFDataExtractor &block_extractor =
+                      debug_info_data.GetDWARFDataExtractor(
+                          (const uint8_t *)str);
                   uint32_t string_offset =
-                      str - (const char *)debug_info_data.GetDataStart();
+                      str - (const char *)block_extractor.GetDataStart();
                   uint32_t string_length = strlen(str) + 1;
-                  location.CopyOpcodeData(module, debug_info_data,
+                  location.CopyOpcodeData(module, block_extractor,
                                           string_offset, string_length);
                 }
               }
             }
             break;
           case DW_AT_location: {
             location_is_const_value_data = false;
             has_explicit_location = true;
             if (DWARFFormValue::IsBlockForm(form_value.Form())) {
               auto data = die.GetData();
 
+              const DWARFDataExtractor &block_extractor =
+                  data.GetDWARFDataExtractor(form_value.BlockData());
               uint32_t block_offset =
-                  form_value.BlockData() - data.GetDataStart();
+                  form_value.BlockData() - block_extractor.GetDataStart();
               uint32_t block_length = form_value.Unsigned();
-              location.CopyOpcodeData(module, data, block_offset, block_length);
+              location.CopyOpcodeData(
+      module, block_extractor, block_offset, block_length);
             } else {
               const DWARFDataExtractor &debug_loc_data = get_debug_loc_data();
               const dw_offset_t debug_loc_offset = form_value.Unsigned();
 
               size_t loc_list_length = DWARFExpression::LocationListSize(
                   die.GetCU(), debug_loc_data, debug_loc_offset);
               if (loc_list_length > 0) {
                 location.CopyOpcodeData(module, debug_loc_data,