[dsymutil] Implement DWARF5 accelerator tables.

[JDevlieghere]

Summary

This patch add support for emitting DWARF5 accelerator tables
(.debug_names) from dsymutil. Just as with the Apple style accelerator
tables, it's possible to update existing dSYMs. This patch includes a
test that show how you can convert back and forth between the two types.

[JDevlieghere]

• Add default option that uses accelerator table kind from input.

[labath]

Are you worried about the case where the dSYM bundle contains CUs with different dwarf versions?

ObjC code in DWARF 4 compile units will not work correctly because of the emit-objc-methods-in-classes issue. Assuming this is a corner case which you don't-want-to-support-much-but-it-would-still-be-good-if-it-works, a better solution would be to just not put these kinds of units into the index. This way, the debugger will be slower as it will have to index them manually, but it will at least be correct.

But if you say this is good enough for you, then I'm fine with that too...

[JDevlieghere]

Update logic for picking the default accelerator table kind:

If we've only seen either Apple or DWARF accelerator tables, we pick the
respective one. If we've seen neither or both, the choice is determined
based on the lower DWARF version encountered.

[labath]

In D49137#1164857, @JDevlieghere wrote:

Update logic for picking the default accelerator table kind:

If we've only seen either Apple or DWARF accelerator tables, we pick the
respective one. If we've seen neither or both, the choice is determined
based on the lower DWARF version encountered.

The new behavior makes more sense.

Apart from the out of date comment, this looks good to me (though I can't say I know much about how dsymutil works internally).

llvm/tools/dsymutil/DwarfLinker.h

164–166 ↗

(On Diff #155851)

I don't know why you deleted the other clear statement, but in any case, the comment is now out of date.

[JDevlieghere]

• Update outdated comment

[JDevlieghere]

llvm/tools/dsymutil/DwarfLinker.h

164–166 ↗

(On Diff #155851)

We can't clear the CUs yet as we need them during emission of the debug_names section to map DIEs to CUs.

[JDevlieghere]

@friss Do you want to have a look before I land this?

[friss]

maybeUpdateAccelKind() will make a final call about the kind of accelerator tables on the first object it encounters which has accelerator tables. This seems pretty arbitrary. I'd prefer if we considered Apple table the default unless all files are Dwarf5 and have Dwarf5 accelerator tables.
Other than that, the implementation looks good. I'd like to know how the LLDB test suite behaves if you force it to use the new accelerator tables.

[friss]

In D49137#1165178, @friss wrote:

maybeUpdateAccelKind() will make a final call about the kind of accelerator tables on the first object it encounters which has accelerator tables. This seems pretty arbitrary. I'd prefer if we considered Apple table the default unless all files are Dwarf5 and have Dwarf5 accelerator tables.
Other than that, the implementation looks good. I'd like to know how the LLDB test suite behaves if you force it to use the new accelerator tables.

Looks like I should have updated the review before going through the code... Let me take another look

[friss]

llvm/tools/dsymutil/DwarfLinker.cpp

2094 ↗

(On Diff #155873)

Why would we need to update the dwarf versions, but not the Accelerator type here?

2305–2311 ↗

(On Diff #155873)

This new iteration is needed because of the registerModuleReference call bellow, right? I'm not sure this is sufficient as I think the registerModuleReference can pull in new debug information from dependencies of the referenced modules.

2313–2327 ↗

(On Diff #155873)

I'd rather we are even more conservative. I'd prefer if we chose Dwarf5 tables only if all the input is Dwarf5 and none of it has Apple tables.

llvm/tools/dsymutil/DwarfLinker.h

164–166 ↗

(On Diff #155851)

What kind of memory does the CU hold? Do we see an increase in overall memory consumption?

[JDevlieghere]

Address code review feedback @friss.

The problem determining the accelerator kind in the default case is that registerModuleReference potentially imports new compile units, which might affect the dwarf version and accelerator table kind. If this is the case, we clone all compile units which means adding entries to the accelerator table. This is problematic because the next such module might affect the decision of which kind of accelerator table to use. I think there's several solutions to this problem:

1. Ignore this and don't let referenced modules affect the decision.
2. Don't clone the CUs at that point would remove the need to populate the tables. This would require restructuring quite a bit of code, and also means keeping unnecessary stuff in memory longer. (e.g. it's CU list, DwarfContext, etc)
3. Have a generic data structure that we can later convert in either an Apple or Dwarf accelerator table. This would require copying info from one struct to another, and we'd need a way to differentiate between what goes where. For the Apple case information will end up in separate tables, while for DWARF there's only one. We also designed the accelerator tables in such a way that they were configurable in their type, by going through another type we'd essentially nullify that effort.
4. Populate both tables as long as we're in limbo about the kind. This means we're duplicating information in memory, but only for modules. As soon as we know the kind we clear the other tables and reclaim the memory.

In this patch I've implemented (4) because it was relatively straightforward and only affects CUs imported by modules.

[JDevlieghere]

In D49137#1165178, @friss wrote:

maybeUpdateAccelKind() will make a final call about the kind of accelerator tables on the first object it encounters which has accelerator tables. This seems pretty arbitrary. I'd prefer if we considered Apple table the default unless all files are Dwarf5 and have Dwarf5 accelerator tables.
Other than that, the implementation looks good. I'd like to know how the LLDB test suite behaves if you force it to use the new accelerator tables.

Failing Tests (2):

lldb-Suite :: python_api/class_members/TestSBTypeClassMembers.py
lldb-Suite :: tools/lldb-server/TestAppleSimulatorOSType.py

The python_api is expected, as it also failed before my change in clang to emit the members as children of the DIE (which is missing in this run, therefore making the failure expected). The last failure was unrelated, as all categories failed.

llvm/tools/dsymutil/DwarfLinker.cpp

2305–2311 ↗

(On Diff #155873)

Correct. The issue is that loadClangModule calls cloneAllCompileUnits which in turn wants to emitAcceleratorEntriesForUnit. Hence at that point we need to have what type of accelerator table we are going to emit, otherwise we might change the decision after the fact. Please see my latest update of the patch for all the details.

2313–2327 ↗

(On Diff #155873)

This is now the case in the latest revision of this patch. As soon as we see an Apple accelerator table we update the default to that.

llvm/tools/dsymutil/DwarfLinker.h

164–166 ↗

(On Diff #155851)

Instead of keeping the whole CU we now only tack the necessary info in EmittedCU (a pointer and an unsigned).

[JDevlieghere]

• Ignore modules for now as discussed offline.
• Document this decision.

[friss]

This LGTM

llvm/trunk/test/tools/dsymutil/X86/accelerator.test

+RUN: dsymutil -accelerator=Dwarf -oso-prepend-path=%p/.. %p/../Inputs/basic.macho.x86_64 -o %t.dwarf.dSYM
+RUN: dsymutil -accelerator=Apple -oso-prepend-path=%p/.. %p/../Inputs/basic.macho.x86_64 -o %t.apple.dSYM
+
+RUN: llvm-dwarfdump -verify %t.dwarf.dSYM
+RUN: llvm-dwarfdump -verify %t.apple.dSYM
+
+RUN: llvm-dwarfdump -debug-names %t.dwarf.dSYM | FileCheck %s -check-prefix=NAMES -check-prefix=DWARF
+RUN: llvm-dwarfdump -apple-names -apple-namespaces -apple-types %t.apple.dSYM | FileCheck %s -check-prefix=NAMES -check-prefix=APPLE
+
+RUN: dsymutil -update -accelerator=Dwarf %t.apple.dSYM -o %t.dwarf.updated.dSYM
+RUN: dsymutil -update -accelerator=Apple %t.dwarf.dSYM -o %t.apple.updated.dSYM
+
+RUN: llvm-dwarfdump -verify %t.dwarf.updated.dSYM
+RUN: llvm-dwarfdump -verify %t.apple.updated.dSYM
+
+RUN: llvm-dwarfdump -debug-names %t.dwarf.updated.dSYM | FileCheck %s -check-prefix=NAMES -check-prefix=DWARF
+RUN: llvm-dwarfdump -apple-names -apple-namespaces -apple-types %t.apple.updated.dSYM | FileCheck %s -check-prefix=NAMES -check-prefix=APPLE
+
+DWARF: .debug_names contents:
+DWARF: Compilation Unit offsets [
+DWARF:   CU[0]
+DWARF:   CU[1]
+DWARF:   CU[2]
+DWARF: ]
+
+APPLE-DAG: .apple_names contents:
+APPLE-DAG: .apple_types contents:
+APPLE-DAG: .apple_namespaces contents:
+
+NAMES-DAG: "private_int"
+NAMES-DAG: "baz"
+NAMES-DAG: "int"
+NAMES-DAG: "bar"
+NAMES-DAG: "foo"
+NAMES-DAG: "inc"
+NAMES-DAG: "val"
+NAMES-DAG: "main"
+NAMES-DAG: "char"

llvm/trunk/test/tools/dsymutil/cmdline.test

 RUN: dsymutil -help 2>&1 | FileCheck --check-prefix=HELP %s
 HELP: OVERVIEW: manipulate archived DWARF debug symbol files.
 HELP: USAGE: dsymutil{{[^ ]*}} [options] <input files>
 HELP-NOT: -reverse-iterate
 HELP: Color Options
 HELP: -color
 HELP: Specific Options:
+HELP: -accelerator
 HELP: -arch=<arch>
 HELP: -dump-debug-map
 HELP: -flat
 HELP: -minimize
 HELP: -no-odr
 HELP: -no-output
 HELP: -no-swiftmodule-timestamp
 HELP: -num-threads=<n>

llvm/trunk/tools/dsymutil/DwarfLinker.h

public:
 
   /// Link the contents of the DebugMap.
   bool link(const DebugMap &);
 
   void reportWarning(const Twine &Warning, const DebugMapObject &DMO,
                      const DWARFDie *DIE = nullptr) const;
 
 private:
-  /// Remembers the newest DWARF version we've seen in a unit.
-  void maybeUpdateMaxDwarfVersion(unsigned Version) {
-    if (MaxDwarfVersion < Version)
-      MaxDwarfVersion = Version;
+  /// Remembers the oldest and newest DWARF version we've seen in a unit.
+  void updateDwarfVersion(unsigned Version) {
+    MaxDwarfVersion = std::max(MaxDwarfVersion, Version);
+    MinDwarfVersion = std::min(MinDwarfVersion, Version);
   }
 
+  /// Remembers the kinds of accelerator tables we've seen in a unit.
+  void updateAccelKind(DWARFContext &Dwarf);
+
   /// Emit warnings as Dwarf compile units to leave a trail after linking.
   bool emitPaperTrailWarnings(const DebugMapObject &DMO, const DebugMap &Map,
                               OffsetsStringPool &StringPool);
 
   /// Keeps track of relocations.
   class RelocationManager {
     struct ValidReloc {
       uint32_t Offset;
LinkContext(const DebugMap &Map, DwarfLinker &Linker, DebugMapObject &DMO)
         ObjectFile = nullptr;
         return;
       }
       auto ErrOrObj = Linker.loadObject(DMO, Map);
       ObjectFile = ErrOrObj ? &*ErrOrObj : nullptr;
       DwarfContext = ObjectFile ? DWARFContext::create(*ObjectFile) : nullptr;
     }
 
-    /// Clear compile units and ranges.
+    /// Clear part of the context that's no longer needed when we're done with
+    /// the debug object.
     void Clear() {
+      DwarfContext.reset(nullptr);
       CompileUnits.clear();
       Ranges.clear();
     }
   };
 
   /// Called at the start of a debug object link.
   void startDebugObject(LinkContext &Context);
 
private:
   /// Extract the line tables from the original dwarf, extract the relevant
   /// parts according to the linked function ranges and emit the result in the
   /// debug_line section.
   void patchLineTableForUnit(CompileUnit &Unit, DWARFContext &OrigDwarf,
                              RangesTy &Ranges, const DebugMapObject &DMO);
 
   /// Emit the accelerator entries for \p Unit.
   void emitAcceleratorEntriesForUnit(CompileUnit &Unit);
+  void emitDwarfAcceleratorEntriesForUnit(CompileUnit &Unit);
+  void emitAppleAcceleratorEntriesForUnit(CompileUnit &Unit);
 
   /// Patch the frame info for an object file and emit it.
   void patchFrameInfoForObject(const DebugMapObject &, RangesTy &Ranges,
                                DWARFContext &, unsigned AddressSize);
 
   /// FoldingSet that uniques the abbreviations.
   FoldingSet<DIEAbbrev> AbbreviationsSet;
 
ErrorOr<const object::ObjectFile &> loadObject(const DebugMapObject &Obj,
                                                  const DebugMap &Map);
   /// @}
 
   raw_fd_ostream &OutFile;
   BinaryHolder &BinHolder;
   LinkOptions Options;
   std::unique_ptr<DwarfStreamer> Streamer;
   uint64_t OutputDebugInfoSize;
+
   unsigned MaxDwarfVersion = 0;
+  unsigned MinDwarfVersion = std::numeric_limits<unsigned>::max();
+
+  bool AtLeastOneAppleAccelTable = false;
+  bool AtLeastOneDwarfAccelTable = false;
 
   /// The CIEs that have been emitted in the output section. The actual CIE
   /// data serves a the key to this StringMap, this takes care of comparing the
   /// semantics of CIEs defined in different object files.
   StringMap<uint32_t> EmittedCIEs;
 
   /// Offset of the last CIE that has been emitted in the output
   /// debug_frame section.
   uint32_t LastCIEOffset = 0;
 
   /// Apple accelerator tables.
+  AccelTable<DWARF5AccelTableStaticData> DebugNames;
   AccelTable<AppleAccelTableStaticOffsetData> AppleNames;
   AccelTable<AppleAccelTableStaticOffsetData> AppleNamespaces;
   AccelTable<AppleAccelTableStaticOffsetData> AppleObjc;
   AccelTable<AppleAccelTableStaticTypeData> AppleTypes;
 
   /// Mapping the PCM filename to the DwoId.
   StringMap<uint64_t> ClangModules;
 
   bool ModuleCacheHintDisplayed = false;
   bool ArchiveHintDisplayed = false;
 };
 
 } // end namespace dsymutil
 } // end namespace llvm
 
 #endif // LLVM_TOOLS_DSYMUTIL_DWARFLINKER_H

llvm/trunk/tools/dsymutil/DwarfLinker.cpp

if (Mapping.Size && Mapping.ObjectAddress)
       Context.Ranges[*Mapping.ObjectAddress] = DebugMapObjectRange(
           *Mapping.ObjectAddress + Mapping.Size,
           int64_t(Mapping.BinaryAddress) - *Mapping.ObjectAddress);
   }
 }
 
 void DwarfLinker::endDebugObject(LinkContext &Context) {
   Context.Clear();
+
   for (auto I = DIEBlocks.begin(), E = DIEBlocks.end(); I != E; ++I)
     (*I)->~DIEBlock();
   for (auto I = DIELocs.begin(), E = DIELocs.end(); I != E; ++I)
     (*I)->~DIELoc();
 
   DIEBlocks.clear();
   DIELocs.clear();
   DIEAlloc.Reset();
else {
     Streamer->emitLineTableForUnit(Params,
                                    LineData.slice(*StmtList + 4, PrologueEnd),
                                    LineTable.Prologue.MinInstLength, NewRows,
                                    Unit.getOrigUnit().getAddressByteSize());
   }
 }
 
 void DwarfLinker::emitAcceleratorEntriesForUnit(CompileUnit &Unit) {
+  switch (Options.TheAccelTableKind) {
+  case AccelTableKind::Apple:
+    emitAppleAcceleratorEntriesForUnit(Unit);
+    break;
+  case AccelTableKind::Dwarf:
+    emitDwarfAcceleratorEntriesForUnit(Unit);
+    break;
+  case AccelTableKind::Default:
+    llvm_unreachable("The default must be updated to a concrete value.");
+    break;
+  }
+}
+
+void DwarfLinker::emitAppleAcceleratorEntriesForUnit(CompileUnit &Unit) {
   // Add namespaces.
   for (const auto &Namespace : Unit.getNamespaces())
     AppleNamespaces.addName(Namespace.Name,
                             Namespace.Die->getOffset() + Unit.getStartOffset());
 
   /// Add names.
   if (!Options.Minimize)
     Streamer->emitPubNamesForUnit(Unit);
AppleTypes.addName(
                                         : 0,
         Pubtype.QualifiedNameHash);
 
   /// Add ObjC names.
   for (const auto &ObjC : Unit.getObjC())
     AppleObjc.addName(ObjC.Name, ObjC.Die->getOffset() + Unit.getStartOffset());
 }
 
+void DwarfLinker::emitDwarfAcceleratorEntriesForUnit(CompileUnit &Unit) {
+  for (const auto &Namespace : Unit.getNamespaces())
+    DebugNames.addName(Namespace.Name, Namespace.Die->getOffset(),
+                       Namespace.Die->getTag(), Unit.getUniqueID());
+  for (const auto &Pubname : Unit.getPubnames())
+    DebugNames.addName(Pubname.Name, Pubname.Die->getOffset(),
+                       Pubname.Die->getTag(), Unit.getUniqueID());
+  for (const auto &Pubtype : Unit.getPubtypes())
+    DebugNames.addName(Pubtype.Name, Pubtype.Die->getOffset(),
+                       Pubtype.Die->getTag(), Unit.getUniqueID());
+}
+
 /// Read the frame info stored in the object, and emit the
 /// patched frame descriptions for the linked binary.
 ///
 /// This is actually pretty easy as the data of the CIEs and FDEs can
 /// be considered as black boxes and moved as is. The only thing to do
 /// is to patch the addresses in the headers.
 void DwarfLinker::patchFrameInfoForObject(const DebugMapObject &DMO,
                                           RangesTy &Ranges,
if (!ErrOrObj) {
     return Error::success();
   }
 
   std::unique_ptr<CompileUnit> Unit;
 
   // Setup access to the debug info.
   auto DwarfContext = DWARFContext::create(*ErrOrObj);
   RelocationManager RelocMgr(*this);
-  for (const auto &CU : DwarfContext->compile_units()) {
-    maybeUpdateMaxDwarfVersion(CU->getVersion());
 
+  for (const auto &CU : DwarfContext->compile_units()) {
+    updateDwarfVersion(CU->getVersion());
     // Recursively get all modules imported by this one.
     auto CUDie = CU->getUnitDIE(false);
     if (!CUDie)
       continue;
     if (!registerModuleReference(CUDie, *CU, ModuleMap, DMO, Ranges, StringPool,
                                  UniquingStringPool, ODRContexts, UnitID,
                                  Indent)) {
       if (Unit) {
for (auto &CurrentUnit : CompileUnits) {
     CurrentUnit->fixupForwardReferences();
     Linker.Streamer->emitCompileUnitHeader(*CurrentUnit);
     if (!CurrentUnit->getOutputUnitDIE())
       continue;
     Linker.Streamer->emitDIE(*CurrentUnit->getOutputUnitDIE());
   }
 }
 
+void DwarfLinker::updateAccelKind(DWARFContext &Dwarf) {
+  if (Options.TheAccelTableKind != AccelTableKind::Default)
+    return;
+
+  auto &DwarfObj = Dwarf.getDWARFObj();
+
+  if (!AtLeastOneDwarfAccelTable &&
+      (!DwarfObj.getAppleNamesSection().Data.empty() ||
+       !DwarfObj.getAppleTypesSection().Data.empty() ||
+       !DwarfObj.getAppleNamespacesSection().Data.empty() ||
+       !DwarfObj.getAppleObjCSection().Data.empty())) {
+    AtLeastOneAppleAccelTable = true;
+  }
+
+  if (!AtLeastOneDwarfAccelTable &&
+      !DwarfObj.getDebugNamesSection().Data.empty()) {
+    AtLeastOneDwarfAccelTable = true;
+  }
+}
+
 bool DwarfLinker::emitPaperTrailWarnings(const DebugMapObject &DMO,
                                          const DebugMap &Map,
                                          OffsetsStringPool &StringPool) {
   if (DMO.getWarnings().empty() || !DMO.empty())
     return false;
 
   Streamer->switchToDebugInfoSection(/* Version */ 2);
   DIE *CUDie = DIE::get(DIEAlloc, dwarf::DW_TAG_compile_unit);
bool DwarfLinker::link(const DebugMap &Map) {
   unsigned UnitID = 0;
   DebugMap ModuleMap(Map.getTriple(), Map.getBinaryPath());
 
   // First populate the data structure we need for each iteration of the
   // parallel loop.
   unsigned NumObjects = Map.getNumberOfObjects();
   std::vector<LinkContext> ObjectContexts;
   ObjectContexts.reserve(NumObjects);
-  for (const auto &Obj : Map.objects())
+  for (const auto &Obj : Map.objects()) {
     ObjectContexts.emplace_back(Map, *this, *Obj.get());
+    LinkContext &LC = ObjectContexts.back();
+    if (LC.ObjectFile)
+      updateAccelKind(*LC.DwarfContext);
+  }
 
   // This Dwarf string pool which is only used for uniquing. This one should
   // never be used for offsets as its not thread-safe or predictable.
   UniquingStringPool UniquingStringPool;
 
   // This Dwarf string pool which is used for emission. It must be used
   // serially as the order of calling getStringOffset matters for
   // reproducibility.
   OffsetsStringPool OffsetsStringPool;
 
   // ODR Contexts for the link.
   DeclContextTree ODRContexts;
 
+  // If we haven't decided on an accelerator table kind yet, we base ourselves
+  // on the DWARF we have seen so far. At this point we haven't pulled in debug
+  // information from modules yet, so it is technically possible that they
+  // would affect the decision. However, as they're built with the same
+  // compiler and flags, it is safe to assume that they will follow the
+  // decision made here.
+  if (Options.TheAccelTableKind == AccelTableKind::Default) {
+    if (AtLeastOneDwarfAccelTable && !AtLeastOneAppleAccelTable)
+      Options.TheAccelTableKind = AccelTableKind::Dwarf;
+    else
+      Options.TheAccelTableKind = AccelTableKind::Apple;
+  }
+
   for (LinkContext &LinkContext : ObjectContexts) {
     if (Options.Verbose)
       outs() << "DEBUG MAP OBJECT: " << LinkContext.DMO.getObjectFilename()
              << "\n";
 
     // N_AST objects (swiftmodule files) should get dumped directly into the
     // appropriate DWARF section.
     if (LinkContext.DMO.getType() == MachO::N_AST) {
for (LinkContext &LinkContext : ObjectContexts) {
     if (!LinkContext.DwarfContext)
       continue;
 
     startDebugObject(LinkContext);
 
     // In a first phase, just read in the debug info and load all clang modules.
     LinkContext.CompileUnits.reserve(
         LinkContext.DwarfContext->getNumCompileUnits());
+
     for (const auto &CU : LinkContext.DwarfContext->compile_units()) {
+      updateDwarfVersion(CU->getVersion());
       auto CUDie = CU->getUnitDIE(false);
       if (Options.Verbose) {
         outs() << "Input compilation unit:";
         DIDumpOptions DumpOpts;
         DumpOpts.RecurseDepth = 0;
         DumpOpts.Verbose = Options.Verbose;
         CUDie.dump(outs(), 0, DumpOpts);
       }
 
       if (!CUDie || LLVM_UNLIKELY(Options.Update) ||
           !registerModuleReference(CUDie, *CU, ModuleMap, LinkContext.DMO,
                                    LinkContext.Ranges, OffsetsStringPool,
                                    UniquingStringPool, ODRContexts, UnitID)) {
         LinkContext.CompileUnits.push_back(llvm::make_unique<CompileUnit>(
             *CU, UnitID++, !Options.NoODR && !Options.Update, ""));
-        maybeUpdateMaxDwarfVersion(CU->getVersion());
       }
     }
   }
 
-  // If we haven't seen any CUs, pick an arbitrary valid Dwarf version anyway,
-  // to be able to emit papertrail warnings.
+  // If we haven't seen any CUs, pick an arbitrary valid Dwarf version anyway.
   if (MaxDwarfVersion == 0)
     MaxDwarfVersion = 3;
 
   // These variables manage the list of processed object files.
   // The mutex and condition variable are to ensure that this is thread safe.
   std::mutex ProcessedFilesMutex;
   std::condition_variable ProcessedFilesConditionVariable;
   BitVector ProcessedFiles(NumObjects, false);
for (unsigned i = 0, e = NumObjects; i != e; ++i) {
       // Clean-up before starting working on the next object.
       endDebugObject(LinkContext);
     }
 
     // Emit everything that's global.
     if (!Options.NoOutput) {
       Streamer->emitAbbrevs(Abbreviations, MaxDwarfVersion);
       Streamer->emitStrings(OffsetsStringPool);
+      switch (Options.TheAccelTableKind) {
+      case AccelTableKind::Apple:
         Streamer->emitAppleNames(AppleNames);
         Streamer->emitAppleNamespaces(AppleNamespaces);
         Streamer->emitAppleTypes(AppleTypes);
         Streamer->emitAppleObjc(AppleObjc);
+        break;
+      case AccelTableKind::Dwarf:
+        Streamer->emitDebugNames(DebugNames);
+        break;
+      case AccelTableKind::Default:
+        llvm_unreachable("Default should have already been resolved.");
+        break;
+      }
     }
   };
 
   // FIXME: The DwarfLinker can have some very deep recursion that can max
   // out the (significantly smaller) stack when using threads. We don't
   // want this limitation when we only have a single thread.
   if (Options.Threads == 1) {
     AnalyzeLambda();
     CloneLambda();
   } else {
     ThreadPool pool(2);
     pool.async(AnalyzeLambda);
     pool.async(CloneLambda);
     pool.wait();
   }
 
   return Options.NoOutput ? true : Streamer->finish(Map);
-}
+} // namespace dsymutil
 
 bool linkDwarf(raw_fd_ostream &OutFile, BinaryHolder &BinHolder,
                const DebugMap &DM, const LinkOptions &Options) {
   DwarfLinker Linker(OutFile, BinHolder, Options);
   return Linker.link(DM);
 }
 
 } // namespace dsymutil
 } // namespace llvm

llvm/trunk/tools/dsymutil/DwarfStreamer.h

 #include "llvm/MC/MCDwarf.h"
 #include "llvm/MC/MCInstrInfo.h"
 #include "llvm/MC/MCObjectFileInfo.h"
 #include "llvm/MC/MCObjectWriter.h"
 #include "llvm/MC/MCRegisterInfo.h"
 #include "llvm/MC/MCSection.h"
 #include "llvm/MC/MCStreamer.h"
 #include "llvm/MC/MCSubtargetInfo.h"
+#include "llvm/MC/MCSymbol.h"
 #include "llvm/MC/MCTargetOptions.h"
 #include "llvm/Target/TargetMachine.h"
 #include "llvm/Target/TargetOptions.h"
 
 #ifndef LLVM_TOOLS_DSYMUTIL_DWARFSTREAMER_H
 #define LLVM_TOOLS_DSYMUTIL_DWARFSTREAMER_H
 
 namespace llvm {
public:
 
   /// Emit a CIE.
   void emitCIE(StringRef CIEBytes);
 
   /// Emit an FDE with data \p Bytes.
   void emitFDE(uint32_t CIEOffset, uint32_t AddreSize, uint32_t Address,
                StringRef Bytes);
 
+  /// Emit DWARF debug names.
+  void emitDebugNames(AccelTable<DWARF5AccelTableStaticData> &Table);
+
   /// Emit Apple namespaces accelerator table.
   void emitAppleNamespaces(AccelTable<AppleAccelTableStaticOffsetData> &Table);
 
   /// Emit Apple names accelerator table.
   void emitAppleNames(AccelTable<AppleAccelTableStaticOffsetData> &Table);
 
   /// Emit Apple Objective-C accelerator table.
   void emitAppleObjc(AccelTable<AppleAccelTableStaticOffsetData> &Table);
private:
 
   LinkOptions Options;
 
   uint32_t RangesSectionSize;
   uint32_t LocSectionSize;
   uint32_t LineSectionSize;
   uint32_t FrameSectionSize;
 
+  /// Keep track of emitted CUs and their Unique ID.
+  struct EmittedUnit {
+    unsigned ID;
+    MCSymbol *LabelBegin;
+  };
+  std::vector<EmittedUnit> EmittedUnits;
+
   /// Emit the pubnames or pubtypes section contribution for \p
   /// Unit into \p Sec. The data is provided in \p Names.
   void emitPubSectionForUnit(MCSection *Sec, StringRef Name,
                              const CompileUnit &Unit,
                              const std::vector<CompileUnit::AccelInfo> &Names);
 };
 
 } // end namespace dsymutil
 } // end namespace llvm
 
 #endif // LLVM_TOOLS_DSYMUTIL_DWARFSTREAMER_H

llvm/trunk/tools/dsymutil/DwarfStreamer.cpp

 //===- tools/dsymutil/DwarfStreamer.cpp - Dwarf Streamer ------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #include "DwarfStreamer.h"
+#include "CompileUnit.h"
 #include "LinkUtils.h"
 #include "MachOUtils.h"
 #include "llvm/ADT/Triple.h"
 #include "llvm/DebugInfo/DWARF/DWARFContext.h"
 #include "llvm/MC/MCTargetOptions.h"
 #include "llvm/MC/MCTargetOptionsCommandFlags.inc"
 #include "llvm/Support/LEB128.h"
 #include "llvm/Support/TargetRegistry.h"
void DwarfStreamer::emitCompileUnitHeader(CompileUnit &Unit) {
   // account for the length field.
   Asm->emitInt32(Unit.getNextUnitOffset() - Unit.getStartOffset() - 4);
   Asm->emitInt16(Version);
 
   // We share one abbreviations table across all units so it's always at the
   // start of the section.
   Asm->emitInt32(0);
   Asm->emitInt8(Unit.getOrigUnit().getAddressByteSize());
+
+  // Remember this CU.
+  EmittedUnits.push_back({Unit.getUniqueID(), Unit.getLabelBegin()});
 }
 
 /// Emit the \p Abbrevs array as the shared abbreviation table
 /// for the linked Dwarf file.
 void DwarfStreamer::emitAbbrevs(
     const std::vector<std::unique_ptr<DIEAbbrev>> &Abbrevs,
     unsigned DwarfVersion) {
   MS->SwitchSection(MOFI->getDwarfAbbrevSection());
if (Entry.getIndex() == -1U)
       break;
     // Emit the string itself.
     Asm->OutStreamer->EmitBytes(Entry.getString());
     // Emit a null terminator.
     Asm->emitInt8(0);
   }
 }
 
+void DwarfStreamer::emitDebugNames(
+    AccelTable<DWARF5AccelTableStaticData> &Table) {
+  if (EmittedUnits.empty())
+    return;
+
+  // Build up data structures needed to emit this section.
+  std::vector<MCSymbol *> CompUnits;
+  DenseMap<unsigned, size_t> UniqueIdToCuMap;
+  unsigned Id = 0;
+  for (auto &CU : EmittedUnits) {
+    CompUnits.push_back(CU.LabelBegin);
+    // We might be omitting CUs, so we need to remap them.
+    UniqueIdToCuMap[CU.ID] = Id++;
+  }
+
+  Asm->OutStreamer->SwitchSection(MOFI->getDwarfDebugNamesSection());
+  emitDWARF5AccelTable(
+      Asm.get(), Table, CompUnits,
+      [&UniqueIdToCuMap](const DWARF5AccelTableStaticData &Entry) {
+        return UniqueIdToCuMap[Entry.getCUIndex()];
+      });
+}
+
 void DwarfStreamer::emitAppleNamespaces(
     AccelTable<AppleAccelTableStaticOffsetData> &Table) {
   Asm->OutStreamer->SwitchSection(MOFI->getDwarfAccelNamespaceSection());
   auto *SectionBegin = Asm->createTempSymbol("namespac_begin");
   Asm->OutStreamer->EmitLabel(SectionBegin);
   emitAppleAccelTable(Asm.get(), Table, "namespac", SectionBegin);
 }
 

llvm/trunk/tools/dsymutil/LinkUtils.h

 namespace llvm {
 namespace dsymutil {
 
 enum class OutputFileType {
   Object,
   Assembly,
 };
 
+/// The kind of accelerator tables we should emit.
+enum class AccelTableKind {
+  Apple,   ///< .apple_names, .apple_namespaces, .apple_types, .apple_objc.
+  Dwarf,   ///< DWARF v5 .debug_names.
+  Default, ///< Dwarf for DWARF5 or later, Apple otherwise.
+};
+
 struct LinkOptions {
   /// Verbosity
   bool Verbose = false;
 
   /// Skip emitting output
   bool NoOutput = false;
 
   /// Do not unique types according to ODR
struct LinkOptions {
   bool NoTimestamp = false;
 
   /// Number of threads.
   unsigned Threads = 1;
 
   // Output file type.
   OutputFileType FileType = OutputFileType::Object;
 
+  /// The accelerator table kind
+  AccelTableKind TheAccelTableKind;
+
   /// -oso-prepend-path
   std::string PrependPath;
 
   LinkOptions() = default;
 };
 
 inline void warn(Twine Warning, Twine Context = {}) {
   WithColor::warning() << Warning + "\n";

llvm/trunk/tools/dsymutil/dsymutil.cpp

 
 static opt<bool> FlatOut("flat",
                          desc("Produce a flat dSYM file (not a bundle)."),
                          init(false), cat(DsymCategory));
 static alias FlatOutA("f", desc("Alias for --flat"), aliasopt(FlatOut));
 
 static opt<bool> Minimize(
     "minimize",
-    desc("When used when creating a dSYM file, this option will suppress\n"
-         "the emission of the .debug_inlines, .debug_pubnames, and\n"
-         ".debug_pubtypes sections since dsymutil currently has better\n"
-         "equivalents: .apple_names and .apple_types. When used in\n"
+    desc("When used when creating a dSYM file with Apple accelerator tables,\n"
+         "this option will suppress the emission of the .debug_inlines, \n"
+         ".debug_pubnames, and .debug_pubtypes sections since dsymutil \n"
+         "has better equivalents: .apple_names and .apple_types. When used in\n"
          "conjunction with --update option, this option will cause redundant\n"
          "accelerator tables to be removed."),
     init(false), cat(DsymCategory));
 static alias MinimizeA("z", desc("Alias for --minimize"), aliasopt(Minimize));
 
 static opt<bool> Update(
     "update",
     desc("Updates existing dSYM files to contain the latest accelerator\n"
-         "tables and other DWARF optimizations. This option will currently\n"
-         "add the new .apple_names and .apple_types hashed accelerator\n"
-         "tables."),
+         "tables and other DWARF optimizations."),
     init(false), cat(DsymCategory));
 static alias UpdateA("u", desc("Alias for --update"), aliasopt(Update));
 
+static cl::opt<AccelTableKind> AcceleratorTable(
+    "accelerator", cl::desc("Output accelerator tables."),
+    cl::values(clEnumValN(AccelTableKind::Default, "Default",
+                          "Default for input."),
+               clEnumValN(AccelTableKind::Apple, "Apple", "Apple"),
+               clEnumValN(AccelTableKind::Dwarf, "Dwarf", "DWARF")),
+    cl::init(AccelTableKind::Default), cat(DsymCategory));
+
 static opt<unsigned> NumThreads(
     "num-threads",
     desc("Specifies the maximum number (n) of simultaneous threads to use\n"
          "when linking multiple architectures."),
     value_desc("n"), init(0), cat(DsymCategory));
 static alias NumThreadsA("j", desc("Alias for --num-threads"),
                          aliasopt(NumThreads));
 
static Expected<LinkOptions> getOptions() {
 
   Options.Verbose = Verbose;
   Options.NoOutput = NoOutput;
   Options.NoODR = NoODR;
   Options.Minimize = Minimize;
   Options.Update = Update;
   Options.NoTimestamp = NoTimestamp;
   Options.PrependPath = OsoPrependPath;
+  Options.TheAccelTableKind = AcceleratorTable;
 
   if (Assembly)
     Options.FileType = OutputFileType::Assembly;
 
   if (Options.Update && std::find(InputFiles.begin(), InputFiles.end(), "-") !=
                             InputFiles.end()) {
     // FIXME: We cannot use stdin for an update because stdin will be
     // consumed by the BinaryHolder during the debugmap parsing, and