[dsymutil] Upstream update feature.

[JDevlieghere]

Summary

Now that dsymutil can generate accelerator tables, we can upstream the
update logic that, as the name implies, updates the accelerator tables
in an existing dSYM bundle. In combination with -minimize this can be
used to remove redundant .debug_(inlines|pubtypes|pubnames).

[aprantl]

Couple nitpicks inline :-)

llvm/tools/dsymutil/DwarfLinker.cpp

512 ↗

(On Diff #132733)

Comment about what this function does?

517 ↗

(On Diff #132733)

should this be a loop instead of recursion? We previously ran into stack limits with dsymutil...

723 ↗

(On Diff #132733)

doxygen comment?

llvm/tools/dsymutil/dsymutil.cpp

90 ↗

(On Diff #132733)

Does this need to go into the manpage, too?

[JDevlieghere]

CR Feedback @aprantl

[davide]

Just a couple of nits.

llvm/tools/dsymutil/DwarfLinker.cpp

2890 ↗

(On Diff #132959)

LLVM_UNLIKELY ? Does this really matter?

3235 ↗

(On Diff #132959)

ditto.

[JDevlieghere]

In D42880#999331, @davide wrote:

Just a couple of nits.

I have to admit that I didn't measure its impact, but it's how it's currently implemented on our internal branch. The DwarfLinker isn't exactly super fast and obviously we don't want to risk it becoming slower for this (really unlikely) update path. If you're really opposed to this I can do some measurements. but in the end I don't think it hurts readability/maintainability which motivates me to err on the side of caution.

[davide]

Nah, it's not worth discussing this further. Go ahead.

llvm/trunk/docs/CommandGuide/dsymutil.rst

  Produce a flat dSYM file. A ``.dwarf`` extension will be appended to the
  executable name unless the output file is specified using the -o option.
 
 
 .. option:: -z, --minimize
 
  When used when creating a dSYM file, this option will suppress the emission of
  the .debug_inlines, .debug_pubnames, and .debug_pubtypes sections since
- dsymutil currently has better equivalents: .apple_names and .apple_types.
+ dsymutil currently has better equivalents: .apple_names and .apple_types. When
+ used in conjunction with --update option, this option will cause redundant
+ accelerator tables to be removed.
 
 .. option:: --no-odr
 
  Do not use ODR (One Definition Rule) for uniquing C++ types.
 
 .. option:: --no-output
 
  Do the link in memory, but do not emit the result file.

 .. option:: --oso-prepend-path=<path>
 
  Specifies a ``path`` to prepend to all debug symbol object file paths.
 
 .. option:: -s, --symtab
 
  Dumps the symbol table found in *executable* or object file(s) and exits.
 
+.. option:: -u, --update
+
+ Update an existing dSYM file to contain the latest accelerator tables and
+ other DWARF optimizations. This option will rebuild the '.apple_names' and
+ '.apple_types' hashed accelerator tables.
+
 .. option:: -v, --verbose
 
  Display verbose information when linking.
 
 .. option:: --version
 
  Display the version of the tool.
 

llvm/trunk/test/tools/dsymutil/X86/basic-linking-x86.test

 RUN: cat %p/../Inputs/basic.macho.x86_64 > %t1
 RUN: llvm-dsymutil -f -oso-prepend-path=%p/.. %t1
 RUN: llvm-dwarfdump -a %t1.dwarf | FileCheck %s
 RUN: llvm-dsymutil -f -o %t2 -oso-prepend-path=%p/.. %p/../Inputs/basic.macho.x86_64
 RUN: llvm-dwarfdump -a %t2 | FileCheck %s
 RUN: llvm-dsymutil -f -o - -oso-prepend-path=%p/.. %p/../Inputs/basic.macho.x86_64 | llvm-dwarfdump -a - | FileCheck %s --check-prefix=CHECK --check-prefix=BASIC
 RUN: llvm-dsymutil -f -o - -oso-prepend-path=%p/.. %p/../Inputs/basic-archive.macho.x86_64 | llvm-dwarfdump -a - | FileCheck %s --check-prefix=CHECK --check-prefix=ARCHIVE
 RUN: llvm-dsymutil -dump-debug-map -oso-prepend-path=%p/.. %p/../Inputs/basic.macho.x86_64 | llvm-dsymutil -f -y -o - - | llvm-dwarfdump -a - | FileCheck %s --check-prefix=CHECK --check-prefix=BASIC
 RUN: llvm-dsymutil -dump-debug-map -oso-prepend-path=%p/.. %p/../Inputs/basic-archive.macho.x86_64 | llvm-dsymutil -f -o - -y - | llvm-dwarfdump -a - | FileCheck %s --check-prefix=CHECK --check-prefix=ARCHIVE
 
+# Update tests
+RUN: rm -rf %t.dir
+RUN: mkdir -p %t.dir
+RUN: cat %p/../Inputs/basic.macho.x86_64 > %t.dir/basic
+RUN: llvm-dsymutil -oso-prepend-path=%p/.. %t.dir/basic
+RUN: llvm-dwarfdump -a %t.dir/basic.dSYM/Contents/Resources/DWARF/basic | FileCheck %s
+RUN: llvm-dsymutil --update %t.dir/basic.dSYM
+RUN: llvm-dwarfdump -a %t.dir/basic.dSYM/Contents/Resources/DWARF/basic | FileCheck %s
+RUN: llvm-dsymutil -u %t.dir/basic.dSYM
+RUN: llvm-dwarfdump -a %t.dir/basic.dSYM/Contents/Resources/DWARF/basic | FileCheck %s
+RUN: llvm-dsymutil --update %t.dir/basic.dSYM -o %t.dir/updated.dSYM
+RUN: llvm-dwarfdump -a %t.dir/updated.dSYM/Contents/Resources/DWARF/basic | FileCheck %s
+RUN: llvm-dsymutil -f -u %t2 -o %t3
+RUN: llvm-dwarfdump -a %t3 | FileCheck %s
+
 CHECK: file format Mach-O 64-bit x86-64
 
 CHECK: debug_info contents
 
 CHECK:  Compile Unit:
 
 CHECK:  DW_TAG_compile_unit
 CHECK:    DW_AT_producer ("Apple LLVM version 6.0 (clang-600.0.39) (based on LLVM 3.5svn)")

llvm/trunk/test/tools/dsymutil/X86/update-one-CU.test

+RUN: llvm-dsymutil -oso-prepend-path=%p/.. %p/../Inputs/objc.macho.x86_64 -o %t.dSYM
+RUN: llvm-dsymutil -update %t.dSYM
+RUN: llvm-dwarfdump -apple-types -apple-objc %t.dSYM/Contents/Resources/DWARF/objc.macho.x86_64 | FileCheck %s
+
+CHECK: .apple_types contents:
+CHECK: Hash 0x2b5e6 [
+CHECK-NEXT:    Name@0x145 {
+CHECK-NEXT:      String: 0x00000066 "A"
+CHECK-NEXT:      Data 0 [
+CHECK-NEXT:        Atom[0]: 0x0000012d
+CHECK-NEXT:        Atom[1]: 0x0013
+CHECK-NEXT:        Atom[2]: 0x02
+CHECK-NEXT:        Atom[3]: 0x0b87b15a
+CHECK-NEXT:      ]
+CHECK-NEXT:    }
+CHECK-NEXT:  ]
+
+CHECK: .apple_objc contents:
+CHECK: Hash 0x2b5e6
+CHECK-NEXT: Name@0x38 {
+CHECK-NEXT:   String: 0x00000066 "A"
+CHECK-NEXT:   Data 0 [
+CHECK-NEXT:     Atom[0]: 0x00000027
+CHECK-NEXT:   ]
+CHECK-NEXT:   Data 1 [
+CHECK-NEXT:     Atom[0]: 0x0000007a
+CHECK-NEXT:   ]
+CHECK-NEXT: }
+CHECK: Hash 0x3fa0f4b5
+CHECK-NEXT: Name@0x4c {
+CHECK-NEXT:   String: 0x0000009d "A(Category)"
+CHECK-NEXT:   Data 0 [
+CHECK-NEXT:     Atom[0]: 0x0000007a
+CHECK-NEXT:   ]
+CHECK-NEXT: }

llvm/trunk/test/tools/dsymutil/cmdline.test

 RUN: llvm-dsymutil -help 2>&1 | FileCheck --check-prefix=HELP %s
 HELP: OVERVIEW: manipulate archived DWARF debug symbol files.
 HELP: USAGE: llvm-dsymutil{{[^ ]*}} [options] <input files>
 HELP-NOT: -reverse-iterate
 HELP: Specific Options:
 HELP: -arch=<arch>
 HELP: -dump-debug-map
 HELP: -flat
 HELP: -minimize
 HELP: -no-odr
 HELP: -no-output
 HELP: -no-swiftmodule-timestamp
 HELP: -num-threads=<n>
 HELP: -o=<filename>
 HELP: -oso-prepend-path=<path>
 HELP: -symtab
+HELP: -update
 HELP: -verbose
 HELP: -verify
 HELP: -y
 HELP-NOT: -reverse-iterate
 
 RUN: llvm-dsymutil --version 2>&1 | FileCheck --check-prefix=VERSION %s
 VERSION: {{ version }}

llvm/trunk/tools/dsymutil/DebugMap.h

 #define LLVM_TOOLS_DSYMUTIL_DEBUGMAP_H
 
 #include "llvm/ADT/DenseMap.h"
 #include "llvm/ADT/Optional.h"
 #include "llvm/ADT/StringMap.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/ADT/Triple.h"
 #include "llvm/ADT/iterator_range.h"
+#include "llvm/Object/MachO.h"
 #include "llvm/Support/Chrono.h"
 #include "llvm/Support/ErrorOr.h"
 #include "llvm/Support/YAMLTraits.h"
 #include <chrono>
 #include <cstddef>
 #include <cstdint>
 #include <memory>
 #include <string>
public:
 
   const_iterator end() const { return Objects.end(); }
 
   /// This function adds an DebugMapObject to the list owned by this
   /// debug map.
   DebugMapObject &
   addDebugMapObject(StringRef ObjectFilePath,
                     sys::TimePoint<std::chrono::seconds> Timestamp,
-                    uint8_t Type);
+                    uint8_t Type = llvm::MachO::N_OSO);
 
   const Triple &getTriple() const { return BinaryTriple; }
 
   StringRef getBinaryPath() const { return BinaryPath; }
 
   void print(raw_ostream &OS) const;
 
 #ifndef NDEBUG

llvm/trunk/tools/dsymutil/DwarfLinker.cpp

 #include <utility>
 #include <vector>
 
 namespace llvm {
 namespace dsymutil {
 
 namespace {
 
+/// Retrieve the section named \a SecName in \a Obj.
+///
+/// To accommodate for platform discrepancies, the name passed should be
+/// (for example) 'debug_info' to match either '__debug_info' or '.debug_info'.
+/// This function will strip the initial platform-specific characters.
+static Optional<object::SectionRef>
+getSectionByName(const object::ObjectFile &Obj, StringRef SecName) {
+  for (const object::SectionRef &Section : Obj.sections()) {
+    StringRef SectionName;
+    Section.getName(SectionName);
+    SectionName = SectionName.substr(SectionName.find_first_not_of("._"));
+    if (SectionName != SecName)
+      continue;
+    return Section;
+  }
+  return None;
+}
+
 template <typename KeyT, typename ValT>
 using HalfOpenIntervalMap =
     IntervalMap<KeyT, ValT, IntervalMapImpl::NodeSizer<KeyT, ValT>::LeafSize,
                 IntervalMapHalfOpenInfo<KeyT>>;
 
 using FunctionIntervals = HalfOpenIntervalMap<uint64_t, int64_t>;
 
 // FIXME: Delete this structure.
private:
 
   /// Did a DIE actually contain a valid reloc?
   bool HasInterestingContent;
 
   /// If this is a Clang module, this holds the module's name.
   std::string ClangModuleName;
 };
 
+/// Check if the DIE at \p Idx is in the scope of a function.
+static bool inFunctionScope(CompileUnit &U, unsigned Idx) {
+  while (Idx) {
+    if (U.getOrigUnit().getDIEAtIndex(Idx).getTag() == dwarf::DW_TAG_subprogram)
+      return true;
+    Idx = U.getInfo(Idx).ParentIdx;
+  }
+  return false;
+}
+
 } // end anonymous namespace
 
 void CompileUnit::markEverythingAsKept() {
-  for (auto &I : Info)
-    // Mark everything that wasn't explicity marked for pruning.
+  unsigned Idx = 0;
+
+  setHasInterestingContent();
+
+  for (auto &I : Info) {
+    // Mark everything that wasn't explicit marked for pruning.
     I.Keep = !I.Prune;
+    auto DIE = OrigUnit.getDIEAtIndex(Idx++);
+
+    // Try to guess which DIEs must go to the accelerator tables. We do that
+    // just for variables, because functions will be handled depending on
+    // whether they carry a DW_AT_low_pc attribute or not.
+    if (DIE.getTag() != dwarf::DW_TAG_variable &&
+        DIE.getTag() != dwarf::DW_TAG_constant)
+      continue;
+
+    Optional<DWARFFormValue> Value;
+    if (!(Value = DIE.find(dwarf::DW_AT_location))) {
+      if ((Value = DIE.find(dwarf::DW_AT_const_value)) &&
+          !inFunctionScope(*this, I.ParentIdx))
+        I.InDebugMap = true;
+      continue;
+    }
+    if (auto Block = Value->getAsBlock()) {
+      if (Block->size() > OrigUnit.getAddressByteSize() &&
+          (*Block)[0] == dwarf::DW_OP_addr)
+        I.InDebugMap = true;
+    }
+  }
 }
 
 uint64_t CompileUnit::computeNextUnitOffset() {
   NextUnitOffset = StartOffset + 11 /* Header size */;
   // The root DIE might be null, meaning that the Unit had nothing to
   // contribute to the linked output. In that case, we will emit the
   // unit header without any actual DIE.
   if (NewUnit)
public:
   void emitLocationsForUnit(const CompileUnit &Unit, DWARFContext &Dwarf);
 
   /// Emit the line table described in \p Rows into the debug_line section.
   void emitLineTableForUnit(MCDwarfLineTableParams Params,
                             StringRef PrologueBytes, unsigned MinInstLength,
                             std::vector<DWARFDebugLine::Row> &Rows,
                             unsigned AdddressSize);
 
+  /// Copy over the debug sections that are not modified when updating.
+  void copyInvariantDebugSection(const object::ObjectFile &Obj, LinkOptions &);
+
   uint32_t getLineSectionSize() const { return LineSectionSize; }
 
   /// Emit the .debug_pubnames contribution for \p Unit.
   void emitPubNamesForUnit(const CompileUnit &Unit);
 
   /// Emit the .debug_pubtypes contribution for \p Unit.
   void emitPubTypesForUnit(const CompileUnit &Unit);
 
if (RowsSinceLastSequence) {
     MS->EmitBytes(EncodingOS.str());
     LineSectionSize += EncodingBuffer.size();
     EncodingBuffer.resize(0);
   }
 
   MS->EmitLabel(LineEndSym);
 }
 
+static void emitSectionContents(const object::ObjectFile &Obj,
+                                StringRef SecName, MCStreamer *MS) {
+  StringRef Contents;
+  if (auto Sec = getSectionByName(Obj, SecName))
+    if (!Sec->getContents(Contents))
+      MS->EmitBytes(Contents);
+}
+
+void DwarfStreamer::copyInvariantDebugSection(const object::ObjectFile &Obj,
+                                              LinkOptions &Options) {
+  MS->SwitchSection(MC->getObjectFileInfo()->getDwarfLineSection());
+  emitSectionContents(Obj, "debug_line", MS);
+
+  MS->SwitchSection(MC->getObjectFileInfo()->getDwarfLocSection());
+  emitSectionContents(Obj, "debug_loc", MS);
+
+  MS->SwitchSection(MC->getObjectFileInfo()->getDwarfRangesSection());
+  emitSectionContents(Obj, "debug_ranges", MS);
+
+  MS->SwitchSection(MC->getObjectFileInfo()->getDwarfFrameSection());
+  emitSectionContents(Obj, "debug_frame", MS);
+
+  MS->SwitchSection(MC->getObjectFileInfo()->getDwarfARangesSection());
+  emitSectionContents(Obj, "debug_aranges", MS);
+}
+
 /// Emit the pubnames or pubtypes section contribution for \p
 /// Unit into \p Sec. The data is provided in \p Names.
 void DwarfStreamer::emitPubSectionForUnit(
     MCSection *Sec, StringRef SecName, const CompileUnit &Unit,
     const std::vector<CompileUnit::AccelInfo> &Names) {
   if (Names.empty())
     return;
 
unsigned cloneAttribute(DIE &Die,
                             CompileUnit &U, const DWARFFormValue &Val,
                             const AttributeSpec AttrSpec, unsigned AttrSize,
                             AttributesInfo &AttrInfo);
 
     /// Clone a string attribute described by \p AttrSpec and add
     /// it to \p Die.
     /// \returns the size of the new attribute.
     unsigned cloneStringAttribute(DIE &Die, AttributeSpec AttrSpec,
-                                  const DWARFFormValue &Val,
-                                  const DWARFUnit &U);
+                                  const DWARFFormValue &Val, const DWARFUnit &U,
+                                  AttributesInfo &Info);
 
     /// Clone an attribute referencing another DIE and add
     /// it to \p Die.
     /// \returns the size of the new attribute.
     unsigned
     cloneDieReferenceAttribute(DIE &Die,
                                const DWARFDie &InputDIE,
                                AttributeSpec AttrSpec, unsigned AttrSize,
if (InSet) {
     Abbrev.setNumber(Abbreviations.size());
     Abbreviations.back()->setNumber(Abbreviations.size());
   }
 }
 
 unsigned DwarfLinker::DIECloner::cloneStringAttribute(DIE &Die,
                                                       AttributeSpec AttrSpec,
                                                       const DWARFFormValue &Val,
-                                                      const DWARFUnit &U) {
+                                                      const DWARFUnit &U,
+                                                      AttributesInfo &Info) {
   // Switch everything to out of line strings.
   const char *String = *Val.getAsCString();
-  unsigned Offset = Linker.StringPool.getStringOffset(String);
+  auto StringEntry = Linker.StringPool.getEntry(String);
+  if (AttrSpec.Attr == dwarf::DW_AT_name)
+    Info.Name = StringEntry;
+  else if (AttrSpec.Attr == dwarf::DW_AT_MIPS_linkage_name ||
+           AttrSpec.Attr == dwarf::DW_AT_linkage_name)
+    Info.MangledName = StringEntry;
   Die.addValue(DIEAlloc, dwarf::Attribute(AttrSpec.Attr), dwarf::DW_FORM_strp,
-               DIEInteger(Offset));
+               DIEInteger(StringEntry.getOffset()));
   return 4;
 }
 
 unsigned DwarfLinker::DIECloner::cloneDieReferenceAttribute(
     DIE &Die, const DWARFDie &InputDIE,
     AttributeSpec AttrSpec, unsigned AttrSize, const DWARFFormValue &Val,
     CompileUnit &Unit) {
   const DWARFUnit &U = Unit.getOrigUnit();
unsigned DwarfLinker::DIECloner::cloneBlockAttribute(DIE &Die,
   return AttrSize;
 }
 
 unsigned DwarfLinker::DIECloner::cloneAddressAttribute(
     DIE &Die, AttributeSpec AttrSpec, const DWARFFormValue &Val,
     const CompileUnit &Unit, AttributesInfo &Info) {
   uint64_t Addr = *Val.getAsAddress();
 
+  if (LLVM_UNLIKELY(Linker.Options.Update)) {
+    if (AttrSpec.Attr == dwarf::DW_AT_low_pc)
+      Info.HasLowPc = true;
+    Die.addValue(DIEAlloc, dwarf::Attribute(AttrSpec.Attr),
+                 dwarf::Form(AttrSpec.Form), DIEInteger(Addr));
+    return Unit.getOrigUnit().getAddressByteSize();
+  }
+
   if (AttrSpec.Attr == dwarf::DW_AT_low_pc) {
     if (Die.getTag() == dwarf::DW_TAG_inlined_subroutine ||
         Die.getTag() == dwarf::DW_TAG_lexical_block)
       // The low_pc of a block or inline subroutine might get
       // relocated because it happens to match the low_pc of the
       // enclosing subprogram. To prevent issues with that, always use
       // the low_pc from the input DIE if relocations have been applied.
       Addr = (Info.OrigLowPc != std::numeric_limits<uint64_t>::max()
unsigned DwarfLinker::DIECloner::cloneAddressAttribute(
   return Unit.getOrigUnit().getAddressByteSize();
 }
 
 unsigned DwarfLinker::DIECloner::cloneScalarAttribute(
     DIE &Die, const DWARFDie &InputDIE, CompileUnit &Unit,
     AttributeSpec AttrSpec, const DWARFFormValue &Val, unsigned AttrSize,
     AttributesInfo &Info) {
   uint64_t Value;
+
+  if (LLVM_UNLIKELY(Linker.Options.Update)) {
+    if (auto OptionalValue = Val.getAsUnsignedConstant())
+      Value = *OptionalValue;
+    else if (auto OptionalValue = Val.getAsSignedConstant())
+      Value = *OptionalValue;
+    else if (auto OptionalValue = Val.getAsSectionOffset())
+      Value = *OptionalValue;
+    else {
+      Linker.reportWarning(
+          "Unsupported scalar attribute form. Dropping attribute.", &InputDIE);
+      return 0;
+    }
+    if (AttrSpec.Attr == dwarf::DW_AT_declaration && Value)
+      Info.IsDeclaration = true;
+    Die.addValue(DIEAlloc, dwarf::Attribute(AttrSpec.Attr),
+                 dwarf::Form(AttrSpec.Form), DIEInteger(Value));
+    return AttrSize;
+  }
+
   if (AttrSpec.Attr == dwarf::DW_AT_high_pc &&
       Die.getTag() == dwarf::DW_TAG_compile_unit) {
     if (Unit.getLowPc() == -1ULL)
       return 0;
     // Dwarf >= 4 high_pc is an size, not an address.
     Value = Unit.getHighPc() - Unit.getLowPc();
   } else if (AttrSpec.Form == dwarf::DW_FORM_sec_offset)
     Value = *Val.getAsSectionOffset();
unsigned DwarfLinker::DIECloner::cloneAttribute(
     DIE &Die, const DWARFDie &InputDIE, CompileUnit &Unit,
     const DWARFFormValue &Val, const AttributeSpec AttrSpec, unsigned AttrSize,
     AttributesInfo &Info) {
   const DWARFUnit &U = Unit.getOrigUnit();
 
   switch (AttrSpec.Form) {
   case dwarf::DW_FORM_strp:
   case dwarf::DW_FORM_string:
-    return cloneStringAttribute(Die, AttrSpec, Val, U);
+    return cloneStringAttribute(Die, AttrSpec, Val, U, Info);
   case dwarf::DW_FORM_ref_addr:
   case dwarf::DW_FORM_ref1:
   case dwarf::DW_FORM_ref2:
   case dwarf::DW_FORM_ref4:
   case dwarf::DW_FORM_ref8:
     return cloneDieReferenceAttribute(Die, InputDIE, AttrSpec, AttrSize, Val,
                                       Unit);
   case dwarf::DW_FORM_block:
DIE *DwarfLinker::DIECloner::cloneDIE(
 
   // We are entering a subprogram. Get and propagate the PCOffset.
   if (Die->getTag() == dwarf::DW_TAG_subprogram)
     PCOffset = Info.AddrAdjust;
   AttrInfo.PCOffset = PCOffset;
 
   if (Abbrev->getTag() == dwarf::DW_TAG_subprogram) {
     Flags |= TF_InFunctionScope;
-    if (!Info.InDebugMap)
+    if (!Info.InDebugMap && LLVM_LIKELY(!Options.Update))
       Flags |= TF_SkipPC;
   }
 
   bool Copied = false;
   for (const auto &AttrSpec : Abbrev->attributes()) {
-    if (shouldSkipAttribute(AttrSpec, Die->getTag(), Info.InDebugMap,
+    if (LLVM_LIKELY(!Options.Update) &&
+        shouldSkipAttribute(AttrSpec, Die->getTag(), Info.InDebugMap,
                             Flags & TF_SkipPC, Flags & TF_InFunctionScope)) {
       DWARFFormValue::skipValue(AttrSpec.Form, Data, &Offset,
                                 U.getFormParams());
       // FIXME: dsymutil-classic keeps the old abbreviation around
       // even if it's not used. We can remove this (and the copyAbbrev
       // helper) as soon as bit-for-bit compatibility is not a goal anymore.
       if (!Copied) {
         copyAbbrev(*InputDIE.getAbbreviationDeclarationPtr(), Unit.hasODR());
if (CurrentUnit->getInfo(0).Keep) {
       // already has a DIE inside of it.
       CurrentUnit->createOutputDIE();
       cloneDIE(InputDIE, *CurrentUnit, 0 /* PC offset */,
                11 /* Unit Header size */, 0, CurrentUnit->getOutputUnitDIE());
     }
     Linker.OutputDebugInfoSize = CurrentUnit->computeNextUnitOffset();
     if (Linker.Options.NoOutput)
       continue;
-    // FIXME: for compatibility with the classic dsymutil, we emit
-    // an empty line table for the unit, even if the unit doesn't
-    // actually exist in the DIE tree.
+
+    if (LLVM_LIKELY(!Linker.Options.Update)) {
+      // FIXME: for compatibility with the classic dsymutil, we emit an empty
+      // line table for the unit, even if the unit doesn't actually exist in
+      // the DIE tree.
       Linker.patchLineTableForUnit(*CurrentUnit, DwarfContext);
       Linker.emitAcceleratorEntriesForUnit(*CurrentUnit);
       Linker.patchRangesForUnit(*CurrentUnit, DwarfContext);
       Linker.Streamer->emitLocationsForUnit(*CurrentUnit, DwarfContext);
+    } else {
+      Linker.emitAcceleratorEntriesForUnit(*CurrentUnit);
+    }
   }
 
   if (Linker.Options.NoOutput)
     return;
 
   // Emit all the compile unit's debug information.
   for (auto &CurrentUnit : CompileUnits) {
+    if (LLVM_LIKELY(!Linker.Options.Update))
       Linker.generateUnitRanges(*CurrentUnit);
     CurrentUnit->fixupForwardReferences();
     Linker.Streamer->emitCompileUnitHeader(*CurrentUnit);
     if (!CurrentUnit->getOutputUnitDIE())
       continue;
     Linker.Streamer->emitDIE(*CurrentUnit->getOutputUnitDIE());
   }
 }
 
for (const auto &Obj : Map.objects()) {
     }
 
     auto ErrOrObj = loadObject(BinHolder, *Obj, Map);
     if (!ErrOrObj)
       continue;
 
     // Look for relocations that correspond to debug map entries.
     RelocationManager RelocMgr(*this);
-    if (!RelocMgr.findValidRelocsInDebugInfo(*ErrOrObj, *Obj)) {
+    if (LLVM_LIKELY(!Options.Update) &&
+        !RelocMgr.findValidRelocsInDebugInfo(*ErrOrObj, *Obj)) {
       if (Options.Verbose)
         outs() << "No valid relocations found. Skipping.\n";
       continue;
     }
 
     // Setup access to the debug info.
     auto DwarfContext = DWARFContext::create(*ErrOrObj);
     startDebugObject(*DwarfContext, *Obj);
 
     // In a first phase, just read in the debug info and load all clang modules.
     for (const auto &CU : DwarfContext->compile_units()) {
       auto CUDie = CU->getUnitDIE(false);
       if (Options.Verbose) {
         outs() << "Input compilation unit:";
         DIDumpOptions DumpOpts;
         DumpOpts.RecurseDepth = 0;
         DumpOpts.Verbose = Options.Verbose;
         CUDie.dump(outs(), 0, DumpOpts);
       }
 
-      if (!registerModuleReference(CUDie, *CU, ModuleMap)) {
-        Units.push_back(llvm::make_unique<CompileUnit>(*CU, UnitID++,
-                                                       !Options.NoODR, ""));
+      if (!CUDie || LLVM_UNLIKELY(Options.Update) ||
+          !registerModuleReference(CUDie, *CU, ModuleMap)) {
+        Units.push_back(llvm::make_unique<CompileUnit>(
+            *CU, UnitID++, !Options.NoODR && !Options.Update, ""));
         maybeUpdateMaxDwarfVersion(CU->getVersion());
       }
     }
 
     // Now build the DIE parent links that we will use during the next phase.
     for (auto &CurrentUnit : Units)
       analyzeContextInfo(CurrentUnit->getOrigUnit().getUnitDIE(), 0, *CurrentUnit,
                          &ODRContexts.getRoot(), StringPool, ODRContexts);
 
     // Then mark all the DIEs that need to be present in the linked
     // output and collect some information about them. Note that this
-    // loop can not be merged with the previous one becaue cross-cu
+    // loop can not be merged with the previous one because cross-CU
     // references require the ParentIdx to be setup for every CU in
     // the object file before calling this.
+    if (LLVM_UNLIKELY(Options.Update)) {
+      for (auto &CurrentUnit : Units)
+        CurrentUnit->markEverythingAsKept();
+      Streamer->copyInvariantDebugSection(*ErrOrObj, Options);
+    } else {
       for (auto &CurrentUnit : Units)
-      lookForDIEsToKeep(RelocMgr, CurrentUnit->getOrigUnit().getUnitDIE(), *Obj,
-                        *CurrentUnit, 0);
+        lookForDIEsToKeep(RelocMgr, CurrentUnit->getOrigUnit().getUnitDIE(),
+                          *Obj, *CurrentUnit, 0);
+    }
 
     // The calls to applyValidRelocs inside cloneDIE will walk the
     // reloc array again (in the same way findValidRelocsInDebugInfo()
     // did). We need to reset the NextValidReloc index to the beginning.
     RelocMgr.resetValidRelocs();
-    if (RelocMgr.hasValidRelocs())
+    if (RelocMgr.hasValidRelocs() || LLVM_UNLIKELY(Options.Update))
       DIECloner(*this, RelocMgr, DIEAlloc, Units, Options)
           .cloneAllCompileUnits(*DwarfContext);
-    if (!Options.NoOutput && !Units.empty())
+    if (!Options.NoOutput && !Units.empty() && LLVM_LIKELY(!Options.Update))
       patchFrameInfoForObject(*Obj, *DwarfContext,
                               Units[0]->getOrigUnit().getAddressByteSize());
 
     // Clean-up before starting working on the next object.
     endDebugObject();
   }
 
   // Emit everything that's global.

llvm/trunk/tools/dsymutil/dsymutil.h

 struct LinkOptions {
   /// Verbosity
   bool Verbose = false;
 
   /// Skip emitting output
   bool NoOutput = false;
 
   /// Do not unique types according to ODR
-  bool NoODR;
+  bool NoODR = false;
+
+  /// Update
+  bool Update = false;
 
   /// Minimize
   bool Minimize = false;
 
   /// Do not check swiftmodule timestamp
   bool NoTimestamp = false;
 
   /// -oso-prepend-path

llvm/trunk/tools/dsymutil/dsymutil.cpp

static opt<bool> FlatOut("flat",
                          init(false), cat(DsymCategory));
 static alias FlatOutA("f", desc("Alias for --flat"), aliasopt(FlatOut));
 
 static opt<bool> Minimize(
     "minimize",
     desc("When used when creating a dSYM file, this option will suppress\n"
          "the emission of the .debug_inlines, .debug_pubnames, and\n"
          ".debug_pubtypes sections since dsymutil currently has better\n"
-         "equivalents: .apple_names and .apple_types."),
+         "equivalents: .apple_names and .apple_types. When used in\n"
+         "conjunction with --update option, this option will cause redundant\n"
+         "accelerator tables to be removed."),
     init(false), cat(DsymCategory));
 static alias MinimizeA("z", desc("Alias for --minimize"), aliasopt(Minimize));
 
+static opt<bool> Update(
+    "update",
+    desc("Updates existing dSYM files to contain the latest accelerator\n"
+         "tables and other DWARF optimizations. This option will currently\n"
+         "add the new .apple_names and .apple_types hashed accelerator\n"
+         "tables."),
+    init(false), cat(DsymCategory));
+static alias UpdateA("u", desc("Alias for --update"), aliasopt(Update));
+
 static opt<unsigned> NumThreads(
     "num-threads",
     desc("Specifies the maximum number (n) of simultaneous threads to use\n"
          "when linking multiple architectures."),
     value_desc("n"), init(0), cat(DsymCategory));
 static alias NumThreadsA("j", desc("Alias for --num-threads"),
                          aliasopt(NumThreads));
 
if (!success)
       errs() << "error: verification failed for " << Arch << '\n';
     return success;
   }
 
   return false;
 }
 
 static std::string getOutputFileName(llvm::StringRef InputFile) {
-  if (FlatOut) {
-    // If a flat dSYM has been requested, things are pretty simple.
+  // When updating, do in place replacement.
+  if (OutputFileOpt.empty() && Update)
+    return InputFile;
+
+  if (FlatOut) {
     if (OutputFileOpt.empty()) {
       if (InputFile == "-")
         return "a.out.dwarf";
       return (InputFile + ".dwarf").str();
     }
 
     return OutputFileOpt;
   }

 
 static Expected<sys::fs::TempFile> createTempFile() {
   llvm::SmallString<128> TmpModel;
   llvm::sys::path::system_temp_directory(true, TmpModel);
   llvm::sys::path::append(TmpModel, "dsym.tmp%%%%%.dwarf");
   return sys::fs::TempFile::create(TmpModel);
 }
 
+/// Parses the command line options into the LinkOptions struct and performs
+/// some sanity checking. Returns an error in case the latter fails.
+static Expected<LinkOptions> getOptions() {
+  LinkOptions Options;
+
+  Options.Verbose = Verbose;
+  Options.NoOutput = NoOutput;
+  Options.NoODR = NoODR;
+  Options.Minimize = Minimize;
+  Options.Update = Update;
+  Options.NoTimestamp = NoTimestamp;
+  Options.PrependPath = OsoPrependPath;
+
+  if (Options.Update && std::find(InputFiles.begin(), InputFiles.end(), "-") !=
+                            InputFiles.end()) {
+    // FIXME: We cannot use stdin for an update because stdin will be
+    // consumed by the BinaryHolder during the debugmap parsing, and
+    // then we will want to consume it again in DwarfLinker. If we
+    // used a unique BinaryHolder object that could cache multiple
+    // binaries this restriction would go away.
+    return make_error<StringError>(
+        "error: standard input cannot be used as input for a dSYM update.",
+        inconvertibleErrorCode());
+  }
+
+  return Options;
+}
+
+/// Return a list of input files. This function has logic for dealing with the
+/// special case where we might have dSYM bundles as input. The function
+/// returns an error when the directory structure doesn't match that of a dSYM
+/// bundle.
+static Expected<std::vector<std::string>> getInputs(bool DsymAsInput) {
+  if (!DsymAsInput)
+    return InputFiles;
+
+  // If we are updating, we might get dSYM bundles as input.
+  std::vector<std::string> Inputs;
+  for (const auto &Input : InputFiles) {
+    if (!llvm::sys::fs::is_directory(Input)) {
+      Inputs.push_back(Input);
+      continue;
+    }
+
+    // Make sure that we're dealing with a dSYM bundle.
+    std::string dSYMDir = Input + "/Contents/Resources/DWARF";
+    if (!llvm::sys::fs::is_directory(dSYMDir))
+      return make_error<StringError>(
+          Input + " is a directory, but doesn't look like a dSYM bundle.",
+          inconvertibleErrorCode());
+
+    // Create a directory iterator to iterate over all the entries in the
+    // bundle.
+    std::error_code EC;
+    llvm::sys::fs::directory_iterator DirIt(dSYMDir, EC);
+    llvm::sys::fs::directory_iterator DirEnd;
+    if (EC)
+      return errorCodeToError(EC);
+
+    // Add each entry to the list of inputs.
+    while (DirIt != DirEnd) {
+      Inputs.push_back(DirIt->path());
+      DirIt.increment(EC);
+      if (EC)
+        return errorCodeToError(EC);
+    }
+  }
+  return Inputs;
+}
+
 namespace {
 struct TempFileVector {
   std::vector<sys::fs::TempFile> Files;
   ~TempFileVector() {
     for (sys::fs::TempFile &Tmp : Files) {
       if (Error E = Tmp.discard())
         errs() << toString(std::move(E));
     }
   }
 };
 } // namespace
 
 int main(int argc, char **argv) {
   llvm::sys::PrintStackTraceOnErrorSignal(argv[0]);
   llvm::PrettyStackTraceProgram StackPrinter(argc, argv);
   llvm::llvm_shutdown_obj Shutdown;
-  LinkOptions Options;
   void *P = (void *)(intptr_t)getOutputFileName;
   std::string SDKPath = llvm::sys::fs::getMainExecutable(argv[0], P);
   SDKPath = llvm::sys::path::parent_path(SDKPath);
 
   HideUnrelatedOptions(DsymCategory);
   llvm::cl::ParseCommandLineOptions(
       argc, argv,
       "manipulate archived DWARF debug symbol files.\n\n"
       "dsymutil links the DWARF debug information found in the object files\n"
       "for the executable <input file> by using debug symbols information\n"
       "contained in its symbol table.\n");
 
   if (Help) {
     PrintHelpMessage();
     return 0;
   }
 
   if (Version) {
     llvm::cl::PrintVersionMessage();
     return 0;
   }
 
-  Options.Verbose = Verbose;
-  Options.NoOutput = NoOutput;
-  Options.NoODR = NoODR;
-  Options.Minimize = Minimize;
-  Options.NoTimestamp = NoTimestamp;
-  Options.PrependPath = OsoPrependPath;
+  auto OptionsOrErr = getOptions();
+  if (!OptionsOrErr) {
+    errs() << "error: " << toString(OptionsOrErr.takeError());
+    return 1;
+  }
 
   llvm::InitializeAllTargetInfos();
   llvm::InitializeAllTargetMCs();
   llvm::InitializeAllTargets();
   llvm::InitializeAllAsmPrinters();
 
+  auto InputsOrErr = getInputs(OptionsOrErr->Update);
+  if (!InputsOrErr) {
+    errs() << "error: " << toString(InputsOrErr.takeError()) << '\n';
+    return 1;
+  }
+
   if (!FlatOut && OutputFileOpt == "-") {
     llvm::errs() << "error: cannot emit to standard output without --flat\n";
     return 1;
   }
 
-  if (InputFiles.size() > 1 && FlatOut && !OutputFileOpt.empty()) {
+  if (InputsOrErr->size() > 1 && FlatOut && !OutputFileOpt.empty()) {
     llvm::errs() << "error: cannot use -o with multiple inputs in flat mode\n";
     return 1;
   }
 
   for (const auto &Arch : ArchFlags)
     if (Arch != "*" && Arch != "all" &&
         !llvm::object::MachOObjectFile::isValidArch(Arch)) {
       llvm::errs() << "error: Unsupported cpu architecture: '" << Arch << "'\n";
       return 1;
     }
 
-  for (auto &InputFile : InputFiles) {
+  for (auto &InputFile : *InputsOrErr) {
     // Dump the symbol table for each input file and requested arch
     if (DumpStab) {
       if (!dumpStab(InputFile, ArchFlags, OsoPrependPath))
         return 1;
       continue;
     }
 
     auto DebugMapPtrsOrErr = parseDebugMap(InputFile, ArchFlags, OsoPrependPath,
                                            Verbose, InputIsYAMLDebugMap);
 
     if (auto EC = DebugMapPtrsOrErr.getError()) {
       llvm::errs() << "error: cannot parse the debug map for \"" << InputFile
                    << "\": " << EC.message() << '\n';
       return 1;
     }
 
+    if (OptionsOrErr->Update) {
+      // The debug map should be empty. Add one object file corresponding to
+      // the input file.
+      for (auto &Map : *DebugMapPtrsOrErr)
+        Map->addDebugMapObject(InputFile,
+                               llvm::sys::TimePoint<std::chrono::seconds>());
+    }
+
+    // Ensure that the debug map is not empty (anymore).
     if (DebugMapPtrsOrErr->empty()) {
       llvm::errs() << "error: no architecture to link\n";
       return 1;
     }
 
     if (NumThreads == 0)
       NumThreads = llvm::thread::hardware_concurrency();
     if (DumpDebugMap || Verbose)
       NumThreads = 1;
     NumThreads = std::min<unsigned>(NumThreads, DebugMapPtrsOrErr->size());
 
     llvm::ThreadPool Threads(NumThreads);
 
     // If there is more than one link to execute, we need to generate
     // temporary files.
-    bool NeedsTempFiles = !DumpDebugMap && (*DebugMapPtrsOrErr).size() != 1;
+    bool NeedsTempFiles =
+        !DumpDebugMap && (OutputFileOpt != "-") &&
+        (DebugMapPtrsOrErr->size() != 1 || OptionsOrErr->Update);
+
     llvm::SmallVector<MachOUtils::ArchAndFilename, 4> TempFiles;
     TempFileVector TempFileStore;
     std::atomic_char AllOK(1);
     for (auto &Map : *DebugMapPtrsOrErr) {
       if (Verbose || DumpDebugMap)
         Map->print(llvm::outs());
 
       if (DumpDebugMap)
for (auto &Map : *DebugMapPtrsOrErr) {
         if (EC) {
           errs() << OutputFile << ": " << EC.message();
           return 1;
         }
       }
 
       auto LinkLambda = [&,
                          OutputFile](std::shared_ptr<raw_fd_ostream> Stream) {
-        AllOK.fetch_and(linkDwarf(*Stream, *Map, Options));
+        AllOK.fetch_and(linkDwarf(*Stream, *Map, *OptionsOrErr));
         Stream->flush();
         if (Verify && !NoOutput)
           AllOK.fetch_and(verify(OutputFile, Map->getTriple().getArchName()));
       };
 
       // FIXME: The DwarfLinker can have some very deep recursion that can max
       // out the (significantly smaller) stack when using threads. We don't
       // want this limitation when we only have a single thread.
       if (NumThreads == 1)
         LinkLambda(OS);
       else
         Threads.async(LinkLambda, OS);
     }
 
     Threads.wait();
 
     if (!AllOK)
       return 1;
 
     if (NeedsTempFiles &&
         !MachOUtils::generateUniversalBinary(
-            TempFiles, getOutputFileName(InputFile), Options, SDKPath))
+            TempFiles, getOutputFileName(InputFile), *OptionsOrErr, SDKPath))
       return 1;
   }
 
   return 0;
 }