[pdb] Don't build the entire source file list up front.

[zturner]

Summary

I decided to run llvm-pdbdump on a very large (~1.5GB) PDB to try and identify show-stopping performance problems. This is my attempt at fixing the first such problem.

When we load the Dbi Stream, before anyone even tries to access any data from it, we build up a set of information about every compiland. This includes, for each compiland, a list of all files. In this particular PDB file, there were about 85 million files. Note these aren't unique files, but since many two compilands can reference the same file (think headers), the combined list is very large.

There is no point doing this unless the user wants to look at the list of files to begin with, and even in that case there is no point building up a vector. If someone requests file 7 of module 6, we should do everything on the fly: Get the list of file offsets for module 6, take the 7th one, seek to that offset in the file names buffer, and return it.

In other words, for a fixed module with n source files, random access to that module's list of files is already O(1), so there is no point to spend O(n) time constructing a vector just to give you O(1) access, which you already had.

Moreover, precomputing the entire table (as was done before) is O(m*n) where m is the number of modules and n is the average number of source files / module.

This patch changes the up-front cost to O(m). It iterates the list of modules once, because they are variable length structures in a flat buffer. So by iterating it once we can get offsets to the beginning of each record in the buffer, thereby providing random access, thereby giving us future constant-time random access to both modules and files within modules.

There are some more performance improvements that need to be made in follow up patches, but I'll keep them separate.

[amccarth]

llvm/include/llvm/DebugInfo/PDB/Native/DbiModuleList.h

89 ↗

(On Diff #97840)

It seems ModeIndexArray is not needed? At least, we don't need to keep it around. Perhaps it should just be a local variable where you read it in order to skip past it.

llvm/lib/DebugInfo/PDB/Native/DbiModuleList.cpp

40 ↗

(On Diff #97840)

s/They/Their/

118 ↗

(On Diff #97840)

The naming here seems misleading. Once you add Filei, it's no longer the _first_ offset. How about:

uint32_t Offset = Modules->ModuleInitialFileIndex[Modi] + Filei;

135 ↗

(On Diff #97840)

If I'm following, it seems >= is just defensive and that == should be sufficient. In that case, what if you assert(Filei <= Modules->getSourceFileCount(Modi)) to help catch bugs that the defensiveness would ordinarily hide?

151 ↗

(On Diff #97840)

Huh? Couldn't they both be the end of the same module? I don't see where you've ruled that out at this point.

198 ↗

(On Diff #97840)

I'm having trouble understanding this comment, for a few reasons:

1. I'm having trouble figuring out exactly what some of the "it"s refer to.
2. It says we're not going to use the first array, but, in fact, you save it into a member variable ModIndexArray. If we're not going to use it, why keep it around?
3. Are you using "modules" to mean more than one thing here? We're talking about compilands here, right?
4. I don't follow that logic that >64K source files means >64K object files. Does source files include header files? If so, isn't it likely that there are fewer object files than source files?

207 ↗

(On Diff #97840)

But the comment above just said we don't use NumSourceFiles, so why sum them up. I assume that's a comment bug.

[zturner]

Fixed issues pointed out by amccarth.

[zturner]

llvm/lib/DebugInfo/PDB/Native/DbiModuleList.cpp

151 ↗

(On Diff #97840)

Good catch. I think this newer logic is much easier to follow.

198 ↗

(On Diff #97840)

The comment was actually all kinds of wrong, hopefully it's more clear now.

207 ↗

(On Diff #97840)

It was confusingly referring to NumSourceFiles inside of the header. Fixed.

216–219 ↗

(On Diff #97861)

This comment was also wrong btw, so it's also fixed.

[zturner]

*Still* got the isCompatible check wrong. I think this is right now.

[amccarth]

Thanks for the changes. Those make it much easier to understand.

llvm/trunk/include/llvm/DebugInfo/PDB/Native/DbiModuleDescriptor.h

public:
   uint32_t getRecordLength() const;
 
 private:
   StringRef ModuleName;
   StringRef ObjFileName;
   const ModuleInfoHeader *Layout = nullptr;
 };
 
-struct ModuleInfoEx {
-  ModuleInfoEx(const DbiModuleDescriptor &Info) : Info(Info) {}
-  ModuleInfoEx(const ModuleInfoEx &Ex) = default;
-
-  DbiModuleDescriptor Info;
-  std::vector<StringRef> SourceFiles;
-};
-
 } // end namespace pdb
 
 template <> struct VarStreamArrayExtractor<pdb::DbiModuleDescriptor> {
   typedef void ContextType;
   static Error extract(BinaryStreamRef Stream, uint32_t &Length,
                        pdb::DbiModuleDescriptor &Info) {
     if (auto EC = pdb::DbiModuleDescriptor::initialize(Stream, Info))
       return EC;
     Length = Info.getRecordLength();
     return Error::success();
   }
 };
 
 } // end namespace llvm
 
 #endif // LLVM_DEBUGINFO_PDB_RAW_DBIMODULEDESCRIPTOR_H

llvm/trunk/include/llvm/DebugInfo/PDB/Native/DbiModuleList.h

+//===- DbiModuleList.h - PDB module information list ------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_DEBUGINFO_PDB_RAW_DBIMODULELIST_H
+#define LLVM_DEBUGINFO_PDB_RAW_DBIMODULELIST_H
+
+#include "llvm/ADT/StringRef.h"
+#include "llvm/ADT/iterator.h"
+#include "llvm/DebugInfo/PDB/Native/DbiModuleDescriptor.h"
+#include "llvm/Support/BinaryStreamArray.h"
+#include "llvm/Support/BinaryStreamRef.h"
+#include "llvm/Support/Endian.h"
+#include "llvm/Support/Error.h"
+#include <cstdint>
+#include <vector>
+
+namespace llvm {
+namespace pdb {
+
+class DbiModuleList;
+struct FileInfoSubstreamHeader;
+
+class DbiModuleSourceFilesIterator
+    : public iterator_facade_base<DbiModuleSourceFilesIterator,
+                                  std::random_access_iterator_tag, StringRef> {
+  typedef iterator_facade_base<DbiModuleSourceFilesIterator,
+                               std::random_access_iterator_tag, StringRef>
+      BaseType;
+
+public:
+  DbiModuleSourceFilesIterator(const DbiModuleList &Modules, uint32_t Modi,
+                               uint16_t Filei);
+  DbiModuleSourceFilesIterator() = default;
+  DbiModuleSourceFilesIterator &
+  operator=(const DbiModuleSourceFilesIterator &R) = default;
+
+  bool operator==(const DbiModuleSourceFilesIterator &R) const;
+
+  const StringRef &operator*() const { return ThisValue; }
+  StringRef &operator*() { return ThisValue; }
+
+  bool operator<(const DbiModuleSourceFilesIterator &RHS) const;
+  std::ptrdiff_t operator-(const DbiModuleSourceFilesIterator &R) const;
+  DbiModuleSourceFilesIterator &operator+=(std::ptrdiff_t N);
+  DbiModuleSourceFilesIterator &operator-=(std::ptrdiff_t N);
+
+private:
+  void setValue();
+
+  bool isEnd() const;
+  bool isCompatible(const DbiModuleSourceFilesIterator &R) const;
+  bool isUniversalEnd() const;
+
+  StringRef ThisValue;
+  const DbiModuleList *Modules{nullptr};
+  uint32_t Modi{0};
+  uint16_t Filei{0};
+};
+
+class DbiModuleList {
+  friend DbiModuleSourceFilesIterator;
+
+public:
+  Error initialize(BinaryStreamRef ModInfo, BinaryStreamRef FileInfo);
+
+  Expected<StringRef> getFileName(uint32_t Index) const;
+  uint32_t getModuleCount() const;
+  uint32_t getSourceFileCount() const;
+  uint16_t getSourceFileCount(uint32_t Modi) const;
+
+  iterator_range<DbiModuleSourceFilesIterator>
+  source_files(uint32_t Modi) const;
+
+  DbiModuleDescriptor getModuleDescriptor(uint32_t Modi) const;
+
+private:
+  Error initializeModInfo(BinaryStreamRef ModInfo);
+  Error initializeFileInfo(BinaryStreamRef FileInfo);
+
+  VarStreamArray<DbiModuleDescriptor> Descriptors;
+
+  FixedStreamArray<support::little32_t> FileNameOffsets;
+  FixedStreamArray<support::ulittle16_t> ModFileCountArray;
+
+  // For each module, there are multiple filenames, which can be obtained by
+  // knowing the index of the file.  Given the index of the file, one can use
+  // that as an offset into the FileNameOffsets array, which contains the
+  // absolute offset of the file name in NamesBuffer.  Thus, for each module
+  // we store the first index in the FileNameOffsets array for this module.
+  // The number of files for the corresponding module is stored in
+  // ModFileCountArray.
+  std::vector<uint32_t> ModuleInitialFileIndex;
+
+  // In order to provide random access into the Descriptors array, we iterate it
+  // once up front to find the offsets of the individual items and store them in
+  // this array.
+  std::vector<uint32_t> ModuleDescriptorOffsets;
+
+  const FileInfoSubstreamHeader *FileInfoHeader = nullptr;
+
+  BinaryStreamRef ModInfoSubstream;
+  BinaryStreamRef FileInfoSubstream;
+  BinaryStreamRef NamesBuffer;
+};
+}
+}
+
+#endif // LLVM_DEBUGINFO_PDB_RAW_DBIMODULELIST_H
 No newline at end of file

llvm/trunk/include/llvm/DebugInfo/PDB/Native/DbiStream.h

 //===- DbiStream.h - PDB Dbi Stream (Stream 3) Access -----------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_DEBUGINFO_PDB_RAW_PDBDBISTREAM_H
 #define LLVM_DEBUGINFO_PDB_RAW_PDBDBISTREAM_H
 
 #include "llvm/DebugInfo/CodeView/ModuleDebugFragment.h"
 #include "llvm/DebugInfo/MSF/MappedBlockStream.h"
 #include "llvm/DebugInfo/PDB/Native/DbiModuleDescriptor.h"
+#include "llvm/DebugInfo/PDB/Native/DbiModuleList.h"
 #include "llvm/DebugInfo/PDB/Native/PDBStringTable.h"
 #include "llvm/DebugInfo/PDB/Native/RawConstants.h"
 #include "llvm/DebugInfo/PDB/Native/RawTypes.h"
 #include "llvm/DebugInfo/PDB/PDBTypes.h"
 #include "llvm/Support/BinaryStreamArray.h"
 #include "llvm/Support/BinaryStreamArray.h"
 #include "llvm/Support/BinaryStreamRef.h"
 #include "llvm/Support/BinaryStreamRef.h"
public:
   uint32_t getSymRecordStreamIndex() const;
 
   PDB_Machine getMachineType() const;
 
   /// If the given stream type is present, returns its stream index. If it is
   /// not present, returns InvalidStreamIndex.
   uint32_t getDebugStreamIndex(DbgHeaderType Type) const;
 
-  ArrayRef<ModuleInfoEx> modules() const;
-
-  Expected<StringRef> getFileNameForIndex(uint32_t Index) const;
+  const DbiModuleList &modules() const;
 
   FixedStreamArray<object::coff_section> getSectionHeaders();
 
   FixedStreamArray<object::FpoData> getFpoRecords();
 
   FixedStreamArray<SecMapEntry> getSectionMap() const;
   void visitSectionContributions(ISectionContribVisitor &Visitor) const;
 
 private:
-  Error initializeModInfoArray();
   Error initializeSectionContributionData();
   Error initializeSectionHeadersData();
   Error initializeSectionMapData();
-  Error initializeFileInfo();
   Error initializeFpoRecords();
 
   PDBFile &Pdb;
   std::unique_ptr<msf::MappedBlockStream> Stream;
 
-  std::vector<ModuleInfoEx> ModuleInfos;
   PDBStringTable ECNames;
 
-  BinaryStreamRef ModInfoSubstream;
   BinaryStreamRef SecContrSubstream;
   BinaryStreamRef SecMapSubstream;
-  BinaryStreamRef FileInfoSubstream;
   BinaryStreamRef TypeServerMapSubstream;
   BinaryStreamRef ECSubstream;
 
-  BinaryStreamRef NamesBuffer;
+  DbiModuleList Modules;
 
   FixedStreamArray<support::ulittle16_t> DbgStreams;
 
   PdbRaw_DbiSecContribVer SectionContribVersion;
   FixedStreamArray<SectionContrib> SectionContribs;
   FixedStreamArray<SectionContrib2> SectionContribs2;
   FixedStreamArray<SecMapEntry> SectionMap;
-  FixedStreamArray<support::little32_t> FileNameOffsets;
 
   std::unique_ptr<msf::MappedBlockStream> SectionHeaderStream;
   FixedStreamArray<object::coff_section> SectionHeaders;
 
   std::unique_ptr<msf::MappedBlockStream> FpoStream;
   FixedStreamArray<object::FpoData> FpoRecords;
 
   const DbiStreamHeader *Header;
 };
 }
 }
 
 #endif

llvm/trunk/include/llvm/DebugInfo/PDB/Native/NativeCompilandSymbol.h

 #include "llvm/DebugInfo/PDB/Native/DbiModuleDescriptor.h"
 #include "llvm/DebugInfo/PDB/Native/NativeRawSymbol.h"
 
 namespace llvm {
 namespace pdb {
 
 class NativeCompilandSymbol : public NativeRawSymbol {
 public:
-  NativeCompilandSymbol(NativeSession &Session, const ModuleInfoEx &MI);
+  NativeCompilandSymbol(NativeSession &Session, DbiModuleDescriptor MI);
   PDB_SymType getSymTag() const override;
   bool isEditAndContinueEnabled() const override;
   uint32_t getLexicalParentId() const override;
   std::string getLibraryName() const override;
   std::string getName() const override;
 
 private:
-  ModuleInfoEx Module;
+  DbiModuleDescriptor Module;
 };
 
 } // namespace pdb
 } // namespace llvm
 
 #endif

llvm/trunk/include/llvm/DebugInfo/PDB/Native/NativeEnumModules.h

 #define LLVM_DEBUGINFO_PDB_NATIVE_NATIVEENUMMODULES_H
 
 #include "llvm/DebugInfo/PDB/IPDBEnumChildren.h"
 #include "llvm/DebugInfo/PDB/Native/DbiModuleDescriptor.h"
 #include "llvm/DebugInfo/PDB/PDBSymbol.h"
 namespace llvm {
 namespace pdb {
 
+class DbiModuleList;
 class NativeSession;
 
 class NativeEnumModules : public IPDBEnumChildren<PDBSymbol> {
 public:
-  explicit NativeEnumModules(NativeSession &Session,
-                             ArrayRef<ModuleInfoEx> Modules,
-                             uint32_t Index = 0);
+  NativeEnumModules(NativeSession &Session, const DbiModuleList &Modules,
 
   uint32_t getChildCount() const override;
   std::unique_ptr<PDBSymbol> getChildAtIndex(uint32_t Index) const override;
   std::unique_ptr<PDBSymbol> getNext() override;
   void reset() override;
   NativeEnumModules *clone() const override;
 
 private:
   NativeSession &Session;
-  ArrayRef<ModuleInfoEx> Modules;
+  const DbiModuleList &Modules;
   uint32_t Index;
 };
 }
 }
 
 #endif

llvm/trunk/include/llvm/DebugInfo/PDB/Native/RawTypes.h

struct ModInfoFlags {
   ///  uint16_t iTSM : 8;       // Type Server Index for this module
   static const uint16_t HasECFlagMask = 0x2;
 
   static const uint16_t TypeServerIndexMask = 0xFF00;
   static const uint16_t TypeServerIndexShift = 8;
 };
 
 /// The header preceeding each entry in the Module Info substream of the DBI
-/// stream.
+/// stream.  Corresponds to the type MODI in the reference implementation.
 struct ModuleInfoHeader {
   /// Currently opened module. This field is a pointer in the reference
   /// implementation, but that won't work on 64-bit systems, and anyway it
   /// doesn't make sense to read a pointer from a file. For now it is unused,
   /// so just ignore it.
   support::ulittle32_t Mod;
 
   /// First section contribution of this module.
struct ModuleInfoHeader {
   support::ulittle32_t C13Bytes;
 
   /// Number of files contributing to this module
   support::ulittle16_t NumFiles;
 
   /// Padding so the next field is 4-byte aligned.
   char Padding1[2];
 
-  /// Array of [0..NumFiles) DBI name buffer offsets.  This field is a pointer
-  /// in the reference implementation, but as with `Mod`, we ignore it for now
-  /// since it is unused.
+  /// Array of [0..NumFiles) DBI name buffer offsets.  In the reference
+  /// implementation this field is a pointer.  But since you can't portably
+  /// serialize a pointer, on 64-bit platforms they copy all the values except
+  /// this one into the 32-bit version of the struct and use that for
+  /// serialization.  Regardless, this field is unused, it is only there to
+  /// store a pointer that can be accessed at runtime.
   support::ulittle32_t FileNameOffs;
 
   /// Name Index for src file name
   support::ulittle32_t SrcFileNameNI;
 
   /// Name Index for path to compiler PDB
   support::ulittle32_t PdbFilePathNI;
 

llvm/trunk/include/llvm/Support/BinaryStreamArray.h

Value &operator*() {
     assert(Array && !HasError);
     return ThisValue;
   }
 
   IterType &operator+=(unsigned N) {
     for (unsigned I = 0; I < N; ++I) {
       // We are done with the current record, discard it so that we are
       // positioned at the next record.
+      AbsOffset += ThisLen;
       IterRef = IterRef.drop_front(ThisLen);
       if (IterRef.getLength() == 0) {
         // There is nothing after the current record, we must make this an end
         // iterator.
         moveToEnd();
       } else {
         // There is some data after the current record.
         auto EC = Ctx->template invoke<Extractor>(IterRef, ThisLen, ThisValue);
         if (EC) {
           consumeError(std::move(EC));
           markError();
         } else if (ThisLen == 0) {
           // An empty record? Make this an end iterator.
           moveToEnd();
         }
       }
     }
     return *this;
   }
 
+  uint32_t offset() const { return AbsOffset; }
+
 private:
   void moveToEnd() {
     Array = nullptr;
     ThisLen = 0;
   }
   void markError() {
     moveToEnd();
     HasError = true;
     if (HadError != nullptr)
       *HadError = true;
   }
 
   Value ThisValue;
   BinaryStreamRef IterRef;
   const WrappedCtx *Ctx{nullptr};
   const ArrayType *Array{nullptr};
   uint32_t ThisLen{0};
+  uint32_t AbsOffset{0};
   bool HasError{false};
   bool *HadError{nullptr};
 };
 
 template <typename T, typename Context> struct ContextWrapper {
   ContextWrapper() = default;
 
   explicit ContextWrapper(Context &&Ctx) : Ctx(Ctx) {}

llvm/trunk/lib/DebugInfo/PDB/CMakeLists.txt

add_pdb_impl_folder(DIA
     )
 
     set(LIBPDB_ADDITIONAL_HEADER_DIRS "${LLVM_MAIN_INCLUDE_DIR}/llvm/DebugInfo/PDB/DIA")
 endif()
 
 add_pdb_impl_folder(Native
   Native/DbiModuleDescriptor.cpp
   Native/DbiModuleDescriptorBuilder.cpp
+  Native/DbiModuleList.cpp
   Native/DbiStream.cpp
   Native/DbiStreamBuilder.cpp
   Native/EnumTables.cpp
   Native/GlobalsStream.cpp
   Native/GSI.cpp
   Native/Hash.cpp
   Native/HashTable.cpp
   Native/InfoStream.cpp

llvm/trunk/lib/DebugInfo/PDB/Native/DbiModuleList.cpp

+//===- DbiModuleList.cpp - PDB module information list ----------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+#include "llvm/DebugInfo/PDB/Native/DbiModuleList.h"
+
+#include "llvm/DebugInfo/PDB/Native/RawError.h"
+#include "llvm/Support/Error.h"
+
+using namespace llvm;
+using namespace llvm::pdb;
+
+DbiModuleSourceFilesIterator::DbiModuleSourceFilesIterator(
+    const DbiModuleList &Modules, uint32_t Modi, uint16_t Filei)
+    : Modules(&Modules), Modi(Modi), Filei(Filei) {
+  setValue();
+}
+
+bool DbiModuleSourceFilesIterator::
+operator==(const DbiModuleSourceFilesIterator &R) const {
+  // incompatible iterators are never equal
+  if (!isCompatible(R))
+    return false;
+
+  // If they're compatible, and they're both ends, then they're equal.
+  if (isEnd() && R.isEnd())
+    return true;
+
+  // If one is an end and the other is not, they're not equal.
+  if (isEnd() != R.isEnd())
+    return false;
+
+  // Now we know:
+  // - They're compatible
+  // - They're not *both* end iterators
+  // - Their endness is the same.
+  // Thus, they're compatible iterators pointing to a valid file on the same
+  // module.  All we need to check are the file indices.
+  assert(Modules == R.Modules);
+  assert(Modi == R.Modi);
+  assert(!isEnd());
+  assert(!R.isEnd());
+
+  return (Filei == R.Filei);
+}
+
+bool DbiModuleSourceFilesIterator::
+operator<(const DbiModuleSourceFilesIterator &R) const {
+  assert(isCompatible(R));
+
+  // It's not sufficient to compare the file indices, because default
+  // constructed iterators could be equal to iterators with valid indices.  To
+  // account for this, early-out if they're equal.
+  if (*this == R)
+    return false;
+
+  return Filei < R.Filei;
+}
+
+std::ptrdiff_t DbiModuleSourceFilesIterator::
+operator-(const DbiModuleSourceFilesIterator &R) const {
+  assert(isCompatible(R));
+  assert(!(*this < R));
+
+  // If they're both end iterators, the distance is 0.
+  if (isEnd() && R.isEnd())
+    return 0;
+
+  assert(!R.isEnd());
+
+  // At this point, R cannot be end, but *this can, which means that *this
+  // might be a universal end iterator with none of its fields set.  So in that
+  // case have to rely on R as the authority to figure out how many files there
+  // are to compute the distance.
+  uint32_t Thisi = Filei;
+  if (isEnd()) {
+    uint32_t RealModi = R.Modi;
+    Thisi = R.Modules->getSourceFileCount(RealModi);
+  }
+
+  assert(Thisi >= R.Filei);
+  return Thisi - R.Filei;
+}
+
+DbiModuleSourceFilesIterator &DbiModuleSourceFilesIterator::
+operator+=(std::ptrdiff_t N) {
+  assert(!isEnd());
+
+  Filei += N;
+  assert(Filei <= Modules->getSourceFileCount(Modi));
+  setValue();
+  return *this;
+}
+
+DbiModuleSourceFilesIterator &DbiModuleSourceFilesIterator::
+operator-=(std::ptrdiff_t N) {
+  // Note that we can subtract from an end iterator, but not a universal end
+  // iterator.
+  assert(!isUniversalEnd());
+
+  assert(N <= Filei);
+
+  Filei -= N;
+  return *this;
+}
+
+void DbiModuleSourceFilesIterator::setValue() {
+  if (isEnd()) {
+    ThisValue = "";
+    return;
+  }
+
+  uint32_t Off = Modules->ModuleInitialFileIndex[Modi] + Filei;
+  auto ExpectedValue = Modules->getFileName(Off);
+  if (!ExpectedValue) {
+    consumeError(ExpectedValue.takeError());
+    Filei = Modules->getSourceFileCount(Modi);
+  } else
+    ThisValue = *ExpectedValue;
+}
+
+bool DbiModuleSourceFilesIterator::isEnd() const {
+  if (isUniversalEnd())
+    return true;
+
+  assert(Modules);
+  assert(Modi <= Modules->getModuleCount());
+  assert(Filei <= Modules->getSourceFileCount(Modi));
+
+  if (Modi == Modules->getModuleCount())
+    return true;
+  if (Filei == Modules->getSourceFileCount(Modi))
+    return true;
+  return false;
+}
+
+bool DbiModuleSourceFilesIterator::isUniversalEnd() const { return !Modules; }
+
+bool DbiModuleSourceFilesIterator::isCompatible(
+    const DbiModuleSourceFilesIterator &R) const {
+  // Universal iterators are compatible with any other iterator.
+  if (isUniversalEnd() || R.isUniversalEnd())
+    return true;
+
+  // At this point, neither iterator is a universal end iterator, although one
+  // or both might be non-universal end iterators.  Regardless, the module index
+  // is valid, so they are compatible if and only if they refer to the same
+  // module.
+  return Modi == R.Modi;
+}
+
+Error DbiModuleList::initialize(BinaryStreamRef ModInfo,
+                                BinaryStreamRef FileInfo) {
+  if (auto EC = initializeModInfo(ModInfo))
+    return EC;
+  if (auto EC = initializeFileInfo(FileInfo))
+    return EC;
+
+  return Error::success();
+}
+
+Error DbiModuleList::initializeModInfo(BinaryStreamRef ModInfo) {
+  ModInfoSubstream = ModInfo;
+
+  if (ModInfo.getLength() == 0)
+    return Error::success();
+
+  BinaryStreamReader Reader(ModInfo);
+
+  if (auto EC = Reader.readArray(Descriptors, ModInfo.getLength()))
+    return EC;
+
+  return Error::success();
+}
+
+Error DbiModuleList::initializeFileInfo(BinaryStreamRef FileInfo) {
+  FileInfoSubstream = FileInfo;
+
+  if (FileInfo.getLength() == 0)
+    return Error::success();
+
+  BinaryStreamReader FISR(FileInfo);
+  if (auto EC = FISR.readObject(FileInfoHeader))
+    return EC;
+
+  // First is an array of `NumModules` module indices.  This does not seem to be
+  // used for anything meaningful, so we ignore it.
+  FixedStreamArray<support::ulittle16_t> ModuleIndices;
+  if (auto EC = FISR.readArray(ModuleIndices, FileInfoHeader->NumModules))
+    return EC;
+  if (auto EC = FISR.readArray(ModFileCountArray, FileInfoHeader->NumModules))
+    return EC;
+
+  // Compute the real number of source files.  We can't trust the value in
+  // `FileInfoHeader->NumSourceFiles` because it is a unit16, and the sum of all
+  // source file counts might be larger than a unit16.  So we compute the real
+  // count by summing up the individual counts.
+  uint32_t NumSourceFiles = 0;
+  for (auto Count : ModFileCountArray)
+    NumSourceFiles += Count;
+
+  // In the reference implementation, this array is where the pointer documented
+  // at the definition of ModuleInfoHeader::FileNameOffs points to.  Note that
+  // although the field in ModuleInfoHeader is ignored this array is not, as it
+  // is the authority on where each filename begins in the names buffer.
+  if (auto EC = FISR.readArray(FileNameOffsets, NumSourceFiles))
+    return EC;
+
+  if (auto EC = FISR.readStreamRef(NamesBuffer))
+    return EC;
+
+  auto DescriptorIter = Descriptors.begin();
+  uint32_t NextFileIndex = 0;
+  ModuleInitialFileIndex.resize(FileInfoHeader->NumModules);
+  ModuleDescriptorOffsets.resize(FileInfoHeader->NumModules);
+  for (size_t I = 0; I < FileInfoHeader->NumModules; ++I) {
+    assert(DescriptorIter != Descriptors.end());
+    ModuleInitialFileIndex[I] = NextFileIndex;
+    ModuleDescriptorOffsets[I] = DescriptorIter.offset();
+
+    NextFileIndex += ModFileCountArray[I];
+    ++DescriptorIter;
+  }
+
+  assert(DescriptorIter == Descriptors.end());
+  assert(NextFileIndex == NumSourceFiles);
+
+  return Error::success();
+}
+
+uint32_t DbiModuleList::getModuleCount() const {
+  return FileInfoHeader->NumModules;
+}
+
+uint32_t DbiModuleList::getSourceFileCount() const {
+  return FileNameOffsets.size();
+}
+
+uint16_t DbiModuleList::getSourceFileCount(uint32_t Modi) const {
+  return ModFileCountArray[Modi];
+}
+
+DbiModuleDescriptor DbiModuleList::getModuleDescriptor(uint32_t Modi) const {
+  assert(Modi < getModuleCount());
+  uint32_t Offset = ModuleDescriptorOffsets[Modi];
+  auto Iter = Descriptors.at(Offset);
+  assert(Iter != Descriptors.end());
+  return *Iter;
+}
+
+iterator_range<DbiModuleSourceFilesIterator>
+DbiModuleList::source_files(uint32_t Modi) const {
+  return make_range<DbiModuleSourceFilesIterator>(
+      DbiModuleSourceFilesIterator(*this, Modi, 0),
+      DbiModuleSourceFilesIterator());
+}
+
+Expected<StringRef> DbiModuleList::getFileName(uint32_t Index) const {
+  BinaryStreamReader Names(NamesBuffer);
+  if (Index >= getSourceFileCount())
+    return make_error<RawError>(raw_error_code::index_out_of_bounds);
+
+  uint32_t FileOffset = FileNameOffsets[Index];
+  Names.setOffset(FileOffset);
+  StringRef Name;
+  if (auto EC = Names.readCString(Name))
+    return std::move(EC);
+  return Name;
+}

llvm/trunk/lib/DebugInfo/PDB/Native/DbiStream.cpp

return make_error<RawError>(raw_error_code::corrupt_file,
                                 "DBI section map substream not aligned.");
   if (Header->FileInfoSize % sizeof(uint32_t) != 0)
     return make_error<RawError>(raw_error_code::corrupt_file,
                                 "DBI file info substream not aligned.");
   if (Header->TypeServerSize % sizeof(uint32_t) != 0)
     return make_error<RawError>(raw_error_code::corrupt_file,
                                 "DBI type server substream not aligned.");
 
+  BinaryStreamRef ModInfoSubstream;
+  BinaryStreamRef FileInfoSubstream;
   if (auto EC =
           Reader.readStreamRef(ModInfoSubstream, Header->ModiSubstreamSize))
     return EC;
-  if (auto EC = initializeModInfoArray())
-    return EC;
 
   if (auto EC = Reader.readStreamRef(SecContrSubstream,
                                      Header->SecContrSubstreamSize))
     return EC;
   if (auto EC = Reader.readStreamRef(SecMapSubstream, Header->SectionMapSize))
     return EC;
   if (auto EC = Reader.readStreamRef(FileInfoSubstream, Header->FileInfoSize))
     return EC;
   if (auto EC =
           Reader.readStreamRef(TypeServerMapSubstream, Header->TypeServerSize))
     return EC;
   if (auto EC = Reader.readStreamRef(ECSubstream, Header->ECSubstreamSize))
     return EC;
   if (auto EC = Reader.readArray(
           DbgStreams, Header->OptionalDbgHdrSize / sizeof(ulittle16_t)))
     return EC;
 
+  if (auto EC = Modules.initialize(ModInfoSubstream, FileInfoSubstream))
+    return EC;
+
   if (auto EC = initializeSectionContributionData())
     return EC;
   if (auto EC = initializeSectionHeadersData())
     return EC;
   if (auto EC = initializeSectionMapData())
     return EC;
-  if (auto EC = initializeFileInfo())
-    return EC;
   if (auto EC = initializeFpoRecords())
     return EC;
 
   if (Reader.bytesRemaining() > 0)
     return make_error<RawError>(raw_error_code::corrupt_file,
                                 "Found unexpected bytes in DBI Stream.");
 
   if (ECSubstream.getLength() > 0) {

 FixedStreamArray<object::coff_section> DbiStream::getSectionHeaders() {
   return SectionHeaders;
 }
 
 FixedStreamArray<object::FpoData> DbiStream::getFpoRecords() {
   return FpoRecords;
 }
 
-ArrayRef<ModuleInfoEx> DbiStream::modules() const { return ModuleInfos; }
+const DbiModuleList &DbiStream::modules() const { return Modules; }
+
 FixedStreamArray<SecMapEntry> DbiStream::getSectionMap() const {
   return SectionMap;
 }
 
 void DbiStream::visitSectionContributions(
     ISectionContribVisitor &Visitor) const {
   if (SectionContribVersion == DbiSecContribVer60) {
     for (auto &SC : SectionContribs)
if (SectionContribVersion == DbiSecContribVer60)
     return loadSectionContribs<SectionContrib>(SectionContribs, SCReader);
   if (SectionContribVersion == DbiSecContribV2)
     return loadSectionContribs<SectionContrib2>(SectionContribs2, SCReader);
 
   return make_error<RawError>(raw_error_code::feature_unsupported,
                               "Unsupported DBI Section Contribution version");
 }
 
-Error DbiStream::initializeModInfoArray() {
-  if (ModInfoSubstream.getLength() == 0)
-    return Error::success();
-
-  // Since each DbiModuleDescriptor in the stream is a variable length, we have
-  // to iterate
-  // them to know how many there actually are.
-  BinaryStreamReader Reader(ModInfoSubstream);
-
-  VarStreamArray<DbiModuleDescriptor> ModInfoArray;
-  if (auto EC = Reader.readArray(ModInfoArray, ModInfoSubstream.getLength()))
-    return EC;
-  for (auto &Info : ModInfoArray) {
-    ModuleInfos.emplace_back(Info);
-  }
-
-  return Error::success();
-}
-
 // Initializes this->SectionHeaders.
 Error DbiStream::initializeSectionHeadersData() {
   if (DbgStreams.size() == 0)
     return Error::success();
 
   uint32_t StreamNum = getDebugStreamIndex(DbgHeaderType::SectionHdr);
   if (StreamNum >= Pdb.getNumStreams())
     return make_error<RawError>(raw_error_code::no_stream);
Error DbiStream::initializeSectionMapData() {
   const SecMapHeader *Header;
   if (auto EC = SMReader.readObject(Header))
     return EC;
   if (auto EC = SMReader.readArray(SectionMap, Header->SecCount))
     return EC;
   return Error::success();
 }
 
-Error DbiStream::initializeFileInfo() {
-  if (FileInfoSubstream.getLength() == 0)
-    return Error::success();
-
-  const FileInfoSubstreamHeader *FH;
-  BinaryStreamReader FISR(FileInfoSubstream);
-  if (auto EC = FISR.readObject(FH))
-    return EC;
-
-  // The number of modules in the stream should be the same as reported by
-  // the FileInfoSubstreamHeader.
-  if (FH->NumModules != ModuleInfos.size())
-    return make_error<RawError>(raw_error_code::corrupt_file,
-                                "FileInfo substream count doesn't match DBI.");
-
-  FixedStreamArray<ulittle16_t> ModIndexArray;
-  FixedStreamArray<ulittle16_t> ModFileCountArray;
-
-  // First is an array of `NumModules` module indices.  This is not used for the
-  // same reason that `NumSourceFiles` is not used.  It's an array of uint16's,
-  // but it's possible there are more than 64k source files, which would imply
-  // more than 64k modules (e.g. object files) as well.  So we ignore this
-  // field.
-  if (auto EC = FISR.readArray(ModIndexArray, ModuleInfos.size()))
-    return EC;
-  if (auto EC = FISR.readArray(ModFileCountArray, ModuleInfos.size()))
-    return EC;
-
-  // Compute the real number of source files.
-  uint32_t NumSourceFiles = 0;
-  for (auto Count : ModFileCountArray)
-    NumSourceFiles += Count;
-
-  // This is the array that in the reference implementation corresponds to
-  // `DbiModuleDescriptor::FileLayout::FileNameOffs`, which is commented there
-  // as being a
-  // pointer. Due to the mentioned problems of pointers causing difficulty
-  // when reading from the file on 64-bit systems, we continue to ignore that
-  // field in `DbiModuleDescriptor`, and instead build a vector of StringRefs
-  // and stores
-  // them in `ModuleInfoEx`.  The value written to and read from the file is
-  // not used anyway, it is only there as a way to store the offsets for the
-  // purposes of later accessing the names at runtime.
-  if (auto EC = FISR.readArray(FileNameOffsets, NumSourceFiles))
-    return EC;
-
-  if (auto EC = FISR.readStreamRef(NamesBuffer))
-    return EC;
-
-  // We go through each ModuleInfo, determine the number N of source files for
-  // that module, and then get the next N offsets from the Offsets array, using
-  // them to get the corresponding N names from the Names buffer and associating
-  // each one with the corresponding module.
-  uint32_t NextFileIndex = 0;
-  for (size_t I = 0; I < ModuleInfos.size(); ++I) {
-    uint32_t NumFiles = ModFileCountArray[I];
-    ModuleInfos[I].SourceFiles.resize(NumFiles);
-    for (size_t J = 0; J < NumFiles; ++J, ++NextFileIndex) {
-      auto ThisName = getFileNameForIndex(NextFileIndex);
-      if (!ThisName)
-        return ThisName.takeError();
-      ModuleInfos[I].SourceFiles[J] = *ThisName;
-    }
-  }
-
-  return Error::success();
-}
-
 uint32_t DbiStream::getDebugStreamIndex(DbgHeaderType Type) const {
   uint16_t T = static_cast<uint16_t>(Type);
   if (T >= DbgStreams.size())
     return kInvalidStreamIndex;
   return DbgStreams[T];
 }
-
-Expected<StringRef> DbiStream::getFileNameForIndex(uint32_t Index) const {
-  BinaryStreamReader Names(NamesBuffer);
-  if (Index >= FileNameOffsets.size())
-    return make_error<RawError>(raw_error_code::index_out_of_bounds);
-
-  uint32_t FileOffset = FileNameOffsets[Index];
-  Names.setOffset(FileOffset);
-  StringRef Name;
-  if (auto EC = Names.readCString(Name))
-    return std::move(EC);
-  return Name;
-}

llvm/trunk/lib/DebugInfo/PDB/Native/NativeCompilandSymbol.cpp

 //===- NativeCompilandSymbol.cpp - Native impl for compilands ---*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/DebugInfo/PDB/Native/NativeCompilandSymbol.h"
 
 namespace llvm {
 namespace pdb {
 
 NativeCompilandSymbol::NativeCompilandSymbol(NativeSession &Session,
-                                             const ModuleInfoEx &MI)
+                                             DbiModuleDescriptor MI)
     : NativeRawSymbol(Session), Module(MI) {}
 
 PDB_SymType NativeCompilandSymbol::getSymTag() const {
   return PDB_SymType::Compiland;
 }
 
 bool NativeCompilandSymbol::isEditAndContinueEnabled() const {
-  return Module.Info.hasECInfo();
+  return Module.hasECInfo();
 }
 
 uint32_t NativeCompilandSymbol::getLexicalParentId() const { return 0; }
 
 // The usage of getObjFileName for getLibraryName and getModuleName for getName
 // may seem backwards, but it is consistent with DIA, which is what this API
 // was modeled after.  We may rename these methods later to try to eliminate
 // this potential confusion.
 
 std::string NativeCompilandSymbol::getLibraryName() const {
-  return Module.Info.getObjFileName();
+  return Module.getObjFileName();
 }
 
 std::string NativeCompilandSymbol::getName() const {
-  return Module.Info.getModuleName();
+  return Module.getModuleName();
 }
 
 } // namespace pdb
 } // namespace llvm

llvm/trunk/lib/DebugInfo/PDB/Native/NativeEnumModules.cpp

 //==- NativeEnumModules.cpp - Native Symbol Enumerator impl ------*- C++ -*-==//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/DebugInfo/PDB/Native/NativeEnumModules.h"
 
 #include "llvm/DebugInfo/PDB/IPDBEnumChildren.h"
+#include "llvm/DebugInfo/PDB/Native/DbiModuleList.h"
 #include "llvm/DebugInfo/PDB/Native/NativeCompilandSymbol.h"
 #include "llvm/DebugInfo/PDB/Native/NativeSession.h"
 #include "llvm/DebugInfo/PDB/PDBSymbol.h"
 #include "llvm/DebugInfo/PDB/PDBSymbolCompiland.h"
 
 namespace llvm {
 namespace pdb {
 
 NativeEnumModules::NativeEnumModules(NativeSession &PDBSession,
-                                     ArrayRef<ModuleInfoEx> Modules,
+                                     const DbiModuleList &Modules,
                                      uint32_t Index)
     : Session(PDBSession), Modules(Modules), Index(Index) {}
 
 uint32_t NativeEnumModules::getChildCount() const {
-  return static_cast<uint32_t>(Modules.size());
+  return static_cast<uint32_t>(Modules.getModuleCount());
 }
 
 std::unique_ptr<PDBSymbol>
 NativeEnumModules::getChildAtIndex(uint32_t Index) const {
-  if (Index >= Modules.size())
+  if (Index >= Modules.getModuleCount())
     return nullptr;
-  return std::unique_ptr<PDBSymbol>(new PDBSymbolCompiland(Session,
-      std::unique_ptr<IPDBRawSymbol>(
-          new NativeCompilandSymbol(Session, Modules[Index]))));
+  return std::unique_ptr<PDBSymbol>(new PDBSymbolCompiland(
+      Session, std::unique_ptr<IPDBRawSymbol>(new NativeCompilandSymbol(
+                   Session, Modules.getModuleDescriptor(Index)))));
 }
 
 std::unique_ptr<PDBSymbol> NativeEnumModules::getNext() {
-  if (Index >= Modules.size())
+  if (Index >= Modules.getModuleCount())
     return nullptr;
   return getChildAtIndex(Index++);
 }
 
 void NativeEnumModules::reset() { Index = 0; }
 
 NativeEnumModules *NativeEnumModules::clone() const {
   return new NativeEnumModules(Session, Modules, Index);
 }
 
 }
 }

llvm/trunk/lib/DebugInfo/PDB/Native/NativeExeSymbol.cpp

NativeExeSymbol::NativeExeSymbol(NativeSession &Session)
     : NativeRawSymbol(Session), File(Session.getPDBFile()) {}
 
 std::unique_ptr<IPDBEnumSymbols>
 NativeExeSymbol::findChildren(PDB_SymType Type) const {
   switch (Type) {
   case PDB_SymType::Compiland: {
     auto Dbi = File.getPDBDbiStream();
     if (Dbi) {
-      const auto Modules = Dbi->modules();
+      const DbiModuleList &Modules = Dbi->modules();
       return std::unique_ptr<IPDBEnumSymbols>(
           new NativeEnumModules(Session, Modules));
     }
     consumeError(Dbi.takeError());
     break;
   }
   default:
     break;

llvm/trunk/tools/llvm-pdbdump/LLVMOutputStyle.cpp

Error LLVMOutputStyle::dumpDbiStream() {
   std::string DllName;
   raw_string_ostream DllStream(DllName);
   DllStream << "mspdb" << Major << Minor << ".dll version";
   DllStream.flush();
   P.printVersion(DllName, Major, Minor, DS->getPdbDllVersion());
 
   if (DumpModules) {
     ListScope L(P, "Modules");
-    for (auto &Modi : DS->modules()) {
+    const DbiModuleList &Modules = DS->modules();
+    for (uint32_t I = 0; I < Modules.getModuleCount(); ++I) {
+      const DbiModuleDescriptor &Modi = Modules.getModuleDescriptor(I);
       DictScope DD(P);
-      P.printString("Name", Modi.Info.getModuleName().str());
-      P.printNumber("Debug Stream Index", Modi.Info.getModuleStreamIndex());
-      P.printString("Object File Name", Modi.Info.getObjFileName().str());
-      P.printNumber("Num Files", Modi.Info.getNumberOfFiles());
-      P.printNumber("Source File Name Idx", Modi.Info.getSourceFileNameIndex());
-      P.printNumber("Pdb File Name Idx", Modi.Info.getPdbFilePathNameIndex());
-      P.printNumber("Line Info Byte Size", Modi.Info.getC11LineInfoByteSize());
-      P.printNumber("C13 Line Info Byte Size",
-                    Modi.Info.getC13LineInfoByteSize());
-      P.printNumber("Symbol Byte Size", Modi.Info.getSymbolDebugInfoByteSize());
-      P.printNumber("Type Server Index", Modi.Info.getTypeServerIndex());
-      P.printBoolean("Has EC Info", Modi.Info.hasECInfo());
+      P.printString("Name", Modi.getModuleName().str());
+      P.printNumber("Debug Stream Index", Modi.getModuleStreamIndex());
+      P.printString("Object File Name", Modi.getObjFileName().str());
+      P.printNumber("Num Files", Modi.getNumberOfFiles());
+      P.printNumber("Source File Name Idx", Modi.getSourceFileNameIndex());
+      P.printNumber("Pdb File Name Idx", Modi.getPdbFilePathNameIndex());
+      P.printNumber("Line Info Byte Size", Modi.getC11LineInfoByteSize());
+      P.printNumber("C13 Line Info Byte Size", Modi.getC13LineInfoByteSize());
+      P.printNumber("Symbol Byte Size", Modi.getSymbolDebugInfoByteSize());
+      P.printNumber("Type Server Index", Modi.getTypeServerIndex());
+      P.printBoolean("Has EC Info", Modi.hasECInfo());
       if (opts::raw::DumpModuleFiles) {
-        std::string FileListName =
-            to_string(Modi.SourceFiles.size()) + " Contributing Source Files";
+        std::string FileListName = to_string(Modules.getSourceFileCount(I)) +
+                                   " Contributing Source Files";
         ListScope LL(P, FileListName);
-        for (auto File : Modi.SourceFiles)
-          P.printString(File.str());
+        for (auto File : Modules.source_files(I))
+          P.printString(File);
       }
-      bool HasModuleDI =
-          (Modi.Info.getModuleStreamIndex() < File.getNumStreams());
+      bool HasModuleDI = (Modi.getModuleStreamIndex() < File.getNumStreams());
       bool ShouldDumpSymbols =
           (opts::raw::DumpModuleSyms || opts::raw::DumpSymRecordBytes);
       if (HasModuleDI && (ShouldDumpSymbols || opts::raw::DumpLineInfo)) {
         auto ModStreamData = MappedBlockStream::createIndexedStream(
             File.getMsfLayout(), File.getMsfBuffer(),
-            Modi.Info.getModuleStreamIndex());
+            Modi.getModuleStreamIndex());
 
-        ModuleDebugStreamRef ModS(Modi.Info, std::move(ModStreamData));
+        ModuleDebugStreamRef ModS(Modi, std::move(ModStreamData));
         if (auto EC = ModS.reload())
           return EC;
 
         if (ShouldDumpSymbols) {
           if (auto EC = buildTypeDatabase(StreamTPI))
             return EC;
 
           ListScope SS(P, "Symbols");
void visit(const SectionContrib &SC) override {
       P.printNumber("Off", SC.Off);
       P.printNumber("Size", SC.Size);
       P.printFlags("Characteristics", SC.Characteristics,
                    codeview::getImageSectionCharacteristicNames(),
                    COFF::SectionCharacteristics(0x00F00000));
       {
         DictScope DD(P, "Module");
         P.printNumber("Index", SC.Imod);
-        auto M = DS.modules();
-        if (M.size() > SC.Imod) {
-          P.printString("Name", M[SC.Imod].Info.getModuleName());
+        const DbiModuleList &Modules = DS.modules();
+        if (Modules.getModuleCount() > SC.Imod) {
+          P.printString("Name",
+                        Modules.getModuleDescriptor(SC.Imod).getModuleName());
         }
       }
       P.printNumber("Data CRC", SC.DataCrc);
       P.printNumber("Reloc CRC", SC.RelocCrc);
       P.flush();
     }
     void visit(const SectionContrib2 &SC) override {
       visit(SC.Base);

llvm/trunk/tools/llvm-pdbdump/StreamUtil.cpp

 //===- StreamUtil.cpp - PDB stream utilities --------------------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #include "StreamUtil.h"
 
 #include "llvm/ADT/DenseMap.h"
 #include "llvm/ADT/DenseMapInfo.h"
 #include "llvm/DebugInfo/PDB/Native/DbiModuleDescriptor.h"
+#include "llvm/DebugInfo/PDB/Native/DbiModuleList.h"
 #include "llvm/DebugInfo/PDB/Native/DbiStream.h"
 #include "llvm/DebugInfo/PDB/Native/InfoStream.h"
 #include "llvm/DebugInfo/PDB/Native/PDBFile.h"
 #include "llvm/DebugInfo/PDB/Native/TpiStream.h"
 
 namespace llvm {
 namespace pdb {
 void discoverStreamPurposes(PDBFile &File,
                             SmallVectorImpl<std::string> &Purposes) {
 
   // It's OK if we fail to load some of these streams, we still attempt to print
   // what we can.
   auto Dbi = File.getPDBDbiStream();
   auto Tpi = File.getPDBTpiStream();
   auto Ipi = File.getPDBIpiStream();
   auto Info = File.getPDBInfoStream();
 
   uint32_t StreamCount = File.getNumStreams();
-  DenseMap<uint16_t, const ModuleInfoEx *> ModStreams;
+  DenseMap<uint16_t, DbiModuleDescriptor> ModStreams;
   DenseMap<uint16_t, std::string> NamedStreams;
 
   if (Dbi) {
-    for (auto &ModI : Dbi->modules()) {
-      uint16_t SN = ModI.Info.getModuleStreamIndex();
+    const DbiModuleList &Modules = Dbi->modules();
+    for (uint32_t I = 0; I < Modules.getModuleCount(); ++I) {
+      DbiModuleDescriptor Descriptor = Modules.getModuleDescriptor(I);
+      uint16_t SN = Descriptor.getModuleStreamIndex();
       if (SN != kInvalidStreamIndex)
-        ModStreams[SN] = &ModI;
+        ModStreams[SN] = Descriptor;
     }
   }
   if (Info) {
     for (auto &NSE : Info->named_streams()) {
       if (NSE.second != kInvalidStreamIndex)
         NamedStreams[NSE.second] = NSE.first();
     }
   }
else if (Dbi &&
       Value = "Token Rid Data";
     else if (Dbi && StreamIdx == Dbi->getDebugStreamIndex(DbgHeaderType::Xdata))
       Value = "Xdata";
     else {
       auto ModIter = ModStreams.find(StreamIdx);
       auto NSIter = NamedStreams.find(StreamIdx);
       if (ModIter != ModStreams.end()) {
         Value = "Module \"";
-        Value += ModIter->second->Info.getModuleName().str();
+        Value += ModIter->second.getModuleName();
         Value += "\"";
       } else if (NSIter != NamedStreams.end()) {
         Value = "Named Stream \"";
         Value += NSIter->second;
         Value += "\"";
       } else {
         Value = "???";
       }

llvm/trunk/tools/llvm-pdbdump/YAMLOutputStyle.cpp

Error YAMLOutputStyle::dumpDbiStream() {
   Obj.DbiStream->Age = DS.getAge();
   Obj.DbiStream->BuildNumber = DS.getBuildNumber();
   Obj.DbiStream->Flags = DS.getFlags();
   Obj.DbiStream->MachineType = DS.getMachineType();
   Obj.DbiStream->PdbDllRbld = DS.getPdbDllRbld();
   Obj.DbiStream->PdbDllVersion = DS.getPdbDllVersion();
   Obj.DbiStream->VerHeader = DS.getDbiVersion();
   if (opts::pdb2yaml::DbiModuleInfo) {
-    for (const auto &MI : DS.modules()) {
+    const auto &Modules = DS.modules();
+    for (uint32_t I = 0; I < Modules.getModuleCount(); ++I) {
+      DbiModuleDescriptor MI = Modules.getModuleDescriptor(I);
+
       Obj.DbiStream->ModInfos.emplace_back();
       yaml::PdbDbiModuleInfo &DMI = Obj.DbiStream->ModInfos.back();
 
-      DMI.Mod = MI.Info.getModuleName();
-      DMI.Obj = MI.Info.getObjFileName();
-      if (opts::pdb2yaml::DbiModuleSourceFileInfo)
-        DMI.SourceFiles = MI.SourceFiles;
+      DMI.Mod = MI.getModuleName();
+      DMI.Obj = MI.getObjFileName();
+      if (opts::pdb2yaml::DbiModuleSourceFileInfo) {
+        auto Files = Modules.source_files(I);
+        DMI.SourceFiles.assign(Files.begin(), Files.end());
+      }
 
-      uint16_t ModiStream = MI.Info.getModuleStreamIndex();
+      uint16_t ModiStream = MI.getModuleStreamIndex();
       if (ModiStream == kInvalidStreamIndex)
         continue;
 
       auto ModStreamData = msf::MappedBlockStream::createIndexedStream(
           File.getMsfLayout(), File.getMsfBuffer(), ModiStream);
 
-      pdb::ModuleDebugStreamRef ModS(MI.Info, std::move(ModStreamData));
+      pdb::ModuleDebugStreamRef ModS(MI, std::move(ModStreamData));
       if (auto EC = ModS.reload())
         return EC;
 
       if (opts::pdb2yaml::DbiModuleSourceLineInfo) {
         auto ExpectedInfo = getFileLineInfo(ModS);
         if (!ExpectedInfo)
           return ExpectedInfo.takeError();
         DMI.FileLineInfo = *ExpectedInfo;