[pdb] Apply zero-copy algorithms to symbol stream iteration

[zturner]

Summary

There is one PDB stream for each module (compiland). This stream contains (among other things) all of the symbols for each compiland. Generally speaking this is where a large portion of memory associated with the PDB lies.

This patch slightly improves the VarStreamArray class by templatizing it on a value type and changing the length conversion function to a value / length extraction function. In doing so, we can now apply the VarStreamArray to symbol streams and iterate them exactly the same as before, but without copying the entire symbol stream into a contiguous buffer.

Measurements:
PDB File Size: 6,598,656 bytes
Combined size of all symbol streams: 1,985,508 bytes

So this is a savings of 30% of the PDB size. We should be able to claim another 30% or so by applying this to the type info streams.

[zturner]

Also worth pointing out that eventually, once this works with types, we can delete RecordIterator.h. That was the "generic" method that was supposed to be able to be used to treat type and symbol streams the same, but VarStreamArray<T> works with type streams, symbol streams, and any other arbitrary type of variable length record array such as the module info array.

[ruiu]

include/llvm/DebugInfo/CodeView/RecordIterator.h

29

Do you need a separate traits class? Looks like you can merge it into the non-traits class if you change the definition of VarStreamArray.

[zturner]

Not sure I understand, can you elaborate?

[ruiu]

I mean it seems you can define static uint32_t extract(const StreamInterface &Stream, CVRecord &Item) for CVRecord class and template VarStreamArray with CVRecord (instead of the traits class).

So you are currently doing this.

VarStreamArrayIterator(const VarStreamArray<TraitsType> &Array)
    : Array(&Array), IterRef(Array.Stream) {
  ThisLen = TraitsType::extract(IterRef, ThisValue);
}

But looks like it can be written like this if you remove the template class and directly use CVRecord class.

VarStreamArrayIterator(const VarStreamArray<T> &Array)
    : Array(&Array), IterRef(Array.Stream) {
  ThisLen = T::extract(IterRef, ThisValue);
}

no?

[zturner]

Ahh, I didn't do this because I imagined cases where you might want
ValueType to be something you can't change like ArrayRef, or have 2
different VarStreamArrays both with the same ValueType but using a
different algorithm to extract them. Currently we don't but I thought we
might need it later

[ruiu]

OK, I wouldn't make a trait class at this moment as it has already lots of meta programming, but it's your call.

include/llvm/DebugInfo/CodeView/RecordIterator.h

41–44

Don't you want to propagate the error up to the caller? Hiding errors in a deep function call seems risky.

[zturner]

I could pass 2 template arguments instead, ValueType and extraction functor
separately. Similar to std::map how you can pass a comparison functor

[ruiu]

Ah, that would be much better, I think.

[zturner]

Don't use a traits class, instead specify ValueType and extraction functor as separate template arguments.

Following the pattern of STL data structures like std::map, I've specified a default value for the extraction functor, so if you define a specialization of this template, you don't need to pass the second argument.

[ruiu]

LGTM

[Eugene.Zelenko]

Committed in r271025.

include/llvm/DebugInfo/CodeView/CVSymbolVisitor.h

 /// expected to consume the trailing bytes used by the field.
 /// FIXME: Make the visitor interpret the trailing bytes so that clients don't
 /// need to.
 #define SYMBOL_RECORD(EnumName, EnumVal, Name)                                 \
   void visit##Name(SymbolRecordKind Kind, Name &Record) {}
 #define SYMBOL_RECORD_ALIAS(EnumName, EnumVal, Name, AliasName)
 #include "CVSymbolTypes.def"
 
-  void visitSymbolRecord(const SymbolIterator::Record &Record) {
+  void visitSymbolRecord(const CVRecord<SymbolKind> &Record) {
     ArrayRef<uint8_t> Data = Record.Data;
     auto *DerivedThis = static_cast<Derived *>(this);
     DerivedThis->visitSymbolBegin(Record.Type, Data);
     uint32_t RecordOffset = Delegate ? Delegate->getRecordOffset(Data) : 0;
     switch (Record.Type) {
     default:
       DerivedThis->visitUnknownSymbol(Record.Type, Data);
       break;

include/llvm/DebugInfo/CodeView/CVTypeVisitor.h

 #define TYPE_RECORD(EnumName, EnumVal, Name)                                   \
   void visit##Name(TypeLeafKind LeafType, Name##Record &Record) {}
 #define TYPE_RECORD_ALIAS(EnumName, EnumVal, Name, AliasName)
 #define MEMBER_RECORD(EnumName, EnumVal, Name)                                 \
   void visit##Name(TypeLeafKind LeafType, Name##Record &Record) {}
 #define MEMBER_RECORD_ALIAS(EnumName, EnumVal, Name, AliasName)
 #include "TypeRecords.def"
 
-  void visitTypeRecord(const TypeIterator::Record &Record) {
+  void visitTypeRecord(const CVRecord<TypeLeafKind> &Record) {
     ArrayRef<uint8_t> LeafData = Record.Data;
     ArrayRef<uint8_t> RecordData = LeafData;
     auto *DerivedThis = static_cast<Derived *>(this);
     DerivedThis->visitTypeBegin(Record.Type, RecordData);
     switch (Record.Type) {
     default:
       DerivedThis->visitUnknownType(Record.Type, RecordData);
       break;

include/llvm/DebugInfo/CodeView/RecordIterator.h

 //===- RecordIterator.h -----------------------------------------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_DEBUGINFO_CODEVIEW_RECORDITERATOR_H
 #define LLVM_DEBUGINFO_CODEVIEW_RECORDITERATOR_H
 
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/iterator_range.h"
 #include "llvm/DebugInfo/CodeView/RecordSerialization.h"
+#include "llvm/DebugInfo/CodeView/StreamInterface.h"
+#include "llvm/DebugInfo/CodeView/StreamReader.h"
 #include "llvm/Support/Endian.h"
 
 namespace llvm {
 namespace codeview {
 
-// A const input iterator interface to the CodeView record stream.
-template <typename Kind> class RecordIterator {
-public:
-  struct Record {
-    std::size_t Length;
-    Kind Type;
-    ArrayRef<uint8_t> Data;
-  };
+template <typename Kind> struct CVRecord {
+  uint32_t Length;
 
+template <typename Kind> struct VarStreamArrayExtractor<CVRecord<Kind>> {
+  uint32_t operator()(const StreamInterface &Stream,
+                      CVRecord<Kind> &Item) const {
+    const RecordPrefix *Prefix = nullptr;
+    StreamReader Reader(Stream);
+    if (auto EC = Reader.readObject(Prefix)) {
+      consumeError(std::move(EC));
+      return 0;
+    }
+    Item.Length = Prefix->RecordLen;
+    Item.Type = static_cast<Kind>(uint16_t(Prefix->RecordKind));
+    if (auto EC = Reader.readBytes(Item.Length - 2, Item.Data)) {
+      consumeError(std::move(EC));
+      return 0;
+    }
+    return Prefix->RecordLen + 2;
+  }
+};
+
+// A const input iterator interface to the CodeView record stream.
+template <typename Kind> class RecordIterator {
+public:
+
   explicit RecordIterator(const ArrayRef<uint8_t> &RecordBytes, bool *HadError)
       : HadError(HadError), Data(RecordBytes), AtEnd(false) {
     next(); // Prime the pump
   }
 
   RecordIterator() : HadError(nullptr), AtEnd(true) {}
 
   // For iterators to compare equal, they must both point at the same record
   // in the same data stream, or they must both be at the end of a stream.
   friend bool operator==(const RecordIterator<Kind> &lhs,
                          const RecordIterator<Kind> &rhs) {
     return (lhs.Data.begin() == rhs.Data.begin()) || (lhs.AtEnd && rhs.AtEnd);
   }
 
   friend bool operator!=(const RecordIterator<Kind> &lhs,
                          const RecordIterator<Kind> &rhs) {
     return !(lhs == rhs);
   }
 
-  const Record &operator*() const {
+  const CVRecord<Kind> &operator*() const {
     assert(!AtEnd);
     return Current;
   }
 
-  const Record *operator->() const {
+  const CVRecord<Kind> *operator->() const {
     assert(!AtEnd);
     return &Current;
   }
 
   RecordIterator<Kind> &operator++() {
     next();
     return *this;
   }
private:
 
   void parseError() {
     if (HadError)
       *HadError = true;
   }
 
   bool *HadError;
   ArrayRef<uint8_t> Data;
-  Record Current;
+  CVRecord<Kind> Current;
   bool AtEnd;
 };
 
 template <typename Kind>
 inline iterator_range<RecordIterator<Kind>>
 makeRecordRange(ArrayRef<uint8_t> Data, bool *HadError) {
   return make_range(RecordIterator<Kind>(Data, HadError), RecordIterator<Kind>());
 }
 }
 }
 
 #endif

include/llvm/DebugInfo/CodeView/StreamArray.h

 #include "llvm/DebugInfo/CodeView/StreamRef.h"
 
 #include <functional>
 #include <type_traits>
 
 namespace llvm {
 namespace codeview {
 
+template <typename T> struct VarStreamArrayExtractor {};
+
 /// VarStreamArray represents an array of variable length records backed by a
 /// stream.  This could be a contiguous sequence of bytes in memory, it could
 /// be a file on disk, or it could be a PDB stream where bytes are stored as
 /// discontiguous blocks in a file.  Usually it is desirable to treat arrays
 /// as contiguous blocks of memory, but doing so with large PDB files, for
 /// example, could mean allocating huge amounts of memory just to allow
 /// re-ordering of stream data to be contiguous before iterating over it.  By
 /// abstracting this out, we need not duplicate this memory, and we can
 /// iterate over arrays in arbitrarily formatted streams.
-class VarStreamArrayIterator;
+template <typename ValueType, typename Extractor> class VarStreamArrayIterator;
 
+template <typename ValueType,
+          typename Extractor = VarStreamArrayExtractor<ValueType>>
 class VarStreamArray {
-  friend class VarStreamArrayIterator;
-  typedef std::function<uint32_t(const StreamInterface &)> LengthFuncType;
+  friend class VarStreamArrayIterator<ValueType, Extractor>;
 
 public:
-  template <typename LengthFunc>
-  VarStreamArray(StreamRef Stream, const LengthFunc &Len)
-      : Stream(Stream), Len(Len) {}
+  typedef VarStreamArrayIterator<ValueType, Extractor> Iterator;
+
+  VarStreamArray() {}
+
+  VarStreamArray(StreamRef Stream) : Stream(Stream) {}
 
-  VarStreamArrayIterator begin() const;
-  VarStreamArrayIterator end() const;
+  Iterator begin() const { return Iterator(*this); }
+
+  Iterator end() const { return Iterator(); }
 
 private:
   StreamRef Stream;
-  LengthFuncType Len; // Function used to calculate legth of a record
 };
 
-class VarStreamArrayIterator {
+template <typename ValueType, typename Extractor> class VarStreamArrayIterator {
+  typedef VarStreamArrayIterator<ValueType, Extractor> IterType;
+  typedef VarStreamArray<ValueType, Extractor> ArrayType;
+
 public:
-  VarStreamArrayIterator(const VarStreamArray &Array)
+  VarStreamArrayIterator(const ArrayType &Array)
       : Array(&Array), IterRef(Array.Stream) {
-    ThisLen = Array.Len(IterRef);
+    ThisLen = Extract(IterRef, ThisValue);
   }
   VarStreamArrayIterator() : Array(nullptr), IterRef() {}
-  bool operator==(const VarStreamArrayIterator &R) const {
+  bool operator==(const IterType &R) const {
     if (Array && R.Array) {
       // Both have a valid array, make sure they're same.
       assert(Array == R.Array);
       return IterRef == R.IterRef;
     }
 
     // Both iterators are at the end.
     if (!Array && !R.Array)
       return true;
 
     // One is not at the end and one is.
     return false;
   }
 
-  bool operator!=(const VarStreamArrayIterator &R) { return !(*this == R); }
+  bool operator!=(const IterType &R) { return !(*this == R); }
 
-  StreamRef operator*() const {
-    ArrayRef<uint8_t> Result;
-    return IterRef.keep_front(ThisLen);
-  }
+  const ValueType &operator*() const { return ThisValue; }
 
-  VarStreamArrayIterator &operator++() {
-    if (!Array || IterRef.getLength() == 0)
+  IterType &operator++() {
+    if (!Array || IterRef.getLength() == 0 || ThisLen == 0)
       return *this;
     IterRef = IterRef.drop_front(ThisLen);
     if (IterRef.getLength() == 0) {
       Array = nullptr;
       ThisLen = 0;
     } else {
-      ThisLen = Array->Len(IterRef);
+      ThisLen = Extract(IterRef, ThisValue);
     }
     return *this;
   }
 
-  VarStreamArrayIterator operator++(int) {
-    VarStreamArrayIterator Original = *this;
+  IterType operator++(int) {
+    IterType Original = *this;
     ++*this;
     return Original;
   }
 
 private:
-  const VarStreamArray *Array;
+  const ArrayType *Array;
   uint32_t ThisLen;
+  ValueType ThisValue;
   StreamRef IterRef;
+  Extractor Extract;
 };
 
-inline VarStreamArrayIterator VarStreamArray::begin() const {
-  return VarStreamArrayIterator(*this);
-}
-inline VarStreamArrayIterator VarStreamArray::end() const {
-  return VarStreamArrayIterator();
-}
-
 template <typename T> class FixedStreamArrayIterator;
 
 template <typename T> class FixedStreamArray {
   friend class FixedStreamArrayIterator<T>;
 
 public:
   FixedStreamArray() : Stream() {}
   FixedStreamArray(StreamRef Stream) : Stream(Stream) {

include/llvm/DebugInfo/CodeView/StreamReader.h

template <typename T> Error readObject(const T *&Dest) {
     ArrayRef<uint8_t> Buffer;
     if (auto EC = readBytes(sizeof(T), Buffer))
       return EC;
     Dest = reinterpret_cast<const T *>(Buffer.data());
     return Error::success();
   }
 
   template <typename T>
+  Error readArray(VarStreamArray<T> &Array, uint32_t Size) {
+    StreamRef S;
+    if (auto EC = readStreamRef(S, Size))
+      return EC;
+    Array = VarStreamArray<T>(S);
+    return Error::success();
+  }
+
+  template <typename T>
   Error readArray(FixedStreamArray<T> &Array, uint32_t NumItems) {
     if (NumItems == 0) {
       Array = FixedStreamArray<T>();
       return Error::success();
     }
     uint32_t Length = NumItems * sizeof(T);
     if (Offset + Length > Stream.getLength())
       return make_error<CodeViewError>(cv_error_code::insufficient_buffer);

include/llvm/DebugInfo/CodeView/SymbolDumper.h

CVSymbolDumper(ScopedPrinter &W, CVTypeDumper &CVTD,
                  bool PrintRecordBytes)
       : W(W), CVTD(CVTD), ObjDelegate(std::move(ObjDelegate)),
         PrintRecordBytes(PrintRecordBytes) {}
 
   /// Dumps one type record.  Returns false if there was a type parsing error,
   /// and true otherwise.  This should be called in order, since the dumper
   /// maintains state about previous records which are necessary for cross
   /// type references.
-  bool dump(const SymbolIterator::Record &Record);
+  bool dump(const CVRecord<SymbolKind> &Record);
 
   /// Dumps the type records in Data. Returns false if there was a type stream
   /// parse error, and true otherwise.
   bool dump(ArrayRef<uint8_t> Data);
 
 private:
   ScopedPrinter &W;
   CVTypeDumper &CVTD;
   std::unique_ptr<SymbolDumpDelegate> ObjDelegate;
 
   bool PrintRecordBytes;
 };
 } // end namespace codeview
 } // end namespace llvm
 
 #endif // LLVM_DEBUGINFO_CODEVIEW_SYMBOLDUMPER_H

include/llvm/DebugInfo/CodeView/SymbolRecord.h

 #ifndef LLVM_DEBUGINFO_CODEVIEW_SYMBOLRECORD_H
 #define LLVM_DEBUGINFO_CODEVIEW_SYMBOLRECORD_H
 
 #include "llvm/ADT/APSInt.h"
 #include "llvm/ADT/Optional.h"
 #include "llvm/DebugInfo/CodeView/CodeView.h"
 #include "llvm/DebugInfo/CodeView/RecordIterator.h"
 #include "llvm/DebugInfo/CodeView/RecordSerialization.h"
+#include "llvm/DebugInfo/CodeView/StreamArray.h"
+#include "llvm/DebugInfo/CodeView/StreamInterface.h"
 #include "llvm/DebugInfo/CodeView/TypeIndex.h"
 #include "llvm/Support/Endian.h"
 #include "llvm/Support/Error.h"
 
 namespace llvm {
 namespace codeview {
 
 using llvm::support::ulittle16_t;
public:
   }
 
   uint32_t RecordOffset;
   Hdr Header;
   StringRef Name;
 };
 
 typedef RecordIterator<SymbolKind> SymbolIterator;
+typedef CVRecord<SymbolKind> CVSymbol;
+typedef VarStreamArray<CVSymbol> CVSymbolArray;
 
 inline iterator_range<SymbolIterator> makeSymbolRange(ArrayRef<uint8_t> Data,
                                                       bool *HadError) {
   return make_range(SymbolIterator(Data, HadError), SymbolIterator());
 }
 
 } // namespace codeview
 } // namespace llvm
 
 #endif

include/llvm/DebugInfo/CodeView/TypeDumper.h

public:
 
   StringRef getTypeName(TypeIndex TI);
   void printTypeIndex(StringRef FieldName, TypeIndex TI);
 
   /// Dumps one type record.  Returns false if there was a type parsing error,
   /// and true otherwise.  This should be called in order, since the dumper
   /// maintains state about previous records which are necessary for cross
   /// type references.
-  bool dump(const TypeIterator::Record &Record);
+  bool dump(const CVRecord<TypeLeafKind> &Record);
 
   /// Dumps the type records in Data. Returns false if there was a type stream
   /// parse error, and true otherwise.
   bool dump(ArrayRef<uint8_t> Data);
 
   /// Gets the type index for the next type record.
   unsigned getNextTypeIndex() const {
     return 0x1000 + CVUDTNames.size();

include/llvm/DebugInfo/PDB/Raw/DbiStream.h

 //===- DbiStream.h - PDB Dbi Stream (Stream 3) Access -----------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_DEBUGINFO_PDB_RAW_PDBDBISTREAM_H
 #define LLVM_DEBUGINFO_PDB_RAW_PDBDBISTREAM_H
 
-#include "llvm/DebugInfo/CodeView/ByteStream.h"
 #include "llvm/DebugInfo/CodeView/StreamArray.h"
 #include "llvm/DebugInfo/CodeView/StreamRef.h"
 #include "llvm/DebugInfo/PDB/PDBTypes.h"
 #include "llvm/DebugInfo/PDB/Raw/MappedBlockStream.h"
 #include "llvm/DebugInfo/PDB/Raw/ModInfo.h"
 #include "llvm/DebugInfo/PDB/Raw/NameHashTable.h"
 #include "llvm/DebugInfo/PDB/Raw/RawConstants.h"
 #include "llvm/Support/Endian.h"

include/llvm/DebugInfo/PDB/Raw/ModInfo.h

 //===- ModInfo.h - PDB module information -----------------------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_DEBUGINFO_PDB_RAW_MODINFO_H
 #define LLVM_DEBUGINFO_PDB_RAW_MODINFO_H
 
 #include "llvm/ADT/StringRef.h"
+#include "llvm/DebugInfo/CodeView/StreamArray.h"
 #include "llvm/DebugInfo/CodeView/StreamRef.h"
 #include <cstdint>
 #include <vector>
 
 namespace llvm {
 namespace pdb {
 
 class ModInfo {
 private:
   struct FileLayout;
 
 public:
+  ModInfo();
   ModInfo(codeview::StreamRef Stream);
   ModInfo(const ModInfo &Info);
   ~ModInfo();
 
   bool hasECInfo() const;
   uint16_t getTypeServerIndex() const;
   uint16_t getModuleStreamIndex() const;
   uint32_t getSymbolDebugInfoByteSize() const;

 private:
   StringRef ModuleName;
   StringRef ObjFileName;
   const FileLayout *Layout;
 };
 
 struct ModuleInfoEx {
   ModuleInfoEx(codeview::StreamRef Stream) : Info(Stream) {}
+  ModuleInfoEx(const ModInfo &Info) : Info(Info) {}
   ModuleInfoEx(const ModuleInfoEx &Ex)
       : Info(Ex.Info), SourceFiles(Ex.SourceFiles) {}
 
   ModInfo Info;
   std::vector<StringRef> SourceFiles;
 };
 
-inline uint32_t ModInfoRecordLength(const codeview::StreamInterface &Stream) {
-  return ModInfo(Stream).getRecordLength();
+} // end namespace pdb
+
+namespace codeview {
+template <> struct VarStreamArrayExtractor<pdb::ModInfo> {
+  uint32_t operator()(const StreamInterface &Stream, pdb::ModInfo &Info) const {
+    Info = pdb::ModInfo(Stream);
+    return Info.getRecordLength();
+  }
+};
 }
 
-} // end namespace pdb
 } // end namespace llvm
 
 #endif // LLVM_DEBUGINFO_PDB_RAW_MODINFO_H

include/llvm/DebugInfo/PDB/Raw/ModStream.h

 //===- ModStream.h - PDB Module Info Stream Access ------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_DEBUGINFO_PDB_RAW_MODSTREAM_H
 #define LLVM_DEBUGINFO_PDB_RAW_MODSTREAM_H
 
 #include "llvm/ADT/iterator_range.h"
-#include "llvm/DebugInfo/CodeView/ByteStream.h"
+#include "llvm/DebugInfo/CodeView/RecordIterator.h"
 #include "llvm/DebugInfo/CodeView/StreamRef.h"
 #include "llvm/DebugInfo/CodeView/SymbolRecord.h"
 #include "llvm/DebugInfo/PDB/Raw/MappedBlockStream.h"
 #include "llvm/Support/Error.h"
 
 namespace llvm {
 namespace pdb {
 class PDBFile;
 class ModInfo;
 
 class ModStream {
 public:
   ModStream(PDBFile &File, const ModInfo &Module);
   ~ModStream();
 
   Error reload();
 
-  iterator_range<codeview::SymbolIterator> symbols() const;
+  iterator_range<codeview::CVSymbolArray::Iterator> symbols() const;
 
 private:
   const ModInfo &Mod;
 
   MappedBlockStream Stream;
 
-  codeview::ByteStream SymbolsSubstream;
+  codeview::CVSymbolArray SymbolsSubstream;
   codeview::StreamRef LinesSubstream;
   codeview::StreamRef C13LinesSubstream;
   codeview::StreamRef GlobalRefsSubstream;
 };
 }
 }
 
 #endif

lib/DebugInfo/CodeView/SymbolDumper.cpp

 
 void CVSymbolDumperImpl::visitUnknownSymbol(SymbolKind Kind,
                                             ArrayRef<uint8_t> Data) {
   DictScope S(W, "UnknownSym");
   W.printEnum("Kind", uint16_t(Kind), makeArrayRef(SymbolTypeNames));
   W.printNumber("Length", uint32_t(Data.size()));
 }
 
-bool CVSymbolDumper::dump(const SymbolIterator::Record &Record) {
+bool CVSymbolDumper::dump(const CVRecord<SymbolKind> &Record) {
   CVSymbolDumperImpl Dumper(CVTD, ObjDelegate.get(), W, PrintRecordBytes);
   Dumper.visitSymbolRecord(Record);
   return !Dumper.hadError();
 }
 
 bool CVSymbolDumper::dump(ArrayRef<uint8_t> Data) {
   CVSymbolDumperImpl Dumper(CVTD, ObjDelegate.get(), W, PrintRecordBytes);
   Dumper.visitSymbolStream(Data);
   return !Dumper.hadError();
 }

lib/DebugInfo/CodeView/TypeDumper.cpp

void CVTypeDumper::printTypeIndex(StringRef FieldName, TypeIndex TI) {
   if (!TI.isNoType())
     TypeName = getTypeName(TI);
   if (!TypeName.empty())
     W->printHex(FieldName, TypeName, TI.getIndex());
   else
     W->printHex(FieldName, TI.getIndex());
 }
 
-bool CVTypeDumper::dump(const TypeIterator::Record &Record) {
+bool CVTypeDumper::dump(const CVRecord<TypeLeafKind> &Record) {
   assert(W && "printer should not be null");
   CVTypeDumperImpl Dumper(*this, *W, PrintRecordBytes);
   Dumper.visitTypeRecord(Record);
   return !Dumper.hadError();
 }
 
 bool CVTypeDumper::dump(ArrayRef<uint8_t> Data) {
   assert(W && "printer should not be null");

lib/DebugInfo/PDB/Raw/DbiStream.cpp

return make_error<RawError>(raw_error_code::corrupt_file,
                                 "DBI section map substream not aligned.");
   if (Header->FileInfoSize % sizeof(uint32_t) != 0)
     return make_error<RawError>(raw_error_code::corrupt_file,
                                 "DBI file info substream not aligned.");
   if (Header->TypeServerSize % sizeof(uint32_t) != 0)
     return make_error<RawError>(raw_error_code::corrupt_file,
                                 "DBI type server substream not aligned.");
 
-  if (auto EC =
-          Reader.readStreamRef(ModInfoSubstream, Header->ModiSubstreamSize))
-    return EC;
-
   // Since each ModInfo in the stream is a variable length, we have to iterate
   // them to know how many there actually are.
-  codeview::VarStreamArray ModInfoArray(ModInfoSubstream, ModInfoRecordLength);
-  for (auto Info : ModInfoArray) {
+  codeview::VarStreamArray<ModInfo> ModInfoArray;
+  if (auto EC = Reader.readArray(ModInfoArray, Header->ModiSubstreamSize))
+    return EC;
+  for (auto &Info : ModInfoArray) {
     ModuleInfos.emplace_back(Info);
   }
 
   if (auto EC = Reader.readStreamRef(SecContrSubstream,
                                      Header->SecContrSubstreamSize))
     return EC;
   if (auto EC = Reader.readStreamRef(SecMapSubstream, Header->SectionMapSize))
     return EC;

lib/DebugInfo/PDB/Raw/ModInfo.cpp

ulittle32_t FileNameOffs;  // array of [0..NumFiles) DBI name buffer offsets.
                              // implementation, but as with `Mod`, we ignore it
                              // for now since it is unused.
   ulittle32_t SrcFileNameNI; // Name Index for src file name
   ulittle32_t PdbFilePathNI; // Name Index for path to compiler PDB
                              // Null terminated Module name
                              // Null terminated Obj File Name
 };
 
+ModInfo::ModInfo() : Layout(nullptr) {}
+
 ModInfo::ModInfo(codeview::StreamRef Stream) : Layout(nullptr) {
   codeview::StreamReader Reader(Stream);
   if (auto EC = Reader.readObject(Layout)) {
     consumeError(std::move(EC));
     return;
   }
   if (auto EC = Reader.readZeroString(ModuleName)) {
     consumeError(std::move(EC));

lib/DebugInfo/PDB/Raw/ModStream.cpp

 //===- ModStream.cpp - PDB Module Info Stream Access ----------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/DebugInfo/PDB/Raw/ModStream.h"
 
+#include "llvm/DebugInfo/CodeView/RecordIterator.h"
 #include "llvm/DebugInfo/CodeView/StreamReader.h"
 #include "llvm/DebugInfo/PDB/Raw/ModInfo.h"
 #include "llvm/DebugInfo/PDB/Raw/RawError.h"
 
 using namespace llvm;
 using namespace llvm::pdb;
 
 ModStream::ModStream(PDBFile &File, const ModInfo &Module)
     : Mod(Module), Stream(Module.getModuleStreamIndex(), File) {}
 
 ModStream::~ModStream() {}
 
 Error ModStream::reload() {
   codeview::StreamReader Reader(Stream);
 
   uint32_t SymbolSize = Mod.getSymbolDebugInfoByteSize();
   uint32_t C11Size = Mod.getLineInfoByteSize();
   uint32_t C13Size = Mod.getC13LineInfoByteSize();
 
   if (C11Size > 0 && C13Size > 0)
     return llvm::make_error<RawError>(raw_error_code::corrupt_file,
                                       "Module has both C11 and C13 line info");
 
-  if (auto EC = SymbolsSubstream.load(Reader, SymbolSize))
+  codeview::StreamRef S;
+
+  uint32_t SymbolSubstreamSig = 0;
+  if (auto EC = Reader.readInteger(SymbolSubstreamSig))
+    return EC;
+  if (auto EC = Reader.readArray(SymbolsSubstream, SymbolSize - 4))
     return EC;
+
   if (auto EC = Reader.readStreamRef(LinesSubstream, C11Size))
     return EC;
   if (auto EC = Reader.readStreamRef(C13LinesSubstream, C13Size))
     return EC;
 
   uint32_t GlobalRefsSize;
   if (auto EC = Reader.readInteger(GlobalRefsSize))
     return EC;
   if (auto EC = Reader.readStreamRef(GlobalRefsSubstream, GlobalRefsSize))
     return EC;
   if (Reader.bytesRemaining() > 0)
     return llvm::make_error<RawError>(raw_error_code::corrupt_file,
                                       "Unexpected bytes in module stream.");
 
   return Error::success();
 }
 
-iterator_range<codeview::SymbolIterator> ModStream::symbols() const {
-  return codeview::makeSymbolRange(SymbolsSubstream.data().slice(4), nullptr);
+iterator_range<codeview::CVSymbolArray::Iterator> ModStream::symbols() const {
+  return llvm::make_range(SymbolsSubstream.begin(), SymbolsSubstream.end());
 }