[pdb] Try super hard to conserve memory in llvm-pdbdump

[zturner]

Summary

We map the entire PDB into the process, and then when reading various streams, in order to guarantee that we have certain structures in a contiguous format, we read them out of the stream and order them appropriately, essentially copying the bytes.

This is incredibly memory inefficient, as it essentially means that loading a PDB into memory will use roughly double the file size since almost all important data structures are copied.

To address this, I've introduced a number of steps:

1. Introduce a class called StreamView. This is analagous to an ArrayRef. It provides a limited window on top of a larger stream (which can be any type of stream, including another StreamView. This is useful for constraining stream operations to specific substreams or fields, for example when one large stream is broken into multiple logical sections.

2. Introduce a set of 3 "stream data structures". These are currently StreamString, FixedStreamArray<T>, and VarStreamArray. These classes all share the same underlying purpose: Try to return references to values in the source byte stream if it can be done contiguously, but if not, copy them into temporary storage. The first one wraps a String, the second one wraps an array of fixed size records, and the third one wraps an array of variable length records. VarStreamArray will prove particularly useful, because in order to return a reference to the data in the source byte stream, the entire array need not be contiguous, only the single record being requested.

3. Update the StreamReader class to be able to read values of type StreamString, FixedStreamArray<T>, and VarStreamArray.

I updated DBIStream to use these new classes in a number of places, but currently there is not much memory savings because it's not yet being used on the type and symbol records stream, which comprise 95% of the file size. I plan to do that in a subsequent patch but I just wanted to get the infrastructure in place first.

[ruiu]

A more radical idea would be to call mmap multiple times with explicit "map-to" addresses to map parts of a PDB file to stitch the blocks that consist a stream together in the virtual memory space. Then you could use regular StringRef and ArrayRef, and it would probably be very fast. But this wouldn't work if your machine's page size is larger than the file's page size. So I don't know if it's feasible.

include/llvm/DebugInfo/CodeView/StreamArray.h

24

Is "discontiguous blocks in a file" better?

97

It will construct an object of type T. If T's constructor needs a parameter, it wouldn't compile. You want to use

mutable uint8_t TempItem[sizeof(T)];

instead for a buffer.

lib/DebugInfo/CodeView/StreamArray.cpp

20 ↗

(On Diff #58540)

So RecordOffsets maps each byte position to each byte position, correct? I wonder if it's going to be too much. If you access sequentially from 0 to 100MB using operator[], is it going to insert 100 million items to the hash?

51 ↗

(On Diff #58540)

Please add newline at end of file.

lib/DebugInfo/CodeView/StreamString.cpp

1 ↗

(On Diff #58540)

I wouldn't check this file in until I need to. ".h" only should be fine.

[zturner]

Updated based on suggestions

[zturner]

In addition to the above comments, I went ahead and fixed VarStreamArray to work, and I am actually using it to iterate over the ModInfos that we were previously using a custom iterator for. In order to make this work, I had to introduce a new StreamObject class. In the future we may even be able to simplify the code of FixedStreamArray<T> by making it internally use StreamObject<T> to handle some of the logic.

Since this class now works correctly for the ModuleInfo array, I expect it will be simple to extend it to the symbol and type streams, but I want to do that in a followup patch.

include/llvm/DebugInfo/CodeView/StreamArray.h

98

If T has a non-trivial constructor, this class won't work anyway because we are reinterpret-casting a raw byte stream to the object type. So, in fact, this class *should* fail to compile for classes with non-trivial constructors.

lib/DebugInfo/CodeView/StreamArray.cpp

20 ↗

(On Diff #58540)

Yea, maybe so. I think at some point we may need something like this, but on the other hand maybe we can use the integrated hash table for those. For now I've removed this and we support forward only iteration.

[ruiu]

include/llvm/DebugInfo/CodeView/StreamObject.h

26 ↗

(On Diff #58573)

sizeof(T) is a compile-time constant. Do you need to allocate it dynamically?

lib/DebugInfo/CodeView/StreamReader.cpp

76–79

This seems to be copying the string contents to the local std::string object whether the source string is contiguous in memory or not. Am I missing something?

[zturner]

include/llvm/DebugInfo/CodeView/StreamObject.h

26 ↗

(On Diff #58573)

I did this because I didn't want sizeof(StreamObject) to be >= sizeof(T). It's probably minor, but in any case I should be able to fix this after switching to the pooled allocator.

lib/DebugInfo/CodeView/StreamReader.cpp

76–79

This was kind of a weird function to write. Yes, it copies the source string to a temporary std::string object. It does this in order to figure out how long the string is. The logic of scanning one by one for a \0 is already handled in the ovther overload of this function and I didn't want to reproduce it. So after it returns the string in a temporary, it can calculate the length N. Then, it tries to get N bytes contiguously (see the call to getArrayRef below. If it works, yay, make a StreamString with a reference to those N bytes. If it didn't work, make a StreamString using the temporary std::string (which will make a copy of the temporary variable into its own internal std::string, and then return a reference to that).

In any case, I'm re-working all this stuff to use the pooled allocator, because since using the temporary storage is so rare, it doesn't make sense to have the overhead of storing a whole std::string every time.

[rnk]

We discussed in person structuring this so we don't need special stream classes to refer to data from the file or from a memory pool on the side.

include/llvm/DebugInfo/PDB/Raw/ModInfo.h

48–49

Let's try to arrange things so these are just plain StringRefs pointing into the file or optionally a pool on the side.

[zturner]

Updated based on suggestions. With this patch I've deleted StreamObject and StreamString, and fallen back to raw pointers and StringRefs. MappedBlockStream uses a BumpPtrAllocator to allocate space for contiguous memory only in situations where the read would otherwise have crossed a block boundary.

We also removed many uses of ByteStream (a source of memory copying), but a few still remain, in particular with the symbol and type streams since they are more difficult.

I did however get rid of the ByteStream used for the module file info array in DBIStream, and am using VarStreamArray instead, which takes advantage of these zero-copy improvements. This is to show that it works in practice before we move on to applying it to the rest of the streams.

[zturner]

Cache addresses which were previously allocated from the pool so that we don't keep allocating from the pool multiple times for the same stream offset.

[ruiu]

Generally looking good. I think I now fully understand how it works.

include/llvm/DebugInfo/CodeView/StreamArray.h

127–129

If this should never happen, you want to make it an assertion instead of swallowing errors.

lib/DebugInfo/CodeView/StreamReader.cpp

76–79

This looks much better now.

lib/DebugInfo/PDB/Raw/MappedBlockStream.cpp

39–40

How is it likely that a object we want to fetch spans two blocks but the two blocks are contiguous in the file? If it is likely, you might want to optimize for that case. (In that case you don't need to copy contents.)

[zturner]

lib/DebugInfo/PDB/Raw/MappedBlockStream.cpp

39–40

A quick use of llvm-pdbdump -raw-stream-blocks on a large PDB shows this is actually surprisingly common. In fact, streams blocks are almost always contiguous. So this will be a really good optimization. I'll make that change and upload the patch.

[zturner]

Updated based on suggestions. Now can return cross-block references as long as the referenced blocks are contiguous. Also got a big memory savings by not reading the entire names buffer in the NameHashTable class as a giant StringRef. Instead I now just treat it as a StreamRef and read individual streams from it.

With this patch across all streams of a 9MB PDB, the total amount of memory copied is less than 1,000 bytes.

[ruiu]

LGTM

lib/DebugInfo/PDB/Raw/MappedBlockStream.cpp

63

Please write a comment for this function that the function is to try to read contents if they are contiguous in file.

include/llvm/DebugInfo/CodeView/ByteStream.h

 
 namespace llvm {
 namespace codeview {
 class StreamReader;
 
 class ByteStream : public StreamInterface {
 public:
   ByteStream();
-  explicit ByteStream(MutableArrayRef<uint8_t> Bytes);
-  explicit ByteStream(uint32_t Length);
+  explicit ByteStream(MutableArrayRef<uint8_t> Data);
   ~ByteStream() override;
 
   void reset();
-  void initialize(MutableArrayRef<uint8_t> Bytes);
-  void initialize(uint32_t Length);
-  Error initialize(StreamReader &Reader, uint32_t Length);
+
+  void load(uint32_t Length);
+  Error load(StreamReader &Reader, uint32_t Length);
 
   Error readBytes(uint32_t Offset,
                   MutableArrayRef<uint8_t> Buffer) const override;
-
-  Error getArrayRef(uint32_t Offset, ArrayRef<uint8_t> &Buffer,
-                    uint32_t Length) const override;
+  Error readBytes(uint32_t Offset, uint32_t Size,
+                  ArrayRef<uint8_t> &Buffer) const override;
 
   uint32_t getLength() const override;
 
   ArrayRef<uint8_t> data() const { return Data; }
   StringRef str() const;
 
 private:
   MutableArrayRef<uint8_t> Data;
   std::unique_ptr<uint8_t[]> Ownership;
 };
 
 } // end namespace pdb
 } // end namespace llvm
 
 #endif // LLVM_DEBUGINFO_CODEVIEW_BYTESTREAM_H

include/llvm/DebugInfo/CodeView/StreamArray.h

+//===- StreamArray.h - Array backed by an arbitrary stream ----------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_DEBUGINFO_CODEVIEW_STREAMARRAY_H
+#define LLVM_DEBUGINFO_CODEVIEW_STREAMARRAY_H
+
+#include "llvm/DebugInfo/CodeView/StreamRef.h"
+
+#include <functional>
+
+namespace llvm {
+namespace codeview {
+
+/// VarStreamArray represents an array of variable length records backed by a
+/// stream.  This could be a contiguous sequence of bytes in memory, it could
+/// be a file on disk, or it could be a PDB stream where bytes are stored as
+/// discontiguous blocks in a file.  Usually it is desirable to treat arrays
+/// as contiguous blocks of memory, but doing so with large PDB files, for
+/// example, could mean allocating huge amounts of memory just to allow
+/// re-ordering of stream data to be contiguous before iterating over it.  By
+/// abstracting this out, we need not duplicate this memory, and we can
+/// iterate over arrays in arbitrarily formatted streams.
+class VarStreamArrayIterator;
+
+class VarStreamArray {
+  friend class VarStreamArrayIterator;
+  typedef std::function<uint32_t(const StreamInterface &)> LengthFuncType;
+
+public:
+  template <typename LengthFunc>
+  VarStreamArray(StreamRef Stream, const LengthFunc &Len)
+      : Stream(Stream), Len(Len) {}
+
+  VarStreamArrayIterator begin() const;
+  VarStreamArrayIterator end() const;
+
+private:
+  StreamRef Stream;
+  LengthFuncType Len; // Function used to calculate legth of a record
+};
+
+class VarStreamArrayIterator {
+public:
+  VarStreamArrayIterator(const VarStreamArray &Array)
+      : Array(&Array), IterRef(Array.Stream) {
+    ThisLen = Array.Len(IterRef);
+  }
+  VarStreamArrayIterator() : Array(nullptr), IterRef() {}
+  bool operator==(const VarStreamArrayIterator &R) const {
+    if (Array && R.Array) {
+      // Both have a valid array, make sure they're same.
+      assert(Array == R.Array);
+      return IterRef == R.IterRef;
+    }
+
+    // Both iterators are at the end.
+    if (!Array && !R.Array)
+      return true;
+
+    // One is not at the end and one is.
+    return false;
+  }
+
+  bool operator!=(const VarStreamArrayIterator &R) { return !(*this == R); }
+
+  StreamRef operator*() const {
+    ArrayRef<uint8_t> Result;
+    return IterRef.keep_front(ThisLen);
+  }
+
+  VarStreamArrayIterator &operator++() {
+    if (!Array || IterRef.getLength() == 0)
+      return *this;
+    IterRef = IterRef.drop_front(ThisLen);
+    if (IterRef.getLength() == 0) {
+      Array = nullptr;
+      ThisLen = 0;
+    } else {
+      ThisLen = Array->Len(IterRef);
+    }
+    return *this;
+  }
+
+  VarStreamArrayIterator operator++(int) {
+    VarStreamArrayIterator Original = *this;
+    ++*this;
+    return Original;
+  }
+
+private:
+  const VarStreamArray *Array;
+  uint32_t ThisLen;
+  StreamRef IterRef;
+};
+
+inline VarStreamArrayIterator VarStreamArray::begin() const {
+  return VarStreamArrayIterator(*this);
+}
+inline VarStreamArrayIterator VarStreamArray::end() const {
+  return VarStreamArrayIterator();
+}
+
+template <typename T> class FixedStreamArrayIterator;
+
+template <typename T> class FixedStreamArray {
+  friend class FixedStreamArrayIterator<T>;
+  static_assert(std::is_trivially_constructible<T>::value,
+                "FixedStreamArray must be used with trivial types");
+
+public:
+  FixedStreamArray() : Stream() {}
+  FixedStreamArray(StreamRef Stream) : Stream(Stream) {
+    assert(Stream.getLength() % sizeof(T) == 0);
+  }
+
+  const T &operator[](uint32_t Index) const {
+    assert(Index < size());
+    uint32_t Off = Index * sizeof(T);
+    ArrayRef<uint8_t> Data;
+    if (auto EC = Stream.readBytes(Off, sizeof(T), Data)) {
+      // This should never happen since we asserted that the stream length was
+      // an exact multiple of the element size.
+      consumeError(std::move(EC));
+    }
+    return *reinterpret_cast<const T *>(Data.data());
+  }
+
+  uint32_t size() const { return Stream.getLength() / sizeof(T); }
+
+  FixedStreamArrayIterator<T> begin() const {
+    return FixedStreamArrayIterator<T>(*this, 0);
+  }
+  FixedStreamArrayIterator<T> end() const {
+    return FixedStreamArrayIterator<T>(*this);
+  }
+
+private:
+  StreamRef Stream;
+};
+
+template <typename T> class FixedStreamArrayIterator {
+public:
+  FixedStreamArrayIterator(const FixedStreamArray<T> &Array)
+      : Array(Array), Index(uint32_t(-1)) {}
+  FixedStreamArrayIterator(const FixedStreamArray<T> &Array, uint32_t Index)
+      : Array(Array), Index(Index) {}
+
+  bool operator==(const FixedStreamArrayIterator<T> &R) {
+    assert(&Array == &R.Array);
+    return Index == R.Index;
+  }
+
+  bool operator!=(const FixedStreamArrayIterator<T> &R) {
+    return !(*this == R);
+  }
+
+  const T &operator*() const { return Array[Index]; }
+
+  FixedStreamArrayIterator<T> &operator++() {
+    if (Index == uint32_t(-1))
+      return *this;
+    if (++Index >= Array.size())
+      Index = uint32_t(-1);
+    return *this;
+  }
+
+  FixedStreamArrayIterator<T> operator++(int) {
+    FixedStreamArrayIterator<T> Original = *this;
+    ++*this;
+    return Original;
+  }
+
+private:
+  const FixedStreamArray<T> &Array;
+  uint32_t Index;
+};
+
+} // namespace codeview
+} // namespace llvm
+
+#endif // LLVM_DEBUGINFO_CODEVIEW_STREAMARRAY_H

include/llvm/DebugInfo/CodeView/StreamInterface.h

 
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/Support/Error.h"
 #include <cstdint>
 
 namespace llvm {
 namespace codeview {
 
+/// StreamInterface abstracts the notion of a data stream.  This way, an
+/// implementation could implement trivial reading from a contiguous memory
+/// buffer or, as in the case of PDB files, reading from a set of possibly
+/// discontiguous blocks.  The implementation is required to return references
+/// to stable memory, so if this is not possible (for example in the case of
+/// a PDB file with discontiguous blocks, it must keep its own pool of temp
+/// storage.
 class StreamInterface {
 public:
   virtual ~StreamInterface() {}
 
   virtual Error readBytes(uint32_t Offset,
                           MutableArrayRef<uint8_t> Buffer) const = 0;
-  virtual Error getArrayRef(uint32_t Offset, ArrayRef<uint8_t> &Buffer,
-                            uint32_t Length) const = 0;
+  virtual Error readBytes(uint32_t Offset, uint32_t Size,
+                          ArrayRef<uint8_t> &Buffer) const = 0;
 
   virtual uint32_t getLength() const = 0;
 };
 
 } // end namespace codeview
 } // end namespace llvm
 
 #endif // LLVM_DEBUGINFO_CODEVIEW_STREAMINTERFACE_H

include/llvm/DebugInfo/CodeView/StreamReader.h

 //===- StreamReader.h - Reads bytes and objects from a stream ---*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_DEBUGINFO_CODEVIEW_STREAMREADER_H
 #define LLVM_DEBUGINFO_CODEVIEW_STREAMREADER_H
 
 #include "llvm/ADT/ArrayRef.h"
+#include "llvm/DebugInfo/CodeView/CodeViewError.h"
+#include "llvm/DebugInfo/CodeView/StreamArray.h"
 #include "llvm/DebugInfo/CodeView/StreamInterface.h"
 #include "llvm/Support/Endian.h"
 #include "llvm/Support/Error.h"
 
 #include <string>
 
 namespace llvm {
 namespace codeview {
 
+class StreamRef;
+
 class StreamReader {
 public:
   StreamReader(const StreamInterface &S);
 
+  Error readBytes(uint32_t Size, ArrayRef<uint8_t> &Buffer);
   Error readBytes(MutableArrayRef<uint8_t> Buffer);
+  Error readInteger(uint16_t &Dest);
   Error readInteger(uint32_t &Dest);
-  Error readZeroString(std::string &Dest);
+  Error readZeroString(StringRef &Dest);
+  Error readFixedString(StringRef &Dest, uint32_t Length);
+  Error readStreamRef(StreamRef &Ref);
+  Error readStreamRef(StreamRef &Ref, uint32_t Length);
+
+  template <typename T> Error readObject(const T *&Dest) {
+    ArrayRef<uint8_t> Buffer;
+    if (auto EC = readBytes(sizeof(T), Buffer))
+      return EC;
+    Dest = reinterpret_cast<const T *>(Buffer.data());
+    return Error::success();
+  }
 
-  template <typename T> Error readObject(T *Dest) {
-    MutableArrayRef<uint8_t> Buffer(reinterpret_cast<uint8_t *>(Dest),
-                                    sizeof(T));
-    return readBytes(Buffer);
+  template <typename T>
+  Error readArray(FixedStreamArray<T> &Array, uint32_t NumItems) {
+    if (NumItems == 0) {
+      Array = FixedStreamArray<T>();
+      return Error::success();
+    }
+    uint32_t Length = NumItems * sizeof(T);
+    if (Offset + Length > Stream.getLength())
+      return make_error<CodeViewError>(cv_error_code::insufficient_buffer);
+    StreamRef View(Stream, Offset, Length);
+    Array = FixedStreamArray<T>(View);
+    Offset += Length;
+    return Error::success();
   }
 
   template <typename T> Error readArray(MutableArrayRef<T> Array) {
     MutableArrayRef<uint8_t> Casted(reinterpret_cast<uint8_t *>(Array.data()),
                                     Array.size() * sizeof(T));
     return readBytes(Casted);
   }
 
-  Error getArrayRef(ArrayRef<uint8_t> &Array, uint32_t Length);
-
   void setOffset(uint32_t Off) { Offset = Off; }
   uint32_t getOffset() const { return Offset; }
   uint32_t getLength() const { return Stream.getLength(); }
   uint32_t bytesRemaining() const { return getLength() - getOffset(); }
 
 private:
   const StreamInterface &Stream;
   uint32_t Offset;
 };
 } // namespace codeview
 } // namespace llvm
 
 #endif // LLVM_DEBUGINFO_CODEVIEW_STREAMREADER_H

include/llvm/DebugInfo/CodeView/StreamRef.h

+//===- StreamRef.h - A copyable reference to a stream -----------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_DEBUGINFO_CODEVIEW_STREAMREF_H
+#define LLVM_DEBUGINFO_CODEVIEW_STREAMREF_H
+
+#include "llvm/DebugInfo/CodeView/StreamInterface.h"
+
+namespace llvm {
+namespace codeview {
+
+class StreamRef : public StreamInterface {
+public:
+  StreamRef() : Stream(nullptr), ViewOffset(0), Length(0) {}
+  StreamRef(const StreamInterface &Stream)
+      : Stream(&Stream), ViewOffset(0), Length(Stream.getLength()) {}
+  StreamRef(const StreamInterface &Stream, uint32_t Offset, uint32_t Length)
+      : Stream(&Stream), ViewOffset(Offset), Length(Length) {}
+  StreamRef(const StreamRef &Other)
+      : Stream(Other.Stream), ViewOffset(Other.ViewOffset),
+        Length(Other.Length) {}
+
+  Error readBytes(uint32_t Offset,
+                  MutableArrayRef<uint8_t> Buffer) const override {
+    return Stream->readBytes(ViewOffset + Offset, Buffer);
+  }
+
+  Error readBytes(uint32_t Offset, uint32_t Size,
+                  ArrayRef<uint8_t> &Buffer) const override {
+    return Stream->readBytes(ViewOffset + Offset, Size, Buffer);
+  }
+
+  uint32_t getLength() const override { return Length; }
+  StreamRef drop_front(uint32_t N) const {
+    if (!Stream)
+      return StreamRef();
+
+    N = std::min(N, Length);
+    return StreamRef(*Stream, ViewOffset + N, Length - N);
+  }
+
+  StreamRef keep_front(uint32_t N) const {
+    if (!Stream)
+      return StreamRef();
+    N = std::min(N, Length);
+    return StreamRef(*Stream, ViewOffset, N);
+  }
+
+  bool operator==(const StreamRef &Other) const {
+    if (Stream != Other.Stream)
+      return false;
+    if (ViewOffset != Other.ViewOffset)
+      return false;
+    if (Length != Other.Length)
+      return false;
+    return true;
+  }
+
+  bool operator!=(const StreamRef &Other) const { return !(*this == Other); }
+
+  StreamRef &operator=(const StreamRef &Other) {
+    Stream = Other.Stream;
+    ViewOffset = Other.ViewOffset;
+    Length = Other.Length;
+    return *this;
+  }
+
+private:
+  const StreamInterface *Stream;
+  uint32_t ViewOffset;
+  uint32_t Length;
+};
+}
+}
+
+#endif // LLVM_DEBUGINFO_CODEVIEW_STREAMREF_H
 No newline at end of file

include/llvm/DebugInfo/PDB/Raw/DbiStream.h

 //===- DbiStream.h - PDB Dbi Stream (Stream 3) Access -----------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_DEBUGINFO_PDB_RAW_PDBDBISTREAM_H
 #define LLVM_DEBUGINFO_PDB_RAW_PDBDBISTREAM_H
 
 #include "llvm/DebugInfo/CodeView/ByteStream.h"
+#include "llvm/DebugInfo/CodeView/StreamArray.h"
+#include "llvm/DebugInfo/CodeView/StreamRef.h"
 #include "llvm/DebugInfo/PDB/PDBTypes.h"
 #include "llvm/DebugInfo/PDB/Raw/MappedBlockStream.h"
 #include "llvm/DebugInfo/PDB/Raw/ModInfo.h"
 #include "llvm/DebugInfo/PDB/Raw/NameHashTable.h"
 #include "llvm/DebugInfo/PDB/Raw/RawConstants.h"
 #include "llvm/Support/Endian.h"
 #include "llvm/Support/Error.h"
 
private:
   Error initializeFileInfo();
 
   PDBFile &Pdb;
   MappedBlockStream Stream;
 
   std::vector<ModuleInfoEx> ModuleInfos;
   NameHashTable ECNames;
 
-  codeview::ByteStream ModInfoSubstream;
-  codeview::ByteStream SecContrSubstream;
-  codeview::ByteStream SecMapSubstream;
-  codeview::ByteStream FileInfoSubstream;
-  codeview::ByteStream TypeServerMapSubstream;
-  codeview::ByteStream ECSubstream;
-  codeview::ByteStream DbgHeader;
+  codeview::StreamRef ModInfoSubstream;
+  codeview::StreamRef SecContrSubstream;
+  codeview::StreamRef SecMapSubstream;
+  codeview::StreamRef FileInfoSubstream;
+  codeview::StreamRef TypeServerMapSubstream;
+  codeview::StreamRef ECSubstream;
 
-  std::unique_ptr<HeaderInfo> Header;
+  codeview::FixedStreamArray<support::ulittle16_t> DbgStreams;
+
+  const HeaderInfo *Header;
 };
 }
 }
 
 #endif

include/llvm/DebugInfo/PDB/Raw/MappedBlockStream.h

 //===- MappedBlockStream.h - Reads stream data from a PDBFile ---*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_DEBUGINFO_PDB_RAW_MAPPEDBLOCKSTREAM_H
 #define LLVM_DEBUGINFO_PDB_RAW_MAPPEDBLOCKSTREAM_H
 
 #include "llvm/ADT/ArrayRef.h"
+#include "llvm/ADT/DenseMap.h"
 #include "llvm/DebugInfo/CodeView/StreamInterface.h"
+#include "llvm/Support/Allocator.h"
 #include "llvm/Support/Error.h"
 #include <cstdint>
 #include <vector>
 
 namespace llvm {
 namespace pdb {
 
 class PDBFile;
 
 class MappedBlockStream : public codeview::StreamInterface {
 public:
   MappedBlockStream(uint32_t StreamIdx, const PDBFile &File);
 
   Error readBytes(uint32_t Offset,
                   MutableArrayRef<uint8_t> Buffer) const override;
-  Error getArrayRef(uint32_t Offset, ArrayRef<uint8_t> &Buffer,
-                    uint32_t Length) const override;
+  Error readBytes(uint32_t Offset, uint32_t Size,
+                  ArrayRef<uint8_t> &Buffer) const override;
 
   uint32_t getLength() const override { return StreamLength; }
 
 private:
   uint32_t StreamLength;
   std::vector<uint32_t> BlockList;
+  mutable llvm::BumpPtrAllocator Pool;
+  mutable DenseMap<uint32_t, uint8_t *> CacheMap;
   const PDBFile &Pdb;
 };
 
 } // end namespace pdb
 } // end namespace llvm
 
 #endif // LLVM_DEBUGINFO_PDB_RAW_MAPPEDBLOCKSTREAM_H

include/llvm/DebugInfo/PDB/Raw/ModInfo.h

 //===- ModInfo.h - PDB module information -----------------------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_DEBUGINFO_PDB_RAW_MODINFO_H
 #define LLVM_DEBUGINFO_PDB_RAW_MODINFO_H
 
 #include "llvm/ADT/StringRef.h"
+#include "llvm/DebugInfo/CodeView/StreamRef.h"
 #include <cstdint>
 #include <vector>
 
 namespace llvm {
 namespace pdb {
 
 class ModInfo {
 private:
   struct FileLayout;
 
 public:
-  ModInfo(const uint8_t *Bytes);
+  ModInfo(codeview::StreamRef Stream);
+  ModInfo(const ModInfo &Info);
   ~ModInfo();
 
   bool hasECInfo() const;
   uint16_t getTypeServerIndex() const;
   uint16_t getModuleStreamIndex() const;
   uint32_t getSymbolDebugInfoByteSize() const;
   uint32_t getLineInfoByteSize() const;
   uint32_t getC13LineInfoByteSize() const;
   uint32_t getNumberOfFiles() const;
   uint32_t getSourceFileNameIndex() const;
   uint32_t getPdbFilePathNameIndex() const;
 
   StringRef getModuleName() const;
   StringRef getObjFileName() const;
 
+  uint32_t getRecordLength() const;
+
 private:
+  StringRef ModuleName;
+  StringRef ObjFileName;
   const FileLayout *Layout;
 };
 
 struct ModuleInfoEx {
-  ModuleInfoEx(ModInfo Module) : Info(Module) {}
+  ModuleInfoEx(codeview::StreamRef Stream) : Info(Stream) {}
+  ModuleInfoEx(const ModuleInfoEx &Ex)
+      : Info(Ex.Info), SourceFiles(Ex.SourceFiles) {}
 
   ModInfo Info;
   std::vector<StringRef> SourceFiles;
 };
 
-class ModInfoIterator {
-public:
-  ModInfoIterator(const uint8_t *Stream);
-  ModInfoIterator(const ModInfoIterator &Other);
-
-  ModInfo operator*();
-  ModInfoIterator &operator++();
-  ModInfoIterator operator++(int);
-  bool operator==(const ModInfoIterator &Other);
-  bool operator!=(const ModInfoIterator &Other);
-  ModInfoIterator &operator=(const ModInfoIterator &Other);
-
-private:
-  const uint8_t *Bytes;
-};
+inline uint32_t ModInfoRecordLength(const codeview::StreamInterface &Stream) {
+  return ModInfo(Stream).getRecordLength();
+}
 
 } // end namespace pdb
 } // end namespace llvm
 
 #endif // LLVM_DEBUGINFO_PDB_RAW_MODINFO_H

include/llvm/DebugInfo/PDB/Raw/ModStream.h

 //===- ModStream.h - PDB Module Info Stream Access ------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_DEBUGINFO_PDB_RAW_MODSTREAM_H
 #define LLVM_DEBUGINFO_PDB_RAW_MODSTREAM_H
 
 #include "llvm/ADT/iterator_range.h"
 #include "llvm/DebugInfo/CodeView/ByteStream.h"
+#include "llvm/DebugInfo/CodeView/StreamRef.h"
 #include "llvm/DebugInfo/CodeView/SymbolRecord.h"
 #include "llvm/DebugInfo/PDB/Raw/MappedBlockStream.h"
 #include "llvm/Support/Error.h"
 
 namespace llvm {
 namespace pdb {
 class PDBFile;
 class ModInfo;
 
 class ModStream {
 public:
   ModStream(PDBFile &File, const ModInfo &Module);
   ~ModStream();
 
   Error reload();
 
   iterator_range<codeview::SymbolIterator> symbols() const;
 
 private:
   const ModInfo &Mod;
 
   MappedBlockStream Stream;
 
   codeview::ByteStream SymbolsSubstream;
-  codeview::ByteStream LinesSubstream;
-  codeview::ByteStream C13LinesSubstream;
-  codeview::ByteStream GlobalRefsSubstream;
+  codeview::StreamRef LinesSubstream;
+  codeview::StreamRef C13LinesSubstream;
+  codeview::StreamRef GlobalRefsSubstream;
 };
 }
 }
 
 #endif

include/llvm/DebugInfo/PDB/Raw/NameHashTable.h

 //===- NameHashTable.h - PDB Name Hash Table --------------------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_DEBUGINFO_PDB_RAW_NAMEHASHTABLE_H
 #define LLVM_DEBUGINFO_PDB_RAW_NAMEHASHTABLE_H
 
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/StringRef.h"
-#include "llvm/DebugInfo/CodeView/ByteStream.h"
+#include "llvm/DebugInfo/CodeView/StreamRef.h"
 #include "llvm/Support/Error.h"
 #include <cstdint>
 #include <vector>
 
 namespace llvm {
 namespace codeview {
 class StreamReader;
 }
public:
   uint32_t getSignature() const { return Signature; }
 
   StringRef getStringForID(uint32_t ID) const;
   uint32_t getIDForString(StringRef Str) const;
 
   ArrayRef<uint32_t> name_ids() const;
 
 private:
-  codeview::ByteStream NamesBuffer;
+  StringRef NamesBuffer;
   std::vector<uint32_t> IDs;
   uint32_t Signature;
   uint32_t HashVersion;
   uint32_t NameCount;
 };
 
 } // end namespace pdb
 } // end namespace llvm
 
 #endif // LLVM_DEBUGINFO_PDB_RAW_NAMEHASHTABLE_H

include/llvm/DebugInfo/PDB/Raw/PublicsStream.h

private:
   MappedBlockStream Stream;
   uint32_t NumBuckets = 0;
   std::vector<HashRecord> HashRecords;
   std::vector<uint32_t> HashBuckets;
   std::vector<uint32_t> AddressMap;
   std::vector<uint32_t> ThunkMap;
   std::vector<uint32_t> SectionOffsets;
 
-  std::unique_ptr<HeaderInfo> Header;
-  std::unique_ptr<GSIHashHeader> HashHdr;
+  const HeaderInfo *Header;
+  const GSIHashHeader *HashHdr;
 };
 }
 }
 
 #endif

include/llvm/DebugInfo/PDB/Raw/TpiStream.h

 //===- TpiStream.cpp - PDB Type Info (TPI) Stream 2 Access ------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_DEBUGINFO_PDB_RAW_PDBTPISTREAM_H
 #define LLVM_DEBUGINFO_PDB_RAW_PDBTPISTREAM_H
 
 #include "llvm/DebugInfo/CodeView/ByteStream.h"
+#include "llvm/DebugInfo/CodeView/StreamRef.h"
 #include "llvm/DebugInfo/CodeView/TypeStream.h"
 #include "llvm/DebugInfo/PDB/PDBTypes.h"
 #include "llvm/DebugInfo/PDB/Raw/MappedBlockStream.h"
 #include "llvm/DebugInfo/PDB/Raw/RawConstants.h"
 
 #include "llvm/Support/Error.h"
 
 namespace llvm {
public:
   iterator_range<codeview::TypeIterator> types(bool *HadError) const;
 
 private:
   PDBFile &Pdb;
   MappedBlockStream Stream;
   HashFunctionType HashFunction;
 
   codeview::ByteStream RecordsBuffer;
-  codeview::ByteStream TypeIndexOffsetBuffer;
-  codeview::ByteStream HashValuesBuffer;
-  codeview::ByteStream HashAdjBuffer;
+  codeview::StreamRef TypeIndexOffsetBuffer;
+  codeview::StreamRef HashValuesBuffer;
+  codeview::StreamRef HashAdjBuffer;
 
-  std::unique_ptr<HeaderInfo> Header;
+  const HeaderInfo *Header;
 };
 }
 }
 
 #endif

lib/DebugInfo/CodeView/ByteStream.cpp

 #include "llvm/DebugInfo/CodeView/StreamReader.h"
 #include <cstring>
 
 using namespace llvm;
 using namespace llvm::codeview;
 
 ByteStream::ByteStream() {}
 
-ByteStream::ByteStream(MutableArrayRef<uint8_t> Bytes) { initialize(Bytes); }
+ByteStream::ByteStream(MutableArrayRef<uint8_t> Data) : Data(Data) {}
 
-ByteStream::ByteStream(uint32_t Length) { initialize(Length); }
-
-ByteStream::~ByteStream() { reset(); }
+ByteStream::~ByteStream() {}
 
 void ByteStream::reset() {
   Ownership.reset();
   Data = MutableArrayRef<uint8_t>();
 }
 
-void ByteStream::initialize(MutableArrayRef<uint8_t> Bytes) {
-  reset();
-  Data = Bytes;
-}
-
-void ByteStream::initialize(uint32_t Length) {
+void ByteStream::load(uint32_t Length) {
   reset();
   if (Length > 0)
     Data = MutableArrayRef<uint8_t>(new uint8_t[Length], Length);
   Ownership.reset(Data.data());
 }
 
-Error ByteStream::initialize(StreamReader &Reader, uint32_t Length) {
-  initialize(Length);
+Error ByteStream::load(StreamReader &Reader, uint32_t Length) {
+  load(Length);
   auto EC = Reader.readBytes(Data);
   if (EC)
     reset();
   return EC;
 }
 
 Error ByteStream::readBytes(uint32_t Offset,
                             MutableArrayRef<uint8_t> Buffer) const {
   if (Data.size() < Buffer.size() + Offset)
     return make_error<CodeViewError>(cv_error_code::insufficient_buffer);
-  ::memcpy(Buffer.data(), Data.data() + Offset, Buffer.size());
+  ::memcpy(Buffer.data() + Offset, Data.data(), Buffer.size());
   return Error::success();
 }
 
-Error ByteStream::getArrayRef(uint32_t Offset, ArrayRef<uint8_t> &Buffer,
-                              uint32_t Length) const {
-  if (Data.size() < Length + Offset)
+Error ByteStream::readBytes(uint32_t Offset, uint32_t Size,
+                            ArrayRef<uint8_t> &Buffer) const {
+  if (Data.size() < Buffer.size() + Offset)
     return make_error<CodeViewError>(cv_error_code::insufficient_buffer);
-  Buffer = Data.slice(Offset, Length);
+  Buffer = Data.slice(Offset, Size);
   return Error::success();
 }
 
 uint32_t ByteStream::getLength() const { return Data.size(); }
 
 StringRef ByteStream::str() const {
   const char *CharData = reinterpret_cast<const char *>(Data.data());
   return StringRef(CharData, Data.size());
 }

lib/DebugInfo/CodeView/StreamReader.cpp

 //===- StreamReader.cpp - Reads bytes and objects from a stream -----------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/DebugInfo/CodeView/StreamReader.h"
 
 #include "llvm/DebugInfo/CodeView/CodeViewError.h"
+#include "llvm/DebugInfo/CodeView/StreamRef.h"
 
 using namespace llvm;
 using namespace llvm::codeview;
 
 StreamReader::StreamReader(const StreamInterface &S) : Stream(S), Offset(0) {}
 
+Error StreamReader::readBytes(uint32_t Size, ArrayRef<uint8_t> &Buffer) {
+  if (auto EC = Stream.readBytes(Offset, Size, Buffer))
+    return EC;
+  Offset += Size;
+  return Error::success();
+}
+
 Error StreamReader::readBytes(MutableArrayRef<uint8_t> Buffer) {
   if (auto EC = Stream.readBytes(Offset, Buffer))
     return EC;
   Offset += Buffer.size();
   return Error::success();
 }
 
+Error StreamReader::readInteger(uint16_t &Dest) {
+  const support::ulittle16_t *P;
+  if (auto EC = readObject(P))
+    return EC;
+  Dest = *P;
+  return Error::success();
+}
+
 Error StreamReader::readInteger(uint32_t &Dest) {
-  support::ulittle32_t P;
-  if (auto EC = readObject(&P))
+  const support::ulittle32_t *P;
+  if (auto EC = readObject(P))
     return EC;
-  Dest = P;
+  Dest = *P;
   return Error::success();
 }
 
-Error StreamReader::readZeroString(std::string &Dest) {
-  Dest.clear();
-  char C;
+Error StreamReader::readZeroString(StringRef &Dest) {
+  uint32_t Length = 0;
+  // First compute the length of the string by reading 1 byte at a time.
+  uint32_t OriginalOffset = getOffset();
+  const char *C;
   do {
-    if (auto EC = readObject(&C))
+    if (auto EC = readObject(C))
       return EC;
-    if (C != '\0')
-      Dest.push_back(C);
-  } while (C != '\0');
+    if (*C != '\0')
+      ++Length;
+  } while (*C != '\0');
+  // Now go back and request a reference for that many bytes.
+  uint32_t NewOffset = getOffset();
+  setOffset(OriginalOffset);
+
+  ArrayRef<uint8_t> Data;
+  if (auto EC = readBytes(Length, Data))
+    return EC;
+  Dest = StringRef(reinterpret_cast<const char *>(Data.begin()), Data.size());
+
+  // Now set the offset back to where it was after we calculated the length.
+  setOffset(NewOffset);
   return Error::success();
 }
 
-Error StreamReader::getArrayRef(ArrayRef<uint8_t> &Array, uint32_t Length) {
-  if (auto EC = Stream.getArrayRef(Offset, Array, Length))
+Error StreamReader::readFixedString(StringRef &Dest, uint32_t Length) {
+  ArrayRef<uint8_t> Bytes;
+  if (auto EC = readBytes(Length, Bytes))
     return EC;
+  Dest = StringRef(reinterpret_cast<const char *>(Bytes.begin()), Bytes.size());
+  return Error::success();
+}
+
+Error StreamReader::readStreamRef(StreamRef &Ref) {
+  return readStreamRef(Ref, bytesRemaining());
+}
+
+Error StreamReader::readStreamRef(StreamRef &Ref, uint32_t Length) {
+  if (bytesRemaining() < Length)
+    return make_error<CodeViewError>(cv_error_code::insufficient_buffer);
+  Ref = StreamRef(Stream, Offset, Length);
   Offset += Length;
   return Error::success();
 }

lib/DebugInfo/PDB/CMakeLists.txt

add_pdb_impl_folder(Raw
   Raw/NameMap.cpp
   Raw/PDBFile.cpp
   Raw/PublicsStream.cpp
   Raw/RawError.cpp
   Raw/RawSession.cpp
   Raw/SymbolStream.cpp
   Raw/TpiStream.cpp)
 
+list(APPEND LIBPDB_ADDITIONAL_HEADER_DIRS "${LLVM_MAIN_INCLUDE_DIR}/llvm/DebugInfo/PDB/Raw")
 list(APPEND LIBPDB_ADDITIONAL_HEADER_DIRS "${LLVM_MAIN_INCLUDE_DIR}/llvm/DebugInfo/PDB")
 
 add_llvm_library(LLVMDebugInfoPDB
   GenericError.cpp
   IPDBSourceFile.cpp
   PDB.cpp
   PDBContext.cpp
   PDBExtras.cpp

lib/DebugInfo/PDB/Raw/DbiStream.cpp

 //===- DbiStream.cpp - PDB Dbi Stream (Stream 3) Access -------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
-
 #include "llvm/DebugInfo/PDB/Raw/DbiStream.h"
 
+#include "llvm/DebugInfo/CodeView/StreamArray.h"
 #include "llvm/DebugInfo/CodeView/StreamReader.h"
 #include "llvm/DebugInfo/PDB/Raw/InfoStream.h"
 #include "llvm/DebugInfo/PDB/Raw/ModInfo.h"
 #include "llvm/DebugInfo/PDB/Raw/NameHashTable.h"
 #include "llvm/DebugInfo/PDB/Raw/PDBFile.h"
 #include "llvm/DebugInfo/PDB/Raw/RawConstants.h"
 #include "llvm/DebugInfo/PDB/Raw/RawError.h"
 
struct DbiStream::HeaderInfo {
   little32_t OptionalDbgHdrSize;  // Size of DbgHeader info
   little32_t ECSubstreamSize;     // Size of EC stream (what is EC?)
   ulittle16_t Flags;              // See DbiFlags enum.
   ulittle16_t MachineType;        // See PDB_MachineType enum.
 
   ulittle32_t Reserved; // Pad to 64 bytes
 };
 
-DbiStream::DbiStream(PDBFile &File) : Pdb(File), Stream(StreamDBI, File) {
+DbiStream::DbiStream(PDBFile &File)
+    : Pdb(File), Stream(StreamDBI, File), Header(nullptr) {
   static_assert(sizeof(HeaderInfo) == 64, "Invalid HeaderInfo size!");
 }
 
 DbiStream::~DbiStream() {}
 
 Error DbiStream::reload() {
   codeview::StreamReader Reader(Stream);
 
-  Header.reset(new HeaderInfo());
-
   if (Stream.getLength() < sizeof(HeaderInfo))
     return make_error<RawError>(raw_error_code::corrupt_file,
                                 "DBI Stream does not contain a header.");
-  if (auto EC = Reader.readObject(Header.get()))
+  if (auto EC = Reader.readObject(Header))
     return make_error<RawError>(raw_error_code::corrupt_file,
                                 "DBI Stream does not contain a header.");
 
   if (Header->VersionSignature != -1)
     return make_error<RawError>(raw_error_code::corrupt_file,
                                 "Invalid DBI version signature.");
 
   // Require at least version 7, which should be present in all PDBs
return make_error<RawError>(raw_error_code::corrupt_file,
                                 "DBI section map substream not aligned.");
   if (Header->FileInfoSize % sizeof(uint32_t) != 0)
     return make_error<RawError>(raw_error_code::corrupt_file,
                                 "DBI file info substream not aligned.");
   if (Header->TypeServerSize % sizeof(uint32_t) != 0)
     return make_error<RawError>(raw_error_code::corrupt_file,
                                 "DBI type server substream not aligned.");
 
-  if (auto EC = ModInfoSubstream.initialize(Reader, Header->ModiSubstreamSize))
+  if (auto EC =
+          Reader.readStreamRef(ModInfoSubstream, Header->ModiSubstreamSize))
     return EC;
 
   // Since each ModInfo in the stream is a variable length, we have to iterate
   // them to know how many there actually are.
-  auto Range =
-      llvm::make_range(ModInfoIterator(&ModInfoSubstream.data().front()),
-                       ModInfoIterator(&ModInfoSubstream.data().back() + 1));
-  for (auto Info : Range)
-    ModuleInfos.push_back(ModuleInfoEx(Info));
+  codeview::VarStreamArray ModInfoArray(ModInfoSubstream, ModInfoRecordLength);
+  for (auto Info : ModInfoArray) {
+    ModuleInfos.emplace_back(Info);
+  }
 
-  if (auto EC =
-          SecContrSubstream.initialize(Reader, Header->SecContrSubstreamSize))
+  if (auto EC = Reader.readStreamRef(SecContrSubstream,
+                                     Header->SecContrSubstreamSize))
     return EC;
-  if (auto EC = SecMapSubstream.initialize(Reader, Header->SectionMapSize))
+  if (auto EC = Reader.readStreamRef(SecMapSubstream, Header->SectionMapSize))
     return EC;
-  if (auto EC = FileInfoSubstream.initialize(Reader, Header->FileInfoSize))
+  if (auto EC = Reader.readStreamRef(FileInfoSubstream, Header->FileInfoSize))
     return EC;
   if (auto EC =
-          TypeServerMapSubstream.initialize(Reader, Header->TypeServerSize))
+          Reader.readStreamRef(TypeServerMapSubstream, Header->TypeServerSize))
     return EC;
-  if (auto EC = ECSubstream.initialize(Reader, Header->ECSubstreamSize))
+  if (auto EC = Reader.readStreamRef(ECSubstream, Header->ECSubstreamSize))
     return EC;
-  if (auto EC = DbgHeader.initialize(Reader, Header->OptionalDbgHdrSize))
+  if (auto EC = Reader.readArray(DbgStreams, Header->OptionalDbgHdrSize /
+                                                 sizeof(ulittle16_t)))
     return EC;
 
   if (auto EC = initializeFileInfo())
     return EC;
 
   if (Reader.bytesRemaining() > 0)
     return make_error<RawError>(raw_error_code::corrupt_file,
                                 "Found unexpected bytes in DBI Stream.");
Error DbiStream::initializeFileInfo() {
   //   ulittle16_t ModIndices[NumModules];
   //   ulittle16_t ModFileCounts[NumModules];
   //   ulittle32_t FileNameOffsets[NumSourceFiles];
   //   char Names[][NumSourceFiles];
   // };
   // with the caveat that `NumSourceFiles` cannot be trusted, so
   // it is computed by summing `ModFileCounts`.
   //
-  const uint8_t *Buf = &FileInfoSubstream.data().front();
-  auto FI = reinterpret_cast<const FileInfoSubstreamHeader *>(Buf);
-  Buf += sizeof(FileInfoSubstreamHeader);
+  const FileInfoSubstreamHeader *FH;
+  codeview::StreamReader FISR(FileInfoSubstream);
+  if (auto EC = FISR.readObject(FH))
+    return EC;
+
   // The number of modules in the stream should be the same as reported by
   // the FileInfoSubstreamHeader.
-  if (FI->NumModules != ModuleInfos.size())
+  if (FH->NumModules != ModuleInfos.size())
     return make_error<RawError>(raw_error_code::corrupt_file,
                                 "FileInfo substream count doesn't match DBI.");
 
+  codeview::FixedStreamArray<ulittle16_t> ModIndexArray;
+  codeview::FixedStreamArray<ulittle16_t> ModFileCountArray;
+  codeview::FixedStreamArray<little32_t> FileNameOffsets;
+
   // First is an array of `NumModules` module indices.  This is not used for the
   // same reason that `NumSourceFiles` is not used.  It's an array of uint16's,
   // but it's possible there are more than 64k source files, which would imply
   // more than 64k modules (e.g. object files) as well.  So we ignore this
   // field.
-  llvm::ArrayRef<ulittle16_t> ModIndexArray(
-      reinterpret_cast<const ulittle16_t *>(Buf), ModuleInfos.size());
-
-  llvm::ArrayRef<ulittle16_t> ModFileCountArray(ModIndexArray.end(),
-                                                ModuleInfos.size());
+  if (auto EC = FISR.readArray(ModIndexArray, ModuleInfos.size()))
+    return EC;
+  if (auto EC = FISR.readArray(ModFileCountArray, ModuleInfos.size()))
+    return EC;
 
   // Compute the real number of source files.
   uint32_t NumSourceFiles = 0;
   for (auto Count : ModFileCountArray)
     NumSourceFiles += Count;
 
   // This is the array that in the reference implementation corresponds to
   // `ModInfo::FileLayout::FileNameOffs`, which is commented there as being a
   // pointer. Due to the mentioned problems of pointers causing difficulty
   // when reading from the file on 64-bit systems, we continue to ignore that
   // field in `ModInfo`, and instead build a vector of StringRefs and stores
   // them in `ModuleInfoEx`.  The value written to and read from the file is
   // not used anyway, it is only there as a way to store the offsets for the
   // purposes of later accessing the names at runtime.
-  llvm::ArrayRef<little32_t> FileNameOffsets(
-      reinterpret_cast<const little32_t *>(ModFileCountArray.end()),
-      NumSourceFiles);
+  if (auto EC = FISR.readArray(FileNameOffsets, NumSourceFiles))
+    return EC;
 
-  const char *Names = reinterpret_cast<const char *>(FileNameOffsets.end());
+  codeview::StreamRef NamesBufferRef;
+  if (auto EC = FISR.readStreamRef(NamesBufferRef))
+    return EC;
+  codeview::StreamReader Names(NamesBufferRef);
 
   // We go through each ModuleInfo, determine the number N of source files for
   // that module, and then get the next N offsets from the Offsets array, using
   // them to get the corresponding N names from the Names buffer and associating
   // each one with the corresponding module.
   uint32_t NextFileIndex = 0;
   for (size_t I = 0; I < ModuleInfos.size(); ++I) {
     uint32_t NumFiles = ModFileCountArray[I];
     ModuleInfos[I].SourceFiles.resize(NumFiles);
     for (size_t J = 0; J < NumFiles; ++J, ++NextFileIndex) {
-      uint32_t FileIndex = FileNameOffsets[NextFileIndex];
-      ModuleInfos[I].SourceFiles[J] = StringRef(Names + FileIndex);
+      uint32_t FileOffset = FileNameOffsets[NextFileIndex];
+      Names.setOffset(FileOffset);
+      if (auto EC = Names.readZeroString(ModuleInfos[I].SourceFiles[J]))
+        return EC;
     }
   }
 
   return Error::success();
 }
 
 uint32_t DbiStream::getDebugStreamIndex(DbgHeaderType Type) const {
-  ArrayRef<uint8_t> DbgData;
-  if (auto EC = DbgHeader.getArrayRef(0, DbgData, DbgHeader.getLength())) {
-    consumeError(std::move(EC));
-    return uint32_t(-1);
-  }
-  ArrayRef<ulittle16_t> DebugStreams(
-      reinterpret_cast<const ulittle16_t *>(DbgData.data()),
-      DbgData.size() / sizeof(ulittle16_t));
-  return DebugStreams[static_cast<uint16_t>(Type)];
+  return DbgStreams[static_cast<uint16_t>(Type)];
 }

lib/DebugInfo/PDB/Raw/InfoStream.cpp

Error InfoStream::reload() {
 
   struct Header {
     support::ulittle32_t Version;
     support::ulittle32_t Signature;
     support::ulittle32_t Age;
     PDB_UniqueId Guid;
   };
 
-  Header H;
-  if (auto EC = Reader.readObject(&H))
+  const Header *H;
+  if (auto EC = Reader.readObject(H))
     return make_error<RawError>(raw_error_code::corrupt_file,
                                 "PDB Stream does not contain a header.");
 
-  if (H.Version < PdbRaw_ImplVer::PdbImplVC70)
+  if (H->Version < PdbRaw_ImplVer::PdbImplVC70)
     return make_error<RawError>(raw_error_code::corrupt_file,
                                 "Unsupported PDB stream version.");
 
-  Version = H.Version;
-  Signature = H.Signature;
-  Age = H.Age;
-  Guid = H.Guid;
+  Version = H->Version;
+  Signature = H->Signature;
+  Age = H->Age;
+  Guid = H->Guid;
 
   return NamedStreams.load(Reader);
 }
 
 uint32_t InfoStream::getNamedStreamIndex(llvm::StringRef Name) const {
   uint32_t Result;
   if (!NamedStreams.tryGetValue(Name, Result))
     return 0;

lib/DebugInfo/PDB/Raw/MappedBlockStream.cpp

MappedBlockStream::MappedBlockStream(uint32_t StreamIdx, const PDBFile &File) : Pdb(File) {
   if (StreamIdx >= Pdb.getNumStreams()) {
     StreamLength = 0;
   } else {
     StreamLength = Pdb.getStreamByteSize(StreamIdx);
     BlockList = Pdb.getStreamBlockList(StreamIdx);
   }
 }
 
+Error MappedBlockStream::readBytes(uint32_t Offset, uint32_t Size,
+                                   ArrayRef<uint8_t> &Buffer) const {
+  static uint32_t AllocSize = 0;
+  uint32_t BlockNum = Offset / Pdb.getBlockSize();
+  uint32_t OffsetInBlock = Offset % Pdb.getBlockSize();
+
+  // Make sure we aren't trying to read beyond the end of the stream.
+  if (Buffer.size() > StreamLength)
+    return make_error<RawError>(raw_error_code::insufficient_buffer);
+  if (Offset > StreamLength - Buffer.size())
+    return make_error<RawError>(raw_error_code::insufficient_buffer);
+
+  uint32_t BytesInFirstBlock = Pdb.getBlockSize() - OffsetInBlock;
+  // If the entire request can be satisfied from one block, return an
+  // ArrayRef directly into the block.
+  if (BytesInFirstBlock >= Size) {
+    uint32_t StreamBlockAddr = BlockList[BlockNum];
+    StringRef Data = Pdb.getBlockData(StreamBlockAddr, Pdb.getBlockSize());
+    ArrayRef<uint8_t> BlockData(Data.bytes_begin(), Data.bytes_end());
+    Buffer = BlockData.slice(OffsetInBlock, Size);
+    return Error::success();
+  }
+
+  auto CacheIter = CacheMap.find(Offset);
+  if (CacheIter != CacheMap.end()) {
+    // In a more general solution, we would need to guarantee that the
+    // cached allocation is at least the requested size.  In practice, since
+    // these are CodeView / PDB records, we know they are always formatted
+    // the same way and never change, so we should never be requesting two
+    // allocations from the same address with different sizes.
+    Buffer = ArrayRef<uint8_t>(CacheIter->second, Size);
+    return Error::success();
+  }
+
+  // Otherwise allocate a large enough buffer in the pool, memcpy the data
+  // into it, and return an ArrayRef to that.
+  // TODO: A possible option might be to cache the pool allocations as a
+  // set of (offset, size) so that if we get the same request again (very
+  // likely since iteration patterns are almost always exactly the same)
+  // we don't have to allocate the same memory a second time.
+  AllocSize += Size;
+  uint8_t *WriteBuffer = Pool.Allocate<uint8_t>(Size);
+
+  if (auto EC = readBytes(Offset, MutableArrayRef<uint8_t>(WriteBuffer, Size)))
+    return EC;
+  CacheMap.insert(std::make_pair(Offset, WriteBuffer));
+  Buffer = ArrayRef<uint8_t>(WriteBuffer, Size);
+  return Error::success();
+}
+
 Error MappedBlockStream::readBytes(uint32_t Offset,
                                    MutableArrayRef<uint8_t> Buffer) const {
   uint32_t BlockNum = Offset / Pdb.getBlockSize();
   uint32_t OffsetInBlock = Offset % Pdb.getBlockSize();
 
   // Make sure we aren't trying to read beyond the end of the stream.
   if (Buffer.size() > StreamLength)
     return make_error<RawError>(raw_error_code::insufficient_buffer);
while (BytesLeft > 0) {
 
     BytesWritten += BytesInChunk;
     BytesLeft -= BytesInChunk;
     ++BlockNum;
     OffsetInBlock = 0;
   }
 
   return Error::success();
-}
 
-Error MappedBlockStream::getArrayRef(uint32_t Offset, ArrayRef<uint8_t> &Buffer,
-                                     uint32_t Length) const {
-  uint32_t BlockNum = Offset / Pdb.getBlockSize();
-  uint32_t OffsetInBlock = Offset % Pdb.getBlockSize();
-  uint32_t BytesAvailableInBlock = Pdb.getBlockSize() - OffsetInBlock;
-
-  // If this is the last block in the stream, not all of the data is valid.
-  if (BlockNum == BlockList.size() - 1) {
-    uint32_t AllocatedBytesInBlock = StreamLength % Pdb.getBlockSize();
-    if (AllocatedBytesInBlock < BytesAvailableInBlock)
-      BytesAvailableInBlock = AllocatedBytesInBlock;
-  }
-  if (BytesAvailableInBlock < Length)
-    return make_error<RawError>(raw_error_code::feature_unsupported);
-
-  uint32_t StreamBlockAddr = BlockList[BlockNum];
-  StringRef Data = Pdb.getBlockData(StreamBlockAddr, Pdb.getBlockSize());
-  Data = Data.substr(OffsetInBlock, Length);
-
-  Buffer = ArrayRef<uint8_t>(Data.bytes_begin(), Data.bytes_end());
-  return Error::success();
 }

lib/DebugInfo/PDB/Raw/ModInfo.cpp

 //===- ModInfo.cpp - PDB module information -------------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/DebugInfo/PDB/Raw/ModInfo.h"
+
+#include "llvm/DebugInfo/CodeView/StreamReader.h"
 #include "llvm/DebugInfo/PDB/Raw/PDBFile.h"
 #include "llvm/Support/Endian.h"
 
 using namespace llvm;
 using namespace llvm::pdb;
 using namespace llvm::support;
 
 namespace {
+
 struct SCBytes {
   ulittle16_t Section;
   char Padding1[2];
   little32_t Offset;
   little32_t Size;
   ulittle32_t Characteristics;
   ulittle16_t ModuleIndex;
   char Padding2[2];
struct ModInfo::FileLayout {
   ulittle16_t NumFiles;      // Number of files contributing to this module
   char Padding1[2];          // Padding so the next field is 4-byte aligned.
   ulittle32_t FileNameOffs;  // array of [0..NumFiles) DBI name buffer offsets.
                              // This field is a pointer in the reference
                              // implementation, but as with `Mod`, we ignore it
                              // for now since it is unused.
   ulittle32_t SrcFileNameNI; // Name Index for src file name
   ulittle32_t PdbFilePathNI; // Name Index for path to compiler PDB
-  char VarInfo[1];           // Module name followed by Obj File Name
-
-  StringRef getModuleName() const { return StringRef(VarInfo); }
+                             // Null terminated Module name
+                             // Null terminated Obj File Name
+};
 
-  StringRef getObjectFileName() const {
-    return StringRef(getModuleName().end() + 1);
-  }
-};
+ModInfo::ModInfo(codeview::StreamRef Stream) : Layout(nullptr) {
+  codeview::StreamReader Reader(Stream);
+  if (auto EC = Reader.readObject(Layout)) {
+    consumeError(std::move(EC));
+    return;
+  }
+  if (auto EC = Reader.readZeroString(ModuleName)) {
+    consumeError(std::move(EC));
+    return;
+  }
+  if (auto EC = Reader.readZeroString(ObjFileName)) {
+    consumeError(std::move(EC));
+    return;
+  }
 
-ModInfo::ModInfo(const uint8_t *Bytes)
-    : Layout(reinterpret_cast<const FileLayout *>(Bytes)) {}
+ModInfo::ModInfo(const ModInfo &Info)
+    : Layout(Info.Layout), ModuleName(Info.ModuleName),
+      ObjFileName(Info.ObjFileName) {}
 
 ModInfo::~ModInfo() {}
 
 bool ModInfo::hasECInfo() const { return (Layout->Flags & HasECFlagMask) != 0; }
 
 uint16_t ModInfo::getTypeServerIndex() const {
   return (Layout->Flags & TypeServerIndexMask) >> TypeServerIndexShift;
 }

 uint32_t ModInfo::getSourceFileNameIndex() const {
   return Layout->SrcFileNameNI;
 }
 
 uint32_t ModInfo::getPdbFilePathNameIndex() const {
   return Layout->PdbFilePathNI;
 }
 
-llvm::StringRef ModInfo::getModuleName() const {
-  return Layout->getModuleName();
-}
-
-llvm::StringRef ModInfo::getObjFileName() const {
-  return Layout->getObjectFileName();
-}
-
-ModInfoIterator::ModInfoIterator(const uint8_t *Stream) : Bytes(Stream) {}
-
-ModInfoIterator::ModInfoIterator(const ModInfoIterator &Other)
-    : Bytes(Other.Bytes) {}
-
-ModInfo ModInfoIterator::operator*() { return ModInfo(Bytes); }
-
-ModInfoIterator &ModInfoIterator::operator++() {
-  StringRef Obj = ModInfo(Bytes).getObjFileName();
-  Bytes = Obj.bytes_end() + 1;
-  Bytes = reinterpret_cast<const uint8_t *>(llvm::alignAddr(Bytes, 4));
-
-  return *this;
-}
+StringRef ModInfo::getModuleName() const { return ModuleName; }
 
-ModInfoIterator ModInfoIterator::operator++(int) {
-  ModInfoIterator Copy(*this);
-  ++(*this);
-  return Copy;
-}
-
-bool ModInfoIterator::operator==(const ModInfoIterator &Other) {
-  return Bytes == Other.Bytes;
-}
-
-bool ModInfoIterator::operator!=(const ModInfoIterator &Other) {
-  return !(*this == Other);
-}
+StringRef ModInfo::getObjFileName() const { return ObjFileName; }
 
-ModInfoIterator &ModInfoIterator::operator=(const ModInfoIterator &Other) {
-  Bytes = Other.Bytes;
-  return *this;
+uint32_t ModInfo::getRecordLength() const {
+  uint32_t M = ModuleName.str().size() + 1;
+  uint32_t O = ObjFileName.str().size() + 1;
+  uint32_t Size = sizeof(FileLayout) + M + O;
+  Size = llvm::alignTo(Size, 4);
+  return Size;
 }

lib/DebugInfo/PDB/Raw/ModStream.cpp

Error ModStream::reload() {
   uint32_t SymbolSize = Mod.getSymbolDebugInfoByteSize();
   uint32_t C11Size = Mod.getLineInfoByteSize();
   uint32_t C13Size = Mod.getC13LineInfoByteSize();
 
   if (C11Size > 0 && C13Size > 0)
     return llvm::make_error<RawError>(raw_error_code::corrupt_file,
                                       "Module has both C11 and C13 line info");
 
-  if (auto EC = SymbolsSubstream.initialize(Reader, SymbolSize))
+  if (auto EC = SymbolsSubstream.load(Reader, SymbolSize))
     return EC;
-  if (auto EC = LinesSubstream.initialize(Reader, C11Size))
+  if (auto EC = Reader.readStreamRef(LinesSubstream, C11Size))
     return EC;
-  if (auto EC = C13LinesSubstream.initialize(Reader, C13Size))
+  if (auto EC = Reader.readStreamRef(C13LinesSubstream, C13Size))
     return EC;
 
   uint32_t GlobalRefsSize;
   if (auto EC = Reader.readInteger(GlobalRefsSize))
     return EC;
-  if (auto EC = GlobalRefsSubstream.initialize(Reader, GlobalRefsSize))
+  if (auto EC = Reader.readStreamRef(GlobalRefsSubstream, GlobalRefsSize))
     return EC;
   if (Reader.bytesRemaining() > 0)
     return llvm::make_error<RawError>(raw_error_code::corrupt_file,
                                       "Unexpected bytes in module stream.");
 
   return Error::success();
 }
 
 iterator_range<codeview::SymbolIterator> ModStream::symbols() const {
   return codeview::makeSymbolRange(SymbolsSubstream.data().slice(4), nullptr);
 }

lib/DebugInfo/PDB/Raw/NameHashTable.cpp

 
 Error NameHashTable::load(codeview::StreamReader &Stream) {
   struct Header {
     support::ulittle32_t Signature;
     support::ulittle32_t HashVersion;
     support::ulittle32_t ByteSize;
   };
 
-  Header H;
-  if (auto EC = Stream.readObject(&H))
+  const Header *H;
+  if (auto EC = Stream.readObject(H))
     return EC;
 
-  if (H.Signature != 0xEFFEEFFE)
+  if (H->Signature != 0xEFFEEFFE)
     return make_error<RawError>(raw_error_code::corrupt_file,
                                 "Invalid hash table signature");
-  if (H.HashVersion != 1 && H.HashVersion != 2)
+  if (H->HashVersion != 1 && H->HashVersion != 2)
     return make_error<RawError>(raw_error_code::corrupt_file,
                                 "Unsupported hash version");
 
-  Signature = H.Signature;
-  HashVersion = H.HashVersion;
-  if (auto EC = NamesBuffer.initialize(Stream, H.ByteSize))
+  Signature = H->Signature;
+  HashVersion = H->HashVersion;
+  if (auto EC = Stream.readFixedString(NamesBuffer, H->ByteSize))
     return make_error<RawError>(raw_error_code::corrupt_file,
                                 "Invalid hash table byte length");
 
-  support::ulittle32_t HashCount;
-  if (auto EC = Stream.readObject(&HashCount))
+  const support::ulittle32_t *HashCount;
+  if (auto EC = Stream.readObject(HashCount))
     return EC;
 
-  std::vector<support::ulittle32_t> BucketArray(HashCount);
+  std::vector<support::ulittle32_t> BucketArray(*HashCount);
   if (auto EC = Stream.readArray<support::ulittle32_t>(BucketArray))
     return make_error<RawError>(raw_error_code::corrupt_file,
                                 "Could not read bucket array");
   IDs.assign(BucketArray.begin(), BucketArray.end());
 
   if (Stream.bytesRemaining() < sizeof(support::ulittle32_t))
     return make_error<RawError>(raw_error_code::corrupt_file,
                                 "Missing name count");
 
   if (auto EC = Stream.readInteger(NameCount))
     return EC;
   return Error::success();
 }
 
 StringRef NameHashTable::getStringForID(uint32_t ID) const {
   if (ID == IDs[0])
     return StringRef();
 
-  return StringRef(NamesBuffer.str().begin() + ID);
+  // NamesBuffer is a buffer of null terminated strings back to back.  ID is
+  // the starting offset of the string we're looking for.  We can't just use
+  // drop_front because that would include all the other subsequent strings
+  // so we have to explicitly use the constructor of StringRef that takes a
+  // const char* so it finds the null terminator.
+  return StringRef(NamesBuffer.drop_front(ID).data());
 }
 
 uint32_t NameHashTable::getIDForString(StringRef Str) const {
   uint32_t Hash = (HashVersion == 1) ? HashStringV1(Str) : HashStringV2(Str);
   size_t Count = IDs.size();
   uint32_t Start = Hash % Count;
   for (size_t I = 0; I < Count; ++I) {
     // The hash is just a starting point for the search, but if it

lib/DebugInfo/PDB/Raw/NameMap.cpp

for (uint32_t I = 0; I < MaxNumberOfStrings; ++I) {
     if (Stream.readInteger(NameIndex))
       return make_error<RawError>(raw_error_code::corrupt_file,
                                   "Expected name map name index");
 
     // Compute the offset of the start of the string relative to the stream.
     uint32_t StringOffset = StringsOffset + NameOffset;
     uint32_t OldOffset = Stream.getOffset();
     // Pump out our c-string from the stream.
-    std::string Str;
+    StringRef Str;
     Stream.setOffset(StringOffset);
     if (Stream.readZeroString(Str))
       return make_error<RawError>(raw_error_code::corrupt_file,
                                   "Expected name map name");
 
     Stream.setOffset(OldOffset);
     // Add this to a string-map from name to stream number.
     Mapping.insert({Str, NameIndex});

lib/DebugInfo/PDB/Raw/PublicsStream.cpp

Error PublicsStream::reload() {
   codeview::StreamReader Reader(Stream);
 
   // Check stream size.
   if (Reader.bytesRemaining() < sizeof(HeaderInfo) + sizeof(GSIHashHeader))
     return make_error<RawError>(raw_error_code::corrupt_file,
                                 "Publics Stream does not contain a header.");
 
   // Read PSGSIHDR and GSIHashHdr structs.
-  Header.reset(new HeaderInfo());
-  if (Reader.readObject(Header.get()))
+  if (Reader.readObject(Header))
     return make_error<RawError>(raw_error_code::corrupt_file,
                                 "Publics Stream does not contain a header.");
-  HashHdr.reset(new GSIHashHeader());
-  if (Reader.readObject(HashHdr.get()))
+
+  if (Reader.readObject(HashHdr))
     return make_error<RawError>(raw_error_code::corrupt_file,
                                 "Publics Stream does not contain a header.");
 
   // An array of HashRecord follows. Read them.
   if (HashHdr->HrSize % sizeof(HashRecord))
     return make_error<RawError>(raw_error_code::corrupt_file,
                                 "Invalid HR array size.");
   HashRecords.resize(HashHdr->HrSize / sizeof(HashRecord));

lib/DebugInfo/PDB/Raw/SymbolStream.cpp

 SymbolStream::SymbolStream(PDBFile &File, uint32_t StreamNum)
     : MappedStream(StreamNum, File) {}
 
 SymbolStream::~SymbolStream() {}
 
 Error SymbolStream::reload() {
   codeview::StreamReader Reader(MappedStream);
 
-  if (Stream.initialize(Reader, MappedStream.getLength()))
+  if (Stream.load(Reader, MappedStream.getLength()))
     return make_error<RawError>(raw_error_code::corrupt_file,
                                 "Could not load symbol stream.");
 
   return Error::success();
 }
 
 iterator_range<codeview::SymbolIterator> SymbolStream::getSymbols() const {
   using codeview::SymbolIterator;
   ArrayRef<uint8_t> Data;
-  if (auto Error = Stream.getArrayRef(0, Data, Stream.getLength())) {
+  if (auto Error = Stream.readBytes(0, Stream.getLength(), Data)) {
     consumeError(std::move(Error));
     return iterator_range<SymbolIterator>(SymbolIterator(), SymbolIterator());
   }
 
   return codeview::makeSymbolRange(Data, nullptr);
 }

lib/DebugInfo/PDB/Raw/TpiStream.cpp

 
 Error TpiStream::reload() {
   codeview::StreamReader Reader(Stream);
 
   if (Reader.bytesRemaining() < sizeof(HeaderInfo))
     return make_error<RawError>(raw_error_code::corrupt_file,
                                 "TPI Stream does not contain a header.");
 
-  Header.reset(new HeaderInfo());
-  if (Reader.readObject(Header.get()))
+  if (Reader.readObject(Header))
     return make_error<RawError>(raw_error_code::corrupt_file,
                                 "TPI Stream does not contain a header.");
 
   if (Header->Version != PdbTpiV80)
     return make_error<RawError>(raw_error_code::corrupt_file,
                                 "Unsupported TPI Version.");
 
   if (Header->HeaderSize != sizeof(HeaderInfo))
     return make_error<RawError>(raw_error_code::corrupt_file,
                                 "Corrupt TPI Header size.");
 
   if (Header->HashKeySize != sizeof(ulittle32_t))
     return make_error<RawError>(raw_error_code::corrupt_file,
                                 "TPI Stream expected 4 byte hash key size.");
 
   if (Header->NumHashBuckets < MinHashBuckets ||
       Header->NumHashBuckets > MaxHashBuckets)
     return make_error<RawError>(raw_error_code::corrupt_file,
                                 "TPI Stream Invalid number of hash buckets.");
 
   HashFunction = HashBufferV8;
 
   // The actual type records themselves come from this stream
-  if (auto EC = RecordsBuffer.initialize(Reader, Header->TypeRecordBytes))
+  if (auto EC = RecordsBuffer.load(Reader, Header->TypeRecordBytes))
     return EC;
 
   // Hash indices, hash values, etc come from the hash stream.
   MappedBlockStream HS(Header->HashStreamIndex, Pdb);
   codeview::StreamReader HSR(HS);
   HSR.setOffset(Header->HashValueBuffer.Off);
   if (auto EC =
-          HashValuesBuffer.initialize(HSR, Header->HashValueBuffer.Length))
+          HSR.readStreamRef(HashValuesBuffer, Header->HashValueBuffer.Length))
     return EC;
 
   HSR.setOffset(Header->HashAdjBuffer.Off);
-  if (auto EC = HashAdjBuffer.initialize(HSR, Header->HashAdjBuffer.Length))
+  if (auto EC = HSR.readStreamRef(HashAdjBuffer, Header->HashAdjBuffer.Length))
     return EC;
 
   HSR.setOffset(Header->IndexOffsetBuffer.Off);
-  if (auto EC = TypeIndexOffsetBuffer.initialize(
-          HSR, Header->IndexOffsetBuffer.Length))
+  if (auto EC = HSR.readStreamRef(TypeIndexOffsetBuffer,
+                                  Header->IndexOffsetBuffer.Length))
     return EC;
 
   return Error::success();
 }
 
 PdbRaw_TpiVer TpiStream::getTpiVersion() const {
   uint32_t Value = Header->Version;
   return static_cast<PdbRaw_TpiVer>(Value);

tools/llvm-pdbdump/llvm-pdbdump.cpp

else if (StreamIdx == DS.getDebugStreamIndex(DbgHeaderType::TokenRidMap))
       Value = "Token Rid Data";
     else if (StreamIdx == DS.getDebugStreamIndex(DbgHeaderType::Xdata))
       Value = "Xdata";
     else {
       auto ModIter = ModStreams.find(StreamIdx);
       auto NSIter = NamedStreams.find(StreamIdx);
       if (ModIter != ModStreams.end()) {
         Value = "Module \"";
-        Value += ModIter->second->Info.getModuleName();
+        Value += ModIter->second->Info.getModuleName().str();
         Value += "\"";
       } else if (NSIter != NamedStreams.end()) {
         Value = "Named Stream \"";
         Value += NSIter->second;
         Value += "\"";
       } else {
         Value = "???";
       }
if (DumpStreamStr.getAsInteger(/*Radix=*/0U, DumpStreamNum) ||
     return Error::success();
 
   MappedBlockStream S(DumpStreamNum, File);
   codeview::StreamReader R(S);
   while (R.bytesRemaining() > 0) {
     ArrayRef<uint8_t> Data;
     uint32_t BytesToReadInBlock = std::min(
         R.bytesRemaining(), static_cast<uint32_t>(File.getBlockSize()));
-    if (auto EC = R.getArrayRef(Data, BytesToReadInBlock))
+    if (auto EC = R.readBytes(BytesToReadInBlock, Data))
       return EC;
     P.printBinaryBlock(
         "Data",
         StringRef(reinterpret_cast<const char *>(Data.begin()), Data.size()));
   }
   return Error::success();
 }
 
static Error dumpDbiStream(ScopedPrinter &P, PDBFile &File,
   DllStream << "mspdb" << Major << Minor << ".dll version";
   DllStream.flush();
   P.printVersion(DllName, Major, Minor, DS.getPdbDllVersion());
 
   if (DumpModules) {
     ListScope L(P, "Modules");
     for (auto &Modi : DS.modules()) {
       DictScope DD(P);
-      P.printString("Name", Modi.Info.getModuleName());
+      P.printString("Name", Modi.Info.getModuleName().str());
       P.printNumber("Debug Stream Index", Modi.Info.getModuleStreamIndex());
-      P.printString("Object File Name", Modi.Info.getObjFileName());
+      P.printString("Object File Name", Modi.Info.getObjFileName().str());
       P.printNumber("Num Files", Modi.Info.getNumberOfFiles());
       P.printNumber("Source File Name Idx", Modi.Info.getSourceFileNameIndex());
       P.printNumber("Pdb File Name Idx", Modi.Info.getPdbFilePathNameIndex());
       P.printNumber("Line Info Byte Size", Modi.Info.getLineInfoByteSize());
       P.printNumber("C13 Line Info Byte Size",
                     Modi.Info.getC13LineInfoByteSize());
       P.printNumber("Symbol Byte Size", Modi.Info.getSymbolDebugInfoByteSize());
       P.printNumber("Type Server Index", Modi.Info.getTypeServerIndex());
       P.printBoolean("Has EC Info", Modi.Info.hasECInfo());
       if (opts::DumpModuleFiles) {
         std::string FileListName =
             to_string(Modi.SourceFiles.size()) + " Contributing Source Files";
         ListScope LL(P, FileListName);
         for (auto File : Modi.SourceFiles)
-          P.printString(File);
+          P.printString(File.str());
       }
       bool HasModuleDI =
           (Modi.Info.getModuleStreamIndex() < File.getNumStreams());
       bool ShouldDumpSymbols =
           (opts::DumpModuleSyms || opts::DumpSymRecordBytes);
       if (HasModuleDI && ShouldDumpSymbols) {
         ListScope SS(P, "Symbols");
         ModStream ModS(File, Modi.Info);