More strongly separate the PDB reading interfaces and PDB writing interfaces

[zturner]

Summary

This is a huge patch, so instead of asking you guys to look at it in detail, I'll summarize the highlights (I also have 7 other smaller patches leading up to this which are not in yet. Point being that I tried as hard as I could to break it down into something smaller, but this is the best I could do). Mostly looking for high level feedback. Even this is long, but it's better than reading a 5,000 line patch :)

Problem: A separation started to occur between reading interfaces and writing interfaces. E.g. PDBFile, DbiStream, InfoStream, etc were the reading interfaces, and PDBFileBuilder, MsfBuilder, InfoStreamBuilder, etc were the writing interfaces. But due to the coupling of the reading interfaces and the underlying file buffer, the only way to write changes back out to disk was to do so through the reading interface. So we had functions like PDBFile::commit(), and DbiStream::commit(), even though these were supposed to be the reading interfaces. The way you would use the api is to create a PdbFileBuilder, fill out the fields, call PdbFileBuilder::build() on it and that would return you a PDBFile. Then you would call commit() on it.

So the problem was to get these commit() methods over to the builders, so that the reading interfaces would be truly read only.

Solution: Until this patch, there was no clear distinction between interfaces which could read and interfaces which could write. Everything could read and everything could write. This causes a problem though, because your backing data is not always writable to begin with. For example, when reading a PDB from disk and dumping it, we read it into a MemoryBuffer, which is immutable. So we would have to use hacks like returning an llvm::Error if the backing store is not writable. It's sort of analagous to if everything was a MutableArrayRef<>, but methods returned an error if the underlying memory was const. Ideally you want the types to dictate what operations are possible.

With that in mind, I split all the stream based types into readable streams, writable streams, and read/write streams. This is supported by the following fundamental types:

/// An abstract base class which defines an interface for reading data from an arbitrary backing store which may or may not be contiguous.
class ReadableStream

/// An abstract base class which defines an interface for writing data to an arbitrary backing store which may or may not be contiguous.
class WritableStream

/// A stream which is backed by a source which is both readable and writable.
/// This is mostly equivalent to what was formerly known as `StreamInterface`.
class ReadWriteStream : public ReadableStream, public WritableStream

These are the basis for everything. From here there are concrete implementations that are backed by contiguous in-memory byte sequences (ByteStream and MutableByteStream) which are useful when you are building your own streams, or when you have a single buffer which contains code view type data, etc.

There are concrete implementations that are backed by discontiguous block streams in an Msf file (MappedBlockStream and WritableMappedBlockStream).

There are concrete implementations that are backed by disk files (FileBufferByteStream) that are useful when you want to commit to a disk.

But it's not always useful to just have an entire stream. Sometimes you just want part of a stream. For example, maybe starting at offset 364 in a big stream is a "substream". We want some way to get a smaller view over a larger stream. For this we have ReadableStreamRef, WritableStreamRef, and ReadWriteStreamRef. StreamRefs are to Streams as ArrayRefs are to arrays. They can be trivially copied and passed by value, and provide convenience methods for dropping items, slicing, etc. But they are backed by ReadableStream, WritableStream, and ReadWriteStream, which again means they can refer to any kind of stream, whether it be a ByteStream, a MappedBlockStream, or any other kind of stream.

Finally, we have StreamReader and StreamWriter. While the underlying stream only knows how to read and write byte sequences, StreamReader and StreamWriter know how to read *types*. integers, objects, strings, etc. They wrap ReadableStreamRef and WritableStreamRef respectively, and maintain an offset into the stream where the next read or write will occur at.

these abstractions largely mirror the abstractions of before this patch, with the exception that reading and writing responsibilities are enforced by the type system.

There is one breaking change to be aware of between this patch and the previous patch. Previously, MappedBlockStream (which supported both reading and writing at the time) handled the case where you read across a block boundary by allocating from an internal pool. And if you ever requested that offset again, it would return the value allocated from the pool. Since it also supported writing, it had to handle the case where you overwrote some of the values that were previously cached from the pool. Because otherwise, if someone were to try to read that value again, they would see it was cached and return the stale value from the cache. So MappedBlockStream was smart enough to deal with this.

But now, MappedBlockStream is read-only and WritableMappedBlockStream is write-only, and just like if you make 2 unique_ptrs to the same data, bad things will happen. The only use case I had found for this advanced cache coherency was in simplicity of writing tests, so it is now up to the user to make sure he/she coordinates reading and writing appropriately and explicitly clears the read cache after any write is performed on the same backing stream.

[zturner]

The breaking change I mentioned from before is no longer an issue. In addition, this patch of makes things a little bit more sane. Previously it was possible to have a write-only interface. Now all writable interfaces are also readable. This mimics the way MutableArrayRef / ArrayRef and other similar classes in LLVM work. Because of this, WritableMappedBlockStream can both read and write, and it now has to deal with the same issue that MappedBlockStream used to have to deal with. While it adds some that complexity back, it has been working and well tested, and removes the possibility of having strange errors pop up that are difficult to figure out. (Plus it makes tests cleaner)

[amccarth]

At a high level, this seems reasonable to me. At an API level, I find it surprising that the write interfaces are const.

include/llvm/DebugInfo/Msf/StreamInterface.h

45 ↗

(On Diff #65144)

It's surprising that the writeBytes method is const. I understand why this works, but as a user of the API, it's surprising.

lib/DebugInfo/Msf/MappedBlockStream.cpp

191 ↗

(On Diff #65144)

Is this correct? If Size is two blocks and BytesFromFirstBlock is less than BlockSize, then we'll get two additional blocks instead of one.

[majnemer]

lib/DebugInfo/Msf/MappedBlockStream.cpp

191 ↗

(On Diff #65144)

Perhaps:

uint32_t NumAdditionalBlocks = (llvm::alignTo(Size, BlockSize) / BlockSize) - 1;

?

[zturner]

lib/DebugInfo/Msf/MappedBlockStream.cpp

191 ↗

(On Diff #65144)

Are you sure? Here's a concrete example:

// Starting from the beginning of block 2, read 2 full blocks of data.
BlockSize = 4096
Offset = 8192
Size = 8192

  // BlockNum = 8192 / 4096 = 2;
  uint32_t BlockNum = Offset / BlockSize;

  // OffsetInBlock = 8192 % 4096 = 0;
  uint32_t OffsetInBlock = Offset % BlockSize;

  // BytesFromFirstBlock = min(8192, 4096 - 0) = 4096
  uint32_t BytesFromFirstBlock = std::min(Size, BlockSize - OffsetInBlock);

  // NumAdditionalBlocks = alignTo(8192 - 4096, 4096) / 4096 = 4096 / 4096 = 1
  uint32_t NumAdditionalBlocks =
      llvm::alignTo(Size - BytesFromFirstBlock, BlockSize) / BlockSize;

[amccarth]

lib/DebugInfo/Msf/MappedBlockStream.cpp

191 ↗

(On Diff #65144)

I didn't understand what BytesFromFirstBlock was measuring. Now that I do, I think the original code is correct.

[zturner]

Just to clarify, here's a diagram:

      BytesFromFirstBlock
--------=============----------------------------------------------------------------------------------------------------------
|       x            |                    |                    |                    |         y          |                    |
-------------------------------------------------------------------------------------------------------------------------------
                     |                     NumAdditionalBlocks                                           |

BytesFromFirstBlock is therefore equal to BlockSize - OffsetInBlock, and `NumAdditionalBlocks is however many bytes are left over, rounded up to the block size, divided by the block size.

[ruiu]

At a high level, it looks good to me, but as I probably mentioned earlier, I have a little concern that we are having an API for PDB that is too complete at this moment. The COFF linker is 7000 lines long in total, and if we keep this pace, we'll probably end up having a PDB library which is larger than the linker. Even if we'd eventually need it, I'm more comfortable if we do as-needed basis.

llvm/trunk/include/llvm/DebugInfo/CodeView/CVRecord.h

template <typename Kind> struct CVRecord {
   ArrayRef<uint8_t> Data;
   ArrayRef<uint8_t> RawData;
 };
 }
 namespace msf {
 
 template <typename Kind>
 struct VarStreamArrayExtractor<codeview::CVRecord<Kind>> {
-  Error operator()(StreamRef Stream, uint32_t &Len,
+  Error operator()(ReadableStreamRef Stream, uint32_t &Len,
                    codeview::CVRecord<Kind> &Item) const {
     using namespace codeview;
     const RecordPrefix *Prefix = nullptr;
     StreamReader Reader(Stream);
     uint32_t Offset = Reader.getOffset();
 
     if (auto EC = Reader.readObject(Prefix))
       return EC;

llvm/trunk/include/llvm/DebugInfo/CodeView/ModuleSubstream.h

 struct ColumnNumberEntry {
   support::ulittle16_t StartColumn;
   support::ulittle16_t EndColumn;
 };
 
 class ModuleSubstream {
 public:
   ModuleSubstream();
-  ModuleSubstream(ModuleSubstreamKind Kind, msf::StreamRef Data);
-  static Error initialize(msf::StreamRef Stream, ModuleSubstream &Info);
+  ModuleSubstream(ModuleSubstreamKind Kind, msf::ReadableStreamRef Data);
+  static Error initialize(msf::ReadableStreamRef Stream, ModuleSubstream &Info);
   uint32_t getRecordLength() const;
   ModuleSubstreamKind getSubstreamKind() const;
-  msf::StreamRef getRecordData() const;
+  msf::ReadableStreamRef getRecordData() const;
 
 private:
   ModuleSubstreamKind Kind;
-  msf::StreamRef Data;
+  msf::ReadableStreamRef Data;
 };
 
 typedef msf::VarStreamArray<ModuleSubstream> ModuleSubstreamArray;
 } // namespace codeview
 
 namespace msf {
 template <> struct VarStreamArrayExtractor<codeview::ModuleSubstream> {
-  Error operator()(StreamRef Stream, uint32_t &Length,
+  Error operator()(ReadableStreamRef Stream, uint32_t &Length,
                    codeview::ModuleSubstream &Info) const {
     if (auto EC = codeview::ModuleSubstream::initialize(Stream, Info))
       return EC;
     Length = Info.getRecordLength();
     return Error::success();
   }
 };
 } // namespace msf
 } // namespace llvm
 
 #endif // LLVM_DEBUGINFO_CODEVIEW_MODULESUBSTREAM_H

llvm/trunk/include/llvm/DebugInfo/CodeView/ModuleSubstreamVisitor.h

 
 typedef msf::VarStreamArray<LineColumnEntry> LineInfoArray;
 typedef msf::VarStreamArray<FileChecksumEntry> FileChecksumArray;
 
 class IModuleSubstreamVisitor {
 public:
   virtual ~IModuleSubstreamVisitor() {}
 
-  virtual Error visitUnknown(ModuleSubstreamKind Kind, msf::StreamRef Data) = 0;
-  virtual Error visitSymbols(msf::StreamRef Data);
-  virtual Error visitLines(msf::StreamRef Data,
+  virtual Error visitUnknown(ModuleSubstreamKind Kind,
+                             msf::ReadableStreamRef Data) = 0;
+  virtual Error visitSymbols(msf::ReadableStreamRef Data);
+  virtual Error visitLines(msf::ReadableStreamRef Data,
                            const LineSubstreamHeader *Header,
                            const LineInfoArray &Lines);
-  virtual Error visitStringTable(msf::StreamRef Data);
-  virtual Error visitFileChecksums(msf::StreamRef Data,
+  virtual Error visitStringTable(msf::ReadableStreamRef Data);
+  virtual Error visitFileChecksums(msf::ReadableStreamRef Data,
                                    const FileChecksumArray &Checksums);
-  virtual Error visitFrameData(msf::StreamRef Data);
-  virtual Error visitInlineeLines(msf::StreamRef Data);
-  virtual Error visitCrossScopeImports(msf::StreamRef Data);
-  virtual Error visitCrossScopeExports(msf::StreamRef Data);
-  virtual Error visitILLines(msf::StreamRef Data);
-  virtual Error visitFuncMDTokenMap(msf::StreamRef Data);
-  virtual Error visitTypeMDTokenMap(msf::StreamRef Data);
-  virtual Error visitMergedAssemblyInput(msf::StreamRef Data);
-  virtual Error visitCoffSymbolRVA(msf::StreamRef Data);
+  virtual Error visitFrameData(msf::ReadableStreamRef Data);
+  virtual Error visitInlineeLines(msf::ReadableStreamRef Data);
+  virtual Error visitCrossScopeImports(msf::ReadableStreamRef Data);
+  virtual Error visitCrossScopeExports(msf::ReadableStreamRef Data);
+  virtual Error visitILLines(msf::ReadableStreamRef Data);
+  virtual Error visitFuncMDTokenMap(msf::ReadableStreamRef Data);
+  virtual Error visitTypeMDTokenMap(msf::ReadableStreamRef Data);
+  virtual Error visitMergedAssemblyInput(msf::ReadableStreamRef Data);
+  virtual Error visitCoffSymbolRVA(msf::ReadableStreamRef Data);
 };
 
 Error visitModuleSubstream(const ModuleSubstream &R,
                            IModuleSubstreamVisitor &V);
 } // namespace codeview
 
 namespace msf {
 template <> class VarStreamArrayExtractor<codeview::LineColumnEntry> {
 public:
   VarStreamArrayExtractor(const codeview::LineSubstreamHeader *Header)
       : Header(Header) {}
 
-  Error operator()(StreamRef Stream, uint32_t &Len,
+  Error operator()(ReadableStreamRef Stream, uint32_t &Len,
                    codeview::LineColumnEntry &Item) const {
     using namespace codeview;
     const LineFileBlockHeader *BlockHeader;
     StreamReader Reader(Stream);
     if (auto EC = Reader.readObject(BlockHeader))
       return EC;
     bool HasColumn = Header->Flags & LineFlags::HaveColumns;
     uint32_t LineInfoSize =
public:
   }
 
 private:
   const codeview::LineSubstreamHeader *Header;
 };
 
 template <> class VarStreamArrayExtractor<codeview::FileChecksumEntry> {
 public:
-  Error operator()(StreamRef Stream, uint32_t &Len,
+  Error operator()(ReadableStreamRef Stream, uint32_t &Len,
                    codeview::FileChecksumEntry &Item) const {
     using namespace codeview;
     const FileChecksum *Header;
     StreamReader Reader(Stream);
     if (auto EC = Reader.readObject(Header))
       return EC;
     Item.FileNameOffset = Header->FileNameOffset;
     Item.Kind = static_cast<FileChecksumKind>(Header->ChecksumKind);

llvm/trunk/include/llvm/DebugInfo/Msf/ByteStream.h

 //===- ByteStream.h - Reads stream data from a byte sequence ----*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_DEBUGINFO_MSF_BYTESTREAM_H
 #define LLVM_DEBUGINFO_MSF_BYTESTREAM_H
 
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/StringRef.h"
+#include "llvm/DebugInfo/Msf/MsfError.h"
 #include "llvm/DebugInfo/Msf/StreamInterface.h"
 #include "llvm/Support/Error.h"
+#include "llvm/Support/FileOutputBuffer.h"
+#include "llvm/Support/MemoryBuffer.h"
 #include <cstdint>
 #include <memory>
 #include <type_traits>
 
 namespace llvm {
 namespace msf {
-class StreamReader;
-
-template <bool Writable = false> class ByteStream : public StreamInterface {
-  typedef typename std::conditional<Writable, MutableArrayRef<uint8_t>,
-                                    ArrayRef<uint8_t>>::type ArrayType;
 
+class ByteStream : public ReadableStream {
 public:
   ByteStream() {}
-  explicit ByteStream(ArrayType Data) : Data(Data) {}
-  ~ByteStream() override {}
+  explicit ByteStream(ArrayRef<uint8_t> Data) : Data(Data) {}
 
   Error readBytes(uint32_t Offset, uint32_t Size,
-                  ArrayRef<uint8_t> &Buffer) const override;
+                  ArrayRef<uint8_t> &Buffer) const override {
+    if (Offset > Data.size())
+      return make_error<MsfError>(msf_error_code::insufficient_buffer);
+    if (Data.size() < Size + Offset)
+      return make_error<MsfError>(msf_error_code::insufficient_buffer);
+    Buffer = Data.slice(Offset, Size);
+    return Error::success();
+  }
   Error readLongestContiguousChunk(uint32_t Offset,
-                                   ArrayRef<uint8_t> &Buffer) const override;
+                                   ArrayRef<uint8_t> &Buffer) const override {
+    if (Offset >= Data.size())
+      return make_error<MsfError>(msf_error_code::insufficient_buffer);
+    Buffer = Data.slice(Offset);
+    return Error::success();
+  }
 
-  Error writeBytes(uint32_t Offset, ArrayRef<uint8_t> Buffer) const override;
+  uint32_t getLength() const override { return Data.size(); }
 
-  uint32_t getLength() const override;
+  ArrayRef<uint8_t> data() const { return Data; }
 
-  Error commit() const override;
+  StringRef str() const {
+    const char *CharData = reinterpret_cast<const char *>(Data.data());
+    return StringRef(CharData, Data.size());
+  }
 
-  ArrayRef<uint8_t> data() const { return Data; }
-  StringRef str() const;
+protected:
+  ArrayRef<uint8_t> Data;
+};
+
+// MemoryBufferByteStream behaves like a read-only ByteStream, but has its data
+// backed by an llvm::MemoryBuffer.  It also owns the underlying MemoryBuffer.
+class MemoryBufferByteStream : public ByteStream {
+public:
+  explicit MemoryBufferByteStream(std::unique_ptr<MemoryBuffer> Buffer)
+      : ByteStream(ArrayRef<uint8_t>(Buffer->getBuffer().bytes_begin(),
+                                     Buffer->getBuffer().bytes_end())),
+        MemBuffer(std::move(Buffer)) {}
+
+  std::unique_ptr<MemoryBuffer> MemBuffer;
+};
+
+class MutableByteStream : public WritableStream {
+public:
+  MutableByteStream() {}
+  explicit MutableByteStream(MutableArrayRef<uint8_t> Data)
+      : Data(Data), ImmutableStream(Data) {}
+
+  Error readBytes(uint32_t Offset, uint32_t Size,
+                  ArrayRef<uint8_t> &Buffer) const override {
+    return ImmutableStream.readBytes(Offset, Size, Buffer);
+  }
+  Error readLongestContiguousChunk(uint32_t Offset,
+                                   ArrayRef<uint8_t> &Buffer) const override {
+    return ImmutableStream.readLongestContiguousChunk(Offset, Buffer);
+  }
+
+  uint32_t getLength() const override { return ImmutableStream.getLength(); }
+
+  Error writeBytes(uint32_t Offset, ArrayRef<uint8_t> Buffer) const override {
+    if (Data.size() < Buffer.size())
+      return make_error<MsfError>(msf_error_code::insufficient_buffer);
+    if (Offset > Buffer.size() - Data.size())
+      return make_error<MsfError>(msf_error_code::insufficient_buffer);
+
+    uint8_t *DataPtr = const_cast<uint8_t *>(Data.data());
+    ::memcpy(DataPtr + Offset, Buffer.data(), Buffer.size());
+    return Error::success();
+  }
+
+  Error commit() const override { return Error::success(); }
+
+  MutableArrayRef<uint8_t> data() const { return Data; }
+
+private:
+  MutableArrayRef<uint8_t> Data;
+  ByteStream ImmutableStream;
+};
+
+// A simple adapter that acts like a ByteStream but holds ownership over
+// and underlying FileOutputBuffer.
+class FileBufferByteStream : public WritableStream {
+private:
+  class StreamImpl : public MutableByteStream {
+  public:
+    StreamImpl(std::unique_ptr<FileOutputBuffer> Buffer)
+        : MutableByteStream(MutableArrayRef<uint8_t>(Buffer->getBufferStart(),
+                                                     Buffer->getBufferEnd())),
+          FileBuffer(std::move(Buffer)) {}
+
+    Error commit() const override {
+      if (FileBuffer->commit())
+        return llvm::make_error<MsfError>(msf_error_code::not_writable);
+      return Error::success();
+    }
+
+  private:
+    std::unique_ptr<FileOutputBuffer> FileBuffer;
+  };
+
+public:
+  explicit FileBufferByteStream(std::unique_ptr<FileOutputBuffer> Buffer)
+      : Impl(std::move(Buffer)) {}
+
+  Error readBytes(uint32_t Offset, uint32_t Size,
+                  ArrayRef<uint8_t> &Buffer) const override {
+    return Impl.readBytes(Offset, Size, Buffer);
+  }
+  Error readLongestContiguousChunk(uint32_t Offset,
+                                   ArrayRef<uint8_t> &Buffer) const override {
+    return Impl.readLongestContiguousChunk(Offset, Buffer);
+  }
+
+  uint32_t getLength() const override { return Impl.getLength(); }
+
+  Error writeBytes(uint32_t Offset, ArrayRef<uint8_t> Data) const override {
+    return Impl.writeBytes(Offset, Data);
+  }
+  Error commit() const override { return Impl.commit(); }
 
 private:
-  ArrayType Data;
+  StreamImpl Impl;
 };
 
-extern template class ByteStream<true>;
-extern template class ByteStream<false>;
 
 } // end namespace msf
 } // end namespace llvm
 
 #endif // LLVM_DEBUGINFO_MSF_BYTESTREAM_H

llvm/trunk/include/llvm/DebugInfo/Msf/MappedBlockStream.h

 #include "llvm/Support/Endian.h"
 #include "llvm/Support/Error.h"
 #include <cstdint>
 #include <vector>
 
 namespace llvm {
 namespace msf {
 
-class IMsfFile;
+struct MsfLayout;
 
 /// MappedBlockStream represents data stored in an Msf file into chunks of a
 /// particular size (called the Block Size), and whose chunks may not be
 /// necessarily contiguous.  The arrangement of these chunks within the file
 /// is described by some other metadata contained within the Msf file.  In
 /// the case of a standard Msf Stream, the layout of the stream's blocks
 /// is described by the Msf "directory", but in the case of the directory
 /// itself, the layout is described by an array at a fixed location within
 /// the Msf.  MappedBlockStream provides methods for reading from and writing
 /// to one of these streams transparently, as if it were a contiguous sequence
 /// of bytes.
-class MappedBlockStream : public StreamInterface {
+class MappedBlockStream : public ReadableStream {
+  friend class WritableMappedBlockStream;
+
 public:
+  static std::unique_ptr<MappedBlockStream>
+  createStream(uint32_t BlockSize, uint32_t NumBlocks,
+               const MsfStreamLayout &Layout, const ReadableStream &MsfData);
+
+  static std::unique_ptr<MappedBlockStream>
+  createIndexedStream(const MsfLayout &Layout, const ReadableStream &MsfData,
+                      uint32_t StreamIndex);
+
+  static std::unique_ptr<MappedBlockStream>
+  createDirectoryStream(const MsfLayout &Layout, const ReadableStream &MsfData);
+
   Error readBytes(uint32_t Offset, uint32_t Size,
                   ArrayRef<uint8_t> &Buffer) const override;
   Error readLongestContiguousChunk(uint32_t Offset,
                                    ArrayRef<uint8_t> &Buffer) const override;
-  Error writeBytes(uint32_t Offset, ArrayRef<uint8_t> Buffer) const override;
 
   uint32_t getLength() const override;
-  Error commit() const override;
 
   uint32_t getNumBytesCopied() const;
 
-  static Expected<std::unique_ptr<MappedBlockStream>>
-  createIndexedStream(uint32_t StreamIdx, const IMsfFile &File);
-  static Expected<std::unique_ptr<MappedBlockStream>>
-  createDirectoryStream(uint32_t Length, ArrayRef<support::ulittle32_t> Blocks,
-                        const IMsfFile &File);
-
   llvm::BumpPtrAllocator &getAllocator() { return Pool; }
 
+  void invalidateCache();
+
+  uint32_t getBlockSize() const { return BlockSize; }
+  uint32_t getNumBlocks() const { return NumBlocks; }
+  uint32_t getStreamLength() const { return StreamLayout.Length; }
+
 protected:
-  MappedBlockStream(const MsfStreamLayout &Layout, const IMsfFile &File);
+  MappedBlockStream(uint32_t BlockSize, uint32_t NumBlocks,
+                    const MsfStreamLayout &StreamLayout,
+                    const ReadableStream &MsfData);
+
+private:
+  const MsfStreamLayout &getStreamLayout() const { return StreamLayout; }
+  void fixCacheAfterWrite(uint32_t Offset, ArrayRef<uint8_t> Data) const;
 
   Error readBytes(uint32_t Offset, MutableArrayRef<uint8_t> Buffer) const;
   bool tryReadContiguously(uint32_t Offset, uint32_t Size,
                            ArrayRef<uint8_t> &Buffer) const;
 
-  const IMsfFile &Msf;
-  const MsfStreamLayout Layout;
+  const uint32_t BlockSize;
+  const uint32_t NumBlocks;
+  const MsfStreamLayout StreamLayout;
+  const ReadableStream &MsfData;
 
   typedef MutableArrayRef<uint8_t> CacheEntry;
   mutable llvm::BumpPtrAllocator Pool;
   mutable DenseMap<uint32_t, std::vector<CacheEntry>> CacheMap;
 };
 
+class WritableMappedBlockStream : public WritableStream {
+public:
+  static std::unique_ptr<WritableMappedBlockStream>
+  createStream(uint32_t BlockSize, uint32_t NumBlocks,
+               const MsfStreamLayout &Layout, const WritableStream &MsfData);
+
+  static std::unique_ptr<WritableMappedBlockStream>
+  createIndexedStream(const MsfLayout &Layout, const WritableStream &MsfData,
+                      uint32_t StreamIndex);
+
+  static std::unique_ptr<WritableMappedBlockStream>
+  createDirectoryStream(const MsfLayout &Layout, const WritableStream &MsfData);
+
+  Error readBytes(uint32_t Offset, uint32_t Size,
+                  ArrayRef<uint8_t> &Buffer) const override;
+  Error readLongestContiguousChunk(uint32_t Offset,
+                                   ArrayRef<uint8_t> &Buffer) const override;
+  uint32_t getLength() const override;
+
+  Error writeBytes(uint32_t Offset, ArrayRef<uint8_t> Buffer) const override;
+
+  Error commit() const override;
+
+  const MsfStreamLayout &getStreamLayout() const {
+    return ReadInterface.getStreamLayout();
+  }
+  uint32_t getBlockSize() const { return ReadInterface.getBlockSize(); }
+  uint32_t getNumBlocks() const { return ReadInterface.getNumBlocks(); }
+  uint32_t getStreamLength() const { return ReadInterface.getStreamLength(); }
+
+protected:
+  WritableMappedBlockStream(uint32_t BlockSize, uint32_t NumBlocks,
+                            const MsfStreamLayout &StreamLayout,
+                            const WritableStream &MsfData);
+
+private:
+  MappedBlockStream ReadInterface;
+
+  const WritableStream &WriteInterface;
+};
+
 } // end namespace pdb
 } // end namespace llvm
 
 #endif // LLVM_DEBUGINFO_MSF_MAPPEDBLOCKSTREAM_H

llvm/trunk/include/llvm/DebugInfo/Msf/MsfBuilder.h

public:
   /// to `getNumUsedBlocks() + getNumFreeBlocks()`.
   uint32_t getTotalBlockCount() const;
 
   /// Check whether a particular block is allocated or free.
   bool isBlockFree(uint32_t Idx) const;
 
   /// Finalize the layout and build the headers and structures that describe the
   /// MSF layout and can be written directly to the MSF file.
-  Expected<Layout> build();
+  Expected<MsfLayout> build();
 
 private:
   MsfBuilder(uint32_t BlockSize, uint32_t MinBlockCount, bool CanGrow,
              BumpPtrAllocator &Allocator);
 
   Error allocateBlocks(uint32_t NumBlocks, MutableArrayRef<uint32_t> Blocks);
   uint32_t computeDirectoryByteSize() const;
 

llvm/trunk/include/llvm/DebugInfo/Msf/MsfCommon.h

struct SuperBlock {
   // This contains the number of bytes which make up the directory.
   support::ulittle32_t NumDirectoryBytes;
   // This field's purpose is not yet known.
   support::ulittle32_t Unknown1;
   // This contains the block # of the block map.
   support::ulittle32_t BlockMapAddr;
 };
 
-struct Layout {
-  Layout() : SB(nullptr) {}
+struct MsfLayout {
+  MsfLayout() : SB(nullptr) {}
   const SuperBlock *SB;
   ArrayRef<support::ulittle32_t> DirectoryBlocks;
   ArrayRef<support::ulittle32_t> StreamSizes;
   std::vector<ArrayRef<support::ulittle32_t>> StreamMap;
 };
 
 inline bool isValidBlockSize(uint32_t Size) {
   switch (Size) {

llvm/trunk/include/llvm/DebugInfo/Msf/StreamArray.h

 /// Item appropriately to represent the current record.
 ///
 /// You can specialize this template for your own custom value types to avoid
 /// having to specify a second template argument to VarStreamArray (documented
 /// below).
 template <typename T> struct VarStreamArrayExtractor {
   // Method intentionally deleted.  You must provide an explicit specialization
   // with the following method implemented.
-  Error operator()(StreamRef Stream, uint32_t &Len, T &Item) const = delete;
+  Error operator()(ReadableStreamRef Stream, uint32_t &Len,
+                   T &Item) const = delete;
 };
 
 /// VarStreamArray represents an array of variable length records backed by a
 /// stream.  This could be a contiguous sequence of bytes in memory, it could
 /// be a file on disk, or it could be a PDB stream where bytes are stored as
 /// discontiguous blocks in a file.  Usually it is desirable to treat arrays
 /// as contiguous blocks of memory, but doing so with large PDB files, for
 /// example, could mean allocating huge amounts of memory just to allow
class VarStreamArray {
   friend class VarStreamArrayIterator<ValueType, Extractor>;
 
 public:
   typedef VarStreamArrayIterator<ValueType, Extractor> Iterator;
 
   VarStreamArray() {}
   explicit VarStreamArray(const Extractor &E) : E(E) {}
 
-  explicit VarStreamArray(StreamRef Stream) : Stream(Stream) {}
-  VarStreamArray(StreamRef Stream, const Extractor &E) : Stream(Stream), E(E) {}
+  explicit VarStreamArray(ReadableStreamRef Stream) : Stream(Stream) {}
+  VarStreamArray(ReadableStreamRef Stream, const Extractor &E)
+      : Stream(Stream), E(E) {}
 
   VarStreamArray(const VarStreamArray<ValueType, Extractor> &Other)
       : Stream(Other.Stream), E(Other.E) {}
 
   Iterator begin(bool *HadError = nullptr) const {
     return Iterator(*this, E, HadError);
   }
 
   Iterator end() const { return Iterator(E); }
 
   const Extractor &getExtractor() const { return E; }
 
-  StreamRef getUnderlyingStream() const { return Stream; }
+  ReadableStreamRef getUnderlyingStream() const { return Stream; }
 
 private:
-  StreamRef Stream;
+  ReadableStreamRef Stream;
   Extractor E;
 };
 
 template <typename ValueType, typename Extractor> class VarStreamArrayIterator {
   typedef VarStreamArrayIterator<ValueType, Extractor> IterType;
   typedef VarStreamArray<ValueType, Extractor> ArrayType;
 
 public:
private:
   void markError() {
     moveToEnd();
     HasError = true;
     if (HadError != nullptr)
       *HadError = true;
   }
 
   ValueType ThisValue;
-  StreamRef IterRef;
+  ReadableStreamRef IterRef;
   const ArrayType *Array{nullptr};
   uint32_t ThisLen{0};
   bool HasError{false};
   bool *HadError{nullptr};
   Extractor Extract;
 };
 
 template <typename T> class FixedStreamArrayIterator;
 
 template <typename T> class FixedStreamArray {
   friend class FixedStreamArrayIterator<T>;
 
 public:
   FixedStreamArray() : Stream() {}
-  FixedStreamArray(StreamRef Stream) : Stream(Stream) {
+  FixedStreamArray(ReadableStreamRef Stream) : Stream(Stream) {
     assert(Stream.getLength() % sizeof(T) == 0);
   }
 
   const T &operator[](uint32_t Index) const {
     assert(Index < size());
     uint32_t Off = Index * sizeof(T);
     ArrayRef<uint8_t> Data;
     if (auto EC = Stream.readBytes(Off, sizeof(T), Data)) {
public:
 
   FixedStreamArrayIterator<T> begin() const {
     return FixedStreamArrayIterator<T>(*this, 0);
   }
   FixedStreamArrayIterator<T> end() const {
     return FixedStreamArrayIterator<T>(*this, size());
   }
 
-  StreamRef getUnderlyingStream() const { return Stream; }
+  ReadableStreamRef getUnderlyingStream() const { return Stream; }
 
 private:
-  StreamRef Stream;
+  ReadableStreamRef Stream;
 };
 
 template <typename T> class FixedStreamArrayIterator {
 public:
   FixedStreamArrayIterator(const FixedStreamArray<T> &Array, uint32_t Index)
       : Array(Array), Index(Index) {}
 
   bool operator==(const FixedStreamArrayIterator<T> &R) {

llvm/trunk/include/llvm/DebugInfo/Msf/StreamInterface.h

 
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/Support/Error.h"
 #include <cstdint>
 
 namespace llvm {
 namespace msf {
 
-/// StreamInterface abstracts the notion of a data stream.  This way, an
-/// implementation could implement trivial reading from a contiguous memory
-/// buffer or, as in the case of PDB files, reading from a set of possibly
-/// discontiguous blocks.  The implementation is required to return references
-/// to stable memory, so if this is not possible (for example in the case of
-/// a PDB file with discontiguous blocks, it must keep its own pool of temp
-/// storage.
-class StreamInterface {
+class ReadableStream {
 public:
-  virtual ~StreamInterface() {}
+  virtual ~ReadableStream() {}
 
   // Given an offset into the stream and a number of bytes, attempt to read
   // the bytes and set the output ArrayRef to point to a reference into the
   // stream, without copying any data.
   virtual Error readBytes(uint32_t Offset, uint32_t Size,
                           ArrayRef<uint8_t> &Buffer) const = 0;
 
   // Given an offset into the stream, read as much as possible without copying
   // any data.
   virtual Error readLongestContiguousChunk(uint32_t Offset,
                                            ArrayRef<uint8_t> &Buffer) const = 0;
 
+  virtual uint32_t getLength() const = 0;
+};
+
+class WritableStream : public ReadableStream {
+public:
+  virtual ~WritableStream() {}
+
   // Attempt to write the given bytes into the stream at the desired offset.
   // This will always necessitate a copy.  Cannot shrink or grow the stream,
   // only writes into existing allocated space.
   virtual Error writeBytes(uint32_t Offset, ArrayRef<uint8_t> Data) const = 0;
 
-  virtual uint32_t getLength() const = 0;
-
   virtual Error commit() const = 0;
 };
 
 } // end namespace msf
 } // end namespace llvm
 
 #endif // LLVM_DEBUGINFO_MSF_STREAMINTERFACE_H

llvm/trunk/include/llvm/DebugInfo/Msf/StreamReader.h

 //===- StreamReader.h - Reads bytes and objects from a stream ---*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_DEBUGINFO_MSF_STREAMREADER_H
 #define LLVM_DEBUGINFO_MSF_STREAMREADER_H
 
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/DebugInfo/Msf/MsfError.h"
 #include "llvm/DebugInfo/Msf/StreamArray.h"
 #include "llvm/DebugInfo/Msf/StreamInterface.h"
+#include "llvm/DebugInfo/Msf/StreamRef.h"
 #include "llvm/Support/Endian.h"
 #include "llvm/Support/Error.h"
 
 #include <string>
 
 namespace llvm {
 namespace msf {
 
-class StreamRef;
-
 class StreamReader {
 public:
-  StreamReader(StreamRef Stream);
+  StreamReader(ReadableStreamRef Stream);
 
   Error readLongestContiguousChunk(ArrayRef<uint8_t> &Buffer);
   Error readBytes(ArrayRef<uint8_t> &Buffer, uint32_t Size);
   Error readInteger(uint16_t &Dest);
   Error readInteger(uint32_t &Dest);
   Error readZeroString(StringRef &Dest);
   Error readFixedString(StringRef &Dest, uint32_t Length);
-  Error readStreamRef(StreamRef &Ref);
-  Error readStreamRef(StreamRef &Ref, uint32_t Length);
+  Error readStreamRef(ReadableStreamRef &Ref);
+  Error readStreamRef(ReadableStreamRef &Ref, uint32_t Length);
 
   template <typename T> Error readEnum(T &Dest) {
     typename std::underlying_type<T>::type N;
     if (auto EC = readInteger(N))
       return EC;
     Dest = static_cast<T>(N);
     return Error::success();
   }
Error readArray(ArrayRef<T> &Array, uint32_t NumElements) {
     if (auto EC = readBytes(Bytes, NumElements * sizeof(T)))
       return EC;
     Array = ArrayRef<T>(reinterpret_cast<const T *>(Bytes.data()), NumElements);
     return Error::success();
   }
 
   template <typename T, typename U>
   Error readArray(VarStreamArray<T, U> &Array, uint32_t Size) {
-    StreamRef S;
+    ReadableStreamRef S;
     if (auto EC = readStreamRef(S, Size))
       return EC;
     Array = VarStreamArray<T, U>(S, Array.getExtractor());
     return Error::success();
   }
 
   template <typename T>
   Error readArray(FixedStreamArray<T> &Array, uint32_t NumItems) {
     if (NumItems == 0) {
       Array = FixedStreamArray<T>();
       return Error::success();
     }
     uint32_t Length = NumItems * sizeof(T);
     if (Length / sizeof(T) != NumItems)
       return make_error<MsfError>(msf_error_code::invalid_format);
     if (Offset + Length > Stream.getLength())
       return make_error<MsfError>(msf_error_code::insufficient_buffer);
-    StreamRef View = Stream.slice(Offset, Length);
+    ReadableStreamRef View = Stream.slice(Offset, Length);
     Array = FixedStreamArray<T>(View);
     Offset += Length;
     return Error::success();
   }
 
   void setOffset(uint32_t Off) { Offset = Off; }
   uint32_t getOffset() const { return Offset; }
   uint32_t getLength() const { return Stream.getLength(); }
   uint32_t bytesRemaining() const { return getLength() - getOffset(); }
 
 private:
-  StreamRef Stream;
+  ReadableStreamRef Stream;
   uint32_t Offset;
 };
 } // namespace msf
 } // namespace llvm
 
 #endif // LLVM_DEBUGINFO_MSF_STREAMREADER_H

llvm/trunk/include/llvm/DebugInfo/Msf/StreamRef.h

 #ifndef LLVM_DEBUGINFO_MSF_STREAMREF_H
 #define LLVM_DEBUGINFO_MSF_STREAMREF_H
 
 #include "llvm/DebugInfo/Msf/MsfError.h"
 #include "llvm/DebugInfo/Msf/StreamInterface.h"
 
 namespace llvm {
 namespace msf {
-
-class StreamRef {
+template <class StreamType, class RefType> class StreamRefBase {
 public:
-  StreamRef() : Stream(nullptr), ViewOffset(0), Length(0) {}
-  StreamRef(const StreamInterface &Stream)
-      : Stream(&Stream), ViewOffset(0), Length(Stream.getLength()) {}
-  StreamRef(const StreamInterface &Stream, uint32_t Offset, uint32_t Length)
+  StreamRefBase() : Stream(nullptr), Length(0), ViewOffset(0) {}
+  StreamRefBase(const StreamType &Stream, uint32_t Offset, uint32_t Length)
       : Stream(&Stream), ViewOffset(Offset), Length(Length) {}
 
+  uint32_t getLength() const { return Length; }
+  const StreamType *getStream() const { return Stream; }
+
+  RefType drop_front(uint32_t N) const {
+    if (!Stream)
+      return RefType();
+
+    N = std::min(N, Length);
+    return RefType(*Stream, ViewOffset + N, Length - N);
+  }
+
+  RefType keep_front(uint32_t N) const {
+    if (!Stream)
+      return RefType();
+    N = std::min(N, Length);
+    return RefType(*Stream, ViewOffset, N);
+  }
+
+  RefType slice(uint32_t Offset, uint32_t Len) const {
+    return drop_front(Offset).keep_front(Len);
+  }
+
+  bool operator==(const RefType &Other) const {
+    if (Stream != Other.Stream)
+      return false;
+    if (ViewOffset != Other.ViewOffset)
+      return false;
+    if (Length != Other.Length)
+      return false;
+    return true;
+  }
+
+protected:
+  const StreamType *Stream;
+  uint32_t ViewOffset;
+  uint32_t Length;
+};
+
+class ReadableStreamRef
+    : public StreamRefBase<ReadableStream, ReadableStreamRef> {
+public:
+  ReadableStreamRef() : StreamRefBase() {}
+  ReadableStreamRef(const ReadableStream &Stream)
+      : StreamRefBase(Stream, 0, Stream.getLength()) {}
+  ReadableStreamRef(const ReadableStream &Stream, uint32_t Offset,
+                    uint32_t Length)
+      : StreamRefBase(Stream, Offset, Length) {}
+
   // Use StreamRef.slice() instead.
-  StreamRef(const StreamRef &S, uint32_t Offset, uint32_t Length) = delete;
+  ReadableStreamRef(const ReadableStreamRef &S, uint32_t Offset,
+                    uint32_t Length) = delete;
 
   Error readBytes(uint32_t Offset, uint32_t Size,
                   ArrayRef<uint8_t> &Buffer) const {
     if (ViewOffset + Offset < Offset)
       return make_error<MsfError>(msf_error_code::insufficient_buffer);
     if (Size + Offset > Length)
       return make_error<MsfError>(msf_error_code::insufficient_buffer);
     return Stream->readBytes(ViewOffset + Offset, Size, Buffer);
Error readLongestContiguousChunk(uint32_t Offset,
     // This StreamRef might refer to a smaller window over a larger stream.  In
     // that case we will have read out more bytes than we should return, because
     // we should not read past the end of the current view.
     uint32_t MaxLength = Length - Offset;
     if (Buffer.size() > MaxLength)
       Buffer = Buffer.slice(0, MaxLength);
     return Error::success();
   }
+};
+
+class WritableStreamRef
+    : public StreamRefBase<WritableStream, WritableStreamRef> {
+public:
+  WritableStreamRef() : StreamRefBase() {}
+  WritableStreamRef(const WritableStream &Stream)
+      : StreamRefBase(Stream, 0, Stream.getLength()) {}
+  WritableStreamRef(const WritableStream &Stream, uint32_t Offset,
+                    uint32_t Length)
+      : StreamRefBase(Stream, Offset, Length) {}
+
+  // Use StreamRef.slice() instead.
+  WritableStreamRef(const WritableStreamRef &S, uint32_t Offset,
+                    uint32_t Length) = delete;
 
   Error writeBytes(uint32_t Offset, ArrayRef<uint8_t> Data) const {
     if (Data.size() + Offset > Length)
       return make_error<MsfError>(msf_error_code::insufficient_buffer);
     return Stream->writeBytes(ViewOffset + Offset, Data);
   }
 
-  uint32_t getLength() const { return Length; }
-
   Error commit() const { return Stream->commit(); }
-
-  StreamRef drop_front(uint32_t N) const {
-    if (!Stream)
-      return StreamRef();
-
-    N = std::min(N, Length);
-    return StreamRef(*Stream, ViewOffset + N, Length - N);
-  }
-
-  StreamRef keep_front(uint32_t N) const {
-    if (!Stream)
-      return StreamRef();
-    N = std::min(N, Length);
-    return StreamRef(*Stream, ViewOffset, N);
-  }
-
-  StreamRef slice(uint32_t Offset, uint32_t Len) const {
-    return drop_front(Offset).keep_front(Len);
-  }
-
-  bool operator==(const StreamRef &Other) const {
-    if (Stream != Other.Stream)
-      return false;
-    if (ViewOffset != Other.ViewOffset)
-      return false;
-    if (Length != Other.Length)
-      return false;
-    return true;
-  }
-
-private:
-  const StreamInterface *Stream;
-  uint32_t ViewOffset;
-  uint32_t Length;
 };
+
 } // namespace msf
 } // namespace llvm
 
 #endif // LLVM_DEBUGINFO_MSF_STREAMREF_H

llvm/trunk/include/llvm/DebugInfo/Msf/StreamWriter.h

 //===- StreamWriter.h - Writes bytes and objects to a stream ----*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_DEBUGINFO_MSF_STREAMWRITER_H
 #define LLVM_DEBUGINFO_MSF_STREAMWRITER_H
 
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/DebugInfo/Msf/MsfError.h"
 #include "llvm/DebugInfo/Msf/StreamArray.h"
 #include "llvm/DebugInfo/Msf/StreamInterface.h"
+#include "llvm/DebugInfo/Msf/StreamRef.h"
 #include "llvm/Support/Endian.h"
 #include "llvm/Support/Error.h"
 
 #include <string>
 
 namespace llvm {
 namespace msf {
 
-class StreamRef;
-
 class StreamWriter {
 public:
-  StreamWriter(StreamRef Stream);
+  StreamWriter(WritableStreamRef Stream);
 
   Error writeBytes(ArrayRef<uint8_t> Buffer);
   Error writeInteger(uint16_t Dest);
   Error writeInteger(uint32_t Dest);
   Error writeZeroString(StringRef Str);
   Error writeFixedString(StringRef Str);
-  Error writeStreamRef(StreamRef Ref);
-  Error writeStreamRef(StreamRef Ref, uint32_t Size);
+  Error writeStreamRef(ReadableStreamRef Ref);
+  Error writeStreamRef(ReadableStreamRef Ref, uint32_t Size);
 
   template <typename T> Error writeEnum(T Num) {
     return writeInteger(
         static_cast<typename std::underlying_type<T>::type>(Num));
   }
 
   template <typename T> Error writeObject(const T &Obj) {
     static_assert(!std::is_pointer<T>::value,
public:
   }
 
   void setOffset(uint32_t Off) { Offset = Off; }
   uint32_t getOffset() const { return Offset; }
   uint32_t getLength() const { return Stream.getLength(); }
   uint32_t bytesRemaining() const { return getLength() - getOffset(); }
 
 private:
-  StreamRef Stream;
+  WritableStreamRef Stream;
   uint32_t Offset;
 };
 } // namespace msf
 } // namespace llvm
 
 #endif // LLVM_DEBUGINFO_MSF_STREAMWRITER_H

llvm/trunk/include/llvm/DebugInfo/PDB/PDBTypes.h

 //===- PDBTypes.h - Defines enums for various fields contained in PDB ---*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_DEBUGINFO_PDB_PDBTYPES_H
 #define LLVM_DEBUGINFO_PDB_PDBTYPES_H
 
 #include "llvm/Config/llvm-config.h"
 #include "llvm/DebugInfo/CodeView/CodeView.h"
-#include <functional>
+#include "llvm/DebugInfo/PDB/Raw/RawTypes.h"
 #include <cstdint>
 #include <cstring>
+#include <functional>
 
 namespace llvm {
 namespace pdb {
 
 class PDBSymDumper;
 class PDBSymbol;
 
 class IPDBDataStream;

 
 /// Specifies which PDB reader implementation is to be used.  Only a value
 /// of PDB_ReaderType::DIA is supported.
 enum class PDB_ReaderType {
   DIA = 0,
   Raw = 1,
 };
 
-/// Defines a 128-bit unique identifier.  This maps to a GUID on Windows, but
-/// is abstracted here for the purposes of non-Windows platforms that don't have
-/// the GUID structure defined.
-struct PDB_UniqueId {
-  char Guid[16];
-};
-
 /// An enumeration indicating the type of data contained in this table.
 enum class PDB_TableType {
   Symbols,
   SourceFiles,
   LineNumbers,
   SectionContribs,
   Segments,
   InjectedSources,

llvm/trunk/include/llvm/DebugInfo/PDB/Raw/DbiStream.h

public:
 
   msf::FixedStreamArray<object::coff_section> getSectionHeaders();
 
   msf::FixedStreamArray<object::FpoData> getFpoRecords();
 
   msf::FixedStreamArray<SecMapEntry> getSectionMap() const;
   void visitSectionContributions(ISectionContribVisitor &Visitor) const;
 
-  Error commit();
-
 private:
   Error initializeModInfoArray();
   Error initializeSectionContributionData();
   Error initializeSectionHeadersData();
   Error initializeSectionMapData();
   Error initializeFileInfo();
   Error initializeFpoRecords();
 
   PDBFile &Pdb;
   std::unique_ptr<msf::MappedBlockStream> Stream;
 
   std::vector<ModuleInfoEx> ModuleInfos;
   NameHashTable ECNames;
 
-  msf::StreamRef ModInfoSubstream;
-  msf::StreamRef SecContrSubstream;
-  msf::StreamRef SecMapSubstream;
-  msf::StreamRef FileInfoSubstream;
-  msf::StreamRef TypeServerMapSubstream;
-  msf::StreamRef ECSubstream;
+  msf::ReadableStreamRef ModInfoSubstream;
+  msf::ReadableStreamRef SecContrSubstream;
+  msf::ReadableStreamRef SecMapSubstream;
+  msf::ReadableStreamRef FileInfoSubstream;
+  msf::ReadableStreamRef TypeServerMapSubstream;
+  msf::ReadableStreamRef ECSubstream;
 
-  msf::StreamRef NamesBuffer;
+  msf::ReadableStreamRef NamesBuffer;
 
   msf::FixedStreamArray<support::ulittle16_t> DbgStreams;
 
   PdbRaw_DbiSecContribVer SectionContribVersion;
   msf::FixedStreamArray<SectionContrib> SectionContribs;
   msf::FixedStreamArray<SectionContrib2> SectionContribs2;
   msf::FixedStreamArray<SecMapEntry> SectionMap;
   msf::FixedStreamArray<support::little32_t> FileNameOffsets;

llvm/trunk/include/llvm/DebugInfo/PDB/Raw/DbiStreamBuilder.h

 #include "llvm/DebugInfo/Msf/StreamReader.h"
 #include "llvm/DebugInfo/PDB/PDBTypes.h"
 #include "llvm/DebugInfo/PDB/Raw/PDBFile.h"
 #include "llvm/DebugInfo/PDB/Raw/RawConstants.h"
 
 namespace llvm {
 namespace pdb {
 class DbiStream;
+struct DbiStreamHeader;
 class PDBFile;
 
 class DbiStreamBuilder {
 public:
   DbiStreamBuilder(BumpPtrAllocator &Allocator);
 
   DbiStreamBuilder(const DbiStreamBuilder &) = delete;
   DbiStreamBuilder &operator=(const DbiStreamBuilder &) = delete;
 
   void setVersionHeader(PdbRaw_DbiVer V);
   void setAge(uint32_t A);
   void setBuildNumber(uint16_t B);
   void setPdbDllVersion(uint16_t V);
   void setPdbDllRbld(uint16_t R);
   void setFlags(uint16_t F);
   void setMachineType(PDB_Machine M);
 
   uint32_t calculateSerializedLength() const;
 
   Error addModuleInfo(StringRef ObjFile, StringRef Module);
   Error addModuleSourceFile(StringRef Module, StringRef File);
 
-  Expected<std::unique_ptr<DbiStream>> build(PDBFile &File);
+  Expected<std::unique_ptr<DbiStream>> build(PDBFile &File,
+                                             const msf::WritableStream &Buffer);
+  Error commit(const msf::MsfLayout &Layout,
+               const msf::WritableStream &Buffer) const;
 
 private:
+  Error finalize();
   uint32_t calculateModiSubstreamSize() const;
   uint32_t calculateFileInfoSubstreamSize() const;
   uint32_t calculateNamesBufferSize() const;
 
   Error generateModiSubstream();
   Error generateFileInfoSubstream();
 
   struct ModuleInfo {
     std::vector<StringRef> SourceFiles;
     StringRef Obj;
     StringRef Mod;
   };
 
   BumpPtrAllocator &Allocator;
 
   Optional<PdbRaw_DbiVer> VerHeader;
   uint32_t Age;
   uint16_t BuildNumber;
   uint16_t PdbDllVersion;
   uint16_t PdbDllRbld;
   uint16_t Flags;
   PDB_Machine MachineType;
 
+  const DbiStreamHeader *Header;
+
   StringMap<std::unique_ptr<ModuleInfo>> ModuleInfos;
   std::vector<ModuleInfo *> ModuleInfoList;
 
   StringMap<uint32_t> SourceFileNames;
 
-  msf::StreamRef NamesBuffer;
-  msf::ByteStream<true> ModInfoBuffer;
-  msf::ByteStream<true> FileInfoBuffer;
+  msf::WritableStreamRef NamesBuffer;
+  msf::MutableByteStream ModInfoBuffer;
+  msf::MutableByteStream FileInfoBuffer;
 };
 }
 }
 
 #endif

llvm/trunk/include/llvm/DebugInfo/PDB/Raw/InfoStream.h

 namespace llvm {
 namespace pdb {
 class InfoStreamBuilder;
 class PDBFile;
 
 class InfoStream {
   friend class InfoStreamBuilder;
 
-  struct HeaderInfo {
-    support::ulittle32_t Version;
-    support::ulittle32_t Signature;
-    support::ulittle32_t Age;
-    PDB_UniqueId Guid;
-  };
-
 public:
   InfoStream(std::unique_ptr<msf::MappedBlockStream> Stream);
 
   Error reload();
-  Error commit();
 
   PdbRaw_ImplVer getVersion() const;
   uint32_t getSignature() const;
   uint32_t getAge() const;
   PDB_UniqueId getGuid() const;
 
   uint32_t getNamedStreamIndex(llvm::StringRef Name) const;
   iterator_range<StringMapConstIterator<uint32_t>> named_streams() const;

llvm/trunk/include/llvm/DebugInfo/PDB/Raw/InfoStreamBuilder.h

 #include "llvm/Support/Error.h"
 
 #include "llvm/DebugInfo/PDB/PDBTypes.h"
 #include "llvm/DebugInfo/PDB/Raw/NameMapBuilder.h"
 #include "llvm/DebugInfo/PDB/Raw/PDBFile.h"
 #include "llvm/DebugInfo/PDB/Raw/RawConstants.h"
 
 namespace llvm {
+namespace msf {
+class StreamWriter;
+}
 namespace pdb {
 class PDBFile;
 
 class InfoStreamBuilder {
 public:
   InfoStreamBuilder();
   InfoStreamBuilder(const InfoStreamBuilder &) = delete;
   InfoStreamBuilder &operator=(const InfoStreamBuilder &) = delete;
 
   void setVersion(PdbRaw_ImplVer V);
   void setSignature(uint32_t S);
   void setAge(uint32_t A);
   void setGuid(PDB_UniqueId G);
 
   NameMapBuilder &getNamedStreamsBuilder();
 
   uint32_t calculateSerializedLength() const;
 
-  Expected<std::unique_ptr<InfoStream>> build(PDBFile &File);
+  Expected<std::unique_ptr<InfoStream>>
+  build(PDBFile &File, const msf::WritableStream &Buffer);
+
+  Error commit(const msf::MsfLayout &Layout,
+               const msf::WritableStream &Buffer) const;
 
 private:
-  Optional<PdbRaw_ImplVer> Ver;
-  Optional<uint32_t> Sig;
-  Optional<uint32_t> Age;
-  Optional<PDB_UniqueId> Guid;
+  PdbRaw_ImplVer Ver;
+  uint32_t Sig;
+  uint32_t Age;
+  PDB_UniqueId Guid;
 
   NameMapBuilder NamedStreams;
 };
 }
 }
 
 #endif

llvm/trunk/include/llvm/DebugInfo/PDB/Raw/ModInfo.h

 class ModInfo {
   friend class DbiStreamBuilder;
 
 public:
   ModInfo();
   ModInfo(const ModInfo &Info);
   ~ModInfo();
 
-  static Error initialize(msf::StreamRef Stream, ModInfo &Info);
+  static Error initialize(msf::ReadableStreamRef Stream, ModInfo &Info);
 
   bool hasECInfo() const;
   uint16_t getTypeServerIndex() const;
   uint16_t getModuleStreamIndex() const;
   uint32_t getSymbolDebugInfoByteSize() const;
   uint32_t getLineInfoByteSize() const;
   uint32_t getC13LineInfoByteSize() const;
   uint32_t getNumberOfFiles() const;
struct ModuleInfoEx {
   ModInfo Info;
   std::vector<StringRef> SourceFiles;
 };
 
 } // end namespace pdb
 
 namespace msf {
 template <> struct VarStreamArrayExtractor<pdb::ModInfo> {
-  Error operator()(StreamRef Stream, uint32_t &Length,
+  Error operator()(ReadableStreamRef Stream, uint32_t &Length,
                    pdb::ModInfo &Info) const {
     if (auto EC = pdb::ModInfo::initialize(Stream, Info))
       return EC;
     Length = Info.getRecordLength();
     return Error::success();
   }
 };
 } // end namespace msf
 
 } // end namespace llvm
 
 #endif // LLVM_DEBUGINFO_PDB_RAW_MODINFO_H

llvm/trunk/include/llvm/DebugInfo/PDB/Raw/ModStream.h

public:
   Error commit();
 
 private:
   const ModInfo &Mod;
 
   std::unique_ptr<msf::MappedBlockStream> Stream;
 
   codeview::CVSymbolArray SymbolsSubstream;
-  msf::StreamRef LinesSubstream;
-  msf::StreamRef C13LinesSubstream;
-  msf::StreamRef GlobalRefsSubstream;
+  msf::ReadableStreamRef LinesSubstream;
+  msf::ReadableStreamRef C13LinesSubstream;
+  msf::ReadableStreamRef GlobalRefsSubstream;
 
   codeview::ModuleSubstreamArray LineInfo;
 };
 }
 }
 
 #endif

llvm/trunk/include/llvm/DebugInfo/PDB/Raw/NameHashTable.h

public:
   uint32_t getSignature() const { return Signature; }
 
   StringRef getStringForID(uint32_t ID) const;
   uint32_t getIDForString(StringRef Str) const;
 
   msf::FixedStreamArray<support::ulittle32_t> name_ids() const;
 
 private:
-  msf::StreamRef NamesBuffer;
+  msf::ReadableStreamRef NamesBuffer;
   msf::FixedStreamArray<support::ulittle32_t> IDs;
   uint32_t Signature;
   uint32_t HashVersion;
   uint32_t NameCount;
 };
 
 } // end namespace pdb
 } // end namespace llvm
 
 #endif // LLVM_DEBUGINFO_PDB_RAW_NAMEHASHTABLE_H

llvm/trunk/include/llvm/DebugInfo/PDB/Raw/NameMap.h

 class NameMapBuilder;
 class NameMap {
   friend NameMapBuilder;
 
 public:
   NameMap();
 
   Error load(msf::StreamReader &Stream);
-  Error commit(msf::StreamWriter &Writer);
 
   bool tryGetValue(StringRef Name, uint32_t &Value) const;
 
   iterator_range<StringMapConstIterator<uint32_t>> entries() const;
 
 private:
   StringMap<uint32_t> Mapping;
 };
 
 } // end namespace pdb
 } // end namespace llvm
 
 #endif // LLVM_DEBUGINFO_PDB_RAW_PDBNAMEMAP_H

llvm/trunk/include/llvm/DebugInfo/PDB/Raw/NameMapBuilder.h

 
 #include "llvm/ADT/StringMap.h"
 #include "llvm/Support/Error.h"
 
 #include <cstdint>
 #include <memory>
 
 namespace llvm {
+namespace msf {
+class StreamWriter;
+}
 namespace pdb {
 class NameMap;
 
 class NameMapBuilder {
 public:
   NameMapBuilder();
 
   void addMapping(StringRef Name, uint32_t Mapping);
 
   Expected<std::unique_ptr<NameMap>> build();
+  Error commit(msf::StreamWriter &Writer) const;
 
   uint32_t calculateSerializedLength() const;
 
 private:
   StringMap<uint32_t> Map;
   uint32_t StringDataBytes = 0;
 };
 
 } // end namespace pdb
 } // end namespace llvm
 
 #endif // LLVM_DEBUGINFO_PDB_RAW_PDBNAMEMAPBUILDER_H

llvm/trunk/include/llvm/DebugInfo/PDB/Raw/PDBFile.h

 #include "llvm/Support/MathExtras.h"
 
 #include <memory>
 
 namespace llvm {
 
 namespace msf {
 class MappedBlockStream;
-class StreamInterface;
+class WritableStream;
 }
 
 namespace pdb {
 class DbiStream;
 class InfoStream;
 class NameHashTable;
 class PDBFileBuilder;
 class PublicsStream;
 class SymbolStream;
 class TpiStream;
 
 class PDBFile : public msf::IMsfFile {
   friend PDBFileBuilder;
 
 public:
-  PDBFile(std::unique_ptr<msf::StreamInterface> PdbFileBuffer,
+  PDBFile(std::unique_ptr<msf::ReadableStream> PdbFileBuffer,
           BumpPtrAllocator &Allocator);
   ~PDBFile() override;
 
   uint32_t getFreeBlockMapBlock() const;
   uint32_t getUnknown1() const;
 
   uint32_t getBlockSize() const override;
   uint32_t getBlockCount() const override;
public:
   uint32_t getFileSize() const;
 
   Expected<ArrayRef<uint8_t>> getBlockData(uint32_t BlockIndex,
                                            uint32_t NumBytes) const override;
   Error setBlockData(uint32_t BlockIndex, uint32_t Offset,
                      ArrayRef<uint8_t> Data) const override;
 
   ArrayRef<support::ulittle32_t> getStreamSizes() const {
-    return MsfLayout.StreamSizes;
+    return ContainerLayout.StreamSizes;
   }
   ArrayRef<ArrayRef<support::ulittle32_t>> getStreamMap() const {
-    return MsfLayout.StreamMap;
+    return ContainerLayout.StreamMap;
   }
 
+  const msf::MsfLayout &getMsfLayout() const { return ContainerLayout; }
+  const msf::ReadableStream &getMsfBuffer() const { return *Buffer; }
+
   ArrayRef<support::ulittle32_t> getDirectoryBlockArray() const;
 
   Error parseFileHeaders();
   Error parseStreamData();
 
   Expected<InfoStream &> getPDBInfoStream();
   Expected<DbiStream &> getPDBDbiStream();
   Expected<TpiStream &> getPDBTpiStream();
   Expected<TpiStream &> getPDBIpiStream();
   Expected<PublicsStream &> getPDBPublicsStream();
   Expected<SymbolStream &> getPDBSymbolStream();
   Expected<NameHashTable &> getStringTable();
 
-  Error commit();
-
 private:
   BumpPtrAllocator &Allocator;
 
-  std::unique_ptr<msf::StreamInterface> Buffer;
+  std::unique_ptr<msf::ReadableStream> Buffer;
 
-  msf::Layout MsfLayout;
+  msf::MsfLayout ContainerLayout;
 
   std::unique_ptr<InfoStream> Info;
   std::unique_ptr<DbiStream> Dbi;
   std::unique_ptr<TpiStream> Tpi;
   std::unique_ptr<TpiStream> Ipi;
   std::unique_ptr<PublicsStream> Publics;
   std::unique_ptr<SymbolStream> Symbols;
   std::unique_ptr<msf::MappedBlockStream> DirectoryStream;
   std::unique_ptr<msf::MappedBlockStream> StringTableStream;
   std::unique_ptr<NameHashTable> StringTable;
 };
 }
 }
 
 #endif

llvm/trunk/include/llvm/DebugInfo/PDB/Raw/PDBFileBuilder.h

 #include "llvm/Support/Error.h"
 
 #include <memory>
 #include <vector>
 
 namespace llvm {
 namespace msf {
 class MsfBuilder;
-class StreamInterface;
 }
 namespace pdb {
 class DbiStreamBuilder;
 class InfoStreamBuilder;
 
 class PDBFileBuilder {
 public:
   explicit PDBFileBuilder(BumpPtrAllocator &Allocator);
   PDBFileBuilder(const PDBFileBuilder &) = delete;
   PDBFileBuilder &operator=(const PDBFileBuilder &) = delete;
 
   Error initialize(const msf::SuperBlock &Super);
 
   msf::MsfBuilder &getMsfBuilder();
   InfoStreamBuilder &getInfoBuilder();
   DbiStreamBuilder &getDbiBuilder();
 
   Expected<std::unique_ptr<PDBFile>>
-  build(std::unique_ptr<msf::StreamInterface> PdbFileBuffer);
+  build(std::unique_ptr<msf::WritableStream> PdbFileBuffer);
+
+  Error commit(const msf::WritableStream &Buffer);
 
 private:
-  Expected<msf::Layout> finalizeMsfLayout() const;
+  Expected<msf::MsfLayout> finalizeMsfLayout() const;
 
   BumpPtrAllocator &Allocator;
 
   std::unique_ptr<msf::MsfBuilder> Msf;
   std::unique_ptr<InfoStreamBuilder> Info;
   std::unique_ptr<DbiStreamBuilder> Dbi;
 };
 }
 }
 
 #endif

llvm/trunk/include/llvm/DebugInfo/PDB/Raw/RawTypes.h

struct ModuleInfoHeader {
   /// Name Index for path to compiler PDB
   support::ulittle32_t PdbFilePathNI;
 
   /// Following this header are two zero terminated strings.
   /// char ModuleName[];
   /// char ObjFileName[];
 };
 
+/// Defines a 128-bit unique identifier.  This maps to a GUID on Windows, but
+/// is abstracted here for the purposes of non-Windows platforms that don't have
+/// the GUID structure defined.
+struct PDB_UniqueId {
+  char Guid[16];
+};
+
+/// The header preceeding the global PDB Stream (Stream 1)
+struct InfoStreamHeader {
+  support::ulittle32_t Version;
+  support::ulittle32_t Signature;
+  support::ulittle32_t Age;
+  PDB_UniqueId Guid;
+};
+
 } // namespace pdb
 } // namespace llvm
 
 #endif

llvm/trunk/lib/CodeGen/AsmPrinter/CodeViewDebug.cpp

TypeTable.ForEachRecord(
           OS.emitRawComment(
               CommentOS.str().drop_front(CommentPrefix.size() - 1).rtrim());
         } else {
 #ifndef NDEBUG
           // Assert that the type data is valid even if we aren't dumping
           // comments. The MSVC linker doesn't do much type record validation,
           // so the first link of an invalid type record can succeed while
           // subsequent links will fail with LNK1285.
-          ByteStream<> Stream({Record.bytes_begin(), Record.bytes_end()});
+          ByteStream Stream({Record.bytes_begin(), Record.bytes_end()});
           CVTypeArray Types;
           StreamReader Reader(Stream);
           Error E = Reader.readArray(Types, Reader.getLength());
           if (!E) {
             TypeVisitorCallbacks C;
             E = CVTypeVisitor(C).visitTypeStream(Types);
           }
           if (E) {

llvm/trunk/lib/DebugInfo/CodeView/ModuleSubstream.cpp

 #include "llvm/DebugInfo/Msf/StreamReader.h"
 
 using namespace llvm;
 using namespace llvm::codeview;
 using namespace llvm::msf;
 
 ModuleSubstream::ModuleSubstream() : Kind(ModuleSubstreamKind::None) {}
 
-ModuleSubstream::ModuleSubstream(ModuleSubstreamKind Kind, StreamRef Data)
+ModuleSubstream::ModuleSubstream(ModuleSubstreamKind Kind,
+                                 ReadableStreamRef Data)
     : Kind(Kind), Data(Data) {}
 
-Error ModuleSubstream::initialize(StreamRef Stream, ModuleSubstream &Info) {
+Error ModuleSubstream::initialize(ReadableStreamRef Stream,
+                                  ModuleSubstream &Info) {
   const ModuleSubsectionHeader *Header;
   StreamReader Reader(Stream);
   if (auto EC = Reader.readObject(Header))
     return EC;
 
   ModuleSubstreamKind Kind =
       static_cast<ModuleSubstreamKind>(uint32_t(Header->Kind));
   if (auto EC = Reader.readStreamRef(Info.Data, Header->Length))
     return EC;
   Info.Kind = Kind;
   return Error::success();
 }
 
 uint32_t ModuleSubstream::getRecordLength() const {
   return sizeof(ModuleSubsectionHeader) + Data.getLength();
 }
 
 ModuleSubstreamKind ModuleSubstream::getSubstreamKind() const { return Kind; }
 
-StreamRef ModuleSubstream::getRecordData() const { return Data; }
+ReadableStreamRef ModuleSubstream::getRecordData() const { return Data; }

llvm/trunk/lib/DebugInfo/CodeView/ModuleSubstreamVisitor.cpp

 #include "llvm/DebugInfo/CodeView/ModuleSubstreamVisitor.h"
 #include "llvm/DebugInfo/Msf/StreamReader.h"
 #include "llvm/DebugInfo/Msf/StreamRef.h"
 
 using namespace llvm;
 using namespace llvm::codeview;
 using namespace llvm::msf;
 
-Error IModuleSubstreamVisitor::visitSymbols(StreamRef Data) {
+Error IModuleSubstreamVisitor::visitSymbols(ReadableStreamRef Data) {
   return visitUnknown(ModuleSubstreamKind::Symbols, Data);
 }
-Error IModuleSubstreamVisitor::visitLines(StreamRef Data,
+Error IModuleSubstreamVisitor::visitLines(ReadableStreamRef Data,
                                           const LineSubstreamHeader *Header,
                                           const LineInfoArray &Lines) {
   return visitUnknown(ModuleSubstreamKind::Lines, Data);
 }
-Error IModuleSubstreamVisitor::visitStringTable(StreamRef Data) {
+Error IModuleSubstreamVisitor::visitStringTable(ReadableStreamRef Data) {
   return visitUnknown(ModuleSubstreamKind::StringTable, Data);
 }
 Error IModuleSubstreamVisitor::visitFileChecksums(
-    StreamRef Data, const FileChecksumArray &Checksums) {
+    ReadableStreamRef Data, const FileChecksumArray &Checksums) {
   return visitUnknown(ModuleSubstreamKind::FileChecksums, Data);
 }
-Error IModuleSubstreamVisitor::visitFrameData(StreamRef Data) {
+Error IModuleSubstreamVisitor::visitFrameData(ReadableStreamRef Data) {
   return visitUnknown(ModuleSubstreamKind::FrameData, Data);
 }
-Error IModuleSubstreamVisitor::visitInlineeLines(StreamRef Data) {
+Error IModuleSubstreamVisitor::visitInlineeLines(ReadableStreamRef Data) {
   return visitUnknown(ModuleSubstreamKind::InlineeLines, Data);
 }
-Error IModuleSubstreamVisitor::visitCrossScopeImports(StreamRef Data) {
+Error IModuleSubstreamVisitor::visitCrossScopeImports(ReadableStreamRef Data) {
   return visitUnknown(ModuleSubstreamKind::CrossScopeExports, Data);
 }
-Error IModuleSubstreamVisitor::visitCrossScopeExports(StreamRef Data) {
+Error IModuleSubstreamVisitor::visitCrossScopeExports(ReadableStreamRef Data) {
   return visitUnknown(ModuleSubstreamKind::CrossScopeImports, Data);
 }
-Error IModuleSubstreamVisitor::visitILLines(StreamRef Data) {
+Error IModuleSubstreamVisitor::visitILLines(ReadableStreamRef Data) {
   return visitUnknown(ModuleSubstreamKind::ILLines, Data);
 }
-Error IModuleSubstreamVisitor::visitFuncMDTokenMap(StreamRef Data) {
+Error IModuleSubstreamVisitor::visitFuncMDTokenMap(ReadableStreamRef Data) {
   return visitUnknown(ModuleSubstreamKind::FuncMDTokenMap, Data);
 }
-Error IModuleSubstreamVisitor::visitTypeMDTokenMap(StreamRef Data) {
+Error IModuleSubstreamVisitor::visitTypeMDTokenMap(ReadableStreamRef Data) {
   return visitUnknown(ModuleSubstreamKind::TypeMDTokenMap, Data);
 }
-Error IModuleSubstreamVisitor::visitMergedAssemblyInput(StreamRef Data) {
+Error IModuleSubstreamVisitor::visitMergedAssemblyInput(
+    ReadableStreamRef Data) {
   return visitUnknown(ModuleSubstreamKind::MergedAssemblyInput, Data);
 }
-Error IModuleSubstreamVisitor::visitCoffSymbolRVA(StreamRef Data) {
+Error IModuleSubstreamVisitor::visitCoffSymbolRVA(ReadableStreamRef Data) {
   return visitUnknown(ModuleSubstreamKind::CoffSymbolRVA, Data);
 }
 
 Error llvm::codeview::visitModuleSubstream(const ModuleSubstream &R,
                                            IModuleSubstreamVisitor &V) {
   switch (R.getSubstreamKind()) {
   case ModuleSubstreamKind::Symbols:
     return V.visitSymbols(R.getRecordData());

llvm/trunk/lib/DebugInfo/CodeView/TypeDumper.cpp

Error CVTypeDumper::dump(const CVTypeArray &Types) {
   assert(W && "printer should not be null");
   CVTypeVisitor Visitor(*this);
   if (auto EC = Visitor.visitTypeStream(Types))
     return EC;
   return Error::success();
 }
 
 Error CVTypeDumper::dump(ArrayRef<uint8_t> Data) {
-  msf::ByteStream<> Stream(Data);
+  msf::ByteStream Stream(Data);
   CVTypeArray Types;
   msf::StreamReader Reader(Stream);
   if (auto EC = Reader.readArray(Types, Reader.getLength()))
     return EC;
 
   return dump(Types);
 }
 
 void CVTypeDumper::setPrinter(ScopedPrinter *P) {
   static ScopedPrinter NullP(llvm::nulls());
   W = P ? P : &NullP;
 }

llvm/trunk/lib/DebugInfo/Msf/ByteStream.cpp

-//===- ByteStream.cpp - Reads stream data from a byte sequence ------------===//
-//
-//                     The LLVM Compiler Infrastructure
-//
-// This file is distributed under the University of Illinois Open Source
-// License. See LICENSE.TXT for details.
-//
-//===----------------------------------------------------------------------===//
-
-#include "llvm/DebugInfo/Msf/ByteStream.h"
-#include "llvm/DebugInfo/Msf/MsfError.h"
-#include "llvm/DebugInfo/Msf/StreamReader.h"
-#include <cstring>
-
-using namespace llvm;
-using namespace llvm::msf;
-
-static Error writeBytes(uint32_t Offset, ArrayRef<uint8_t> Src,
-                        ArrayRef<uint8_t> Dest) {
-  return make_error<MsfError>(msf_error_code::not_writable,
-                              "ByteStream is immutable.");
-}
-
-static Error writeBytes(uint32_t Offset, ArrayRef<uint8_t> Src,
-                        MutableArrayRef<uint8_t> Dest) {
-  if (Dest.size() < Src.size())
-    return make_error<MsfError>(msf_error_code::insufficient_buffer);
-  if (Offset > Src.size() - Dest.size())
-    return make_error<MsfError>(msf_error_code::insufficient_buffer);
-
-  ::memcpy(Dest.data() + Offset, Src.data(), Src.size());
-  return Error::success();
-}
-
-template <bool Writable>
-Error ByteStream<Writable>::readBytes(uint32_t Offset, uint32_t Size,
-                                      ArrayRef<uint8_t> &Buffer) const {
-  if (Offset > Data.size())
-    return make_error<MsfError>(msf_error_code::insufficient_buffer);
-  if (Data.size() < Size + Offset)
-    return make_error<MsfError>(msf_error_code::insufficient_buffer);
-  Buffer = Data.slice(Offset, Size);
-  return Error::success();
-}
-
-template <bool Writable>
-Error ByteStream<Writable>::readLongestContiguousChunk(
-    uint32_t Offset, ArrayRef<uint8_t> &Buffer) const {
-  if (Offset >= Data.size())
-    return make_error<MsfError>(msf_error_code::insufficient_buffer);
-  Buffer = Data.slice(Offset);
-  return Error::success();
-}
-
-template <bool Writable>
-Error ByteStream<Writable>::writeBytes(uint32_t Offset,
-                                       ArrayRef<uint8_t> Buffer) const {
-  return ::writeBytes(Offset, Buffer, Data);
-}
-
-template <bool Writable> uint32_t ByteStream<Writable>::getLength() const {
-  return Data.size();
-}
-
-template <bool Writable> Error ByteStream<Writable>::commit() const {
-  return Error::success();
-}
-
-template <bool Writable> StringRef ByteStream<Writable>::str() const {
-  const char *CharData = reinterpret_cast<const char *>(Data.data());
-  return StringRef(CharData, Data.size());
-}
-
-namespace llvm {
-namespace msf {
-template class ByteStream<true>;
-template class ByteStream<false>;
-}
-}

llvm/trunk/lib/DebugInfo/Msf/CMakeLists.txt

 add_llvm_library(LLVMDebugInfoMsf
-  ByteStream.cpp
   MappedBlockStream.cpp
   MsfBuilder.cpp
   MsfCommon.cpp
   MsfError.cpp
   StreamReader.cpp
   StreamWriter.cpp
   ADDITIONAL_HEADER_DIRS
   "${LLVM_MAIN_INCLUDE_DIR}/llvm/DebugInfo/MSF"
   )

llvm/trunk/lib/DebugInfo/Msf/MappedBlockStream.cpp

 //===- MappedBlockStream.cpp - Reads stream data from an MSF file ---------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/DebugInfo/Msf/MappedBlockStream.h"
 
 #include "llvm/DebugInfo/Msf/IMsfFile.h"
+#include "llvm/DebugInfo/Msf/MsfCommon.h"
 #include "llvm/DebugInfo/Msf/MsfError.h"
 #include "llvm/DebugInfo/Msf/MsfStreamLayout.h"
 
 using namespace llvm;
 using namespace llvm::msf;
 
 namespace {
-// This exists so that we can use make_unique (which requires a public default
-// constructor, while still keeping the constructor of MappedBlockStream
-// protected, forcing users to go through the `create` interface.
-class MappedBlockStreamImpl : public MappedBlockStream {
+template <typename Base> class MappedBlockStreamImpl : public Base {
 public:
-  MappedBlockStreamImpl(const MsfStreamLayout &Layout, const IMsfFile &File)
-      : MappedBlockStream(Layout, File) {}
+  template <typename... Args>
+  MappedBlockStreamImpl(Args &&... Params)
+      : Base(std::forward<Args>(Params)...) {}
 };
 }
 
 typedef std::pair<uint32_t, uint32_t> Interval;
 static Interval intersect(const Interval &I1, const Interval &I2) {
   return std::make_pair(std::max(I1.first, I2.first),
                         std::min(I1.second, I2.second));
 }
 
-MappedBlockStream::MappedBlockStream(const MsfStreamLayout &Layout,
-                                     const IMsfFile &File)
-    : Msf(File), Layout(Layout) {}
+MappedBlockStream::MappedBlockStream(uint32_t BlockSize, uint32_t NumBlocks,
+                                     const MsfStreamLayout &Layout,
+                                     const ReadableStream &MsfData)
+    : BlockSize(BlockSize), NumBlocks(NumBlocks), StreamLayout(Layout),
+      MsfData(MsfData) {}
+
+std::unique_ptr<MappedBlockStream>
+MappedBlockStream::createStream(uint32_t BlockSize, uint32_t NumBlocks,
+                                const MsfStreamLayout &Layout,
+                                const ReadableStream &MsfData) {
+  return llvm::make_unique<MappedBlockStreamImpl<MappedBlockStream>>(
+      BlockSize, NumBlocks, Layout, MsfData);
+}
+
+std::unique_ptr<MappedBlockStream>
+MappedBlockStream::createIndexedStream(const MsfLayout &Layout,
+                                       const ReadableStream &MsfData,
+                                       uint32_t StreamIndex) {
+  MsfStreamLayout SL;
+  SL.Blocks = Layout.StreamMap[StreamIndex];
+  SL.Length = Layout.StreamSizes[StreamIndex];
+  return llvm::make_unique<MappedBlockStreamImpl<MappedBlockStream>>(
+      Layout.SB->BlockSize, Layout.SB->NumBlocks, SL, MsfData);
+}
+
+std::unique_ptr<MappedBlockStream>
+MappedBlockStream::createDirectoryStream(const MsfLayout &Layout,
+                                         const ReadableStream &MsfData) {
+  MsfStreamLayout SL;
+  SL.Blocks = Layout.DirectoryBlocks;
+  SL.Length = Layout.SB->NumDirectoryBytes;
+  return createStream(Layout.SB->BlockSize, Layout.SB->NumBlocks, SL, MsfData);
+}
 
 Error MappedBlockStream::readBytes(uint32_t Offset, uint32_t Size,
                                    ArrayRef<uint8_t> &Buffer) const {
   // Make sure we aren't trying to read beyond the end of the stream.
-  if (Size > Layout.Length)
+  if (Size > StreamLayout.Length)
     return make_error<MsfError>(msf_error_code::insufficient_buffer);
-  if (Offset > Layout.Length - Size)
+  if (Offset > StreamLayout.Length - Size)
     return make_error<MsfError>(msf_error_code::insufficient_buffer);
 
   if (tryReadContiguously(Offset, Size, Buffer))
     return Error::success();
 
   auto CacheIter = CacheMap.find(Offset);
   if (CacheIter != CacheMap.end()) {
     // Try to find an alloc that was large enough for this request.
Error MappedBlockStream::readBytes(uint32_t Offset, uint32_t Size,
   }
   Buffer = ArrayRef<uint8_t>(WriteBuffer, Size);
   return Error::success();
 }
 
 Error MappedBlockStream::readLongestContiguousChunk(
     uint32_t Offset, ArrayRef<uint8_t> &Buffer) const {
   // Make sure we aren't trying to read beyond the end of the stream.
-  if (Offset >= Layout.Length)
+  if (Offset >= StreamLayout.Length)
     return make_error<MsfError>(msf_error_code::insufficient_buffer);
-  uint32_t First = Offset / Msf.getBlockSize();
+  uint32_t First = Offset / BlockSize;
   uint32_t Last = First;
 
-  while (Last < Msf.getBlockCount() - 1) {
-    if (Layout.Blocks[Last] != Layout.Blocks[Last + 1] - 1)
+  while (Last < NumBlocks - 1) {
+    if (StreamLayout.Blocks[Last] != StreamLayout.Blocks[Last + 1] - 1)
       break;
     ++Last;
   }
 
-  uint32_t OffsetInFirstBlock = Offset % Msf.getBlockSize();
-  uint32_t BytesFromFirstBlock = Msf.getBlockSize() - OffsetInFirstBlock;
+  uint32_t OffsetInFirstBlock = Offset % BlockSize;
+  uint32_t BytesFromFirstBlock = BlockSize - OffsetInFirstBlock;
   uint32_t BlockSpan = Last - First + 1;
-  uint32_t ByteSpan =
-      BytesFromFirstBlock + (BlockSpan - 1) * Msf.getBlockSize();
-  auto Result = Msf.getBlockData(Layout.Blocks[First], Msf.getBlockSize());
-  if (!Result)
-    return Result.takeError();
-  Buffer = Result->drop_front(OffsetInFirstBlock);
-  Buffer = ArrayRef<uint8_t>(Buffer.data(), ByteSpan);
+  uint32_t ByteSpan = BytesFromFirstBlock + (BlockSpan - 1) * BlockSize;
+
+  ArrayRef<uint8_t> BlockData;
+  uint32_t MsfOffset = blockToOffset(StreamLayout.Blocks[First], BlockSize);
+  if (auto EC = MsfData.readBytes(MsfOffset, BlockSize, BlockData))
+    return EC;
+
+  BlockData = BlockData.drop_front(OffsetInFirstBlock);
+  Buffer = ArrayRef<uint8_t>(BlockData.data(), ByteSpan);
   return Error::success();
 }
 
-uint32_t MappedBlockStream::getLength() const { return Layout.Length; }
-
-Error MappedBlockStream::commit() const { return Error::success(); }
+uint32_t MappedBlockStream::getLength() const { return StreamLayout.Length; }
 
 bool MappedBlockStream::tryReadContiguously(uint32_t Offset, uint32_t Size,
                                             ArrayRef<uint8_t> &Buffer) const {
   // Attempt to fulfill the request with a reference directly into the stream.
   // This can work even if the request crosses a block boundary, provided that
   // all subsequent blocks are contiguous.  For example, a 10k read with a 4k
   // block size can be filled with a reference if, from the starting offset,
   // 3 blocks in a row are contiguous.
-  uint32_t BlockNum = Offset / Msf.getBlockSize();
-  uint32_t OffsetInBlock = Offset % Msf.getBlockSize();
-  uint32_t BytesFromFirstBlock =
-      std::min(Size, Msf.getBlockSize() - OffsetInBlock);
+  uint32_t BlockNum = Offset / BlockSize;
+  uint32_t OffsetInBlock = Offset % BlockSize;
+  uint32_t BytesFromFirstBlock = std::min(Size, BlockSize - OffsetInBlock);
   uint32_t NumAdditionalBlocks =
-      llvm::alignTo(Size - BytesFromFirstBlock, Msf.getBlockSize()) /
-      Msf.getBlockSize();
+      llvm::alignTo(Size - BytesFromFirstBlock, BlockSize) / BlockSize;
 
   uint32_t RequiredContiguousBlocks = NumAdditionalBlocks + 1;
-  uint32_t E = Layout.Blocks[BlockNum];
+  uint32_t E = StreamLayout.Blocks[BlockNum];
   for (uint32_t I = 0; I < RequiredContiguousBlocks; ++I, ++E) {
-    if (Layout.Blocks[I + BlockNum] != E)
+    if (StreamLayout.Blocks[I + BlockNum] != E)
       return false;
   }
 
-  uint32_t FirstBlockAddr = Layout.Blocks[BlockNum];
-  auto Result = Msf.getBlockData(FirstBlockAddr, Msf.getBlockSize());
-  if (!Result) {
-    consumeError(Result.takeError());
+  // Read out the entire block where the requested offset starts.  Then drop
+  // bytes from the beginning so that the actual starting byte lines up with
+  // the requested starting byte.  Then, since we know this is a contiguous
+  // cross-block span, explicitly resize the ArrayRef to cover the entire
+  // request length.
+  ArrayRef<uint8_t> BlockData;
+  uint32_t FirstBlockAddr = StreamLayout.Blocks[BlockNum];
+  uint32_t MsfOffset = blockToOffset(FirstBlockAddr, BlockSize);
+  if (auto EC = MsfData.readBytes(MsfOffset, BlockSize, BlockData)) {
+    consumeError(std::move(EC));
     return false;
   }
-  auto Data = Result->drop_front(OffsetInBlock);
-  Buffer = ArrayRef<uint8_t>(Data.data(), Size);
+  BlockData = BlockData.drop_front(OffsetInBlock);
+  Buffer = ArrayRef<uint8_t>(BlockData.data(), Size);
   return true;
 }
 
 Error MappedBlockStream::readBytes(uint32_t Offset,
                                    MutableArrayRef<uint8_t> Buffer) const {
-  uint32_t BlockNum = Offset / Msf.getBlockSize();
-  uint32_t OffsetInBlock = Offset % Msf.getBlockSize();
+  uint32_t BlockNum = Offset / BlockSize;
+  uint32_t OffsetInBlock = Offset % BlockSize;
 
   // Make sure we aren't trying to read beyond the end of the stream.
-  if (Buffer.size() > Layout.Length)
+  if (Buffer.size() > StreamLayout.Length)
     return make_error<MsfError>(msf_error_code::insufficient_buffer);
-  if (Offset > Layout.Length - Buffer.size())
+  if (Offset > StreamLayout.Length - Buffer.size())
     return make_error<MsfError>(msf_error_code::insufficient_buffer);
 
   uint32_t BytesLeft = Buffer.size();
   uint32_t BytesWritten = 0;
   uint8_t *WriteBuffer = Buffer.data();
   while (BytesLeft > 0) {
-    uint32_t StreamBlockAddr = Layout.Blocks[BlockNum];
+    uint32_t StreamBlockAddr = StreamLayout.Blocks[BlockNum];
 
-    auto Result = Msf.getBlockData(StreamBlockAddr, Msf.getBlockSize());
-    if (!Result)
-      return Result.takeError();
-
-    auto Data = *Result;
-    const uint8_t *ChunkStart = Data.data() + OffsetInBlock;
-    uint32_t BytesInChunk =
-        std::min(BytesLeft, Msf.getBlockSize() - OffsetInBlock);
+    ArrayRef<uint8_t> BlockData;
+    uint32_t Offset = blockToOffset(StreamBlockAddr, BlockSize);
+    if (auto EC = MsfData.readBytes(Offset, BlockSize, BlockData))
+      return EC;
+
+    const uint8_t *ChunkStart = BlockData.data() + OffsetInBlock;
+    uint32_t BytesInChunk = std::min(BytesLeft, BlockSize - OffsetInBlock);
     ::memcpy(WriteBuffer + BytesWritten, ChunkStart, BytesInChunk);
 
     BytesWritten += BytesInChunk;
     BytesLeft -= BytesInChunk;
     ++BlockNum;
     OffsetInBlock = 0;
   }
 
   return Error::success();
 }
 
-Error MappedBlockStream::writeBytes(uint32_t Offset,
-                                    ArrayRef<uint8_t> Buffer) const {
-  // Make sure we aren't trying to write beyond the end of the stream.
-  if (Buffer.size() > Layout.Length)
-    return make_error<MsfError>(msf_error_code::insufficient_buffer);
-
-  if (Offset > Layout.Length - Buffer.size())
-    return make_error<MsfError>(msf_error_code::insufficient_buffer);
-
-  uint32_t BlockNum = Offset / Msf.getBlockSize();
-  uint32_t OffsetInBlock = Offset % Msf.getBlockSize();
-
-  uint32_t BytesLeft = Buffer.size();
-  uint32_t BytesWritten = 0;
-  while (BytesLeft > 0) {
-    uint32_t StreamBlockAddr = Layout.Blocks[BlockNum];
-    uint32_t BytesToWriteInChunk =
-        std::min(BytesLeft, Msf.getBlockSize() - OffsetInBlock);
-
-    const uint8_t *Chunk = Buffer.data() + BytesWritten;
-    ArrayRef<uint8_t> ChunkData(Chunk, BytesToWriteInChunk);
-    if (auto EC = Msf.setBlockData(StreamBlockAddr, OffsetInBlock, ChunkData))
-      return EC;
-
-    BytesLeft -= BytesToWriteInChunk;
-    BytesWritten += BytesToWriteInChunk;
-    ++BlockNum;
-    OffsetInBlock = 0;
-  }
+uint32_t MappedBlockStream::getNumBytesCopied() const {
+  return static_cast<uint32_t>(Pool.getBytesAllocated());
 
+void MappedBlockStream::invalidateCache() { CacheMap.shrink_and_clear(); }
+
+void MappedBlockStream::fixCacheAfterWrite(uint32_t Offset,
+                                           ArrayRef<uint8_t> Data) const {
   // If this write overlapped a read which previously came from the pool,
   // someone may still be holding a pointer to that alloc which is now invalid.
   // Compute the overlapping range and update the cache entry, so any
   // outstanding buffers are automatically updated.
   for (const auto &MapEntry : CacheMap) {
     // If the end of the written extent precedes the beginning of the cached
     // extent, ignore this map entry.
-    if (Offset + BytesWritten < MapEntry.first)
+    if (Offset + Data.size() < MapEntry.first)
       continue;
     for (const auto &Alloc : MapEntry.second) {
       // If the end of the cached extent precedes the beginning of the written
       // extent, ignore this alloc.
       if (MapEntry.first + Alloc.size() < Offset)
         continue;
 
       // If we get here, they are guaranteed to overlap.
-      Interval WriteInterval = std::make_pair(Offset, Offset + BytesWritten);
+      Interval WriteInterval = std::make_pair(Offset, Offset + Data.size());
       Interval CachedInterval =
           std::make_pair(MapEntry.first, MapEntry.first + Alloc.size());
       // If they overlap, we need to write the new data into the overlapping
       // range.
       auto Intersection = intersect(WriteInterval, CachedInterval);
       assert(Intersection.first <= Intersection.second);
 
       uint32_t Length = Intersection.second - Intersection.first;
       uint32_t SrcOffset =
           AbsoluteDifference(WriteInterval.first, Intersection.first);
       uint32_t DestOffset =
           AbsoluteDifference(CachedInterval.first, Intersection.first);
-      ::memcpy(Alloc.data() + DestOffset, Buffer.data() + SrcOffset, Length);
-    }
-  }
+      ::memcpy(Alloc.data() + DestOffset, Data.data() + SrcOffset, Length);
+    }
 
-  return Error::success();
+WritableMappedBlockStream::WritableMappedBlockStream(
+    uint32_t BlockSize, uint32_t NumBlocks, const MsfStreamLayout &Layout,
+    const WritableStream &MsfData)
+    : ReadInterface(BlockSize, NumBlocks, Layout, MsfData),
+      WriteInterface(MsfData) {}
+
+std::unique_ptr<WritableMappedBlockStream>
+WritableMappedBlockStream::createStream(uint32_t BlockSize, uint32_t NumBlocks,
+                                        const MsfStreamLayout &Layout,
+                                        const WritableStream &MsfData) {
+  return llvm::make_unique<MappedBlockStreamImpl<WritableMappedBlockStream>>(
+      BlockSize, NumBlocks, Layout, MsfData);
 }
 
-uint32_t MappedBlockStream::getNumBytesCopied() const {
-  return static_cast<uint32_t>(Pool.getBytesAllocated());
+std::unique_ptr<WritableMappedBlockStream>
+WritableMappedBlockStream::createIndexedStream(const MsfLayout &Layout,
+                                               const WritableStream &MsfData,
+                                               uint32_t StreamIndex) {
+  MsfStreamLayout SL;
+  SL.Blocks = Layout.StreamMap[StreamIndex];
+  SL.Length = Layout.StreamSizes[StreamIndex];
+  return createStream(Layout.SB->BlockSize, Layout.SB->NumBlocks, SL, MsfData);
 }
 
-Expected<std::unique_ptr<MappedBlockStream>>
-MappedBlockStream::createIndexedStream(uint32_t StreamIdx,
-                                       const IMsfFile &File) {
-  if (StreamIdx >= File.getNumStreams())
-    return make_error<MsfError>(msf_error_code::no_stream);
-  MsfStreamLayout L;
-  L.Blocks = File.getStreamBlockList(StreamIdx);
-  L.Length = File.getStreamByteSize(StreamIdx);
-  return llvm::make_unique<MappedBlockStreamImpl>(L, File);
-}
-
-Expected<std::unique_ptr<MappedBlockStream>>
-MappedBlockStream::createDirectoryStream(uint32_t Length,
-                                         ArrayRef<support::ulittle32_t> Blocks,
-                                         const IMsfFile &File) {
-  MsfStreamLayout L;
-  L.Blocks = Blocks;
-  L.Length = Length;
+std::unique_ptr<WritableMappedBlockStream>
+WritableMappedBlockStream::createDirectoryStream(
+    const MsfLayout &Layout, const WritableStream &MsfData) {
+  MsfStreamLayout SL;
+  SL.Blocks = Layout.DirectoryBlocks;
+  SL.Length = Layout.SB->NumDirectoryBytes;
+  return createStream(Layout.SB->BlockSize, Layout.SB->NumBlocks, SL, MsfData);
+}
+
+Error WritableMappedBlockStream::readBytes(uint32_t Offset, uint32_t Size,
+                                           ArrayRef<uint8_t> &Buffer) const {
+  return ReadInterface.readBytes(Offset, Size, Buffer);
+}
+
+Error WritableMappedBlockStream::readLongestContiguousChunk(
+    uint32_t Offset, ArrayRef<uint8_t> &Buffer) const {
+  return ReadInterface.readLongestContiguousChunk(Offset, Buffer);
+}
+
+uint32_t WritableMappedBlockStream::getLength() const {
+  return ReadInterface.getLength();
+}
+
+Error WritableMappedBlockStream::writeBytes(uint32_t Offset,
+                                            ArrayRef<uint8_t> Buffer) const {
+  // Make sure we aren't trying to write beyond the end of the stream.
+  if (Buffer.size() > getStreamLength())
+    return make_error<MsfError>(msf_error_code::insufficient_buffer);
+
+  if (Offset > getStreamLayout().Length - Buffer.size())
+    return make_error<MsfError>(msf_error_code::insufficient_buffer);
+
+  uint32_t BlockNum = Offset / getBlockSize();
+  uint32_t OffsetInBlock = Offset % getBlockSize();
+
+  uint32_t BytesLeft = Buffer.size();
+  uint32_t BytesWritten = 0;
+  while (BytesLeft > 0) {
+    uint32_t StreamBlockAddr = getStreamLayout().Blocks[BlockNum];
+    uint32_t BytesToWriteInChunk =
+        std::min(BytesLeft, getBlockSize() - OffsetInBlock);
+
+    const uint8_t *Chunk = Buffer.data() + BytesWritten;
+    ArrayRef<uint8_t> ChunkData(Chunk, BytesToWriteInChunk);
+    uint32_t MsfOffset = blockToOffset(StreamBlockAddr, getBlockSize());
+    MsfOffset += OffsetInBlock;
+    if (auto EC = WriteInterface.writeBytes(MsfOffset, ChunkData))
+      return EC;
+
+    BytesLeft -= BytesToWriteInChunk;
+    BytesWritten += BytesToWriteInChunk;
+    ++BlockNum;
+    OffsetInBlock = 0;
+  }
+
+  ReadInterface.fixCacheAfterWrite(Offset, Buffer);
+
+  return Error::success();
+}
 
-  return llvm::make_unique<MappedBlockStreamImpl>(L, File);
+Error WritableMappedBlockStream::commit() const {
+  return WriteInterface.commit();
 }

llvm/trunk/lib/DebugInfo/Msf/MsfBuilder.cpp

-//===- MSFBuilder.cpp - MSF Directory & Metadata Builder --------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
for (const auto &D : StreamData) {
     uint32_t ExpectedNumBlocks = bytesToBlocks(D.first, BlockSize);
     assert(ExpectedNumBlocks == D.second.size() &&
            "Unexpected number of blocks");
     Size += ExpectedNumBlocks * sizeof(ulittle32_t);
   }
   return Size;
 }
 
-Expected<Layout> MsfBuilder::build() {
+Expected<MsfLayout> MsfBuilder::build() {
   SuperBlock *SB = Allocator.Allocate<SuperBlock>();
-  Layout L;
+  MsfLayout L;
   L.SB = SB;
 
   std::memcpy(SB->MagicBytes, Magic, sizeof(Magic));
   SB->BlockMapAddr = BlockMapAddr;
   SB->BlockSize = BlockSize;
   SB->NumDirectoryBytes = computeDirectoryByteSize();
   SB->FreeBlockMapBlock = FreePageMap;
   SB->Unknown1 = Unknown1;

llvm/trunk/lib/DebugInfo/Msf/StreamReader.cpp

 #include "llvm/DebugInfo/Msf/StreamReader.h"
 
 #include "llvm/DebugInfo/Msf/MsfError.h"
 #include "llvm/DebugInfo/Msf/StreamRef.h"
 
 using namespace llvm;
 using namespace llvm::msf;
 
-StreamReader::StreamReader(StreamRef S) : Stream(S), Offset(0) {}
+StreamReader::StreamReader(ReadableStreamRef S) : Stream(S), Offset(0) {}
 
 Error StreamReader::readLongestContiguousChunk(ArrayRef<uint8_t> &Buffer) {
   if (auto EC = Stream.readLongestContiguousChunk(Offset, Buffer))
     return EC;
   Offset += Buffer.size();
   return Error::success();
 }
 

 Error StreamReader::readFixedString(StringRef &Dest, uint32_t Length) {
   ArrayRef<uint8_t> Bytes;
   if (auto EC = readBytes(Bytes, Length))
     return EC;
   Dest = StringRef(reinterpret_cast<const char *>(Bytes.begin()), Bytes.size());
   return Error::success();
 }
 
-Error StreamReader::readStreamRef(StreamRef &Ref) {
+Error StreamReader::readStreamRef(ReadableStreamRef &Ref) {
   return readStreamRef(Ref, bytesRemaining());
 }
 
-Error StreamReader::readStreamRef(StreamRef &Ref, uint32_t Length) {
+Error StreamReader::readStreamRef(ReadableStreamRef &Ref, uint32_t Length) {
   if (bytesRemaining() < Length)
     return make_error<MsfError>(msf_error_code::insufficient_buffer);
   Ref = Stream.slice(Offset, Length);
   Offset += Length;
   return Error::success();
 }

llvm/trunk/lib/DebugInfo/Msf/StreamWriter.cpp

 
 #include "llvm/DebugInfo/Msf/MsfError.h"
 #include "llvm/DebugInfo/Msf/StreamReader.h"
 #include "llvm/DebugInfo/Msf/StreamRef.h"
 
 using namespace llvm;
 using namespace llvm::msf;
 
-StreamWriter::StreamWriter(StreamRef S) : Stream(S), Offset(0) {}
+StreamWriter::StreamWriter(WritableStreamRef S) : Stream(S), Offset(0) {}
 
 Error StreamWriter::writeBytes(ArrayRef<uint8_t> Buffer) {
   if (auto EC = Stream.writeBytes(Offset, Buffer))
     return EC;
   Offset += Buffer.size();
   return Error::success();
 }
 
Error StreamWriter::writeFixedString(StringRef Str) {
   ArrayRef<uint8_t> Bytes(Str.bytes_begin(), Str.bytes_end());
   if (auto EC = Stream.writeBytes(Offset, Bytes))
     return EC;
 
   Offset += Str.size();
   return Error::success();
 }
 
-Error StreamWriter::writeStreamRef(StreamRef Ref) {
+Error StreamWriter::writeStreamRef(ReadableStreamRef Ref) {
   if (auto EC = writeStreamRef(Ref, Ref.getLength()))
     return EC;
   // Don't increment Offset here, it is done by the overloaded call to
   // writeStreamRef.
   return Error::success();
 }
 
-Error StreamWriter::writeStreamRef(StreamRef Ref, uint32_t Length) {
+Error StreamWriter::writeStreamRef(ReadableStreamRef Ref, uint32_t Length) {
   Ref = Ref.slice(0, Length);
 
   StreamReader SrcReader(Ref);
   // This is a bit tricky.  If we just call readBytes, we are requiring that it
   // return us the entire stream as a contiguous buffer.  For large streams this
   // will allocate a huge amount of space from the pool.  Instead, iterate over
   // each contiguous chunk until we've consumed the entire stream.
   while (SrcReader.bytesRemaining() > 0) {
     ArrayRef<uint8_t> Chunk;
     if (auto EC = SrcReader.readLongestContiguousChunk(Chunk))
       return EC;
     if (auto EC = writeBytes(Chunk))
       return EC;
   }
   return Error::success();
 }

llvm/trunk/lib/DebugInfo/PDB/Raw/DbiStream.cpp

 Error DbiStream::initializeSectionHeadersData() {
   if (DbgStreams.size() == 0)
     return Error::success();
 
   uint32_t StreamNum = getDebugStreamIndex(DbgHeaderType::SectionHdr);
   if (StreamNum >= Pdb.getNumStreams())
     return make_error<RawError>(raw_error_code::no_stream);
 
-  auto SHS = MappedBlockStream::createIndexedStream(StreamNum, Pdb);
-  if (!SHS)
-    return SHS.takeError();
+  auto SHS = MappedBlockStream::createIndexedStream(
+      Pdb.getMsfLayout(), Pdb.getMsfBuffer(), StreamNum);
 
-  size_t StreamLen = (*SHS)->getLength();
+  size_t StreamLen = SHS->getLength();
   if (StreamLen % sizeof(object::coff_section))
     return make_error<RawError>(raw_error_code::corrupt_file,
                                 "Corrupted section header stream.");
 
   size_t NumSections = StreamLen / sizeof(object::coff_section);
-  msf::StreamReader Reader(**SHS);
+  msf::StreamReader Reader(*SHS);
   if (auto EC = Reader.readArray(SectionHeaders, NumSections))
     return make_error<RawError>(raw_error_code::corrupt_file,
                                 "Could not read a bitmap.");
 
-  SectionHeaderStream = std::move(*SHS);
+  SectionHeaderStream = std::move(SHS);
   return Error::success();
 }
 
 // Initializes this->Fpos.
 Error DbiStream::initializeFpoRecords() {
   if (DbgStreams.size() == 0)
     return Error::success();
 
   uint32_t StreamNum = getDebugStreamIndex(DbgHeaderType::NewFPO);
 
   // This means there is no FPO data.
   if (StreamNum == kInvalidStreamIndex)
     return Error::success();
 
   if (StreamNum >= Pdb.getNumStreams())
     return make_error<RawError>(raw_error_code::no_stream);
 
-  auto FS = MappedBlockStream::createIndexedStream(StreamNum, Pdb);
-  if (!FS)
-    return FS.takeError();
+  auto FS = MappedBlockStream::createIndexedStream(
+      Pdb.getMsfLayout(), Pdb.getMsfBuffer(), StreamNum);
 
-  size_t StreamLen = (*FS)->getLength();
+  size_t StreamLen = FS->getLength();
   if (StreamLen % sizeof(object::FpoData))
     return make_error<RawError>(raw_error_code::corrupt_file,
                                 "Corrupted New FPO stream.");
 
   size_t NumRecords = StreamLen / sizeof(object::FpoData);
-  msf::StreamReader Reader(**FS);
+  msf::StreamReader Reader(*FS);
   if (auto EC = Reader.readArray(FpoRecords, NumRecords))
     return make_error<RawError>(raw_error_code::corrupt_file,
                                 "Corrupted New FPO stream.");
-  FpoStream = std::move(*FS);
+  FpoStream = std::move(FS);
   return Error::success();
 }
 
 Error DbiStream::initializeSectionMapData() {
   if (SecMapSubstream.getLength() == 0)
     return Error::success();
 
   StreamReader SMReader(SecMapSubstream);
Expected<StringRef> DbiStream::getFileNameForIndex(uint32_t Index) const {
 
   uint32_t FileOffset = FileNameOffsets[Index];
   Names.setOffset(FileOffset);
   StringRef Name;
   if (auto EC = Names.readZeroString(Name))
     return std::move(EC);
   return Name;
 }
-
-Error DbiStream::commit() {
-  StreamWriter Writer(*Stream);
-  if (auto EC = Writer.writeObject(*Header))
-    return EC;
-
-  if (auto EC = Writer.writeStreamRef(ModInfoSubstream))
-    return EC;
-
-  if (auto EC = Writer.writeStreamRef(SecContrSubstream,
-                                      SecContrSubstream.getLength()))
-    return EC;
-  if (auto EC =
-          Writer.writeStreamRef(SecMapSubstream, SecMapSubstream.getLength()))
-    return EC;
-  if (auto EC = Writer.writeStreamRef(FileInfoSubstream,
-                                      FileInfoSubstream.getLength()))
-    return EC;
-  if (auto EC = Writer.writeStreamRef(TypeServerMapSubstream,
-                                      TypeServerMapSubstream.getLength()))
-    return EC;
-  if (auto EC = Writer.writeStreamRef(ECSubstream, ECSubstream.getLength()))
-    return EC;
-
-  if (Writer.bytesRemaining() > 0)
-    return make_error<RawError>(raw_error_code::invalid_format,
-                                "Unexpected bytes found in DBI Stream");
-  return Error::success();
-}

llvm/trunk/lib/DebugInfo/PDB/Raw/DbiStreamBuilder.cpp

 using namespace llvm::pdb;
 
 namespace {
 class ModiSubstreamBuilder {};
 }
 
 DbiStreamBuilder::DbiStreamBuilder(BumpPtrAllocator &Allocator)
     : Allocator(Allocator), Age(1), BuildNumber(0), PdbDllVersion(0),
-      PdbDllRbld(0), Flags(0), MachineType(PDB_Machine::x86) {}
+      PdbDllRbld(0), Flags(0), MachineType(PDB_Machine::x86), Header(nullptr) {}
 
 void DbiStreamBuilder::setVersionHeader(PdbRaw_DbiVer V) { VerHeader = V; }
 
 void DbiStreamBuilder::setAge(uint32_t A) { Age = A; }
 
 void DbiStreamBuilder::setBuildNumber(uint16_t B) { BuildNumber = B; }
 
 void DbiStreamBuilder::setPdbDllVersion(uint16_t V) { PdbDllVersion = V; }
uint32_t DbiStreamBuilder::calculateNamesBufferSize() const {
   }
   return Size;
 }
 
 Error DbiStreamBuilder::generateModiSubstream() {
   uint32_t Size = calculateModiSubstreamSize();
   auto Data = Allocator.Allocate<uint8_t>(Size);
 
-  ModInfoBuffer = ByteStream<true>(MutableArrayRef<uint8_t>(Data, Size));
+  ModInfoBuffer = MutableByteStream(MutableArrayRef<uint8_t>(Data, Size));
 
   StreamWriter ModiWriter(ModInfoBuffer);
   for (const auto &M : ModuleInfoList) {
     ModuleInfoHeader Layout = {};
     Layout.ModDiStream = kInvalidStreamIndex;
     Layout.NumFiles = M->SourceFiles.size();
     if (auto EC = ModiWriter.writeObject(Layout))
       return EC;

 }
 
 Error DbiStreamBuilder::generateFileInfoSubstream() {
   uint32_t Size = calculateFileInfoSubstreamSize();
   uint32_t NameSize = calculateNamesBufferSize();
   auto Data = Allocator.Allocate<uint8_t>(Size);
   uint32_t NamesOffset = Size - NameSize;
 
-  FileInfoBuffer = ByteStream<true>(MutableArrayRef<uint8_t>(Data, Size));
+  FileInfoBuffer = MutableByteStream(MutableArrayRef<uint8_t>(Data, Size));
 
-  StreamRef MetadataBuffer = StreamRef(FileInfoBuffer).keep_front(NamesOffset);
+  WritableStreamRef MetadataBuffer =
+      WritableStreamRef(FileInfoBuffer).keep_front(NamesOffset);
   StreamWriter MetadataWriter(MetadataBuffer);
 
   uint16_t ModiCount = std::min<uint16_t>(UINT16_MAX, ModuleInfos.size());
   uint16_t FileCount = std::min<uint16_t>(UINT16_MAX, SourceFileNames.size());
   if (auto EC = MetadataWriter.writeInteger(ModiCount)) // NumModules
     return EC;
   if (auto EC = MetadataWriter.writeInteger(FileCount)) // NumSourceFiles
     return EC;
   for (uint16_t I = 0; I < ModiCount; ++I) {
     if (auto EC = MetadataWriter.writeInteger(I)) // Mod Indices
       return EC;
   }
   for (const auto MI : ModuleInfoList) {
     FileCount = static_cast<uint16_t>(MI->SourceFiles.size());
     if (auto EC = MetadataWriter.writeInteger(FileCount)) // Mod File Counts
       return EC;
   }
 
   // Before writing the FileNameOffsets array, write the NamesBuffer array.
   // A side effect of this is that this will actually compute the various
   // file name offsets, so we can then go back and write the FileNameOffsets
   // array to the other substream.
-  NamesBuffer = StreamRef(FileInfoBuffer).drop_front(NamesOffset);
+  NamesBuffer = WritableStreamRef(FileInfoBuffer).drop_front(NamesOffset);
   StreamWriter NameBufferWriter(NamesBuffer);
   for (auto &Name : SourceFileNames) {
     Name.second = NameBufferWriter.getOffset();
     if (auto EC = NameBufferWriter.writeZeroString(Name.getKey()))
       return EC;
   }
 
   for (const auto MI : ModuleInfoList) {
Error DbiStreamBuilder::generateFileInfoSubstream() {
   if (MetadataWriter.bytesRemaining() > 0)
     return make_error<RawError>(
         raw_error_code::invalid_format,
         "The metadata buffer contained unexpected data.");
 
   return Error::success();
 }
 
-Expected<std::unique_ptr<DbiStream>> DbiStreamBuilder::build(PDBFile &File) {
-  if (!VerHeader.hasValue())
-    return make_error<RawError>(raw_error_code::unspecified,
-                                "Missing DBI Stream Version");
+Error DbiStreamBuilder::finalize() {
+  if (Header)
+    return Error::success();
 
-  auto DbiS = MappedBlockStream::createIndexedStream(StreamDBI, File);
-  if (!DbiS)
-    return DbiS.takeError();
-  auto DS = std::move(*DbiS);
-  DbiStreamHeader *H = DS->getAllocator().Allocate<DbiStreamHeader>(1);
+  DbiStreamHeader *H = Allocator.Allocate<DbiStreamHeader>();
 
   if (auto EC = generateModiSubstream())
     return std::move(EC);
   if (auto EC = generateFileInfoSubstream())
     return std::move(EC);
 
   H->VersionHeader = *VerHeader;
   H->VersionSignature = -1;
Error DbiStreamBuilder::finalize() {
   H->SecContrSubstreamSize = 0;
   H->SectionMapSize = 0;
   H->TypeServerSize = 0;
   H->SymRecordStreamIndex = kInvalidStreamIndex;
   H->PublicSymbolStreamIndex = kInvalidStreamIndex;
   H->MFCTypeServerIndex = kInvalidStreamIndex;
   H->GlobalSymbolStreamIndex = kInvalidStreamIndex;
 
-  auto Dbi = llvm::make_unique<DbiStream>(File, std::move(DS));
-  Dbi->Header = H;
-  Dbi->FileInfoSubstream = StreamRef(FileInfoBuffer);
-  Dbi->ModInfoSubstream = StreamRef(ModInfoBuffer);
+  Header = H;
+  return Error::success();
+}
+
+Expected<std::unique_ptr<DbiStream>>
+DbiStreamBuilder::build(PDBFile &File, const msf::WritableStream &Buffer) {
+  if (!VerHeader.hasValue())
+    return make_error<RawError>(raw_error_code::unspecified,
+                                "Missing DBI Stream Version");
+  if (auto EC = finalize())
+    return std::move(EC);
+
+  auto StreamData = MappedBlockStream::createIndexedStream(File.getMsfLayout(),
+                                                           Buffer, StreamDBI);
+  auto Dbi = llvm::make_unique<DbiStream>(File, std::move(StreamData));
+  Dbi->Header = Header;
+  Dbi->FileInfoSubstream = ReadableStreamRef(FileInfoBuffer);
+  Dbi->ModInfoSubstream = ReadableStreamRef(ModInfoBuffer);
   if (auto EC = Dbi->initializeModInfoArray())
     return std::move(EC);
   if (auto EC = Dbi->initializeFileInfo())
     return std::move(EC);
   return std::move(Dbi);
 }
+
+Error DbiStreamBuilder::commit(const msf::MsfLayout &Layout,
+                               const msf::WritableStream &Buffer) const {
+  auto InfoS =
+      WritableMappedBlockStream::createIndexedStream(Layout, Buffer, StreamDBI);
+
+  StreamWriter Writer(*InfoS);
+  if (auto EC = Writer.writeObject(*Header))
+    return EC;
+
+  if (auto EC = Writer.writeStreamRef(ModInfoBuffer))
+    return EC;
+  if (auto EC = Writer.writeStreamRef(FileInfoBuffer))
+    return EC;
+
+  if (Writer.bytesRemaining() > 0)
+    return make_error<RawError>(raw_error_code::invalid_format,
+                                "Unexpected bytes found in DBI Stream");
+  return Error::success();
+}

llvm/trunk/lib/DebugInfo/PDB/Raw/InfoStream.cpp

 #include "llvm/DebugInfo/PDB/Raw/InfoStream.h"
 #include "llvm/ADT/BitVector.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/DebugInfo/Msf/StreamReader.h"
 #include "llvm/DebugInfo/Msf/StreamWriter.h"
 #include "llvm/DebugInfo/PDB/Raw/PDBFile.h"
 #include "llvm/DebugInfo/PDB/Raw/RawConstants.h"
 #include "llvm/DebugInfo/PDB/Raw/RawError.h"
+#include "llvm/DebugInfo/PDB/Raw/RawTypes.h"
 
 using namespace llvm;
 using namespace llvm::codeview;
 using namespace llvm::msf;
 using namespace llvm::pdb;
 
 InfoStream::InfoStream(std::unique_ptr<MappedBlockStream> Stream)
     : Stream(std::move(Stream)) {}
 
 Error InfoStream::reload() {
   StreamReader Reader(*Stream);
 
-  const HeaderInfo *H;
+  const InfoStreamHeader *H;
   if (auto EC = Reader.readObject(H))
     return joinErrors(
         std::move(EC),
         make_error<RawError>(raw_error_code::corrupt_file,
                              "PDB Stream does not contain a header."));
 
   switch (H->Version) {
   case PdbImplVC70:
PdbRaw_ImplVer InfoStream::getVersion() const {
   return static_cast<PdbRaw_ImplVer>(Version);
 }
 
 uint32_t InfoStream::getSignature() const { return Signature; }
 
 uint32_t InfoStream::getAge() const { return Age; }
 
 PDB_UniqueId InfoStream::getGuid() const { return Guid; }
-
-Error InfoStream::commit() {
-  StreamWriter Writer(*Stream);
-
-  HeaderInfo H;
-  H.Age = Age;
-  H.Signature = Signature;
-  H.Version = Version;
-  H.Guid = Guid;
-  if (auto EC = Writer.writeObject(H))
-    return EC;
-
-  return NamedStreams.commit(Writer);
-}

llvm/trunk/lib/DebugInfo/PDB/Raw/InfoStreamBuilder.cpp

 //===- InfoStreamBuilder.cpp - PDB Info Stream Creation ---------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/DebugInfo/PDB/Raw/InfoStreamBuilder.h"
 
 #include "llvm/DebugInfo/Msf/MappedBlockStream.h"
 #include "llvm/DebugInfo/Msf/StreamWriter.h"
 #include "llvm/DebugInfo/PDB/Raw/InfoStream.h"
 #include "llvm/DebugInfo/PDB/Raw/RawError.h"
+#include "llvm/DebugInfo/PDB/Raw/RawTypes.h"
 
 using namespace llvm;
 using namespace llvm::codeview;
 using namespace llvm::msf;
 using namespace llvm::pdb;
 
-InfoStreamBuilder::InfoStreamBuilder() {}
+InfoStreamBuilder::InfoStreamBuilder()
+    : Ver(PdbRaw_ImplVer::PdbImplVC70), Sig(-1), Age(0) {}
 
 void InfoStreamBuilder::setVersion(PdbRaw_ImplVer V) { Ver = V; }
 
 void InfoStreamBuilder::setSignature(uint32_t S) { Sig = S; }
 
 void InfoStreamBuilder::setAge(uint32_t A) { Age = A; }
 
 void InfoStreamBuilder::setGuid(PDB_UniqueId G) { Guid = G; }
 
 NameMapBuilder &InfoStreamBuilder::getNamedStreamsBuilder() {
   return NamedStreams;
 }
 
 uint32_t InfoStreamBuilder::calculateSerializedLength() const {
-  return sizeof(InfoStream::HeaderInfo) +
-         NamedStreams.calculateSerializedLength();
+  return sizeof(InfoStreamHeader) + NamedStreams.calculateSerializedLength();
 }
 
-Expected<std::unique_ptr<InfoStream>> InfoStreamBuilder::build(PDBFile &File) {
-  if (!Ver.hasValue())
-    return make_error<RawError>(raw_error_code::unspecified,
-                                "Missing PDB Stream Version");
-  if (!Sig.hasValue())
-    return make_error<RawError>(raw_error_code::unspecified,
-                                "Missing PDB Stream Signature");
-  if (!Age.hasValue())
-    return make_error<RawError>(raw_error_code::unspecified,
-                                "Missing PDB Stream Age");
-  if (!Guid.hasValue())
-    return make_error<RawError>(raw_error_code::unspecified,
-                                "Missing PDB Stream Guid");
-
-  auto InfoS = MappedBlockStream::createIndexedStream(StreamPDB, File);
-  if (!InfoS)
-    return InfoS.takeError();
-  auto Info = llvm::make_unique<InfoStream>(std::move(*InfoS));
-  Info->Version = *Ver;
-  Info->Signature = *Sig;
-  Info->Age = *Age;
-  Info->Guid = *Guid;
+Expected<std::unique_ptr<InfoStream>>
+InfoStreamBuilder::build(PDBFile &File, const msf::WritableStream &Buffer) {
+  auto StreamData = MappedBlockStream::createIndexedStream(File.getMsfLayout(),
+                                                           Buffer, StreamPDB);
+  auto Info = llvm::make_unique<InfoStream>(std::move(StreamData));
+  Info->Version = Ver;
+  Info->Signature = Sig;
+  Info->Age = Age;
+  Info->Guid = Guid;
   auto NS = NamedStreams.build();
   if (!NS)
     return NS.takeError();
   Info->NamedStreams = **NS;
   return std::move(Info);
 }
+
+Error InfoStreamBuilder::commit(const msf::MsfLayout &Layout,
+                                const msf::WritableStream &Buffer) const {
+  auto InfoS =
+      WritableMappedBlockStream::createIndexedStream(Layout, Buffer, StreamPDB);
+  StreamWriter Writer(*InfoS);
+
+  InfoStreamHeader H;
+  H.Age = Age;
+  H.Signature = Sig;
+  H.Version = Ver;
+  H.Guid = Guid;
+  if (auto EC = Writer.writeObject(H))
+    return EC;
+
+  return NamedStreams.commit(Writer);
+}

llvm/trunk/lib/DebugInfo/PDB/Raw/ModInfo.cpp

 ModInfo::ModInfo() : Layout(nullptr) {}
 
 ModInfo::ModInfo(const ModInfo &Info)
     : ModuleName(Info.ModuleName), ObjFileName(Info.ObjFileName),
       Layout(Info.Layout) {}
 
 ModInfo::~ModInfo() {}
 
-Error ModInfo::initialize(StreamRef Stream, ModInfo &Info) {
+Error ModInfo::initialize(ReadableStreamRef Stream, ModInfo &Info) {
   StreamReader Reader(Stream);
   if (auto EC = Reader.readObject(Info.Layout))
     return EC;
 
   if (auto EC = Reader.readZeroString(Info.ModuleName))
     return EC;
 
   if (auto EC = Reader.readZeroString(Info.ObjFileName))

llvm/trunk/lib/DebugInfo/PDB/Raw/ModStream.cpp

Error ModStream::reload() {
   uint32_t SymbolSize = Mod.getSymbolDebugInfoByteSize();
   uint32_t C11Size = Mod.getLineInfoByteSize();
   uint32_t C13Size = Mod.getC13LineInfoByteSize();
 
   if (C11Size > 0 && C13Size > 0)
     return llvm::make_error<RawError>(raw_error_code::corrupt_file,
                                       "Module has both C11 and C13 line info");
 
-  StreamRef S;
+  ReadableStreamRef S;
 
   uint32_t SymbolSubstreamSig = 0;
   if (auto EC = Reader.readInteger(SymbolSubstreamSig))
     return EC;
   if (auto EC = Reader.readArray(SymbolsSubstream, SymbolSize - 4))
     return EC;
 
   if (auto EC = Reader.readStreamRef(LinesSubstream, C11Size))

llvm/trunk/lib/DebugInfo/PDB/Raw/NameMap.cpp

for (unsigned I : Present) {
     Stream.setOffset(OldOffset);
     // Add this to a string-map from name to stream number.
     Mapping.insert({Str, NameIndex});
   }
 
   return Error::success();
 }
 
-Error NameMap::commit(StreamWriter &Writer) {
-  // The first field is the number of bytes of string data.  So add
-  // up the length of all strings plus a null terminator for each
-  // one.
-  uint32_t NumBytes = 0;
-  for (auto B = Mapping.begin(), E = Mapping.end(); B != E; ++B) {
-    NumBytes += B->getKeyLength() + 1;
-  }
-
-  if (auto EC = Writer.writeInteger(NumBytes)) // Number of bytes of string data
-    return EC;
-  // Now all of the string data itself.
-  for (auto B = Mapping.begin(), E = Mapping.end(); B != E; ++B) {
-    if (auto EC = Writer.writeZeroString(B->getKey()))
-      return EC;
-  }
-
-  if (auto EC = Writer.writeInteger(Mapping.size())) // Hash Size
-    return EC;
-
-  if (auto EC = Writer.writeInteger(Mapping.size())) // Max Number of Strings
-    return EC;
-
-  if (auto EC = Writer.writeInteger(Mapping.size())) // Num Present Words
-    return EC;
-
-  // For each entry in the mapping, write a bit mask which represents a bucket
-  // to store it in.  We don't use this, so the value we write isn't important
-  // to us, it just has to be there.
-  for (auto B = Mapping.begin(), E = Mapping.end(); B != E; ++B) {
-    if (auto EC = Writer.writeInteger(1U))
-      return EC;
-  }
-
-  if (auto EC = Writer.writeInteger(0U)) // Num Deleted Words
-    return EC;
-
-  // Mappings of each word.
-  uint32_t OffsetSoFar = 0;
-  for (auto B = Mapping.begin(), E = Mapping.end(); B != E; ++B) {
-    // This is a list of key value pairs where the key is the offset into the
-    // strings buffer, and the value is a stream number.  Write each pair.
-    if (auto EC = Writer.writeInteger(OffsetSoFar))
-      return EC;
-
-    if (auto EC = Writer.writeInteger(B->second))
-      return EC;
-
-    OffsetSoFar += B->getKeyLength() + 1;
-  }
-
-  return Error::success();
-}
-
 iterator_range<StringMapConstIterator<uint32_t>> NameMap::entries() const {
   return llvm::make_range<StringMapConstIterator<uint32_t>>(Mapping.begin(),
                                                             Mapping.end());
 }
 
 bool NameMap::tryGetValue(StringRef Name, uint32_t &Value) const {
   auto Iter = Mapping.find(Name);
   if (Iter == Mapping.end())
     return false;
   Value = Iter->second;
   return true;
 }

llvm/trunk/lib/DebugInfo/PDB/Raw/NameMapBuilder.cpp

 //===- NameMapBuilder.cpp - PDB Name Map Builder ----------------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/DebugInfo/PDB/Raw/NameMapBuilder.h"
 
+#include "llvm/DebugInfo/Msf/StreamWriter.h"
 #include "llvm/DebugInfo/PDB/Raw/NameMap.h"
 #include "llvm/Support/Endian.h"
 
 using namespace llvm;
 using namespace llvm::pdb;
 
 NameMapBuilder::NameMapBuilder() {}
 
uint32_t NameMapBuilder::calculateSerializedLength() const {
 
   // For each present word, which we are treating as equivalent to the number of
   // entries in the table, we have a pair of integers.  An offset into the
   // string data, and a corresponding stream number.
   TotalLength += Map.size() * 2 * sizeof(support::ulittle32_t);
 
   return TotalLength;
 }
+
+Error NameMapBuilder::commit(msf::StreamWriter &Writer) const {
+  // The first field is the number of bytes of string data.  So add
+  // up the length of all strings plus a null terminator for each
+  // one.
+  uint32_t NumBytes = 0;
+  for (auto B = Map.begin(), E = Map.end(); B != E; ++B) {
+    NumBytes += B->getKeyLength() + 1;
+  }
+
+  if (auto EC = Writer.writeInteger(NumBytes)) // Number of bytes of string data
+    return EC;
+  // Now all of the string data itself.
+  for (auto B = Map.begin(), E = Map.end(); B != E; ++B) {
+    if (auto EC = Writer.writeZeroString(B->getKey()))
+      return EC;
+  }
+
+  if (auto EC = Writer.writeInteger(Map.size())) // Hash Size
+    return EC;
+
+  if (auto EC = Writer.writeInteger(Map.size())) // Max Number of Strings
+    return EC;
+
+  if (auto EC = Writer.writeInteger(Map.size())) // Num Present Words
+    return EC;
+
+  // For each entry in the mapping, write a bit mask which represents a bucket
+  // to store it in.  We don't use this, so the value we write isn't important
+  // to us, it just has to be there.
+  for (auto B = Map.begin(), E = Map.end(); B != E; ++B) {
+    if (auto EC = Writer.writeInteger(1U))
+      return EC;
+  }
+
+  if (auto EC = Writer.writeInteger(0U)) // Num Deleted Words
+    return EC;
+
+  // Mappings of each word.
+  uint32_t OffsetSoFar = 0;
+  for (auto B = Map.begin(), E = Map.end(); B != E; ++B) {
+    // This is a list of key value pairs where the key is the offset into the
+    // strings buffer, and the value is a stream number.  Write each pair.
+    if (auto EC = Writer.writeInteger(OffsetSoFar))
+      return EC;
+
+    if (auto EC = Writer.writeInteger(B->second))
+      return EC;
+
+    OffsetSoFar += B->getKeyLength() + 1;
+  }
+
+  return Error::success();
+}

llvm/trunk/lib/DebugInfo/PDB/Raw/PDBFile.cpp

 using namespace llvm::codeview;
 using namespace llvm::msf;
 using namespace llvm::pdb;
 
 namespace {
 typedef FixedStreamArray<support::ulittle32_t> ulittle_array;
 }
 
-PDBFile::PDBFile(std::unique_ptr<StreamInterface> PdbFileBuffer,
+PDBFile::PDBFile(std::unique_ptr<ReadableStream> PdbFileBuffer,
                  BumpPtrAllocator &Allocator)
     : Allocator(Allocator), Buffer(std::move(PdbFileBuffer)) {}
 
 PDBFile::~PDBFile() {}
 
-uint32_t PDBFile::getBlockSize() const { return MsfLayout.SB->BlockSize; }
+uint32_t PDBFile::getBlockSize() const { return ContainerLayout.SB->BlockSize; }
 
 uint32_t PDBFile::getFreeBlockMapBlock() const {
-  return MsfLayout.SB->FreeBlockMapBlock;
+  return ContainerLayout.SB->FreeBlockMapBlock;
 }
 
-uint32_t PDBFile::getBlockCount() const { return MsfLayout.SB->NumBlocks; }
+uint32_t PDBFile::getBlockCount() const {
+  return ContainerLayout.SB->NumBlocks;
+}
 
 uint32_t PDBFile::getNumDirectoryBytes() const {
-  return MsfLayout.SB->NumDirectoryBytes;
+  return ContainerLayout.SB->NumDirectoryBytes;
 }
 
 uint32_t PDBFile::getBlockMapIndex() const {
-  return MsfLayout.SB->BlockMapAddr;
+  return ContainerLayout.SB->BlockMapAddr;
 }
 
-uint32_t PDBFile::getUnknown1() const { return MsfLayout.SB->Unknown1; }
+uint32_t PDBFile::getUnknown1() const { return ContainerLayout.SB->Unknown1; }
 
 uint32_t PDBFile::getNumDirectoryBlocks() const {
-  return msf::bytesToBlocks(MsfLayout.SB->NumDirectoryBytes,
-                            MsfLayout.SB->BlockSize);
+  return msf::bytesToBlocks(ContainerLayout.SB->NumDirectoryBytes,
+                            ContainerLayout.SB->BlockSize);
 }
 
 uint64_t PDBFile::getBlockMapOffset() const {
-  return (uint64_t)MsfLayout.SB->BlockMapAddr * MsfLayout.SB->BlockSize;
+  return (uint64_t)ContainerLayout.SB->BlockMapAddr *
+         ContainerLayout.SB->BlockSize;
 }
 
-uint32_t PDBFile::getNumStreams() const { return MsfLayout.StreamSizes.size(); }
+uint32_t PDBFile::getNumStreams() const {
+  return ContainerLayout.StreamSizes.size();
+}
 
 uint32_t PDBFile::getStreamByteSize(uint32_t StreamIndex) const {
-  return MsfLayout.StreamSizes[StreamIndex];
+  return ContainerLayout.StreamSizes[StreamIndex];
 }
 
 ArrayRef<support::ulittle32_t>
 PDBFile::getStreamBlockList(uint32_t StreamIndex) const {
-  return MsfLayout.StreamMap[StreamIndex];
+  return ContainerLayout.StreamMap[StreamIndex];
 }
 
 uint32_t PDBFile::getFileSize() const { return Buffer->getLength(); }
 
 Expected<ArrayRef<uint8_t>> PDBFile::getBlockData(uint32_t BlockIndex,
                                                   uint32_t NumBytes) const {
   uint64_t StreamBlockOffset = msf::blockToOffset(BlockIndex, getBlockSize());
 
   ArrayRef<uint8_t> Result;
   if (auto EC = Buffer->readBytes(StreamBlockOffset, NumBytes, Result))
     return std::move(EC);
   return Result;
 }
 
 Error PDBFile::setBlockData(uint32_t BlockIndex, uint32_t Offset,
                             ArrayRef<uint8_t> Data) const {
-  if (Offset >= getBlockSize())
-    return make_error<RawError>(
-        raw_error_code::invalid_block_address,
-        "setBlockData attempted to write out of block bounds.");
-  if (Data.size() > getBlockSize() - Offset)
-    return make_error<RawError>(
-        raw_error_code::invalid_block_address,
-        "setBlockData attempted to write out of block bounds.");
-
-  uint64_t StreamBlockOffset = msf::blockToOffset(BlockIndex, getBlockSize());
-  StreamBlockOffset += Offset;
-  return Buffer->writeBytes(StreamBlockOffset, Data);
+  return make_error<RawError>(raw_error_code::not_writable,
+                              "PDBFile is immutable");
 }
 
 Error PDBFile::parseFileHeaders() {
   StreamReader Reader(*Buffer);
 
   const msf::SuperBlock *SB = nullptr;
   if (auto EC = Reader.readObject(SB)) {
     consumeError(std::move(EC));
     return make_error<RawError>(raw_error_code::corrupt_file,
                                 "Does not contain superblock");
   }
 
   if (auto EC = msf::validateSuperBlock(*SB))
     return EC;
 
   if (Buffer->getLength() % SB->BlockSize != 0)
     return make_error<RawError>(raw_error_code::corrupt_file,
                                 "File size is not a multiple of block size");
-  MsfLayout.SB = SB;
+  ContainerLayout.SB = SB;
 
   Reader.setOffset(getBlockMapOffset());
-  if (auto EC =
-          Reader.readArray(MsfLayout.DirectoryBlocks, getNumDirectoryBlocks()))
+  if (auto EC = Reader.readArray(ContainerLayout.DirectoryBlocks,
+                                 getNumDirectoryBlocks()))
     return EC;
 
   return Error::success();
 }
 
 Error PDBFile::parseStreamData() {
-  assert(MsfLayout.SB);
+  assert(ContainerLayout.SB);
   if (DirectoryStream)
     return Error::success();
 
   uint32_t NumStreams = 0;
 
   // Normally you can't use a MappedBlockStream without having fully parsed the
   // PDB file, because it accesses the directory and various other things, which
   // is exactly what we are attempting to parse.  By specifying a custom
   // subclass of IPDBStreamData which only accesses the fields that have already
   // been parsed, we can avoid this and reuse MappedBlockStream.
-  auto DS = MappedBlockStream::createDirectoryStream(
-      MsfLayout.SB->NumDirectoryBytes, getDirectoryBlockArray(), *this);
-  if (!DS)
-    return DS.takeError();
-  StreamReader Reader(**DS);
+  auto DS = MappedBlockStream::createDirectoryStream(ContainerLayout, *Buffer);
+  StreamReader Reader(*DS);
   if (auto EC = Reader.readInteger(NumStreams))
     return EC;
 
-  if (auto EC = Reader.readArray(MsfLayout.StreamSizes, NumStreams))
+  if (auto EC = Reader.readArray(ContainerLayout.StreamSizes, NumStreams))
     return EC;
   for (uint32_t I = 0; I < NumStreams; ++I) {
     uint32_t StreamSize = getStreamByteSize(I);
     // FIXME: What does StreamSize ~0U mean?
     uint64_t NumExpectedStreamBlocks =
         StreamSize == UINT32_MAX
             ? 0
-            : msf::bytesToBlocks(StreamSize, MsfLayout.SB->BlockSize);
+            : msf::bytesToBlocks(StreamSize, ContainerLayout.SB->BlockSize);
 
     // For convenience, we store the block array contiguously.  This is because
     // if someone calls setStreamMap(), it is more convenient to be able to call
     // it with an ArrayRef instead of setting up a StreamRef.  Since the
     // DirectoryStream is cached in the class and thus lives for the life of the
     // class, we can be guaranteed that readArray() will return a stable
     // reference, even if it has to allocate from its internal pool.
     ArrayRef<support::ulittle32_t> Blocks;
     if (auto EC = Reader.readArray(Blocks, NumExpectedStreamBlocks))
       return EC;
     for (uint32_t Block : Blocks) {
-      uint64_t BlockEndOffset = (uint64_t)(Block + 1) * MsfLayout.SB->BlockSize;
+      uint64_t BlockEndOffset =
+          (uint64_t)(Block + 1) * ContainerLayout.SB->BlockSize;
       if (BlockEndOffset > getFileSize())
         return make_error<RawError>(raw_error_code::corrupt_file,
                                     "Stream block map is corrupt.");
     }
-    MsfLayout.StreamMap.push_back(Blocks);
+    ContainerLayout.StreamMap.push_back(Blocks);
   }
 
   // We should have read exactly SB->NumDirectoryBytes bytes.
   assert(Reader.bytesRemaining() == 0);
-  DirectoryStream = std::move(*DS);
+  DirectoryStream = std::move(DS);
   return Error::success();
 }
 
 llvm::ArrayRef<support::ulittle32_t> PDBFile::getDirectoryBlockArray() const {
-  return MsfLayout.DirectoryBlocks;
+  return ContainerLayout.DirectoryBlocks;
 }
 
 Expected<InfoStream &> PDBFile::getPDBInfoStream() {
   if (!Info) {
-    auto InfoS = MappedBlockStream::createIndexedStream(StreamPDB, *this);
-    if (!InfoS)
-      return InfoS.takeError();
-    auto TempInfo = llvm::make_unique<InfoStream>(std::move(*InfoS));
+    auto InfoS = MappedBlockStream::createIndexedStream(ContainerLayout,
+                                                        *Buffer, StreamPDB);
+    auto TempInfo = llvm::make_unique<InfoStream>(std::move(InfoS));
     if (auto EC = TempInfo->reload())
       return std::move(EC);
     Info = std::move(TempInfo);
   }
   return *Info;
 }
 
 Expected<DbiStream &> PDBFile::getPDBDbiStream() {
   if (!Dbi) {
-    auto DbiS = MappedBlockStream::createIndexedStream(StreamDBI, *this);
-    if (!DbiS)
-      return DbiS.takeError();
-    auto TempDbi = llvm::make_unique<DbiStream>(*this, std::move(*DbiS));
+    auto DbiS = MappedBlockStream::createIndexedStream(ContainerLayout, *Buffer,
+                                                       StreamDBI);
+    auto TempDbi = llvm::make_unique<DbiStream>(*this, std::move(DbiS));
     if (auto EC = TempDbi->reload())
       return std::move(EC);
     Dbi = std::move(TempDbi);
   }
   return *Dbi;
 }
 
 Expected<TpiStream &> PDBFile::getPDBTpiStream() {
   if (!Tpi) {
-    auto TpiS = MappedBlockStream::createIndexedStream(StreamTPI, *this);
-    if (!TpiS)
-      return TpiS.takeError();
-    auto TempTpi = llvm::make_unique<TpiStream>(*this, std::move(*TpiS));
+    auto TpiS = MappedBlockStream::createIndexedStream(ContainerLayout, *Buffer,
+                                                       StreamTPI);
+    auto TempTpi = llvm::make_unique<TpiStream>(*this, std::move(TpiS));
     if (auto EC = TempTpi->reload())
       return std::move(EC);
     Tpi = std::move(TempTpi);
   }
   return *Tpi;
 }
 
 Expected<TpiStream &> PDBFile::getPDBIpiStream() {
   if (!Ipi) {
-    auto IpiS = MappedBlockStream::createIndexedStream(StreamIPI, *this);
-    if (!IpiS)
-      return IpiS.takeError();
-    auto TempIpi = llvm::make_unique<TpiStream>(*this, std::move(*IpiS));
+    auto IpiS = MappedBlockStream::createIndexedStream(ContainerLayout, *Buffer,
+                                                       StreamIPI);
+    auto TempIpi = llvm::make_unique<TpiStream>(*this, std::move(IpiS));
     if (auto EC = TempIpi->reload())
       return std::move(EC);
     Ipi = std::move(TempIpi);
   }
   return *Ipi;
 }
 
 Expected<PublicsStream &> PDBFile::getPDBPublicsStream() {
   if (!Publics) {
     auto DbiS = getPDBDbiStream();
     if (!DbiS)
       return DbiS.takeError();
 
     uint32_t PublicsStreamNum = DbiS->getPublicSymbolStreamIndex();
 
-    auto PublicS =
-        MappedBlockStream::createIndexedStream(PublicsStreamNum, *this);
-    if (!PublicS)
-      return PublicS.takeError();
+    auto PublicS = MappedBlockStream::createIndexedStream(
+        ContainerLayout, *Buffer, PublicsStreamNum);
     auto TempPublics =
-        llvm::make_unique<PublicsStream>(*this, std::move(*PublicS));
+        llvm::make_unique<PublicsStream>(*this, std::move(PublicS));
     if (auto EC = TempPublics->reload())
       return std::move(EC);
     Publics = std::move(TempPublics);
   }
   return *Publics;
 }
 
 Expected<SymbolStream &> PDBFile::getPDBSymbolStream() {
   if (!Symbols) {
     auto DbiS = getPDBDbiStream();
     if (!DbiS)
       return DbiS.takeError();
 
     uint32_t SymbolStreamNum = DbiS->getSymRecordStreamIndex();
+    auto SymbolS = MappedBlockStream::createIndexedStream(
+        ContainerLayout, *Buffer, SymbolStreamNum);
 
-    auto SymbolS =
-        MappedBlockStream::createIndexedStream(SymbolStreamNum, *this);
-    if (!SymbolS)
-      return SymbolS.takeError();
-    auto TempSymbols = llvm::make_unique<SymbolStream>(std::move(*SymbolS));
+    auto TempSymbols = llvm::make_unique<SymbolStream>(std::move(SymbolS));
     if (auto EC = TempSymbols->reload())
       return std::move(EC);
     Symbols = std::move(TempSymbols);
   }
   return *Symbols;
 }
 
 Expected<NameHashTable &> PDBFile::getStringTable() {
   if (!StringTable || !StringTableStream) {
     auto IS = getPDBInfoStream();
     if (!IS)
       return IS.takeError();
 
     uint32_t NameStreamIndex = IS->getNamedStreamIndex("/names");
 
     if (NameStreamIndex == 0)
       return make_error<RawError>(raw_error_code::no_stream);
     if (NameStreamIndex >= getNumStreams())
       return make_error<RawError>(raw_error_code::no_stream);
+    auto NS = MappedBlockStream::createIndexedStream(ContainerLayout, *Buffer,
+                                                     NameStreamIndex);
 
-    auto NS = MappedBlockStream::createIndexedStream(NameStreamIndex, *this);
-    if (!NS)
-      return NS.takeError();
-
-    StreamReader Reader(**NS);
+    StreamReader Reader(*NS);
     auto N = llvm::make_unique<NameHashTable>();
     if (auto EC = N->load(Reader))
       return std::move(EC);
     StringTable = std::move(N);
-    StringTableStream = std::move(*NS);
+    StringTableStream = std::move(NS);
   }
   return *StringTable;
 }
-
-Error PDBFile::commit() {
-  StreamWriter Writer(*Buffer);
-
-  if (auto EC = Writer.writeObject(*MsfLayout.SB))
-    return EC;
-  Writer.setOffset(getBlockMapOffset());
-  if (auto EC = Writer.writeArray(MsfLayout.DirectoryBlocks))
-    return EC;
-
-  auto DS = MappedBlockStream::createDirectoryStream(
-      MsfLayout.SB->NumDirectoryBytes, getDirectoryBlockArray(), *this);
-  if (!DS)
-    return DS.takeError();
-  auto DirStream = std::move(*DS);
-  StreamWriter DW(*DirStream);
-  if (auto EC = DW.writeInteger(this->getNumStreams()))
-    return EC;
-
-  if (auto EC = DW.writeArray(MsfLayout.StreamSizes))
-    return EC;
-
-  for (const auto &Blocks : MsfLayout.StreamMap) {
-    if (auto EC = DW.writeArray(Blocks))
-      return EC;
-  }
-
-  if (Info) {
-    if (auto EC = Info->commit())
-      return EC;
-  }
-
-  if (Dbi) {
-    if (auto EC = Dbi->commit())
-      return EC;
-  }
-
-  if (Symbols) {
-    if (auto EC = Symbols->commit())
-      return EC;
-  }
-
-  if (Publics) {
-    if (auto EC = Publics->commit())
-      return EC;
-  }
-
-  if (Tpi) {
-    if (auto EC = Tpi->commit())
-      return EC;
-  }
-
-  if (Ipi) {
-    if (auto EC = Ipi->commit())
-      return EC;
-  }
-
-  return Buffer->commit();
-}

llvm/trunk/lib/DebugInfo/PDB/Raw/PDBFileBuilder.cpp

 }
 
 DbiStreamBuilder &PDBFileBuilder::getDbiBuilder() {
   if (!Dbi)
     Dbi = llvm::make_unique<DbiStreamBuilder>(Allocator);
   return *Dbi;
 }
 
-Expected<msf::Layout> PDBFileBuilder::finalizeMsfLayout() const {
+Expected<msf::MsfLayout> PDBFileBuilder::finalizeMsfLayout() const {
   if (Info) {
     uint32_t Length = Info->calculateSerializedLength();
     if (auto EC = Msf->setStreamSize(StreamPDB, Length))
       return std::move(EC);
   }
   if (Dbi) {
     uint32_t Length = Dbi->calculateSerializedLength();
     if (auto EC = Msf->setStreamSize(StreamDBI, Length))
       return std::move(EC);
   }
 
   return Msf->build();
 }
 
 Expected<std::unique_ptr<PDBFile>>
-PDBFileBuilder::build(std::unique_ptr<msf::StreamInterface> PdbFileBuffer) {
+PDBFileBuilder::build(std::unique_ptr<msf::WritableStream> PdbFileBuffer) {
   auto ExpectedLayout = finalizeMsfLayout();
   if (!ExpectedLayout)
     return ExpectedLayout.takeError();
 
   auto File = llvm::make_unique<PDBFile>(std::move(PdbFileBuffer), Allocator);
-  File->MsfLayout = *ExpectedLayout;
+  File->ContainerLayout = *ExpectedLayout;
 
   if (Info) {
-    auto ExpectedInfo = Info->build(*File);
+    auto ExpectedInfo = Info->build(*File, *PdbFileBuffer);
     if (!ExpectedInfo)
       return ExpectedInfo.takeError();
     File->Info = std::move(*ExpectedInfo);
   }
 
   if (Dbi) {
-    auto ExpectedDbi = Dbi->build(*File);
+    auto ExpectedDbi = Dbi->build(*File, *PdbFileBuffer);
     if (!ExpectedDbi)
       return ExpectedDbi.takeError();
     File->Dbi = std::move(*ExpectedDbi);
   }
 
   if (File->Info && File->Dbi && File->Info->getAge() != File->Dbi->getAge())
     return llvm::make_error<RawError>(
         raw_error_code::corrupt_file,
         "PDB Stream Age doesn't match Dbi Stream Age!");
 
   return std::move(File);
 }
+
+Error PDBFileBuilder::commit(const msf::WritableStream &Buffer) {
+  StreamWriter Writer(Buffer);
+  auto ExpectedLayout = finalizeMsfLayout();
+  if (!ExpectedLayout)
+    return ExpectedLayout.takeError();
+  auto &Layout = *ExpectedLayout;
+
+  if (auto EC = Writer.writeObject(*Layout.SB))
+    return EC;
+  uint32_t BlockMapOffset =
+      msf::blockToOffset(Layout.SB->BlockMapAddr, Layout.SB->BlockSize);
+  Writer.setOffset(BlockMapOffset);
+  if (auto EC = Writer.writeArray(Layout.DirectoryBlocks))
+    return EC;
+
+  auto DirStream =
+      WritableMappedBlockStream::createDirectoryStream(Layout, Buffer);
+  StreamWriter DW(*DirStream);
+  if (auto EC = DW.writeInteger(Layout.StreamSizes.size()))
+    return EC;
+
+  if (auto EC = DW.writeArray(Layout.StreamSizes))
+    return EC;
+
+  for (const auto &Blocks : Layout.StreamMap) {
+    if (auto EC = DW.writeArray(Blocks))
+      return EC;
+  }
+
+  if (Info) {
+    if (auto EC = Info->commit(Layout, Buffer))
+      return EC;
+  }
+
+  if (Dbi) {
+    if (auto EC = Dbi->commit(Layout, Buffer))
+      return EC;
+  }
+
+  return Buffer.commit();
+}
 No newline at end of file

llvm/trunk/lib/DebugInfo/PDB/Raw/RawSession.cpp

 
 #include "llvm/Support/ErrorOr.h"
 #include "llvm/Support/MemoryBuffer.h"
 
 using namespace llvm;
 using namespace llvm::msf;
 using namespace llvm::pdb;
 
-namespace {
-// We need a class which behaves like an immutable ByteStream, but whose data
-// is backed by an llvm::MemoryBuffer.  It also needs to own the underlying
-// MemoryBuffer, so this simple adapter is a good way to achieve that.
-class InputByteStream : public ByteStream<false> {
-public:
-  explicit InputByteStream(std::unique_ptr<MemoryBuffer> Buffer)
-      : ByteStream(ArrayRef<uint8_t>(Buffer->getBuffer().bytes_begin(),
-                                     Buffer->getBuffer().bytes_end())),
-        MemBuffer(std::move(Buffer)) {}
-
-  std::unique_ptr<MemoryBuffer> MemBuffer;
-};
-}
-
 RawSession::RawSession(std::unique_ptr<PDBFile> PdbFile,
                        std::unique_ptr<BumpPtrAllocator> Allocator)
     : Pdb(std::move(PdbFile)), Allocator(std::move(Allocator)) {}
 
 RawSession::~RawSession() {}
 
 Error RawSession::createFromPdb(StringRef Path,
                                 std::unique_ptr<IPDBSession> &Session) {
 
   ErrorOr<std::unique_ptr<MemoryBuffer>> ErrorOrBuffer =
       MemoryBuffer::getFileOrSTDIN(Path, /*FileSize=*/-1,
                                    /*RequiresNullTerminator=*/false);
   if (!ErrorOrBuffer)
     return llvm::make_error<GenericError>(generic_error_code::invalid_path);
 
   std::unique_ptr<MemoryBuffer> Buffer = std::move(*ErrorOrBuffer);
-  auto Stream = llvm::make_unique<InputByteStream>(std::move(Buffer));
+  auto Stream = llvm::make_unique<MemoryBufferByteStream>(std::move(Buffer));
 
   auto Allocator = llvm::make_unique<BumpPtrAllocator>();
   auto File = llvm::make_unique<PDBFile>(std::move(Stream), *Allocator);
   if (auto EC = File->parseFileHeaders())
     return EC;
   if (auto EC = File->parseStreamData())
     return EC;
 

llvm/trunk/lib/DebugInfo/PDB/Raw/TpiStream.cpp

Error TpiStream::reload() {
   // The actual type records themselves come from this stream
   if (auto EC = Reader.readArray(TypeRecords, Header->TypeRecordBytes))
     return EC;
 
   // Hash indices, hash values, etc come from the hash stream.
   if (Header->HashStreamIndex >= Pdb.getNumStreams())
     return make_error<RawError>(raw_error_code::corrupt_file,
                                 "Invalid TPI hash stream index.");
-
-  auto HS =
-      MappedBlockStream::createIndexedStream(Header->HashStreamIndex, Pdb);
-  if (!HS)
-    return HS.takeError();
-  StreamReader HSR(**HS);
+  auto HS = MappedBlockStream::createIndexedStream(
+      Pdb.getMsfLayout(), Pdb.getMsfBuffer(), Header->HashStreamIndex);
+  StreamReader HSR(*HS);
 
   uint32_t NumHashValues = Header->HashValueBuffer.Length / sizeof(ulittle32_t);
   if (NumHashValues != NumTypeRecords())
     return make_error<RawError>(
         raw_error_code::corrupt_file,
         "TPI hash count does not match with the number of type records.");
   HSR.setOffset(Header->HashValueBuffer.Off);
   if (auto EC = HSR.readArray(HashValues, NumHashValues))
     return EC;
 
   HSR.setOffset(Header->IndexOffsetBuffer.Off);
   uint32_t NumTypeIndexOffsets =
       Header->IndexOffsetBuffer.Length / sizeof(TypeIndexOffset);
   if (auto EC = HSR.readArray(TypeIndexOffsets, NumTypeIndexOffsets))
     return EC;
 
   HSR.setOffset(Header->HashAdjBuffer.Off);
   uint32_t NumHashAdjustments =
       Header->HashAdjBuffer.Length / sizeof(TypeIndexOffset);
   if (auto EC = HSR.readArray(HashAdjustments, NumHashAdjustments))
     return EC;
 
-  HashStream = std::move(*HS);
+  HashStream = std::move(HS);
 
   // TPI hash table is a parallel array for the type records.
   // Verify that the hash values match with type records.
   if (auto EC = verifyHashValues())
     return EC;
 
   return Error::success();
 }

llvm/trunk/tools/llvm-pdbdump/LLVMOutputStyle.cpp

Error LLVMOutputStyle::dumpStreamData() {
   StringRef DumpStreamStr = opts::raw::DumpStreamDataIdx;
   uint32_t DumpStreamNum;
   if (DumpStreamStr.getAsInteger(/*Radix=*/0U, DumpStreamNum))
     return Error::success();
 
   if (DumpStreamNum >= StreamCount)
     return make_error<RawError>(raw_error_code::no_stream);
 
-  auto S = MappedBlockStream::createIndexedStream(DumpStreamNum, File);
-  if (!S)
-    return S.takeError();
-  StreamReader R(**S);
+  auto S = MappedBlockStream::createIndexedStream(
+      File.getMsfLayout(), File.getMsfBuffer(), DumpStreamNum);
+  StreamReader R(*S);
   while (R.bytesRemaining() > 0) {
     ArrayRef<uint8_t> Data;
     uint32_t BytesToReadInBlock = std::min(
         R.bytesRemaining(), static_cast<uint32_t>(File.getBlockSize()));
     if (auto EC = R.readBytes(Data, BytesToReadInBlock))
       return EC;
     P.printBinaryBlock(
         "Data",
Error LLVMOutputStyle::dumpNamedStream() {
 
   if (NameStreamIndex != 0) {
     std::string Name("Stream '");
     Name += opts::raw::DumpStreamDataName;
     Name += "'";
     DictScope D(P, Name);
     P.printNumber("Index", NameStreamIndex);
 
-    auto NameStream =
-        MappedBlockStream::createIndexedStream(NameStreamIndex, File);
-    if (!NameStream)
-      return NameStream.takeError();
-    StreamReader Reader(**NameStream);
+    auto NameStream = MappedBlockStream::createIndexedStream(
+        File.getMsfLayout(), File.getMsfBuffer(), NameStreamIndex);
+    StreamReader Reader(*NameStream);
 
     NameHashTable NameTable;
     if (auto EC = NameTable.load(Reader))
       return EC;
 
     P.printHex("Signature", NameTable.getSignature());
     P.printNumber("Version", NameTable.getHashVersion());
     P.printNumber("Name Count", NameTable.getNameCount());
for (auto &Modi : DS->modules()) {
           P.printString(File.str());
       }
       bool HasModuleDI =
           (Modi.Info.getModuleStreamIndex() < File.getNumStreams());
       bool ShouldDumpSymbols =
           (opts::raw::DumpModuleSyms || opts::raw::DumpSymRecordBytes);
       if (HasModuleDI && (ShouldDumpSymbols || opts::raw::DumpLineInfo)) {
         auto ModStreamData = MappedBlockStream::createIndexedStream(
-            Modi.Info.getModuleStreamIndex(), File);
-        if (!ModStreamData)
-          return ModStreamData.takeError();
-        ModStream ModS(Modi.Info, std::move(*ModStreamData));
+            File.getMsfLayout(), File.getMsfBuffer(),
+            Modi.Info.getModuleStreamIndex());
+
+        ModStream ModS(Modi.Info, std::move(ModStreamData));
         if (auto EC = ModS.reload())
           return EC;
 
         if (ShouldDumpSymbols) {
           ListScope SS(P, "Symbols");
           codeview::CVSymbolDumper SD(P, TD, nullptr, false);
           bool HadError = false;
           for (const auto &S : ModS.symbols(&HadError)) {
for (auto &Modi : DS->modules()) {
           ListScope SS(P, "LineInfo");
           bool HadError = false;
           // Define a locally scoped visitor to print the different
           // substream types types.
           class RecordVisitor : public codeview::IModuleSubstreamVisitor {
           public:
             RecordVisitor(ScopedPrinter &P, PDBFile &F) : P(P), F(F) {}
             Error visitUnknown(ModuleSubstreamKind Kind,
-                               StreamRef Stream) override {
+                               ReadableStreamRef Stream) override {
               DictScope DD(P, "Unknown");
               ArrayRef<uint8_t> Data;
               StreamReader R(Stream);
               if (auto EC = R.readBytes(Data, R.bytesRemaining())) {
                 return make_error<RawError>(
                     raw_error_code::corrupt_file,
                     "DBI stream contained corrupt line info record");
               }
               P.printBinaryBlock("Data", Data);
               return Error::success();
             }
             Error
-            visitFileChecksums(StreamRef Data,
+            visitFileChecksums(ReadableStreamRef Data,
                                const FileChecksumArray &Checksums) override {
               DictScope DD(P, "FileChecksums");
               for (const auto &C : Checksums) {
                 DictScope DDD(P, "Checksum");
                 if (auto Result = getFileNameForOffset(C.FileNameOffset))
                   P.printString("FileName", Result.get());
                 else
                   return Result.takeError();
                 P.flush();
                 P.printEnum("Kind", uint8_t(C.Kind), getFileChecksumNames());
                 P.printBinaryBlock("Checksum", C.Checksum);
               }
               return Error::success();
             }
 
-            Error visitLines(StreamRef Data, const LineSubstreamHeader *Header,
+            Error visitLines(ReadableStreamRef Data,
+                             const LineSubstreamHeader *Header,
                              const LineInfoArray &Lines) override {
               DictScope DD(P, "Lines");
               for (const auto &L : Lines) {
                 if (auto Result = getFileNameForOffset2(L.NameIndex))
                   P.printString("FileName", Result.get());
                 else
                   return Result.takeError();
                 P.flush();

llvm/trunk/tools/llvm-pdbdump/llvm-pdbdump.cpp

 #include "llvm/Support/Signals.h"
 #include "llvm/Support/raw_ostream.h"
 
 using namespace llvm;
 using namespace llvm::codeview;
 using namespace llvm::msf;
 using namespace llvm::pdb;
 
-namespace {
-// A simple adapter that acts like a ByteStream but holds ownership over
-// and underlying FileOutputBuffer.
-class FileBufferByteStream : public ByteStream<true> {
-public:
-  FileBufferByteStream(std::unique_ptr<FileOutputBuffer> Buffer)
-      : ByteStream(MutableArrayRef<uint8_t>(Buffer->getBufferStart(),
-                                            Buffer->getBufferEnd())),
-        FileBuffer(std::move(Buffer)) {}
-
-  Error commit() const override {
-    if (FileBuffer->commit())
-      return llvm::make_error<RawError>(raw_error_code::not_writable);
-    return Error::success();
-  }
-
-private:
-  std::unique_ptr<FileOutputBuffer> FileBuffer;
-};
-}
-
 namespace opts {
 
 cl::SubCommand RawSubcommand("raw", "Dump raw structure of the PDB file");
 cl::SubCommand
     PrettySubcommand("pretty",
                      "Dump semantic information about types and symbols");
 cl::SubCommand
     YamlToPdbSubcommand("yaml2pdb",
if (YamlObj.DbiStream.hasValue()) {
     DbiBuilder.setVersionHeader(YamlObj.DbiStream->VerHeader);
     for (const auto &MI : YamlObj.DbiStream->ModInfos) {
       ExitOnErr(DbiBuilder.addModuleInfo(MI.Obj, MI.Mod));
       for (auto S : MI.SourceFiles)
         ExitOnErr(DbiBuilder.addModuleSourceFile(MI.Mod, S));
     }
   }
 
-  auto Pdb = Builder.build(std::move(FileByteStream));
-  ExitOnErr(Pdb.takeError());
-
-  auto &PdbFile = *Pdb;
-  ExitOnErr(PdbFile->commit());
+  ExitOnErr(Builder.commit(*FileByteStream));
 }
 
 static void pdb2Yaml(StringRef Path) {
   std::unique_ptr<IPDBSession> Session;
   ExitOnErr(loadDataForPDB(PDB_ReaderType::Raw, Path, Session));
 
   RawSession *RS = static_cast<RawSession *>(Session.get());
   PDBFile &File = RS->getPDBFile();

llvm/trunk/tools/llvm-readobj/COFFDumper.cpp

void COFFDumper::printCodeViewSymbolsSubsection(StringRef Subsection,
                                                 const SectionRef &Section,
                                                 StringRef SectionContents) {
   ArrayRef<uint8_t> BinaryData(Subsection.bytes_begin(),
                                Subsection.bytes_end());
   auto CODD = llvm::make_unique<COFFObjectDumpDelegate>(*this, Section, Obj,
                                                         SectionContents);
 
   CVSymbolDumper CVSD(W, CVTD, std::move(CODD), opts::CodeViewSubsectionBytes);
-  ByteStream<> Stream(BinaryData);
+  ByteStream Stream(BinaryData);
   CVSymbolArray Symbols;
   StreamReader Reader(Stream);
   if (auto EC = Reader.readArray(Symbols, Reader.getLength())) {
     consumeError(std::move(EC));
     W.flush();
     error(object_error::parse_failed);
   }
 
if (SectionName == ".debug$T") {
       StringRef Data;
       error(S.getContents(Data));
       uint32_t Magic;
       error(consume(Data, Magic));
       if (Magic != 4)
         error(object_error::parse_failed);
       ArrayRef<uint8_t> Bytes(reinterpret_cast<const uint8_t *>(Data.data()),
                               Data.size());
-      ByteStream<> Stream(Bytes);
+      ByteStream Stream(Bytes);
       CVTypeArray Types;
       StreamReader Reader(Stream);
       if (auto EC = Reader.readArray(Types, Reader.getLength())) {
         consumeError(std::move(EC));
         W.flush();
         error(object_error::parse_failed);
       }
 

llvm/trunk/unittests/DebugInfo/PDB/MappedBlockStreamTest.cpp

 using namespace llvm;
 using namespace llvm::msf;
 
 namespace {
 
 static const uint32_t BlocksAry[] = {0, 1, 2, 5, 4, 3, 6, 7, 8, 9};
 static uint8_t DataAry[] = {'A', 'B', 'C', 'F', 'E', 'D', 'G', 'H', 'I', 'J'};
 
-class DiscontiguousFile : public IMsfFile {
+class DiscontiguousStream : public WritableStream {
 public:
-  DiscontiguousFile(ArrayRef<uint32_t> Blocks, MutableArrayRef<uint8_t> Data)
+  DiscontiguousStream(ArrayRef<uint32_t> Blocks, MutableArrayRef<uint8_t> Data)
       : Blocks(Blocks.begin(), Blocks.end()), Data(Data.begin(), Data.end()) {}
 
-  uint32_t getBlockSize() const override { return 1; }
-  uint32_t getBlockCount() const override { return Blocks.size(); }
-  uint32_t getNumStreams() const override { return 1; }
-  uint32_t getStreamByteSize(uint32_t StreamIndex) const override {
-    return getBlockCount() * getBlockSize();
-  }
-  ArrayRef<support::ulittle32_t>
-  getStreamBlockList(uint32_t StreamIndex) const override {
-    if (StreamIndex != 0)
-      return ArrayRef<support::ulittle32_t>();
-    return Blocks;
-  }
-  Expected<ArrayRef<uint8_t>> getBlockData(uint32_t BlockIndex,
-                                           uint32_t NumBytes) const override {
-    return ArrayRef<uint8_t>(&Data[BlockIndex], NumBytes);
-  }
-
-  Error setBlockData(uint32_t BlockIndex, uint32_t Offset,
-                     ArrayRef<uint8_t> SrcData) const override {
-    if (BlockIndex >= Blocks.size())
+  uint32_t block_size() const { return 1; }
+  uint32_t block_count() const { return Blocks.size(); }
+
+  Error readBytes(uint32_t Offset, uint32_t Size,
+                  ArrayRef<uint8_t> &Buffer) const override {
+    if (Offset + Size > Data.size())
       return make_error<MsfError>(msf_error_code::insufficient_buffer);
-    if (Offset > getBlockSize() - SrcData.size())
+    Buffer = Data.slice(Offset, Size);
+    return Error::success();
+  }
+
+  Error readLongestContiguousChunk(uint32_t Offset,
+                                   ArrayRef<uint8_t> &Buffer) const override {
+    if (Offset >= Data.size())
+      return make_error<MsfError>(msf_error_code::insufficient_buffer);
+    Buffer = Data.drop_front(Offset);
+    return Error::success();
+  }
+
+  uint32_t getLength() const override { return Data.size(); }
+
+  Error writeBytes(uint32_t Offset, ArrayRef<uint8_t> SrcData) const override {
+    if (Offset + SrcData.size() > Data.size())
       return make_error<MsfError>(msf_error_code::insufficient_buffer);
-    ::memcpy(&Data[BlockIndex] + Offset, SrcData.data(), SrcData.size());
+    ::memcpy(&Data[Offset], SrcData.data(), SrcData.size());
     return Error::success();
   }
+  Error commit() const override { return Error::success(); }
+
+  MsfStreamLayout layout() const {
+    return MsfStreamLayout{Data.size(), Blocks};
+  }
 
 private:
   std::vector<support::ulittle32_t> Blocks;
   MutableArrayRef<uint8_t> Data;
 };
 
 // Tests that a read which is entirely contained within a single block works
 // and does not allocate.
 TEST(MappedBlockStreamTest, ReadBeyondEndOfStreamRef) {
-  DiscontiguousFile F(BlocksAry, DataAry);
-  auto ExpectedS = MappedBlockStream::createIndexedStream(0, F);
-  EXPECT_EXPECTED(ExpectedS);
-  auto &S = *ExpectedS;
+  DiscontiguousStream F(BlocksAry, DataAry);
+  auto S = MappedBlockStream::createStream(F.block_size(), F.block_count(),
+                                           F.layout(), F);
 
   StreamReader R(*S);
-  StreamRef SR;
+  ReadableStreamRef SR;
   EXPECT_NO_ERROR(R.readStreamRef(SR, 0U));
   ArrayRef<uint8_t> Buffer;
   EXPECT_ERROR(SR.readBytes(0U, 1U, Buffer));
   EXPECT_NO_ERROR(R.readStreamRef(SR, 1U));
   EXPECT_ERROR(SR.readBytes(1U, 1U, Buffer));
 }
 
 // Tests that a read which outputs into a full destination buffer works and
 // does not fail due to the length of the output buffer.
 TEST(MappedBlockStreamTest, ReadOntoNonEmptyBuffer) {
-  DiscontiguousFile F(BlocksAry, DataAry);
-  auto ExpectedS = MappedBlockStream::createIndexedStream(0, F);
-  EXPECT_EXPECTED(ExpectedS);
-  auto &S = *ExpectedS;
+  DiscontiguousStream F(BlocksAry, DataAry);
+  auto S = MappedBlockStream::createStream(F.block_size(), F.block_count(),
+                                           F.layout(), F);
+
   StreamReader R(*S);
   StringRef Str = "ZYXWVUTSRQPONMLKJIHGFEDCBA";
   EXPECT_NO_ERROR(R.readFixedString(Str, 1));
   EXPECT_EQ(Str, StringRef("A"));
   EXPECT_EQ(0U, S->getNumBytesCopied());
 }
 
 // Tests that a read which crosses a block boundary, but where the subsequent
 // blocks are still contiguous in memory to the previous block works and does
 // not allocate memory.
 TEST(MappedBlockStreamTest, ZeroCopyReadContiguousBreak) {
-  DiscontiguousFile F(BlocksAry, DataAry);
-  auto ExpectedS = MappedBlockStream::createIndexedStream(0, F);
-  EXPECT_EXPECTED(ExpectedS);
-  auto &S = *ExpectedS;
+  DiscontiguousStream F(BlocksAry, DataAry);
+  auto S = MappedBlockStream::createStream(F.block_size(), F.block_count(),
+                                           F.layout(), F);
   StreamReader R(*S);
   StringRef Str;
   EXPECT_NO_ERROR(R.readFixedString(Str, 2));
   EXPECT_EQ(Str, StringRef("AB"));
   EXPECT_EQ(0U, S->getNumBytesCopied());
 
   R.setOffset(6);
   EXPECT_NO_ERROR(R.readFixedString(Str, 4));
   EXPECT_EQ(Str, StringRef("GHIJ"));
   EXPECT_EQ(0U, S->getNumBytesCopied());
 }
 
 // Tests that a read which crosses a block boundary and cannot be referenced
 // contiguously works and allocates only the precise amount of bytes
 // requested.
 TEST(MappedBlockStreamTest, CopyReadNonContiguousBreak) {
-  DiscontiguousFile F(BlocksAry, DataAry);
-  auto ExpectedS = MappedBlockStream::createIndexedStream(0, F);
-  EXPECT_EXPECTED(ExpectedS);
-  auto &S = *ExpectedS;
+  DiscontiguousStream F(BlocksAry, DataAry);
+  auto S = MappedBlockStream::createStream(F.block_size(), F.block_count(),
+                                           F.layout(), F);
   StreamReader R(*S);
   StringRef Str;
   EXPECT_NO_ERROR(R.readFixedString(Str, 10));
   EXPECT_EQ(Str, StringRef("ABCDEFGHIJ"));
   EXPECT_EQ(10U, S->getNumBytesCopied());
 }
 
 // Test that an out of bounds read which doesn't cross a block boundary
 // fails and allocates no memory.
 TEST(MappedBlockStreamTest, InvalidReadSizeNoBreak) {
-  DiscontiguousFile F(BlocksAry, DataAry);
-  auto ExpectedS = MappedBlockStream::createIndexedStream(0, F);
-  EXPECT_EXPECTED(ExpectedS);
-  auto &S = *ExpectedS;
+  DiscontiguousStream F(BlocksAry, DataAry);
+  auto S = MappedBlockStream::createStream(F.block_size(), F.block_count(),
+                                           F.layout(), F);
   StreamReader R(*S);
   StringRef Str;
 
   R.setOffset(10);
   EXPECT_ERROR(R.readFixedString(Str, 1));
   EXPECT_EQ(0U, S->getNumBytesCopied());
 }
 
 // Test that an out of bounds read which crosses a contiguous block boundary
 // fails and allocates no memory.
 TEST(MappedBlockStreamTest, InvalidReadSizeContiguousBreak) {
-  DiscontiguousFile F(BlocksAry, DataAry);
-  auto ExpectedS = MappedBlockStream::createIndexedStream(0, F);
-  EXPECT_EXPECTED(ExpectedS);
-  auto &S = *ExpectedS;
+  DiscontiguousStream F(BlocksAry, DataAry);
+  auto S = MappedBlockStream::createStream(F.block_size(), F.block_count(),
+                                           F.layout(), F);
   StreamReader R(*S);
   StringRef Str;
 
   R.setOffset(6);
   EXPECT_ERROR(R.readFixedString(Str, 5));
   EXPECT_EQ(0U, S->getNumBytesCopied());
 }
 
 // Test that an out of bounds read which crosses a discontiguous block
 // boundary fails and allocates no memory.
 TEST(MappedBlockStreamTest, InvalidReadSizeNonContiguousBreak) {
-  DiscontiguousFile F(BlocksAry, DataAry);
-  auto ExpectedS = MappedBlockStream::createIndexedStream(0, F);
-  EXPECT_EXPECTED(ExpectedS);
-  auto &S = *ExpectedS;
+  DiscontiguousStream F(BlocksAry, DataAry);
+  auto S = MappedBlockStream::createStream(F.block_size(), F.block_count(),
+                                           F.layout(), F);
   StreamReader R(*S);
   StringRef Str;
 
   EXPECT_ERROR(R.readFixedString(Str, 11));
   EXPECT_EQ(0U, S->getNumBytesCopied());
 }
 
 // Tests that a read which is entirely contained within a single block but
 // beyond the end of a StreamRef fails.
 TEST(MappedBlockStreamTest, ZeroCopyReadNoBreak) {
-  DiscontiguousFile F(BlocksAry, DataAry);
-  auto ExpectedS = MappedBlockStream::createIndexedStream(0, F);
-  EXPECT_EXPECTED(ExpectedS);
-  auto &S = *ExpectedS;
+  DiscontiguousStream F(BlocksAry, DataAry);
+  auto S = MappedBlockStream::createStream(F.block_size(), F.block_count(),
+                                           F.layout(), F);
   StreamReader R(*S);
   StringRef Str;
   EXPECT_NO_ERROR(R.readFixedString(Str, 1));
   EXPECT_EQ(Str, StringRef("A"));
   EXPECT_EQ(0U, S->getNumBytesCopied());
 }
 
 // Tests that a read which is not aligned on the same boundary as a previous
 // cached request, but which is known to overlap that request, shares the
 // previous allocation.
 TEST(MappedBlockStreamTest, UnalignedOverlappingRead) {
-  DiscontiguousFile F(BlocksAry, DataAry);
-  auto ExpectedS = MappedBlockStream::createIndexedStream(0, F);
-  EXPECT_EXPECTED(ExpectedS);
-  auto &S = *ExpectedS;
+  DiscontiguousStream F(BlocksAry, DataAry);
+  auto S = MappedBlockStream::createStream(F.block_size(), F.block_count(),
+                                           F.layout(), F);
   StreamReader R(*S);
   StringRef Str1;
   StringRef Str2;
   EXPECT_NO_ERROR(R.readFixedString(Str1, 7));
   EXPECT_EQ(Str1, StringRef("ABCDEFG"));
   EXPECT_EQ(7U, S->getNumBytesCopied());
 
   R.setOffset(2);
   EXPECT_NO_ERROR(R.readFixedString(Str2, 3));
   EXPECT_EQ(Str2, StringRef("CDE"));
   EXPECT_EQ(Str1.data() + 2, Str2.data());
   EXPECT_EQ(7U, S->getNumBytesCopied());
 }
 
 // Tests that a read which is not aligned on the same boundary as a previous
 // cached request, but which only partially overlaps a previous cached request,
 // still works correctly and allocates again from the shared pool.
 TEST(MappedBlockStreamTest, UnalignedOverlappingReadFail) {
-  DiscontiguousFile F(BlocksAry, DataAry);
-  auto ExpectedS = MappedBlockStream::createIndexedStream(0, F);
-  EXPECT_EXPECTED(ExpectedS);
-  auto &S = *ExpectedS;
+  DiscontiguousStream F(BlocksAry, DataAry);
+  auto S = MappedBlockStream::createStream(F.block_size(), F.block_count(),
+                                           F.layout(), F);
   StreamReader R(*S);
   StringRef Str1;
   StringRef Str2;
   EXPECT_NO_ERROR(R.readFixedString(Str1, 6));
   EXPECT_EQ(Str1, StringRef("ABCDEF"));
   EXPECT_EQ(6U, S->getNumBytesCopied());
 
   R.setOffset(4);
   EXPECT_NO_ERROR(R.readFixedString(Str2, 4));
   EXPECT_EQ(Str2, StringRef("EFGH"));
   EXPECT_EQ(10U, S->getNumBytesCopied());
 }
 
 TEST(MappedBlockStreamTest, WriteBeyondEndOfStream) {
   static uint8_t Data[] = {'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J'};
   static uint8_t LargeBuffer[] = {'0', '1', '2', '3', '4', '5',
                                   '6', '7', '8', '9', 'A'};
   static uint8_t SmallBuffer[] = {'0', '1', '2'};
   static_assert(sizeof(LargeBuffer) > sizeof(Data),
                 "LargeBuffer is not big enough");
 
-  DiscontiguousFile F(BlocksAry, Data);
-  auto ExpectedS = MappedBlockStream::createIndexedStream(0, F);
-  EXPECT_EXPECTED(ExpectedS);
-  auto &S = *ExpectedS;
+  DiscontiguousStream F(BlocksAry, Data);
+  auto S = WritableMappedBlockStream::createStream(
+      F.block_size(), F.block_count(), F.layout(), F);
   ArrayRef<uint8_t> Buffer;
 
   EXPECT_ERROR(S->writeBytes(0, ArrayRef<uint8_t>(LargeBuffer)));
   EXPECT_NO_ERROR(S->writeBytes(0, ArrayRef<uint8_t>(SmallBuffer)));
   EXPECT_NO_ERROR(S->writeBytes(7, ArrayRef<uint8_t>(SmallBuffer)));
   EXPECT_ERROR(S->writeBytes(8, ArrayRef<uint8_t>(SmallBuffer)));
 }
 
 TEST(MappedBlockStreamTest, TestWriteBytesNoBreakBoundary) {
   static uint8_t Data[] = {'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J'};
-  DiscontiguousFile F(BlocksAry, Data);
-  auto ExpectedS = MappedBlockStream::createIndexedStream(0, F);
-  EXPECT_EXPECTED(ExpectedS);
-  auto &S = *ExpectedS;
+  DiscontiguousStream F(BlocksAry, Data);
+  auto S = WritableMappedBlockStream::createStream(
+      F.block_size(), F.block_count(), F.layout(), F);
   ArrayRef<uint8_t> Buffer;
 
   EXPECT_NO_ERROR(S->readBytes(0, 1, Buffer));
   EXPECT_EQ(Buffer, ArrayRef<uint8_t>('A'));
   EXPECT_NO_ERROR(S->readBytes(9, 1, Buffer));
   EXPECT_EQ(Buffer, ArrayRef<uint8_t>('J'));
 
   EXPECT_NO_ERROR(S->writeBytes(0, ArrayRef<uint8_t>('J')));

 }
 
 TEST(MappedBlockStreamTest, TestWriteBytesBreakBoundary) {
   static uint8_t Data[] = {'0', '0', '0', '0', '0', '0', '0', '0', '0', '0'};
   static uint8_t TestData[] = {'T', 'E', 'S', 'T', 'I', 'N', 'G', '.'};
   static uint8_t Expected[] = {'T', 'E', 'S', 'N', 'I',
                                'T', 'G', '.', '0', '0'};
 
-  DiscontiguousFile F(BlocksAry, Data);
-  auto ExpectedS = MappedBlockStream::createIndexedStream(0, F);
-  EXPECT_EXPECTED(ExpectedS);
-  auto &S = *ExpectedS;
+  DiscontiguousStream F(BlocksAry, Data);
+  auto S = WritableMappedBlockStream::createStream(
+      F.block_size(), F.block_count(), F.layout(), F);
   ArrayRef<uint8_t> Buffer;
 
   EXPECT_NO_ERROR(S->writeBytes(0, TestData));
   // First just compare the memory, then compare the result of reading the
   // string out.
   EXPECT_EQ(ArrayRef<uint8_t>(Data), ArrayRef<uint8_t>(Expected));
 
   EXPECT_NO_ERROR(S->readBytes(0, 8, Buffer));
   EXPECT_EQ(Buffer, ArrayRef<uint8_t>(TestData));
 }
 
 TEST(MappedBlockStreamTest, TestWriteThenRead) {
   std::vector<uint8_t> DataBytes(10);
   MutableArrayRef<uint8_t> Data(DataBytes);
   const uint32_t Blocks[] = {2, 1, 0, 6, 3, 4, 5, 7, 9, 8};
 
-  DiscontiguousFile F(Blocks, Data);
-  auto ExpectedS = MappedBlockStream::createIndexedStream(0, F);
-  EXPECT_EXPECTED(ExpectedS);
-  auto &S = *ExpectedS;
+  DiscontiguousStream F(Blocks, Data);
+  auto S = WritableMappedBlockStream::createStream(
+      F.block_size(), F.block_count(), F.layout(), F);
 
   enum class MyEnum : uint32_t { Val1 = 2908234, Val2 = 120891234 };
   using support::ulittle32_t;
 
   uint16_t u16[] = {31468, 0};
   uint32_t u32[] = {890723408, 0};
   MyEnum Enum[] = {MyEnum::Val1, MyEnum::Val2};
   StringRef ZStr[] = {"Zero Str", ""};

 TEST(MappedBlockStreamTest, TestWriteContiguousStreamRef) {
   std::vector<uint8_t> DestDataBytes(10);
   MutableArrayRef<uint8_t> DestData(DestDataBytes);
   const uint32_t DestBlocks[] = {2, 1, 0, 6, 3, 4, 5, 7, 9, 8};
 
   std::vector<uint8_t> SrcDataBytes(10);
   MutableArrayRef<uint8_t> SrcData(SrcDataBytes);
 
-  DiscontiguousFile F(DestBlocks, DestData);
-  auto ExpectedS = MappedBlockStream::createIndexedStream(0, F);
-  EXPECT_EXPECTED(ExpectedS);
-  auto &DestStream = *ExpectedS;
+  DiscontiguousStream F(DestBlocks, DestData);
+  auto DestStream = WritableMappedBlockStream::createStream(
+      F.block_size(), F.block_count(), F.layout(), F);
 
   // First write "Test Str" into the source stream.
-  ByteStream<true> SourceStream(SrcData);
+  MutableByteStream SourceStream(SrcData);
   StreamWriter SourceWriter(SourceStream);
   EXPECT_NO_ERROR(SourceWriter.writeZeroString("Test Str"));
   EXPECT_EQ(SrcDataBytes, std::vector<uint8_t>(
                               {'T', 'e', 's', 't', ' ', 'S', 't', 'r', 0, 0}));
 
   // Then write the source stream into the dest stream.
   StreamWriter DestWriter(*DestStream);
   EXPECT_NO_ERROR(DestWriter.writeStreamRef(SourceStream));
TEST(MappedBlockStreamTest, TestWriteDiscontiguousStreamRef) {
   std::vector<uint8_t> DestDataBytes(10);
   MutableArrayRef<uint8_t> DestData(DestDataBytes);
   const uint32_t DestBlocks[] = {2, 1, 0, 6, 3, 4, 5, 7, 9, 8};
 
   std::vector<uint8_t> SrcDataBytes(10);
   MutableArrayRef<uint8_t> SrcData(SrcDataBytes);
   const uint32_t SrcBlocks[] = {1, 0, 6, 3, 4, 5, 2, 7, 8, 9};
 
-  DiscontiguousFile DestFile(DestBlocks, DestData);
-  DiscontiguousFile SrcFile(SrcBlocks, SrcData);
+  DiscontiguousStream DestF(DestBlocks, DestData);
+  DiscontiguousStream SrcF(SrcBlocks, SrcData);
 
-  auto ExpectedDest = MappedBlockStream::createIndexedStream(0, DestFile);
-  auto ExpectedSrc = MappedBlockStream::createIndexedStream(0, SrcFile);
-  EXPECT_EXPECTED(ExpectedDest);
-  EXPECT_EXPECTED(ExpectedSrc);
-  auto &DestStream = *ExpectedDest;
-  auto &SrcStream = *ExpectedSrc;
+  auto Dest = WritableMappedBlockStream::createStream(
+      DestF.block_size(), DestF.block_count(), DestF.layout(), DestF);
+  auto Src = WritableMappedBlockStream::createStream(
+      SrcF.block_size(), SrcF.block_count(), SrcF.layout(), SrcF);
 
   // First write "Test Str" into the source stream.
-  StreamWriter SourceWriter(*SrcStream);
+  StreamWriter SourceWriter(*Src);
   EXPECT_NO_ERROR(SourceWriter.writeZeroString("Test Str"));
   EXPECT_EQ(SrcDataBytes, std::vector<uint8_t>(
                               {'e', 'T', 't', 't', ' ', 'S', 's', 'r', 0, 0}));
 
   // Then write the source stream into the dest stream.
-  StreamWriter DestWriter(*DestStream);
-  EXPECT_NO_ERROR(DestWriter.writeStreamRef(*SrcStream));
+  StreamWriter DestWriter(*Dest);
+  EXPECT_NO_ERROR(DestWriter.writeStreamRef(*Src));
   EXPECT_EQ(DestDataBytes, std::vector<uint8_t>(
                                {'s', 'e', 'T', ' ', 'S', 't', 't', 'r', 0, 0}));
 
   // Then read the string back out of the dest stream.
   StringRef Result;
-  StreamReader DestReader(*DestStream);
+  StreamReader DestReader(*Dest);
   EXPECT_NO_ERROR(DestReader.readZeroString(Result));
   EXPECT_EQ(Result, "Test Str");
 }
 
 } // end anonymous namespace

llvm/trunk/unittests/DebugInfo/PDB/MsfBuilderTest.cpp

TEST_F(MsfBuilderTest, TestAddStreamNoDirectoryBlockIncrease) {
   // tests the case where the directory *DOES NOT* grow large enough that it
   // crosses a Block boundary.
   auto ExpectedMsf = MsfBuilder::create(Allocator, 4096);
   EXPECT_EXPECTED(ExpectedMsf);
   auto &Msf = *ExpectedMsf;
 
   auto ExpectedL1 = Msf.build();
   EXPECT_EXPECTED(ExpectedL1);
-  Layout &L1 = *ExpectedL1;
+  MsfLayout &L1 = *ExpectedL1;
 
   auto OldDirBlocks = L1.DirectoryBlocks;
   EXPECT_EQ(1U, OldDirBlocks.size());
 
   auto ExpectedMsf2 = MsfBuilder::create(Allocator, 4096);
   EXPECT_EXPECTED(ExpectedMsf2);
   auto &Msf2 = *ExpectedMsf2;
 
   EXPECT_NO_ERROR(Msf2.addStream(4000));
   EXPECT_EQ(1U, Msf2.getNumStreams());
   EXPECT_EQ(4000U, Msf2.getStreamSize(0));
   auto Blocks = Msf2.getStreamBlocks(0);
   EXPECT_EQ(1U, Blocks.size());
 
   auto ExpectedL2 = Msf2.build();
   EXPECT_EXPECTED(ExpectedL2);
-  Layout &L2 = *ExpectedL2;
+  MsfLayout &L2 = *ExpectedL2;
   auto NewDirBlocks = L2.DirectoryBlocks;
   EXPECT_EQ(1U, NewDirBlocks.size());
 }
 
 TEST_F(MsfBuilderTest, TestAddStreamWithDirectoryBlockIncrease) {
   // Test that adding a new stream correctly updates the directory.  This only
   // tests the case where the directory *DOES* grow large enough that it
   // crosses a Block boundary.  This is because the newly added stream occupies
   // so many Blocks that need to be indexed in the directory that the directory
   // crosses a Block boundary.
   auto ExpectedMsf = MsfBuilder::create(Allocator, 4096);
   EXPECT_EXPECTED(ExpectedMsf);
   auto &Msf = *ExpectedMsf;
 
   EXPECT_NO_ERROR(Msf.addStream(4096 * 4096 / sizeof(uint32_t)));
 
   auto ExpectedL1 = Msf.build();
   EXPECT_EXPECTED(ExpectedL1);
-  Layout &L1 = *ExpectedL1;
+  MsfLayout &L1 = *ExpectedL1;
   auto DirBlocks = L1.DirectoryBlocks;
   EXPECT_EQ(2U, DirBlocks.size());
 }
 
 TEST_F(MsfBuilderTest, TestGrowStreamNoBlockIncrease) {
   // Test growing an existing stream by a value that does not affect the number
   // of blocks it occupies.
   auto ExpectedMsf = MsfBuilder::create(Allocator, 4096);
for (int I = 0; I < 3; ++I) {
     EXPECT_NO_ERROR(Msf.addStream(StreamSizes[I]));
     NumUsedBlocks += bytesToBlocks(StreamSizes[I], 4096);
     EXPECT_EQ(NumUsedBlocks, Msf.getNumUsedBlocks());
     EXPECT_EQ(0U, Msf.getNumFreeBlocks());
   }
 }
 
 TEST_F(MsfBuilderTest, BuildMsfLayout) {
-  // Test that we can generate an Msf Layout structure from a valid layout
+  // Test that we can generate an Msf MsfLayout structure from a valid layout
   // specification.
   auto ExpectedMsf = MsfBuilder::create(Allocator, 4096);
   EXPECT_EXPECTED(ExpectedMsf);
   auto &Msf = *ExpectedMsf;
 
   const uint32_t StreamSizes[] = {4000, 6193, 189723};
   uint32_t ExpectedNumBlocks = msf::getMinimumBlockCount();
   for (int I = 0; I < 3; ++I) {
     EXPECT_NO_ERROR(Msf.addStream(StreamSizes[I]));
     ExpectedNumBlocks += bytesToBlocks(StreamSizes[I], 4096);
   }
   ++ExpectedNumBlocks; // The directory itself should use 1 block
 
   auto ExpectedLayout = Msf.build();
   EXPECT_EXPECTED(ExpectedLayout);
-  Layout &L = *ExpectedLayout;
+  MsfLayout &L = *ExpectedLayout;
   EXPECT_EQ(4096U, L.SB->BlockSize);
   EXPECT_EQ(ExpectedNumBlocks, L.SB->NumBlocks);
 
   EXPECT_EQ(1U, L.DirectoryBlocks.size());
 
   EXPECT_EQ(3U, L.StreamMap.size());
   EXPECT_EQ(3U, L.StreamSizes.size());
   for (int I = 0; I < 3; ++I) {
TEST_F(MsfBuilderTest, UseDirectoryBlockHint) {
   auto &Msf = *ExpectedMsf;
 
   uint32_t B = msf::getFirstUnreservedBlock();
   EXPECT_NO_ERROR(Msf.setDirectoryBlocksHint({B + 1}));
   EXPECT_NO_ERROR(Msf.addStream(2048, {B + 2}));
 
   auto ExpectedLayout = Msf.build();
   EXPECT_EXPECTED(ExpectedLayout);
-  Layout &L = *ExpectedLayout;
+  MsfLayout &L = *ExpectedLayout;
   EXPECT_EQ(msf::getMinimumBlockCount() + 2, L.SB->NumBlocks);
   EXPECT_EQ(1U, L.DirectoryBlocks.size());
   EXPECT_EQ(1U, L.StreamMap[0].size());
 
   EXPECT_EQ(B + 1, L.DirectoryBlocks[0]);
   EXPECT_EQ(B + 2, L.StreamMap[0].front());
 }
 
 TEST_F(MsfBuilderTest, DirectoryBlockHintInsufficient) {
   Expected<MsfBuilder> ExpectedMsf =
       MsfBuilder::create(Allocator, 4096, msf::getMinimumBlockCount() + 2);
   EXPECT_EXPECTED(ExpectedMsf);
   auto &Msf = *ExpectedMsf;
   uint32_t B = msf::getFirstUnreservedBlock();
   EXPECT_NO_ERROR(Msf.setDirectoryBlocksHint({B + 1}));
 
   uint32_t Size = 4096 * 4096 / 4;
   EXPECT_NO_ERROR(Msf.addStream(Size));
 
   auto ExpectedLayout = Msf.build();
   EXPECT_EXPECTED(ExpectedLayout);
-  Layout &L = *ExpectedLayout;
+  MsfLayout &L = *ExpectedLayout;
   EXPECT_EQ(2U, L.DirectoryBlocks.size());
   EXPECT_EQ(B + 1, L.DirectoryBlocks[0]);
 }
 
 TEST_F(MsfBuilderTest, DirectoryBlockHintOverestimated) {
   Expected<MsfBuilder> ExpectedMsf =
       MsfBuilder::create(Allocator, 4096, msf::getMinimumBlockCount() + 2);
   EXPECT_EXPECTED(ExpectedMsf);
   auto &Msf = *ExpectedMsf;
 
   uint32_t B = msf::getFirstUnreservedBlock();
   EXPECT_NO_ERROR(Msf.setDirectoryBlocksHint({B + 1, B + 2}));
 
   EXPECT_NO_ERROR(Msf.addStream(2048));
 
   auto ExpectedLayout = Msf.build();
   EXPECT_EXPECTED(ExpectedLayout);
-  Layout &L = *ExpectedLayout;
+  MsfLayout &L = *ExpectedLayout;
   EXPECT_EQ(1U, L.DirectoryBlocks.size());
   EXPECT_EQ(B + 1, L.DirectoryBlocks[0]);
 }