Refactor bitcode reader to simplify control.

[kschimpf]

Summary

Modifies the bitcode reader such that the same logic is used for
both memory buffers and data streams. The incremental parsing
was factored into startParse, continueParse, and finishParse.
All parses (incremental or non-incremental) begin with startParse.
Then zero (or more) calls to continueParse incrementally read more
input, picking up from where the last call left off. finishParse
materializes any additional parts, based on the flags passed to startParse.

[jvoung]

A couple nits for now, but still trying to read through and understand what's happening in the streaming and lazy cases...

lib/Bitcode/Reader/BitcodeReader.cpp

275

http://llvm.org/docs/CodingStandards.html#doxygen-use-in-documentation-comments

says "\brief" instead of @brief

283

lowercase first letter of function name -- should probably do it for these new functions since you're touching it (but leave existing functions alone?)

288

\returns

397

lowercase first letter for new function (and I guess update the commit message if you do)

401

same

407

extra space in between updateParseState and (

409

In the review, I've tried to look for where ParseState gets set, and there are various ways to grep for that...

one is ParseState = X... another is updateParseState(Y, ...), or updateParseState(Z);

Is there any way you can make the number of variations smaller?

424

This is a bit weird to me... you have a bunch of function called "updateParseState" but some variants modify ParseState and some variants don't.

2752

NextUnreadBit is no longer set -- wanted to check if that is okay now (and why)?

2823

NOte -> Note

3241

I don't quite understand how "ShouldMaterializeAll = false" is supposed to work for the streaming case, if this isn't checked until after:

while (ParseState < NoMoreInput) {

  if (std::error_code EC = ContinueParse()) {
    return EC;
  }
}

How do you delay reading until materialize(GV) for streaming?

3245

This used to be early, in "getLazyBitcodeModuleImpl". This looks like it is now happening late in FinishParse after the loop until NoMoreInput.

Why is this okay now? Was the early call to "materializeForwardReferencedFunctions" actually extraneous because of the call in materialize(GV), or what?

Make sure to check the lazy case with blockaddresses for computed gotos, if there isn't already a unittest for that.

4473

no need for extra space

[jvoung]

lib/Bitcode/Reader/BitcodeReader.cpp

235

be consistent about capitalization in comments

one line starts with "parsed input" and another line starts with "Parsed input" =)

257

Don't need explicit anymore, though that transition was a while back so not really related to this CL.

284

Variable name is different from comment "MaterializeAll" vs "ShouldMaterializeAll" -- make them the same? I see that in the actual definition you're trying to avoid conflicting with the field name...

[kschimpf]

• Merge branch 'master' of http://llvm.org/git/llvm into readfac1
• Working version. Save state.
• Cleanup startParse.
• Cleaned up code.
• Fix nit.
• Merge branch 'master' of http://llvm.org/git/llvm into readfac1

[kschimpf]

DId all changes except adding test that forward block address references get resolved on lazy loads.

lib/Bitcode/Reader/BitcodeReader.cpp

235

Done.

257

Done.

275

Sorry, my fault. I followed the syntax of ConstantPlaceHolder below. Changing to follow coding standards.

283

Good point. Fixing.

284

Fixing the names to be consistent. Fixing name conflict by prefixing assignments of field names with "this->".

288

Done.

397

Done.

401

Done.

409

I guess the first part of the problem is that there are 2 notions of parse state:

1. The field ParseState that names the state of the parser, and
2. NextUnreadBit which defines where to continue the parse on return (in case a function body gets parsed between calls).

I also overloaded the return value with this update. Refactoring to do less and be more clear.

424

The refactoring is a bit better now. Hopefully good enough.

2823

Done.

3241

After talking to Derek, I realized what was the issue I was missing. I'l summarize what I understand:

1. When streaming, we want to "return" as soon as possible, without having to force all bitcode to be scanned. This reduces the cost of (potential) blocking calls to the data streamer.

2. Control can return to the caller without having completed the parse. However, the parsed portions must be consistent (i.e. forward block address references have been resolved).

Based on this, I've modified the code to lift the materializeForwardReferencedFunctions into startParse.

3245

I agree that there should be some type of forward reference unit test to verify we can lazy evaluate these forward-referenced block addresses.

I also agree that for the use by llvm-dis, my original code worked because it eventually calls materializeAllPermanently. I think I may have been confused about the full expectation of "streamed" (or lazy) was because of this.

4473

Done.

[• rafael]

include/llvm/Support/StreamingMemoryObject.h

75

Why do you need this? The streamer will return how many bytes were read and can handle a larger request.

Also, why does it need to be part of this patch? It looks like this patch has many independent changes in it.

92

Why the extra logic? If objectsize is known it is the same as BytesRead, no?

[kschimpf]

include/llvm/Support/StreamingMemoryObject.h

75

This was added to handle the case of when one is parsing a wrapped bitcode file. In such cases, you do not need to do another read (which may block until it succeeds). That was the intent of this change. However, a simpler approach would be to allow the extra read, and then not set ObjectSize (below) if already set.

I will remove this change, and add the conditional assignment to ObjectSize below.

I changed it in this CL because it didn't cause a problem until I fixed that materializing a module (when streaming) didn't actually read all of the bitcode file. When that change was added, tests failed and this issue was exposed.

I will remove the changes StreamingMemoryObject.{h,cpp} and put in a separate CL.

92

No, they aren't necessarily the same.

The problem happens when you have a wrapped bitcode file, and was not exposed until I fixed the case that we weren't reading the entire bitcode when materializing lazily. Then, a bunch of test cases failed.

When I looked into it, this is what I discovered:

1. The wrapped bitcode was smaller than kChunkSize. Hence, the initial read set BytesRead to the size of the wrapped file on first read.

2. The wrapper was then read, and set ObjectSize, which corresponded to 4 bytes smaller than BytesRead.

This is the reason I changed this file as I did.

[jvoung]

lib/Bitcode/Reader/BitcodeReader.cpp

243

nit: This is usually 0 or 1, but it seems unexpected for this field to be named "NumModulesParsed", and yet have the type be "bool". Rename or change type?

424

Thanks -- this is better.

For a while I was also wondering how many places need to be aware of setting the state to ParseError, but I think it's just continueParse() because most/all searching for bit position, etc. goes through that.

3132

Does this need to be cleanupOnError(EC) also?

3168

This covers two states? InsideModule and AtTopLevel? It might be more clear if you list them out so it's clear what the "break" corresponds to (AtTopLevel).

Previously, the Stream.JumpToBit(NextUnreadBit); was only needed when InsideModule... is it now needed for AtTopLevel too?

3269

"NoMorInput" -> "NoMoreInput"

It could also be that more states >= NoMoreInput are added as code evolves, but not handled here. Can the compiler accept/handle a "static_assert(ParseState < NoMoreInput, "...") to catch what happens if more states are added after NoMoreInput but not handled by this switch?

4530

Is this necessary at this point? Should that already be covered by the " // Iterate over the module, deserializing any functions that are still on disk" loop?

4534

The "promise" comment from "above" is removed now, so you could update this comment.

[kschimpf]

Fix issues raised by jvoung, and remove changes now in D8907.

[kschimpf]

lib/Bitcode/Reader/BitcodeReader.cpp

243

Good catch. I did meant to use size_t. Fixing.

424

That is correct and was the intent. State updates (and bit positioning) is intentionally now localized to continueParse.

The only exception is in ParseModule, which updates the state to state whether it returned without completing.

3132

Yes. Good catch. Fixing.

3168

The jumpToBit is needed because various "materialize" methods may be called between calls to continueParse. By forcing a jumpToBit to happen at all calls to continueParse, we no longer need to know where the materialize methods leave the bitcursor.

While I did not see an example of an error caused by interleaved calls to materialize, I was very suspicious that they could occur, and wanted to make sure that this would not happen. Hence, I made sure that continueParse always resets the position to where it left off.

I will fix to not use default, so that a corresponding warning will be generated if a new value is added to the enumeration.

3269

Fixed string. Also removed "default" case and made all states explicit. This will force a warning if a new state is added.

4508

Removing the comment about being after a function body. This is no longer true. A call to materializeMetadata would put us some place else in the bitcode file.

4530

In correct bitcode files, you are right. However, if the function doesn't define any function blocks, but (incorrectly) references function block addresses, this code will cause the error to be generated.

However, looking at the following instruction, this is checked anyway. Removing.

4534

Done.

[kschimpf]

Fix issues in diff 23431.

[kschimpf]

• Merge branch 'master' into readfac1
• Fix issues raised by merge.
• Merge branch 'master' of http://llvm.org/git/llvm into readfac1
• Fix tests to use old-style parser.

[kschimpf]

• Fix nits.

[kschimpf]

Now that the issues with the streaming memory object has been fixed, I have updated this CL for review.

Note that I added a CL flag "-old-lazy-bitcode-parser". This was done to deal with a bug fixed by this CL. That is, in the old code, when you materialized a module, it didn't check if there was any additional data in the bitcode file. The new code fixes this by calling "finishParse". However, there are a couple of (bitcode binary) tests that were generated with this violation. Hence, the flag was added to fix this problem.

I'm willing to remove this flag in either (1) a later review, or (2) in a later revision. However, for this review I made the issues explicit so that the problem can be seen.

[filcab]

In D8786#179791, @kschimpf wrote:

However, there are a couple of (bitcode binary) tests that were generated with this violation. Hence, the flag was added to fix this problem.

I'm willing to remove this flag in either (1) a later review, or (2) in a later revision. However, for this review I made the issues explicit so that the problem can be seen.

Let me know which ones have bugs. They might be easy-ish to reconstruct (especially since I have some additional practice in fiddling with bc files, now (Can't promise to deal with them very quickly, though).

lib/Bitcode/Reader/BitcodeReader.cpp

293–294

Why the empty line?

306

http://llvm.org/docs/CodingStandards.html#doxygen-use-in-documentation-comments
^^ This changed recently.

Omit \brief if the brief description is just a sentence. (You might want to add the '.' at the end, though)

771

Omit \brief.

785

Omit \brief.

3190

We're already at top level, no? (line 3234)
I might be missing something, but it looks like we're at top level, and saw an EndBlock. Shouldn't this be an error?

[kschimpf]

• Merge branch 'master' of http://llvm.org/git/llvm into readfac1
• Fix issues raised by filcab.

[kschimpf]

Fixes based on feedback by Filipe.

lib/Bitcode/Reader/BitcodeReader.cpp

293–294

Removed.

771

Done.

785

Done.

3190

I agree that we should be at top level.

I also agree that it appears weird that we allow extra (unmathced) EndBlocks. This has been allowed by the bitcode reader/writer for years. I just wasn't willing to make the leap that I should remove this.

However, I tried removing it (and making it an error), and no tests failed. Hence, Converting this to an error.

4521

Moved the iterating of functions to before the call to finishParse. This deals with the problems I was having with tests in test/Bitcode/invalid.test (i.e. Inputs/invalid-fwdref-type-mismatch-2.bc and Inputs/invalid-load-ptr-type.bc). These two files had multiple errors (the one they intended which was inside a function body, and the one probably not intended - extraneous stuff at the end of the bitcode file).

This removes the need for the command line flag UseOldLazyBitcodeParse, and I have deleted it.

[filcab]

Hi Karl,

The files came straight from the fuzzer, so it is likely have more than one error. If, in order to support them (where support is: keep the test working and diagnosing what we want), you have to change the code in a convoluted way, I would prefer to change the test.

If the change is minimal and not a problem (doesn't impact legibility or architecture), then keeping the tests as they are is not a problem either. I just want to avoid having worse code just so we don't have to re-do some tests.

Of course, if we started crashing on the tests, that's a problem :-)

Thanks,

Filipe

(Phab butchered the comment in my email. Editing it to get a complete history in Phabricator)

[kschimpf]

• Fix invalid bitcode tests with more than one error.
• Merge branch 'master' into readfac1

[kschimpf]

In addition to moving the code back in materializeModule, I fixed three tests. Two of them were fuzz tests where I "truncated" the file to the end of the module block. The other test did not have anything after a bad abbreviation definition, and the code file was incomplete. So I generated a replacement test that was well structured otherwise (i.e. only had that one error in it).

include/llvm/Bitcode/BitstreamReader.h

328 ↗

(On Diff #26923)

This code fixes the state of the bit streamer when no more input is found. As a result, method AtEndOfStream now works correctly.

lib/Bitcode/Reader/BitcodeReader.cpp

3189–3190

Discovered that the Bitstream::EndBLock was "hiding" a bug int the bitstream reader when processing a data stream. That is, when using a data stream, the size is not set until after the eof is reached. Hence, when Stream.AtEndOfStream() was called above, it would return false even when at the eof. The actual problem was in FillCurWord, which did not set the bit position correctly when there was no more input.

The old code worked because the read (at eof) would return zero, and is understood as an end block. By returning success for this value, it would hide this problem.

I also improved the error message so that once can see where the reader thought the eof should be, if there is miscellaneous stuff at the end of the bit code file. This makes it easier to know where to cut a test file in such cases.

4522

Now that the eof checking is fixed, I moved this back where it was in an earlier version of this CL.

[filcab]

Hi Karl,

Really sorry for the delay.
LGTM on my part, as long as you add the test for the error message and do the fix.

Thank you,

Filipe

lib/Bitcode/Reader/BitcodeReader.cpp

772

Nit: Put more words on the first line.

776

Nit: If it's for docs, it's probably best to start with an uppercase letter.

3193

Thank you!

3197

errs()? Or StrBuf?
Please also add a test for this error message.

[kschimpf]

• Merge branch 'master' of http://llvm.org/git/llvm into readfac1
• Fixes associated with review by Filipe.

[kschimpf]

Fixed issues raised by Filipe.

lib/Bitcode/Reader/BitcodeReader.cpp

772

Done.

776

Done.

3197

Good catch. I meant StrBuf, so that we can use the same API for all errors.

[• rafael]

Applying the patch locally to take a better look.

lib/Bitcode/Reader/BitcodeReader.cpp

290–292

Please commit the pure cleanup bits first:

using \ instead of @
starting functions with lowercase names.

3198

This looks a bit much to be honest. Corrupted files are not that common and it is trivial to set a breakpoint to find the state.

3199–3200

This is always 0 or 1. Use a boolean instead.

[• rafael]

I got the following test failures locally:

LLVM :: Bitcode/invalid.test
LLVM :: tools/gold/invalid.ll

[kschimpf]

This CL has gotten a bit long, and hard to read (to many versions). Moved to a new CL http://reviews.llvm.org/D10518

lib/Bitcode/Reader/BitcodeReader.cpp

290–292

I assume this has already been done. The new CL doesn't have suc cases anymore.

3198

Simplified in new CL to do same as before.

3199–3200

This was already fixed in master.

[kschimpf]

This CL has gotten a bit long, and hard to read (to many versions). Moved to a new CL http://reviews.llvm.org/D10518

include/llvm/Support/StreamingMemoryObject.h

public:
   void setKnownObjectSize(size_t size);
 
 private:
   const static uint32_t kChunkSize = 4096 * 4;
   mutable std::vector<unsigned char> Bytes;
   std::unique_ptr<DataStreamer> Streamer;
   mutable size_t BytesRead;   // Bytes read from stream
   size_t BytesSkipped;// Bytes skipped at start of stream (e.g. wrapper/header)
-  mutable size_t ObjectSize; // 0 if unknown, set if wrapper seen or EOF reached
-  mutable bool EOFReached;
-
-  // Fetch enough bytes such that Pos can be read or EOF is reached
-  // (i.e. BytesRead > Pos). Return true if Pos can be read.
-  // Unlike most of the functions in BitcodeReader, returns true on success.
-  // Most of the requests will be small, but we fetch at kChunkSize bytes
-  // at a time to avoid making too many potentially expensive GetBytes calls
+  mutable size_t ObjectSize; // 0 if unknown, set if wrapper seen or end of
+                             // object reached.
+  mutable bool EOOReached;  // end of object reached.
+
+  // Fetch enough bytes such that Pos can be read or end of object is
+  // reached (i.e. BytesRead > Pos). Note: EOF sets end of object if
+  // not already defined.  Returns true if Pos can be read.  Unlike
+  // most of the functions in BitcodeReader, returns true on success.
+  // Most of the requests will be small, but we fetch at kChunkSize
+  // bytes at a time to avoid making too many potentially expensive
+  // GetBytes calls
   bool fetchToPos(size_t Pos) const {
-    if (EOFReached)
+    if (EOOReached)
       return Pos < ObjectSize;
+
     while (Pos >= BytesRead) {
-      Bytes.resize(BytesRead + BytesSkipped + kChunkSize);
+      size_t NextChunkSize = kChunkSize;
+      if (ObjectSize && ObjectSize < BytesRead + kChunkSize) {
+        if (BytesRead >= ObjectSize) {
+          EOOReached = true;
+          return false;
+        }
+        NextChunkSize = ObjectSize - BytesRead;
+      }
+      Bytes.resize(BytesRead + BytesSkipped + NextChunkSize);
       size_t bytes = Streamer->GetBytes(&Bytes[BytesRead + BytesSkipped],
-                                        kChunkSize);
+                                        NextChunkSize);
       BytesRead += bytes;
-      if (bytes != kChunkSize) { // reached EOF/ran out of bytes
+      if (bytes == 0) { // reached EOF/ran out of bytes
         ObjectSize = BytesRead;
-        EOFReached = true;
+        EOOReached = true;
         break;
       }
     }
-    return Pos < BytesRead;
+    return (Pos < BytesRead) || (ObjectSize && Pos < ObjectSize);
   }
 
   StreamingMemoryObject(const StreamingMemoryObject&) = delete;
   void operator=(const StreamingMemoryObject&) = delete;
 };
 
 MemoryObject *getNonStreamedMemoryObject(
     const unsigned char *Start, const unsigned char *End);
 
 }
 #endif  // STREAMINGMEMORYOBJECT_H_

lib/Bitcode/Reader/BitcodeReader.cpp

public:
   void AssignValue(Metadata *MD, unsigned Idx);
   void tryToResolveCycles();
 };
 
 class BitcodeReader : public GVMaterializer {
   LLVMContext &Context;
   DiagnosticHandlerFunction DiagnosticHandler;
   Module *TheModule;
+  // The following two fields define the type of memory to parse.
   std::unique_ptr<MemoryBuffer> Buffer;
+  DataStreamer *Streamer;
   std::unique_ptr<BitstreamReader> StreamFile;
   BitstreamCursor Stream;
-  DataStreamer *LazyStreamer;
-  uint64_t NextUnreadBit;
   bool SeenValueSymbolTable;
 
   std::vector<Type*> TypeList;
   BitcodeReaderValueList ValueList;
   BitcodeReaderMDValueList MDValueList;
   std::vector<Comdat *> ComdatList;
   SmallVector<Instruction *, 64> InstructionList;
 
class BitcodeReader : public GVMaterializer {
   /// for a more compact encoding.  Some instruction operands are not
   /// relative to the instruction ID: basic block numbers, and types.
   /// Once the old style function blocks have been phased out, we would
   /// not need this flag.
   bool UseRelativeIDs;
 
   /// True if all functions will be materialized, negating the need to process
   /// (e.g.) blockaddress forward references.
-  bool WillMaterializeAllForwardRefs;
+  bool WillMaterializeAllForwardRefs = false;
 
   /// Functions that have block addresses taken.  This is usually empty.
   SmallPtrSet<const Function *, 4> BlockAddressesTaken;
 
   /// True if any Metadata block has been materialized.
-  bool IsMetadataMaterialized;
+  bool IsMetadataMaterialized = false;
+
+  /// True if the module is materialized.
+  bool IsModuleMaterialized = false;
+
+  /// True if meta data should initially be skipped.
+  bool ShouldLazyLoadMetadata = false;
+
+  /// True if everything should materialize all before finishing parsing.
+  bool ShouldMaterializeAll = false;
+
+  /// The state of the parse.
+  enum BitcodeReaderState {
+    AtStart,
+    AtTopLevel,    // Processing top-level records.
+    InsideModule,  // processing records inside a module block.
+    // All states below here represent cases where input shouldn't be parsed.
+    NoMoreInput,   // Generic marker for having parsed input.
+    ReachedEof,    // parsed input, but not necessary materializations.
+    FinishedParse, // Parsed input and materialized necessary parts.
+    ParseError,    // An error has occurred, stop parsing.
+  } ParseState = AtStart;
+
+  /// The position (within the bitcode) where parsing left off when
+  /// incrementally parsing.
+  uint64_t NextUnreadBit = 0;
+
+  /// The number of modules read at the top level.
+  bool NumModulesParsed = 0;
 
   bool StripDebugInfo = false;
 
 public:
   std::error_code Error(BitcodeError E, const Twine &Message);
   std::error_code Error(BitcodeError E);
   std::error_code Error(const Twine &Message);
 
-  explicit BitcodeReader(MemoryBuffer *buffer, LLVMContext &C,
+  explicit BitcodeReader(MemoryBuffer *Buffer, LLVMContext &C,
                          DiagnosticHandlerFunction DiagnosticHandler);
-  explicit BitcodeReader(DataStreamer *streamer, LLVMContext &C,
+  explicit BitcodeReader(DataStreamer *Streamer, LLVMContext &C,
                          DiagnosticHandlerFunction DiagnosticHandler);
   ~BitcodeReader() { FreeState(); }
 
   std::error_code materializeForwardReferencedFunctions();
 
   void FreeState();
 
   void releaseBuffer();
 
   bool isDematerializable(const GlobalValue *GV) const override;
   std::error_code materialize(GlobalValue *GV) override;
   std::error_code MaterializeModule(Module *M) override;
   std::vector<StructType *> getIdentifiedStructTypes() const override;
   void Dematerialize(GlobalValue *GV) override;
 
-  /// @brief Main interface to parsing a bitcode buffer.
+  /// @brief Starts an incremental parse for module M. Reads enough to
+  /// define global values. The flags define what should happen before
+  /// finishing the parse. That is:
+  /// ShouldMaterializeAll: When true, the module should be materialized
+  ///    completely. Otherwise, function bodies are only loaded on demand.
+  /// ShouldLazyLoadMetadata: When true, the metadata blocks should be
+  ///    parsed.
   /// @returns true if an error occurred.
-  std::error_code ParseBitcodeInto(Module *M,
-                                   bool ShouldLazyLoadMetadata = false);
+  std::error_code StartParse(Module *M,
+                             bool MaterializeAll,
 
+  /// @brief Parses bitcode. Materializes based on flags.
+  /// @returns true if an error occurred.
+  std::error_code ParseBitcodeInto(Module *M,
+                                   bool ShouleMaterializeAll,
+                                   bool ShouldLazyLoadMetadata);
+
+
   /// @brief Cheap mechanism to just extract module triple
   /// @returns true if an error occurred.
   ErrorOr<std::string> parseTriple();
 
   static uint64_t decodeSignRotatedValue(uint64_t V);
 
   /// Materialize any deferred Metadata block.
   std::error_code materializeMetadata() override;
 
   void setStripDebugInfo() override;
 
 private:
   std::vector<StructType *> IdentifiedStructTypes;
   StructType *createIdentifiedStructType(LLVMContext &Context, StringRef Name);
   StructType *createIdentifiedStructType(LLVMContext &Context);
 
   Type *getTypeByID(unsigned ID);
   Value *getFnValueByID(unsigned ID, Type *Ty) {
     if (Ty && Ty->isMetadataTy())
       return MetadataAsValue::get(Ty->getContext(), getFnMetadataByID(ID));
     return ValueList.getValueFwdRef(ID, Ty);
Value *getValueSigned(SmallVectorImpl<uint64_t> &Record, unsigned Slot,
     if (Slot == Record.size()) return nullptr;
     unsigned ValNo = (unsigned)decodeSignRotatedValue(Record[Slot]);
     // Adjust the ValNo, if it was encoded relative to the InstNum.
     if (UseRelativeIDs)
       ValNo = InstNum - ValNo;
     return getFnValueByID(ValNo, Ty);
   }
 
+  // Continue incremental parse to next skipped block, or eof, whichever
+  // comes first.
+  std::error_code ContinueParse();
+
+  // Finish the parse and then materialize based on flags passed to
+  // StartParse().
+  std::error_code FinishParse();
+
+  // The updateParseState methods update the parse state with the
+  // given information, so that the next call to ContinueParse can
+  // continue. Returns an error code to be returned by
+  // ContinueParse().
+  std::error_code &updateParseState (BitcodeReaderState NewValue,
+                                     std::error_code &EC) {
+    ParseState = EC ? ParseError : NewValue;
+    NextUnreadBit = Stream.GetCurrentBitNo();
+    return EC;
+  }
+
+  std::error_code updateParseState(std::error_code EC) {
+    return updateParseState(ParseState, EC);
+  }
+
+  std::error_code updateParseState(BitcodeReaderState NewValue) {
+    ParseState = NewValue;
+    NextUnreadBit = Stream.GetCurrentBitNo();
+    return std::error_code();
+  }
+
+  std::error_code updateParseState() {
+    NextUnreadBit = Stream.GetCurrentBitNo();
+    return std::error_code();
+  }
+
   /// Converts alignment exponent (i.e. power of two (or zero)) to the
   /// corresponding alignment to use. If alignment is too large, returns
   /// a corresponding error code.
   std::error_code parseAlignmentValue(uint64_t Exponent, unsigned &Alignment);
   std::error_code ParseAttrKind(uint64_t Code, Attribute::AttrKind *Kind);
-  std::error_code ParseModule(bool Resume, bool ShouldLazyLoadMetadata = false);
+  std::error_code ParseModule();
   std::error_code ParseAttributeBlock();
   std::error_code ParseAttributeGroupBlock();
   std::error_code ParseTypeTable();
   std::error_code ParseTypeTableBody();
 
   std::error_code ParseValueSymbolTable();
   std::error_code ParseConstants();
   std::error_code RememberAndSkipFunctionBody();

 
 static DiagnosticHandlerFunction getDiagHandler(DiagnosticHandlerFunction F,
                                                 LLVMContext &C) {
   if (F)
     return F;
   return [&C](const DiagnosticInfo &DI) { C.diagnose(DI); };
 }
 
-BitcodeReader::BitcodeReader(MemoryBuffer *buffer, LLVMContext &C,
+BitcodeReader::BitcodeReader(MemoryBuffer *Buffer, LLVMContext &C,
                              DiagnosticHandlerFunction DiagnosticHandler)
     : Context(C), DiagnosticHandler(getDiagHandler(DiagnosticHandler, C)),
-      TheModule(nullptr), Buffer(buffer), LazyStreamer(nullptr),
-      NextUnreadBit(0), SeenValueSymbolTable(false), ValueList(C),
-      MDValueList(C), SeenFirstFunctionBody(false), UseRelativeIDs(false),
-      WillMaterializeAllForwardRefs(false), IsMetadataMaterialized(false) {}
+      TheModule(nullptr), Buffer(Buffer), Streamer(nullptr),
+      SeenValueSymbolTable(false), ValueList(C),
+      MDValueList(C), SeenFirstFunctionBody(false), UseRelativeIDs(false) {}
 
-BitcodeReader::BitcodeReader(DataStreamer *streamer, LLVMContext &C,
+BitcodeReader::BitcodeReader(DataStreamer *Streamer, LLVMContext &C,
                              DiagnosticHandlerFunction DiagnosticHandler)
     : Context(C), DiagnosticHandler(getDiagHandler(DiagnosticHandler, C)),
-      TheModule(nullptr), Buffer(nullptr), LazyStreamer(streamer),
-      NextUnreadBit(0), SeenValueSymbolTable(false), ValueList(C),
-      MDValueList(C), SeenFirstFunctionBody(false), UseRelativeIDs(false),
-      WillMaterializeAllForwardRefs(false), IsMetadataMaterialized(false) {}
+      TheModule(nullptr), Buffer(nullptr), Streamer(Streamer),
+      SeenValueSymbolTable(false), ValueList(C),
+      MDValueList(C), SeenFirstFunctionBody(false), UseRelativeIDs(false) {}
 
 std::error_code BitcodeReader::materializeForwardReferencedFunctions() {
   if (WillMaterializeAllForwardRefs)
     return std::error_code();
 
   // Prevent recursion.
   WillMaterializeAllForwardRefs = true;
 
static void UpgradeDLLImportExportLinkage(llvm::GlobalValue *GV, unsigned Val) {
   switch (Val) {
   case 5: GV->setDLLStorageClass(GlobalValue::DLLImportStorageClass); break;
   case 6: GV->setDLLStorageClass(GlobalValue::DLLExportStorageClass); break;
   }
 }
 
 namespace llvm {
 namespace {
   /// @brief A class for maintaining the slot number definition
   /// as a placeholder for the actual definition for forward constants defs.
   class ConstantPlaceHolder : public ConstantExpr {
     void operator=(const ConstantPlaceHolder &) = delete;
   public:
     // allocate space for exactly one operand
     void *operator new(size_t s) {
       return User::operator new(s, 1);
     }
     explicit ConstantPlaceHolder(Type *Ty, LLVMContext& Context)
       : ConstantExpr(Ty, Instruction::UserOp1, &Op<0>(), 1) {
       Op<0>() = UndefValue::get(Type::getInt32Ty(Context));
     }
 
     /// @brief Methods to support type inquiry through isa, cast, and dyn_cast.
     static bool classof(const Value *V) {
       return isa<ConstantExpr>(V) &&
              cast<ConstantExpr>(V)->getOpcode() == Instruction::UserOp1;
     }
 
 
     /// Provide fast operand accessors
     DECLARE_TRANSPARENT_OPERAND_ACCESSORS(Value);
std::error_code BitcodeReader::GlobalCleanup() {
 
   // Force deallocation of memory for these vectors to favor the client that
   // want lazy deserialization.
   std::vector<std::pair<GlobalVariable*, unsigned> >().swap(GlobalInits);
   std::vector<std::pair<GlobalAlias*, unsigned> >().swap(AliasInits);
   return std::error_code();
 }
 
-std::error_code BitcodeReader::ParseModule(bool Resume,
-                                           bool ShouldLazyLoadMetadata) {
-  if (Resume)
-    Stream.JumpToBit(NextUnreadBit);
-  else if (Stream.EnterSubBlock(bitc::MODULE_BLOCK_ID))
+std::error_code BitcodeReader::ParseModule() {
+  if (ParseState == AtTopLevel) {
+    if (Stream.EnterSubBlock(bitc::MODULE_BLOCK_ID))
       return Error("Invalid record");
+    ParseState = InsideModule;
+  } else {
+    assert(ParseState == InsideModule);
+  }
 
   SmallVector<uint64_t, 64> Record;
   std::vector<std::string> SectionTable;
   std::vector<std::string> GCTable;
 
   // Read all the records for this module.
   while (1) {
     BitstreamEntry Entry = Stream.advance();
 
     switch (Entry.Kind) {
     case BitstreamEntry::Error:
       return Error("Malformed block");
     case BitstreamEntry::EndBlock:
+      ParseState = AtTopLevel;
       return GlobalCleanup();
 
     case BitstreamEntry::SubBlock:
       switch (Entry.ID) {
       default:  // Skip unknown content.
         if (Stream.SkipBlock())
           return Error("Invalid record");
         break;
case BitstreamEntry::SubBlock:
           std::reverse(FunctionsWithBodies.begin(), FunctionsWithBodies.end());
           if (std::error_code EC = GlobalCleanup())
             return EC;
           SeenFirstFunctionBody = true;
         }
 
         if (std::error_code EC = RememberAndSkipFunctionBody())
           return EC;
-        // For streaming bitcode, suspend parsing when we reach the function
-        // bodies. Subsequent materialization calls will resume it when
-        // necessary. For streaming, the function bodies must be at the end of
-        // the bitcode. If the bitcode file is old, the symbol table will be
-        // at the end instead and will not have been seen yet. In this case,
-        // just finish the parse now.
-        if (LazyStreamer && SeenValueSymbolTable) {
-          NextUnreadBit = Stream.GetCurrentBitNo();
+        // Suspend parsing when we reach a function body, assuming we
+        // have already associated names with global values. NOte: If
+        // the bitcode file is old, the symbol table will be at the
+        // end instead and will not have been seen yet.
+        if (SeenValueSymbolTable)
           return std::error_code();
-        }
         break;
       case bitc::USELIST_BLOCK_ID:
         if (std::error_code EC = ParseUseLists())
           return EC;
         break;
       }
       continue;
 
case bitc::MODULE_CODE_FUNCTION: {
 
       ValueList.push_back(Func);
 
       // If this is a function with a body, remember the prototype we are
       // creating now, so that we can match up the body with them later.
       if (!isProto) {
         Func->setIsMaterializable(true);
         FunctionsWithBodies.push_back(Func);
-        if (LazyStreamer)
-          DeferredFunctionInfo[Func] = 0;
       }
       break;
     }
     // ALIAS: [alias type, aliasee val#, linkage]
     // ALIAS: [alias type, aliasee val#, linkage, visibility, dllstorageclass]
     case bitc::MODULE_CODE_ALIAS: {
       if (Record.size() < 3)
         return Error("Invalid record");
case bitc::MODULE_CODE_PURGEVALS:
       ValueList.shrinkTo(Record[0]);
       break;
     }
     Record.clear();
   }
 }
 
 std::error_code BitcodeReader::ParseBitcodeInto(Module *M,
+                                                bool ShouldMaterializeAll,
                                                 bool ShouldLazyLoadMetadata) {
-  TheModule = nullptr;
+  auto cleanupOnError = [&](std::error_code EC) {
+    releaseBuffer(); // Never take ownership on error.
+    return EC;
+  };
+
+  if (std::error_code EC =
+      StartParse(M, ShouldMaterializeAll, ShouldLazyLoadMetadata))
+    return cleanupOnError(EC);
+
+  if (std::error_code EC = FinishParse())
+    return cleanupOnError(EC);
+
+  return std::error_code();
+}
+
+std::error_code BitcodeReader::StartParse(Module *M,
+                                          bool MaterializeAll,
+                                          bool LazyLoadMetadata) {
+  assert(ParseState == AtStart);
+  TheModule = M;
+  ShouldLazyLoadMetadata = LazyLoadMetadata;
+  ShouldMaterializeAll = MaterializeAll;
 
   if (std::error_code EC = InitStream())
     return EC;
 
   // Sniff for the signature.
   if (Stream.Read(8) != 'B' ||
       Stream.Read(8) != 'C' ||
       Stream.Read(4) != 0x0 ||
       Stream.Read(4) != 0xC ||
       Stream.Read(4) != 0xE ||
       Stream.Read(4) != 0xD)
     return Error("Invalid bitcode signature");
+  return ContinueParse();
+}
+
+std::error_code BitcodeReader::ContinueParse() {
+  switch (ParseState) {
+  case AtStart:
+    ParseState = AtTopLevel;
+    break;
+  case ReachedEof:
+  case FinishedParse:
+    return updateParseState();
+  case ParseError:
+    return updateParseState(
+        Error("Can't continue, bitcode error already found"));
+  default:
+    Stream.JumpToBit(NextUnreadBit);
+    if (ParseState == InsideModule) {
+      std::error_code EC = ParseModule();
+      return updateParseState(EC);
+    }
+    break;
+  }
 
   // We expect a number of well-defined blocks, though we don't necessarily
   // need to understand them all.
   while (1) {
     if (Stream.AtEndOfStream())
-      return std::error_code();
+      return updateParseState(ReachedEof);
 
+    assert(ParseState == AtTopLevel);
     BitstreamEntry Entry =
       Stream.advance(BitstreamCursor::AF_DontAutoprocessAbbrevs);
 
     switch (Entry.Kind) {
     case BitstreamEntry::Error:
-      return Error("Malformed block");
+      return updateParseState(Error("Malformed block"));
     case BitstreamEntry::EndBlock:
-      return std::error_code();
+      return updateParseState(AtTopLevel);
 
     case BitstreamEntry::SubBlock:
       switch (Entry.ID) {
       case bitc::BLOCKINFO_BLOCK_ID:
         if (Stream.ReadBlockInfoBlock())
-          return Error("Malformed block");
+          return updateParseState(Error("Malformed block"));
         break;
       case bitc::MODULE_BLOCK_ID:
         // Reject multiple MODULE_BLOCK's in a single bitstream.
-        if (TheModule)
-          return Error("Invalid multiple blocks");
-        TheModule = M;
-        if (std::error_code EC = ParseModule(false, ShouldLazyLoadMetadata))
-          return EC;
-        if (LazyStreamer)
-          return std::error_code();
+        if (NumModulesParsed++)
+          return updateParseState(Error("Invalid multiple blocks"));
+        if (std::error_code EC = ParseModule())
+          return updateParseState(EC);
+        return updateParseState();
         break;
       default:
         if (Stream.SkipBlock())
-          return Error("Invalid record");
+          return updateParseState(Error("Invalid record"));
         break;
       }
       continue;
     case BitstreamEntry::Record:
       // There should be no records in the top-level of blocks.
 
       // The ranlib in Xcode 4 will align archive members by appending newlines
       // to the end of them. If this file size is a multiple of 4 but not 8, we
       // have to read and ignore these final 4 bytes :-(
       if (Stream.getAbbrevIDWidth() == 2 && Entry.ID == 2 &&
           Stream.Read(6) == 2 && Stream.Read(24) == 0xa0a0a &&
           Stream.AtEndOfStream())
-        return std::error_code();
+        return updateParseState();
+
+      return updateParseState(Error("Invalid record"));
+    }
+  }
+}
+
+std::error_code BitcodeReader::FinishParse() {
+  if (ParseState == FinishedParse)
 
-      return Error("Invalid record");
+  while (ParseState < NoMoreInput) {
+    if (std::error_code EC = ContinueParse()) {
+      return EC;
     }
   }
+
+  assert(TheModule);
+  ParseState = FinishedParse;
+
+  if (ShouldMaterializeAll) {
+    if (std::error_code EC = MaterializeModule(TheModule))
+      return EC;
+  } else {
+    if (std::error_code EC = materializeForwardReferencedFunctions())
+      return EC;
+  }
+
+  return std::error_code();
 }
 
 ErrorOr<std::string> BitcodeReader::parseModuleTriple() {
   if (Stream.EnterSubBlock(bitc::MODULE_BLOCK_ID))
     return Error("Invalid record");
 
   SmallVector<uint64_t, 64> Record;
 
   std::string Triple;
   // Read all the records for this module.
   while (1) {
     BitstreamEntry Entry = Stream.advanceSkippingSubblocks();
 
     switch (Entry.Kind) {
     case BitstreamEntry::SubBlock: // Handled for us already.
     case BitstreamEntry::Error:
       return Error("Malformed block");
     case BitstreamEntry::EndBlock:
       return Triple;
     case BitstreamEntry::Record:
       // The interesting case.
       break;
     }
 
     // Read a record.
     switch (Stream.readRecord(Entry.ID, Record)) {
     default: break;  // Default behavior, ignore unknown content.
     case bitc::MODULE_CODE_TRIPLE: {  // TRIPLE: [strchr x N]
OutOfRecordLoop:
   return std::error_code();
 }
 
 /// Find the function body in the bitcode stream
 std::error_code BitcodeReader::FindFunctionInStream(
     Function *F,
     DenseMap<Function *, uint64_t>::iterator DeferredFunctionInfoIterator) {
   while (DeferredFunctionInfoIterator->second == 0) {
-    if (Stream.AtEndOfStream())
-      return Error("Could not find function in stream");
-    // ParseModule will parse the next body in the stream and set its
+    if (ParseState < NoMoreInput) {
+      // Continue will parse the next body in the stream and set its
       // position in the DeferredFunctionInfo map.
-    if (std::error_code EC = ParseModule(true))
+      if (std::error_code EC = ContinueParse())
         return EC;
+      break;
+    }
+    return Error("Could not find function in stream");
   }
   return std::error_code();
 }
 
 //===----------------------------------------------------------------------===//
 // GVMaterializer implementation
 //===----------------------------------------------------------------------===//
 
 void BitcodeReader::releaseBuffer() { Buffer.release(); }
 
 std::error_code BitcodeReader::materialize(GlobalValue *GV) {
   if (std::error_code EC = materializeMetadata())
     return EC;
 
   Function *F = dyn_cast<Function>(GV);
   // If it's not a function or is already material, ignore the request.
   if (!F || !F->isMaterializable())
     return std::error_code();
 
   DenseMap<Function*, uint64_t>::iterator DFII = DeferredFunctionInfo.find(F);
   assert(DFII != DeferredFunctionInfo.end() && "Deferred function not found!");
   // If its position is recorded as 0, its body is somewhere in the stream
   // but we haven't seen it yet.
-  if (DFII->second == 0 && LazyStreamer)
+  if (DFII->second == 0)
     if (std::error_code EC = FindFunctionInStream(F, DFII))
       return EC;
 
   // Move the bit stream to the saved position of the deferred function body.
   Stream.JumpToBit(DFII->second);
 
   if (std::error_code EC = ParseFunctionBody(F))
     return EC;
if (I->first != I->second) {
         if (CallInst* CI = dyn_cast<CallInst>(*UI++))
           UpgradeIntrinsicCall(CI, I->second);
       }
     }
   }
 
   // Bring in any functions that this function forward-referenced via
   // blockaddresses.
-  return materializeForwardReferencedFunctions();
+  return  materializeForwardReferencedFunctions();
 }
 
 bool BitcodeReader::isDematerializable(const GlobalValue *GV) const {
   const Function *F = dyn_cast<Function>(GV);
   if (!F || F->isDeclaration())
     return false;
 
   // Dematerializing F would leave dangling references that wouldn't be
void BitcodeReader::Dematerialize(GlobalValue *GV) {
   F->dropAllReferences();
   F->setIsMaterializable(true);
 }
 
 std::error_code BitcodeReader::MaterializeModule(Module *M) {
   assert(M == TheModule &&
          "Can only Materialize the Module this BitcodeReader is attached to.");
 
-  if (std::error_code EC = materializeMetadata())
+  if (IsModuleMaterialized)
+    return std::error_code();
+
+  // Set flag now so that FinishParse will not recursively apply this
+  // function.
+  IsModuleMaterialized = true;
+
+  // At this point, if there are any function bodies, the current bit is
+  // pointing to the END_BLOCK record after them. Now make sure the rest
+  // of the bits in the module have been read.
+  if (std::error_code EC = FinishParse())
     return EC;
 
-  // Promise to materialize all forward references.
-  WillMaterializeAllForwardRefs = true;
+  if (std::error_code EC = materializeMetadata())
+    return EC;
 
   // Iterate over the module, deserializing any functions that are still on
   // disk.
   for (Module::iterator F = TheModule->begin(), E = TheModule->end();
        F != E; ++F) {
     if (std::error_code EC = materialize(F))
       return EC;
   }
-  // At this point, if there are any function bodies, the current bit is
-  // pointing to the END_BLOCK record after them. Now make sure the rest
-  // of the bits in the module have been read.
-  if (NextUnreadBit)
-    ParseModule(true);
+
+  if (std::error_code EC = materializeForwardReferencedFunctions())
+    return EC;
 
   // Check that all block address forward references got resolved (as we
   // promised above).
   if (!BasicBlockFwdRefs.empty())
     return Error("Never resolved function from blockaddress");
 
   // Upgrade any intrinsic calls that slipped through (should not happen!) and
   // delete the old functions to clean up. We can't do this unless the entire
   // module is materialized because there could always be another function body
   // with calls to the old function.
   for (std::vector<std::pair<Function*, Function*> >::iterator I =
std::error_code BitcodeReader::MaterializeModule(Module *M) {
   return std::error_code();
 }
 
 std::vector<StructType *> BitcodeReader::getIdentifiedStructTypes() const {
   return IdentifiedStructTypes;
 }
 
 std::error_code BitcodeReader::InitStream() {
-  if (LazyStreamer)
+  if (Streamer)
     return InitLazyStream();
   return InitStreamFromBuffer();
 }
 
 std::error_code BitcodeReader::InitStreamFromBuffer() {
   const unsigned char *BufPtr = (const unsigned char*)Buffer->getBufferStart();
   const unsigned char *BufEnd = BufPtr+Buffer->getBufferSize();
 
std::error_code BitcodeReader::InitStreamFromBuffer() {
   Stream.init(&*StreamFile);
 
   return std::error_code();
 }
 
 std::error_code BitcodeReader::InitLazyStream() {
   // Check and strip off the bitcode wrapper; BitstreamReader expects never to
   // see it.
-  auto OwnedBytes = llvm::make_unique<StreamingMemoryObject>(LazyStreamer);
+  auto OwnedBytes = llvm::make_unique<StreamingMemoryObject>(Streamer);
   StreamingMemoryObject &Bytes = *OwnedBytes;
   StreamFile = llvm::make_unique<BitstreamReader>(std::move(OwnedBytes));
   Stream.init(&*StreamFile);
 
   unsigned char buf[16];
   if (Bytes.readBytes(buf, 16, 0) != 16)
     return Error("Invalid bitcode signature");
 
getLazyBitcodeModuleImpl(std::unique_ptr<MemoryBuffer> &&Buffer,
                          LLVMContext &Context, bool WillMaterializeAll,
                          DiagnosticHandlerFunction DiagnosticHandler,
                          bool ShouldLazyLoadMetadata = false) {
   Module *M = new Module(Buffer->getBufferIdentifier(), Context);
   BitcodeReader *R =
       new BitcodeReader(Buffer.get(), Context, DiagnosticHandler);
   M->setMaterializer(R);
 
-  auto cleanupOnError = [&](std::error_code EC) {
+  // Delay parsing Metadata if ShouldLazyLoadMetadata is true.
+  if (std::error_code EC =
+      R->ParseBitcodeInto(M, WillMaterializeAll, ShouldLazyLoadMetadata)) {
     R->releaseBuffer(); // Never take ownership on error.
     delete M;  // Also deletes R.
     return EC;
-  };
-
-  // Delay parsing Metadata if ShouldLazyLoadMetadata is true.
-  if (std::error_code EC = R->ParseBitcodeInto(M, ShouldLazyLoadMetadata))
-    return cleanupOnError(EC);
-
-  if (!WillMaterializeAll)
-    // Resolve forward references from blockaddresses.
-    if (std::error_code EC = R->materializeForwardReferencedFunctions())
-      return cleanupOnError(EC);
+  }
 
   Buffer.release(); // The BitcodeReader owns it now.
   return M;
 }
 
 ErrorOr<Module *>
 llvm::getLazyBitcodeModule(std::unique_ptr<MemoryBuffer> &&Buffer,
                            LLVMContext &Context,
                            DiagnosticHandlerFunction DiagnosticHandler,
                            bool ShouldLazyLoadMetadata) {
   return getLazyBitcodeModuleImpl(std::move(Buffer), Context, false,
                                   DiagnosticHandler, ShouldLazyLoadMetadata);
 }
 
 ErrorOr<std::unique_ptr<Module>>
 llvm::getStreamedBitcodeModule(StringRef Name, DataStreamer *Streamer,
                                LLVMContext &Context,
                                DiagnosticHandlerFunction DiagnosticHandler) {
   std::unique_ptr<Module> M = make_unique<Module>(Name, Context);
   BitcodeReader *R = new BitcodeReader(Streamer, Context, DiagnosticHandler);
   M->setMaterializer(R);
-  if (std::error_code EC = R->ParseBitcodeInto(M.get()))
+  if (std::error_code EC = R->ParseBitcodeInto(M.get(), false, false))
     return EC;
   return std::move(M);
 }
 
 ErrorOr<Module *>
 llvm::parseBitcodeFile(MemoryBufferRef Buffer, LLVMContext &Context,
                        DiagnosticHandlerFunction DiagnosticHandler) {
   std::unique_ptr<MemoryBuffer> Buf = MemoryBuffer::getMemBuffer(Buffer, false);
   ErrorOr<Module *> ModuleOrErr = getLazyBitcodeModuleImpl(
       std::move(Buf), Context, true, DiagnosticHandler);
   if (!ModuleOrErr)
     return ModuleOrErr;
   Module *M = ModuleOrErr.get();
-  // Read in the entire module, and destroy the BitcodeReader.
-  if (std::error_code EC = M->materializeAllPermanently()) {
-    delete M;
-    return EC;
-  }
 
   // TODO: Restore the use-lists to the in-memory state when the bitcode was
   // written.  We must defer until the Module has been fully materialized.
 
   return M;
 }
 
 std::string

lib/Bitcode/Reader/BitstreamReader.cpp

switch (readRecord(Entry.ID, Record)) {
           Name += (char)Record[i];
         CurBlockInfo->RecordNames.push_back(std::make_pair((unsigned)Record[0],
                                                            Name));
         break;
       }
     }
   }
 }
-

lib/Support/StreamingMemoryObject.cpp

uint64_t StreamingMemoryObject::getExtent() const {
   // keep fetching until we run out of bytes
   while (fetchToPos(pos)) pos += kChunkSize;
   return ObjectSize;
 }
 
 uint64_t StreamingMemoryObject::readBytes(uint8_t *Buf, uint64_t Size,
                                           uint64_t Address) const {
   fetchToPos(Address + Size - 1);
-  if (Address >= BytesRead)
+  if (Address >= BytesRead || (ObjectSize && Address >= ObjectSize))
     return 0;
 
   uint64_t End = Address + Size;
-  if (End > BytesRead)
+  if (ObjectSize) {
+    if (End > ObjectSize) {
+      End = ObjectSize;
+    }
+  } else if (End > BytesRead)
     End = BytesRead;
-  assert(static_cast<int64_t>(End - Address) >= 0);
+  assert(End >= Address);
   Size = End - Address;
   memcpy(Buf, &Bytes[Address + BytesSkipped], Size);
   return Size;
 }
 
 bool StreamingMemoryObject::dropLeadingBytes(size_t s) {
   if (BytesRead < s) return true;
   BytesSkipped = s;
   BytesRead -= s;
   return false;
 }
 
 void StreamingMemoryObject::setKnownObjectSize(size_t size) {
   ObjectSize = size;
   Bytes.reserve(size);
 }
 
 MemoryObject *getNonStreamedMemoryObject(const unsigned char *Start,
                                          const unsigned char *End) {
   return new RawMemoryObject(Start, End);
 }
 
 StreamingMemoryObject::StreamingMemoryObject(DataStreamer *streamer) :
   Bytes(kChunkSize), Streamer(streamer), BytesRead(0), BytesSkipped(0),
-  ObjectSize(0), EOFReached(false) {
+  ObjectSize(0), EOOReached(false) {
   BytesRead = streamer->GetBytes(&Bytes[0], kChunkSize);
 }
 }