Allow for unfinished #if blocks in preambles.

[erikjv]

Summary

Previously, a preamble only included #if blocks (and friends like
ifdef) if there was a corresponding #endif before any declaration or
definition. The problem is that any header file that uses include guards
will not have a preamble generated, which can make code-completion very
slow.

To prevent errors about unbalanced preprocessor conditionals in the
preamble, and unbalanced preprocessor conditionals after a preamble
containing unfinished conditionals, the conditional stack is stored
in the pch file.

This fixes PR26045.

[bkramer]

After pondering this patch for a while I couldn't come up with a case where this would cause problems, but it provides great improvements for editing headers in an interactive setup. So LGTM from my side.

[erikjv]

This will give errors about unbalanced #if/#endif for header guards.

[erikjv]

This version stores/loads the conditional stack in the preprocessor the the generated pch file. It suppresses errors about unfinished conditional preprocessor blocks, and after restoring, it will check if the block is closed in the main file.

[Eugene.Zelenko]

Looks like patch was not committed.

[erikjv]

Yes, the patch was not committed, because there were errors in it. So, I fixed it and put up a new version, but I have no idea how to reset the state.

Still waiting for rsmith to review...

[rsmith]

include/clang/Lex/Preprocessor.h

287–290

What's the difference between the Off and Done states? They seem to be the same to me?

321

{ on previous line.

330

Start function names with a lowercase letter.

333

We don't really care much about the Preprocessor object getting a few dozen bytes larger, since there will typically only be one of them. I don't think the complexity of a dynamically-allocated stack and a PointerIntPair is worthwhile here. Just store a Stack and a State directly as members.

1966–1970

Start function names with a lowercase letter.

lib/Lex/Lexer.cpp

626

You can remove the special-case handling for #if/#else/#endif here.

2521–2524

We should only do this if we reach the end of the main source file. If we reach the end of a #include'd file with a #if still open, we should diagnose that.

lib/Serialization/ASTReader.cpp

2902

Why can't we set the conditional stack on the PPLexer directly from here? (Why do we need to store it separately?)

[erikjv]

lib/Serialization/ASTReader.cpp

2902

Because there is no PPLexer yet. The deserialization is kicked off by ASTUnit::CodeComplete in Act->BeginSourceFile, while the lexer is created in Act->Execute.

[rsmith]

include/clang/Lex/Preprocessor.h

310

Return an ArrayRef rather than exposing the type of the storage (which is an implementation detail), here and transitively through the callers of this.

322

Please pass in an ArrayRef here rather than a SmallVector.

328

I don't see a need to heap-allocate this separately from the Preprocessor.

include/clang/Lex/PreprocessorOptions.h

84

Please add a doc comment for this option. Also, maybe GeneratePreamble to avoid ambiguity as to whether this is some kind of generation number for the preamble?

lib/Lex/PPLexerChange.cpp

139–142

I think this would make more sense two callers up, in Preprocessor::EnterMainSourceFile

[arphaman]

Thanks for working on this! I have some comments below:

include/clang/Lex/Preprocessor.h

291

setState is redundant IMHO, just assign to ConditionalStackState when you use setState below.

301

the const is redundant for the return type.

310

You can pass ArrayRef by value.

1974

the const is redundant for the return type.

1978

Please use ArrayRef instead of SmallVector here and in the function below.

include/clang/Lex/PreprocessorLexer.h

181

ArrayRef can be passed by value.

include/clang/Lex/PreprocessorOptions.h

87

Typo: save

lib/Serialization/ASTReader.cpp

2896

Loc should be capitalized.

lib/Serialization/ASTWriter.cpp

2269

I think you should add an assertion here that verifies that the ASTWriter isn't creating a module.

[erikjv]

Fixed with the next patch.

[nik]

Ping?!

[rsmith]

Some comments, but I'm happy for you to go ahead and commit after addressing them. Thanks!

include/clang/Lex/Preprocessor.h

1975

Please put the { on the previous line and the } on the next line. We don't use this "sandwich braces" style in clang except when defining the function on the same line as the declaration.

include/clang/Lex/PreprocessorLexer.h

182

{ on the previous line, please.

lib/Lex/Lexer.cpp

2522

This should be in the isInPreamble() conditional below, too. We don't want to make a redundant copy of the ConditionalStack at the end of each #included file, just at the end of the overall preamble.

lib/Lex/PPLexerChange.cpp

49

I think this would be better named as isInMainFile: we don't care whether we're in the preamble, or even if there is one, here (the caller checks that).

include/clang/Lex/Preprocessor.h

class Preprocessor {
 
   /// \brief The number of bytes that we will initially skip when entering the
   /// main file, along with a flag that indicates whether skipping this number
   /// of bytes will place the lexer at the start of a line.
   ///
   /// This is used when loading a precompiled preamble.
   std::pair<int, bool> SkipMainFilePreamble;
 
+  class PreambleConditionalStackStore {
+    enum State {
+      Off = 0,
+      Recording = 1,
+      Replaying = 2,
+    };
+
+    void setState(State s) { ConditionalStackState = s; }
+
+  public:
+    PreambleConditionalStackStore() : ConditionalStackState(Off) {}
+
+    void startRecording() { ConditionalStackState = Recording; }
+    void startReplaying() { ConditionalStackState = Replaying; }
+    bool isRecording() const { return ConditionalStackState == Recording; }
+    bool isReplaying() const { return ConditionalStackState == Replaying; }
+
+    const ArrayRef<PPConditionalInfo> getStack() const {
+      return ConditionalStack;
+    }
+
+    void doneReplaying() {
+      ConditionalStack.clear();
+      setState(Off);
+    }
+
+    void setStack(const ArrayRef<PPConditionalInfo> &s) {
+      if (!isRecording() && !isReplaying())
+        return;
+      ConditionalStack.clear();
+      ConditionalStack.append(s.begin(), s.end());
+    }
+
+    bool hasRecordedPreamble() const { return !ConditionalStack.empty(); }
+
+  private:
+    SmallVector<PPConditionalInfo, 4> ConditionalStack;
+    State ConditionalStackState;
+  } PreambleConditionalStack;
+
   /// \brief The current top of the stack that we're lexing from if
   /// not expanding a macro and we are lexing directly from source code.
   ///
   /// Only one of CurLexer, CurPTHLexer, or CurTokenLexer will be non-null.
   std::unique_ptr<Lexer> CurLexer;
 
   /// \brief The current top of stack that we're lexing from if
   /// not expanding from a macro and we are lexing from a PTH cache.
   ///
   /// Only one of CurLexer, CurPTHLexer, or CurTokenLexer will be non-null.
   std::unique_ptr<PTHLexer> CurPTHLexer;
 
   /// \brief The current top of the stack what we're lexing from
   /// if not expanding a macro.
   ///
   /// This is an alias for either CurLexer or  CurPTHLexer.
   PreprocessorLexer *CurPPLexer;
 
public:
   ///
   /// This allows us to implement \#include_next and find directory-specific
   /// properties.
   const DirectoryLookup *GetCurDirLookup() { return CurDirLookup; }
 
   /// \brief Return true if we're in the top-level file, not in a \#include.
   bool isInPrimaryFile() const;
 
+  /// \brief Return true if we're in the 1-level deep \#include. This is used
+  /// by the lexer to determine if it needs to generate errors about
+  /// unterminated \#if directives.
+  bool isInPreamble() const;
+
   /// \brief Handle cases where the \#include name is expanded
   /// from a macro as multiple tokens, which need to be glued together. 
   ///
   /// This occurs for code like:
   /// \code
   ///    \#define FOO <x/y.h>
   ///    \#include FOO
   /// \endcode
public:
   ///        effect occurred (eg, where a desired entity was declared).
   ///
   /// \return A file that can be #included to import a module containing MLoc.
   ///         Null if no such file could be determined or if a #include is not
   ///         appropriate.
   const FileEntry *getModuleHeaderToIncludeForDiagnostics(SourceLocation IncLoc,
                                                           SourceLocation MLoc);
 
+  bool isRecordingPreamble() const {
+    return PreambleConditionalStack.isRecording();
+  }
+
+  bool hasRecordedPreamble() const {
+    return PreambleConditionalStack.hasRecordedPreamble();
+  }
+
+  const ArrayRef<PPConditionalInfo> getPreambleConditionalStack() const
+  { return PreambleConditionalStack.getStack(); }
+
+  void setRecordedPreambleConditionalStack(
+      const SmallVector<PPConditionalInfo, 4> &s) {
+    PreambleConditionalStack.setStack(s);
+  }
+
+  void setReplayablePreambleConditionalStack(
+      const SmallVector<PPConditionalInfo, 4> &s) {
+    PreambleConditionalStack.startReplaying();
+    PreambleConditionalStack.setStack(s);
+  }
+
 private:
   // Macro handling.
   void HandleDefineDirective(Token &Tok, bool ImmediatelyAfterTopLevelIfndef);
   void HandleUndefDirective();
 
   // Conditional Inclusion.
   void HandleIfdefDirective(Token &Tok, bool isIfndef,
                             bool ReadAnyTokensBeforeDirective);

include/clang/Lex/PreprocessorLexer.h

 ///
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_CLANG_LEX_PREPROCESSORLEXER_H
 #define LLVM_CLANG_LEX_PREPROCESSORLEXER_H
 
 #include "clang/Lex/MultipleIncludeOpt.h"
 #include "clang/Lex/Token.h"
+#include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/SmallVector.h"
 
 namespace clang {
 
 class FileEntry;
 class Preprocessor;
 
 class PreprocessorLexer {
typedef SmallVectorImpl<PPConditionalInfo>::const_iterator
     conditional_iterator;
 
   conditional_iterator conditional_begin() const { 
     return ConditionalStack.begin(); 
   }
   conditional_iterator conditional_end() const { 
     return ConditionalStack.end(); 
   }
+
+  void setConditionalLevels(const ArrayRef<PPConditionalInfo> &CL)
+  {
+    ConditionalStack.clear();
+    ConditionalStack.append(CL.begin(), CL.end());
+  }
 };
 
 }  // end namespace clang
 
 #endif

include/clang/Lex/PreprocessorOptions.h

public:
   std::set<std::string> DeserializedPCHDeclsToErrorOn;
 
   /// \brief If non-zero, the implicit PCH include is actually a precompiled
   /// preamble that covers this number of bytes in the main source file.
   ///
   /// The boolean indicates whether the preamble ends at the start of a new
   /// line.
   std::pair<unsigned, bool> PrecompiledPreambleBytes;
-  
+  /// \brief True indicates that a preamble is being generated.
+  ///
+  /// When the lexer is done, one of the things that need to be preserved is the
+  /// conditional #if stack, so the ASTWriter/ASTReader can safe/restore it when
+  /// processing the rest of the file.
+  bool GeneratePreamble;
+
   /// The implicit PTH input included at the start of the translation unit, or
   /// empty.
   std::string ImplicitPTHInclude;
 
   /// If given, a PTH cache file to use for speeding up header parsing.
   std::string TokenCache;
 
   /// \brief True if the SourceManager should report the original file name for
public:
   std::shared_ptr<FailedModulesSet> FailedModules;
 
 public:
   PreprocessorOptions() : UsePredefines(true), DetailedRecord(false),
                           DisablePCHValidation(false),
                           AllowPCHWithCompilerErrors(false),
                           DumpDeserializedPCHDecls(false),
                           PrecompiledPreambleBytes(0, true),
+                          GeneratePreamble(false),
                           RemappedFilesKeepOriginalName(true),
                           RetainRemappedFileBuffers(false),
                           ObjCXXARCStandardLibrary(ARCXX_nolib) { }
 
   void addMacroDef(StringRef Name) { Macros.emplace_back(Name, false); }
   void addMacroUndef(StringRef Name) { Macros.emplace_back(Name, true); }
   void addRemappedFile(StringRef From, StringRef To) {
     RemappedFiles.emplace_back(From, To);

include/clang/Serialization/ASTBitCodes.h

enum ASTRecordTypes {
 
       /// \brief Record code for types associated with OpenCL extensions.
       OPENCL_EXTENSION_TYPES = 58,
 
       /// \brief Record code for declarations associated with OpenCL extensions.
       OPENCL_EXTENSION_DECLS = 59,
 
       MODULAR_CODEGEN_DECLS = 60,
+
+      /// \brief The stack of open #ifs/#ifdefs recorded in a preamble.
+      PP_CONDITIONAL_STACK = 61,
     };
 
     /// \brief Record types used within a source manager block.
     enum SourceManagerRecordTypes {
       /// \brief Describes a source location entry (SLocEntry) for a
       /// file.
       SM_SLOC_FILE_ENTRY = 1,
       /// \brief Describes a source location entry (SLocEntry) for a

lib/Frontend/ASTUnit.cpp

ASTUnit *ASTUnit::LoadFromCommandLine(
   // Override any files that need remapping
   for (const auto &RemappedFile : RemappedFiles) {
     CI->getPreprocessorOpts().addRemappedFile(RemappedFile.first,
                                               RemappedFile.second);
   }
   PreprocessorOptions &PPOpts = CI->getPreprocessorOpts();
   PPOpts.RemappedFilesKeepOriginalName = RemappedFilesKeepOriginalName;
   PPOpts.AllowPCHWithCompilerErrors = AllowPCHWithCompilerErrors;
+  PPOpts.GeneratePreamble = PrecompilePreambleAfterNParses != 0;
   
   // Override the resources path.
   CI->getHeaderSearchOpts().ResourceDir = ResourceFilesPath;
 
   CI->getFrontendOpts().SkipFunctionBodies = SkipFunctionBodies;
 
   if (ModuleFormat)
     CI->getHeaderSearchOpts().ModuleFormat = ModuleFormat.getValue();

lib/Lex/Lexer.cpp

assert(FileLocInfo.first == BeginFileLocInfo.first &&
         FileLocInfo.second >= BeginFileLocInfo.second);
  return Loc.getLocWithOffset(BeginFileLocInfo.second - FileLocInfo.second);
 }
 
 namespace {
 
   enum PreambleDirectiveKind {
     PDK_Skipped,
-    PDK_StartIf,
-    PDK_EndIf,
     PDK_Unknown
   };
 
 } // end anonymous namespace
 
 std::pair<unsigned, bool> Lexer::ComputePreamble(StringRef Buffer,
                                                  const LangOptions &LangOpts,
                                                  unsigned MaxLines) {
   // Create a lexer starting at the beginning of the file. Note that we use a
   // "fake" file source location at offset 1 so that the lexer will track our
   // position within the file.
   const unsigned StartOffset = 1;
   SourceLocation FileLoc = SourceLocation::getFromRawEncoding(StartOffset);
   Lexer TheLexer(FileLoc, LangOpts, Buffer.begin(), Buffer.begin(),
                  Buffer.end());
   TheLexer.SetCommentRetentionState(true);
 
   // StartLoc will differ from FileLoc if there is a BOM that was skipped.
   SourceLocation StartLoc = TheLexer.getSourceLocation();
 
   bool InPreprocessorDirective = false;
   Token TheTok;
-  Token IfStartTok;
-  unsigned IfCount = 0;
   SourceLocation ActiveCommentLoc;
 
   unsigned MaxLineOffset = 0;
   if (MaxLines) {
     const char *CurPtr = Buffer.begin();
     unsigned CurLine = 0;
     while (CurPtr != Buffer.end()) {
       char ch = *CurPtr++;
if (TheTok.isAtStartOfLine() && TheTok.getKind() == tok::hash) {
       ActiveCommentLoc = SourceLocation();
       
       // Figure out which directive this is. Since we're lexing raw tokens,
       // we don't have an identifier table available. Instead, just look at
       // the raw identifier to recognize and categorize preprocessor directives.
       TheLexer.LexFromRawLexer(TheTok);
       if (TheTok.getKind() == tok::raw_identifier && !TheTok.needsCleaning()) {
         StringRef Keyword = TheTok.getRawIdentifier();
         PreambleDirectiveKind PDK
           = llvm::StringSwitch<PreambleDirectiveKind>(Keyword)
               .Case("include", PDK_Skipped)
               .Case("__include_macros", PDK_Skipped)
               .Case("define", PDK_Skipped)
               .Case("undef", PDK_Skipped)
               .Case("line", PDK_Skipped)
               .Case("error", PDK_Skipped)
               .Case("pragma", PDK_Skipped)
               .Case("import", PDK_Skipped)
               .Case("include_next", PDK_Skipped)
               .Case("warning", PDK_Skipped)
               .Case("ident", PDK_Skipped)
               .Case("sccs", PDK_Skipped)
               .Case("assert", PDK_Skipped)
               .Case("unassert", PDK_Skipped)
-              .Case("if", PDK_StartIf)
-              .Case("ifdef", PDK_StartIf)
-              .Case("ifndef", PDK_StartIf)
+              .Case("if", PDK_Skipped)
+              .Case("ifdef", PDK_Skipped)
+              .Case("ifndef", PDK_Skipped)
               .Case("elif", PDK_Skipped)
               .Case("else", PDK_Skipped)
-              .Case("endif", PDK_EndIf)
+              .Case("endif", PDK_Skipped)
               .Default(PDK_Unknown);
 
         switch (PDK) {
         case PDK_Skipped:
           continue;
 
-        case PDK_StartIf:
-          if (IfCount == 0)
-            IfStartTok = HashTok;
-            
-          ++IfCount;
-          continue;
-            
-        case PDK_EndIf:
-          // Mismatched #endif. The preamble ends here.
-          if (IfCount == 0)
-            break;
-
-          --IfCount;
-          continue;
-            
         case PDK_Unknown:
           // We don't know what this directive is; stop at the '#'.
           break;
         }
       }
       
       // We only end up here if we didn't recognize the preprocessor
       // directive or it was one that can't occur in the preamble at this
       // point. Roll back the current token to the location of the '#'.
       InPreprocessorDirective = false;
       TheTok = HashTok;
     }
 
     // We hit a token that we don't recognize as being in the
     // "preprocessing only" part of the file, so we're no longer in
     // the preamble.
     break;
   } while (true);
   
   SourceLocation End;
-  if (IfCount)
-    End = IfStartTok.getLocation();
-  else if (ActiveCommentLoc.isValid())
+  if (ActiveCommentLoc.isValid())
     End = ActiveCommentLoc; // don't truncate a decl comment.
   else
     End = TheTok.getLocation();
 
   return std::make_pair(End.getRawEncoding() - StartLoc.getRawEncoding(),
-                        IfCount? IfStartTok.isAtStartOfLine()
-                               : TheTok.isAtStartOfLine());
+                        TheTok.isAtStartOfLine());
 }
 
 /// AdvanceToTokenCharacter - Given a location that specifies the start of a
 /// token, return a new location that specifies a character within the token.
 SourceLocation Lexer::AdvanceToTokenCharacter(SourceLocation TokStart,
                                               unsigned CharNo,
                                               const SourceManager &SM,
                                               const LangOptions &LangOpts) {
bool Lexer::LexEndOfFile(Token &Result, const char *CurPtr) {
   // that put us in raw mode handle the event.
   if (isLexingRawMode()) {
     Result.startToken();
     BufferPtr = BufferEnd;
     FormTokenWithChars(Result, BufferEnd, tok::eof);
     return true;
   }
   
+  if (PP->isRecordingPreamble()) {
+    PP->setRecordedPreambleConditionalStack(ConditionalStack);
+    if (PP->isInPreamble())
+      ConditionalStack.clear();
+  }
+
   // Issue diagnostics for unterminated #if and missing newline.
 
   // If we are in a #if directive, emit an error.
   while (!ConditionalStack.empty()) {
     if (PP->getCodeCompletionFileLoc() != FileLoc)
       PP->Diag(ConditionalStack.back().IfLoc,
                diag::err_pp_unterminated_conditional);
     ConditionalStack.pop_back();

lib/Lex/PPLexerChange.cpp

bool Preprocessor::isInPrimaryFile() const {
   assert(IsFileLexer(IncludeMacroStack[0]) &&
          "Top level include stack isn't our primary lexer?");
   return std::none_of(IncludeMacroStack.begin() + 1, IncludeMacroStack.end(),
                       [this](const IncludeStackInfo &ISI) -> bool {
     return IsFileLexer(ISI);
   });
 }
 
+bool Preprocessor::isInPreamble() const {
+  if (IsFileLexer())
+    return IncludeMacroStack.size() == 1;
+  return false;
+}
+
 /// getCurrentLexer - Return the current file lexer being lexed from.  Note
 /// that this ignores any potentially active macro expansions and _Pragma
 /// expansions going on at the time.
 PreprocessorLexer *Preprocessor::getCurrentFileLexer() const {
   if (IsFileLexer())
     return CurPPLexer;
 
   // Look for a stacked lexer.
void Preprocessor::EnterSourceFileWithLexer(Lexer *TheLexer,
   if (Callbacks && !CurLexer->Is_PragmaLexer) {
     SrcMgr::CharacteristicKind FileType =
        SourceMgr.getFileCharacteristic(CurLexer->getFileLoc());
 
     Callbacks->FileChanged(CurLexer->getFileLoc(),
                            PPCallbacks::EnterFile, FileType);
   }
 }
 
 /// EnterSourceFileWithPTH - Add a source file to the top of the include stack
 /// and start getting tokens from it using the PTH cache.
 void Preprocessor::EnterSourceFileWithPTH(PTHLexer *PL,
                                           const DirectoryLookup *CurDir) {
 
   if (CurPPLexer || CurTokenLexer)
     PushIncludeMacroStack();
 
   CurDirLookup = CurDir;
   CurPTHLexer.reset(PL);
   CurPPLexer = CurPTHLexer.get();

lib/Lex/Preprocessor.cpp

if(LangOpts.Borland) {
     Ident_AbnormalTermination    = getIdentifierInfo("AbnormalTermination");
   } else {
     Ident__exception_info = Ident__exception_code = nullptr;
     Ident__abnormal_termination = Ident___exception_info = nullptr;
     Ident___exception_code = Ident___abnormal_termination = nullptr;
     Ident_GetExceptionInfo = Ident_GetExceptionCode = nullptr;
     Ident_AbnormalTermination = nullptr;
   }
+
+  if (this->PPOpts->GeneratePreamble)
+    PreambleConditionalStack.startRecording();
 }
 
 Preprocessor::~Preprocessor() {
   assert(BacktrackPositions.empty() && "EnableBacktrack/Backtrack imbalance!");
 
   IncludeMacroStack.clear();
 
   // Destroy any macro definitions.
std::unique_ptr<llvm::MemoryBuffer> SB =
     llvm::MemoryBuffer::getMemBufferCopy(Predefines, "<built-in>");
   assert(SB && "Cannot create predefined source buffer");
   FileID FID = SourceMgr.createFileID(std::move(SB));
   assert(FID.isValid() && "Could not create FileID for predefines?");
   setPredefinesFileID(FID);
 
   // Start parsing the predefines.
   EnterSourceFile(FID, nullptr, SourceLocation());
+
+  // Restore the conditional stack from the preamble, if there is one.
+  if (PreambleConditionalStack.isReplaying()) {
+    CurPPLexer->setConditionalLevels(PreambleConditionalStack.getStack());
+    PreambleConditionalStack.doneReplaying();
+  }
 }
 
 void Preprocessor::EndSourceFile() {
   // Notify the client that we reached the end of the source file.
   if (Callbacks)
     Callbacks->EndOfMainFile();
 }
 

lib/Serialization/ASTReader.cpp

case REFERENCED_SELECTOR_POOL:
           ReferencedSelectorsData.push_back(getGlobalSelectorID(F,
                                                                 Record[Idx++]));
           ReferencedSelectorsData.push_back(ReadSourceLocation(F, Record, Idx).
                                               getRawEncoding());
         }
       }
       break;
 
+    case PP_CONDITIONAL_STACK:
+      if (!Record.empty()) {
+        SmallVector<PPConditionalInfo, 4> ConditionalStack;
+        for (unsigned Idx = 0, N = Record.size() - 1; Idx < N; /* in loop */) {
+          auto loc = ReadSourceLocation(F, Record, Idx);
+          bool WasSkipping = Record[Idx++];
+          bool FoundNonSkip = Record[Idx++];
+          bool FoundElse = Record[Idx++];
+          ConditionalStack.push_back({ loc, WasSkipping, FoundNonSkip, FoundElse });
+        }
+        PP.setReplayablePreambleConditionalStack(ConditionalStack);
+      }
+      break;
+
     case PP_COUNTER_VALUE:
       if (!Record.empty() && Listener)
         Listener->ReadCounter(F, Record[0]);
       break;
 
     case FILE_SORTED_DECLS:
       F.FileSortedDecls = (const DeclID *)Blob.data();
       F.NumFileSortedDecls = Record[0];

lib/Serialization/ASTWriter.cpp

#define RECORD(X) EmitRecordID(X, #X, Stream, Record)
   RECORD(UNDEFINED_BUT_USED);
   RECORD(LATE_PARSED_TEMPLATE);
   RECORD(OPTIMIZE_PRAGMA_OPTIONS);
   RECORD(MSSTRUCT_PRAGMA_OPTIONS);
   RECORD(POINTERS_TO_MEMBERS_PRAGMA_OPTIONS);
   RECORD(UNUSED_LOCAL_TYPEDEF_NAME_CANDIDATES);
   RECORD(DELETE_EXPRS_TO_ANALYZE);
   RECORD(CUDA_PRAGMA_FORCE_HOST_DEVICE_DEPTH);
+  RECORD(PP_CONDITIONAL_STACK);
 
   // SourceManager Block.
   BLOCK(SOURCE_MANAGER_BLOCK);
   RECORD(SM_SLOC_FILE_ENTRY);
   RECORD(SM_SLOC_BUFFER_ENTRY);
   RECORD(SM_SLOC_BUFFER_BLOB);
   RECORD(SM_SLOC_BUFFER_BLOB_COMPRESSED);
   RECORD(SM_SLOC_EXPANSION_ENTRY);
void ASTWriter::WritePreprocessor(const Preprocessor &PP, bool IsModule) {
   RecordData ModuleMacroRecord;
 
   // If the preprocessor __COUNTER__ value has been bumped, remember it.
   if (PP.getCounterValue() != 0) {
     RecordData::value_type Record[] = {PP.getCounterValue()};
     Stream.EmitRecord(PP_COUNTER_VALUE, Record);
   }
 
+  if (PP.isRecordingPreamble() && PP.hasRecordedPreamble()) {
+    for (const auto &Cond : PP.getPreambleConditionalStack()) {
+      AddSourceLocation(Cond.IfLoc, Record);
+      Record.push_back(Cond.WasSkipping);
+      Record.push_back(Cond.FoundNonSkip);
+      Record.push_back(Cond.FoundElse);
+    }
+    Stream.EmitRecord(PP_CONDITIONAL_STACK, Record);
+    Record.clear();
+  }
+
   // Enter the preprocessor block.
   Stream.EnterSubblock(PREPROCESSOR_BLOCK_ID, 3);
 
   // If the AST file contains __DATE__ or __TIME__ emit a warning about this.
   // FIXME: Include a location for the use, and say which one was used.
   if (PP.SawDateOrTime())
     PP.Diag(SourceLocation(), diag::warn_module_uses_date_time) << IsModule;
 

test/Lexer/preamble.c

 // Preamble detection test: see below for comments and test commands.
 //* A BCPL comment that includes '/*'
 #include <blah>
 #ifndef FOO
 #else
 #ifdef BAR
 #elif WIBBLE
 #endif
 #pragma unknown
 #endif
 #ifdef WIBBLE
-#include "honk"
-#else
-int foo();
+#include "foo"
+int bar;
 #endif
 
 // This test checks for detection of the preamble of a file, which
-// includes all of the starting comments and #includes. Note that any
-// changes to the preamble part of this file must be mirrored in
-// Inputs/preamble.txt, since we diff against it.
+// includes all of the starting comments and #includes.
 
 // RUN: %clang_cc1 -print-preamble %s > %t
 // RUN: echo END. >> %t
 // RUN: FileCheck < %t %s
 
 // CHECK: // Preamble detection test: see below for comments and test commands.
 // CHECK-NEXT: //* A BCPL comment that includes '/*'
 // CHECK-NEXT: #include <blah>
 // CHECK-NEXT: #ifndef FOO
 // CHECK-NEXT: #else
 // CHECK-NEXT: #ifdef BAR
 // CHECK-NEXT: #elif WIBBLE
 // CHECK-NEXT: #endif
 // CHECK-NEXT: #pragma unknown
 // CHECK-NEXT: #endif
+// CHECK-NEXT: #ifdef WIBBLE
+// CHECK-NEXT: #include "foo"
 // CHECK-NEXT: END.

test/Lexer/preamble2.c

+// Preamble detection test: header with an include guard.
+#ifndef HEADER_H
+#define HEADER_H
+#include "foo"
+int bar;
+#endif
+
+// This test checks for detection of the preamble of a file, which
+// includes all of the starting comments and #includes.
+
+// RUN: %clang_cc1 -print-preamble %s > %t
+// RUN: echo END. >> %t
+// RUN: FileCheck < %t %s
+
+// CHECK: // Preamble detection test: header with an include guard.
+// CHECK-NEXT: #ifndef HEADER_H
+// CHECK-NEXT: #define HEADER_H
+// CHECK-NEXT: #include "foo"
+// CHECK-NEXT: END.