Refactor clang-format's formatter.

[klimek]

Summary

a) Pull out a class LevelIndentTracker whose responsibility is to keep track

of the indent of levels across multiple annotated lines.

b) Put all responsibility for merging lines into the LineJoiner; make the

LineJoiner iterate over the lines so we never operate on a line that might
be merged later; this makes the interface safer to use.

c) Move formatting of the end-of-file whitespace into formatFirstToken.

Fix bugs that became obvious after the refactoring:

1. We would not format lines with offsets correctly inside nested blocks if only the outer expression was affected: int x = s({ clang-format only this line class X { public: ^ this starts at the non-modified indnent level; previously we would // not fix this, now we correctly outdent it. void f(); }; });
2. We would incorrectly align comments across lines that do not have comments for lines with nested blocks: int expression; with comment int x = s({ int y; comment int z; we would incorrectly align this comment with the comment on 'expression' });

[djasper]

I like it.

lib/Format/UnwrappedLineFormatter.cpp

28

I think "Tracks the indent level of \c AnnotatedLines across levels" is slightly clearer as this tracks the indent for multiple lines.

29

I think this should be a class, not a struct.

34

Just use:

for (unsigned i = 0; i != StartLevel; ++i) ...

StartLevel already is a POD type (unsigned even) and we don't get anything.

39–43

Ignore.. Phab doesn't let me delete this comment (Eclipse-like behavior).

62

Isn't this missing an

Offset = getIndentOffset(*Line.First);

106–110

As obvious from my comment above where I probably show that I am not understanding something, it is unclear how this needs to be a class variable instead of a variable calculated per line. Please comment.

142–154

Remove all the braces.

446–447

Can you change the order? My OCD finds it weird to have Next after End ;-).

815

This assert seems a bit odd ..

821–824

The actions for "Format" and "!Format" seem pretty interleaved here. Couldn't we group them more?

821–825

Both, the comment and the variable name are a bit confusing. Lets discuss this offline.

831–837

No braces.. Here and below.

894

Merge this line with the next (there already is a clang-format bug for that).

lib/Format/UnwrappedLineFormatter.h

53–55

If the implementation isn't in the header anymore, this might deserve a small comment.

[klimek]

Implement review comments.

lib/Format/UnwrappedLineFormatter.cpp

28

Done.

29

Yep.

34

Done.

39–43

Ignored</irony>

62

Nope. Added class comment and local method comment.

106–110

I hope the additional comments on the methods clarify...

142–154

Done. ALL.

446–447

The problem is that my OCD finds it weird to have a non-const member in between the const ones. Here, I added an empty line, perhaps that helps?

815

Resolved via chat: moved around and duplicated the condition.

821–824

Resolved via chat: moved some of the code around.

821–825

Resolved via that: moved into the !Format part and rewrote comment.

831–837

Done.

894

Done.

lib/Format/UnwrappedLineFormatter.h

53–55

Yep.

[djasper]

Looks very nice now.

lib/Format/UnwrappedLineFormatter.cpp

30

This should be stronger, i.e. updateIndentForLine must be called...

34

This should somehow include that updateLevelIndentFromUnmodifiedLine needs to be called *in addition* to updateIndentForLine.

51

I'd probably just call this nextLine.

71

maybe adjustToUnmodifiedLine

[klimek]

lib/Format/UnwrappedLineFormatter.cpp

30

Done.

34

Done.

51

Done.

71

Done.

[klimek]

Commited as r237104

lib/Format/UnwrappedLineFormatter.h

UnwrappedLineFormatter(ContinuationIndenter *Indenter,
         Keywords(Keywords), IncompleteFormat(IncompleteFormat) {}
 
   /// \brief Format the current block and return the penalty.
   unsigned format(const SmallVectorImpl<AnnotatedLine *> &Lines,
                   bool DryRun = false, int AdditionalIndent = 0,
                   bool FixBadIndentation = false);
 
 private:
-  /// \brief Get the offset of the line relatively to the level.
-  ///
-  /// For example, 'public:' labels in classes are offset by 1 or 2
-  /// characters to the left from their level.
-  int getIndentOffset(const FormatToken &RootToken) {
-    if (Style.Language == FormatStyle::LK_Java ||
-        Style.Language == FormatStyle::LK_JavaScript)
-      return 0;
-    if (RootToken.isAccessSpecifier(false) ||
-        RootToken.isObjCAccessSpecifier() ||
-        (RootToken.is(Keywords.kw_signals) && RootToken.Next &&
-         RootToken.Next->is(tok::colon)))
-      return Style.AccessModifierOffset;
-    return 0;
-  }
-
   /// \brief Add a new line and the required indent before the first Token
   /// of the \c UnwrappedLine if there was no structural parsing error.
   void formatFirstToken(FormatToken &RootToken,
                         const AnnotatedLine *PreviousLine, unsigned IndentLevel,
                         unsigned Indent, bool InPPDirective);
 
-  /// \brief Get the indent of \p Level from \p IndentForLevel.
-  ///
-  /// \p IndentForLevel must contain the indent for the level \c l
-  /// at \p IndentForLevel[l], or a value < 0 if the indent for
-  /// that level is unknown.
-  unsigned getIndent(ArrayRef<int> IndentForLevel, unsigned Level);
-
-  void join(AnnotatedLine &A, const AnnotatedLine &B);
-
-  unsigned getColumnLimit(bool InPPDirective) const {
-    // In preprocessor directives reserve two chars for trailing " \"
-    return Style.ColumnLimit - (InPPDirective ? 2 : 0);
-  }
+  /// \brief Returns the column limit for a line, taking into account whether we
+  /// need an escaped newline due to a continued preprocessor directive.
+  unsigned getColumnLimit(bool InPPDirective, const AnnotatedLine *NextLine) const;
 
   // Cache to store the penalty of formatting a vector of AnnotatedLines
   // starting from a specific additional offset. Improves performance if there
   // are many nested blocks.
   std::map<std::pair<const SmallVectorImpl<AnnotatedLine *> *, unsigned>,
            unsigned> PenaltyCache;
 
   ContinuationIndenter *Indenter;

lib/Format/UnwrappedLineFormatter.cpp

 
 bool startsExternCBlock(const AnnotatedLine &Line) {
   const FormatToken *Next = Line.First->getNextNonComment();
   const FormatToken *NextNext = Next ? Next->getNextNonComment() : nullptr;
   return Line.First->is(tok::kw_extern) && Next && Next->isStringLiteral() &&
          NextNext && NextNext->is(tok::l_brace);
 }
 
+/// \brief Tracks the indent level of \c AnnotatedLines across levels.
+///
+/// When \c updateIndentForLine is called for each \c AnnotatedLine, \c
+/// getIndent() will return the indent for the last line \c updateIndentForLine
+/// was called with.
+/// If the line is not formatted (and thus the indent does not change), calling
+/// \c updateLevelIndentFromUnmodifiedLine will cause subsequent lines on the
+/// same level to be indented at the same level as the given line.
+class LevelIndentTracker {
+public:
+  LevelIndentTracker(const FormatStyle &Style,
+                     const AdditionalKeywords &Keywords, unsigned StartLevel,
+                     int AdditionalIndent)
+      : Style(Style), Keywords(Keywords) {
+    for (unsigned i = 0; i != StartLevel; ++i)
+      IndentForLevel.push_back(Style.IndentWidth * i + AdditionalIndent);
+  }
+
+  /// \brief Returns the indent for the current line.
+  unsigned getIndent() const { return Indent; }
+
+  /// \brief Update the indent state given that \p Line is going to be formatted
+  /// next.
+  void updateIndentForLine(const AnnotatedLine &Line) {
+    Offset = getIndentOffset(*Line.First);
+    if (Line.InPPDirective) {
+      Indent = Line.Level * Style.IndentWidth;
+    } else {
+      while (IndentForLevel.size() <= Line.Level)
+        IndentForLevel.push_back(-1);
+      IndentForLevel.resize(Line.Level + 1);
+      Indent = getIndent(IndentForLevel, Line.Level);
+    }
+    if (static_cast<int>(Indent) + Offset >= 0)
+      Indent += Offset;
+  }
+
+  /// \brief Update the level indent to adapt to the given \p Line.
+  ///
+  /// When a line is not formatted, we move the subsequent lines on the same
+  /// level to the same indent.
+  /// Note that \c updateIndentForLine must have been called before this
+  /// method.
+  void updateLevelIndentFromUnmodifiedLine(const AnnotatedLine &Line) {
+    unsigned LevelIndent = Line.First->OriginalColumn;
+    if (static_cast<int>(LevelIndent) - Offset >= 0)
+      LevelIndent -= Offset;
+    if ((Line.First->isNot(tok::comment) || IndentForLevel[Line.Level] == -1) &&
+        !Line.InPPDirective)
+      IndentForLevel[Line.Level] = LevelIndent;
+  }
+
+private:
+  /// \brief Get the offset of the line relatively to the level.
+  ///
+  /// For example, 'public:' labels in classes are offset by 1 or 2
+  /// characters to the left from their level.
+  int getIndentOffset(const FormatToken &RootToken) {
+    if (Style.Language == FormatStyle::LK_Java ||
+        Style.Language == FormatStyle::LK_JavaScript)
+      return 0;
+    if (RootToken.isAccessSpecifier(false) ||
+        RootToken.isObjCAccessSpecifier() ||
+        (RootToken.is(Keywords.kw_signals) && RootToken.Next &&
+         RootToken.Next->is(tok::colon)))
+      return Style.AccessModifierOffset;
+    return 0;
+  }
+
+  /// \brief Get the indent of \p Level from \p IndentForLevel.
+  ///
+  /// \p IndentForLevel must contain the indent for the level \c l
+  /// at \p IndentForLevel[l], or a value < 0 if the indent for
+  /// that level is unknown.
+  unsigned getIndent(ArrayRef<int> IndentForLevel, unsigned Level) {
+    if (IndentForLevel[Level] != -1)
+      return IndentForLevel[Level];
+    if (Level == 0)
+      return 0;
+    return getIndent(IndentForLevel, Level - 1) + Style.IndentWidth;
+  }
+
+  const FormatStyle &Style;
+  const AdditionalKeywords &Keywords;
+
+  /// \brief The indent in characters for each level.
+  std::vector<int> IndentForLevel;
+
+  /// \brief Offset of the current line relative to the indent level.
+  ///
+  /// For example, the 'public' keywords is often indented with a negative
+  /// offset.
+  int Offset = 0;
+
+  /// \brief The current line's indent.
+  unsigned Indent = 0;
+};
+
 class LineJoiner {
 public:
-  LineJoiner(const FormatStyle &Style, const AdditionalKeywords &Keywords)
-      : Style(Style), Keywords(Keywords) {}
+  LineJoiner(const FormatStyle &Style, const AdditionalKeywords &Keywords,
+             const SmallVectorImpl<AnnotatedLine *> &Lines)
+      : Style(Style), Keywords(Keywords), End(Lines.end()),
+        Next(Lines.begin()) {}
+
+  /// \brief Returns the next line, merging multiple lines into one if possible.
+  const AnnotatedLine *getNextMergedLine(bool DryRun,
+                                         LevelIndentTracker &IndentTracker) {
+    if (Next == End)
+      return nullptr;
+    const AnnotatedLine *Current = *Next;
+    IndentTracker.updateIndentForLine(*Current);
+    unsigned MergedLines =
+        tryFitMultipleLinesInOne(IndentTracker.getIndent(), Next, End);
+    if (MergedLines > 0 && Style.ColumnLimit == 0)
+      // Disallow line merging if there is a break at the start of one of the
+      // input lines.
+      for (unsigned i = 0; i < MergedLines; ++i)
+        if (Next[i + 1]->First->NewlinesBefore > 0)
+          MergedLines = 0;
+    if (!DryRun)
+      for (unsigned i = 0; i < MergedLines; ++i)
+        join(*Next[i], *Next[i + 1]);
+    Next = Next + MergedLines + 1;
+    return Current;
+  }
 
+private:
   /// \brief Calculates how many lines can be merged into 1 starting at \p I.
   unsigned
   tryFitMultipleLinesInOne(unsigned Indent,
                            SmallVectorImpl<AnnotatedLine *>::const_iterator I,
                            SmallVectorImpl<AnnotatedLine *>::const_iterator E) {
     // Can't join the last line with anything.
     if (I + 1 == E)
       return 0;
tryFitMultipleLinesInOne(unsigned Indent,
     }
     if (TheLine->InPPDirective &&
         (TheLine->First->HasUnescapedNewline || TheLine->First->IsFirst)) {
       return tryMergeSimplePPDirective(I, E, Limit);
     }
     return 0;
   }
 
-private:
   unsigned
   tryMergeSimplePPDirective(SmallVectorImpl<AnnotatedLine *>::const_iterator I,
                             SmallVectorImpl<AnnotatedLine *>::const_iterator E,
                             unsigned Limit) {
     if (Limit == 0)
       return 0;
     if (I + 2 != E && I[2]->InPPDirective && !I[2]->First->HasUnescapedNewline)
       return 0;
private:
   bool containsMustBreak(const AnnotatedLine *Line) {
     for (const FormatToken *Tok = Line->First; Tok; Tok = Tok->Next) {
       if (Tok->MustBreakBefore)
         return true;
     }
     return false;
   }
 
+  void join(AnnotatedLine &A, const AnnotatedLine &B) {
+    assert(!A.Last->Next);
+    assert(!B.First->Previous);
+    if (B.Affected)
+      A.Affected = true;
+    A.Last->Next = B.First;
+    B.First->Previous = A.Last;
+    B.First->CanBreakBefore = true;
+    unsigned LengthA = A.Last->TotalLength + B.First->SpacesRequiredBefore;
+    for (FormatToken *Tok = B.First; Tok; Tok = Tok->Next) {
+      Tok->TotalLength += LengthA;
+      A.Last = Tok;
+    }
+  }
+
   const FormatStyle &Style;
   const AdditionalKeywords &Keywords;
+  const SmallVectorImpl<AnnotatedLine*>::const_iterator End;
+
+  SmallVectorImpl<AnnotatedLine*>::const_iterator Next;
 };
 
 static void markFinalized(FormatToken *Tok) {
   for (; Tok; Tok = Tok->Next) {
     Tok->Finalized = true;
     for (AnnotatedLine *Child : Tok->Children)
       markFinalized(Child->First);
   }

 };
 
 } // namespace
 
 unsigned
 UnwrappedLineFormatter::format(const SmallVectorImpl<AnnotatedLine *> &Lines,
                                bool DryRun, int AdditionalIndent,
                                bool FixBadIndentation) {
-  LineJoiner Joiner(Style, Keywords);
+  LineJoiner Joiner(Style, Keywords, Lines);
 
   // Try to look up already computed penalty in DryRun-mode.
   std::pair<const SmallVectorImpl<AnnotatedLine *> *, unsigned> CacheKey(
       &Lines, AdditionalIndent);
   auto CacheIt = PenaltyCache.find(CacheKey);
   if (DryRun && CacheIt != PenaltyCache.end())
     return CacheIt->second;
 
   assert(!Lines.empty());
   unsigned Penalty = 0;
-  std::vector<int> IndentForLevel;
-  for (unsigned i = 0, e = Lines[0]->Level; i != e; ++i)
-    IndentForLevel.push_back(Style.IndentWidth * i + AdditionalIndent);
+  LevelIndentTracker IndentTracker(Style, Keywords, Lines[0]->Level,
+                                   AdditionalIndent);
   const AnnotatedLine *PreviousLine = nullptr;
-  for (SmallVectorImpl<AnnotatedLine *>::const_iterator I = Lines.begin(),
-                                                        E = Lines.end();
-       I != E; ++I) {
-    const AnnotatedLine &TheLine = **I;
-    const FormatToken *FirstTok = TheLine.First;
-    int Offset = getIndentOffset(*FirstTok);
-
-    // Determine indent and try to merge multiple unwrapped lines.
-    unsigned Indent;
-    if (TheLine.InPPDirective) {
-      Indent = TheLine.Level * Style.IndentWidth;
-    } else {
-      while (IndentForLevel.size() <= TheLine.Level)
-        IndentForLevel.push_back(-1);
-      IndentForLevel.resize(TheLine.Level + 1);
-      Indent = getIndent(IndentForLevel, TheLine.Level);
-    }
-    unsigned LevelIndent = Indent;
-    if (static_cast<int>(Indent) + Offset >= 0)
-      Indent += Offset;
-
-    // Merge multiple lines if possible.
-    unsigned MergedLines = Joiner.tryFitMultipleLinesInOne(Indent, I, E);
-    if (MergedLines > 0 && Style.ColumnLimit == 0) {
-      // Disallow line merging if there is a break at the start of one of the
-      // input lines.
-      for (unsigned i = 0; i < MergedLines; ++i) {
-        if (I[i + 1]->First->NewlinesBefore > 0)
-          MergedLines = 0;
-      }
-    }
-    if (!DryRun) {
-      for (unsigned i = 0; i < MergedLines; ++i) {
-        join(*I[i], *I[i + 1]);
-      }
-    }
-    I += MergedLines;
-
+  const AnnotatedLine *NextLine = nullptr;
+  for (const AnnotatedLine *Line =
+           Joiner.getNextMergedLine(DryRun, IndentTracker);
+       Line; Line = NextLine) {
+    const AnnotatedLine &TheLine = *Line;
+    unsigned Indent = IndentTracker.getIndent();
     bool FixIndentation =
-        FixBadIndentation && (LevelIndent != FirstTok->OriginalColumn);
+        FixBadIndentation && (Indent != TheLine.First->OriginalColumn);
     bool ShouldFormat = TheLine.Affected || FixIndentation;
-    if (TheLine.First->is(tok::eof)) {
-      if (PreviousLine && PreviousLine->Affected && !DryRun) {
-        // Remove the file's trailing whitespace.
-        unsigned Newlines = std::min(FirstTok->NewlinesBefore, 1u);
-        Whitespaces->replaceWhitespace(*TheLine.First, Newlines,
-                                       /*IndentLevel=*/0, /*Spaces=*/0,
-                                       /*TargetColumn=*/0);
-      }
-    } else if (TheLine.Type != LT_Invalid && ShouldFormat) {
-      if (FirstTok->WhitespaceRange.isValid()) {
-        if (!DryRun)
-          formatFirstToken(*TheLine.First, PreviousLine, TheLine.Level, Indent,
-                           TheLine.InPPDirective);
-      } else {
-        Indent = LevelIndent = FirstTok->OriginalColumn;
-      }
+    // We cannot format this line; if the reason is that the line had a
+    // parsing error, remember that.
+    if (ShouldFormat && TheLine.Type == LT_Invalid && IncompleteFormat)
+      *IncompleteFormat = true;
+
+    if (ShouldFormat && TheLine.Type != LT_Invalid) {
 
-      // If everything fits on a single line, just put it there.
-      unsigned ColumnLimit = Style.ColumnLimit;
-      if (I + 1 != E) {
-        AnnotatedLine *NextLine = I[1];
-        if (NextLine->InPPDirective && !NextLine->First->HasUnescapedNewline)
-          ColumnLimit = getColumnLimit(TheLine.InPPDirective);
-      }
+      NextLine = Joiner.getNextMergedLine(DryRun, IndentTracker);
+      unsigned ColumnLimit = getColumnLimit(TheLine.InPPDirective, NextLine);
+      bool FitsIntoOneLine =
+          TheLine.Last->TotalLength + Indent <= ColumnLimit ||
+          TheLine.Type == LT_ImportStatement;
 
-      if (TheLine.Last->TotalLength + Indent <= ColumnLimit ||
-          TheLine.Type == LT_ImportStatement) {
-        Penalty += NoLineBreakFormatter(Indenter, Whitespaces, Style, this)
-                       .formatLine(TheLine, Indent, DryRun);
-      } else if (Style.ColumnLimit == 0) {
+      if (Style.ColumnLimit == 0)
         NoColumnLimitLineFormatter(Indenter, Whitespaces, Style, this)
             .formatLine(TheLine, Indent, DryRun);
-      } else {
+      else if (FitsIntoOneLine)
+        Penalty += NoLineBreakFormatter(Indenter, Whitespaces, Style, this)
+                       .formatLine(TheLine, Indent, DryRun);
+      else
         Penalty += OptimizingLineFormatter(Indenter, Whitespaces, Style, this)
                        .formatLine(TheLine, Indent, DryRun);
-      }
-
-      if (!TheLine.InPPDirective)
-        IndentForLevel[TheLine.Level] = LevelIndent;
-    } else if (TheLine.ChildrenAffected) {
-      format(TheLine.Children, DryRun);
     } else {
-      // Format the first token if necessary, and notify the WhitespaceManager
-      // about the unchanged whitespace.
-      for (FormatToken *Tok = TheLine.First; Tok; Tok = Tok->Next) {
-        if (Tok == TheLine.First && (Tok->NewlinesBefore > 0 || Tok->IsFirst)) {
-          unsigned LevelIndent = Tok->OriginalColumn;
-          if (!DryRun) {
-            // Remove trailing whitespace of the previous line.
-            if ((PreviousLine && PreviousLine->Affected) ||
-                TheLine.LeadingEmptyLinesAffected) {
-              formatFirstToken(*Tok, PreviousLine, TheLine.Level, LevelIndent,
+      // If no token in the current line is affected, we still need to format
+      // affected children.
+      if (TheLine.ChildrenAffected)
+        format(TheLine.Children, DryRun);
+
+      // Adapt following lines on the current indent level to the same level
+      // unless the current \c AnnotatedLine is not at the beginning of a line.
+      bool StartsNewLine =
+          TheLine.First->NewlinesBefore > 0 || TheLine.First->IsFirst;
+      if (StartsNewLine)
+        IndentTracker.updateLevelIndentFromUnmodifiedLine(TheLine);
+        bool ReformatLeadingWhitespace =
+            StartsNewLine && ((PreviousLine && PreviousLine->Affected) ||
+                              TheLine.LeadingEmptyLinesAffected);
+        // Format the first token.
+        if (ReformatLeadingWhitespace)
+          formatFirstToken(*TheLine.First, PreviousLine, TheLine.Level,
+                           TheLine.First->OriginalColumn,
+                           TheLine.InPPDirective);
+        else
+          Whitespaces->addUntouchableToken(*TheLine.First,
                                            TheLine.InPPDirective);
-            } else {
-              Whitespaces->addUntouchableToken(*Tok, TheLine.InPPDirective);
-            }
-          }
 
-          if (static_cast<int>(LevelIndent) - Offset >= 0)
-            LevelIndent -= Offset;
-          if ((Tok->isNot(tok::comment) ||
-               IndentForLevel[TheLine.Level] == -1) &&
-              !TheLine.InPPDirective)
-            IndentForLevel[TheLine.Level] = LevelIndent;
-        } else if (!DryRun) {
+        // Notify the WhitespaceManager about the unchanged whitespace.
+        for (FormatToken *Tok = TheLine.First->Next; Tok; Tok = Tok->Next)
           Whitespaces->addUntouchableToken(*Tok, TheLine.InPPDirective);
-        }
-      }
-    }
-    if (TheLine.Type == LT_Invalid && ShouldFormat && IncompleteFormat)
-      *IncompleteFormat = true;
+
+      NextLine = Joiner.getNextMergedLine(DryRun, IndentTracker);
     if (!DryRun)
       markFinalized(TheLine.First);
-    PreviousLine = *I;
+    PreviousLine = &TheLine;
   }
   PenaltyCache[CacheKey] = Penalty;
   return Penalty;
 }
 
 void UnwrappedLineFormatter::formatFirstToken(FormatToken &RootToken,
                                               const AnnotatedLine *PreviousLine,
                                               unsigned IndentLevel,
                                               unsigned Indent,
                                               bool InPPDirective) {
+  if (RootToken.is(tok::eof)) {
+    unsigned Newlines = std::min(RootToken.NewlinesBefore, 1u);
+    Whitespaces->replaceWhitespace(RootToken, Newlines, /*IndentLevel=*/0,
+                                   /*Spaces=*/0, /*TargetColumn=*/0);
+    return;
+  }
   unsigned Newlines =
       std::min(RootToken.NewlinesBefore, Style.MaxEmptyLinesToKeep + 1);
   // Remove empty lines before "}" where applicable.
   if (RootToken.is(tok::r_brace) &&
       (!RootToken.Next ||
        (RootToken.Next->is(tok::semi) && !RootToken.Next->Next)))
     Newlines = std::min(Newlines, 1u);
   if (Newlines == 0 && !RootToken.IsFirst)
if (PreviousLine && PreviousLine->First->isAccessSpecifier() &&
       (!PreviousLine->InPPDirective || !RootToken.HasUnescapedNewline))
     Newlines = std::min(1u, Newlines);
 
   Whitespaces->replaceWhitespace(RootToken, Newlines, IndentLevel, Indent,
                                  Indent, InPPDirective &&
                                              !RootToken.HasUnescapedNewline);
 }
 
-/// \brief Get the indent of \p Level from \p IndentForLevel.
-///
-/// \p IndentForLevel must contain the indent for the level \c l
-/// at \p IndentForLevel[l], or a value < 0 if the indent for
-/// that level is unknown.
-unsigned UnwrappedLineFormatter::getIndent(ArrayRef<int> IndentForLevel,
-                                           unsigned Level) {
-  if (IndentForLevel[Level] != -1)
-    return IndentForLevel[Level];
-  if (Level == 0)
-    return 0;
-  return getIndent(IndentForLevel, Level - 1) + Style.IndentWidth;
-}
-
-void UnwrappedLineFormatter::join(AnnotatedLine &A, const AnnotatedLine &B) {
-  assert(!A.Last->Next);
-  assert(!B.First->Previous);
-  if (B.Affected)
-    A.Affected = true;
-  A.Last->Next = B.First;
-  B.First->Previous = A.Last;
-  B.First->CanBreakBefore = true;
-  unsigned LengthA = A.Last->TotalLength + B.First->SpacesRequiredBefore;
-  for (FormatToken *Tok = B.First; Tok; Tok = Tok->Next) {
-    Tok->TotalLength += LengthA;
-    A.Last = Tok;
-  }
+unsigned
+UnwrappedLineFormatter::getColumnLimit(bool InPPDirective,
+                                       const AnnotatedLine *NextLine) const {
+  // In preprocessor directives reserve two chars for trailing " \" if the
+  // next line continues the preprocessor directive.
+  bool ContinuesPPDirective =
+      InPPDirective && NextLine && NextLine->InPPDirective &&
+      // If there is an unescaped newline between this line and the next, the
+      // next line starts a new preprocessor directive.
+      !NextLine->First->HasUnescapedNewline;
+  return Style.ColumnLimit - (ContinuesPPDirective ? 2 : 0);
 }
 
 } // namespace format
 } // namespace clang

unittests/Format/FormatTest.cpp

TEST_F(FormatTest, MacroDefinitionsWithIncompleteCode) {
   verifyIncompleteFormat("#define STR(x) #x\n"
                          "f(STR(this_is_a_string_literal{));");
   verifyFormat("#pragma omp threadprivate( \\\n"
                "    y)), // expected-warning",
                getLLVMStyleWithColumns(28));
   verifyFormat("#d, = };");
   verifyFormat("#if \"a");
   verifyIncompleteFormat("({\n"
-                         "#define b }\\\n"
+                         "#define b     \\\n"
+                         "  }           \\\n"
                          "  a\n"
-                         "a");
+                         "a", getLLVMStyleWithColumns(15));
   verifyFormat("#define A     \\\n"
                "  {           \\\n"
                "    {\n"
                "#define B     \\\n"
                "  }           \\\n"
                "  }",
                getLLVMStyleWithColumns(15));
   verifyNoCrash("#if a\na(\n#else\n#endif\n{a");
verifyFormat(
       "      int i; // break\n"
       "    },\n"
       "    aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa(bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb,\n"
       "                                     ccccccccccccccccc));");
   verifyFormat("DEBUG({\n"
                "  if (a)\n"
                "    f();\n"
                "});");
+  EXPECT_EQ("int longlongname; // comment\n"
+            "int x = f({\n"
+            "  int x; // comment\n"
+            "  int y; // comment\n"
+            "});",
+            format("int longlongname; // comment\n"
+                   "int x = f({\n"
+                   "  int x; // comment\n"
+                   "  int y; // comment\n"
+                   "});",
+                   65, 0, getLLVMStyle()));
+  EXPECT_EQ("int s = f({\n"
+            "  class X {\n"
+            "  public:\n"
+            "    void f();\n"
+            "  };\n"
+            "});",
+            format("int s = f({\n"
+                   "  class X {\n"
+                   "    public:\n"
+                   "    void f();\n"
+                   "  };\n"
+                   "});",
+                   0, 0, getLLVMStyle()));
 }
 
 TEST_F(FormatTest, LayoutBlockInsideStatement) {
   EXPECT_EQ("SOME_MACRO { int i; }\n"
             "int i;",
             format("  SOME_MACRO  {int i;}  int i;"));
 }