[clang-format] use spaces for alignment with UT_ForContinuationAndIndentation

[fickert]

Summary

Use spaces instead of tabs for alignment with UT_ForContinuationAndIndentation to make the code aligned for any tab/indent width.

Fixes https://bugs.llvm.org/show_bug.cgi?id=38381

[fickert]

I added an IsAligned flag to ParenState and Change to track if an indent was created for alignment. I adapted the corresponding test cases where alignments were previously done with tabs, including a test case where leading whitespace in a comment is interpreted as alignment.

This is my first contribution to clang-format so I appreciate any suggestions.

[MyDeveloperDay]

If you are changing unit tests then I'm nervous, imagine if this many tests change what the impact is on a large code base. I think we need to understand more about why you are making this change and why you think its ok to change/remove existing tests which assert some behavior which up to now we have considered correct.

[MyDeveloperDay]

you need to add any diff as a full context diff (normally by adding -U 9999999) to the diff command

[fickert]

uploaded full context diff

[fickert]

I understand the caution around changing the unit tests. The tests I modified assumed that alignment would be made with mixed tabs and spaces, which is arguably not the expected behavior with UT_ForContinuationAndIndentation (see https://bugs.llvm.org/show_bug.cgi?id=38381). I removed a test case that was a duplicate (the one in line 10293 is identical to the one in line 10182, and there are no changes to the formatting configuration in between). I added two test cases to ensure that the behavior is consistent for alignment in multi-line comments, e.g. the formatted code should be

{
\t/* aaaa
\t   bbbb
}

instead of

{
\t/* aaaa
\t\t bbbb
}

when using UT_ForContinuationAndIndentation and a tab width of 2.

[MyDeveloperDay]

clang/unittests/Format/FormatTest.cpp

10335

Sorry this isn't clear to be which test this is a duplicate of, as this is an example where there are tabs in the comment and trailing tabs

[MyDeveloperDay]

Is this intentional, or is it an oversight?

This is the key question, which I'm not sure how we can answer without the help of the original authors @djasper and @klimek

For example I can't understand with the example below if the TabWith is 8 why are you putting 13 spaces in, rather than the original test which had 1 tab (assuming 8 characters) and 5 spaces? (to make up 13)

Could you explain why this is the correct change for this example why its correct to put 13 spaces? (is this not continuation or indentation?)

Tab.UseTab = FormatStyle::UT_ForContinuationAndIndentation;
Tab.TabWidth = 8;
Tab.IndentWidth = 8;
verifyFormat("class X {\n"
             "\tvoid f() {\n"
             "\t\tsomeFunction(parameter1,\n"
             "\t\t.............parameter2);\n"
             "\t}\n"
             "};",
             Tab);

[fickert]

clang/unittests/Format/FormatTest.cpp

10335

This test case is identical to the one in line 10182-10190. There are several test cases that appear multiple times, but usually with changes to the formatting settings in between. This is not the case for these two test cases (no changes in between), so this one seems to be obsolete.

[fickert]

In D75034#1889107, @MyDeveloperDay wrote:

Is this intentional, or is it an oversight?

This is the key question, which I'm not sure how we can answer without the help of the original authors @djasper and @klimek

For example I can't understand with the example below if the TabWith is 8 why are you putting 13 spaces in, rather than the original test which had 1 tab (assuming 8 characters) and 5 spaces? (to make up 13)

Could you explain why this is the correct change for this example why its correct to put 13 spaces? (is this not continuation or indentation?)

Tab.UseTab = FormatStyle::UT_ForContinuationAndIndentation;
Tab.TabWidth = 8;
Tab.IndentWidth = 8;
verifyFormat("class X {\n"
             "\tvoid f() {\n"
             "\t\tsomeFunction(parameter1,\n"
             "\t\t.............parameter2);\n"
             "\t}\n"
             "};",
             Tab);

In this example, the first two tabs are for indentation, and the remaining whitespace before parameter2 is for alignment (not continuation) with parameter1. If tabs are used in the alignment, the formatted code will only look aligned with the correct tab width, whereas if the alignment is done with spaces, the code will look aligned with arbitrary tab width.

I believe this to be the expected behavior for UT_ForContinuationAndIndentation, and tabs should only be used for alignment with UT_Always, but I would also be happy about comments from the original authors.

[MyDeveloperDay]

I tend to agree with your assessment, I think its worth waiting for some others to comment.

clang/unittests/Format/FormatTest.cpp

10335

thank you for highlighting that.

[mitchell-stellar]

Digging through the history, it seems this behavior was explicitly desired: https://reviews.llvm.org/D19028. @mxbOctasic was the original author and perhaps could comment. Regardless, I think there should be a new option.

[fickert]

From the original commit, it does seem that this behavior was desired.

However, from the name of the option and description in the documentation ("Use tabs only for line continuation and indentation"), I would expect it to use tabs only for indentation and continuation, but spaces for alignment. The current behavior uses spaces for some types of alignment that appear in the middle of a line (e.g. consecutive assignments/declarations), but not for others that occur at the beginning of a line (e.g. function parameters). The bug report seems to indicate that other people also expected a different behavior given the description.

Does anyone else want to comment? Otherwise I can move the changed behavior into a new option; in that case, I'd be happy about naming suggestions.

[MyDeveloperDay]

I want to believe that it was a mistake on the original developers part, but I just can't tell.

A search of github shows people using this in their repos

https://github.com/search?q=ForContinuationAndIndentation&type=Code

I almost feel unless the original author can comment we are at an impasse, maybe we have to add:

ForAllContinuationAndIndentation

And if at some point in the future we can collapse both options into 1 then that would be better, but I just don't feel qualified to say change everyone when I don't use that option myself because I don't use tabs.

Whatever you add need to be heavily document that we mean everywhere, and if you ever find any cases where its doesn't work we intend for them to be changed. i.e. be very explicit in the review/commit comments

[MyDeveloperDay]

is there overlap here D76197: clang-format: Use block indentation for braced initializations

[fickert]

In D75034#1938626, @MyDeveloperDay wrote:

is there overlap here D76197: clang-format: Use block indentation for braced initializations

I think these changes are unrelated.

[fickert]

I moved the changed behavior to a new option (UT_AlignWithSpaces), and added corresponding unit tests by copying and adapting the tests for UT_ForContinuationAndIndentation.

[fickert]

clang/unittests/Format/FormatTest.cpp

10164–10165

This test seems to be missing the tab configuration (Tab) as an argument to format, and the same test with missing Tab argument appears again in line 10337. Adding the Tab argument will also change the expected result. I think the test case here should look like this:

EXPECT_EQ("{\n"
          "        /*\n"
          "         * Comment\n"
          "         */\n"
          "        int i;\n"
          "}",
          format("{\n"
                 "\t/*\n"
                 "\t * Comment\n"
                 "\t */\n"
                 "\t int i;\n"
                 "}",
                 Tab));

and a similar change should be made to the one in line 10337.

I assume this should be fixed in a separate commit?

[fickert]

Added documentation and missing test cases for 0-width tabs.

[MyDeveloperDay]

This LGTM, thank you

[fickert]

Great, thank you!

I do not have push access to the repository, so someone else would need to commit this for me: Maximilian Fickert <[e-mail]>

Should I add the fix for the test cases missing the tab configuration (see my last inline comment: https://reviews.llvm.org/D75034#inline-703423)? Or submit it as a separate change set?

[fickert]

rebased, fixed two unit tests

clang/docs/ClangFormatStyleOptions.rst

**UseTab** (``UseTabStyle``)
 
   * ``UT_Never`` (in configuration: ``Never``)
     Never use tab.
 
   * ``UT_ForIndentation`` (in configuration: ``ForIndentation``)
     Use tabs only for indentation.
 
   * ``UT_ForContinuationAndIndentation`` (in configuration: ``ForContinuationAndIndentation``)
-    Use tabs only for line continuation and indentation.
+    Fill all leading whitespace with tabs, and use spaces for alignment that
+    appears within a line (e.g. consecutive assignments and declarations).
+
+  * ``UT_AlignWithSpaces`` (in configuration: ``AlignWithSpaces``)
+    Use tabs for line continuation and indentation, and spaces for alignment.
 
   * ``UT_Always`` (in configuration: ``Always``)
     Use tabs whenever we need to fill whitespace that spans at least from
     one tab stop to the next one.
 
 
 
 .. END_FORMAT_STYLE_OPTIONS

clang/include/clang/Format/Format.h

   unsigned TabWidth;
 
   /// Different ways to use tab in formatting.
   enum UseTabStyle {
     /// Never use tab.
     UT_Never,
     /// Use tabs only for indentation.
     UT_ForIndentation,
-    /// Use tabs only for line continuation and indentation.
+    /// Fill all leading whitespace with tabs, and use spaces for alignment that
+    /// appears within a line (e.g. consecutive assignments and declarations).
     UT_ForContinuationAndIndentation,
+    /// Use tabs for line continuation and indentation, and spaces for
+    /// alignemnt.
+    UT_AlignWithSpaces,
     /// Use tabs whenever we need to fill whitespace that spans at least from
     /// one tab stop to the next one.
     UT_Always
   };
 
   /// \brief Use ``\r\n`` instead of ``\n`` for line breaks.
   /// Also used as fallback if ``DeriveLineEnding`` is true.
   bool UseCRLF;

clang/lib/Format/BreakableToken.cpp

 
 void BreakableLineCommentSection::reflow(unsigned LineIndex,
                                          WhitespaceManager &Whitespaces) const {
   if (LineIndex > 0 && Tokens[LineIndex] != Tokens[LineIndex - 1]) {
     // Reflow happens between tokens. Replace the whitespace between the
     // tokens by the empty string.
     Whitespaces.replaceWhitespace(
         *Tokens[LineIndex], /*Newlines=*/0, /*Spaces=*/0,
-        /*StartOfTokenColumn=*/StartColumn, /*InPPDirective=*/false);
+        /*StartOfTokenColumn=*/StartColumn, /*IsAligned=*/true,
+        /*InPPDirective=*/false);
   } else if (LineIndex > 0) {
     // In case we're reflowing after the '\' in:
     //
     //   // line comment \
     //   // line 2
     //
     // the reflow happens inside the single comment token (it is a single line
     // comment with an unescaped newline).
if (LineIndex > 0 && Tokens[LineIndex] != Tokens[LineIndex - 1]) {
     // We always want to create a replacement instead of adding an untouchable
     // token, even if LineColumn is the same as the original column of the
     // token. This is because WhitespaceManager doesn't align trailing
     // comments if they are untouchable.
     Whitespaces.replaceWhitespace(*Tokens[LineIndex],
                                   /*Newlines=*/1,
                                   /*Spaces=*/LineColumn,
                                   /*StartOfTokenColumn=*/LineColumn,
+                                  /*IsAligned=*/true,
                                   /*InPPDirective=*/false);
   }
   if (OriginalPrefix[LineIndex] != Prefix[LineIndex]) {
     // Adjust the prefix if necessary.
 
     // Take care of the space possibly introduced after a decoration.
     assert(Prefix[LineIndex] == (OriginalPrefix[LineIndex] + " ").str() &&
            "Expecting a line comment prefix to differ from original by at most "

clang/lib/Format/ContinuationIndenter.h

private:
   llvm::Regex CommentPragmasRegex;
   const RawStringFormatStyleManager RawStringFormats;
 };
 
 struct ParenState {
   ParenState(const FormatToken *Tok, unsigned Indent, unsigned LastSpace,
              bool AvoidBinPacking, bool NoLineBreak)
       : Tok(Tok), Indent(Indent), LastSpace(LastSpace),
-        NestedBlockIndent(Indent), BreakBeforeClosingBrace(false),
-        AvoidBinPacking(AvoidBinPacking), BreakBeforeParameter(false),
-        NoLineBreak(NoLineBreak), NoLineBreakInOperand(false),
-        LastOperatorWrapped(true), ContainsLineBreak(false),
-        ContainsUnwrappedBuilder(false), AlignColons(true),
-        ObjCSelectorNameFound(false), HasMultipleNestedBlocks(false),
-        NestedBlockInlined(false), IsInsideObjCArrayLiteral(false),
-        IsCSharpGenericTypeConstraint(false) {}
+        NestedBlockIndent(Indent), IsAligned(false),
+        BreakBeforeClosingBrace(false), AvoidBinPacking(AvoidBinPacking),
+        BreakBeforeParameter(false), NoLineBreak(NoLineBreak),
+        NoLineBreakInOperand(false), LastOperatorWrapped(true),
+        ContainsLineBreak(false), ContainsUnwrappedBuilder(false),
+        AlignColons(true), ObjCSelectorNameFound(false),
+        HasMultipleNestedBlocks(false), NestedBlockInlined(false),
+        IsInsideObjCArrayLiteral(false), IsCSharpGenericTypeConstraint(false) {}
 
   /// \brief The token opening this parenthesis level, or nullptr if this level
   /// is opened by fake parenthesis.
   ///
   /// Not considered for memoization as it will always have the same value at
   /// the same token.
   const FormatToken *Tok;
 
struct ParenState {
   /// contains the start column of the second line. Otherwise 0.
   unsigned CallContinuation = 0;
 
   /// The column of the first variable name in a variable declaration.
   ///
   /// Used to align further variables if necessary.
   unsigned VariablePos = 0;
 
+  /// Whether this block's indentation is used for alignment.
+  bool IsAligned : 1;
+
   /// Whether a newline needs to be inserted before the block's closing
   /// brace.
   ///
   /// We only want to insert a newline before the closing brace if there also
   /// was a newline after the beginning left brace.
   bool BreakBeforeClosingBrace : 1;
 
   /// Avoid bin packing, i.e. multiple parameters/elements on multiple
bool operator<(const ParenState &Other) const {
     if (Indent != Other.Indent)
       return Indent < Other.Indent;
     if (LastSpace != Other.LastSpace)
       return LastSpace < Other.LastSpace;
     if (NestedBlockIndent != Other.NestedBlockIndent)
       return NestedBlockIndent < Other.NestedBlockIndent;
     if (FirstLessLess != Other.FirstLessLess)
       return FirstLessLess < Other.FirstLessLess;
+    if (IsAligned != Other.IsAligned)
+      return IsAligned;
     if (BreakBeforeClosingBrace != Other.BreakBeforeClosingBrace)
       return BreakBeforeClosingBrace;
     if (QuestionColumn != Other.QuestionColumn)
       return QuestionColumn < Other.QuestionColumn;
     if (AvoidBinPacking != Other.AvoidBinPacking)
       return AvoidBinPacking;
     if (BreakBeforeParameter != Other.BreakBeforeParameter)
       return BreakBeforeParameter;

clang/lib/Format/ContinuationIndenter.cpp

if (Style.AlignAfterOpenBracket == FormatStyle::BAS_AlwaysBreak &&
       Current.FakeLParens.back() > prec::Unknown)
     State.Stack.back().NoLineBreak = true;
   if (Previous.is(TT_TemplateString) && Previous.opensScope())
     State.Stack.back().NoLineBreak = true;
 
   if (Style.AlignAfterOpenBracket != FormatStyle::BAS_DontAlign &&
       !State.Stack.back().IsCSharpGenericTypeConstraint &&
       Previous.opensScope() && Previous.isNot(TT_ObjCMethodExpr) &&
-      (Current.isNot(TT_LineComment) || Previous.BlockKind == BK_BracedInit))
+      (Current.isNot(TT_LineComment) || Previous.BlockKind == BK_BracedInit)) {
     State.Stack.back().Indent = State.Column + Spaces;
+    State.Stack.back().IsAligned = true;
+  }
   if (State.Stack.back().AvoidBinPacking && startsNextParameter(Current, Style))
     State.Stack.back().NoLineBreak = true;
   if (startsSegmentOfBuilderTypeCall(Current) &&
       State.Column > getNewLineColumn(State))
     State.Stack.back().ContainsUnwrappedBuilder = true;
 
   if (Current.is(TT_LambdaArrow) && Style.Language == FormatStyle::LK_Java)
     State.Stack.back().NoLineBreak = true;
if (Current.is(tok::r_brace) && Current.MatchingParen &&
       // about removing empty lines on closing blocks. Special case them here.
       MaxEmptyLinesToKeep = 1;
     }
     unsigned Newlines =
         std::max(1u, std::min(Current.NewlinesBefore, MaxEmptyLinesToKeep));
     bool ContinuePPDirective =
         State.Line->InPPDirective && State.Line->Type != LT_ImportStatement;
     Whitespaces.replaceWhitespace(Current, Newlines, State.Column, State.Column,
+                                  State.Stack.back().IsAligned,
                                   ContinuePPDirective);
   }
 
   if (!Current.isTrailingComment())
     State.Stack.back().LastSpace = State.Column;
   if (Current.is(tok::lessless))
     // If we are breaking before a "<<", we always want to indent relative to
     // RHS. This is necessary only for "<<", as we special-case it and don't

clang/lib/Format/Format.cpp

template <> struct ScalarEnumerationTraits<FormatStyle::UseTabStyle> {
   static void enumeration(IO &IO, FormatStyle::UseTabStyle &Value) {
     IO.enumCase(Value, "Never", FormatStyle::UT_Never);
     IO.enumCase(Value, "false", FormatStyle::UT_Never);
     IO.enumCase(Value, "Always", FormatStyle::UT_Always);
     IO.enumCase(Value, "true", FormatStyle::UT_Always);
     IO.enumCase(Value, "ForIndentation", FormatStyle::UT_ForIndentation);
     IO.enumCase(Value, "ForContinuationAndIndentation",
                 FormatStyle::UT_ForContinuationAndIndentation);
+    IO.enumCase(Value, "AlignWithSpaces", FormatStyle::UT_AlignWithSpaces);
   }
 };
 
 template <> struct ScalarEnumerationTraits<FormatStyle::JavaScriptQuoteStyle> {
   static void enumeration(IO &IO, FormatStyle::JavaScriptQuoteStyle &Value) {
     IO.enumCase(Value, "Leave", FormatStyle::JSQS_Leave);
     IO.enumCase(Value, "Single", FormatStyle::JSQS_Single);
     IO.enumCase(Value, "Double", FormatStyle::JSQS_Double);

clang/lib/Format/UnwrappedLineFormatter.cpp

bool formatChildren(LineState &State, bool NewLine, bool DryRun,
     // We add +2 for the trailing " }".
     if (Style.ColumnLimit > 0 &&
         Child->Last->TotalLength + State.Column + 2 > Style.ColumnLimit)
       return false;
 
     if (!DryRun) {
       Whitespaces->replaceWhitespace(
           *Child->First, /*Newlines=*/0, /*Spaces=*/1,
-          /*StartOfTokenColumn=*/State.Column, State.Line->InPPDirective);
+          /*StartOfTokenColumn=*/State.Column, /*IsAligned=*/false,
+          State.Line->InPPDirective);
     }
     Penalty +=
         formatLine(*Child, State.Column + 1, /*FirstStartColumn=*/0, DryRun);
 
     State.Column += 1 + Child->Last->TotalLength;
     return true;
   }
 
void UnwrappedLineFormatter::formatFirstToken(
 
   // Preprocessor directives get indented before the hash only if specified
   if (Style.IndentPPDirectives != FormatStyle::PPDIS_BeforeHash &&
       (Line.Type == LT_PreprocessorDirective ||
        Line.Type == LT_ImportStatement))
     Indent = 0;
 
   Whitespaces->replaceWhitespace(RootToken, Newlines, Indent, Indent,
+                                 /*IsAligned=*/false,
                                  Line.InPPDirective &&
                                      !RootToken.HasUnescapedNewline);
 }
 
 unsigned
 UnwrappedLineFormatter::getColumnLimit(bool InPPDirective,
                                        const AnnotatedLine *NextLine) const {
   // In preprocessor directives reserve two chars for trailing " \" if the

clang/lib/Format/WhitespaceManager.h

public:
 
   /// Replaces the whitespace in front of \p Tok. Only call once for
   /// each \c AnnotatedToken.
   ///
   /// \p StartOfTokenColumn is the column at which the token will start after
   /// this replacement. It is needed for determining how \p Spaces is turned
   /// into tabs and spaces for some format styles.
   void replaceWhitespace(FormatToken &Tok, unsigned Newlines, unsigned Spaces,
-                         unsigned StartOfTokenColumn,
+                         unsigned StartOfTokenColumn, bool isAligned = false,
                          bool InPPDirective = false);
 
   /// Adds information about an unchangeable token's whitespace.
   ///
   /// Needs to be called for every token for which \c replaceWhitespace
   /// was not called.
   void addUntouchableToken(const FormatToken &Tok, bool InPPDirective);
 
struct Change {
     /// and \p CurrentLinePrefix.
     ///
     /// \p StartOfTokenColumn and \p InPPDirective will be used to lay out
     /// trailing comments and escaped newlines.
     Change(const FormatToken &Tok, bool CreateReplacement,
            SourceRange OriginalWhitespaceRange, int Spaces,
            unsigned StartOfTokenColumn, unsigned NewlinesBefore,
            StringRef PreviousLinePostfix, StringRef CurrentLinePrefix,
-           bool ContinuesPPDirective, bool IsInsideToken);
+           bool IsAligned, bool ContinuesPPDirective, bool IsInsideToken);
 
     // The kind of the token whose whitespace this change replaces, or in which
     // this change inserts whitespace.
     // FIXME: Currently this is not set correctly for breaks inside comments, as
     // the \c BreakableToken is still doing its own alignment.
     const FormatToken *Tok;
 
     bool CreateReplacement;
     // Changes might be in the middle of a token, so we cannot just keep the
     // FormatToken around to query its information.
     SourceRange OriginalWhitespaceRange;
     unsigned StartOfTokenColumn;
     unsigned NewlinesBefore;
     std::string PreviousLinePostfix;
     std::string CurrentLinePrefix;
+    bool IsAligned;
     bool ContinuesPPDirective;
 
     // The number of spaces in front of the token or broken part of the token.
     // This will be adapted when aligning tokens.
     // Can be negative to retain information about the initial relative offset
     // of the lines in a block comment. This is used when aligning trailing
     // comments. Uncompensated negative offset is truncated to 0.
     int Spaces;
private:
 
   /// Stores \p Text as the replacement for the whitespace in \p Range.
   void storeReplacement(SourceRange Range, StringRef Text);
   void appendNewlineText(std::string &Text, unsigned Newlines);
   void appendEscapedNewlineText(std::string &Text, unsigned Newlines,
                                 unsigned PreviousEndOfTokenColumn,
                                 unsigned EscapedNewlineColumn);
   void appendIndentText(std::string &Text, unsigned IndentLevel,
-                        unsigned Spaces, unsigned WhitespaceStartColumn);
+                        unsigned Spaces, unsigned WhitespaceStartColumn,
+                        bool IsAligned);
+  unsigned appendTabIndent(std::string &Text, unsigned Spaces,
+                           unsigned Indentation);
 
   SmallVector<Change, 16> Changes;
   const SourceManager &SourceMgr;
   tooling::Replacements Replaces;
   const FormatStyle &Style;
   bool UseCRLF;
 };
 
 } // namespace format
 } // namespace clang
 
 #endif

clang/lib/Format/WhitespaceManager.cpp

 }
 
 WhitespaceManager::Change::Change(const FormatToken &Tok,
                                   bool CreateReplacement,
                                   SourceRange OriginalWhitespaceRange,
                                   int Spaces, unsigned StartOfTokenColumn,
                                   unsigned NewlinesBefore,
                                   StringRef PreviousLinePostfix,
-                                  StringRef CurrentLinePrefix,
+                                  StringRef CurrentLinePrefix, bool IsAligned,
                                   bool ContinuesPPDirective, bool IsInsideToken)
     : Tok(&Tok), CreateReplacement(CreateReplacement),
       OriginalWhitespaceRange(OriginalWhitespaceRange),
       StartOfTokenColumn(StartOfTokenColumn), NewlinesBefore(NewlinesBefore),
       PreviousLinePostfix(PreviousLinePostfix),
-      CurrentLinePrefix(CurrentLinePrefix),
+      CurrentLinePrefix(CurrentLinePrefix), IsAligned(IsAligned),
       ContinuesPPDirective(ContinuesPPDirective), Spaces(Spaces),
       IsInsideToken(IsInsideToken), IsTrailingComment(false), TokenLength(0),
       PreviousEndOfTokenColumn(0), EscapedNewlineColumn(0),
       StartOfBlockComment(nullptr), IndentationOffset(0) {}
 
 void WhitespaceManager::replaceWhitespace(FormatToken &Tok, unsigned Newlines,
                                           unsigned Spaces,
                                           unsigned StartOfTokenColumn,
-                                          bool InPPDirective) {
+                                          bool IsAligned, bool InPPDirective) {
   if (Tok.Finalized)
     return;
   Tok.Decision = (Newlines > 0) ? FD_Break : FD_Continue;
   Changes.push_back(Change(Tok, /*CreateReplacement=*/true, Tok.WhitespaceRange,
                            Spaces, StartOfTokenColumn, Newlines, "", "",
-                           InPPDirective && !Tok.IsFirst,
+                           IsAligned, InPPDirective && !Tok.IsFirst,
                            /*IsInsideToken=*/false));
 }
 
 void WhitespaceManager::addUntouchableToken(const FormatToken &Tok,
                                             bool InPPDirective) {
   if (Tok.Finalized)
     return;
   Changes.push_back(Change(Tok, /*CreateReplacement=*/false,
                            Tok.WhitespaceRange, /*Spaces=*/0,
                            Tok.OriginalColumn, Tok.NewlinesBefore, "", "",
-                           InPPDirective && !Tok.IsFirst,
+                           /*IsAligned=*/false, InPPDirective && !Tok.IsFirst,
                            /*IsInsideToken=*/false));
 }
 
 llvm::Error
 WhitespaceManager::addReplacement(const tooling::Replacement &Replacement) {
   return Replaces.add(Replacement);
 }
 
 void WhitespaceManager::replaceWhitespaceInToken(
     const FormatToken &Tok, unsigned Offset, unsigned ReplaceChars,
     StringRef PreviousPostfix, StringRef CurrentPrefix, bool InPPDirective,
     unsigned Newlines, int Spaces) {
   if (Tok.Finalized)
     return;
   SourceLocation Start = Tok.getStartOfNonWhitespace().getLocWithOffset(Offset);
   Changes.push_back(
       Change(Tok, /*CreateReplacement=*/true,
              SourceRange(Start, Start.getLocWithOffset(ReplaceChars)), Spaces,
              std::max(0, Spaces), Newlines, PreviousPostfix, CurrentPrefix,
-             InPPDirective && !Tok.IsFirst, /*IsInsideToken=*/true));
+             /*IsAligned=*/true, InPPDirective && !Tok.IsFirst,
+             /*IsInsideToken=*/true));
 }
 
 const tooling::Replacements &WhitespaceManager::generateReplacements() {
   if (Changes.empty())
     return Replaces;
 
   llvm::sort(Changes, Change::IsBeforeInFile(SourceMgr));
   calculateLineBreakInformation();
for (unsigned i = 0, e = Changes.size(); i != e; ++i) {
     if (C.CreateReplacement) {
       std::string ReplacementText = C.PreviousLinePostfix;
       if (C.ContinuesPPDirective)
         appendEscapedNewlineText(ReplacementText, C.NewlinesBefore,
                                  C.PreviousEndOfTokenColumn,
                                  C.EscapedNewlineColumn);
       else
         appendNewlineText(ReplacementText, C.NewlinesBefore);
-      appendIndentText(ReplacementText, C.Tok->IndentLevel,
-                       std::max(0, C.Spaces),
-                       C.StartOfTokenColumn - std::max(0, C.Spaces));
+      appendIndentText(
+          ReplacementText, C.Tok->IndentLevel, std::max(0, C.Spaces),
+          C.StartOfTokenColumn - std::max(0, C.Spaces), C.IsAligned);
       ReplacementText.append(C.CurrentLinePrefix);
       storeReplacement(C.OriginalWhitespaceRange, ReplacementText);
     }
   }
 }
 
 void WhitespaceManager::storeReplacement(SourceRange Range, StringRef Text) {
   unsigned WhitespaceLength = SourceMgr.getFileOffset(Range.getEnd()) -
for (unsigned i = 0; i < Newlines; ++i) {
       Text.append(UseCRLF ? "\\\r\n" : "\\\n");
       Spaces = std::max<int>(0, EscapedNewlineColumn - 1);
     }
   }
 }
 
 void WhitespaceManager::appendIndentText(std::string &Text,
                                          unsigned IndentLevel, unsigned Spaces,
-                                         unsigned WhitespaceStartColumn) {
+                                         unsigned WhitespaceStartColumn,
+                                         bool IsAligned) {
   switch (Style.UseTab) {
   case FormatStyle::UT_Never:
     Text.append(Spaces, ' ');
     break;
   case FormatStyle::UT_Always: {
     if (Style.TabWidth) {
       unsigned FirstTabWidth =
           Style.TabWidth - WhitespaceStartColumn % Style.TabWidth;
case FormatStyle::UT_Always: {
     } else if (Spaces == 1) {
       Text.append(Spaces, ' ');
     }
     break;
   }
   case FormatStyle::UT_ForIndentation:
     if (WhitespaceStartColumn == 0) {
       unsigned Indentation = IndentLevel * Style.IndentWidth;
-      // This happens, e.g. when a line in a block comment is indented less than
-      // the first one.
-      if (Indentation > Spaces)
-        Indentation = Spaces;
-      if (Style.TabWidth) {
-        unsigned Tabs = Indentation / Style.TabWidth;
-        Text.append(Tabs, '\t');
-        Spaces -= Tabs * Style.TabWidth;
-      }
+      Spaces = appendTabIndent(Text, Spaces, Indentation);
     }
     Text.append(Spaces, ' ');
     break;
   case FormatStyle::UT_ForContinuationAndIndentation:
-    if (WhitespaceStartColumn == 0 && Style.TabWidth) {
-      unsigned Tabs = Spaces / Style.TabWidth;
-      Text.append(Tabs, '\t');
-      Spaces -= Tabs * Style.TabWidth;
+    if (WhitespaceStartColumn == 0)
+      Spaces = appendTabIndent(Text, Spaces, Spaces);
+    Text.append(Spaces, ' ');
+    break;
+  case FormatStyle::UT_AlignWithSpaces:
+    if (WhitespaceStartColumn == 0) {
+      unsigned Indentation =
+          IsAligned ? IndentLevel * Style.IndentWidth : Spaces;
+      Spaces = appendTabIndent(Text, Spaces, Indentation);
     }
     Text.append(Spaces, ' ');
     break;
   }
 }
 
+unsigned WhitespaceManager::appendTabIndent(std::string &Text, unsigned Spaces,
+                                            unsigned Indentation) {
+  // This happens, e.g. when a line in a block comment is indented less than the
+  // first one.
+  if (Indentation > Spaces)
+    Indentation = Spaces;
+  if (Style.TabWidth) {
+    unsigned Tabs = Indentation / Style.TabWidth;
+    Text.append(Tabs, '\t');
+    Spaces -= Tabs * Style.TabWidth;
+  }
+  return Spaces;
+}
+
 } // namespace format
 } // namespace clang

clang/unittests/Format/FormatTest.cpp

EXPECT_EQ("int a; /* some\n"
                    " \t \t    comment */",
                    Tab));
   EXPECT_EQ("f(\"\t\t\"); /* some\n"
             "    comment */",
             format(" \t \t f(\"\t\t\"); /* some\n"
                    " \t \t    comment */",
                    Tab));
   EXPECT_EQ("{\n"
-            "  /*\n"
-            "   * Comment\n"
-            "   */\n"
-            "  int i;\n"
+            "        /*\n"
+            "         * Comment\n"
+            "         */\n"
+            "        int i;\n"
             "}",
             format("{\n"
                    "\t/*\n"
                    "\t * Comment\n"
                    "\t */\n"
                    "\t int i;\n"
-                   "}"));
+                   "}",
+                   Tab));
 
   Tab.UseTab = FormatStyle::UT_ForContinuationAndIndentation;
   Tab.TabWidth = 8;
   Tab.IndentWidth = 8;
   EXPECT_EQ("if (aaaaaaaa && // q\n"
             "    bb)         // w\n"
             "\t;",
             format("if (aaaaaaaa &&// q\n"
EXPECT_EQ("{\n"
             "\t*/\n"
             "}",
             format("{\n"
                    "\t/*\n"
                    " asdf\n"
                    "\t*/\n"
                    "}",
                    Tab));
+  EXPECT_EQ("/* some\n"
+            "   comment */",
+            format(" \t \t /* some\n"
+                   " \t \t    comment */",
+                   Tab));
+  EXPECT_EQ("int a; /* some\n"
+            "   comment */",
+            format(" \t \t int a; /* some\n"
+                   " \t \t    comment */",
+                   Tab));
+  EXPECT_EQ("int a; /* some\n"
+            "comment */",
+            format(" \t \t int\ta; /* some\n"
+                   " \t \t    comment */",
+                   Tab));
+  EXPECT_EQ("f(\"\t\t\"); /* some\n"
+            "    comment */",
+            format(" \t \t f(\"\t\t\"); /* some\n"
+                   " \t \t    comment */",
+                   Tab));
+  EXPECT_EQ("{\n"
+            "\t/*\n"
+            "\t * Comment\n"
+            "\t */\n"
+            "\tint i;\n"
+            "}",
+            format("{\n"
+                   "\t/*\n"
+                   "\t * Comment\n"
+                   "\t */\n"
+                   "\t int i;\n"
+                   "}",
+                   Tab));
+  Tab.TabWidth = 2;
+  Tab.IndentWidth = 2;
+  EXPECT_EQ("{\n"
+            "\t/* aaaa\n"
+            "\t\t bbbb */\n"
+            "}",
+            format("{\n"
+                   "/* aaaa\n"
+                   "\t bbbb */\n"
+                   "}",
+                   Tab));
+  EXPECT_EQ("{\n"
+            "\t/*\n"
+            "\t\taaaaaaaaaaaaaaaaaaaaaaaaaa\n"
+            "\t\tbbbbbbbbbbbbb\n"
+            "\t*/\n"
+            "}",
+            format("{\n"
+                   "/*\n"
+                   "\taaaaaaaaaaaaaaaaaaaaaaaaaa bbbbbbbbbbbbb\n"
+                   "*/\n"
+                   "}",
+                   Tab));
+  Tab.AlignConsecutiveAssignments = true;
+  Tab.AlignConsecutiveDeclarations = true;
+  Tab.TabWidth = 4;
+  Tab.IndentWidth = 4;
+  verifyFormat("class Assign {\n"
+               "\tvoid f() {\n"
+               "\t\tint         x      = 123;\n"
+               "\t\tint         random = 4;\n"
+               "\t\tstd::string alphabet =\n"
+               "\t\t\t\"abcdefghijklmnopqrstuvwxyz\";\n"
+               "\t}\n"
+               "};",
+               Tab);
+
+  Tab.UseTab = FormatStyle::UT_AlignWithSpaces;
+  Tab.TabWidth = 8;
+  Tab.IndentWidth = 8;
+  EXPECT_EQ("if (aaaaaaaa && // q\n"
+            "    bb)         // w\n"
+            "\t;",
+            format("if (aaaaaaaa &&// q\n"
+                   "bb)// w\n"
+                   ";",
+                   Tab));
+  EXPECT_EQ("if (aaa && bbb) // w\n"
+            "\t;",
+            format("if(aaa&&bbb)// w\n"
+                   ";",
+                   Tab));
+  verifyFormat("class X {\n"
+               "\tvoid f() {\n"
+               "\t\tsomeFunction(parameter1,\n"
+               "\t\t             parameter2);\n"
+               "\t}\n"
+               "};",
+               Tab);
+  verifyFormat("#define A                        \\\n"
+               "\tvoid f() {               \\\n"
+               "\t\tsomeFunction(    \\\n"
+               "\t\t    parameter1,  \\\n"
+               "\t\t    parameter2); \\\n"
+               "\t}",
+               Tab);
+  Tab.TabWidth = 4;
+  Tab.IndentWidth = 8;
+  verifyFormat("class TabWidth4Indent8 {\n"
+               "\t\tvoid f() {\n"
+               "\t\t\t\tsomeFunction(parameter1,\n"
+               "\t\t\t\t             parameter2);\n"
+               "\t\t}\n"
+               "};",
+               Tab);
+  Tab.TabWidth = 4;
+  Tab.IndentWidth = 4;
+  verifyFormat("class TabWidth4Indent4 {\n"
+               "\tvoid f() {\n"
+               "\t\tsomeFunction(parameter1,\n"
+               "\t\t             parameter2);\n"
+               "\t}\n"
+               "};",
+               Tab);
+  Tab.TabWidth = 8;
+  Tab.IndentWidth = 4;
+  verifyFormat("class TabWidth8Indent4 {\n"
+               "    void f() {\n"
+               "\tsomeFunction(parameter1,\n"
+               "\t             parameter2);\n"
+               "    }\n"
+               "};",
+               Tab);
+  Tab.TabWidth = 8;
+  Tab.IndentWidth = 8;
   EXPECT_EQ("/*\n"
-            "\t      a\t\tcomment\n"
-            "\t      in multiple lines\n"
+            "              a\t\tcomment\n"
+            "              in multiple lines\n"
             "       */",
             format("   /*\t \t \n"
                    " \t \t a\t\tcomment\t \t\n"
                    " \t \t in multiple lines\t\n"
                    " \t  */",
                    Tab));
+  verifyFormat("{\n"
+               "\taaaaaaaaaaaaaaaaaaaaaaaaaaaa();\n"
+               "\taaaaaaaaaaaaaaaaaaaaaaaaaaaa();\n"
+               "\taaaaaaaaaaaaaaaaaaaaaaaaaaaa();\n"
+               "\taaaaaaaaaaaaaaaaaaaaaaaaaaaa();\n"
+               "\taaaaaaaaaaaaaaaaaaaaaaaaaaaa();\n"
+               "\taaaaaaaaaaaaaaaaaaaaaaaaaaaa();\n"
+               "};",
+               Tab);
+  verifyFormat("enum AA {\n"
+               "\ta1, // Force multiple lines\n"
+               "\ta2,\n"
+               "\ta3\n"
+               "};",
+               Tab);
+  EXPECT_EQ("if (aaaaaaaa && // q\n"
+            "    bb)         // w\n"
+            "\t;",
+            format("if (aaaaaaaa &&// q\n"
+                   "bb)// w\n"
+                   ";",
+                   Tab));
+  verifyFormat("class X {\n"
+               "\tvoid f() {\n"
+               "\t\tsomeFunction(parameter1,\n"
+               "\t\t             parameter2);\n"
+               "\t}\n"
+               "};",
+               Tab);
+  verifyFormat("{\n"
+               "\tQ(\n"
+               "\t    {\n"
+               "\t\t    int a;\n"
+               "\t\t    someFunction(aaaaaaaa,\n"
+               "\t\t                 bbbbbbb);\n"
+               "\t    },\n"
+               "\t    p);\n"
+               "}",
+               Tab);
+  EXPECT_EQ("{\n"
+            "\t/* aaaa\n"
+            "\t   bbbb */\n"
+            "}",
+            format("{\n"
+                   "/* aaaa\n"
+                   "   bbbb */\n"
+                   "}",
+                   Tab));
+  EXPECT_EQ("{\n"
+            "\t/*\n"
+            "\t  aaaaaaaaaaaaaaaaaaaaaaaaaa\n"
+            "\t  bbbbbbbbbbbbb\n"
+            "\t*/\n"
+            "}",
+            format("{\n"
+                   "/*\n"
+                   "  aaaaaaaaaaaaaaaaaaaaaaaaaa bbbbbbbbbbbbb\n"
+                   "*/\n"
+                   "}",
+                   Tab));
+  EXPECT_EQ("{\n"
+            "\t// aaaaaaaaaaaaaaaaaaaaaaaaaa\n"
+            "\t// bbbbbbbbbbbbb\n"
+            "}",
+            format("{\n"
+                   "\t// aaaaaaaaaaaaaaaaaaaaaaaaaa bbbbbbbbbbbbb\n"
+                   "}",
+                   Tab));
+  EXPECT_EQ("{\n"
+            "\t/*\n"
+            "\t  aaaaaaaaaaaaaaaaaaaaaaaaaa\n"
+            "\t  bbbbbbbbbbbbb\n"
+            "\t*/\n"
+            "}",
+            format("{\n"
+                   "\t/*\n"
+                   "\t  aaaaaaaaaaaaaaaaaaaaaaaaaa bbbbbbbbbbbbb\n"
+                   "\t*/\n"
+                   "}",
+                   Tab));
+  EXPECT_EQ("{\n"
+            "\t/*\n"
+            "\n"
+            "\t*/\n"
+            "}",
+            format("{\n"
+                   "\t/*\n"
+                   "\n"
+                   "\t*/\n"
+                   "}",
+                   Tab));
+  EXPECT_EQ("{\n"
+            "\t/*\n"
+            " asdf\n"
+            "\t*/\n"
+            "}",
+            format("{\n"
+                   "\t/*\n"
+                   " asdf\n"
+                   "\t*/\n"
+                   "}",
+                   Tab));
   EXPECT_EQ("/* some\n"
             "   comment */",
             format(" \t \t /* some\n"
                    " \t \t    comment */",
                    Tab));
   EXPECT_EQ("int a; /* some\n"
             "   comment */",
             format(" \t \t int a; /* some\n"
                    " \t \t    comment */",
                    Tab));
   EXPECT_EQ("int a; /* some\n"
             "comment */",
             format(" \t \t int\ta; /* some\n"
                    " \t \t    comment */",
                    Tab));
   EXPECT_EQ("f(\"\t\t\"); /* some\n"
             "    comment */",
             format(" \t \t f(\"\t\t\"); /* some\n"
                    " \t \t    comment */",
                    Tab));
   EXPECT_EQ("{\n"
-            "  /*\n"
-            "   * Comment\n"
-            "   */\n"
-            "  int i;\n"
+            "\t/*\n"
+            "\t * Comment\n"
+            "\t */\n"
+            "\tint i;\n"
             "}",
             format("{\n"
                    "\t/*\n"
                    "\t * Comment\n"
                    "\t */\n"
                    "\t int i;\n"
-                   "}"));
+                   "}",
+                   Tab));
+  Tab.TabWidth = 2;
+  Tab.IndentWidth = 2;
+  EXPECT_EQ("{\n"
+            "\t/* aaaa\n"
+            "\t   bbbb */\n"
+            "}",
+            format("{\n"
+                   "/* aaaa\n"
+                   "   bbbb */\n"
+                   "}",
+                   Tab));
+  EXPECT_EQ("{\n"
+            "\t/*\n"
+            "\t  aaaaaaaaaaaaaaaaaaaaaaaaaa\n"
+            "\t  bbbbbbbbbbbbb\n"
+            "\t*/\n"
+            "}",
+            format("{\n"
+                   "/*\n"
+                   "  aaaaaaaaaaaaaaaaaaaaaaaaaa bbbbbbbbbbbbb\n"
+                   "*/\n"
+                   "}",
+                   Tab));
   Tab.AlignConsecutiveAssignments = true;
   Tab.AlignConsecutiveDeclarations = true;
   Tab.TabWidth = 4;
   Tab.IndentWidth = 4;
   verifyFormat("class Assign {\n"
                "\tvoid f() {\n"
                "\t\tint         x      = 123;\n"
                "\t\tint         random = 4;\n"
TEST_F(FormatTest, ZeroTabWidth) {
   EXPECT_EQ("void a(){\n"
             "    // line starts with '\t'\n"
             "};",
             format("void a(){\n"
                    "\t\t// line starts with '\t'\n"
                    "};",
                    Tab));
 
+  Tab.UseTab = FormatStyle::UT_AlignWithSpaces;
+  EXPECT_EQ("void a(){\n"
+            "    // line starts with '\t'\n"
+            "};",
+            format("void a(){\n"
+                   "\t// line starts with '\t'\n"
+                   "};",
+                   Tab));
+
+  EXPECT_EQ("void a(){\n"
+            "    // line starts with '\t'\n"
+            "};",
+            format("void a(){\n"
+                   "\t\t// line starts with '\t'\n"
+                   "};",
+                   Tab));
+
   Tab.UseTab = FormatStyle::UT_Always;
   EXPECT_EQ("void a(){\n"
             "// line starts with '\t'\n"
             "};",
             format("void a(){\n"
                    "\t// line starts with '\t'\n"
                    "};",
                    Tab));
CHECK_PARSE("AlignEscapedNewlinesLeft: false", AlignEscapedNewlines,
               FormatStyle::ENAS_Right);
 
   Style.UseTab = FormatStyle::UT_ForIndentation;
   CHECK_PARSE("UseTab: Never", UseTab, FormatStyle::UT_Never);
   CHECK_PARSE("UseTab: ForIndentation", UseTab, FormatStyle::UT_ForIndentation);
   CHECK_PARSE("UseTab: Always", UseTab, FormatStyle::UT_Always);
   CHECK_PARSE("UseTab: ForContinuationAndIndentation", UseTab,
               FormatStyle::UT_ForContinuationAndIndentation);
+  CHECK_PARSE("UseTab: AlignWithSpaces", UseTab,
+              FormatStyle::UT_AlignWithSpaces);
   // For backward compatibility:
   CHECK_PARSE("UseTab: false", UseTab, FormatStyle::UT_Never);
   CHECK_PARSE("UseTab: true", UseTab, FormatStyle::UT_Always);
 
   Style.AllowShortBlocksOnASingleLine = FormatStyle::SBS_Empty;
   CHECK_PARSE("AllowShortBlocksOnASingleLine: Never",
               AllowShortBlocksOnASingleLine, FormatStyle::SBS_Never);
   CHECK_PARSE("AllowShortBlocksOnASingleLine: Empty",