[clang-format] Fix the bug that joins template closer and > or >>

[owenpan]

Summary

Also fixes a buggy test case.

See PR42404

[MyDeveloperDay]

LGTM

[• Quuxplusone]

Drive-by observation: My experiments with https://zed0.co.uk/clang-format-configurator/ show that there is a similar bug where clang-format accidentally produces >= via reformatting of template<enable_if_t<Foo, int> =0>, pi<int> =3;, and so on. Is it possible to fix that bug as part of this patch, and/or would you be interested in patching that bug as a follow-up to this one?

[owenpan]

In D66332#1633448, @Quuxplusone wrote:

Drive-by observation: My experiments with https://zed0.co.uk/clang-format-configurator/ show that there is a similar bug where clang-format accidentally produces >= via reformatting of template<enable_if_t<Foo, int> =0>, pi<int> =3;, and so on. Is it possible to fix that bug as part of this patch, and/or would you be interested in patching that bug as a follow-up to this one?

Do you have SpaceBeforeAssignmentOperators off? I am more than happy to fix it as a follow-up, but the current behavior of SpaceBeforeAssignmentOperators is:
true:

int a = 5;
a += 42;

false:

int a= 5;
a+= 42;

It's weird that there is a space after the assignment operators regardless.

The documentation should be fixed.

[• Quuxplusone]

In D66332#1633749, @owenpan wrote:

Do you have SpaceBeforeAssignmentOperators off?

Yes; I mean, that's what I tested in order to produce the buggy behavior. I don't believe anyone should be using that option in production, though, and would totally support a patch to remove it completely because it can't possibly be doing anyone any good. It feels like someone meant to implement SpacesAroundAssignmentOperators but then got the name wrong, and then someone fixed the behavior to match the name, and at this point it's just completely broken. :)

The documentation should be fixed.

Also agreed.

Are there any other clang-format options that might lead to a lack-of-space before >, >=, ==, or =? I brought up enable_if_t<Foo, int>=0 because that specifically is a construction I've run into in my own code, even though I've never tried to clang-format it, let alone have it formatted incorrectly.

(Clang-format probably does not promise to preserve the AST of pathological input like i++ ++; or C++03-specific input such as A<B<int> > or A< ::B>.)

[owenpan]

Are there any other clang-format options that might lead to a lack-of-space before >, >=, ==, or =? I brought up enable_if_t<Foo, int>=0 because that specifically is a construction I've run into in my own code, even though I've never tried to clang-format it, let alone have it formatted incorrectly.

Any token that starts with >, e.g., >, >>, >=, and >>=, are already taken care of by this patch. For tokens starting with =, only the assignment operator = has a problem and it only occurs when SpaceBeforeAssignmentOperators is set to false.

I can insert a space between a template closer > and an assignment operator = despite the value of SpaceBeforeAssignmentOperators, but the formatted code may be inconsistent...

[hans]

This seems to have caused https://bugs.llvm.org/show_bug.cgi?id=45218
Please take a look.

[owenpan]

The culprit is AnnotatingParser::parseAngle() in clang/lib/Format/TokenAnnotator.cpp. This commit merely uncovered it. :)

Upon reading a < token, parseAngle() tries to scan the rest of the line to find a matching >. If found, it's given the type TT_TemplateCloser. This should be fixed.

clang/lib/Format/TokenAnnotator.cpp

bool consumeToken() {
     case tok::r_brace:
       // Lines can start with '}'.
       if (Tok->Previous)
         return false;
       break;
     case tok::greater:
       if (Style.Language != FormatStyle::LK_TextProto)
         Tok->Type = TT_BinaryOperator;
+      if (Tok->Previous && Tok->Previous->is(TT_TemplateCloser))
+        Tok->SpacesRequiredBefore = 1;
       break;
     case tok::kw_operator:
       if (Style.Language == FormatStyle::LK_TextProto ||
           Style.Language == FormatStyle::LK_Proto)
         break;
       while (CurrentToken &&
              !CurrentToken->isOneOf(tok::l_paren, tok::semi, tok::r_paren)) {
         if (CurrentToken->isOneOf(tok::star, tok::amp))

clang/unittests/Format/FormatTest.cpp

TEST_F(FormatTest, UnderstandsTemplateParameters) {
   verifyGoogleFormat("A< ::A<int> > a;");
   EXPECT_EQ("A<A<A<A>>> a;", format("A<A<A<A> >> a;", getGoogleStyle()));
   EXPECT_EQ("A<A<A<A>>> a;", format("A<A<A<A>> > a;", getGoogleStyle()));
   EXPECT_EQ("A<::A<int>> a;", format("A< ::A<int>> a;", getGoogleStyle()));
   EXPECT_EQ("A<::A<int>> a;", format("A<::A<int> > a;", getGoogleStyle()));
   EXPECT_EQ("auto x = [] { A<A<A<A>>> a; };",
             format("auto x=[]{A<A<A<A> >> a;};", getGoogleStyle()));
 
-  verifyFormat("A<A>> a;", getChromiumStyle(FormatStyle::LK_Cpp));
+  verifyFormat("A<A<int>> a;", getChromiumStyle(FormatStyle::LK_Cpp));
+
+  verifyFormat("int i = a<1> >> 1;");
+  verifyFormat("bool b = a<1> > 1;");
 
   verifyFormat("test >> a >> b;");
   verifyFormat("test << a >> b;");
 
   verifyFormat("f<int>();");
   verifyFormat("template <typename T> void f() {}");
   verifyFormat("struct A<std::enable_if<sizeof(T2) < sizeof(int32)>::type>;");
   verifyFormat("struct A<std::enable_if<sizeof(T2) ? sizeof(int32) : "