[clang-format] spacesRequiredBetween is not honouring clang-format off/on

[MyDeveloperDay]

Summary

https://github.com/llvm/llvm-project/issues/52881

It seems that clang-format off/on is not being honoured in regard to adding spaces.

My understanding of clang-format off/on is that it marks the token as finalized based on whether formatting is currently enabled or disabled.

This was causing a space to be added between the < and << in the Cuda kernel foo<<<1, 1>>>();

This if doesn't solve this actual issue but ensure that clang-format is at least honoured.

[MyDeveloperDay]

Switch to verifyFormat in the tests

[curdeius]

LGTM.

clang/lib/Format/TokenAnnotator.cpp

 
 bool TokenAnnotator::spaceRequiredBefore(const AnnotatedLine &Line,
                                          const FormatToken &Right) {
   const FormatToken &Left = *Right.Previous;
   auto HasExistingWhitespace = [&Right]() {
     return Right.WhitespaceRange.getBegin() != Right.WhitespaceRange.getEnd();
   };
 
+  // If the token is finalized don't touch it (as it could be in a
+  // clang-format-off section).
+  if (Left.Finalized)
+    return HasExistingWhitespace();
+
   if (Right.Tok.getIdentifierInfo() && Left.Tok.getIdentifierInfo())
     return true; // Never ever merge two identifiers.
 
   // Leave a space between * and /* to avoid C4138 `comment end` found outside
   // of comment.
   if (Left.is(tok::star) && Right.is(tok::comment))
     return true;
 

clang/unittests/Format/FormatTest.cpp

TEST_F(FormatTest, SpacesInAngles) {
   verifyFormat("Foo< int, Bar > x6;", Spaces);
   verifyFormat("Foo<::int, ::Bar> x7;", Spaces);
   verifyFormat("Foo< ::int, ::Bar > x7;", Spaces);
 
   verifyFormat("A<A<int>>();", Spaces);
   verifyFormat("A< A< int > >();", Spaces);
   verifyFormat("A<A<int > >();", Spaces);
   verifyFormat("A< A< int>>();", Spaces);
+
+  Spaces.SpacesInAngles = FormatStyle::SIAS_Always;
+  verifyFormat("// clang-format off\n"
+               "foo<<<1, 1>>>();\n"
+               "// clang-format on\n",
+               Spaces);
+  verifyFormat("// clang-format off\n"
+               "foo< < <1, 1> > >();\n"
+               "// clang-format on\n",
+               Spaces);
 }
 
 TEST_F(FormatTest, SpaceAfterTemplateKeyword) {
   FormatStyle Style = getLLVMStyle();
   Style.SpaceAfterTemplateKeyword = false;
   verifyFormat("template<int> void foo();", Style);
 }