[clang-format] ColumnLimit check for trailing comments alignment acts wrong for multi-byte UTF-8 #47624

[HazardyKnusperkeks]

Summary

Fix for
https://github.com/llvm/llvm-project/issues/47624
https://bugs.llvm.org/show_bug.cgi?id=48280

Used token for counting length instead of logical symbols.
That leads to wrong length count with utf8 symbols

[HazardyKnusperkeks]

Hi,

could you please reupload your patch with the full diff context? And please add a regression test.

[StailGot]

In D124260#3468773, @HazardyKnusperkeks wrote:

Hi,

could you please reupload your patch with the full diff context? And please add a regression test.

Done!

[HazardyKnusperkeks]

There seem to be failing tests. Did you run the FormatTests?

clang/unittests/Format/FormatTestComments.cpp

2861–2864 ↗

(On Diff #424754)

Here is something wrong.

[HazardyKnusperkeks]

Do you still intend to work on this? Otherwise I will commandeer and abandon.

[HazardyKnusperkeks]

You can reopen this, if you wish to continue working on it.

clang/lib/Format/WhitespaceManager.cpp

 
     if (Style.ColumnLimit == 0)
       ChangeMaxColumn = UINT_MAX;
-    else if (Style.ColumnLimit >= Changes[i].TokenLength)
-      ChangeMaxColumn = Style.ColumnLimit - Changes[i].TokenLength;
+    else if (Style.ColumnLimit >= Changes[i].Tok->ColumnWidth)
+      ChangeMaxColumn = Style.ColumnLimit - Changes[i].Tok->ColumnWidth;
     else
       ChangeMaxColumn = ChangeMinColumn;