[clang-format] Handle '_' in ud-suffix for IntegerLiteralSeparator

[owenpan]

Summary

Fixes https://github.com/llvm/llvm-project/issues/61676.

[rymiel]

Could this possibly be an issue for more esoteric underscore-less UDLs, like i?
Does the code need to search for a suffix, could it not detect the absence of a digit? Sorry if the questions are silly, I haven't really looked at this part of the formatter in depth

[owenpan]

In D146844#4220894, @rymiel wrote:

Could this possibly be an issue for more esoteric underscore-less UDLs, like i?
Does the code need to search for a suffix, could it not detect the absence of a digit? Sorry if the questions are silly, I haven't really looked at this part of the formatter in depth

Good questions. I had chosen searching for suffixes instead of the absence of digits/separators because the former is simpler and runs a little faster. I'll switch to the latter if and when there is a need for inserting separators into integer literals with ud-suffixes that don't start with an underscore, which are allowed in the standard library only.

clang/lib/Format/IntegerLiteralSeparatorFixer.cpp

125

There is no need to search the prefix. Will update it when landing.

[owenpan]

Don't format imaginary numbers.

[owenpan]

Treats imaginary number suffixes (starting with an i) just like any user-defined suffixes (starting with an underscore). Also handles ud-suffixes (e.g. _km and _Pa) containing letters that can be part of a floating-point number.

[MyDeveloperDay]

I'm not going to use it, but I like what you've done ;-)

[owenpan]

In D146844#4226558, @MyDeveloperDay wrote:

I'm not going to use it, but I like what you've done ;-)

Thanks! I won't use it either. :)

clang/lib/Format/IntegerLiteralSeparatorFixer.cpp

for (bool Skip = false; !Lex.LexFromRawLexer(Tok);) {
     const auto B = getBase(Text);
     const bool IsBase2 = B == Base::Binary;
     const bool IsBase10 = B == Base::Decimal;
     const bool IsBase16 = B == Base::Hex;
     if ((IsBase2 && SkipBinary) || (IsBase10 && SkipDecimal) ||
         (IsBase16 && SkipHex) || B == Base::Other) {
       continue;
     }
+    if (Style.isCpp()) {
+      if (const auto Pos = Text.find_first_of("_i"); Pos != StringRef::npos) {
+        Text = Text.substr(0, Pos);
+        Length = Pos;
+      }
+    }
     if ((IsBase10 && Text.find_last_of(".eEfFdDmM") != StringRef::npos) ||
         (IsBase16 && Text.find_last_of(".pP") != StringRef::npos)) {
       continue;
     }
     if (((IsBase2 && Binary < 0) || (IsBase10 && Decimal < 0) ||
          (IsBase16 && Hex < 0)) &&
         Text.find(Separator) == StringRef::npos) {
       continue;
     }
     const auto Start = Text[0] == '0' ? 2 : 0;
-    auto End = Text.find_first_of("uUlLzZn");
+    auto End = Text.find_first_of("uUlLzZn", Start);
     if (End == StringRef::npos)
       End = Length;
     if (Start > 0 || End < Length) {
       Length = End - Start;
       Text = Text.substr(Start, Length);
     }
     auto DigitsPerGroup = Decimal;
     if (IsBase2)

clang/unittests/Format/IntegerLiteralSeparatorTest.cpp

TEST_F(IntegerLiteralSeparatorTest, SingleQuoteAsSeparator) {
   verifyFormat(Hex, Style);
   Style.IntegerLiteralSeparator.Hex = -1;
   verifyFormat("h = 0xDEADBEEFDEADBEEFuz;", Hex, Style);
   Style.IntegerLiteralSeparator.Hex = 2;
   verifyFormat("h = 0xDE'AD'BE'EF'DE'AD'BE'EFuz;", Hex, Style);
 
   verifyFormat("o0 = 0;\n"
                "o1 = 07;\n"
-               "o5 = 012345",
+               "o5 = 012345;",
+               Style);
+
+  verifyFormat("bi = 0b1'0000i;\n"
+               "dif = 1'234if;\n"
+               "hil = 0xA'BCil;",
+               "bi = 0b10000i;\n"
+               "dif = 1234if;\n"
+               "hil = 0xABCil;",
+               Style);
+
+  verifyFormat("d = 5'678_km;\n"
+               "h = 0xD'EF_u16;",
+               "d = 5678_km;\n"
+               "h = 0xDEF_u16;",
                Style);
 }
 
 TEST_F(IntegerLiteralSeparatorTest, UnderscoreAsSeparator) {
   FormatStyle Style = getLLVMStyle();
   const StringRef Binary("B = 0B10011_11_0110_1;");
   const StringRef Decimal("d = 184467_440737_0_95505_92;");
   const StringRef Hex("H = 0XDEAD_BEEF_DE_AD_BEE_F;");