Enable the support for digit separators in C mode in Clang (to be compatible with MSVC)

[Manna]

Summary

This patch enables the support for digit separators in C mode in Clang. The digit separators are not supported in C mode but are supported in C++ mode in Clang.

Digit Separators are supported in both C and C++ modes in MS VC/C++ and Intel C/C++ compiler.

To be compatible with MSVC, I have enabled the support for digit separators in C mode in clang.

[rnk]

lib/Lex/Lexer.cpp

1765

This is a conforming extension, so please check MicrosoftExt instead of MSVCCompat.

[Manna]

Thanks for looking into the patch. I have made changes to check MicrosoftExt instead of MSVCCompat.

[Manna]

lib/Lex/Lexer.cpp

1765

checked MicrosoftExt instead of MSVCCompat

[rnk]

test/SemaCXX/digit_separators.c

2

You should explicitly pass -std=c++11 here and -fms-extensions, otherwise the test will fail now.

[rsmith]

It's actually not (quite) a conforming extension: http://eel.is/c++draft/diff.cpp11.lex#1

Another example:

#define X(a)
X(1'a"b')

This is valid without the extension (you get a 1 token followed by a 'a"b' token) but undefined behavior with this extension due to an unmatched ".

But it's fairly close to being a conforming extension. I'm not sure how pedantic we want to be about this.

lib/Lex/Lexer.cpp

1765

Please remove the redundant parentheses.

[Manna]

Thanks for the feedback. The patch is updated based on it.

[Manna]

test/SemaCXX/digit_separators.c

2

This test case is a ‘.c’ file, so it is compiling as a C program.

"-std=c++11" can't be explicitly passed here.
The test fails with this option "-std=c++11" and generates error:
invalid argument '-std=c++11' not allowed with 'C'.

The test passes without this option: "-std=c++11".

lib/Lex/Lexer.cpp

if (!LangOpts.C99) {
                std::find(BufferPtr, CurPtr, '_') != CurPtr)
         IsHexFloat = false;
     }
     if (IsHexFloat)
       return LexNumericConstant(Result, ConsumeChar(CurPtr, Size, Result));
   }
 
   // If we have a digit separator, continue.
-  if (C == '\'' && getLangOpts().CPlusPlus14) {
+  if (C == '\'' && (getLangOpts().CPlusPlus14 || getLangOpts().MicrosoftExt)) {
     unsigned NextSize;
     char Next = getCharAndSizeNoWarn(CurPtr + Size, NextSize, getLangOpts());
     if (isIdentifierBody(Next)) {
       if (!isLexingRawMode())
         Diag(CurPtr, diag::warn_cxx11_compat_digit_separator);
       CurPtr = ConsumeChar(CurPtr, Size, Result);
       CurPtr = ConsumeChar(CurPtr, NextSize, Result);
       return LexNumericConstant(Result, CurPtr);
     }

test/SemaCXX/digit_separators.c

+// RUN: %clang -cc1 -fsyntax-only -verify -fms-compatibility -fms-extensions %s
+// expected-no-diagnostics
+
+int x = 1'000'000 ;