[clang] Fix for "Bug 27113 - MSVC-compat __identifier implementation incomplete"

[super_concat]

Summary

this patch fixes Bug 27113 by adding support for string literals to the implementation of the MS extension __identifier.

[super_concat]

Ping

[super_concat]

Ping

[super_concat]

Ping

[hans]

Interesting, I hadn't seen __identifier before. It seems like a pretty esoteric feature.

clang/lib/Lex/PPMacroExpansion.cpp

1818

I'm not sure this is the right way to get the string out of a string_literal.. this pattern seems more common: https://github.com/llvm/llvm-project/blob/llvmorg-12.0.0/clang/lib/Lex/Pragma.cpp#L755

For example, what should happen with __identifier("foo" "bar")?

1820

It says in the bug report that MSVC emits a warning for this. What does the warning look like, and would it make sense for Clang too warn too (it might not, just wondering if we should consider it).

clang/test/Parser/MicrosoftExtensions.cpp

273

Maybe instead of removing this, move it to the cases that work above. For example, I guess this should work now?

int __identifier("foo") = 42;
int bar = foo;

282

It would be interesting to see a non-error test case too, perhaps with a mangled name like in the PR, and to see that it makes it through to LLVM IR correctly..

[super_concat]

Correctly handle string literals and add more test cases. Also MSVC by default gives an error when trying to use __identifier with a string literal but disabling/suppressing that error allows string literals to be used with __identifier. However given how esoteric it is, it wouldn't really be necessary for clang to give a warning or error like MSVC

[hans]

lgtm

[super_concat]

@hans, could you land this for me? I do not have commit access.

clang/lib/Lex/PPMacroExpansion.cpp

 #include "clang/Basic/ObjCRuntime.h"
 #include "clang/Basic/SourceLocation.h"
 #include "clang/Basic/TargetInfo.h"
 #include "clang/Lex/CodeCompletionHandler.h"
 #include "clang/Lex/DirectoryLookup.h"
 #include "clang/Lex/ExternalPreprocessorSource.h"
 #include "clang/Lex/HeaderSearch.h"
 #include "clang/Lex/LexDiagnostic.h"
+#include "clang/Lex/LiteralSupport.h"
 #include "clang/Lex/MacroArgs.h"
 #include "clang/Lex/MacroInfo.h"
 #include "clang/Lex/Preprocessor.h"
 #include "clang/Lex/PreprocessorLexer.h"
 #include "clang/Lex/PreprocessorOptions.h"
 #include "clang/Lex/Token.h"
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/DenseMap.h"
if (Tok.isNot(tok::l_paren)) {
       return;
     }
 
     SourceLocation LParenLoc = Tok.getLocation();
     LexNonComment(Tok);
 
     if (!Tok.isAnnotation() && Tok.getIdentifierInfo())
       Tok.setKind(tok::identifier);
-    else {
+    else if (Tok.is(tok::string_literal) && !Tok.hasUDSuffix()) {
+      StringLiteralParser Literal(Tok, *this);
+      if (Literal.hadError)
+        return;
+
+      Tok.setIdentifierInfo(getIdentifierInfo(Literal.GetString()));
+      Tok.setKind(tok::identifier);
+    } else {
       Diag(Tok.getLocation(), diag::err_pp_identifier_arg_not_identifier)
         << Tok.getKind();
       // Don't walk past anything that's not a real token.
       if (Tok.isOneOf(tok::eof, tok::eod) || Tok.isAnnotation())
         return;
     }
 
     // Discard the ')', preserving 'Tok' as our result.

clang/test/Parser/MicrosoftExtensions.cpp

 
 int __identifier for; // expected-error {{missing '(' after '__identifier'}}
 int __identifier(else} = __identifier(for); // expected-error {{missing ')' after identifier}} expected-note {{to match this '('}}
 #define identifier_weird(x) __identifier(x
 int k = identifier_weird(if)); // expected-error {{use of undeclared identifier 'if'}}
 
 extern int __identifier(and);
 
+int __identifier("baz") = 0;
+int bar = baz;
+
+void mangled_function();
+extern "C" void __identifier("?mangled_function@@YAXXZ")() {}
+
 void f() {
   __identifier(() // expected-error {{cannot convert '(' token to an identifier}}
   __identifier(void) // expected-error {{use of undeclared identifier 'void'}}
   __identifier()) // expected-error {{cannot convert ')' token to an identifier}}
   // FIXME: We should pick a friendlier display name for this token kind.
   __identifier(1) // expected-error {{cannot convert <numeric_constant> token to an identifier}}
   __identifier(+) // expected-error {{cannot convert '+' token to an identifier}}
-  __identifier("foo") // expected-error {{cannot convert <string_literal> token to an identifier}}
   __identifier(;) // expected-error {{cannot convert ';' token to an identifier}}
+  __identifier("1"); // expected-error {{use of undeclared identifier '1'}}
+  __identifier("+"); // expected-error {{use of undeclared identifier '+'}}
+  __identifier(";"); // expected-error {{use of undeclared identifier ';'}}
 }
 
 class inline_definition_pure_spec {
    virtual int f() = 0 { return 0; }// expected-warning {{function definition with pure-specifier is a Microsoft extension}}
    virtual int f2() = 0;
 };
 
 struct pure_virtual_dtor {