Implement P2361 Unevaluated string literals

[cor3ntin]

Summary

This patch proposes to handle in an uniform fashion
the parsing of strings that are never evaluated,
in asm statement, static assert, attrributes, extern,
etc.

Unevaluated strings are UTF-8 internally and so currently
behave as narrow strings, but these things will diverge with
D93031.

The big question both for this patch and the P2361 paper
is whether we risk breaking code by disallowing
encoding prefixes in this context.
I hope this patch may allow to gather some data on that.

Future work:
Improve the rendering of unicode characters, line break
and so forth in static-assert messages

[aaron.ballman]

In general, I think this is shaping up nicely and is almost complete. I'm adding some additional reviewer though, as this is a somewhat experimental patch for a WG21 proposal that has not been accepted yet and I want to make sure that I'm not missing something. That may also solve the few open questions that still remain.

clang/lib/AST/Expr.cpp

1110–1111

I'd recommend running the entire patch through clang-format though: https://clang.llvm.org/docs/ClangFormat.html#script-for-patch-reformatting

clang/lib/Lex/LiteralSupport.cpp

1707

Looks like this comment is still missing punctuation.

[cor3ntin]

Formatting and missing punctuation

[erichkeane]

clang/include/clang/Basic/DiagnosticLexKinds.td

257

Is there value to combining these two diagnostics with a %select?

clang/lib/Lex/LiteralSupport.cpp

95–96

I might consider rejecting ANY character escape in the less-than-32 part of the table.

For consistency at least, I don't see value in allowing \a if we're rejecting layout things like \t.

114

This is like the 3rd time we're using 'Unevaluated' as a bool parameter. I have a pretty strong preference for making it a scoped-enum in 'Basic' somewhere.

1822

Is this OK? It looks like we're passing a ton of parameters to a diag type that doesn't have any wildcards?

[cor3ntin]

Replace Unevaluated by an enum.

[aaron.ballman]

clang/include/clang/Basic/DiagnosticLexKinds.td

257

I waffled when doing this review, so it's funny you mention it. :-D

We could do: an unevaluated string literal cannot %select{have an encoding prefix|be a user-defined literal}0 but there was just enough text in the select that I felt it wasn't critical to combine. But I don't feel strongly either way.

clang/lib/Lex/LiteralSupport.cpp

95–96

But that's just it, we're accepting \t and \n with this code.

1822

Good catch! The first two are not helpful (the diag engine will silently ignore them), but the second two are for underlines in the diagnostic and are useful.

[erichkeane]

clang/lib/Lex/LiteralSupport.cpp

95–96

Ah! I missed that this is an allow-list instead of a deny-list. That makes me way more comfortable with this code.

IMO, I'd suggest we we allow '\r' (since wouldn't we have problems on Windows at that point, being unable to accept a printable newline for windows?), but disallow \a for now unless someone comes up with a really good reason to allow it.

[cor3ntin]

Accept \r as an escape sequence n unevaluated string literal

[cor3ntin]

Rename commit

[erichkeane]

A couple of small things, otherwise I'm happy; but Aaron has some bigger opens above, plus clang-format, plus the modules from Richard.

clang/include/clang/Basic/DiagnosticLexKinds.td

257

I was waffly on this too, so your waffling + my waffling I think is sufficient reason to not deal with this now.

clang/lib/AST/Expr.cpp

1083

minor preference (perhaps 'nit' level) to move this whole CharByteWidth + IsPascal calculation into its own function. This constructor is absurdly long as it is.

clang/lib/Lex/LiteralSupport.cpp

98

For future clarification, the ones from the 'simple' list here: https://en.cppreference.com/w/cpp/language/escape

that we are missing are: \a \b \f and \v.

I personally think I'm ok with that until someone else says they care.

1706

Hrm.... this is unfortunate. Is there no way to combine the loops? I guess (hope?) that hte list of tokens is at least going to be short...

[cor3ntin]

Formatting

[cor3ntin]

Get rid of the extra loop by using a lambda

[aaron.ballman]

Some naming nits. There are two open questions also: one about module behavior and one about a TODO comment in the patch. If we don't hear back about the modules question, I think that can be handled in a follow-up.

clang/include/clang/Lex/LiteralSupport.h

207

Slight renaming so nobody thinks this is going to be about wide vs narrow vs u8, etc.

235

We should rename anything mentioning StringKind similarly -- this will also help avoid confusion with the StringKind type in Expr.h.

249

Can we make this private now rather than letting callers access it directly?

clang/lib/Lex/LiteralSupport.cpp

1707

When I hear "check" I think it'll return a value; I think this name is a bit more clear.

1824–1826

[cor3ntin]

Address Aaron's comments

clang/lib/Lex/LiteralSupport.cpp

114

Any suggestion for where to

1824–1826

This are actually used by err_string_concat_mixed_suffix

[cor3ntin]

clang/lib/Lex/LiteralSupport.cpp

114

NVM

[erichkeane]

clang/lib/Lex/LiteralSupport.cpp

1824–1826

right, i guess it is just super awkward to have unused parameters passed like this. I know we only check the other direction, but seems awkward. Aaron, thoughts?

[aaron.ballman]

clang/lib/Lex/LiteralSupport.cpp

1824–1826

I'd split it into two calls at this point. e.g.,

if (UnevaluatedStringHasUDL)
  Diags->Report(TokLoc, diag::err_unevaluated_string_udl) << ...;
else
  Diags->Report(TokLoc, diag::err_string_concat_mixed_suffix) << ...;

[cor3ntin]

Cleanup Diagnostics In LiteralSupport

[aaron.ballman]

LGTM aside from two small nits. As for the modules question, if @rsmith doesn't get back to us, I think it's fine to address that post-commit.

clang/include/clang/Lex/LiteralSupport.h

235

Did this one get missed?

clang/test/CXX/dcl.dcl/p4-0x.cpp

24

Can you add the newline back to the end of the file?

[cor3ntin]

Fix EOF & unrenamed StringKind

[cor3ntin]

As approved by CWG
Updates to cxx_status / doc will come later :)

[shafik]

clang/lib/Parse/ParseExpr.cpp

3177

[cor3ntin]

Address Shafik's comment

[shafik]

LGTM but I don't see asm covered in the tests.

clang/lib/AST/Expr.cpp

1062

Why not grouped w/ Ordinary above?

1087

Isn't this the same as Length?

1123

Isn't Str.size() the same as ByteLength?

clang/lib/Lex/LiteralSupport.cpp

89

Should we use Is as a prefix here? Right now it should like we are modifying something.

[cor3ntin]

Rename /EscapeValidInUnevaluatedStringLiteral/IsEscapeValidInUnevaluatedStringLiteral

[cor3ntin]

clang/lib/AST/Expr.cpp

1087

Only when CharByteWidth == 1

1123

ByteLength isn't defined in this scope, I guess i could move it.

[aaron.ballman]

This also should update the cxx_status page and have a release note.

clang/include/clang/Basic/Attr.td

1411 ↗

(On Diff #534201)

What is the plan for non-standard attributes? Are you planning to handle those in a follow-up, or should we be investigating those right now?

clang/include/clang/Parse/Parser.h

1820–1822

Two default bool params is a bad thing but three default bool params seems like we should fix the interface at this point. WDYT?

Also, it's not clear what the new parameter will do, the function could use comments unless fixing the interface makes it sufficiently clear.

clang/lib/AST/Expr.cpp

1062

Specifically because we want the host encoding, not the target encoding.

1087

It is -- I think we can get rid of ByteLength, but it's possible that this exists because of the optimization comment below. I don't insist, but it would be nice to know if we can replace the switch with Length /= CharByteWidth these days.

1112

Add assert(!Pascal && "Can't make an unevaluated Pascal string"); ?

1123

I think it's more clear to use Str.size() because we're copying from Str.data().

1160

clang/lib/Lex/LiteralSupport.cpp

89

+1, I think Is would be an improvement.

[aaron.ballman]

clang/lib/Lex/LiteralSupport.cpp

97

We're still missing support for some escape characters from: http://eel.is/c++draft/lex#nt:simple-escape-sequence-char

Just to verify, UCNs have already been handled by the time we get here, so we don't need to care about those, correct?

1747

Doesn't returning here leave the object in a partially-initialized state? That seems bad.

1900–1903

Is there test coverage that we diagnose this properly?

clang/lib/Lex/PPMacroExpansion.cpp

1811–1813

Test coverage for this change?

clang/lib/Lex/Pragma.cpp

807

Pinging @ChuanqiXu for opinions.

clang/lib/Parse/ParseExpr.cpp

3387–3388

I'm surprised we need special logic in ParseExpressionList() for handling unevaluated string literals; I would have expected that to be needed when parsing a string literal. Nothing changed in the grammar for http://eel.is/c++draft/expr.post.general#nt:expression-list (or initializer-list), so these changes seem wrong. Can you explain the changes a bit more?

clang/lib/Sema/SemaDeclAttr.cpp

854–855

Test coverage for these changes?

clang/lib/Sema/SemaDeclCXX.cpp

16106

Test coverage for changes?

[cor3ntin]

clang/lib/AST/Expr.cpp

1062

an unevaluated string is a sequence of 1-byte even on platforms were sizeof(char) would be 2 or 4. It's never influenced by the target's properties

[cor3ntin]

clang/include/clang/Basic/Attr.td

1411 ↗

(On Diff #534201)

I don't feel I'm qualified to answer that. Ideally, attributes that expect string literals that are not evaluated should follow suite.

clang/include/clang/Parse/Parser.h

1820–1822

I'm still not sure that's the best solution. AllowEvaluatedString would only ever be false for attributes, I consider duplicating the function, except it does quite a bit for variadics, which apparently attribute support

Maybe would could have

ParseAttributeArgumentList
ParseExpressionList
ParseExpressionListImpl?

?

clang/lib/AST/Expr.cpp

1087

I think we should.

clang/lib/Lex/LiteralSupport.cpp

97

Just to verify, UCNs have already been handled by the time we get here, so we don't need to care about those, correct?

They are dealt with elsewhere yes (and supported)

1900–1903

What sort of test would you like to see?

clang/lib/Parse/ParseExpr.cpp

3387–3388

We use ParseExpressionList when parsing attribute arguments, and some attributes have unevaluate string as argument - I agree with you that I'd rather find a better solution for attributes, but I came up empty. There is no further reason for this change, and you are right it does not match the grammar.

clang/lib/Sema/SemaDeclAttr.cpp

854–855

There is one somewhere, I don;t remember where, The reason we need to do that is that Unevaluated StringLiterals don''t have types

clang/lib/Sema/SemaDeclCXX.cpp

16106

There are some in dcl.link/p2.cpp

[cor3ntin]

Address some of Aaron's comments

[ChuanqiXu]

clang/lib/Lex/Pragma.cpp

807

I think the both options (to modify it or not) are acceptable. Because the input here should be the output of the clang itself. See https://github.com/llvm/llvm-project/blob/ebd0b8a0472b865b7eb6e1a32af97ae31d829033/clang/lib/Basic/Module.cpp#L229-L231 and https://github.com/llvm/llvm-project/blob/ebd0b8a0472b865b7eb6e1a32af97ae31d829033/clang/lib/Frontend/Rewrite/FrontendActions.cpp#L238-L240.

We can see there is no deprecated prefix. So while it is acceptable to modify this since its pattern matches the paper, it doesn't matter really since we can control the input completely.

Personally, I prefer to not touch it. Since I feel like this use case doesn't have been used a lot. So the effort here may not be worthy.

[aaron.ballman]

clang/include/clang/Basic/Attr.td

1411 ↗

(On Diff #534201)

Let's do them in a follow-up. Normally I'd suggest working with @erichkeane on which attributes to apply that to, but he's about to go on a sabbatical and might not have time to help with that. So maybe you can take a first pass at it as best you can and then rope me in to help finalize it, if that'd work for you?

clang/lib/Lex/LiteralSupport.cpp

1900–1903

Pascal strings enabled and using something like [[deprecated("\pOh no, a Pascal string!")]] (or some other unevaluated uses).

clang/lib/Parse/ParseExpr.cpp

3387–3388

I was thinking we'd use a new kind of evaluation context for this. We'd enter the evaluation context when we know we need to parse an expression that is an unevaluated string literal which the string literal parser would pay attention to. This would require knowing up-front when we want to parse an unevaluated string literal, but we should have that information available to us at parse time (I think).

clang/lib/Sema/SemaDeclAttr.cpp

854–855

Let's try to track that down, but... an unevaluated string literal still has a type, surely? It'd be const char[] for C++?

[cor3ntin]

clang/lib/Parse/ParseExpr.cpp

3387–3388

After offline discussion, i think what we want to be doing is to have a

ParseAtttributeArgumentList function that is aware of whether the Nth argument is an unevaluated string - by means of modifying tablegen,
and doing the right parsing accordingly.
It would take care of all attributes automatically.
Alas that's a tad more involved.

[aaron.ballman]

clang/lib/Parse/ParseExpr.cpp

3387–3388

+1

I agree it's more involved, but it's also a more general solution that fits nicely in the parser design (we do this sort of thing for other parts of attribute parsing).

[cor3ntin]

clang/lib/Sema/SemaDeclAttr.cpp

854–855

It doesn't because it doesn't exist past phase 6.
It's not unevaluated as in decltype, it's more unevaluated as it's a weird token that never participate in the program, the same way a pragma or an attribute don't have a type.
Note that we can revert that change if we do the whole tablegen thing

The relevant test is in test/SemaCXX/warn-thread-safety-parsing.cpp, L17

[cor3ntin]

Add tests for pascal strings (which are not a thing in C++ apparently)

[cor3ntin]

Parse attribute as unevaluated string if they
are declare StringLiteralArgument in the Attr.td file.

WIP

@aaron.ballman Do we agree on direction before I
fix the remaining broken tests?

There are a few limitations, which I'm hoping not to fix there

• It doesn't support variadic string arguments
• checking the type of argument ahead of time seems like a good idea overall, maybe we want to expand that system?

[aaron.ballman]

In D105759#4453440, @cor3ntin wrote:

Parse attribute as unevaluated string if they
are declare StringLiteralArgument in the Attr.td file.

WIP

@aaron.ballman Do we agree on direction before I
fix the remaining broken tests?

Mostly agreed, though I left a comment where I think the direction should change slightly.

There are a few limitations, which I'm hoping not to fix there

• It doesn't support variadic string arguments

That's reasonable; let's leave them as evaluated strings for now so there's no behavioral change.

• checking the type of argument ahead of time seems like a good idea overall, maybe we want to expand that system?

I agree; we currently have the common handler checking argument counts (https://github.com/llvm/llvm-project/blob/1e010c5c4fae43c52d6f5f1c8e8920c26bcc6cc7/clang/lib/Sema/SemaAttr.cpp#L1419), but we don't generate any code for checking argument types. But certainly doesn't need to be done as part of your work.

clang/include/clang/Basic/Attr.td

3048 ↗

(On Diff #535073)

I don't think we should reuse this flag this way. This flag is for the traditional sense of "unevaluated", but unevaluated string literals are a different kind of beast. I think that should be tracked on the argument level. We can either adjust:

class StringArgument<string name, bit opt = 0> : Argument<name, opt>;

so that it takes another bit for whether the string is unevaluated or not, or we could add a new subclass for UnevaluatedStringArgument. Then ClangAttrEmitter.cpp would look at this information when emitting the switch cases.

llvm/cmake/modules/HandleLLVMOptions.cmake

608 ↗

(On Diff #535073)

Spurious change. ;-)

[aaron.ballman]

Clearing the "accepted" status so it's not confusing as to the state of things.

[cor3ntin]

clang/include/clang/Basic/Attr.td

3048 ↗

(On Diff #535073)

This is the previous approach i forgot to fixup everywhere.
My current approach is to always consider StringArgument unevaluated.
I don't think it make sense to have both StringArgument and UnevaluatedStringArgument.
Currently in all the places we accept StringArgument, we check it's a possibly parenthesized StringLiteral

If you want an evaluated string literal, an expression that produce a const char* or something should work

llvm/cmake/modules/HandleLLVMOptions.cmake

608 ↗

(On Diff #535073)

I've been battling with that for weeks, that flag completely breaks my IDE, not sure why. It was inevitable that it ended up in a commit :|

[aaron.ballman]

clang/include/clang/Basic/Attr.td

3048 ↗

(On Diff #535073)

My current approach is to always consider StringArgument unevaluated.
I don't think it make sense to have both StringArgument and UnevaluatedStringArgument.

I think that's potentially a pretty significant change in behavior until we actually evaluate (ahahaha, puns!) all the vendor attributes using a StringArgument. Also, I thought you mentioned you planned to leave variadic string arguments as evaluated strings, so there would be a pretty surprising inconsistency to the behavior there. I would feel more comfortable not changing the behavior of attributes we've not validated are still correct when using unevaluated strings.

[cor3ntin]

Fix tests and handle variadic attributes.

• With that all normal attributes are handled. Only attributes with custop parsing code and those specified as an enum are left untouched.

[cor3ntin]

Note that I have confirmed that all the change attributes require a StringLiteral and go through checkStringLiteralArgumentAttr

[cor3ntin]

revert accidental changes to cmake

[erichkeane]

Just 2 small nits, otherwise this all LGTM.

clang/lib/Parse/ParseDecl.cpp

406

Please put a newline between unchained 'if' statements... it makes tehse really hard to read without it.

It happens a few times here.

clang/lib/Sema/SemaDeclAttr.cpp

854

Unrelated change here? What is this for?

[cor3ntin]

Address Erich's comments

[cor3ntin]

clang/lib/Sema/SemaDeclAttr.cpp

854

Some test i failed to fully revert. good catch!

[aaron.ballman]

I don't think it's correct to assume that all string arguments to attributes are unevaluated, but it is hard to tell where to draw the line sometimes. Backing up a step, as I understand P2361, an unevaluated string is one which is not converted into the execution character set (effectively). Is that correct? If so, then as an example, [[clang::annotate()]] should almost certainly be using an evaluated string because the argument is passed down to LLVM IR and is used in ways we cannot predict. What's more, an unevaluated string cannot have some kinds of escape characters (numeric and conditional escape sequences) and those are currently allowed by clang::annotate and could potentially be used by a backend plugin.

I think other attributes may have similar issues. For example, the alias attribute is a bit of a question mark for me -- that takes a string literal representing an external identifier that is looked up. I'm not certain whether that should be in the execution character set or not, but we do support escape sequences for it: https://godbolt.org/z/v65Yd7a68

I think we need to track evaluated vs not on the argument level so that the attributes in Attr.td can decide which form to use. I think we should default to "evaluated" for any attribute we're on the fence about because that's the current behavior they get today (so we should avoid regressions).

clang/include/clang/Sema/ParsedAttr.h

919 ↗

(On Diff #535373)

clang/lib/Parse/ParseDecl.cpp

285

Comment doesn't match the function name. ;-)

429–430

What are these lines intended to do? We assign to E but nothing ever reads from it after this assignment and we reset it on the next iteration through the loop.

clang/lib/Parse/ParseExpr.cpp

3387

Can revert these two changes now.

[aaron.ballman]

In D105759#4456864, @aaron.ballman wrote:

I don't think it's correct to assume that all string arguments to attributes are unevaluated, but it is hard to tell where to draw the line sometimes. Backing up a step, as I understand P2361, an unevaluated string is one which is not converted into the execution character set (effectively). Is that correct? If so, then as an example, [[clang::annotate()]] should almost certainly be using an evaluated string because the argument is passed down to LLVM IR and is used in ways we cannot predict. What's more, an unevaluated string cannot have some kinds of escape characters (numeric and conditional escape sequences) and those are currently allowed by clang::annotate and could potentially be used by a backend plugin.

I think other attributes may have similar issues. For example, the alias attribute is a bit of a question mark for me -- that takes a string literal representing an external identifier that is looked up. I'm not certain whether that should be in the execution character set or not, but we do support escape sequences for it: https://godbolt.org/z/v65Yd7a68

I took a quick pass over our existing attributes, and here's my intuition on them regarding encoding of the literal:

Unevaluated Strings are Fine:
AbiTag
TLSModel
Availability
Deprecated
EnableIf/DiagnoseIf
ObjCRuntimeName
PragmaClangBSSSection/PragmaClangDataSection/PragmaClangRodataSection/PragmaClangRelroSection/PragmaClangTextSection (only created implicitly)
Suppress
Target/TargetVersion/TargetClones
Unavailable
Uuid
WarnUnusedResult
NoSanitize
Capability
Assumption
NoBuiltin (it names a builtin name, so this is probably fine to leave unevaluated?)
AcquireHandle/UseHandle/ReleaseHandle
Error
HLSLResourceBinding

Unevaluated String are Potentially Bad:
Annotate
AnnotateType

Unevaluated String Needs More Thinking (common thread is that they survive to LLVM IR):
Alias
AsmLabel
IFunc
BTFDeclTag/BTFTypeTag (is emitted to DWARF with -g so probably evaluated?)
WebAssemblyExportName/WebAssemblyImportModule/WebAssemblyImportModule
ExternalSourceSymbol
SwiftAsyncName/SwiftAttr/SwiftBridge/SwiftName
Section/CodeSeg/InitSeg
WeakRef
EnforceTCB/EnforceTCBLeaf

There's also the escape sequences issue where use of an escape sequence will go from accepted to rejected in these contexts. I did some hunting to see if I could find uses of numeric escape sequences in asm labels or alias attributes, to see if there's some signs this is done in practice:

Testing we can find numeric escape sequences at all:
https://sourcegraph.com/search?q=context:global+%5C%28%5C%22%5B%5B:alpha:%5D%5D*%28%5C%5C%5B%5B:digit:%5D%5D%2B%29%2B%5B%5B:alpha:%5D%5D*%5C%22%5C%29+lang:C+lang:C%2B%2B&patternType=regexp&case=yes&sm=1&groupBy=repo

Testing we can find asm labels at all:
https://sourcegraph.com/search?q=context:global+asm%5C%28%5C%22%5B%5B:alpha:%5D%5D*%5B%5B:alpha:%5D%5D*%5C%22%5C%29+lang:C+lang:C%2B%2B&patternType=regexp&case=yes&sm=1&groupBy=repo

Testing we can find asm labels with numeric escapes:
https://sourcegraph.com/search?q=context:global+asm%5C%28%5C%22%5B%5B:alpha:%5D%5D*%28%5C%5C%5B%5B:digit:%5D%5D%2B%29%2B%5B%5B:alpha:%5D%5D*%5C%22%5C%29+lang:C+lang:C%2B%2B&patternType=regexp&case=yes&sm=1&groupBy=repo

Testing we can find alias attributes at all:
https://sourcegraph.com/search?q=context:global+alias%5C%28%5C%22%5B%5B:alpha:%5D%5D*%5B%5B:alpha:%5D%5D*%5C%22%5C%29+lang:C+lang:C%2B%2B&patternType=regexp&case=yes&sm=1&groupBy=repo

Testing we can find alias attributes with numeric escapes:
https://sourcegraph.com/search?q=context:global+alias%5C%28%5C%22%5B%5B:alpha:%5D%5D*%28%5C%5C%5B%5B:digit:%5D%5D%2B%29%2B%5B%5B:alpha:%5D%5D*%5C%22%5C%29+lang:C+lang:C%2B%2B&patternType=regexp&case=yes&sm=1&groupBy=repo

I think this leaves me with three open questions:

• Do we know of any uses of the annotate attribute that rely on the string literal being in the execution character set? I do not know of any but I know this is used by plugins quite often.
• Do we know of any attributes in the "needs more thinking" list that should have the string literal encoded in the execution character set? I think most of these are for referring to identifiers in source and I expect those would want source character set and not execution character set strings.
• Do we know of any significant body of code using numeric escape sequences in these string literals that could not be relatively easily modified to compile again? I would be surprised, but I think someone should probably run more of the attributes on the "needs more thinking" list through similar searches on source graph and we can use that as an approximation.

If all these answers come back "no" as best we can figure, then I think we can punt on argument-level handling of this until we add an attribute that really does need an execution-encoded (or numeric escape sequence-using) string literal. I think we've got enough time before the Clang 17 branch to hear if the changes cause problems after we've done this due diligence. WDYT?

[cor3ntin]

In D105759#4456864, @aaron.ballman wrote:

I don't think it's correct to assume that all string arguments to attributes are unevaluated, but it is hard to tell where to draw the line sometimes. Backing up a step, as I understand P2361, an unevaluated string is one which is not converted into the execution character set (effectively). Is that correct? If so, then as an example, [[clang::annotate()]] should almost certainly be using an evaluated string because the argument is passed down to LLVM IR and is used in ways we cannot predict. What's more, an unevaluated string cannot have some kinds of escape characters (numeric and conditional escape sequences) and those are currently allowed by clang::annotate and could potentially be used by a backend plugin.

I think other attributes may have similar issues. For example, the alias attribute is a bit of a question mark for me -- that takes a string literal representing an external identifier that is looked up. I'm not certain whether that should be in the execution character set or not, but we do support escape sequences for it: https://godbolt.org/z/v65Yd7a68

I think we need to track evaluated vs not on the argument level so that the attributes in Attr.td can decide which form to use. I think we should default to "evaluated" for any attribute we're on the fence about because that's the current behavior they get today (so we should avoid regressions).

I really don't think it makes sense to have both "unevaluated" and "evaluated" arguments.
We chatted offline and we struggle to find places where escape sequences are used, or examples of attributes intended to be in the execution character set.

My suggestion would be to land the non-attributes changes now, and the attributes bits in early clang 18.
If we find clear example of attributes expecting execution character set, they should be able to be described as an expression, which will be checked as a string literal anyway, hopefully?

In the case of annotate, if these are fed, for example to a debugger, their may need to convert to whatever the debugger expect as encoding, which is not necessarily the execution charset,
Same for plugins, they certainly not expect ebcdic data, for example.
I would expect for example static analyzers and code generator to keep working after the introduction of fexec-charset
So it's important that it remains unevaluated in the front end so that it can be correctly converted to the appropriate encoding of the various consumers. Which doesn't have a single answer

Do we know of any attributes in the "needs more thinking" list that should have the string literal encoded in the execution character set? I think most of these are for referring to identifiers in source and I expect those would want source character set and not execution character set strings.

Identifiers and symbol names are in UTF8, and may get mangle through, for example replacing non-ascii codepoints by UCN. The source character set is never relevant
This address the WebAsm attributes

BTFDeclTag/BTFTypeTag (is emitted to DWARF with -g so probably evaluated?)

Is it correct to assume the debugger file encoding is always the same as the program's ? Probably not!
If need be, we can then transcode the strings when doing codegen for these things

[aaron.ballman]

In D105759#4457041, @cor3ntin wrote:

In D105759#4456864, @aaron.ballman wrote:

I don't think it's correct to assume that all string arguments to attributes are unevaluated, but it is hard to tell where to draw the line sometimes. Backing up a step, as I understand P2361, an unevaluated string is one which is not converted into the execution character set (effectively). Is that correct? If so, then as an example, [[clang::annotate()]] should almost certainly be using an evaluated string because the argument is passed down to LLVM IR and is used in ways we cannot predict. What's more, an unevaluated string cannot have some kinds of escape characters (numeric and conditional escape sequences) and those are currently allowed by clang::annotate and could potentially be used by a backend plugin.

I think other attributes may have similar issues. For example, the alias attribute is a bit of a question mark for me -- that takes a string literal representing an external identifier that is looked up. I'm not certain whether that should be in the execution character set or not, but we do support escape sequences for it: https://godbolt.org/z/v65Yd7a68

I think we need to track evaluated vs not on the argument level so that the attributes in Attr.td can decide which form to use. I think we should default to "evaluated" for any attribute we're on the fence about because that's the current behavior they get today (so we should avoid regressions).

I really don't think it makes sense to have both "unevaluated" and "evaluated" arguments.
We chatted offline and we struggle to find places where escape sequences are used, or examples of attributes intended to be in the execution character set.

In general I agree, but the one scenario that I keep coming back to are attributes like diagnose_if where they take an expression we're evaluating at compile time (condition expression) and a string literal that's not evaluated (warning vs error, diagnostic message itself). But I think the "evaluating at compile time" is part of why I don't think we intend the attribute to be considering the execution character set.

My suggestion would be to land the non-attributes changes now, and the attributes bits in early clang 18.

I think we're almost safe enough to make the attribute changes in Clang 17 so that no attribute uses an evaluated argument, but given that there's less than a month before we make the 17 branch, I think it's probably a good idea to make these changes after the branch point so folks have longer to react. Adding clang-vendors to the review for awareness of the potential for a breaking change.

[cor3ntin]

I removed the changes to attributes.
Nothing else changes except cxx_status/ReleaseNotes.

Unevaluated strings in attributes will be back (in a separate PR)

[aaron.ballman]

LGTM with a minor tweak to the wording on the status page, thank you!

clang/www/cxx_status.html

118–120 ↗

(On Diff #538073)

[barannikov88]

clang/docs/ReleaseNotes.rst

138 ↗

(On Diff #538078)

Looks like a copy&paste bug.

[cor3ntin]

clang/docs/ReleaseNotes.rst

138 ↗

(On Diff #538078)

Nice catch, thanks

[barannikov88]

@cor3ntin
I've been working on pretty much the same functionality in our downstream fork. I was not aware of the paper, nor of the ongoing work in this direction, and so I unfortunately missed the review.
Thanks for this patch, it significantly reduces the number of changes downstream and makes it easier to merge with upstream in the future.

I have a couple of questions about future work:

• IIUC the paper initially addressed this issue with #line directive, but the changes were reverted(?). Is there any chance they can get back?
• Are there any plans for making similar changes to asm statement parsing?

[cor3ntin]

In D105759#4482543, @barannikov88 wrote:

@cor3ntin
I've been working on pretty much the same functionality in our downstream fork. I was not aware of the paper, nor of the ongoing work in this direction, and so I unfortunately missed the review.
Thanks for this patch, it significantly reduces the number of changes downstream and makes it easier to merge with upstream in the future.

I have a couple of questions about future work:

• IIUC the paper initially addressed this issue with #line directive, but the changes were reverted(?). Is there any chance they can get back?

There is a core issue tracking that, ie the c++ committee was concerned about escape sequences in header names
https://www.open-std.org/jtc1/sc22/wg21/docs/cwg_active.html#2693

I'd be happy to bring that back to clang though, as the concerned is unlikely to be warranted for us.

• Are there any plans for making similar changes to asm statement parsing?

The direction of the c++ committee is that what's in asm() is now strictly implementation-defined, so we could but last time there were concerns about escape sequences in there too.

[hubert.reinterpretcast]

I hope this patch may allow to gather some data on that.

@cor3ntin, I have reports that applications having encoding prefixes in static_assert are failing to build. The committee did not adopt the subject paper as a "DR resolution". Is it possible to downgrade to a warning?

[cor3ntin]

In D105759#4540716, @hubert.reinterpretcast wrote:

I hope this patch may allow to gather some data on that.

@cor3ntin, I have reports that applications having encoding prefixes in static_assert are failing to build. The committee did not adopt the subject paper as a "DR resolution". Is it possible to downgrade to a warning?

You know how frequent it is?

Making it a warning is possible but not straightforward.

Previously with a prefix, it was parsed as a wide (for example) string, and we relied on the fact that L was UTF-16/32 to sometimes print a reasonable diagnostics - and sometimes not https://godbolt.org/z/f3Pj4T5aj
This is going to be worse when we add -fexec-charset:

• static_assert(true, L"やあ") is going to be ill-formed when coded as, eg, EBCDIC because it's not representable, and if it is representable we need to either output mojibake or convert the string back to UTF-8 which we are currently not doing.

Another solution maybe to lexically ignore prefixes by replacing the string literal token on the fly such that they are still parsed as unevaluated strings and not encoded, i could look into that.

[hubert.reinterpretcast]

In D105759#4541813, @cor3ntin wrote:

In D105759#4540716, @hubert.reinterpretcast wrote:

I hope this patch may allow to gather some data on that.

@cor3ntin, I have reports that applications having encoding prefixes in static_assert are failing to build. The committee did not adopt the subject paper as a "DR resolution". Is it possible to downgrade to a warning?

You know how frequent it is?

No, but I am concerned that this came up even before we deployed an LLVM 17-based solution (in pre-release testing). I believe that reverting for LLVM 17 is the prudent course of action.

Previously with a prefix, it was parsed as a wide (for example) string, and we relied on the fact that L was UTF-16/32 to sometimes print a reasonable diagnostics - and sometimes not https://godbolt.org/z/f3Pj4T5aj
This is going to be worse when we add -fexec-charset:

• static_assert(true, L"やあ") is going to be ill-formed when coded as, eg, EBCDIC because it's not representable, and if it is representable we need to either output mojibake or convert the string back to UTF-8 which we are currently not doing.

This may be the motivation for the prefixes in the applications in the first place in the context of other compilers: They may have needed the prefix to avoid unrepresentable character issues (e.g., if the other compiler rejects the unprefixed string, but manages to emit the error to the terminal because both the terminal and the compiler use the source encoding).

Another solution maybe to lexically ignore prefixes by replacing the string literal token on the fly such that they are still parsed as unevaluated strings and not encoded, i could look into that.

This sounds good.

[aaron.ballman]

In D105759#4543184, @hubert.reinterpretcast wrote:

In D105759#4541813, @cor3ntin wrote:

In D105759#4540716, @hubert.reinterpretcast wrote:

I hope this patch may allow to gather some data on that.

@cor3ntin, I have reports that applications having encoding prefixes in static_assert are failing to build. The committee did not adopt the subject paper as a "DR resolution". Is it possible to downgrade to a warning?

You know how frequent it is?

No, but I am concerned that this came up even before we deployed an LLVM 17-based solution (in pre-release testing). I believe that reverting for LLVM 17 is the prudent course of action.

Early reports of user code getting tripped up on this is something we should react to while we still can; I'd recommend we change the diagnostic to be a warning that defaults to an error so that users who are caught by the changes can still disable the diagnostic rather than be stuck; for Clang 18, we can explore other solutions to the issue. Would this work for you @hubert.reinterpretcast?

[hubert.reinterpretcast]

In D105759#4543246, @aaron.ballman wrote:

I'd recommend we change the diagnostic to be a warning that defaults to an error so that users who are caught by the changes can still disable the diagnostic rather than be stuck; for Clang 18, we can explore other solutions to the issue. Would this work for you @hubert.reinterpretcast?

I think there are questions about whether an error (or even warning) by default is appropriate. This seems to be a change for C++2c that does not have "DR" treatment from the committee. Considering this a warning controlled by c++2c-compat is a potential direction. Indeed, if we are going to accept the code, we might as well allow it as an extension in C++2c modes. With this line of logic, I don't see why we would want user-side churn of making a migration effort.

[cor3ntin]

In D105759#4543685, @hubert.reinterpretcast wrote:

In D105759#4543246, @aaron.ballman wrote:

I'd recommend we change the diagnostic to be a warning that defaults to an error so that users who are caught by the changes can still disable the diagnostic rather than be stuck; for Clang 18, we can explore other solutions to the issue. Would this work for you @hubert.reinterpretcast?

I think there are questions about whether an error (or even warning) by default is appropriate. This seems to be a change for C++2c that does not have "DR" treatment from the committee. Considering this a warning controlled by c++2c-compat is a potential direction. Indeed, if we are going to accept the code, we might as well allow it as an extension in C++2c modes. With this line of logic, I don't see why we would want user-side churn of making a migration effort.

I will endeavor to have a patch by the beginning of the week.

I think the implementation effort is going to be the same whether it is an error by default or not so we can discuss that. I don't have a strong opinion.
Ideally, that would depend on how many users are affected.

However, I don't think nothing at all is a reasonable expectation here, L in static_assert message does either not work or is ignored. In no case does it do what the user wants https://godbolt.org/z/fYnMqT38P
Text encodings are sufficiently confusing that we should not add to the confusion by not telling users their encodings prefix have no effects.

And, given that prior to c++20 the standard simply ignores encoding prefixes, we could also discuss whether it was ever intended for prefixes to be supported or whether it was an oversight to begin with.

clang-tools-extra/test/clang-tidy/checkers/modernize-unary-static-assert.cpp

 // RUN: %check_clang_tidy -std=c++17-or-later %s modernize-unary-static-assert %t
 
 #define FOO static_assert(sizeof(a) <= 15, "");
 #define MSG ""
 
 void f_textless(int a) {
   static_assert(sizeof(a) <= 10, "");
   // CHECK-MESSAGES: :[[@LINE-1]]:3: warning: use unary 'static_assert' when the string literal is an empty string [modernize-unary-static-assert]
   // CHECK-FIXES: {{^}}  static_assert(sizeof(a) <= 10 );{{$}}
-  static_assert(sizeof(a) <= 12, L"");
-  // CHECK-MESSAGES: :[[@LINE-1]]:3: warning: use unary 'static_assert' when
-  // CHECK-FIXES: {{^}}  static_assert(sizeof(a) <= 12 );{{$}}
   FOO
   // CHECK-FIXES: {{^}}  FOO{{$}}
   static_assert(sizeof(a) <= 17, MSG);
   // CHECK-FIXES: {{^}}  static_assert(sizeof(a) <= 17, MSG);{{$}}
 }
 
 void f_with_tex(int a) {
   static_assert(sizeof(a) <= 10, "Size of variable a is out of range!");
 }
 
 void f_unary(int a) { static_assert(sizeof(a) <= 10); }
 
 void f_incorrect_assert() { static_assert(""); }

clang/include/clang/AST/Expr.h

class StringLiteral final
   ///   consider moving it inside StringLiteral.
   ///
   /// * An array of getNumConcatenated() SourceLocation, one for each of the
   ///   token this string is made of.
   ///
   /// * An array of getByteLength() char used to store the string data.
 
 public:
-  enum StringKind { Ascii, Wide, UTF8, UTF16, UTF32 };
+  enum StringKind { Ascii, Wide, UTF8, UTF16, UTF32, Unevaluated };
 
 private:
   unsigned numTrailingObjects(OverloadToken<unsigned>) const { return 1; }
   unsigned numTrailingObjects(OverloadToken<SourceLocation>) const {
     return getNumConcatenated();
   }
 
   unsigned numTrailingObjects(OverloadToken<char>) const {
public:
   }
 
   /// Construct an empty string literal.
   static StringLiteral *CreateEmpty(const ASTContext &Ctx,
                                     unsigned NumConcatenated, unsigned Length,
                                     unsigned CharByteWidth);
 
   StringRef getString() const {
-    assert(getCharByteWidth() == 1 &&
+    assert((isUnevaluated() || getCharByteWidth() == 1) &&
            "This function is used in places that assume strings use char");
     return StringRef(getStrDataAsChar(), getByteLength());
   }
 
   /// Allow access to clients that need the byte representation, such as
   /// ASTWriterStmt::VisitStringLiteral().
   StringRef getBytes() const {
     // FIXME: StringRef may not be the right type to use as a result for this.
     return StringRef(getStrDataAsChar(), getByteLength());
   }
StringKind getKind() const {
     return static_cast<StringKind>(StringLiteralBits.Kind);
   }
 
   bool isAscii() const { return getKind() == Ascii; }
   bool isWide() const { return getKind() == Wide; }
   bool isUTF8() const { return getKind() == UTF8; }
   bool isUTF16() const { return getKind() == UTF16; }
   bool isUTF32() const { return getKind() == UTF32; }
+  bool isUnevaluated() const { return getKind() == Unevaluated; }
   bool isPascal() const { return StringLiteralBits.IsPascal; }
 
   bool containsNonAscii() const {
     for (auto c : getString())
       if (!isASCII(c))
         return true;
     return false;
   }

clang/include/clang/Basic/DiagnosticLexKinds.td

 def ext_reserved_user_defined_literal : ExtWarn<
   "invalid suffix on literal; C++11 requires a space between literal and "
   "identifier">, InGroup<ReservedUserDefinedLiteral>, DefaultError;
 def ext_ms_reserved_user_defined_literal : ExtWarn<
   "invalid suffix on literal; C++11 requires a space between literal and "
   "identifier">, InGroup<ReservedUserDefinedLiteral>;
 def err_unsupported_string_concat : Error<
   "unsupported non-standard concatenation of string literals">;
+
+def err_unevaluated_string_prefix : Error<
+  "an unevaluated string literal cannot have an encoding prefix">;
+def err_unevaluated_string_udl : Error<
+  "an unevaluated string literal cannot be a user-defined literal">;
+def err_unevaluated_string_invalid_escape_sequence : Error<
+  "invalid escape sequence '%0' in an unevaluated string literal">;
 def err_string_concat_mixed_suffix : Error<
   "differing user-defined suffixes ('%0' and '%1') in string literal "
   "concatenation">;
 def err_pp_invalid_udl : Error<
   "%select{character|integer}0 literal with user-defined suffix "
   "cannot be used in preprocessor constant expression">;
 def err_bad_string_encoding : Error<
   "illegal character encoding in string literal">;

clang/include/clang/Basic/DiagnosticSemaKinds.td

def ext_implicit_function_decl : ExtWarn<
   "implicit declaration of function %0 is invalid in C99">,
   InGroup<ImplicitFunctionDeclare>;
 def note_function_suggestion : Note<"did you mean %0?">;
 
 def err_ellipsis_first_param : Error<
   "ISO C requires a named parameter before '...'">;
 def err_declarator_need_ident : Error<"declarator requires an identifier">;
 def err_language_linkage_spec_unknown : Error<"unknown linkage language">;
-def err_language_linkage_spec_not_ascii : Error<
-  "string literal in language linkage specifier cannot have an "
-  "encoding-prefix">;
 def ext_use_out_of_scope_declaration : ExtWarn<
   "use of out-of-scope declaration of %0%select{| whose type is not "
   "compatible with that of an implicit declaration}1">,
   InGroup<DiagGroup<"out-of-scope-function">>;
 def err_inline_non_function : Error<
   "'inline' can only appear on functions%select{| and non-local variables}0">;
 def err_noreturn_non_function : Error<
   "'_Noreturn' can only appear on functions">;

clang/include/clang/Lex/LiteralSupport.h

public:
   uint64_t getValue() const { return Value; }
   StringRef getUDSuffix() const { return UDSuffixBuf; }
   unsigned getUDSuffixOffset() const {
     assert(!UDSuffixBuf.empty() && "no ud-suffix");
     return UDSuffixOffset;
   }
 };
 
+enum class StringLiteralEvalMethod {
+  Evaluated,
+  Unevaluated,
+};
+
 /// StringLiteralParser - This decodes string escape characters and performs
 /// wide string analysis and Translation Phase #6 (concatenation of string
 /// literals) (C99 5.1.1.2p1).
 class StringLiteralParser {
   const SourceManager &SM;
   const LangOptions &Features;
   const TargetInfo &Target;
   DiagnosticsEngine *Diags;
 
   unsigned MaxTokenLength;
   unsigned SizeBound;
   unsigned CharByteWidth;
   tok::TokenKind Kind;
   SmallString<512> ResultBuf;
   char *ResultPtr; // cursor
   SmallString<32> UDSuffixBuf;
   unsigned UDSuffixToken;
   unsigned UDSuffixOffset;
+  StringLiteralEvalMethod EvalMethod;
+
 public:
-  StringLiteralParser(ArrayRef<Token> StringToks,
-                      Preprocessor &PP);
-  StringLiteralParser(ArrayRef<Token> StringToks,
-                      const SourceManager &sm, const LangOptions &features,
-                      const TargetInfo &target,
+  StringLiteralParser(
+      ArrayRef<Token> StringToks, Preprocessor &PP,
+      StringLiteralEvalMethod StringKind = StringLiteralEvalMethod::Evaluated);
+  StringLiteralParser(ArrayRef<Token> StringToks, const SourceManager &sm,
+                      const LangOptions &features, const TargetInfo &target,
                       DiagnosticsEngine *diags = nullptr)
       : SM(sm), Features(features), Target(target), Diags(diags),
         MaxTokenLength(0), SizeBound(0), CharByteWidth(0), Kind(tok::unknown),
-      ResultPtr(ResultBuf.data()), hadError(false), Pascal(false) {
+        ResultPtr(ResultBuf.data()),
+        EvalMethod(StringLiteralEvalMethod::Evaluated), hadError(false),
+        Pascal(false) {
     init(StringToks);
   }
 
-
   bool hadError;
   bool Pascal;
 
   StringRef GetString() const {
     return StringRef(ResultBuf.data(), GetStringLength());
   }
   unsigned GetStringLength() const { return ResultPtr-ResultBuf.data(); }
 
   unsigned GetNumStringChars() const {
     return GetStringLength() / CharByteWidth;
   }
   /// getOffsetOfStringByte - This function returns the offset of the
   /// specified byte of the string data represented by Token.  This handles
   /// advancing over escape sequences in the string.
   ///
   /// If the Diagnostics pointer is non-null, then this will do semantic
   /// checking of the string literal and emit errors and warnings.
   unsigned getOffsetOfStringByte(const Token &TheTok, unsigned ByteNo) const;
 
   bool isAscii() const { return Kind == tok::string_literal; }
   bool isWide() const { return Kind == tok::wide_string_literal; }
   bool isUTF8() const { return Kind == tok::utf8_string_literal; }
   bool isUTF16() const { return Kind == tok::utf16_string_literal; }
   bool isUTF32() const { return Kind == tok::utf32_string_literal; }
   bool isPascal() const { return Pascal; }
+  bool isUnevaluated() const {
+    return EvalMethod == StringLiteralEvalMethod::Unevaluated;
+  }
 
   StringRef getUDSuffix() const { return UDSuffixBuf; }
 
   /// Get the index of a token containing a ud-suffix.
   unsigned getUDSuffixToken() const {
     assert(!UDSuffixBuf.empty() && "no ud-suffix");
     return UDSuffixToken;
   }

clang/include/clang/Parse/Parser.h

public:
   // Expr that doesn't include commas.
   ExprResult ParseAssignmentExpression(TypeCastState isTypeCast = NotTypeCast);
 
   ExprResult ParseMSAsmIdentifier(llvm::SmallVectorImpl<Token> &LineToks,
                                   unsigned &NumLineToksConsumed,
                                   bool IsUnevaluated);
 
   ExprResult ParseStringLiteralExpression(bool AllowUserDefinedLiteral = false);
+  ExprResult ParseUnevaluatedStringLiteralExpression();
 
 private:
+  ExprResult ParseStringLiteralExpression(bool AllowUserDefinedLiteral,
+                                          bool Unevaluated);
+
   ExprResult ParseExpressionWithLeadingAt(SourceLocation AtLoc);
 
   ExprResult ParseExpressionWithLeadingExtension(SourceLocation ExtLoc);
 
   ExprResult ParseRHSOfBinaryExpression(ExprResult LHS,
                                         prec::Level MinPrec);
   /// Control what ParseCastExpression will parse.
   enum CastParseKind {
ExprResult ParseExprAfterUnaryExprOrTypeTrait(const Token &OpTok,
                                                      ParsedType &CastTy,
                                                      SourceRange &CastRange);
 
   typedef SmallVector<SourceLocation, 20> CommaLocsTy;
 
   /// ParseExpressionList - Used for C/C++ (argument-)expression-list.
   bool ParseExpressionList(SmallVectorImpl<Expr *> &Exprs,
                            SmallVectorImpl<SourceLocation> &CommaLocs,
                            llvm::function_ref<void()> ExpressionStarts =
                                llvm::function_ref<void()>());
 
   /// ParseSimpleExpressionList - A simple comma-separated list of expressions,
   /// used for misc language extensions.
   bool ParseSimpleExpressionList(SmallVectorImpl<Expr*> &Exprs,
                                  SmallVectorImpl<SourceLocation> &CommaLocs);
 
 
   /// ParenParseOption - Control what ParseParenExpression will parse.
   enum ParenParseOption {
private:
   /// list with the given attribute syntax. Returns the number of arguments
   /// parsed for the attribute.
   unsigned
   ParseAttributeArgsCommon(IdentifierInfo *AttrName, SourceLocation AttrNameLoc,
                            ParsedAttributes &Attrs, SourceLocation *EndLoc,
                            IdentifierInfo *ScopeName, SourceLocation ScopeLoc,
                            ParsedAttr::Syntax Syntax);
 
+  ExprResult
+  ParseAttributeArgAsUnevaluatedLiteralOrExpression(ParsedAttr::Kind Kind);
+
   enum ParseAttrKindMask {
     PAKM_GNU = 1 << 0,
     PAKM_Declspec = 1 << 1,
     PAKM_CXX11 = 1 << 2,
   };
 
   /// \brief Parse attributes based on what syntaxes are desired, allowing for
   /// the order to vary. e.g. with PAKM_GNU | PAKM_Declspec:

clang/include/clang/Sema/Sema.h

ExprResult ActOnParenListExpr(SourceLocation L,
                                 SourceLocation R,
                                 MultiExprArg Val);
 
   /// ActOnStringLiteral - The specified tokens were lexed as pasted string
   /// fragments (e.g. "foo" "bar" L"baz").
   ExprResult ActOnStringLiteral(ArrayRef<Token> StringToks,
                                 Scope *UDLScope = nullptr);
 
+  ExprResult ActOnUnevaluatedStringLiteral(ArrayRef<Token> StringToks);
+
   ExprResult ActOnGenericSelectionExpr(SourceLocation KeyLoc,
                                        SourceLocation DefaultLoc,
                                        SourceLocation RParenLoc,
                                        Expr *ControllingExpr,
                                        ArrayRef<ParsedType> ArgTypes,
                                        ArrayRef<Expr *> ArgExprs);
   ExprResult CreateGenericSelectionExpr(SourceLocation KeyLoc,
                                         SourceLocation DefaultLoc,

clang/lib/AST/Expr.cpp

case Wide:
     CharByteWidth = Target.getWCharWidth();
     break;
   case UTF16:
     CharByteWidth = Target.getChar16Width();
     break;
   case UTF32:
     CharByteWidth = Target.getChar32Width();
     break;
+  case Unevaluated:
+    return sizeof(char); // Host;
+    break;
   }
   assert((CharByteWidth & 7) == 0 && "Assumes character size is byte multiple");
   CharByteWidth /= 8;
   assert((CharByteWidth == 1 || CharByteWidth == 2 || CharByteWidth == 4) &&
          "The only supported character byte widths are 1,2 and 4!");
   return CharByteWidth;
 }
 
 StringLiteral::StringLiteral(const ASTContext &Ctx, StringRef Str,
                              StringKind Kind, bool Pascal, QualType Ty,
                              const SourceLocation *Loc,
                              unsigned NumConcatenated)
     : Expr(StringLiteralClass, Ty, VK_LValue, OK_Ordinary) {
+
+  unsigned Length = Str.size();
+
+  StringLiteralBits.Kind = Kind;
+  StringLiteralBits.NumConcatenated = NumConcatenated;
+
+  if (Kind != StringKind::Unevaluated) {
     assert(Ctx.getAsConstantArrayType(Ty) &&
            "StringLiteral must be of constant array type!");
     unsigned CharByteWidth = mapCharByteWidth(Ctx.getTargetInfo(), Kind);
     unsigned ByteLength = Str.size();
     assert((ByteLength % CharByteWidth == 0) &&
            "The size of the data must be a multiple of CharByteWidth!");
 
     // Avoid the expensive division. The compiler should be able to figure it
     // out by itself. However as of clang 7, even with the appropriate
     // llvm_unreachable added just here, it is not able to do so.
-  unsigned Length;
     switch (CharByteWidth) {
     case 1:
       Length = ByteLength;
       break;
     case 2:
       Length = ByteLength / 2;
       break;
     case 4:
       Length = ByteLength / 4;
       break;
     default:
       llvm_unreachable("Unsupported character width!");
     }
 
-  StringLiteralBits.Kind = Kind;
     StringLiteralBits.CharByteWidth = CharByteWidth;
     StringLiteralBits.IsPascal = Pascal;
-  StringLiteralBits.NumConcatenated = NumConcatenated;
+  } else {
+    StringLiteralBits.CharByteWidth = 1;
+    StringLiteralBits.IsPascal = false;
+  }
+
   *getTrailingObjects<unsigned>() = Length;
 
   // Initialize the trailing array of SourceLocation.
   // This is safe since SourceLocation is POD-like.
   std::memcpy(getTrailingObjects<SourceLocation>(), Loc,
               NumConcatenated * sizeof(SourceLocation));
 
   // Initialize the trailing array of char holding the string data.
-  std::memcpy(getTrailingObjects<char>(), Str.data(), ByteLength);
+  std::memcpy(getTrailingObjects<char>(), Str.data(), Str.size());
 
   setDependence(ExprDependence::None);
 }
 
 StringLiteral::StringLiteral(EmptyShell Empty, unsigned NumConcatenated,
                              unsigned Length, unsigned CharByteWidth)
     : Expr(StringLiteralClass, Empty) {
   StringLiteralBits.CharByteWidth = CharByteWidth;
void *Mem = Ctx.Allocate(totalSizeToAlloc<unsigned, SourceLocation, char>(
                                1, NumConcatenated, Length * CharByteWidth),
                            alignof(StringLiteral));
   return new (Mem)
       StringLiteral(EmptyShell(), NumConcatenated, Length, CharByteWidth);
 }
 
 void StringLiteral::outputString(raw_ostream &OS) const {
   switch (getKind()) {
+  case Unevaluated:  // fallthrough. no prefix.
   case Ascii: break; // no prefix.
   case Wide:  OS << 'L'; break;
   case UTF8:  OS << "u8"; break;
   case UTF16: OS << 'u'; break;
   case UTF32: OS << 'U'; break;
   }
   OS << '"';
   static const char Hex[] = "0123456789ABCDEF";
 

 /// string.
 ///
 SourceLocation
 StringLiteral::getLocationOfByte(unsigned ByteNo, const SourceManager &SM,
                                  const LangOptions &Features,
                                  const TargetInfo &Target, unsigned *StartToken,
                                  unsigned *StartTokenByteOffset) const {
   assert((getKind() == StringLiteral::Ascii ||
-          getKind() == StringLiteral::UTF8) &&
+          getKind() == StringLiteral::UTF8 ||
+          getKind() == StringLiteral::Unevaluated) &&
          "Only narrow string literals are currently supported");
 
   // Loop over all of the tokens in this string until we find the one that
   // contains the byte we're looking for.
   unsigned TokNo = 0;
   unsigned StringOffset = 0;
   if (StartToken)
     TokNo = *StartToken;

clang/lib/Frontend/FrontendAction.cpp

if (Lexer::getSpelling(LineNoLoc, Buffer, SourceMgr, CI.getLangOpts())
             .getAsInteger(10, LineNo))
       return SourceLocation();
   }
 
   RawLexer->LexFromRawLexer(T);
   if (T.isAtStartOfLine() || T.getKind() != tok::string_literal)
     return SourceLocation();
 
-  StringLiteralParser Literal(T, CI.getPreprocessor());
+  StringLiteralParser Literal(T, CI.getPreprocessor(),
+                              StringLiteralEvalMethod::Unevaluated);
   if (Literal.hadError)
     return SourceLocation();
   RawLexer->LexFromRawLexer(T);
   if (T.isNot(tok::eof) && !T.isAtStartOfLine())
     return SourceLocation();
   InputFile = Literal.GetString().str();
 
   if (IsModuleMap)

clang/lib/Lex/LiteralSupport.cpp

static DiagnosticBuilder Diag(DiagnosticsEngine *Diags,
                               const char *TokRangeEnd, unsigned DiagID) {
   SourceLocation Begin =
     Lexer::AdvanceToTokenCharacter(TokLoc, TokRangeBegin - TokBegin,
                                    TokLoc.getManager(), Features);
   return Diags->Report(Begin, DiagID) <<
     MakeCharSourceRange(Features, TokLoc, TokBegin, TokRangeBegin, TokRangeEnd);
 }
 
+static bool EscapeValidInUnevaluatedStringLiteral(char Escape) {
+  switch (Escape) {
+  case '\\':
+  case '\'':
+  case '"':
+  case '?':
+  case 'n':
+  case 't':
+  case 'r':
+    return true;
+  }
+  return false;
+}
+
 /// ProcessCharEscape - Parse a standard C escape sequence, which can occur in
 /// either a character or a string literal.
 static unsigned ProcessCharEscape(const char *ThisTokBegin,
                                   const char *&ThisTokBuf,
                                   const char *ThisTokEnd, bool &HadError,
                                   FullSourceLoc Loc, unsigned CharWidth,
                                   DiagnosticsEngine *Diags,
-                                  const LangOptions &Features) {
+                                  const LangOptions &Features,
+                                  StringLiteralEvalMethod EvalMethod) {
   const char *EscapeBegin = ThisTokBuf;
   bool Delimited = false;
   bool EndDelimiterFound = false;
 
   // Skip the '\' char.
   ++ThisTokBuf;
 
   // We know that this character can't be off the end of the buffer, because
   // that would have been \", which would not have been the end of string.
   unsigned ResultChar = *ThisTokBuf++;
   switch (ResultChar) {
else
            diag::ext_unknown_escape)
         << "x" + llvm::utohexstr(ResultChar);
     break;
   }
 
   if (Delimited && Diags) {
     if (!EndDelimiterFound)
       Diag(Diags, Features, Loc, ThisTokBegin, EscapeBegin, ThisTokBuf,
            diag::err_expected)
           << tok::r_brace;
     else if (!HadError) {
       Diag(Diags, Features, Loc, ThisTokBegin, EscapeBegin, ThisTokBuf,
            diag::ext_delimited_escape_sequence);
     }
   }
 
+  if (EvalMethod == StringLiteralEvalMethod::Unevaluated &&
+      !EscapeValidInUnevaluatedStringLiteral(*EscapeBegin)) {
+    Diag(Diags, Features, Loc, ThisTokBegin, EscapeBegin, ThisTokBuf,
+         diag::err_unevaluated_string_invalid_escape_sequence)
+        << StringRef(EscapeBegin + 1, 1);
+  }
   return ResultChar;
 }
 
 static void appendCodePoint(unsigned Codepoint,
                             llvm::SmallVectorImpl<char> &Str) {
   char ResultBuf[4];
   char *ResultPtr = ResultBuf;
   bool Res = llvm::ConvertCodePointToUTF8(Codepoint, ResultPtr);
if (begin[1] == 'u' || begin[1] == 'U') {
         PP.Diag(Loc, diag::err_character_too_large);
       }
 
       ++buffer_begin;
       continue;
     }
     unsigned CharWidth = getCharWidth(Kind, PP.getTargetInfo());
     uint64_t result =
         ProcessCharEscape(TokBegin, begin, end, HadError,
-                        FullSourceLoc(Loc,PP.getSourceManager()),
-                        CharWidth, &PP.getDiagnostics(), PP.getLangOpts());
+                          FullSourceLoc(Loc, PP.getSourceManager()), CharWidth,
+                          &PP.getDiagnostics(), PP.getLangOpts(),
+                          StringLiteralEvalMethod::Evaluated);
     *buffer_begin++ = result;
   }
 
   unsigned NumCharsSoFar = buffer_begin - &codepoint_buffer.front();
 
   if (NumCharsSoFar > 1) {
     if (isAscii() && NumCharsSoFar == 4)
       PP.Diag(Loc, diag::warn_four_char_character_literal);

 ///         hexadecimal-escape-sequence hexadecimal-digit
 ///       universal-character-name:
 ///         \u hex-quad
 ///         \U hex-quad hex-quad
 ///       hex-quad:
 ///         hex-digit hex-digit hex-digit hex-digit
 /// \endverbatim
 ///
-StringLiteralParser::
-StringLiteralParser(ArrayRef<Token> StringToks,
-                    Preprocessor &PP)
+StringLiteralParser::StringLiteralParser(ArrayRef<Token> StringToks,
+                                         Preprocessor &PP,
+                                         StringLiteralEvalMethod EvalMethod)
     : SM(PP.getSourceManager()), Features(PP.getLangOpts()),
       Target(PP.getTargetInfo()), Diags(&PP.getDiagnostics()),
       MaxTokenLength(0), SizeBound(0), CharByteWidth(0), Kind(tok::unknown),
-    ResultPtr(ResultBuf.data()), hadError(false), Pascal(false) {
+      ResultPtr(ResultBuf.data()), EvalMethod(EvalMethod), hadError(false),
+      Pascal(false) {
   init(StringToks);
 }
 
 void StringLiteralParser::init(ArrayRef<Token> StringToks){
   // The literal token may have come from an invalid source location (e.g. due
   // to a PCH error), in which case the token length will be 0.
   if (StringToks.empty() || StringToks[0].getLength() < 2)
     return DiagnoseLexingError(SourceLocation());
 
   // Scan all of the string portions, remember the max individual token length,
   // computing a bound on the concatenated string length, and see whether any
   // piece is a wide-string.  If any of the string portions is a wide-string
   // literal, the result is a wide-string literal [C99 6.4.5p4].
   assert(!StringToks.empty() && "expected at least one token");
   MaxTokenLength = StringToks[0].getLength();
   assert(StringToks[0].getLength() >= 2 && "literal token is invalid!");
   SizeBound = StringToks[0].getLength()-2;  // -2 for "".
-  Kind = StringToks[0].getKind();
-
   hadError = false;
 
-  // Implement Translation Phase #6: concatenation of string literals
+  // Determines the kind of string from the prefix
+  Kind = tok::string_literal;
+
+  auto DiagWrongConcatenationAndEvalMethod = [&](const Token &Tok) {
+    if (isUnevaluated() && Tok.getKind() != tok::string_literal) {
+      if (Diags)
+        Diags->Report(Tok.getLocation(), diag::err_unevaluated_string_prefix);
+      hadError = true;
+      return;
+    }
+    if (Tok.is(tok::string_literal))
+      return;
+    if (Tok.is(Kind) || Kind == tok::string_literal) {
+      Kind = Tok.getKind();
+      return;
+    }
+    if (Diags) {
+      Diags->Report(Tok.getLocation(), diag::err_unsupported_string_concat);
+      hadError = true;
+    }
+  };
+
+  DiagWrongConcatenationAndEvalMethod(StringToks[0]);
+
   /// (C99 5.1.1.2p1).  The common case is only one string fragment.
   for (unsigned i = 1; i != StringToks.size(); ++i) {
     if (StringToks[i].getLength() < 2)
       return DiagnoseLexingError(StringToks[i].getLocation());
 
+    DiagWrongConcatenationAndEvalMethod(StringToks[i]);
+
     // The string could be shorter than this if it needs cleaning, but this is a
     // reasonable bound, which is all we need.
     assert(StringToks[i].getLength() >= 2 && "literal token is invalid!");
     SizeBound += StringToks[i].getLength()-2;  // -2 for "".
 
     // Remember maximum string piece length.
     if (StringToks[i].getLength() > MaxTokenLength)
       MaxTokenLength = StringToks[i].getLength();
-
-    // Remember if we see any wide or utf-8/16/32 strings.
-    // Also check for illegal concatenations.
-    if (StringToks[i].isNot(Kind) && StringToks[i].isNot(tok::string_literal)) {
-      if (isAscii()) {
-        Kind = StringToks[i].getKind();
-      } else {
-        if (Diags)
-          Diags->Report(StringToks[i].getLocation(),
-                        diag::err_unsupported_string_concat);
-        hadError = true;
-      }
-    }
   }
 
   // Include space for the null terminator.
   ++SizeBound;
 
   // TODO: K&R warning: "traditional C rejects string constant concatenation"
 
   // Get the width in bytes of char/wchar_t/char16_t/char32_t
   CharByteWidth = getCharWidth(Kind, Target);
   assert((CharByteWidth & 7) == 0 && "Assumes character size is byte multiple");
   CharByteWidth /= 8;
 
   // The output buffer size needs to be large enough to hold wide characters.
if (ThisTokEnd[-1] != '"') {
           expandUCNs(ExpandedUDSuffix, UDSuffix);
           UDSuffix = ExpandedUDSuffix;
         }
 
         // C++11 [lex.ext]p8: At the end of phase 6, if a string literal is the
         // result of a concatenation involving at least one user-defined-string-
         // literal, all the participating user-defined-string-literals shall
         // have the same ud-suffix.
-        if (UDSuffixBuf != UDSuffix) {
+        bool UnevaluatedStringHasUDL = isUnevaluated() && !UDSuffix.empty();
+        if (UDSuffixBuf != UDSuffix || UnevaluatedStringHasUDL) {
           if (Diags) {
             SourceLocation TokLoc = StringToks[i].getLocation();
+            if (UnevaluatedStringHasUDL) {
+              Diags->Report(TokLoc, diag::err_unevaluated_string_udl)
+                  << SourceRange(TokLoc, TokLoc);
+            } else {
               Diags->Report(TokLoc, diag::err_string_concat_mixed_suffix)
                   << UDSuffixBuf << UDSuffix
-              << SourceRange(UDSuffixTokLoc, UDSuffixTokLoc)
-              << SourceRange(TokLoc, TokLoc);
+                  << SourceRange(UDSuffixTokLoc, UDSuffixTokLoc);
+            }
           }
           hadError = true;
         }
       }
     }
 
     // Strip the end quote.
     --ThisTokEnd;
if (ThisTokBuf[0] == 'R') {
       if (ThisTokBuf[0] != '"') {
         // The file may have come from PCH and then changed after loading the
         // PCH; Fail gracefully.
         return DiagnoseLexingError(StringToks[i].getLocation());
       }
       ++ThisTokBuf; // skip "
 
       // Check if this is a pascal string
-      if (Features.PascalStrings && ThisTokBuf + 1 != ThisTokEnd &&
-          ThisTokBuf[0] == '\\' && ThisTokBuf[1] == 'p') {
+      if (!isUnevaluated() && Features.PascalStrings &&
+          ThisTokBuf + 1 != ThisTokEnd && ThisTokBuf[0] == '\\' &&
+          ThisTokBuf[1] == 'p') {
 
         // If the \p sequence is found in the first token, we have a pascal string
         // Otherwise, if we already have a pascal string, ignore the first \p
         if (i == 0) {
           ++ThisTokBuf;
           Pascal = true;
         } else if (Pascal)
           ThisTokBuf += 2;
       }
if (ThisTokBuf[0] == 'R') {
           EncodeUCNEscape(ThisTokBegin, ThisTokBuf, ThisTokEnd,
                           ResultPtr, hadError,
                           FullSourceLoc(StringToks[i].getLocation(), SM),
                           CharByteWidth, Diags, Features);
           continue;
         }
         // Otherwise, this is a non-UCN escape character.  Process it.
         unsigned ResultChar =
             ProcessCharEscape(ThisTokBegin, ThisTokBuf, ThisTokEnd, hadError,
                               FullSourceLoc(StringToks[i].getLocation(), SM),
-                            CharByteWidth*8, Diags, Features);
+                              CharByteWidth * 8, Diags, Features, EvalMethod);
 
         if (CharByteWidth == 4) {
           // FIXME: Make the type of the result buffer correct instead of
           // using reinterpret_cast.
           llvm::UTF32 *ResultWidePtr = reinterpret_cast<llvm::UTF32*>(ResultPtr);
           *ResultWidePtr = ResultChar;
           ResultPtr += 4;
         } else if (CharByteWidth == 2) {
           // FIXME: Make the type of the result buffer correct instead of
           // using reinterpret_cast.
           llvm::UTF16 *ResultWidePtr = reinterpret_cast<llvm::UTF16*>(ResultPtr);
           *ResultWidePtr = ResultChar & 0xFFFF;
           ResultPtr += 2;
         } else {
           assert(CharByteWidth == 1 && "Unexpected char width");
           *ResultPtr++ = ResultChar & 0xFF;
         }
       }
     }
   }
 
+  assert((!Pascal || !isUnevaluated()) &&
+         "Pascal string in unevaluated context");
   if (Pascal) {
     if (CharByteWidth == 4) {
       // FIXME: Make the type of the result buffer correct instead of
       // using reinterpret_cast.
       llvm::UTF32 *ResultWidePtr = reinterpret_cast<llvm::UTF32*>(ResultBuf.data());
       ResultWidePtr[0] = GetNumStringChars() - 1;
     } else if (CharByteWidth == 2) {
       // FIXME: Make the type of the result buffer correct instead of
if (SpellingPtr[1] == 'u' || SpellingPtr[1] == 'U') {
       if (Len > ByteNo) {
         // ByteNo is somewhere within the escape sequence.
         SpellingPtr = EscapePtr;
         break;
       }
       ByteNo -= Len;
     } else {
       ProcessCharEscape(SpellingStart, SpellingPtr, SpellingEnd, HadError,
-                        FullSourceLoc(Tok.getLocation(), SM),
-                        CharByteWidth*8, Diags, Features);
+                        FullSourceLoc(Tok.getLocation(), SM), CharByteWidth * 8,
+                        Diags, Features, StringLiteralEvalMethod::Evaluated);
       --ByteNo;
     }
     assert(!HadError && "This method isn't valid on erroneous strings");
   }
 
   return SpellingPtr-SpellingStart;
 }
 
 /// Determine whether a suffix is a valid ud-suffix. We avoid treating reserved
 /// suffixes as ud-suffixes, because the diagnostic experience is better if we
 /// treat it as an invalid suffix.
 bool StringLiteralParser::isValidUDSuffix(const LangOptions &LangOpts,
                                           StringRef Suffix) {
   return NumericLiteralParser::isValidUDSuffix(LangOpts, Suffix) ||
          Suffix == "sv";
 }

clang/lib/Lex/PPDirectives.cpp

else if (StrTok.isNot(tok::string_literal)) {
     DiscardUntilEndOfDirective();
     return;
   } else if (StrTok.hasUDSuffix()) {
     Diag(StrTok, diag::err_invalid_string_udl);
     DiscardUntilEndOfDirective();
     return;
   } else {
     // Parse and validate the string, converting it into a unique ID.
-    StringLiteralParser Literal(StrTok, *this);
-    assert(Literal.isAscii() && "Didn't allow wide strings in");
+    StringLiteralParser Literal(StrTok, *this,
+                                StringLiteralEvalMethod::Unevaluated);
     if (Literal.hadError) {
       DiscardUntilEndOfDirective();
       return;
     }
-    if (Literal.Pascal) {
-      Diag(StrTok, diag::err_pp_linemarker_invalid_filename);
-      DiscardUntilEndOfDirective();
-      return;
-    }
     FilenameID = SourceMgr.getLineTableFilenameID(Literal.GetString());
 
     // Verify that there is nothing after the string, other than EOD.  Because
     // of C99 6.10.4p5, macros that expand to empty tokens are ok.
     CheckEndOfDirective("line", true);
   }
 
   // Take the file kind of the file containing the #line directive. #line
if (StrTok.is(tok::eod)) {
     DiscardUntilEndOfDirective();
     return;
   } else if (StrTok.hasUDSuffix()) {
     Diag(StrTok, diag::err_invalid_string_udl);
     DiscardUntilEndOfDirective();
     return;
   } else {
     // Parse and validate the string, converting it into a unique ID.
-    StringLiteralParser Literal(StrTok, *this);
-    assert(Literal.isAscii() && "Didn't allow wide strings in");
+    StringLiteralParser Literal(StrTok, *this,
+                                StringLiteralEvalMethod::Unevaluated);
     if (Literal.hadError) {
       DiscardUntilEndOfDirective();
       return;
     }
-    if (Literal.Pascal) {
-      Diag(StrTok, diag::err_pp_linemarker_invalid_filename);
-      DiscardUntilEndOfDirective();
-      return;
-    }
     FilenameID = SourceMgr.getLineTableFilenameID(Literal.GetString());
 
     // If a filename was present, read any flags that are present.
     if (ReadLineMarkerFlags(IsFileEntry, IsFileExit, FileKind, *this))
       return;
   }
 
   // Create a line note with this information.

clang/lib/Lex/PPMacroExpansion.cpp

if (II == Ident__LINE__) {
     }
 
     SourceLocation LParenLoc = Tok.getLocation();
     LexNonComment(Tok);
 
     if (!Tok.isAnnotation() && Tok.getIdentifierInfo())
       Tok.setKind(tok::identifier);
     else if (Tok.is(tok::string_literal) && !Tok.hasUDSuffix()) {
-      StringLiteralParser Literal(Tok, *this);
+      StringLiteralParser Literal(Tok, *this,
+                                  StringLiteralEvalMethod::Unevaluated);
       if (Literal.hadError)
         return;
 
       Tok.setIdentifierInfo(getIdentifierInfo(Literal.GetString()));
       Tok.setKind(tok::identifier);
     } else {
       Diag(Tok.getLocation(), diag::err_pp_identifier_arg_not_identifier)
         << Tok.getKind();
       // Don't walk past anything that's not a real token.

clang/lib/Lex/Pragma.cpp

 // Lex a component of a module name: either an identifier or a string literal;
 // for components that can be expressed both ways, the two forms are equivalent.
 static bool LexModuleNameComponent(
     Preprocessor &PP, Token &Tok,
     std::pair<IdentifierInfo *, SourceLocation> &ModuleNameComponent,
     bool First) {
   PP.LexUnexpandedToken(Tok);
   if (Tok.is(tok::string_literal) && !Tok.hasUDSuffix()) {
     StringLiteralParser Literal(Tok, PP);
     if (Literal.hadError)
       return true;
     ModuleNameComponent = std::make_pair(
         PP.getIdentifierInfo(Literal.GetString()), Tok.getLocation());
   } else if (!Tok.isAnnotation() && Tok.getIdentifierInfo()) {
     ModuleNameComponent =
         std::make_pair(Tok.getIdentifierInfo(), Tok.getLocation());
   } else {
if (II->isStr("assert")) {
             << II->getName();
       }
     } else if (II->isStr("diag_mapping")) {
       Token DiagName;
       PP.LexUnexpandedToken(DiagName);
       if (DiagName.is(tok::eod))
         PP.getDiagnostics().dump();
       else if (DiagName.is(tok::string_literal) && !DiagName.hasUDSuffix()) {
-        StringLiteralParser Literal(DiagName, PP);
+        StringLiteralParser Literal(DiagName, PP,
+                                    StringLiteralEvalMethod::Unevaluated);
         if (Literal.hadError)
           return;
         PP.getDiagnostics().dump(Literal.GetString());
       } else {
         PP.Diag(DiagName, diag::warn_pragma_debug_missing_argument)
             << II->getName();
       }
     } else if (II->isStr("llvm_fatal_error")) {

clang/lib/Parse/ParseDecl.cpp

 static bool attributeHasIdentifierArg(const IdentifierInfo &II) {
 #define CLANG_ATTR_IDENTIFIER_ARG_LIST
   return llvm::StringSwitch<bool>(normalizeAttrName(II.getName()))
 #include "clang/Parse/AttrParserStringSwitches.inc"
            .Default(false);
 #undef CLANG_ATTR_IDENTIFIER_ARG_LIST
 }
 
 /// Determine whether the given attribute has a variadic identifier argument.
 static bool attributeHasVariadicIdentifierArg(const IdentifierInfo &II) {
 #define CLANG_ATTR_VARIADIC_IDENTIFIER_ARG_LIST
   return llvm::StringSwitch<bool>(normalizeAttrName(II.getName()))
 #include "clang/Parse/AttrParserStringSwitches.inc"
            .Default(false);
 #undef CLANG_ATTR_VARIADIC_IDENTIFIER_ARG_LIST
 }
 
unsigned Parser::ParseAttributeArgsCommon(
 
   bool ChangeKWThisToIdent = attributeTreatsKeywordThisAsIdentifier(*AttrName);
   bool AttributeIsTypeArgAttr = attributeIsTypeArgAttr(*AttrName);
 
   // Interpret "kw_this" as an identifier if the attributed requests it.
   if (ChangeKWThisToIdent && Tok.is(tok::kw_this))
     Tok.setKind(tok::identifier);
 
+  ParsedAttr::Kind AttrKind =
+      ParsedAttr::getParsedKind(AttrName, ScopeName, Syntax);
+
   ArgsVector ArgExprs;
   if (Tok.is(tok::identifier)) {
     // If this attribute wants an 'identifier' argument, make it so.
     bool IsIdentifierArg = attributeHasIdentifierArg(*AttrName) ||
                            attributeHasVariadicIdentifierArg(*AttrName);
-    ParsedAttr::Kind AttrKind =
-        ParsedAttr::getParsedKind(AttrName, ScopeName, Syntax);
 
     // If we don't know how to parse this attribute, but this is the only
     // token in this argument, assume it's meant to be an identifier.
     if (AttrKind == ParsedAttr::UnknownAttribute ||
         AttrKind == ParsedAttr::IgnoredAttribute) {
       const Token &Next = NextToken();
       IsIdentifierArg = Next.isOneOf(tok::r_paren, tok::comma);
     }
 
     if (IsIdentifierArg)
       ArgExprs.push_back(ParseIdentifierLoc());
   }
 
   ParsedType TheParsedType;
   if (!ArgExprs.empty() ? Tok.is(tok::comma) : Tok.isNot(tok::r_paren)) {
     // Eat the comma.
     if (!ArgExprs.empty())
       ConsumeToken();
 
     // Parse the non-empty comma-separated list of expressions.
     do {
       // Interpret "kw_this" as an identifier if the attributed requests it.
       if (ChangeKWThisToIdent && Tok.is(tok::kw_this))
         Tok.setKind(tok::identifier);
 
       ExprResult ArgExpr;
       if (AttributeIsTypeArgAttr) {
         TypeResult T = ParseTypeName();
         if (T.isInvalid()) {
           SkipUntil(tok::r_paren, StopAtSemi);
           return 0;
         }
         if (T.isUsable())
           TheParsedType = T.get();
         break; // FIXME: Multiple type arguments are not implemented.
       } else if (Tok.is(tok::identifier) &&
                  attributeHasVariadicIdentifierArg(*AttrName)) {
         ArgExprs.push_back(ParseIdentifierLoc());
       } else {
         bool Uneval = attributeParsedArgsUnevaluated(*AttrName);
         EnterExpressionEvaluationContext Unevaluated(
             Actions,
             Uneval ? Sema::ExpressionEvaluationContext::Unevaluated
                    : Sema::ExpressionEvaluationContext::ConstantEvaluated);
 
-        ExprResult ArgExpr(
-            Actions.CorrectDelayedTyposInExpr(ParseAssignmentExpression()));
+        ExprResult ArgExpr(Actions.CorrectDelayedTyposInExpr(
+            ParseAttributeArgAsUnevaluatedLiteralOrExpression(AttrKind)));
         if (ArgExpr.isInvalid()) {
           SkipUntil(tok::r_paren, StopAtSemi);
           return 0;
         }
         ArgExprs.push_back(ArgExpr.get());
       }
       // Eat the comma, move to the next argument
     } while (TryConsumeToken(tok::comma));
   }
 
unsigned Parser::ParseAttributeArgsCommon(
   }
 
   if (EndLoc)
     *EndLoc = RParen;
 
   return static_cast<unsigned>(ArgExprs.size() + !TheParsedType.get().isNull());
 }
 
+ExprResult Parser::ParseAttributeArgAsUnevaluatedLiteralOrExpression(
+    ParsedAttr::Kind Kind) {
+  if (isTokenStringLiteral() && (Kind == ParsedAttr::AT_Deprecated ||
+                                 Kind == ParsedAttr::AT_WarnUnusedResult)) {
+    ExprResult Result = ParseUnevaluatedStringLiteralExpression();
+    if (!Result.isInvalid())
+      return Result;
+  }
+  return ParseAssignmentExpression();
+}
+
 /// Parse the arguments to a parameterized GNU attribute or
 /// a C++11 attribute in "gnu" namespace.
 void Parser::ParseGNUAttributeArgs(IdentifierInfo *AttrName,
                                    SourceLocation AttrNameLoc,
                                    ParsedAttributes &Attrs,
                                    SourceLocation *EndLoc,
                                    IdentifierInfo *ScopeName,
                                    SourceLocation ScopeLoc,
do {
     }
 
     if (Tok.isNot(tok::equal)) {
       Diag(Tok, diag::err_expected_after) << Keyword << tok::equal;
       SkipUntil(tok::r_paren, StopAtSemi);
       return;
     }
     ConsumeToken();
-    if (Keyword == Ident_message || Keyword == Ident_replacement) {
-      if (Tok.isNot(tok::string_literal)) {
-        Diag(Tok, diag::err_expected_string_literal)
-          << /*Source='availability attribute'*/2;
-        SkipUntil(tok::r_paren, StopAtSemi);
-        return;
-      }
-      if (Keyword == Ident_message)
-        MessageExpr = ParseStringLiteralExpression();
-      else
-        ReplacementExpr = ParseStringLiteralExpression();
-      // Also reject wide string literals.
-      if (StringLiteral *MessageStringLiteral =
-              cast_or_null<StringLiteral>(MessageExpr.get())) {
-        if (!MessageStringLiteral->isAscii()) {
-          Diag(MessageStringLiteral->getSourceRange().getBegin(),
-               diag::err_expected_string_literal)
-            << /*Source='availability attribute'*/ 2;
-          SkipUntil(tok::r_paren, StopAtSemi);
-          return;
-        }
-      }
-      if (Keyword == Ident_message)
-        break;
-      else
-        continue;
+    if ((Keyword == Ident_message || Keyword == Ident_replacement) &&
+        !tok::isStringLiteral(Tok.getKind())) {
+          << /*Source='availability attribute'*/ 2;
+    if (Keyword == Ident_message) {
+      MessageExpr = ParseUnevaluatedStringLiteralExpression();
+      if (MessageExpr.isInvalid()) {
+    }
+    if (Keyword == Ident_replacement) {
+      ReplacementExpr = ParseUnevaluatedStringLiteralExpression();
+      if (ReplacementExpr.isInvalid()) {
+        SkipUntil(tok::r_paren, StopAtSemi);
+        return;
+      }
     }
 
     // Special handling of 'NA' only when applied to introduced or
     // deprecated.
     if ((Keyword == Ident_introduced || Keyword == Ident_deprecated) &&
         Tok.is(tok::identifier)) {
       IdentifierInfo *NA = Tok.getIdentifierInfo();
       if (NA->getName() == "NA") {
if (Tok.isNot(tok::string_literal)) {
           << /*language | source container*/ (Keyword != Ident_language);
       SkipUntil(tok::comma, tok::r_paren, StopAtSemi | StopBeforeMatch);
       continue;
     }
     if (Keyword == Ident_language) {
       if (HadLanguage) {
         Diag(KeywordLoc, diag::err_external_source_symbol_duplicate_clause)
             << Keyword;
-        ParseStringLiteralExpression();
+        ParseUnevaluatedStringLiteralExpression();
         continue;
       }
-      Language = ParseStringLiteralExpression();
+      Language = ParseUnevaluatedStringLiteralExpression();
     } else {
       assert(Keyword == Ident_defined_in && "Invalid clause keyword!");
       if (HadDefinedIn) {
         Diag(KeywordLoc, diag::err_external_source_symbol_duplicate_clause)
             << Keyword;
-        ParseStringLiteralExpression();
+        ParseUnevaluatedStringLiteralExpression();
         continue;
       }
-      DefinedInExpr = ParseStringLiteralExpression();
+      DefinedInExpr = ParseUnevaluatedStringLiteralExpression();
     }
   } while (TryConsumeToken(tok::comma));
 
   // Closing ')'.
   if (T.consumeClose())
     return;
   if (EndLoc)
     *EndLoc = T.getCloseLocation();

clang/lib/Parse/ParseDeclCXX.cpp

 /// and just before that, that extern was seen.
 ///
 ///       linkage-specification: [C++ 7.5p2: dcl.link]
 ///         'extern' string-literal '{' declaration-seq[opt] '}'
 ///         'extern' string-literal declaration
 ///
 Decl *Parser::ParseLinkage(ParsingDeclSpec &DS, DeclaratorContext Context) {
   assert(isTokenStringLiteral() && "Not a string literal!");
-  ExprResult Lang = ParseStringLiteralExpression(false);
+  ExprResult Lang = ParseUnevaluatedStringLiteralExpression();
 
   ParseScope LinkageScope(this, Scope::DeclScope);
   Decl *LinkageSpec =
       Lang.isInvalid()
           ? nullptr
           : Actions.ActOnStartLinkageSpecification(
                 getCurScope(), DS.getSourceRange().getBegin(), Lang.get(),
                 Tok.is(tok::l_brace) ? Tok.getLocation() : SourceLocation());
if (Tok.is(tok::r_paren)) {
 
     if (!isTokenStringLiteral()) {
       Diag(Tok, diag::err_expected_string_literal)
         << /*Source='static_assert'*/1;
       SkipMalformedDecl();
       return nullptr;
     }
 
-    AssertMessage = ParseStringLiteralExpression();
+    AssertMessage = ParseUnevaluatedStringLiteralExpression();
     if (AssertMessage.isInvalid()) {
       SkipMalformedDecl();
       return nullptr;
     }
   }
 
   T.consumeClose();
 
if (Tok.is(tok::string_literal)) {
     // ok that the Token points to StrBuffer.
     Token Toks[1];
     Toks[0].startToken();
     Toks[0].setKind(tok::string_literal);
     Toks[0].setLocation(StartLoc);
     Toks[0].setLiteralData(StrBuffer.data());
     Toks[0].setLength(StrBuffer.size());
     StringLiteral *UuidString =
-        cast<StringLiteral>(Actions.ActOnStringLiteral(Toks, nullptr).get());
+        cast<StringLiteral>(Actions.ActOnStringLiteral(Toks).get());
     ArgExprs.push_back(UuidString);
   }
 
   if (!T.consumeClose()) {
     Attrs.addNew(UuidIdent, SourceRange(UuidLoc, T.getCloseLocation()), nullptr,
                  SourceLocation(), ArgExprs.data(), ArgExprs.size(),
                  ParsedAttr::AS_Microsoft);
   }

clang/lib/Parse/ParseExpr.cpp

 /// form string literals, and also handles string concatenation [C99 5.1.1.2,
 /// translation phase #6].
 ///
 /// \verbatim
 ///       primary-expression: [C99 6.5.1]
 ///         string-literal
 /// \verbatim
 ExprResult Parser::ParseStringLiteralExpression(bool AllowUserDefinedLiteral) {
+  return ParseStringLiteralExpression(AllowUserDefinedLiteral, false);
+}
+
+ExprResult Parser::ParseUnevaluatedStringLiteralExpression() {
+  if (!isTokenStringLiteral()) {
+    Diag(Tok.getLocation(), diag::err_expected_string_literal);
+    return ExprError();
+  }
+
+  return ParseStringLiteralExpression(false, true);
+}
+
+ExprResult Parser::ParseStringLiteralExpression(bool AllowUserDefinedLiteral,
+                                                bool Unevaluated) {
   assert(isTokenStringLiteral() && "Not a string literal!");
 
   // String concat.  Note that keywords like __func__ and __FUNCTION__ are not
   // considered to be strings for concatenation purposes.
   SmallVector<Token, 4> StringToks;
 
   do {
     StringToks.push_back(Tok);
     ConsumeStringToken();
   } while (isTokenStringLiteral());
 
+  if (Unevaluated) {
+    assert(!AllowUserDefinedLiteral && "UDL are always evaluated");
+    return Actions.ActOnUnevaluatedStringLiteral(StringToks);
+  }
+
   // Pass the set of string tokens, ready for concatenation, to the actions.
   return Actions.ActOnStringLiteral(StringToks,
                                     AllowUserDefinedLiteral ? getCurScope()
                                                             : nullptr);
 }
 
 /// ParseGenericSelectionExpression - Parse a C11 generic-selection
 /// [C11 6.5.1.1].
bool Parser::ParseExpressionList(SmallVectorImpl<Expr *> &Exprs,
   while (1) {
     if (ExpressionStarts)
       ExpressionStarts();
 
     ExprResult Expr;
     if (getLangOpts().CPlusPlus11 && Tok.is(tok::l_brace)) {
       Diag(Tok, diag::warn_cxx98_compat_generalized_initializer_lists);
       Expr = ParseBraceInitializer();
     } else
       Expr = ParseAssignmentExpression();
 
     if (Tok.is(tok::ellipsis))
       Expr = Actions.ActOnPackExpansion(Expr.get(), ConsumeToken());
     else if (Tok.is(tok::code_completion)) {
       // There's nothing to suggest in here as we parsed a full expression.
       // Instead fail and propogate the error since caller might have something
       // the suggest, e.g. signature help in function call. Note that this is
       // performed before pushing the \p Expr, so that signature help can report

clang/lib/Parse/Parser.cpp

 ///
 ExprResult Parser::ParseAsmStringLiteral(bool ForAsmLabel) {
   if (!isTokenStringLiteral()) {
     Diag(Tok, diag::err_expected_string_literal)
       << /*Source='in...'*/0 << "'asm'";
     return ExprError();
   }
 
-  ExprResult AsmString(ParseStringLiteralExpression());
+  ExprResult AsmString(ParseUnevaluatedStringLiteralExpression());
   if (!AsmString.isInvalid()) {
     const auto *SL = cast<StringLiteral>(AsmString.get());
-    if (!SL->isAscii()) {
-      Diag(Tok, diag::err_asm_operand_wide_string_literal)
-        << SL->isWide()
-        << SL->getSourceRange();
-      return ExprError();
-    }
     if (ForAsmLabel && SL->getString().empty()) {
       Diag(Tok, diag::err_asm_operand_wide_string_literal)
           << 2 /* an empty */ << SL->getSourceRange();
       return ExprError();
     }
   }
   return AsmString;
 }

clang/lib/Sema/SemaDeclAttr.cpp

bool Sema::checkStringLiteralArgumentAttr(const ParsedAttr &AL, unsigned ArgNum,
   }
 
   // Now check for an actual string literal.
   Expr *ArgExpr = AL.getArgAsExpr(ArgNum);
   const auto *Literal = dyn_cast<StringLiteral>(ArgExpr->IgnoreParenCasts());
   if (ArgLocation)
     *ArgLocation = ArgExpr->getBeginLoc();
 
-  if (!Literal || !Literal->isAscii()) {
+  if (!Literal || (!Literal->isUnevaluated() && !Literal->isAscii())) {
     Diag(ArgExpr->getBeginLoc(), diag::err_attribute_argument_type)
         << AL << AANT_ArgumentString;
     return false;
   }
 
   Str = Literal->getString();
   return true;
 }
D->addAttr(::new (S.Context)
                  AllocSizeAttr(S.Context, AL, SizeArgNo, NumberArgNo));
 }
 
 static bool checkTryLockFunAttrCommon(Sema &S, Decl *D, const ParsedAttr &AL,
                                       SmallVectorImpl<Expr *> &Args) {
   if (!AL.checkAtLeastNumArgs(S, 1))
     return false;
 
   if (!isIntOrBool(AL.getArgAsExpr(0))) {
     S.Diag(AL.getLoc(), diag::err_attribute_argument_n_type)
         << AL << 1 << AANT_ArgumentIntOrBool;
     return false;
   }
 
   // check that all arguments are lockable objects
   checkAttrArgsAreCapabilityObjs(S, D, AL, Args, 1);
 
   return true;

clang/lib/Sema/SemaDeclCXX.cpp

 /// the '{'. ExternLoc is the location of the 'extern', Lang is the
 /// language string literal. LBraceLoc, if valid, provides the location of
 /// the '{' brace. Otherwise, this linkage specification does not
 /// have any braces.
 Decl *Sema::ActOnStartLinkageSpecification(Scope *S, SourceLocation ExternLoc,
                                            Expr *LangStr,
                                            SourceLocation LBraceLoc) {
   StringLiteral *Lit = cast<StringLiteral>(LangStr);
-  if (!Lit->isAscii()) {
-    Diag(LangStr->getExprLoc(), diag::err_language_linkage_spec_not_ascii)
-      << LangStr->getSourceRange();
-    return nullptr;
-  }
+  assert(Lit->isUnevaluated() && "Unexpected string literal kind");
 
   StringRef Lang = Lit->getString();
   LinkageSpecDecl::LanguageIDs Language;
   if (Lang == "C")
     Language = LinkageSpecDecl::lang_c;
   else if (Lang == "C++")
     Language = LinkageSpecDecl::lang_cxx;
   else {

clang/lib/Sema/SemaExpr.cpp

if (S.LookupLiteralOperator(Scope, R, llvm::makeArrayRef(ArgTy, Args.size()),
                               /*AllowRaw*/ false, /*AllowTemplate*/ false,
                               /*AllowStringTemplatePack*/ false,
                               /*DiagnoseMissing*/ true) == Sema::LOLR_Error)
     return ExprError();
 
   return S.BuildLiteralOperatorCall(R, OpNameInfo, Args, LitEndLoc);
 }
 
+ExprResult Sema::ActOnUnevaluatedStringLiteral(ArrayRef<Token> StringToks) {
+  StringLiteralParser Literal(StringToks, PP,
+                              StringLiteralEvalMethod::Unevaluated);
+  if (Literal.hadError)
+    return ExprError();
+
+  SmallVector<SourceLocation, 4> StringTokLocs;
+  for (const Token &Tok : StringToks)
+    StringTokLocs.push_back(Tok.getLocation());
+
+  StringLiteral *Lit = StringLiteral::Create(
+      Context, Literal.GetString(), StringLiteral::Unevaluated, false, {},
+      &StringTokLocs[0], StringTokLocs.size());
+
+  if (!Literal.getUDSuffix().empty()) {
+    SourceLocation UDSuffixLoc =
+        getUDSuffixLoc(*this, StringTokLocs[Literal.getUDSuffixToken()],
+                       Literal.getUDSuffixOffset());
+    return ExprError(Diag(UDSuffixLoc, diag::err_invalid_string_udl));
+  }
+
+  return Lit;
+}
+
 /// ActOnStringLiteral - The specified tokens were lexed as pasted string
 /// fragments (e.g. "foo" "bar" L"baz").  The result string has to handle string
 /// concatenation ([C99 5.1.1.2, translation phase #6]), so it may come from
 /// multiple tokens.  However, the common case is that StringToks points to one
 /// string.
 ///
 ExprResult
 Sema::ActOnStringLiteral(ArrayRef<Token> StringToks, Scope *UDLScope) {

clang/lib/Sema/SemaExprCXX.cpp

if (const PointerType *ToPtrType = ToType->getAs<PointerType>())
               // We don't allow UTF literals to be implicitly converted
               break;
             case StringLiteral::Ascii:
               return (ToPointeeType->getKind() == BuiltinType::Char_U ||
                       ToPointeeType->getKind() == BuiltinType::Char_S);
             case StringLiteral::Wide:
               return Context.typesAreCompatible(Context.getWideCharType(),
                                                 QualType(ToPointeeType, 0));
+            case StringLiteral::Unevaluated:
+              assert(false && "Unevaluated string literal in expression");
+              break;
           }
         }
       }
 
   return false;
 }
 
 static ExprResult BuildCXXCastArgument(Sema &S,

clang/lib/Sema/SemaInit.cpp

static StringInitFailureKind IsStringInit(Expr *Init, const ArrayType *AT,
   case StringLiteral::Wide:
     if (Context.typesAreCompatible(Context.getWideCharType(), ElemTy))
       return SIF_None;
     if (ElemTy->isCharType() || ElemTy->isChar8Type())
       return SIF_WideStringIntoChar;
     if (IsWideCharCompatible(ElemTy, Context))
       return SIF_IncompatWideStringIntoWideChar;
     return SIF_Other;
+  case StringLiteral::Unevaluated:
+    assert(false && "Unevaluated string literal in initialization");
+    break;
   }
 
   llvm_unreachable("missed a StringLiteral kind?");
 }
 
 static StringInitFailureKind IsStringInit(Expr *init, QualType declType,
                                           ASTContext &Context) {
   const ArrayType *arrayType = Context.getAsArrayType(declType);

clang/lib/Sema/SemaStmtAsm.cpp

StmtResult Sema::ActOnGCCAsmStmt(SourceLocation AsmLoc, bool IsSimple,
   StringLiteral **Constraints =
     reinterpret_cast<StringLiteral**>(constraints.data());
   StringLiteral *AsmString = cast<StringLiteral>(asmString);
   StringLiteral **Clobbers = reinterpret_cast<StringLiteral**>(clobbers.data());
 
   SmallVector<TargetInfo::ConstraintInfo, 4> OutputConstraintInfos;
 
   // The parser verifies that there is a string literal here.
-  assert(AsmString->isAscii());
+  assert(AsmString->isUnevaluated());
 
   FunctionDecl *FD = dyn_cast<FunctionDecl>(getCurLexicalContext());
   llvm::StringMap<bool> FeatureMap;
   Context.getFunctionFeatureMap(FeatureMap, FD);
 
   for (unsigned i = 0; i != NumOutputs; i++) {
     StringLiteral *Literal = Constraints[i];
-    assert(Literal->isAscii());
+    assert(Literal->isUnevaluated());
 
     StringRef OutputName;
     if (Names[i])
       OutputName = Names[i]->getName();
 
     TargetInfo::ConstraintInfo Info(Literal->getString(), OutputName);
     if (!Context.getTargetInfo().validateOutputConstraint(Info)) {
       targetDiag(Literal->getBeginLoc(),
if (!Context.getTargetInfo().validateOutputSize(
                      NumClobbers, Clobbers, NumLabels, RParenLoc);
     }
   }
 
   SmallVector<TargetInfo::ConstraintInfo, 4> InputConstraintInfos;
 
   for (unsigned i = NumOutputs, e = NumOutputs + NumInputs; i != e; i++) {
     StringLiteral *Literal = Constraints[i];
-    assert(Literal->isAscii());
+    assert(Literal->isUnevaluated());
 
     StringRef InputName;
     if (Names[i])
       InputName = Names[i]->getName();
 
     TargetInfo::ConstraintInfo Info(Literal->getString(), InputName);
     if (!Context.getTargetInfo().validateInputConstraint(OutputConstraintInfos,
                                                          Info)) {
if (!Context.getTargetInfo().validateInputSize(FeatureMap,
              << Info.getConstraintStr();
   }
 
   Optional<SourceLocation> UnwindClobberLoc;
 
   // Check that the clobbers are valid.
   for (unsigned i = 0; i != NumClobbers; i++) {
     StringLiteral *Literal = Clobbers[i];
-    assert(Literal->isAscii());
+    assert(Literal->isUnevaluated());
 
     StringRef Clobber = Literal->getString();
 
     if (!Context.getTargetInfo().isValidClobber(Clobber)) {
       targetDiag(Literal->getBeginLoc(), diag::err_asm_unknown_register_name)
           << Clobber;
       return new (Context)
           GCCAsmStmt(Context, AsmLoc, IsSimple, IsVolatile, NumOutputs,

clang/test/CXX/dcl.dcl/dcl.link/p2.cpp

 // RUN: %clang_cc1 -std=c++11 -verify %s
 
 extern "C" {
   extern R"(C++)" { }
 }
 
 #define plusplus "++"
 extern "C" plusplus {
 }
 
-extern u8"C" {} // expected-error {{string literal in language linkage specifier cannot have an encoding-prefix}}
-extern L"C" {} // expected-error {{string literal in language linkage specifier cannot have an encoding-prefix}}
-extern u"C++" {} // expected-error {{string literal in language linkage specifier cannot have an encoding-prefix}}
-extern U"C" {} // expected-error {{string literal in language linkage specifier cannot have an encoding-prefix}}
+extern u8"C" {}  // expected-error {{an unevaluated string literal cannot have an encoding prefix}}
+extern L"C" {}   // expected-error {{an unevaluated string literal cannot have an encoding prefix}}
+extern u"C++" {} // expected-error {{an unevaluated string literal cannot have an encoding prefix}}
+extern U"C" {}   // expected-error {{an unevaluated string literal cannot have an encoding prefix}}

clang/test/CXX/dcl.dcl/p4-0x.cpp

struct U {
   constexpr operator long() const { return 0; } // expected-note {{candidate}}
 };
 
 static_assert(S(true), "");
 static_assert(S(false), "not so fast"); // expected-error {{not so fast}}
 static_assert(T(), "");
 static_assert(U(), ""); // expected-error {{ambiguous}}
 
-static_assert(false, L"\x14hi" "!" R"x(")x"); // expected-error {{static_assert failed L"\024hi!\""}}
+static_assert(false, L"\x14hi" // expected-error {{an unevaluated string literal cannot have an encoding prefix}}
+                     "!"
+                     R"x(")x");
 No newline at end of file

clang/test/FixIt/fixit-static-assert.cpp

 // RUN: %clang_cc1 -std=c++14 %s -fdiagnostics-parseable-fixits %s 2>&1 | FileCheck %s
 // Ensure no warnings are emitted in c++17.
 // RUN: %clang_cc1 -std=c++17 %s -verify=cxx17
 // RUN: %clang_cc1 -std=c++14 %s -fixit-recompile -fixit-to-temporary -Werror
 
 // cxx17-no-diagnostics
 
 static_assert(true && "String");
 // CHECK-DAG: {[[@LINE-1]]:20-[[@LINE-1]]:22}:","
 
 // String literal prefixes are good.
 static_assert(true && R"(RawString)");
 // CHECK-DAG: {[[@LINE-1]]:20-[[@LINE-1]]:22}:","
-static_assert(true && L"RawString");
-// CHECK-DAG: {[[@LINE-1]]:20-[[@LINE-1]]:22}:","
 
 static_assert(true);
 // CHECK-DAG: {[[@LINE-1]]:19-[[@LINE-1]]:19}:", \"\""
 
 // While its technically possible to transform this to
 // static_assert(true, "String") we don't attempt this fix.
 static_assert("String" && true);
 // CHECK-DAG: {[[@LINE-1]]:31-[[@LINE-1]]:31}:", \"\""
 
 // Don't be smart and look in parentheses.
 static_assert((true && "String"));
 // CHECK-DAG: {[[@LINE-1]]:33-[[@LINE-1]]:33}:", \"\""

clang/test/Parser/asm.c

 // RUN: %clang_cc1 -fsyntax-only -verify %s
 
 #if !__has_extension(gnu_asm)
 #error Extension 'gnu_asm' should be available by default
 #endif
 
 void f1() {
   // PR7673: Some versions of GCC support an empty clobbers section.
   asm ("ret" : : :);
 }
 
 void f2() {
   asm("foo" : "=r" (a)); // expected-error {{use of undeclared identifier 'a'}}
-  asm("foo" : : "r" (b)); // expected-error {{use of undeclared identifier 'b'}} 
+  asm("foo"
+      :
+      : "r"(b)); // expected-error {{use of undeclared identifier 'b'}}
 }
 
 void a() __asm__(""); // expected-error {{cannot use an empty string literal in 'asm'}}
 void a() {
   __asm__(""); // ok
 }
 
 // rdar://5952468
 __asm ; // expected-error {{expected '(' after 'asm'}}
 
 // <rdar://problem/10465079> - Don't crash on wide string literals in 'asm'.
-int foo asm (L"bar"); // expected-error {{cannot use wide string literal in 'asm'}}
+int foo asm(L"bar"); // expected-error {{an unevaluated string literal cannot have an encoding prefix}}
 
 asm() // expected-error {{expected string literal in 'asm'}}
 // expected-error@-1 {{expected ';' after top-level asm block}}
 
 asm(; // expected-error {{expected string literal in 'asm'}}
 
 asm("") // expected-error {{expected ';' after top-level asm block}}
 
 // Unterminated asm strings at the end of the file were causing us to crash, so
 // this needs to be last. rdar://15624081
 // expected-warning@+3 {{missing terminating '"' character}}
 // expected-error@+2 {{expected string literal in 'asm'}}
 // expected-error@+1 {{expected ';' after top-level asm block}}
 asm("

clang/test/Parser/asm.cpp

 // RUN: %clang_cc1 -fsyntax-only -verify -std=c++11 %s
 
 int foo1 asm ("bar1");
-int foo2 asm (L"bar2"); // expected-error {{cannot use wide string literal in 'asm'}}
-int foo3 asm (u8"bar3"); // expected-error {{cannot use unicode string literal in 'asm'}}
-int foo4 asm (u"bar4"); // expected-error {{cannot use unicode string literal in 'asm'}}
-int foo5 asm (U"bar5"); // expected-error {{cannot use unicode string literal in 'asm'}}
+int foo2 asm(L"bar2");   // expected-error {{an unevaluated string literal cannot have an encoding prefix}}
+int foo3 asm(u8"bar3");  // expected-error {{an unevaluated string literal cannot have an encoding prefix}}
+int foo4 asm(u"bar4");   // expected-error {{an unevaluated string literal cannot have an encoding prefix}}
+int foo5 asm(U"bar5");   // expected-error {{an unevaluated string literal cannot have an encoding prefix}}
 int foo6 asm ("bar6"_x); // expected-error {{string literal with user-defined suffix cannot be used here}}
-int foo6 asm ("" L"bar7"); // expected-error {{cannot use wide string literal in 'asm'}}
+int foo6 asm(""
+             L"bar7"); // expected-error {{an unevaluated string literal cannot have an encoding prefix}}

clang/test/Parser/attr-availability-xcore.c

 // Test availability message string type when wide characters are 1 byte.
 // REQUIRES: xcore-registered-target
 // RUN: %clang_cc1 -triple xcore -fsyntax-only -verify %s
 
 #if !__has_feature(attribute_availability)
 #  error 'availability' attribute is not available
 #endif
 
-void f7() __attribute__((availability(macosx,message=L"wide"))); // expected-error {{expected string literal for optional message in 'availability' attribute}}
+void f7() __attribute__((availability(macosx, message = L"wide"))); // expected-error {{an unevaluated string literal cannot have an encoding prefix}}
 
-void f8() __attribute__((availability(macosx,message="a" L"b"))); // expected-error {{expected string literal for optional message in 'availability' attribute}}
+void f8() __attribute__((availability(macosx, message = "a"
+                                                        L"b"))); // expected-error {{an unevaluated string literal cannot have an encoding prefix}}

clang/test/Parser/attr-availability.c

 void f3() __attribute__((availability(ios,deprecated=10.4.7,introduced=10,obsoleted=10.6,introduced=10.2))); // expected-error{{redundant 'introduced' availability change; only the last specified change will be used}}
 
 void f4() __attribute__((availability(macosx,introduced=10.5), availability(ios,unavailable)));
 
 void f5() __attribute__((availability(macosx,introduced=10.5), availability(ios,unavailable, unavailable))); // expected-error{{redundant 'unavailable' availability change; only the last specified change will be used}}
 
 void f6() __attribute__((availability(macosx,unavailable,introduced=10.5))); // expected-warning{{'unavailable' availability overrides all other availability information}}
 
-void f7() __attribute__((availability(macosx,message=L"wide"))); // expected-error {{expected string literal for optional message in 'availability' attribute}}
+void f7() __attribute__((availability(macosx, message = L"wide"))); // expected-error {{an unevaluated string literal cannot have an encoding prefix}}
 
-void f8() __attribute__((availability(macosx,message="a" L"b"))); // expected-error {{expected string literal for optional message in 'availability' attribute}}
+void f8() __attribute__((availability(macosx, message = "a"
+                                                        L"b"))); // expected-error {{an unevaluated string literal cannot have an encoding prefix}}
 
-void f9() __attribute__((availability(macosx,message=u8"b"))); // expected-error {{expected string literal for optional message in 'availability' attribute}}
+void f9() __attribute__((availability(macosx, message = u8"b"))); // expected-error {{an unevaluated string literal cannot have an encoding prefix}}
 
-void f10() __attribute__((availability(macosx,message="a" u8"b"))); // expected-error {{expected string literal for optional message in 'availability' attribute}}
+void f10() __attribute__((availability(macosx, message = "a"
+                                                         u8"b"))); // expected-error {{an unevaluated string literal cannot have an encoding prefix}}
 
-void f11() __attribute__((availability(macosx,message=u"b"))); // expected-error {{expected string literal for optional message in 'availability' attribute}}
+void f11() __attribute__((availability(macosx, message = u"b"))); // expected-error {{an unevaluated string literal cannot have an encoding prefix}}
 
-void f12() __attribute__((availability(macosx,message="a" u"b"))); // expected-error {{expected string literal for optional message in 'availability' attribute}}
+void f12() __attribute__((availability(macosx, message = "a"
+                                                         u"b"))); // expected-error {{an unevaluated string literal cannot have an encoding prefix}}
 
 // rdar://10095131
-enum E{
-    gorf __attribute__((availability(macosx,introduced=8.5, message = 10.0))), // expected-error {{expected string literal for optional message in 'availability' attribute}}
-    garf __attribute__((availability(macosx,introduced=8.5, message))), // expected-error {{expected '=' after 'message'}}
+enum E {
+  gorf __attribute__((availability(macosx, introduced = 8.5, message = 10.0))), // expected-error {{expected string literal}}
+  garf __attribute__((availability(macosx, introduced = 8.5, message))),        // expected-error {{expected '=' after 'message'}}
 
-    foo __attribute__((availability(macosx,introduced=8.5,deprecated=9.0, message="Use CTFontCopyPostScriptName()", deprecated=10.0))) // expected-error {{expected ')'}} \
+  foo __attribute__((availability(macosx, introduced = 8.5, deprecated = 9.0, message = "Use CTFontCopyPostScriptName()", deprecated = 10.0))) // expected-error {{expected ')'}} \
                                                                                                                                        // expected-note {{to match this '('}}
 };

clang/test/Sema/asm.c

void clobbers() {
   register void *no_clobber_conflict asm ("%rax");
   int a,b,c;
   asm ("nop" : "=r" (no_clobber_conflict) : "r" (clobber_conflict) : "%rcx"); // expected-error {{asm-specifier for input or output variable conflicts with asm clobber list}}
   asm ("nop" : "=r" (clobber_conflict) : "r" (no_clobber_conflict) : "%rcx"); // expected-error {{asm-specifier for input or output variable conflicts with asm clobber list}}
   asm ("nop" : "=r" (clobber_conflict) : "r" (clobber_conflict) : "%rcx"); // expected-error {{asm-specifier for input or output variable conflicts with asm clobber list}}
   asm ("nop" : "=c" (a) : "r" (no_clobber_conflict) : "%rcx"); // expected-error {{asm-specifier for input or output variable conflicts with asm clobber list}}
   asm ("nop" : "=r" (no_clobber_conflict) : "c" (c) : "%rcx"); // expected-error {{asm-specifier for input or output variable conflicts with asm clobber list}}
   asm ("nop" : "=r" (clobber_conflict) : "c" (c) : "%rcx"); // expected-error {{asm-specifier for input or output variable conflicts with asm clobber list}}
-  asm ("nop" : "=a" (a) : "b" (b) : "%rcx", "%rbx"); // expected-error {{asm-specifier for input or output variable conflicts with asm clobber list}} 
+  asm("nop"
+      : "=a"(a)
+      : "b"(b)
+      : "%rcx", "%rbx"); // expected-error {{asm-specifier for input or output variable conflicts with asm clobber list}}
 }
 
 // rdar://6094010
 void test3() {
   int x;
-  asm(L"foo" : "=r"(x)); // expected-error {{wide string}}
-  asm("foo" : L"=r"(x)); // expected-error {{wide string}}
+  asm(L"foo" // expected-error {{an unevaluated string literal cannot have an encoding prefix}}
+      : "=r"(x));
+  asm("foo"
+      : L"=r"(x)); // expected-error {{an unevaluated string literal cannot have an encoding prefix}}
 }
 
 // <rdar://problem/6156893>
 void test4(const volatile void *addr)
 {
     asm ("nop" : : "r"(*addr)); // expected-error {{invalid type 'const volatile void' in asm input for constraint 'r'}}
     asm ("nop" : : "m"(*addr));
 

clang/test/SemaCXX/static-assert.cpp

 
 template<typename T> struct S {
     static_assert(sizeof(T) > sizeof(char), "Type not big enough!"); // expected-error {{static_assert failed due to requirement 'sizeof(char) > sizeof(char)' "Type not big enough!"}}
 };
 
 S<char> s1; // expected-note {{in instantiation of template class 'S<char>' requested here}}
 S<int> s2;
 
-static_assert(false, L"\xFFFFFFFF"); // expected-error {{static_assert failed L"\xFFFFFFFF"}}
-static_assert(false, u"\U000317FF"); // expected-error {{static_assert failed u"\U000317FF"}}
+static_assert(false, L"\xFFFFFFFF"); // expected-error {{an unevaluated string literal cannot have an encoding prefix}}
+static_assert(false, u"\U000317FF"); // expected-error {{an unevaluated string literal cannot have an encoding prefix}}
 // FIXME: render this as u8"\u03A9"
-static_assert(false, u8"Ω"); // expected-error {{static_assert failed u8"\316\251"}}
-static_assert(false, L"\u1234"); // expected-error {{static_assert failed L"\x1234"}}
-static_assert(false, L"\x1ff" "0\x123" "fx\xfffff" "goop"); // expected-error {{static_assert failed L"\x1FF""0\x123""fx\xFFFFFgoop"}}
+static_assert(false, u8"Ω");     // expected-error {{an unevaluated string literal cannot have an encoding prefix}}
+static_assert(false, L"\u1234"); // expected-error {{an unevaluated string literal cannot have an encoding prefix}}
+static_assert(false, L"\x1ff"    // expected-error {{an unevaluated string literal cannot have an encoding prefix}}
+                     "0\x123"
+                     "fx\xfffff"
+                     "goop");
 
 template<typename T> struct AlwaysFails {
   // Only give one error here.
   static_assert(false, ""); // expected-error {{static_assert failed}}
 };
 AlwaysFails<int> alwaysFails;
 
 template<typename T> struct StaticAssertProtected {