Implement P2361 Unevaluated string literals

[cor3ntin]

Summary

This patch proposes to handle in an uniform fashion
the parsing of strings that are never evaluated,
in asm statement, static assert, attrributes, extern,
etc.

Unevaluated strings are UTF-8 internally and so currently
behave as narrow strings, but these things will diverge with
D93031.

The big question both for this patch and the P2361 paper
is whether we risk breaking code by disallowing
encoding prefixes in this context.
I hope this patch may allow to gather some data on that.

Future work:
Improve the rendering of unicode characters, line break
and so forth in static-assert messages

[aaron.ballman]

In general, I think this is shaping up nicely and is almost complete. I'm adding some additional reviewer though, as this is a somewhat experimental patch for a WG21 proposal that has not been accepted yet and I want to make sure that I'm not missing something. That may also solve the few open questions that still remain.

clang/lib/AST/Expr.cpp

1187–1188

I'd recommend running the entire patch through clang-format though: https://clang.llvm.org/docs/ClangFormat.html#script-for-patch-reformatting

clang/lib/Lex/LiteralSupport.cpp

1897

Looks like this comment is still missing punctuation.

[cor3ntin]

Formatting and missing punctuation

[erichkeane]

clang/include/clang/Basic/DiagnosticLexKinds.td

280

Is there value to combining these two diagnostics with a %select?

clang/lib/Lex/LiteralSupport.cpp

96–97

I might consider rejecting ANY character escape in the less-than-32 part of the table.

For consistency at least, I don't see value in allowing \a if we're rejecting layout things like \t.

119

This is like the 3rd time we're using 'Unevaluated' as a bool parameter. I have a pretty strong preference for making it a scoped-enum in 'Basic' somewhere.

2005

Is this OK? It looks like we're passing a ton of parameters to a diag type that doesn't have any wildcards?

[cor3ntin]

Replace Unevaluated by an enum.

[aaron.ballman]

clang/include/clang/Basic/DiagnosticLexKinds.td

280

I waffled when doing this review, so it's funny you mention it. :-D

We could do: an unevaluated string literal cannot %select{have an encoding prefix|be a user-defined literal}0 but there was just enough text in the select that I felt it wasn't critical to combine. But I don't feel strongly either way.

clang/lib/Lex/LiteralSupport.cpp

96–97

But that's just it, we're accepting \t and \n with this code.

2005

Good catch! The first two are not helpful (the diag engine will silently ignore them), but the second two are for underlines in the diagnostic and are useful.

[erichkeane]

clang/lib/Lex/LiteralSupport.cpp

96–97

Ah! I missed that this is an allow-list instead of a deny-list. That makes me way more comfortable with this code.

IMO, I'd suggest we we allow '\r' (since wouldn't we have problems on Windows at that point, being unable to accept a printable newline for windows?), but disallow \a for now unless someone comes up with a really good reason to allow it.

[cor3ntin]

Accept \r as an escape sequence n unevaluated string literal

[cor3ntin]

Rename commit

[erichkeane]

A couple of small things, otherwise I'm happy; but Aaron has some bigger opens above, plus clang-format, plus the modules from Richard.

clang/include/clang/Basic/DiagnosticLexKinds.td

280

I was waffly on this too, so your waffling + my waffling I think is sufficient reason to not deal with this now.

clang/lib/AST/Expr.cpp

1160

minor preference (perhaps 'nit' level) to move this whole CharByteWidth + IsPascal calculation into its own function. This constructor is absurdly long as it is.

clang/lib/Lex/LiteralSupport.cpp

99

For future clarification, the ones from the 'simple' list here: https://en.cppreference.com/w/cpp/language/escape

that we are missing are: \a \b \f and \v.

I personally think I'm ok with that until someone else says they care.

1896

Hrm.... this is unfortunate. Is there no way to combine the loops? I guess (hope?) that hte list of tokens is at least going to be short...

[cor3ntin]

Formatting

[cor3ntin]

Get rid of the extra loop by using a lambda

[aaron.ballman]

Some naming nits. There are two open questions also: one about module behavior and one about a TODO comment in the patch. If we don't hear back about the modules question, I think that can be handled in a follow-up.

clang/include/clang/Lex/LiteralSupport.h

215

Slight renaming so nobody thinks this is going to be about wide vs narrow vs u8, etc.

243

We should rename anything mentioning StringKind similarly -- this will also help avoid confusion with the StringKind type in Expr.h.

257

Can we make this private now rather than letting callers access it directly?

clang/lib/Lex/LiteralSupport.cpp

1897

When I hear "check" I think it'll return a value; I think this name is a bit more clear.

2007–2009

[cor3ntin]

Address Aaron's comments

clang/lib/Lex/LiteralSupport.cpp

119

Any suggestion for where to

2007–2009

This are actually used by err_string_concat_mixed_suffix

[cor3ntin]

clang/lib/Lex/LiteralSupport.cpp

119

NVM

[erichkeane]

clang/lib/Lex/LiteralSupport.cpp

2007–2009

right, i guess it is just super awkward to have unused parameters passed like this. I know we only check the other direction, but seems awkward. Aaron, thoughts?

[aaron.ballman]

clang/lib/Lex/LiteralSupport.cpp

2007–2009

I'd split it into two calls at this point. e.g.,

if (UnevaluatedStringHasUDL)
  Diags->Report(TokLoc, diag::err_unevaluated_string_udl) << ...;
else
  Diags->Report(TokLoc, diag::err_string_concat_mixed_suffix) << ...;

[cor3ntin]

Cleanup Diagnostics In LiteralSupport

[aaron.ballman]

LGTM aside from two small nits. As for the modules question, if @rsmith doesn't get back to us, I think it's fine to address that post-commit.

clang/include/clang/Lex/LiteralSupport.h

243

Did this one get missed?

clang/test/CXX/dcl.dcl/p4-0x.cpp

25

Can you add the newline back to the end of the file?

[cor3ntin]

Fix EOF & unrenamed StringKind

[cor3ntin]

As approved by CWG
Updates to cxx_status / doc will come later :)

[shafik]

clang/lib/Parse/ParseExpr.cpp

3268

[cor3ntin]

Address Shafik's comment

[shafik]

LGTM but I don't see asm covered in the tests.

clang/lib/AST/Expr.cpp

1140

Why not grouped w/ Ordinary above?

1164

Isn't this the same as Length?

1201

Isn't Str.size() the same as ByteLength?

clang/lib/Lex/LiteralSupport.cpp

90

Should we use Is as a prefix here? Right now it should like we are modifying something.

[cor3ntin]

Rename /EscapeValidInUnevaluatedStringLiteral/IsEscapeValidInUnevaluatedStringLiteral

[cor3ntin]

clang/lib/AST/Expr.cpp

1164

Only when CharByteWidth == 1

1201

ByteLength isn't defined in this scope, I guess i could move it.

[aaron.ballman]

This also should update the cxx_status page and have a release note.

clang/include/clang/Basic/Attr.td

1411 ↗

(On Diff #534201)

What is the plan for non-standard attributes? Are you planning to handle those in a follow-up, or should we be investigating those right now?

clang/include/clang/Parse/Parser.h

1857–1859

Two default bool params is a bad thing but three default bool params seems like we should fix the interface at this point. WDYT?

Also, it's not clear what the new parameter will do, the function could use comments unless fixing the interface makes it sufficiently clear.

clang/lib/AST/Expr.cpp

1140

Specifically because we want the host encoding, not the target encoding.

1164

It is -- I think we can get rid of ByteLength, but it's possible that this exists because of the optimization comment below. I don't insist, but it would be nice to know if we can replace the switch with Length /= CharByteWidth these days.

1189

Add assert(!Pascal && "Can't make an unevaluated Pascal string"); ?

1201

I think it's more clear to use Str.size() because we're copying from Str.data().

1238

clang/lib/Lex/LiteralSupport.cpp

90

+1, I think Is would be an improvement.

[aaron.ballman]

clang/lib/Lex/LiteralSupport.cpp

98

We're still missing support for some escape characters from: http://eel.is/c++draft/lex#nt:simple-escape-sequence-char

Just to verify, UCNs have already been handled by the time we get here, so we don't need to care about those, correct?

1917

Doesn't returning here leave the object in a partially-initialized state? That seems bad.

2083–2086

Is there test coverage that we diagnose this properly?

clang/lib/Lex/PPMacroExpansion.cpp

1872–1874

Test coverage for this change?

clang/lib/Lex/Pragma.cpp

776

Pinging @ChuanqiXu for opinions.

clang/lib/Parse/ParseExpr.cpp

3497–3498

I'm surprised we need special logic in ParseExpressionList() for handling unevaluated string literals; I would have expected that to be needed when parsing a string literal. Nothing changed in the grammar for http://eel.is/c++draft/expr.post.general#nt:expression-list (or initializer-list), so these changes seem wrong. Can you explain the changes a bit more?

clang/lib/Sema/SemaDeclAttr.cpp

878–879 ↗

(On Diff #534201)

Test coverage for these changes?

clang/lib/Sema/SemaDeclCXX.cpp

16474

Test coverage for changes?

[cor3ntin]

clang/lib/AST/Expr.cpp

1140

an unevaluated string is a sequence of 1-byte even on platforms were sizeof(char) would be 2 or 4. It's never influenced by the target's properties

[cor3ntin]

clang/include/clang/Basic/Attr.td

1411 ↗

(On Diff #534201)

I don't feel I'm qualified to answer that. Ideally, attributes that expect string literals that are not evaluated should follow suite.

clang/include/clang/Parse/Parser.h

1857–1859

I'm still not sure that's the best solution. AllowEvaluatedString would only ever be false for attributes, I consider duplicating the function, except it does quite a bit for variadics, which apparently attribute support

Maybe would could have

ParseAttributeArgumentList
ParseExpressionList
ParseExpressionListImpl?

?

clang/lib/AST/Expr.cpp

1164

I think we should.

clang/lib/Lex/LiteralSupport.cpp

98

Just to verify, UCNs have already been handled by the time we get here, so we don't need to care about those, correct?

They are dealt with elsewhere yes (and supported)

2083–2086

What sort of test would you like to see?

clang/lib/Parse/ParseExpr.cpp

3497–3498

We use ParseExpressionList when parsing attribute arguments, and some attributes have unevaluate string as argument - I agree with you that I'd rather find a better solution for attributes, but I came up empty. There is no further reason for this change, and you are right it does not match the grammar.

clang/lib/Sema/SemaDeclAttr.cpp

878–879 ↗

(On Diff #534201)

There is one somewhere, I don;t remember where, The reason we need to do that is that Unevaluated StringLiterals don''t have types

clang/lib/Sema/SemaDeclCXX.cpp

16474

There are some in dcl.link/p2.cpp

[cor3ntin]

Address some of Aaron's comments

[ChuanqiXu]

clang/lib/Lex/Pragma.cpp

776

I think the both options (to modify it or not) are acceptable. Because the input here should be the output of the clang itself. See https://github.com/llvm/llvm-project/blob/ebd0b8a0472b865b7eb6e1a32af97ae31d829033/clang/lib/Basic/Module.cpp#L229-L231 and https://github.com/llvm/llvm-project/blob/ebd0b8a0472b865b7eb6e1a32af97ae31d829033/clang/lib/Frontend/Rewrite/FrontendActions.cpp#L238-L240.

We can see there is no deprecated prefix. So while it is acceptable to modify this since its pattern matches the paper, it doesn't matter really since we can control the input completely.

Personally, I prefer to not touch it. Since I feel like this use case doesn't have been used a lot. So the effort here may not be worthy.

[aaron.ballman]

clang/include/clang/Basic/Attr.td

1411 ↗

(On Diff #534201)

Let's do them in a follow-up. Normally I'd suggest working with @erichkeane on which attributes to apply that to, but he's about to go on a sabbatical and might not have time to help with that. So maybe you can take a first pass at it as best you can and then rope me in to help finalize it, if that'd work for you?

clang/lib/Lex/LiteralSupport.cpp

2083–2086

Pascal strings enabled and using something like [[deprecated("\pOh no, a Pascal string!")]] (or some other unevaluated uses).

clang/lib/Parse/ParseExpr.cpp

3497–3498

I was thinking we'd use a new kind of evaluation context for this. We'd enter the evaluation context when we know we need to parse an expression that is an unevaluated string literal which the string literal parser would pay attention to. This would require knowing up-front when we want to parse an unevaluated string literal, but we should have that information available to us at parse time (I think).

clang/lib/Sema/SemaDeclAttr.cpp

878–879 ↗

(On Diff #534201)

Let's try to track that down, but... an unevaluated string literal still has a type, surely? It'd be const char[] for C++?

[cor3ntin]

clang/lib/Parse/ParseExpr.cpp

3497–3498

After offline discussion, i think what we want to be doing is to have a

ParseAtttributeArgumentList function that is aware of whether the Nth argument is an unevaluated string - by means of modifying tablegen,
and doing the right parsing accordingly.
It would take care of all attributes automatically.
Alas that's a tad more involved.

[aaron.ballman]

clang/lib/Parse/ParseExpr.cpp

3497–3498

+1

I agree it's more involved, but it's also a more general solution that fits nicely in the parser design (we do this sort of thing for other parts of attribute parsing).

[cor3ntin]

clang/lib/Sema/SemaDeclAttr.cpp

878–879 ↗

(On Diff #534201)

It doesn't because it doesn't exist past phase 6.
It's not unevaluated as in decltype, it's more unevaluated as it's a weird token that never participate in the program, the same way a pragma or an attribute don't have a type.
Note that we can revert that change if we do the whole tablegen thing

The relevant test is in test/SemaCXX/warn-thread-safety-parsing.cpp, L17

[cor3ntin]

Add tests for pascal strings (which are not a thing in C++ apparently)

[cor3ntin]

Parse attribute as unevaluated string if they
are declare StringLiteralArgument in the Attr.td file.

WIP

@aaron.ballman Do we agree on direction before I
fix the remaining broken tests?

There are a few limitations, which I'm hoping not to fix there

• It doesn't support variadic string arguments
• checking the type of argument ahead of time seems like a good idea overall, maybe we want to expand that system?

[aaron.ballman]

In D105759#4453440, @cor3ntin wrote:

Parse attribute as unevaluated string if they
are declare StringLiteralArgument in the Attr.td file.

WIP

@aaron.ballman Do we agree on direction before I
fix the remaining broken tests?

Mostly agreed, though I left a comment where I think the direction should change slightly.

There are a few limitations, which I'm hoping not to fix there

• It doesn't support variadic string arguments

That's reasonable; let's leave them as evaluated strings for now so there's no behavioral change.

• checking the type of argument ahead of time seems like a good idea overall, maybe we want to expand that system?

I agree; we currently have the common handler checking argument counts (https://github.com/llvm/llvm-project/blob/1e010c5c4fae43c52d6f5f1c8e8920c26bcc6cc7/clang/lib/Sema/SemaAttr.cpp#L1419), but we don't generate any code for checking argument types. But certainly doesn't need to be done as part of your work.

clang/include/clang/Basic/Attr.td

3048 ↗

(On Diff #535073)

I don't think we should reuse this flag this way. This flag is for the traditional sense of "unevaluated", but unevaluated string literals are a different kind of beast. I think that should be tracked on the argument level. We can either adjust:

class StringArgument<string name, bit opt = 0> : Argument<name, opt>;

so that it takes another bit for whether the string is unevaluated or not, or we could add a new subclass for UnevaluatedStringArgument. Then ClangAttrEmitter.cpp would look at this information when emitting the switch cases.

llvm/cmake/modules/HandleLLVMOptions.cmake

608 ↗

(On Diff #535073)

Spurious change. ;-)

[aaron.ballman]

Clearing the "accepted" status so it's not confusing as to the state of things.

[cor3ntin]

clang/include/clang/Basic/Attr.td

3048 ↗

(On Diff #535073)

This is the previous approach i forgot to fixup everywhere.
My current approach is to always consider StringArgument unevaluated.
I don't think it make sense to have both StringArgument and UnevaluatedStringArgument.
Currently in all the places we accept StringArgument, we check it's a possibly parenthesized StringLiteral

If you want an evaluated string literal, an expression that produce a const char* or something should work

llvm/cmake/modules/HandleLLVMOptions.cmake

608 ↗

(On Diff #535073)

I've been battling with that for weeks, that flag completely breaks my IDE, not sure why. It was inevitable that it ended up in a commit :|

[aaron.ballman]

clang/include/clang/Basic/Attr.td

3048 ↗

(On Diff #535073)

My current approach is to always consider StringArgument unevaluated.
I don't think it make sense to have both StringArgument and UnevaluatedStringArgument.

I think that's potentially a pretty significant change in behavior until we actually evaluate (ahahaha, puns!) all the vendor attributes using a StringArgument. Also, I thought you mentioned you planned to leave variadic string arguments as evaluated strings, so there would be a pretty surprising inconsistency to the behavior there. I would feel more comfortable not changing the behavior of attributes we've not validated are still correct when using unevaluated strings.

[cor3ntin]

Fix tests and handle variadic attributes.

• With that all normal attributes are handled. Only attributes with custop parsing code and those specified as an enum are left untouched.

[cor3ntin]

Note that I have confirmed that all the change attributes require a StringLiteral and go through checkStringLiteralArgumentAttr

[cor3ntin]

revert accidental changes to cmake

[erichkeane]

Just 2 small nits, otherwise this all LGTM.

clang/lib/Parse/ParseDecl.cpp

430 ↗

(On Diff #535356)

Please put a newline between unchained 'if' statements... it makes tehse really hard to read without it.

It happens a few times here.

clang/lib/Sema/SemaDeclAttr.cpp

878 ↗

(On Diff #535356)

Unrelated change here? What is this for?

[cor3ntin]

Address Erich's comments

[cor3ntin]

clang/lib/Sema/SemaDeclAttr.cpp

878 ↗

(On Diff #535356)

Some test i failed to fully revert. good catch!

[aaron.ballman]

I don't think it's correct to assume that all string arguments to attributes are unevaluated, but it is hard to tell where to draw the line sometimes. Backing up a step, as I understand P2361, an unevaluated string is one which is not converted into the execution character set (effectively). Is that correct? If so, then as an example, [[clang::annotate()]] should almost certainly be using an evaluated string because the argument is passed down to LLVM IR and is used in ways we cannot predict. What's more, an unevaluated string cannot have some kinds of escape characters (numeric and conditional escape sequences) and those are currently allowed by clang::annotate and could potentially be used by a backend plugin.

I think other attributes may have similar issues. For example, the alias attribute is a bit of a question mark for me -- that takes a string literal representing an external identifier that is looked up. I'm not certain whether that should be in the execution character set or not, but we do support escape sequences for it: https://godbolt.org/z/v65Yd7a68

I think we need to track evaluated vs not on the argument level so that the attributes in Attr.td can decide which form to use. I think we should default to "evaluated" for any attribute we're on the fence about because that's the current behavior they get today (so we should avoid regressions).

clang/include/clang/Sema/ParsedAttr.h

919 ↗

(On Diff #535373)

clang/lib/Parse/ParseDecl.cpp

291 ↗

(On Diff #535373)

Comment doesn't match the function name. ;-)

453–454 ↗

(On Diff #535373)

What are these lines intended to do? We assign to E but nothing ever reads from it after this assignment and we reset it on the next iteration through the loop.

clang/lib/Parse/ParseExpr.cpp

3497

Can revert these two changes now.

[aaron.ballman]

In D105759#4456864, @aaron.ballman wrote:

I don't think it's correct to assume that all string arguments to attributes are unevaluated, but it is hard to tell where to draw the line sometimes. Backing up a step, as I understand P2361, an unevaluated string is one which is not converted into the execution character set (effectively). Is that correct? If so, then as an example, [[clang::annotate()]] should almost certainly be using an evaluated string because the argument is passed down to LLVM IR and is used in ways we cannot predict. What's more, an unevaluated string cannot have some kinds of escape characters (numeric and conditional escape sequences) and those are currently allowed by clang::annotate and could potentially be used by a backend plugin.

I think other attributes may have similar issues. For example, the alias attribute is a bit of a question mark for me -- that takes a string literal representing an external identifier that is looked up. I'm not certain whether that should be in the execution character set or not, but we do support escape sequences for it: https://godbolt.org/z/v65Yd7a68

I took a quick pass over our existing attributes, and here's my intuition on them regarding encoding of the literal:

Unevaluated Strings are Fine:
AbiTag
TLSModel
Availability
Deprecated
EnableIf/DiagnoseIf
ObjCRuntimeName
PragmaClangBSSSection/PragmaClangDataSection/PragmaClangRodataSection/PragmaClangRelroSection/PragmaClangTextSection (only created implicitly)
Suppress
Target/TargetVersion/TargetClones
Unavailable
Uuid
WarnUnusedResult
NoSanitize
Capability
Assumption
NoBuiltin (it names a builtin name, so this is probably fine to leave unevaluated?)
AcquireHandle/UseHandle/ReleaseHandle
Error
HLSLResourceBinding

Unevaluated String are Potentially Bad:
Annotate
AnnotateType

Unevaluated String Needs More Thinking (common thread is that they survive to LLVM IR):
Alias
AsmLabel
IFunc
BTFDeclTag/BTFTypeTag (is emitted to DWARF with -g so probably evaluated?)
WebAssemblyExportName/WebAssemblyImportModule/WebAssemblyImportModule
ExternalSourceSymbol
SwiftAsyncName/SwiftAttr/SwiftBridge/SwiftName
Section/CodeSeg/InitSeg
WeakRef
EnforceTCB/EnforceTCBLeaf

There's also the escape sequences issue where use of an escape sequence will go from accepted to rejected in these contexts. I did some hunting to see if I could find uses of numeric escape sequences in asm labels or alias attributes, to see if there's some signs this is done in practice:

Testing we can find numeric escape sequences at all:
https://sourcegraph.com/search?q=context:global+%5C%28%5C%22%5B%5B:alpha:%5D%5D*%28%5C%5C%5B%5B:digit:%5D%5D%2B%29%2B%5B%5B:alpha:%5D%5D*%5C%22%5C%29+lang:C+lang:C%2B%2B&patternType=regexp&case=yes&sm=1&groupBy=repo

Testing we can find asm labels at all:
https://sourcegraph.com/search?q=context:global+asm%5C%28%5C%22%5B%5B:alpha:%5D%5D*%5B%5B:alpha:%5D%5D*%5C%22%5C%29+lang:C+lang:C%2B%2B&patternType=regexp&case=yes&sm=1&groupBy=repo

Testing we can find asm labels with numeric escapes:
https://sourcegraph.com/search?q=context:global+asm%5C%28%5C%22%5B%5B:alpha:%5D%5D*%28%5C%5C%5B%5B:digit:%5D%5D%2B%29%2B%5B%5B:alpha:%5D%5D*%5C%22%5C%29+lang:C+lang:C%2B%2B&patternType=regexp&case=yes&sm=1&groupBy=repo

Testing we can find alias attributes at all:
https://sourcegraph.com/search?q=context:global+alias%5C%28%5C%22%5B%5B:alpha:%5D%5D*%5B%5B:alpha:%5D%5D*%5C%22%5C%29+lang:C+lang:C%2B%2B&patternType=regexp&case=yes&sm=1&groupBy=repo

Testing we can find alias attributes with numeric escapes:
https://sourcegraph.com/search?q=context:global+alias%5C%28%5C%22%5B%5B:alpha:%5D%5D*%28%5C%5C%5B%5B:digit:%5D%5D%2B%29%2B%5B%5B:alpha:%5D%5D*%5C%22%5C%29+lang:C+lang:C%2B%2B&patternType=regexp&case=yes&sm=1&groupBy=repo

I think this leaves me with three open questions:

• Do we know of any uses of the annotate attribute that rely on the string literal being in the execution character set? I do not know of any but I know this is used by plugins quite often.
• Do we know of any attributes in the "needs more thinking" list that should have the string literal encoded in the execution character set? I think most of these are for referring to identifiers in source and I expect those would want source character set and not execution character set strings.
• Do we know of any significant body of code using numeric escape sequences in these string literals that could not be relatively easily modified to compile again? I would be surprised, but I think someone should probably run more of the attributes on the "needs more thinking" list through similar searches on source graph and we can use that as an approximation.

If all these answers come back "no" as best we can figure, then I think we can punt on argument-level handling of this until we add an attribute that really does need an execution-encoded (or numeric escape sequence-using) string literal. I think we've got enough time before the Clang 17 branch to hear if the changes cause problems after we've done this due diligence. WDYT?

[cor3ntin]

In D105759#4456864, @aaron.ballman wrote:

I don't think it's correct to assume that all string arguments to attributes are unevaluated, but it is hard to tell where to draw the line sometimes. Backing up a step, as I understand P2361, an unevaluated string is one which is not converted into the execution character set (effectively). Is that correct? If so, then as an example, [[clang::annotate()]] should almost certainly be using an evaluated string because the argument is passed down to LLVM IR and is used in ways we cannot predict. What's more, an unevaluated string cannot have some kinds of escape characters (numeric and conditional escape sequences) and those are currently allowed by clang::annotate and could potentially be used by a backend plugin.

I think other attributes may have similar issues. For example, the alias attribute is a bit of a question mark for me -- that takes a string literal representing an external identifier that is looked up. I'm not certain whether that should be in the execution character set or not, but we do support escape sequences for it: https://godbolt.org/z/v65Yd7a68

I think we need to track evaluated vs not on the argument level so that the attributes in Attr.td can decide which form to use. I think we should default to "evaluated" for any attribute we're on the fence about because that's the current behavior they get today (so we should avoid regressions).

I really don't think it makes sense to have both "unevaluated" and "evaluated" arguments.
We chatted offline and we struggle to find places where escape sequences are used, or examples of attributes intended to be in the execution character set.

My suggestion would be to land the non-attributes changes now, and the attributes bits in early clang 18.
If we find clear example of attributes expecting execution character set, they should be able to be described as an expression, which will be checked as a string literal anyway, hopefully?

In the case of annotate, if these are fed, for example to a debugger, their may need to convert to whatever the debugger expect as encoding, which is not necessarily the execution charset,
Same for plugins, they certainly not expect ebcdic data, for example.
I would expect for example static analyzers and code generator to keep working after the introduction of fexec-charset
So it's important that it remains unevaluated in the front end so that it can be correctly converted to the appropriate encoding of the various consumers. Which doesn't have a single answer

Do we know of any attributes in the "needs more thinking" list that should have the string literal encoded in the execution character set? I think most of these are for referring to identifiers in source and I expect those would want source character set and not execution character set strings.

Identifiers and symbol names are in UTF8, and may get mangle through, for example replacing non-ascii codepoints by UCN. The source character set is never relevant
This address the WebAsm attributes

BTFDeclTag/BTFTypeTag (is emitted to DWARF with -g so probably evaluated?)

Is it correct to assume the debugger file encoding is always the same as the program's ? Probably not!
If need be, we can then transcode the strings when doing codegen for these things

[aaron.ballman]

In D105759#4457041, @cor3ntin wrote:

In D105759#4456864, @aaron.ballman wrote:

I don't think it's correct to assume that all string arguments to attributes are unevaluated, but it is hard to tell where to draw the line sometimes. Backing up a step, as I understand P2361, an unevaluated string is one which is not converted into the execution character set (effectively). Is that correct? If so, then as an example, [[clang::annotate()]] should almost certainly be using an evaluated string because the argument is passed down to LLVM IR and is used in ways we cannot predict. What's more, an unevaluated string cannot have some kinds of escape characters (numeric and conditional escape sequences) and those are currently allowed by clang::annotate and could potentially be used by a backend plugin.

I think other attributes may have similar issues. For example, the alias attribute is a bit of a question mark for me -- that takes a string literal representing an external identifier that is looked up. I'm not certain whether that should be in the execution character set or not, but we do support escape sequences for it: https://godbolt.org/z/v65Yd7a68

I think we need to track evaluated vs not on the argument level so that the attributes in Attr.td can decide which form to use. I think we should default to "evaluated" for any attribute we're on the fence about because that's the current behavior they get today (so we should avoid regressions).

I really don't think it makes sense to have both "unevaluated" and "evaluated" arguments.
We chatted offline and we struggle to find places where escape sequences are used, or examples of attributes intended to be in the execution character set.

In general I agree, but the one scenario that I keep coming back to are attributes like diagnose_if where they take an expression we're evaluating at compile time (condition expression) and a string literal that's not evaluated (warning vs error, diagnostic message itself). But I think the "evaluating at compile time" is part of why I don't think we intend the attribute to be considering the execution character set.

My suggestion would be to land the non-attributes changes now, and the attributes bits in early clang 18.

I think we're almost safe enough to make the attribute changes in Clang 17 so that no attribute uses an evaluated argument, but given that there's less than a month before we make the 17 branch, I think it's probably a good idea to make these changes after the branch point so folks have longer to react. Adding clang-vendors to the review for awareness of the potential for a breaking change.

[cor3ntin]

I removed the changes to attributes.
Nothing else changes except cxx_status/ReleaseNotes.

Unevaluated strings in attributes will be back (in a separate PR)

[aaron.ballman]

LGTM with a minor tweak to the wording on the status page, thank you!

clang/www/cxx_status.html

118–124

[barannikov88]

clang/docs/ReleaseNotes.rst

138

Looks like a copy&paste bug.

[cor3ntin]

clang/docs/ReleaseNotes.rst

138

Nice catch, thanks

[barannikov88]

@cor3ntin
I've been working on pretty much the same functionality in our downstream fork. I was not aware of the paper, nor of the ongoing work in this direction, and so I unfortunately missed the review.
Thanks for this patch, it significantly reduces the number of changes downstream and makes it easier to merge with upstream in the future.

I have a couple of questions about future work:

• IIUC the paper initially addressed this issue with #line directive, but the changes were reverted(?). Is there any chance they can get back?
• Are there any plans for making similar changes to asm statement parsing?

[cor3ntin]

In D105759#4482543, @barannikov88 wrote:

@cor3ntin
I've been working on pretty much the same functionality in our downstream fork. I was not aware of the paper, nor of the ongoing work in this direction, and so I unfortunately missed the review.
Thanks for this patch, it significantly reduces the number of changes downstream and makes it easier to merge with upstream in the future.

I have a couple of questions about future work:

• IIUC the paper initially addressed this issue with #line directive, but the changes were reverted(?). Is there any chance they can get back?

There is a core issue tracking that, ie the c++ committee was concerned about escape sequences in header names
https://www.open-std.org/jtc1/sc22/wg21/docs/cwg_active.html#2693

I'd be happy to bring that back to clang though, as the concerned is unlikely to be warranted for us.

• Are there any plans for making similar changes to asm statement parsing?

The direction of the c++ committee is that what's in asm() is now strictly implementation-defined, so we could but last time there were concerns about escape sequences in there too.

[hubert.reinterpretcast]

I hope this patch may allow to gather some data on that.

@cor3ntin, I have reports that applications having encoding prefixes in static_assert are failing to build. The committee did not adopt the subject paper as a "DR resolution". Is it possible to downgrade to a warning?

[cor3ntin]

In D105759#4540716, @hubert.reinterpretcast wrote:

I hope this patch may allow to gather some data on that.

@cor3ntin, I have reports that applications having encoding prefixes in static_assert are failing to build. The committee did not adopt the subject paper as a "DR resolution". Is it possible to downgrade to a warning?

You know how frequent it is?

Making it a warning is possible but not straightforward.

Previously with a prefix, it was parsed as a wide (for example) string, and we relied on the fact that L was UTF-16/32 to sometimes print a reasonable diagnostics - and sometimes not https://godbolt.org/z/f3Pj4T5aj
This is going to be worse when we add -fexec-charset:

• static_assert(true, L"やあ") is going to be ill-formed when coded as, eg, EBCDIC because it's not representable, and if it is representable we need to either output mojibake or convert the string back to UTF-8 which we are currently not doing.

Another solution maybe to lexically ignore prefixes by replacing the string literal token on the fly such that they are still parsed as unevaluated strings and not encoded, i could look into that.

[hubert.reinterpretcast]

In D105759#4541813, @cor3ntin wrote:

In D105759#4540716, @hubert.reinterpretcast wrote:

I hope this patch may allow to gather some data on that.

@cor3ntin, I have reports that applications having encoding prefixes in static_assert are failing to build. The committee did not adopt the subject paper as a "DR resolution". Is it possible to downgrade to a warning?

You know how frequent it is?

No, but I am concerned that this came up even before we deployed an LLVM 17-based solution (in pre-release testing). I believe that reverting for LLVM 17 is the prudent course of action.

Previously with a prefix, it was parsed as a wide (for example) string, and we relied on the fact that L was UTF-16/32 to sometimes print a reasonable diagnostics - and sometimes not https://godbolt.org/z/f3Pj4T5aj
This is going to be worse when we add -fexec-charset:

• static_assert(true, L"やあ") is going to be ill-formed when coded as, eg, EBCDIC because it's not representable, and if it is representable we need to either output mojibake or convert the string back to UTF-8 which we are currently not doing.

This may be the motivation for the prefixes in the applications in the first place in the context of other compilers: They may have needed the prefix to avoid unrepresentable character issues (e.g., if the other compiler rejects the unprefixed string, but manages to emit the error to the terminal because both the terminal and the compiler use the source encoding).

Another solution maybe to lexically ignore prefixes by replacing the string literal token on the fly such that they are still parsed as unevaluated strings and not encoded, i could look into that.

This sounds good.

[aaron.ballman]

In D105759#4543184, @hubert.reinterpretcast wrote:

In D105759#4541813, @cor3ntin wrote:

In D105759#4540716, @hubert.reinterpretcast wrote:

I hope this patch may allow to gather some data on that.

@cor3ntin, I have reports that applications having encoding prefixes in static_assert are failing to build. The committee did not adopt the subject paper as a "DR resolution". Is it possible to downgrade to a warning?

You know how frequent it is?

No, but I am concerned that this came up even before we deployed an LLVM 17-based solution (in pre-release testing). I believe that reverting for LLVM 17 is the prudent course of action.

Early reports of user code getting tripped up on this is something we should react to while we still can; I'd recommend we change the diagnostic to be a warning that defaults to an error so that users who are caught by the changes can still disable the diagnostic rather than be stuck; for Clang 18, we can explore other solutions to the issue. Would this work for you @hubert.reinterpretcast?

[hubert.reinterpretcast]

In D105759#4543246, @aaron.ballman wrote:

I'd recommend we change the diagnostic to be a warning that defaults to an error so that users who are caught by the changes can still disable the diagnostic rather than be stuck; for Clang 18, we can explore other solutions to the issue. Would this work for you @hubert.reinterpretcast?

I think there are questions about whether an error (or even warning) by default is appropriate. This seems to be a change for C++2c that does not have "DR" treatment from the committee. Considering this a warning controlled by c++2c-compat is a potential direction. Indeed, if we are going to accept the code, we might as well allow it as an extension in C++2c modes. With this line of logic, I don't see why we would want user-side churn of making a migration effort.

[cor3ntin]

In D105759#4543685, @hubert.reinterpretcast wrote:

In D105759#4543246, @aaron.ballman wrote:

I'd recommend we change the diagnostic to be a warning that defaults to an error so that users who are caught by the changes can still disable the diagnostic rather than be stuck; for Clang 18, we can explore other solutions to the issue. Would this work for you @hubert.reinterpretcast?

I think there are questions about whether an error (or even warning) by default is appropriate. This seems to be a change for C++2c that does not have "DR" treatment from the committee. Considering this a warning controlled by c++2c-compat is a potential direction. Indeed, if we are going to accept the code, we might as well allow it as an extension in C++2c modes. With this line of logic, I don't see why we would want user-side churn of making a migration effort.

I will endeavor to have a patch by the beginning of the week.

I think the implementation effort is going to be the same whether it is an error by default or not so we can discuss that. I don't have a strong opinion.
Ideally, that would depend on how many users are affected.

However, I don't think nothing at all is a reasonable expectation here, L in static_assert message does either not work or is ignored. In no case does it do what the user wants https://godbolt.org/z/fYnMqT38P
Text encodings are sufficiently confusing that we should not add to the confusion by not telling users their encodings prefix have no effects.

And, given that prior to c++20 the standard simply ignores encoding prefixes, we could also discuss whether it was ever intended for prefixes to be supported or whether it was an oversight to begin with.

clang-tools-extra/test/clang-tidy/checkers/modernize/unary-static-assert.cpp

 // RUN: %check_clang_tidy -std=c++17-or-later %s modernize-unary-static-assert %t
 
 #define FOO static_assert(sizeof(a) <= 15, "");
 #define MSG ""
 
 void f_textless(int a) {
   static_assert(sizeof(a) <= 10, "");
   // CHECK-MESSAGES: :[[@LINE-1]]:3: warning: use unary 'static_assert' when the string literal is an empty string [modernize-unary-static-assert]
   // CHECK-FIXES: {{^}}  static_assert(sizeof(a) <= 10 );{{$}}
-  static_assert(sizeof(a) <= 12, L"");
-  // CHECK-MESSAGES: :[[@LINE-1]]:3: warning: use unary 'static_assert' when
-  // CHECK-FIXES: {{^}}  static_assert(sizeof(a) <= 12 );{{$}}
   FOO
   // CHECK-FIXES: {{^}}  FOO{{$}}
   static_assert(sizeof(a) <= 17, MSG);
   // CHECK-FIXES: {{^}}  static_assert(sizeof(a) <= 17, MSG);{{$}}
 }
 
 void f_with_tex(int a) {
   static_assert(sizeof(a) <= 10, "Size of variable a is out of range!");
 }
 
 void f_unary(int a) { static_assert(sizeof(a) <= 10); }
 
 void f_incorrect_assert() { static_assert(""); }

clang/docs/ReleaseNotes.rst

- Implemented partial support for `P2448R2: Relaxing some constexpr restrictions <https://wg21.link/p2448r2>`_
   non-constexpr functions and constructors.
 - Clang now supports `requires cplusplus23` for module maps.
 - Implemented `P2564R3: consteval needs to propagate up <https://wg21.link/P2564R3>`_.
 
 C++2c Feature Support
 ^^^^^^^^^^^^^^^^^^^^^
 - Compiler flags ``-std=c++2c`` and ``-std=gnu++2c`` have been added for experimental C++2c implementation work.
 - Implemented `P2738R1: constexpr cast from void* <https://wg21.link/P2738R1>`_.
+- Partially implemented `P2361R6: constexpr cast from void* <https://wg21.link/P2361R6>`_.
+  The changes to attributes declarations are not part of this release.
 
 Resolutions to C++ Defect Reports
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 - Implemented `DR2397 <https://wg21.link/CWG2397>`_ which allows ``auto`` specifier for pointers
   and reference to arrays.
 
 C Language Changes
 ------------------

clang/include/clang/AST/Expr.h

class StringLiteral final
   ///   consider moving it inside StringLiteral.
   ///
   /// * An array of getNumConcatenated() SourceLocation, one for each of the
   ///   token this string is made of.
   ///
   /// * An array of getByteLength() char used to store the string data.
 
 public:
-  enum StringKind { Ordinary, Wide, UTF8, UTF16, UTF32 };
+  enum StringKind { Ordinary, Wide, UTF8, UTF16, UTF32, Unevaluated };
 
 private:
   unsigned numTrailingObjects(OverloadToken<unsigned>) const { return 1; }
   unsigned numTrailingObjects(OverloadToken<SourceLocation>) const {
     return getNumConcatenated();
   }
 
   unsigned numTrailingObjects(OverloadToken<char>) const {
public:
   }
 
   /// Construct an empty string literal.
   static StringLiteral *CreateEmpty(const ASTContext &Ctx,
                                     unsigned NumConcatenated, unsigned Length,
                                     unsigned CharByteWidth);
 
   StringRef getString() const {
-    assert(getCharByteWidth() == 1 &&
+    assert((isUnevaluated() || getCharByteWidth() == 1) &&
            "This function is used in places that assume strings use char");
     return StringRef(getStrDataAsChar(), getByteLength());
   }
 
   /// Allow access to clients that need the byte representation, such as
   /// ASTWriterStmt::VisitStringLiteral().
   StringRef getBytes() const {
     // FIXME: StringRef may not be the right type to use as a result for this.
     return StringRef(getStrDataAsChar(), getByteLength());
   }
StringKind getKind() const {
     return static_cast<StringKind>(StringLiteralBits.Kind);
   }
 
   bool isOrdinary() const { return getKind() == Ordinary; }
   bool isWide() const { return getKind() == Wide; }
   bool isUTF8() const { return getKind() == UTF8; }
   bool isUTF16() const { return getKind() == UTF16; }
   bool isUTF32() const { return getKind() == UTF32; }
+  bool isUnevaluated() const { return getKind() == Unevaluated; }
   bool isPascal() const { return StringLiteralBits.IsPascal; }
 
   bool containsNonAscii() const {
     for (auto c : getString())
       if (!isASCII(c))
         return true;
     return false;
   }

clang/include/clang/Basic/DiagnosticLexKinds.td

 def ext_reserved_user_defined_literal : ExtWarn<
   "invalid suffix on literal; C++11 requires a space between literal and "
   "identifier">, InGroup<ReservedUserDefinedLiteral>, DefaultError;
 def ext_ms_reserved_user_defined_literal : ExtWarn<
   "invalid suffix on literal; C++11 requires a space between literal and "
   "identifier">, InGroup<ReservedUserDefinedLiteral>;
 def err_unsupported_string_concat : Error<
   "unsupported non-standard concatenation of string literals">;
+
+def err_unevaluated_string_prefix : Error<
+  "an unevaluated string literal cannot have an encoding prefix">;
+def err_unevaluated_string_udl : Error<
+  "an unevaluated string literal cannot be a user-defined literal">;
+def err_unevaluated_string_invalid_escape_sequence : Error<
+  "invalid escape sequence '%0' in an unevaluated string literal">;
 def err_string_concat_mixed_suffix : Error<
   "differing user-defined suffixes ('%0' and '%1') in string literal "
   "concatenation">;
 def err_pp_invalid_udl : Error<
   "%select{character|integer}0 literal with user-defined suffix "
   "cannot be used in preprocessor constant expression">;
 def err_bad_string_encoding : Error<
   "illegal character encoding in string literal">;

clang/include/clang/Basic/DiagnosticSemaKinds.td

def ext_implicit_function_decl_c99 : ExtWarn<
   "call to undeclared function %0; ISO C99 and later do not support implicit "
   "function declarations">, InGroup<ImplicitFunctionDeclare>, DefaultError;
 def note_function_suggestion : Note<"did you mean %0?">;
 
 def err_ellipsis_first_param : Error<
   "ISO C requires a named parameter before '...'">;
 def err_declarator_need_ident : Error<"declarator requires an identifier">;
 def err_language_linkage_spec_unknown : Error<"unknown linkage language">;
-def err_language_linkage_spec_not_ascii : Error<
-  "string literal in language linkage specifier cannot have an "
-  "encoding-prefix">;
 def ext_use_out_of_scope_declaration : ExtWarn<
   "use of out-of-scope declaration of %0%select{| whose type is not "
   "compatible with that of an implicit declaration}1">,
   InGroup<DiagGroup<"out-of-scope-function">>;
 def err_inline_non_function : Error<
   "'inline' can only appear on functions%select{| and non-local variables}0">;
 def err_noreturn_non_function : Error<
   "'_Noreturn' can only appear on functions">;

clang/include/clang/Lex/LiteralSupport.h

public:
   uint64_t getValue() const { return Value; }
   StringRef getUDSuffix() const { return UDSuffixBuf; }
   unsigned getUDSuffixOffset() const {
     assert(!UDSuffixBuf.empty() && "no ud-suffix");
     return UDSuffixOffset;
   }
 };
 
+enum class StringLiteralEvalMethod {
+  Evaluated,
+  Unevaluated,
+};
+
 /// StringLiteralParser - This decodes string escape characters and performs
 /// wide string analysis and Translation Phase #6 (concatenation of string
 /// literals) (C99 5.1.1.2p1).
 class StringLiteralParser {
   const SourceManager &SM;
   const LangOptions &Features;
   const TargetInfo &Target;
   DiagnosticsEngine *Diags;
 
   unsigned MaxTokenLength;
   unsigned SizeBound;
   unsigned CharByteWidth;
   tok::TokenKind Kind;
   SmallString<512> ResultBuf;
   char *ResultPtr; // cursor
   SmallString<32> UDSuffixBuf;
   unsigned UDSuffixToken;
   unsigned UDSuffixOffset;
+  StringLiteralEvalMethod EvalMethod;
+
 public:
-  StringLiteralParser(ArrayRef<Token> StringToks,
-                      Preprocessor &PP);
-  StringLiteralParser(ArrayRef<Token> StringToks,
-                      const SourceManager &sm, const LangOptions &features,
-                      const TargetInfo &target,
+  StringLiteralParser(ArrayRef<Token> StringToks, Preprocessor &PP,
+                      StringLiteralEvalMethod StringMethod =
+                          StringLiteralEvalMethod::Evaluated);
+  StringLiteralParser(ArrayRef<Token> StringToks, const SourceManager &sm,
+                      const LangOptions &features, const TargetInfo &target,
                       DiagnosticsEngine *diags = nullptr)
       : SM(sm), Features(features), Target(target), Diags(diags),
         MaxTokenLength(0), SizeBound(0), CharByteWidth(0), Kind(tok::unknown),
-      ResultPtr(ResultBuf.data()), hadError(false), Pascal(false) {
+        ResultPtr(ResultBuf.data()),
+        EvalMethod(StringLiteralEvalMethod::Evaluated), hadError(false),
+        Pascal(false) {
     init(StringToks);
   }
 
-
   bool hadError;
   bool Pascal;
 
   StringRef GetString() const {
     return StringRef(ResultBuf.data(), GetStringLength());
   }
   unsigned GetStringLength() const { return ResultPtr-ResultBuf.data(); }
 
   unsigned GetNumStringChars() const {
     return GetStringLength() / CharByteWidth;
   }
   /// getOffsetOfStringByte - This function returns the offset of the
   /// specified byte of the string data represented by Token.  This handles
   /// advancing over escape sequences in the string.
   ///
   /// If the Diagnostics pointer is non-null, then this will do semantic
   /// checking of the string literal and emit errors and warnings.
   unsigned getOffsetOfStringByte(const Token &TheTok, unsigned ByteNo) const;
 
   bool isOrdinary() const { return Kind == tok::string_literal; }
   bool isWide() const { return Kind == tok::wide_string_literal; }
   bool isUTF8() const { return Kind == tok::utf8_string_literal; }
   bool isUTF16() const { return Kind == tok::utf16_string_literal; }
   bool isUTF32() const { return Kind == tok::utf32_string_literal; }
   bool isPascal() const { return Pascal; }
+  bool isUnevaluated() const {
+    return EvalMethod == StringLiteralEvalMethod::Unevaluated;
+  }
 
   StringRef getUDSuffix() const { return UDSuffixBuf; }
 
   /// Get the index of a token containing a ud-suffix.
   unsigned getUDSuffixToken() const {
     assert(!UDSuffixBuf.empty() && "no ud-suffix");
     return UDSuffixToken;
   }

clang/include/clang/Parse/Parser.h

public:
   // Expr that doesn't include commas.
   ExprResult ParseAssignmentExpression(TypeCastState isTypeCast = NotTypeCast);
 
   ExprResult ParseMSAsmIdentifier(llvm::SmallVectorImpl<Token> &LineToks,
                                   unsigned &NumLineToksConsumed,
                                   bool IsUnevaluated);
 
   ExprResult ParseStringLiteralExpression(bool AllowUserDefinedLiteral = false);
+  ExprResult ParseUnevaluatedStringLiteralExpression();
 
 private:
+  ExprResult ParseStringLiteralExpression(bool AllowUserDefinedLiteral,
+                                          bool Unevaluated);
+
   ExprResult ParseExpressionWithLeadingAt(SourceLocation AtLoc);
 
   ExprResult ParseExpressionWithLeadingExtension(SourceLocation ExtLoc);
 
   ExprResult ParseRHSOfBinaryExpression(ExprResult LHS,
                                         prec::Level MinPrec);
   /// Control what ParseCastExpression will parse.
   enum CastParseKind {
ExprResult ParseExprAfterUnaryExprOrTypeTrait(const Token &OpTok,
                                                      bool &isCastExpr,
                                                      ParsedType &CastTy,
                                                      SourceRange &CastRange);
 
   /// ParseExpressionList - Used for C/C++ (argument-)expression-list.
   bool ParseExpressionList(SmallVectorImpl<Expr *> &Exprs,
                            llvm::function_ref<void()> ExpressionStarts =
                                llvm::function_ref<void()>(),
                            bool FailImmediatelyOnInvalidExpr = false,
                            bool EarlyTypoCorrection = false);
 
   /// ParseSimpleExpressionList - A simple comma-separated list of expressions,
   /// used for misc language extensions.
   bool ParseSimpleExpressionList(SmallVectorImpl<Expr *> &Exprs);
 
   /// ParenParseOption - Control what ParseParenExpression will parse.
   enum ParenParseOption {
     SimpleExpr,      // Only parse '(' expression ')'
     FoldExpr,        // Also allow fold-expression <anything>

clang/include/clang/Sema/Sema.h

ExprResult ActOnParenListExpr(SourceLocation L,
                                 SourceLocation R,
                                 MultiExprArg Val);
 
   /// ActOnStringLiteral - The specified tokens were lexed as pasted string
   /// fragments (e.g. "foo" "bar" L"baz").
   ExprResult ActOnStringLiteral(ArrayRef<Token> StringToks,
                                 Scope *UDLScope = nullptr);
 
+  ExprResult ActOnUnevaluatedStringLiteral(ArrayRef<Token> StringToks);
+
   /// ControllingExprOrType is either an opaque pointer coming out of a
   /// ParsedType or an Expr *. FIXME: it'd be better to split this interface
   /// into two so we don't take a void *, but that's awkward because one of
   /// the operands is either a ParsedType or an Expr *, which doesn't lend
   /// itself to generic code very well.
   ExprResult ActOnGenericSelectionExpr(SourceLocation KeyLoc,
                                        SourceLocation DefaultLoc,
                                        SourceLocation RParenLoc,

clang/lib/AST/Expr.cpp

case Wide:
     CharByteWidth = Target.getWCharWidth();
     break;
   case UTF16:
     CharByteWidth = Target.getChar16Width();
     break;
   case UTF32:
     CharByteWidth = Target.getChar32Width();
     break;
+  case Unevaluated:
+    return sizeof(char); // Host;
   }
   assert((CharByteWidth & 7) == 0 && "Assumes character size is byte multiple");
   CharByteWidth /= 8;
   assert((CharByteWidth == 1 || CharByteWidth == 2 || CharByteWidth == 4) &&
          "The only supported character byte widths are 1,2 and 4!");
   return CharByteWidth;
 }
 
 StringLiteral::StringLiteral(const ASTContext &Ctx, StringRef Str,
                              StringKind Kind, bool Pascal, QualType Ty,
                              const SourceLocation *Loc,
                              unsigned NumConcatenated)
     : Expr(StringLiteralClass, Ty, VK_LValue, OK_Ordinary) {
+
+  unsigned Length = Str.size();
+
+  StringLiteralBits.Kind = Kind;
+  StringLiteralBits.NumConcatenated = NumConcatenated;
+
+  if (Kind != StringKind::Unevaluated) {
     assert(Ctx.getAsConstantArrayType(Ty) &&
            "StringLiteral must be of constant array type!");
     unsigned CharByteWidth = mapCharByteWidth(Ctx.getTargetInfo(), Kind);
     unsigned ByteLength = Str.size();
     assert((ByteLength % CharByteWidth == 0) &&
            "The size of the data must be a multiple of CharByteWidth!");
 
     // Avoid the expensive division. The compiler should be able to figure it
     // out by itself. However as of clang 7, even with the appropriate
     // llvm_unreachable added just here, it is not able to do so.
-  unsigned Length;
     switch (CharByteWidth) {
     case 1:
       Length = ByteLength;
       break;
     case 2:
       Length = ByteLength / 2;
       break;
     case 4:
       Length = ByteLength / 4;
       break;
     default:
       llvm_unreachable("Unsupported character width!");
     }
 
-  StringLiteralBits.Kind = Kind;
     StringLiteralBits.CharByteWidth = CharByteWidth;
     StringLiteralBits.IsPascal = Pascal;
-  StringLiteralBits.NumConcatenated = NumConcatenated;
+  } else {
+    assert(!Pascal && "Can't make an unevaluated Pascal string");
+    StringLiteralBits.CharByteWidth = 1;
+    StringLiteralBits.IsPascal = false;
+  }
+
   *getTrailingObjects<unsigned>() = Length;
 
   // Initialize the trailing array of SourceLocation.
   // This is safe since SourceLocation is POD-like.
   std::memcpy(getTrailingObjects<SourceLocation>(), Loc,
               NumConcatenated * sizeof(SourceLocation));
 
   // Initialize the trailing array of char holding the string data.
-  std::memcpy(getTrailingObjects<char>(), Str.data(), ByteLength);
+  std::memcpy(getTrailingObjects<char>(), Str.data(), Str.size());
 
   setDependence(ExprDependence::None);
 }
 
 StringLiteral::StringLiteral(EmptyShell Empty, unsigned NumConcatenated,
                              unsigned Length, unsigned CharByteWidth)
     : Expr(StringLiteralClass, Empty) {
   StringLiteralBits.CharByteWidth = CharByteWidth;
void *Mem = Ctx.Allocate(totalSizeToAlloc<unsigned, SourceLocation, char>(
                                1, NumConcatenated, Length * CharByteWidth),
                            alignof(StringLiteral));
   return new (Mem)
       StringLiteral(EmptyShell(), NumConcatenated, Length, CharByteWidth);
 }
 
 void StringLiteral::outputString(raw_ostream &OS) const {
   switch (getKind()) {
+  case Unevaluated:
   case Ordinary:
     break; // no prefix.
   case Wide:  OS << 'L'; break;
   case UTF8:  OS << "u8"; break;
   case UTF16: OS << 'u'; break;
   case UTF32: OS << 'U'; break;
   }
   OS << '"';
   static const char Hex[] = "0123456789ABCDEF";

 /// string.
 ///
 SourceLocation
 StringLiteral::getLocationOfByte(unsigned ByteNo, const SourceManager &SM,
                                  const LangOptions &Features,
                                  const TargetInfo &Target, unsigned *StartToken,
                                  unsigned *StartTokenByteOffset) const {
   assert((getKind() == StringLiteral::Ordinary ||
-          getKind() == StringLiteral::UTF8) &&
+          getKind() == StringLiteral::UTF8 ||
+          getKind() == StringLiteral::Unevaluated) &&
          "Only narrow string literals are currently supported");
 
   // Loop over all of the tokens in this string until we find the one that
   // contains the byte we're looking for.
   unsigned TokNo = 0;
   unsigned StringOffset = 0;
   if (StartToken)
     TokNo = *StartToken;

clang/lib/Lex/LiteralSupport.cpp

static DiagnosticBuilder Diag(DiagnosticsEngine *Diags,
                               const char *TokRangeEnd, unsigned DiagID) {
   SourceLocation Begin =
     Lexer::AdvanceToTokenCharacter(TokLoc, TokRangeBegin - TokBegin,
                                    TokLoc.getManager(), Features);
   return Diags->Report(Begin, DiagID) <<
     MakeCharSourceRange(Features, TokLoc, TokBegin, TokRangeBegin, TokRangeEnd);
 }
 
+static bool IsEscapeValidInUnevaluatedStringLiteral(char Escape) {
+  switch (Escape) {
+  case '\'':
+  case '"':
+  case '?':
+  case '\\':
+  case 'a':
+  case 'b':
+  case 'f':
+  case 'n':
+  case 'r':
+  case 't':
+  case 'v':
+    return true;
+  }
+  return false;
+}
+
 /// ProcessCharEscape - Parse a standard C escape sequence, which can occur in
 /// either a character or a string literal.
 static unsigned ProcessCharEscape(const char *ThisTokBegin,
                                   const char *&ThisTokBuf,
                                   const char *ThisTokEnd, bool &HadError,
                                   FullSourceLoc Loc, unsigned CharWidth,
                                   DiagnosticsEngine *Diags,
-                                  const LangOptions &Features) {
+                                  const LangOptions &Features,
+                                  StringLiteralEvalMethod EvalMethod) {
   const char *EscapeBegin = ThisTokBuf;
   bool Delimited = false;
   bool EndDelimiterFound = false;
 
   // Skip the '\' char.
   ++ThisTokBuf;
 
   // We know that this character can't be off the end of the buffer, because
   // that would have been \", which would not have been the end of string.
   unsigned ResultChar = *ThisTokBuf++;
+  char Escape = ResultChar;
   switch (ResultChar) {
   // These map to themselves.
   case '\\': case '\'': case '"': case '?': break;
 
     // These have fixed mappings.
   case 'a':
     // TODO: K&R: the meaning of '\\a' is different in traditional C
     ResultChar = 7;
else
            diag::ext_unknown_escape)
         << "x" + llvm::utohexstr(ResultChar);
     break;
   }
 
   if (Delimited && Diags) {
     if (!EndDelimiterFound)
       Diag(Diags, Features, Loc, ThisTokBegin, EscapeBegin, ThisTokBuf,
            diag::err_expected)
           << tok::r_brace;
     else if (!HadError) {
       Diag(Diags, Features, Loc, ThisTokBegin, EscapeBegin, ThisTokBuf,
            Features.CPlusPlus23 ? diag::warn_cxx23_delimited_escape_sequence
                                 : diag::ext_delimited_escape_sequence)
           << /*delimited*/ 0 << (Features.CPlusPlus ? 1 : 0);
     }
   }
 
+  if (EvalMethod == StringLiteralEvalMethod::Unevaluated &&
+      !IsEscapeValidInUnevaluatedStringLiteral(Escape)) {
+    Diag(Diags, Features, Loc, ThisTokBegin, EscapeBegin, ThisTokBuf,
+         diag::err_unevaluated_string_invalid_escape_sequence)
+        << StringRef(EscapeBegin, ThisTokBuf - EscapeBegin);
+  }
   return ResultChar;
 }
 
 static void appendCodePoint(unsigned Codepoint,
                             llvm::SmallVectorImpl<char> &Str) {
   char ResultBuf[4];
   char *ResultPtr = ResultBuf;
   if (llvm::ConvertCodePointToUTF8(Codepoint, ResultPtr))
if (begin[1] == 'u' || begin[1] == 'U' || begin[1] == 'N') {
         PP.Diag(Loc, diag::err_character_too_large);
       }
 
       ++buffer_begin;
       continue;
     }
     unsigned CharWidth = getCharWidth(Kind, PP.getTargetInfo());
     uint64_t result =
         ProcessCharEscape(TokBegin, begin, end, HadError,
-                        FullSourceLoc(Loc,PP.getSourceManager()),
-                        CharWidth, &PP.getDiagnostics(), PP.getLangOpts());
+                          FullSourceLoc(Loc, PP.getSourceManager()), CharWidth,
+                          &PP.getDiagnostics(), PP.getLangOpts(),
+                          StringLiteralEvalMethod::Evaluated);
     *buffer_begin++ = result;
   }
 
   unsigned NumCharsSoFar = buffer_begin - &codepoint_buffer.front();
 
   if (NumCharsSoFar > 1) {
     if (isOrdinary() && NumCharsSoFar == 4)
       PP.Diag(Loc, diag::warn_four_char_character_literal);

 ///         hexadecimal-escape-sequence hexadecimal-digit
 ///       universal-character-name:
 ///         \u hex-quad
 ///         \U hex-quad hex-quad
 ///       hex-quad:
 ///         hex-digit hex-digit hex-digit hex-digit
 /// \endverbatim
 ///
-StringLiteralParser::
-StringLiteralParser(ArrayRef<Token> StringToks,
-                    Preprocessor &PP)
+StringLiteralParser::StringLiteralParser(ArrayRef<Token> StringToks,
+                                         Preprocessor &PP,
+                                         StringLiteralEvalMethod EvalMethod)
     : SM(PP.getSourceManager()), Features(PP.getLangOpts()),
       Target(PP.getTargetInfo()), Diags(&PP.getDiagnostics()),
       MaxTokenLength(0), SizeBound(0), CharByteWidth(0), Kind(tok::unknown),
-    ResultPtr(ResultBuf.data()), hadError(false), Pascal(false) {
+      ResultPtr(ResultBuf.data()), EvalMethod(EvalMethod), hadError(false),
+      Pascal(false) {
   init(StringToks);
 }
 
 void StringLiteralParser::init(ArrayRef<Token> StringToks){
   // The literal token may have come from an invalid source location (e.g. due
   // to a PCH error), in which case the token length will be 0.
   if (StringToks.empty() || StringToks[0].getLength() < 2)
     return DiagnoseLexingError(SourceLocation());
 
   // Scan all of the string portions, remember the max individual token length,
   // computing a bound on the concatenated string length, and see whether any
   // piece is a wide-string.  If any of the string portions is a wide-string
   // literal, the result is a wide-string literal [C99 6.4.5p4].
   assert(!StringToks.empty() && "expected at least one token");
   MaxTokenLength = StringToks[0].getLength();
   assert(StringToks[0].getLength() >= 2 && "literal token is invalid!");
-  SizeBound = StringToks[0].getLength()-2;  // -2 for "".
-  Kind = StringToks[0].getKind();
-
+  SizeBound = StringToks[0].getLength() - 2; // -2 for "".
   hadError = false;
 
-  // Implement Translation Phase #6: concatenation of string literals
+  // Determines the kind of string from the prefix
+  Kind = tok::string_literal;
+
   /// (C99 5.1.1.2p1).  The common case is only one string fragment.
   for (const Token &Tok : StringToks) {
     if (Tok.getLength() < 2)
       return DiagnoseLexingError(Tok.getLocation());
 
     // The string could be shorter than this if it needs cleaning, but this is a
     // reasonable bound, which is all we need.
     assert(Tok.getLength() >= 2 && "literal token is invalid!");
     SizeBound += Tok.getLength() - 2; // -2 for "".
 
     // Remember maximum string piece length.
     if (Tok.getLength() > MaxTokenLength)
       MaxTokenLength = Tok.getLength();
 
     // Remember if we see any wide or utf-8/16/32 strings.
     // Also check for illegal concatenations.
-    if (Tok.isNot(Kind) && Tok.isNot(tok::string_literal)) {
+    if (isUnevaluated() && Tok.getKind() != tok::string_literal) {
+      if (Diags)
+        Diags->Report(Tok.getLocation(), diag::err_unevaluated_string_prefix);
+      hadError = true;
+    } else if (Tok.isNot(Kind) && Tok.isNot(tok::string_literal)) {
       if (isOrdinary()) {
         Kind = Tok.getKind();
       } else {
         if (Diags)
           Diags->Report(Tok.getLocation(), diag::err_unsupported_string_concat);
         hadError = true;
       }
     }
if (ThisTokEnd[-1] != '"') {
           expandUCNs(ExpandedUDSuffix, UDSuffix);
           UDSuffix = ExpandedUDSuffix;
         }
 
         // C++11 [lex.ext]p8: At the end of phase 6, if a string literal is the
         // result of a concatenation involving at least one user-defined-string-
         // literal, all the participating user-defined-string-literals shall
         // have the same ud-suffix.
-        if (UDSuffixBuf != UDSuffix) {
+        bool UnevaluatedStringHasUDL = isUnevaluated() && !UDSuffix.empty();
+        if (UDSuffixBuf != UDSuffix || UnevaluatedStringHasUDL) {
           if (Diags) {
             SourceLocation TokLoc = StringToks[i].getLocation();
+            if (UnevaluatedStringHasUDL) {
+              Diags->Report(TokLoc, diag::err_unevaluated_string_udl)
+                  << SourceRange(TokLoc, TokLoc);
+            } else {
               Diags->Report(TokLoc, diag::err_string_concat_mixed_suffix)
                   << UDSuffixBuf << UDSuffix
-              << SourceRange(UDSuffixTokLoc, UDSuffixTokLoc)
-              << SourceRange(TokLoc, TokLoc);
+                  << SourceRange(UDSuffixTokLoc, UDSuffixTokLoc);
+            }
           }
           hadError = true;
         }
       }
     }
 
     // Strip the end quote.
     --ThisTokEnd;
if (ThisTokBuf[0] == 'R') {
       if (ThisTokBuf[0] != '"') {
         // The file may have come from PCH and then changed after loading the
         // PCH; Fail gracefully.
         return DiagnoseLexingError(StringToks[i].getLocation());
       }
       ++ThisTokBuf; // skip "
 
       // Check if this is a pascal string
-      if (Features.PascalStrings && ThisTokBuf + 1 != ThisTokEnd &&
-          ThisTokBuf[0] == '\\' && ThisTokBuf[1] == 'p') {
+      if (!isUnevaluated() && Features.PascalStrings &&
+          ThisTokBuf + 1 != ThisTokEnd && ThisTokBuf[0] == '\\' &&
+          ThisTokBuf[1] == 'p') {
 
         // If the \p sequence is found in the first token, we have a pascal string
         // Otherwise, if we already have a pascal string, ignore the first \p
         if (i == 0) {
           ++ThisTokBuf;
           Pascal = true;
         } else if (Pascal)
           ThisTokBuf += 2;
       }
if (ThisTokBuf[0] == 'R') {
           EncodeUCNEscape(ThisTokBegin, ThisTokBuf, ThisTokEnd,
                           ResultPtr, hadError,
                           FullSourceLoc(StringToks[i].getLocation(), SM),
                           CharByteWidth, Diags, Features);
           continue;
         }
         // Otherwise, this is a non-UCN escape character.  Process it.
         unsigned ResultChar =
             ProcessCharEscape(ThisTokBegin, ThisTokBuf, ThisTokEnd, hadError,
                               FullSourceLoc(StringToks[i].getLocation(), SM),
-                            CharByteWidth*8, Diags, Features);
+                              CharByteWidth * 8, Diags, Features, EvalMethod);
 
         if (CharByteWidth == 4) {
           // FIXME: Make the type of the result buffer correct instead of
           // using reinterpret_cast.
           llvm::UTF32 *ResultWidePtr = reinterpret_cast<llvm::UTF32*>(ResultPtr);
           *ResultWidePtr = ResultChar;
           ResultPtr += 4;
         } else if (CharByteWidth == 2) {
           // FIXME: Make the type of the result buffer correct instead of
           // using reinterpret_cast.
           llvm::UTF16 *ResultWidePtr = reinterpret_cast<llvm::UTF16*>(ResultPtr);
           *ResultWidePtr = ResultChar & 0xFFFF;
           ResultPtr += 2;
         } else {
           assert(CharByteWidth == 1 && "Unexpected char width");
           *ResultPtr++ = ResultChar & 0xFF;
         }
       }
     }
   }
 
+  assert((!Pascal || !isUnevaluated()) &&
+         "Pascal string in unevaluated context");
   if (Pascal) {
     if (CharByteWidth == 4) {
       // FIXME: Make the type of the result buffer correct instead of
       // using reinterpret_cast.
       llvm::UTF32 *ResultWidePtr = reinterpret_cast<llvm::UTF32*>(ResultBuf.data());
       ResultWidePtr[0] = GetNumStringChars() - 1;
     } else if (CharByteWidth == 2) {
       // FIXME: Make the type of the result buffer correct instead of
if (SpellingPtr[1] == 'u' || SpellingPtr[1] == 'U' ||
       if (Len > ByteNo) {
         // ByteNo is somewhere within the escape sequence.
         SpellingPtr = EscapePtr;
         break;
       }
       ByteNo -= Len;
     } else {
       ProcessCharEscape(SpellingStart, SpellingPtr, SpellingEnd, HadError,
-                        FullSourceLoc(Tok.getLocation(), SM),
-                        CharByteWidth*8, Diags, Features);
+                        FullSourceLoc(Tok.getLocation(), SM), CharByteWidth * 8,
+                        Diags, Features, StringLiteralEvalMethod::Evaluated);
       --ByteNo;
     }
     assert(!HadError && "This method isn't valid on erroneous strings");
   }
 
   return SpellingPtr-SpellingStart;
 }
 
 /// Determine whether a suffix is a valid ud-suffix. We avoid treating reserved
 /// suffixes as ud-suffixes, because the diagnostic experience is better if we
 /// treat it as an invalid suffix.
 bool StringLiteralParser::isValidUDSuffix(const LangOptions &LangOpts,
                                           StringRef Suffix) {
   return NumericLiteralParser::isValidUDSuffix(LangOpts, Suffix) ||
          Suffix == "sv";
 }

clang/lib/Lex/PPMacroExpansion.cpp

#include "clang/Basic/TransformTypeTraits.def"
     }
 
     SourceLocation LParenLoc = Tok.getLocation();
     LexNonComment(Tok);
 
     if (!Tok.isAnnotation() && Tok.getIdentifierInfo())
       Tok.setKind(tok::identifier);
     else if (Tok.is(tok::string_literal) && !Tok.hasUDSuffix()) {
-      StringLiteralParser Literal(Tok, *this);
+      StringLiteralParser Literal(Tok, *this,
+                                  StringLiteralEvalMethod::Unevaluated);
       if (Literal.hadError)
         return;
 
       Tok.setIdentifierInfo(getIdentifierInfo(Literal.GetString()));
       Tok.setKind(tok::identifier);
     } else {
       Diag(Tok.getLocation(), diag::err_pp_identifier_arg_not_identifier)
         << Tok.getKind();
       // Don't walk past anything that's not a real token.

clang/lib/Lex/Pragma.cpp

 // Lex a component of a module name: either an identifier or a string literal;
 // for components that can be expressed both ways, the two forms are equivalent.
 static bool LexModuleNameComponent(
     Preprocessor &PP, Token &Tok,
     std::pair<IdentifierInfo *, SourceLocation> &ModuleNameComponent,
     bool First) {
   PP.LexUnexpandedToken(Tok);
   if (Tok.is(tok::string_literal) && !Tok.hasUDSuffix()) {
     StringLiteralParser Literal(Tok, PP);
     if (Literal.hadError)
       return true;
     ModuleNameComponent = std::make_pair(
         PP.getIdentifierInfo(Literal.GetString()), Tok.getLocation());
   } else if (!Tok.isAnnotation() && Tok.getIdentifierInfo()) {
     ModuleNameComponent =
         std::make_pair(Tok.getIdentifierInfo(), Tok.getLocation());
   } else {
if (II->isStr("assert")) {
       DumpAnnot.setAnnotationRange(SourceRange(Tok.getLocation()));
       PP.EnterToken(DumpAnnot, /*IsReinject*/false);
     } else if (II->isStr("diag_mapping")) {
       Token DiagName;
       PP.LexUnexpandedToken(DiagName);
       if (DiagName.is(tok::eod))
         PP.getDiagnostics().dump();
       else if (DiagName.is(tok::string_literal) && !DiagName.hasUDSuffix()) {
-        StringLiteralParser Literal(DiagName, PP);
+        StringLiteralParser Literal(DiagName, PP,
+                                    StringLiteralEvalMethod::Unevaluated);
         if (Literal.hadError)
           return;
         PP.getDiagnostics().dump(Literal.GetString());
       } else {
         PP.Diag(DiagName, diag::warn_pragma_debug_missing_argument)
             << II->getName();
       }
     } else if (II->isStr("llvm_fatal_error")) {

clang/lib/Parse/ParseDeclCXX.cpp

 /// and just before that, that extern was seen.
 ///
 ///       linkage-specification: [C++ 7.5p2: dcl.link]
 ///         'extern' string-literal '{' declaration-seq[opt] '}'
 ///         'extern' string-literal declaration
 ///
 Decl *Parser::ParseLinkage(ParsingDeclSpec &DS, DeclaratorContext Context) {
   assert(isTokenStringLiteral() && "Not a string literal!");
-  ExprResult Lang = ParseStringLiteralExpression(false);
+  ExprResult Lang = ParseUnevaluatedStringLiteralExpression();
 
   ParseScope LinkageScope(this, Scope::DeclScope);
   Decl *LinkageSpec =
       Lang.isInvalid()
           ? nullptr
           : Actions.ActOnStartLinkageSpecification(
                 getCurScope(), DS.getSourceRange().getBegin(), Lang.get(),
                 Tok.is(tok::l_brace) ? Tok.getLocation() : SourceLocation());
if (Tok.is(tok::r_paren)) {
 
     if (!isTokenStringLiteral()) {
       Diag(Tok, diag::err_expected_string_literal)
           << /*Source='static_assert'*/ 1;
       SkipMalformedDecl();
       return nullptr;
     }
 
-    AssertMessage = ParseStringLiteralExpression();
+    AssertMessage = ParseUnevaluatedStringLiteralExpression();
     if (AssertMessage.isInvalid()) {
       SkipMalformedDecl();
       return nullptr;
     }
   }
 
   T.consumeClose();
 

clang/lib/Parse/ParseExpr.cpp

 /// form string literals, and also handles string concatenation [C99 5.1.1.2,
 /// translation phase #6].
 ///
 /// \verbatim
 ///       primary-expression: [C99 6.5.1]
 ///         string-literal
 /// \verbatim
 ExprResult Parser::ParseStringLiteralExpression(bool AllowUserDefinedLiteral) {
+  return ParseStringLiteralExpression(AllowUserDefinedLiteral,
+                                      /*Unevaluated=*/false);
+}
+
+ExprResult Parser::ParseUnevaluatedStringLiteralExpression() {
+  return ParseStringLiteralExpression(/*AllowUserDefinedLiteral=*/false,
+                                      /*Unevaluated=*/true);
+}
+
+ExprResult Parser::ParseStringLiteralExpression(bool AllowUserDefinedLiteral,
+                                                bool Unevaluated) {
   assert(isTokenStringLiteral() && "Not a string literal!");
 
   // String concat.  Note that keywords like __func__ and __FUNCTION__ are not
   // considered to be strings for concatenation purposes.
   SmallVector<Token, 4> StringToks;
 
   do {
     StringToks.push_back(Tok);
     ConsumeStringToken();
   } while (isTokenStringLiteral());
 
+  if (Unevaluated) {
+    assert(!AllowUserDefinedLiteral && "UDL are always evaluated");
+    return Actions.ActOnUnevaluatedStringLiteral(StringToks);
+  }
+
   // Pass the set of string tokens, ready for concatenation, to the actions.
   return Actions.ActOnStringLiteral(StringToks,
                                     AllowUserDefinedLiteral ? getCurScope()
                                                             : nullptr);
 }
 
 /// ParseGenericSelectionExpression - Parse a C11 generic-selection
 /// [C11 6.5.1.1].
bool Parser::ParseExpressionList(SmallVectorImpl<Expr *> &Exprs,
   while (true) {
     if (ExpressionStarts)
       ExpressionStarts();
 
     ExprResult Expr;
     if (getLangOpts().CPlusPlus11 && Tok.is(tok::l_brace)) {
       Diag(Tok, diag::warn_cxx98_compat_generalized_initializer_lists);
       Expr = ParseBraceInitializer();
     } else
       Expr = ParseAssignmentExpression();
 
     if (EarlyTypoCorrection)
       Expr = Actions.CorrectDelayedTyposInExpr(Expr);
 
     if (Tok.is(tok::ellipsis))
       Expr = Actions.ActOnPackExpansion(Expr.get(), ConsumeToken());
     else if (Tok.is(tok::code_completion)) {
       // There's nothing to suggest in here as we parsed a full expression.

clang/lib/Sema/SemaDeclCXX.cpp

 /// the '{'. ExternLoc is the location of the 'extern', Lang is the
 /// language string literal. LBraceLoc, if valid, provides the location of
 /// the '{' brace. Otherwise, this linkage specification does not
 /// have any braces.
 Decl *Sema::ActOnStartLinkageSpecification(Scope *S, SourceLocation ExternLoc,
                                            Expr *LangStr,
                                            SourceLocation LBraceLoc) {
   StringLiteral *Lit = cast<StringLiteral>(LangStr);
-  if (!Lit->isOrdinary()) {
-    Diag(LangStr->getExprLoc(), diag::err_language_linkage_spec_not_ascii)
-      << LangStr->getSourceRange();
-    return nullptr;
-  }
+  assert(Lit->isUnevaluated() && "Unexpected string literal kind");
 
   StringRef Lang = Lit->getString();
   LinkageSpecDecl::LanguageIDs Language;
   if (Lang == "C")
     Language = LinkageSpecDecl::lang_c;
   else if (Lang == "C++")
     Language = LinkageSpecDecl::lang_cxx;
   else {
bool InTemplateDefinition =
         getLangOpts().CPlusPlus && CurContext->isDependentContext();
 
     if (!Failed && !Cond && !InTemplateDefinition) {
 
       SmallString<256> MsgBuffer;
       llvm::raw_svector_ostream Msg(MsgBuffer);
       if (AssertMessage) {
         const auto *MsgStr = cast<StringLiteral>(AssertMessage);
-        if (MsgStr->isOrdinary())
-          Msg << MsgStr->getString();
-        else
-          MsgStr->printPretty(Msg, nullptr, getPrintingPolicy());
       }
 
       Expr *InnerCond = nullptr;
       std::string InnerCondDescription;
       std::tie(InnerCond, InnerCondDescription) =
         findFailedBooleanCondition(Converted.get());
       if (InnerCond && isa<ConceptSpecializationExpr>(InnerCond)) {
         // Drill down into concept specialization expressions to see why they

clang/lib/Sema/SemaExpr.cpp

if (S.LookupLiteralOperator(Scope, R, llvm::ArrayRef(ArgTy, Args.size()),
                               /*AllowRaw*/ false, /*AllowTemplate*/ false,
                               /*AllowStringTemplatePack*/ false,
                               /*DiagnoseMissing*/ true) == Sema::LOLR_Error)
     return ExprError();
 
   return S.BuildLiteralOperatorCall(R, OpNameInfo, Args, LitEndLoc);
 }
 
+ExprResult Sema::ActOnUnevaluatedStringLiteral(ArrayRef<Token> StringToks) {
+  StringLiteralParser Literal(StringToks, PP,
+                              StringLiteralEvalMethod::Unevaluated);
+  if (Literal.hadError)
+    return ExprError();
+
+  SmallVector<SourceLocation, 4> StringTokLocs;
+  for (const Token &Tok : StringToks)
+    StringTokLocs.push_back(Tok.getLocation());
+
+  StringLiteral *Lit = StringLiteral::Create(
+      Context, Literal.GetString(), StringLiteral::Unevaluated, false, {},
+      &StringTokLocs[0], StringTokLocs.size());
+
+  if (!Literal.getUDSuffix().empty()) {
+    SourceLocation UDSuffixLoc =
+        getUDSuffixLoc(*this, StringTokLocs[Literal.getUDSuffixToken()],
+                       Literal.getUDSuffixOffset());
+    return ExprError(Diag(UDSuffixLoc, diag::err_invalid_string_udl));
+  }
+
+  return Lit;
+}
+
 /// ActOnStringLiteral - The specified tokens were lexed as pasted string
 /// fragments (e.g. "foo" "bar" L"baz").  The result string has to handle string
 /// concatenation ([C99 5.1.1.2, translation phase #6]), so it may come from
 /// multiple tokens.  However, the common case is that StringToks points to one
 /// string.
 ///
 ExprResult
 Sema::ActOnStringLiteral(ArrayRef<Token> StringToks, Scope *UDLScope) {

clang/lib/Sema/SemaExprCXX.cpp

if (const PointerType *ToPtrType = ToType->getAs<PointerType>())
               // We don't allow UTF literals to be implicitly converted
               break;
             case StringLiteral::Ordinary:
               return (ToPointeeType->getKind() == BuiltinType::Char_U ||
                       ToPointeeType->getKind() == BuiltinType::Char_S);
             case StringLiteral::Wide:
               return Context.typesAreCompatible(Context.getWideCharType(),
                                                 QualType(ToPointeeType, 0));
+            case StringLiteral::Unevaluated:
+              assert(false && "Unevaluated string literal in expression");
+              break;
           }
         }
       }
 
   return false;
 }
 
 static ExprResult BuildCXXCastArgument(Sema &S,

clang/lib/Sema/SemaInit.cpp

static StringInitFailureKind IsStringInit(Expr *Init, const ArrayType *AT,
   case StringLiteral::Wide:
     if (Context.typesAreCompatible(Context.getWideCharType(), ElemTy))
       return SIF_None;
     if (ElemTy->isCharType() || ElemTy->isChar8Type())
       return SIF_WideStringIntoChar;
     if (IsWideCharCompatible(ElemTy, Context))
       return SIF_IncompatWideStringIntoWideChar;
     return SIF_Other;
+  case StringLiteral::Unevaluated:
+    assert(false && "Unevaluated string literal in initialization");
+    break;
   }
 
   llvm_unreachable("missed a StringLiteral kind?");
 }
 
 static StringInitFailureKind IsStringInit(Expr *init, QualType declType,
                                           ASTContext &Context) {
   const ArrayType *arrayType = Context.getAsArrayType(declType);

clang/test/CXX/dcl.dcl/dcl.link/p2.cpp

 // RUN: %clang_cc1 -std=c++11 -verify %s
 
 extern "C" {
   extern R"(C++)" { }
 }
 
 #define plusplus "++"
 extern "C" plusplus {
 }
 
-extern u8"C" {} // expected-error {{string literal in language linkage specifier cannot have an encoding-prefix}}
-extern L"C" {} // expected-error {{string literal in language linkage specifier cannot have an encoding-prefix}}
-extern u"C++" {} // expected-error {{string literal in language linkage specifier cannot have an encoding-prefix}}
-extern U"C" {} // expected-error {{string literal in language linkage specifier cannot have an encoding-prefix}}
+extern u8"C" {}  // expected-error {{an unevaluated string literal cannot have an encoding prefix}}
+extern L"C" {}   // expected-error {{an unevaluated string literal cannot have an encoding prefix}}
+extern u"C++" {} // expected-error {{an unevaluated string literal cannot have an encoding prefix}}
+extern U"C" {}   // expected-error {{an unevaluated string literal cannot have an encoding prefix}}

clang/test/CXX/dcl.dcl/p4-0x.cpp

struct U {
   constexpr operator long() const { return 0; } // expected-note {{candidate}}
 };
 
 static_assert(S(true), "");
 static_assert(S(false), "not so fast"); // expected-error {{not so fast}}
 static_assert(T(), "");
 static_assert(U(), ""); // expected-error {{ambiguous}}
 
-static_assert(false, L"\x14hi" "!" R"x(")x"); // expected-error {{static assertion failed: L"\024hi!\""}}
+static_assert(false, L"\x14hi" // expected-error {{an unevaluated string literal cannot have an encoding prefix}} \
+                               // expected-error {{invalid escape sequence '\x14' in an unevaluated string literal}}
+                     "!"
+                     R"x(")x");

clang/test/FixIt/fixit-static-assert.cpp

 // RUN: %clang_cc1 -std=c++14 %s -fdiagnostics-parseable-fixits %s 2>&1 | FileCheck %s
 // Ensure no warnings are emitted in c++17.
 // RUN: %clang_cc1 -std=c++17 %s -verify=cxx17
 // RUN: %clang_cc1 -std=c++14 %s -fixit-recompile -fixit-to-temporary -Werror
 
 // cxx17-no-diagnostics
 
 static_assert(true && "String");
 // CHECK-DAG: {[[@LINE-1]]:20-[[@LINE-1]]:22}:","
 
 // String literal prefixes are good.
 static_assert(true && R"(RawString)");
 // CHECK-DAG: {[[@LINE-1]]:20-[[@LINE-1]]:22}:","
-static_assert(true && L"RawString");
-// CHECK-DAG: {[[@LINE-1]]:20-[[@LINE-1]]:22}:","
 
 static_assert(true);
 // CHECK-DAG: {[[@LINE-1]]:19-[[@LINE-1]]:19}:", \"\""
 
 // While its technically possible to transform this to
 // static_assert(true, "String") we don't attempt this fix.
 static_assert("String" && true);
 // CHECK-DAG: {[[@LINE-1]]:31-[[@LINE-1]]:31}:", \"\""
 
 // Don't be smart and look in parentheses.
 static_assert((true && "String"));
 // CHECK-DAG: {[[@LINE-1]]:33-[[@LINE-1]]:33}:", \"\""

clang/test/SemaCXX/static-assert.cpp

 template<typename T> struct S {
     static_assert(sizeof(T) > sizeof(char), "Type not big enough!"); // expected-error {{static assertion failed due to requirement 'sizeof(char) > sizeof(char)': Type not big enough!}} \
                                                                      // expected-note {{1 > 1}}
 };
 
 S<char> s1; // expected-note {{in instantiation of template class 'S<char>' requested here}}
 S<int> s2;
 
-static_assert(false, L"\xFFFFFFFF"); // expected-error {{static assertion failed: L"\xFFFFFFFF"}}
-static_assert(false, u"\U000317FF"); // expected-error {{static assertion failed: u"\U000317FF"}}
-
-static_assert(false, u8"Ω"); // expected-error {{static assertion failed: u8"\316\251"}}
-static_assert(false, L"\u1234"); // expected-error {{static assertion failed: L"\x1234"}}
-static_assert(false, L"\x1ff" "0\x123" "fx\xfffff" "goop"); // expected-error {{static assertion failed: L"\x1FF""0\x123""fx\xFFFFFgoop"}}
-
+static_assert(false, L"\xFFFFFFFF"); // expected-error {{an unevaluated string literal cannot have an encoding prefix}} \
+                                     // expected-error {{invalid escape sequence '\xFFFFFFFF' in an unevaluated string literal}}
+static_assert(false, u"\U000317FF"); // expected-error {{an unevaluated string literal cannot have an encoding prefix}}
+// FIXME: render this as u8"\u03A9"
+static_assert(false, u8"Ω");     // expected-error {{an unevaluated string literal cannot have an encoding prefix}}
+static_assert(false, L"\u1234"); // expected-error {{an unevaluated string literal cannot have an encoding prefix}}
+static_assert(false, L"\x1ff"    // expected-error {{an unevaluated string literal cannot have an encoding prefix}} \
+                                 // expected-error {{invalid escape sequence '\x1ff' in an unevaluated string literal}}
+                     "0\x123"    // expected-error {{invalid escape sequence '\x123' in an unevaluated string literal}}
+                     "fx\xfffff" // expected-error {{invalid escape sequence '\xfffff' in an unevaluated string literal}}
+                     "goop");
+
+static_assert(false, "\'\"\?\\\a\b\f\n\r\t\v"); // expected-error {{'"?\<U+0007><U+0008>}}
+static_assert(true, "\xFF"); // expected-error {{invalid escape sequence '\xFF' in an unevaluated string literal}}
+static_assert(true, "\123"); // expected-error {{invalid escape sequence '\123' in an unevaluated string literal}}
+static_assert(true, "\pOh no, a Pascal string!"); // expected-warning {{unknown escape sequence '\p'}} \
+                                                  // expected-error {{invalid escape sequence '\p' in an unevaluated string literal}}
 static_assert(false, R"(a
 \tb
 c
 )"); // expected-error@-3 {{static assertion failed: a\n\tb\nc\n}}
 
 static_assert(false, "\u0080\u0081\u0082\u0083\u0099\u009A\u009B\u009C\u009D\u009E\u009F");
 // expected-error@-1 {{static assertion failed: <U+0080><U+0081><U+0082><U+0083><U+0099><U+009A><U+009B><U+009C><U+009D><U+009E><U+009F>}}
 

clang/www/cxx_status.html

  <tr>
   <td>Making non-encodable string literals ill-formed</td>
   <td><a href="https://wg21.link/P1854R4">P1854R4</a> (<a href="#dr">DR</a>)</td>
   <td class="full" align="center">Clang 14</td>
  </tr>
  <tr>
   <td>Unevaluated strings</td>
   <td><a href="https://wg21.link/P2361R6">P2361R6</a></td>
-  <td class="none" align="center">No</td>
+  <td class="partial" align="center">
+    <details>
+      <summary>Clang 17 (Partial)</summary>
+      Attributes arguments don't yet parse as unevaluated string literals.
+    </details>
+  </td>
  </tr>
  <tr>
   <td>Add @, $, and ` to the basic character set</td>
   <td><a href="https://wg21.link/P2558R2">P2558R2</a></td>
   <td class="full" align="center">Yes</td>
  </tr>
  <tr>
   <td>constexpr cast from <tt>void*</tt></td>
   <td><a href="https://wg21.link/P2738R1">P2738R1</a></td>