[clang] Add support for “regular” keyword attributes

[rsandifo-arm]

Summary

Platform-specific language extensions often want to provide a way of
indicating that certain functions should be called in a different way,
compiled in a different way, or otherwise treated differently from a
“normal” function. Honoring these indications is often required for
correctness, rather being than an optimization/QoI thing.

If a function declaration has a property P that matters for correctness,
it will be ODR-incompatible with a function that does not have property P.
If a function type has a property P that affects the calling convention,
it will not be two-way compatible with a function type that does not
have property P. These properties therefore affect language semantics.
That in turn means that they cannot be treated as standard [[]]
attributes.

Until now, many of these properties have been specified using GNU-style
attributes instead. GNU attributes have traditionally been more lax
than standard attributes, with many of them having semantic meaning.
Examples include calling conventions and the vector_size attribute.

However, there is a big drawback to using GNU attributes for semantic
information: compilers that don't understand the attributes will
(by default) emit a warning rather than an error. They will go on to
compile the code as though the attributes weren't present, which will
inevitably lead to wrong code in most cases. For users who live
dangerously and disable the warning, this wrong code could even be
generated silently.

A more robust approach would be to specify the properties using
keywords, which older compilers would then reject. Some vendor-specific
extensions have already taken this approach. But traditionally, each
such keyword has been treated as a language extension in its own right.
This has three major drawbacks:

(1) The parsing rules need to be kept up-to-date as the language evolves.

(2) There are often corner cases that similar extensions handle differently.

(3) Each extension requires more custom code than a standard attribute.

The underlying problem for all three is that, unlike for true attributes,
there is no established template that extensions can reuse. The purpose
of this patch series is to try to provide such a template.

One option would have been to pick an existing keyword and do whatever
that keyword does. The problem with that is that most keywords only
apply to specific kinds of types, kinds of decls, etc., and so the
parsing rules are (for good reason) not generally applicable to all
types and decls.

Really, the “only” thing wrong with using standard attributes is that
standard attributes cannot affect semantics. In all other respects
they provide exactly what we need: a well-defined grammar that evolves
with the language, clear rules about what an attribute appertains to,
and so on.

This series therefore adds keyword “attributes” that can appear
exactly where a standard attribute can appear and that appertain
to exactly what a standard attribute would appertain to. The link is
mechanical and no opt-outs or variations are allowed. This should
make the keywords predictable for programmers who are already
familiar with standard attributes.

This does mean that these keywords will be accepted for parsing purposes
in many more places than necessary. Inappropriate uses will then be
diagnosed during semantic analysis. However, the compiler would need
to reject the keywords in those positions whatever happens, and treating
them as ostensible attributes shouldn't be any worse than the alternative.
In some cases it might even be better. For example, SME's
__arm_streaming attribute would make conceptual sense as a statement
attribute, so someone who takes a “try-it-and-see” approach might write:

__arm_streaming { …block-of-code…; }

In fact, we did consider supporting this originally. The reason for
rejecting it was that it was too difficult to implement, rather than
because it didn't make conceptual sense.

One slight disadvantage of the keyword-based approach is that it isn't
possible to use #pragma clang attribute with the keywords. Perhaps we
could add support for that in future, if it turns out to be useful.

For want of a better term, I've called the new attributes "regular"
keyword attributes (in the sense that their parsing is regular wrt
standard attributes), as opposed to "custom" keyword attributes that
have their own parsing rules.

This patch adds the Attr.td support for regular keyword attributes.
Adding an attribute with a RegularKeyword spelling causes tablegen
to define the associated tokens and to record that attributes created
with that syntax are regular keyword attributes rather than custom
keyword attributes.

A follow-on patch contains the main Parse and Sema support,
which is enabled automatically by the Attr.td definition.

Other notes:

• The series does not allow regular keyword attributes to take

arguments, but this could be added in future.

• I wondered about trying to use tablegen for

TypePrinter::printAttributedAfter too, but decided against it.
RegularKeyword is really a spelling-level classification rather
than an attribute-level classification, and in general, an attribute
could have both GNU and RegularKeyword spellings. In contrast,
printAttributedAfter is only given the attribute kind and the type
that results from applying the attribute. AFAIK, it doesn't have
access to the original attribute spelling. This means that some
attribute-specific or type-specific knowledge might be needed
to print the attribute in the best way.

• Generating the tokens automatically from Attr.td means that

pseudo's libgrammar does now depend on tablegen.

• The patch uses the SME __arm_streaming attribute as an example

for testing purposes. The attribute does not do anything at this
stage. Later SME-specific patches will add proper semantics for it,
and add other SME-related keyword attributes.

[erichkeane]

clang/include/clang/Basic/TokenKinds.h

106

Does AttrTokenKinds.inc not do this undef for you? Most of our tablegen'ed/.td files tend to...

Edit now that I've gone through the whole patch:

I see the inclusion into TokenKinds.def makes adding one a pain, but I suggest instead making these be KEYWORD_ATTRIBUTE(...), with all the 'fixings' that comes in TokenKinds.def (see the defines at the top), and then including TokenKinds.def here instead.

[rsandifo-arm]

Make AttrTokenKinds.inc use a new KEYWORD_ATTRIBUTE macro, rather than
using KEYWORD directly. Move the #undef to the .inc file.

[rsandifo-arm]

clang/include/clang/Basic/TokenKinds.h

106

Ah, yeah, that's neater, thanks. It also means that the KEYALL can be moved to TokenKinds.def until such time (hopefully never) as it becomes necessary to vary it based on attribute.

Done in the updated version.

[erichkeane]

Generally ok with all of this, but want some time to think about it/give aaron/etc time to see it.

clang/include/clang/Basic/TokenKinds.def

752

Probably should be: #ifndef KEYWORD_ATTRIBUTE here.

[rsandifo-arm]

Add #ifndef guard.

[rsandifo-arm]

clang/include/clang/Basic/TokenKinds.def

752

Oops, yes, sorry.

[rsandifo-arm]

In D148700#4280833, @erichkeane wrote:

Generally ok with all of this, but want some time to think about it/give aaron/etc time to see it.

Hi @erichkeane & @aaron.ballman Thanks for the reviews so far. Do you have any more thoughts on this series?

[aaron.ballman]

clang/include/clang/Basic/Attr.td

2437–2440

I'd feel more comfortable switching an existing attribute over to use this new functionality instead of introducing a new attribute at the same time. (Also, we ask that there be no new undocumented attributes unless there's a Very Good Reason to leave it undocumented.)

clang/include/clang/Lex/Token.h

122

clang/lib/AST/TypePrinter.cpp

1760–1763

This seems like something that tablegen should automatically handle more generally for these attributes.

[rsandifo-arm]

Thanks for the review.

clang/include/clang/Basic/Attr.td

2437–2440

Could you suggest an attribute that would be suitable? The problem is that, as far as I know, no existing keyword has parse rules that exactly match the rules for standard attributes. (That, and the fact that different keywords have different rules, was one of the main motivations for the patch). So I don't think it would be possible to adopt the new scheme for existing attributes without breaking compatiblity (both ways: some things would become valid that weren't before, and some things that were valid before would become invalid).

E.g.:

__declspec

I don't think anyone was suggesting we try to convert this :-) but for completeness:

int __declspec(align(16)) a1; // OK, but standard attribute rules would say this appertains to the type
int a2 __declspec();          // Not valid, unlike for standard decl attributes

__forceinline

int __forceinline a1() { return 1; } // OK, but standard attribute rules would say this appertains to the type
int a2 __forceinline() { return 1; } // Not valid, but would be for standard decl attributes

Calling convention keywords

These are generally DeclOrType attributes, with attributes sliding from the decl to the type where necessary.

__stdcall int a1();   // OK, but appertains to the decl and relies on sliding behaviour
int a2 __stdcall();   // Not valid, but is another place to put standard decl attributes
int a3() __stdcall;   // Not valid, but is another place to put standard type attributes
int (__stdcall a4)(); // OK, but standard attributes aren't allowed in this position
extern int (*const __stdcall volatile a5) (); // OK, but standard attributes wouldn't be allowed here

Nullability keywords

Like the calling-convention keywords, the nullability keywords appertain to types but are allowed in some (but not all) positions that would normally appertain to decls:

using ptr = int *;
_Nullable ptr a1; // OK, but appertains to the decl and relies on sliding behaviour
ptr a2 _Nullable; // Not valid, but is another place to put standard decl attributes
extern int *const _Nullable volatile a3; // OK, but a standard attribute wouldn't be allowed here

The same distinction applies to Microsoft pointer attributes.

On the documentation side: I admit this is one of the awkward things about using a new keyword to prove the system. The series doesn't actually implement __arm_streaming (@sdesmalen has patches for that, and other SME stuff). So it didn't seem appropriate to document the attribute until it was actually implemented. But the implementation, when it comes, will/should have documentation.

clang/lib/AST/TypePrinter.cpp

1760–1763

I wondered about trying to use tablegen for TypePrinter::printAttributedAfter, but decided against it.
RegularKeyword is really a spelling-level classification rather than an attribute-level classification, and in general, an attribute could have both GNU and RegularKeyword spellings. In contrast, printAttributedAfter is only given the attribute kind and the type that results from applying the attribute. AFAIK, it doesn't have access to the original attribute spelling. This means that some attribute-specific or type-specific knowledge might be needed to print the attribute in the best way.

[rsandifo-arm]

Hi @aaron.ballman and @erichkeane . Do you have any more thoughts on this? In principle, I'm happy to convert an existing attribute over to the new scheme, but in practice, I can't find one that seems to be suitable. If we're going to do that, I think I'll need guidance as to which attribute to convert.

Alternatively, I could try to classify existing keywords into groups based on their current parsing rules and tablegen-ify them based on that. E.g. I could add NullabilityKeyword for _Nonnull, _Nullable, _Nullable_result, _Null_unspecified, and for anything else that happens to use exactly those parsing rules. I could then replace direct checks for the nullability keyword tokens with checks for a NullabilityKeyword spelling.

But the tablegen side wasn't really the main focus of this. It's more the principle of having an optional, mechanical link between attribute keywords and the standard attribute parsing rules.

[erichkeane]

I don't have any good suggestions unfortunately. Perhaps Aaron does? I went through our list as well, and don't believe I see a good candidate. While I'm generally against an undocumented attribute, I'd be OK with either an immediate follow-up patch that documents it with an implementation, or, more acceptably: just a note at the top of the document that says something like, "this is not yet implemented, but will SOON do the following" at the top, which gets removed when implemented (it could perhaps also be a "This is a WIP" disclaimer as well).

[rsandifo-arm]

Add documentation. Fix a comment cut-&-pasto that Aaron pointed out.

[rsandifo-arm]

Thanks @erichkeane . Adding the documentation with that kind of disclaimer sounds good to me. I've done that in the updated version, and also fixed the comment problem that Aaron pointed out.

[rsandifo-arm]

Hi @erichkeane and @aaron.ballman. Does the updated patch look OK?

[aaron.ballman]

This basically LGTM, but we should add documentation to the internals manual too: https://clang.llvm.org/docs/InternalsManual.html#spellings

clang/lib/AST/TypePrinter.cpp

1760–1763

Yeah, we've run into this problem a number of times. Ultimately, we should retain what spelling the user used because that's salient information for the AST (for example, doing AST matching looking for an AlignedAttr attribute where the spelling used matters for the intended semantics. But that's not something you need to address.

clang/lib/Sema/SemaDeclAttr.cpp

5332

[rsandifo-arm]

Add internals documentation. Add FIXME to temporary code.

[rsandifo-arm]

In D148700#4364229, @aaron.ballman wrote:

This basically LGTM, but we should add documentation to the internals manual too: https://clang.llvm.org/docs/InternalsManual.html#spellings

Oops, hadn't seen that documentation. Added in the updated patch, thanks!

[aaron.ballman]

LGTM! Thank you for your patience while we worked through the design details. :-)

[rsandifo-arm]

Thanks @aaron.ballman and @erichkeane for your patience in reviewing the patch and steering me in the right direction.

What do you think about the other two patches in the series:

• https://reviews.llvm.org/D148699 (comes before this one)
• https://reviews.llvm.org/D148702 (comes after this one)

The first one is pretty mechanical. The second is going to be a slog to review, sorry. It's one of those things that's much easier to write than to check afterwards. I've tried to capture all the potentially non-obvious/non-mechanical parts in the covering note, but a long covering note might just make the review even more painful. I'm happy to try presenting it in a different form if you can think of one that would help.

Thanks again for accepting this patch, really appreciate it.

[aaron.ballman]

In D148700#4371518, @rsandifo-arm wrote:

Thanks @aaron.ballman and @erichkeane for your patience in reviewing the patch and steering me in the right direction.

What do you think about the other two patches in the series:

• https://reviews.llvm.org/D148699 (comes before this one)
• https://reviews.llvm.org/D148702 (comes after this one)

The first one is pretty mechanical. The second is going to be a slog to review, sorry. It's one of those things that's much easier to write than to check afterwards. I've tried to capture all the potentially non-obvious/non-mechanical parts in the covering note, but a long covering note might just make the review even more painful. I'm happy to try presenting it in a different form if you can think of one that would help.

Thanks again for accepting this patch, really appreciate it.

Thank you for the reminder about the other patches in the stack, I had lost track of those. I was able to review the first one today, but I don't think I'll have the bandwidth to get the second one done today (I'm about to head out for a ~week's vacation) so that one may take a bit longer.

[thakis]

clang-tools-extra/pseudo/lib/grammar/CMakeLists.txt

5

Did whoever wrote this comment sign up its removal? Looks like it was @sammccall in https://github.com/llvm/llvm-project/commit/dc63ad8878de6d0b5dc1268691f48035e9234763

[rsandifo-arm]

clang-tools-extra/pseudo/lib/grammar/CMakeLists.txt

5

No, sorry, I should have checked.

The type referenced in the comment (tok::TokenKind) is the one that is now being partially generated by tablegen. Given that, there didn't seem any way of avoiding the dependency.

[jyknight]

This makes `CXX11 and C2x` spellings
unsuitable for attributes that affect the type system, that change the
binary interface of the code, or that have other similar semantic meaning.

Yes, standard attributes aren't supposed to be used for things which affect the type system (although, we certainly have many, already, which do, since we expose most GCC-syntax attributes also as C++-standard attribute syntax!)

But I think the C++ standard [[no_unique_address]] attribute is ample precedent that standard-attributes may validly affect ABI. [[no_unique_address]] may indeed be ignored without changing the validity of a program -- but it does change the ABI. So, if you care for ABI compatibility between compilers, all the compilers you're using need to agree on whether they ignore it or not.

[rsmith]

In D148700#4401353, @jyknight wrote:

Yes, standard attributes aren't supposed to be used for things which affect the type system (although, we certainly have many, already, which do, since we expose most GCC-syntax attributes also as C++-standard attribute syntax!)

The rule that standard attributes don't affect semantics only applies to attributes specified by the language standard. There is no expectation that vendor attributes avoid such effects. In particular, I'm concerned by this in the description of this change:

In D148700, @rsandifo-arm wrote:

Really, the “only” thing wrong with using standard attributes is that standard attributes cannot affect semantics.

If the only reason for this patch series is an idea that vendor attributes using [[...]] syntax can't affect program semantics, then I think this change is not justified, because vendor attributes using [[...]] syntax can and usually do affect program semantics. But the documentation change here makes the point that using a keyword attribute may be as good idea in cases where you would always want compilation to fail on a compiler that doesn't understand the annotation, rather than the annotation being ignored (likely with a warning), so maybe that's reasonable justification for this direction.

[rsandifo-arm]

In D148700#4401451, @rsmith wrote:

In D148700#4401353, @jyknight wrote:

Yes, standard attributes aren't supposed to be used for things which affect the type system (although, we certainly have many, already, which do, since we expose most GCC-syntax attributes also as C++-standard attribute syntax!)

The rule that standard attributes don't affect semantics only applies to attributes specified by the language standard. There is no expectation that vendor attributes avoid such effects.

Ah. Does that mean that, when the standard says (sorry for the direct latex quote):

For an \grammarterm{attribute-token} (including an \grammarterm{attribute-scoped-token}) not specified in this document, the behavior is \impldef{behavior of non-standard attributes}; any such \grammarterm{attribute-token} that is not recognized by the implementation is ignored.

the “is ignored” rule only applies to unscoped attributes and to attributes in one of the std:: namespaces? I.e. to things that could reasonably be standardised in future, rather than to vendor attributes?

In particular, I'm concerned by this in the description of this change:

In D148700, @rsandifo-arm wrote:

Really, the “only” thing wrong with using standard attributes is that standard attributes cannot affect semantics.

If the only reason for this patch series is an idea that vendor attributes using [[...]] syntax can't affect program semantics, then I think this change is not justified, because vendor attributes using [[...]] syntax can and usually do affect program semantics. But the documentation change here makes the point that using a keyword attribute may be as good idea in cases where you would always want compilation to fail on a compiler that doesn't understand the annotation, rather than the annotation being ignored (likely with a warning), so maybe that's reasonable justification for this direction.

FWIW, the original plan had been to use standard attribute syntax with the arm vendor namespace. We started out with things like [[arm::streaming]], but then a colleague pointed out that standard attributes aren't supposed to affect semantics. So then we switched to GNU attributes, which have long included things that can't be ignored (such as vector_size). But there was pushback against that because even unrecognised GNU attributes only generate a warning. (I think GCC made a mistake by establishing that unrecognised GNU attributes are only a warning.)

If using standard attributes with a vendor namespace is acceptable for things that affect semantics and ABI, then that would actually be more convenient, for a few reasons:

• It would provide proper scoping (via namespaces), rather than the clunky __arm_ prefixes. E.g. some of the ACLE intrinsics have __arm_streaming __arm_shared_za __arm_preserves_za, which is quite a mouthful :)

• It would allow other attribute-related features to be used, such as #pragma clang attribute

• It would allow attributes to take arguments, without any compatibility concerns. Keyword attributes could support arguments, but the decision about whether an attribute takes arguments would probably have to be made when the attribute is first added. Trying to retrofit arguments to a keyword attribute that didn't previously take arguments could lead to parsing ambiguities. In contrast, the standard attribute syntax would allow optional arguments to be added later without any backward compatibility concerns.

The main drawback would still be that unrecognised attributes only generate a warning. But if vendor attributes are allowed to affect semantics, we could “solve” the warning problem for current and future compilers by reporting an error for unrecognised arm:: attributes. That wouldn't help with older compilers, but TBH, I think older compilers would reject any practical use of SME attributes anyway. E.g. the attributes would almost always (perhaps always) be used in conjunction with the arm_sme.h header, which older compilers don't provide. We also provide a feature preprocessor macro that careful code can check.

So if using an arm namespace is acceptable, if the current line about what can affect semantics is likely to become more fuzzy, and if there's a risk that keyword attributes are a dead end that no-one else adopts, then TBH I'd still be in favour of using arm namespaces for SME.

[aaron.ballman]

In D148700#4401451, @rsmith wrote:

In D148700#4401353, @jyknight wrote:

Yes, standard attributes aren't supposed to be used for things which affect the type system (although, we certainly have many, already, which do, since we expose most GCC-syntax attributes also as C++-standard attribute syntax!)

The rule that standard attributes don't affect semantics only applies to attributes specified by the language standard. There is no expectation that vendor attributes avoid such effects. In particular, I'm concerned by this in the description of this change:

In D148700, @rsandifo-arm wrote:

Really, the “only” thing wrong with using standard attributes is that standard attributes cannot affect semantics.

If the only reason for this patch series is an idea that vendor attributes using [[...]] syntax can't affect program semantics, then I think this change is not justified, because vendor attributes using [[...]] syntax can and usually do affect program semantics. But the documentation change here makes the point that using a keyword attribute may be as good idea in cases where you would always want compilation to fail on a compiler that doesn't understand the annotation, rather than the annotation being ignored (likely with a warning), so maybe that's reasonable justification for this direction.

That is the reason for this direction -- we have enough instances where people want to add a vendor attribute but it is not safe for other implementations to ignore it (due to ABI, correctness, etc). This feature allows adding the attribute as a keyword which can appear anywhere the attribute can appear. Due to it being expressed as a keyword in source, the other implementations will fail to accept the code, whereas use of a vendor-specified attribute would be (potentially silently) accepted with ill-effects.

[rsandifo-arm]

Hi @jyknight , @rsmith

Do you have any more thoughts on the above? Quick version is:

1. Is it OK to have [[…]] attributes in the arm namespace that affect semantics?
2. Is it OK to raise an error for unrecognised attributes in the arm namespace (for a measure of future-proofing)?

Given the differing views, I'm unsure whether to revert the series and do (1) (and possibly (2)), or whether to leave things as they are.

Thanks!

[jyknight]

In D148700#4418767, @rsandifo-arm wrote:

Hi @jyknight , @rsmith

Do you have any more thoughts on the above? Quick version is:

1. Is it OK to have [[…]] attributes in the arm namespace that affect semantics?

I'd say the consensus is that it is.

2. Is it OK to raise an error for unrecognised attributes in the arm namespace (for a measure of future-proofing)?

We already have the -Wunknown-attributes warning enabled by default (as a warning). Is it vital for it to be a default-on error (for arm::*), instead of a default-on warning? ISTM that the default-on warning ought to suffice, but I'm happy to hear people's experience of this going badly in their experience. :)

Given the differing views, I'm unsure whether to revert the series and do (1) (and possibly (2)), or whether to leave things as they are.

I don't really feel strongly about the syntax chosen, but given that you've mentioned a fair number of upsides to using normal [[arm::...]] attributes, I'd say it may indeed be worthwhile to go back to that.

[aaron.ballman]

In D148700#4418767, @rsandifo-arm wrote:

Hi @jyknight , @rsmith

Do you have any more thoughts on the above? Quick version is:

1. Is it OK to have [[…]] attributes in the arm namespace that affect semantics?

Yes, any attribute in a vendor namespace can do anything it wants, including impacting semantics.

1. Is it OK to raise an error for unrecognised attributes in the arm namespace (for a measure of future-proofing)?

Yes, part of "anything it wants" includes diagnostic behavior.

Given the differing views, I'm unsure whether to revert the series and do (1) (and possibly (2)), or whether to leave things as they are.

The reason why we suggested keywords isn't because of concerns with how *Clang* implements the attribute, it's about portability of the code between Clang and other compilers that don't implement the attribute. Compilers do not issue an error diagnostic on unrecognized attributes in a vendor namespace (per spec they're defined to be ignored: https://eel.is/c++draft/dcl.attr#grammar-6.sentence-1 so if you're lucky, you'll get a warning about the attribute being unknown, but there's no guarantee: https://godbolt.org/z/8qaqPzYYG). So when the attribute impacts semantics in such a way that silently ignoring the attribute would lead to Very Bad Outcomes (think: crashes, ABI mismatches, security concerns, etc), using a keyword for the attribute is a better user experience because unknown keywords are not ignored by implementations (they get diagnosed as a syntax error).

clang-tools-extra/pseudo/lib/grammar/CMakeLists.txt

 set(LLVM_LINK_COMPONENTS Support)
 
-# This library is used by the clang-pseudo-gen tool which runs at build time.
-# Dependencies should be minimal to avoid long dep paths in the build graph.
-# It does use clangBasic headers (tok::TokenKind), but linking is not needed.
-# We have no transitive dependencies on tablegen files.
-list(REMOVE_ITEM LLVM_COMMON_DEPENDS clang-tablegen-targets)
 add_clang_library(clangPseudoGrammar
   Grammar.cpp
   GrammarBNF.cpp
   LRGraph.cpp
   LRTable.cpp
   LRTableBuild.cpp
   )
 

clang/docs/InternalsManual.rst

 ^^^^^^^^^^^^^^^^
 Attributes in Clang are handled in three stages: parsing into a parsed attribute
 representation, conversion from a parsed attribute into a semantic attribute,
 and then the semantic handling of the attribute.
 
 Parsing of the attribute is determined by the various syntactic forms attributes
 can take, such as GNU, C++11, and Microsoft style attributes, as well as other
 information provided by the table definition of the attribute. Ultimately, the
-parsed representation of an attribute object is an ``ParsedAttr`` object.
+parsed representation of an attribute object is a ``ParsedAttr`` object.
 These parsed attributes chain together as a list of parsed attributes attached
 to a declarator or declaration specifier. The parsing of attributes is handled
-automatically by Clang, except for attributes spelled as keywords. When
-implementing a keyword attribute, the parsing of the keyword and creation of the
-``ParsedAttr`` object must be done manually.
+automatically by Clang, except for attributes spelled as so-called “custom”
+keywords. When implementing a custom keyword attribute, the parsing of the
+keyword and creation of the ``ParsedAttr`` object must be done manually.
 
 Eventually, ``Sema::ProcessDeclAttributeList()`` is called with a ``Decl`` and
 a ``ParsedAttr``, at which point the parsed attribute can be transformed
 into a semantic attribute. The process by which a parsed attribute is converted
 into a semantic attribute depends on the attribute definition and semantic
 requirements of the attribute. The end result, however, is that the semantic
 attribute object is attached to the ``Decl`` object, and can be obtained by a
 call to ``Decl::getAttr<T>()``. Similarly, for statement attributes,

 Spellings
 ~~~~~~~~~
 All attributes are required to specify a spelling list that denotes the ways in
 which the attribute can be spelled. For instance, a single semantic attribute
 may have a keyword spelling, as well as a C++11 spelling and a GNU spelling. An
 empty spelling list is also permissible and may be useful for attributes which
 are created implicitly. The following spellings are accepted:
 
-  ============  ================================================================
-  Spelling      Description
-  ============  ================================================================
-  ``GNU``       Spelled with a GNU-style ``__attribute__((attr))`` syntax and
-                placement.
-  ``CXX11``     Spelled with a C++-style ``[[attr]]`` syntax with an optional
-                vendor-specific namespace.
-  ``C2x``       Spelled with a C-style ``[[attr]]`` syntax with an optional
-                vendor-specific namespace.
-  ``Declspec``  Spelled with a Microsoft-style ``__declspec(attr)`` syntax.
-  ``Keyword``   The attribute is spelled as a keyword, and required custom
-                parsing.
-  ``GCC``       Specifies two or three spellings: the first is a GNU-style
-                spelling, the second is a C++-style spelling with the ``gnu``
-                namespace, and the third is an optional C-style spelling with
-                the ``gnu`` namespace. Attributes should only specify this
-                spelling for attributes supported by GCC.
-  ``Clang``     Specifies two or three spellings: the first is a GNU-style
-                spelling, the second is a C++-style spelling with the ``clang``
-                namespace, and the third is an optional C-style spelling with
-                the ``clang`` namespace. By default, a C-style spelling is
-                provided.
-  ``Pragma``    The attribute is spelled as a ``#pragma``, and requires custom
-                processing within the preprocessor. If the attribute is meant to
-                be used by Clang, it should set the namespace to ``"clang"``.
-                Note that this spelling is not used for declaration attributes.
-  ============  ================================================================
+  ==================  =========================================================
+  Spelling            Description
+  ==================  =========================================================
+  ``GNU``             Spelled with a GNU-style ``__attribute__((attr))``
+                      syntax and placement.
+  ``CXX11``           Spelled with a C++-style ``[[attr]]`` syntax with an
+                      optional vendor-specific namespace.
+  ``C2x``             Spelled with a C-style ``[[attr]]`` syntax with an
+                      optional vendor-specific namespace.
+  ``Declspec``        Spelled with a Microsoft-style ``__declspec(attr)``
+                      syntax.
+  ``CustomKeyword``   The attribute is spelled as a keyword, and requires
+                      custom parsing.
+  ``RegularKeyword``  The attribute is spelled as a keyword. It can be
+                      used in exactly the places that the standard
+                      ``[[attr]]`` syntax can be used, and appertains to
+                      exactly the same thing that a standard attribute
+                      would appertain to. Lexing and parsing of the keyword
+                      are handled automatically.
+  ``GCC``             Specifies two or three spellings: the first is a
+                      GNU-style spelling, the second is a C++-style spelling
+                      with the ``gnu`` namespace, and the third is an optional
+                      C-style spelling with the ``gnu`` namespace. Attributes
+                      should only specify this spelling for attributes
+                      supported by GCC.
+  ``Clang``           Specifies two or three spellings: the first is a
+                      GNU-style spelling, the second is a C++-style spelling
+                      with the ``clang`` namespace, and the third is an
+                      optional C-style spelling with the ``clang`` namespace.
+                      By default, a C-style spelling is provided.
+  ``Pragma``          The attribute is spelled as a ``#pragma``, and requires
+                      custom processing within the preprocessor. If the
+                      attribute is meant to be used by Clang, it should
+                      set the namespace to ``"clang"``. Note that this
+                      spelling is not used for declaration attributes.
+  ==================  =========================================================
+
+The C++ standard specifies that “any [non-standard attribute] that is not
+recognized by the implementation is ignored” (``[dcl.attr.grammar]``).
+The rule for C is similar. This makes ``CXX11`` and ``C2x`` spellings
+unsuitable for attributes that affect the type system, that change the
+binary interface of the code, or that have other similar semantic meaning.
+
+``RegularKeyword`` provides an alternative way of spelling such attributes.
+It reuses the production rules for standard attributes, but it applies them
+to plain keywords rather than to ``[[…]]`` sequences. Compilers that don't
+recognize the keyword are likely to report an error of some kind.
+
+For example, the ``ArmStreaming`` function type attribute affects
+both the type system and the binary interface of the function.
+It cannot therefore be spelled ``[[arm::streaming]]``, since compilers
+that don't understand ``arm::streaming`` would ignore it and miscompile
+the code. ``ArmStreaming`` is instead spelled ``__arm_streaming``, but it
+can appear wherever a hypothetical ``[[arm::streaming]]`` could appear.
 
 Subjects
 ~~~~~~~~
 Attributes appertain to one or more subjects. If the attribute attempts to
 attach to a subject that is not in the subject list, a diagnostic is issued
 automatically. Whether the diagnostic is a warning or an error depends on how
 the attribute's ``SubjectList`` is defined, but the default behavior is to warn.
 The diagnostics displayed to the user are automatically determined based on the

clang/include/clang/Basic/Attr.td

class C2x<string namespace, string name, int version = 1>
     : Spelling<name, "C2x", version> {
   string Namespace = namespace;
 }
 
 class Keyword<string name, bit hasOwnParseRules>
     : Spelling<name, "Keyword"> {
   bit HasOwnParseRules = hasOwnParseRules;
 }
+
+// A keyword that can appear wherever a standard attribute can appear,
+// and that appertains to whatever a standard attribute would appertain to.
+// This is useful for things that affect semantics but that should otherwise
+// be treated like standard attributes.
+class RegularKeyword<string name> : Keyword<name, 0> {}
+
 // A keyword that has its own individual parsing rules.
 class CustomKeyword<string name> : Keyword<name, 1> {}
 
 class Pragma<string namespace, string name> : Spelling<name, "Pragma"> {
   string Namespace = namespace;
 }
 
 // The GCC spelling implies GNU<name>, CXX11<"gnu", name>, and optionally,
def AArch64VectorPcs: DeclOrTypeAttr {
   let Documentation = [AArch64VectorPcsDocs];
 }
 
 def AArch64SVEPcs: DeclOrTypeAttr {
   let Spellings = [Clang<"aarch64_sve_pcs">];
   let Documentation = [AArch64SVEPcsDocs];
 }
 
+def ArmStreaming : TypeAttr, TargetSpecificAttr<TargetAArch64> {
+  let Spellings = [RegularKeyword<"__arm_streaming">];
+  let Documentation = [ArmStreamingDocs];
+}
+
 def Pure : InheritableAttr {
   let Spellings = [GCC<"pure">];
   let Documentation = [Undocumented];
   let SimpleHandler = 1;
 }
 
 def Regparm : TypeAttr {
   let Spellings = [GCC<"regparm">];

clang/include/clang/Basic/AttrDocs.td

 from secure state. Entering from and returning to non-secure state would switch
 to and from secure state, respectively, and prevent flow of information
 to non-secure state, except via return values. See `ARMv8-M Security Extensions:
 Requirements on Development Tools - Engineering Specification Documentation
 <https://developer.arm.com/docs/ecm0359818/latest/>`_ for more information.
   }];
 }
 
+def ArmStreamingDocs : Documentation {
+  let Category = DocCatType;
+  let Content = [{
+.. Note:: This attribute has not been implemented yet, but once it is
+   implemented, it will behave as described below.
+
+The ``__arm_streaming`` keyword is only available on AArch64 targets.
+It applies to function types and specifies that the function has a
+“streaming interface”.  This means that:
+
+* the function requires the Scalable Matrix Extension (SME)
+
+* the function must be entered in streaming mode (that is, with PSTATE.SM
+  set to 1)
+
+* the function must return in streaming mode
+
+See `Procedure Call Standard for the Arm® 64-bit Architecture (AArch64)
+<https://github.com/ARM-software/abi-aa>`_ for more details about
+streaming-interface functions.
+
+Clang manages PSTATE.SM automatically; it is not the source code's
+responsibility to do this.  For example, if a normal non-streaming
+function calls an ``__arm_streaming`` function, Clang generates code
+that switches into streaming mode before calling the function and
+switches back to non-streaming mode on return.
+
+``__arm_streaming`` can appear anywhere that a standard ``[[…]]`` type
+attribute can appear.
+
+See `Arm C Language Extensions <https://github.com/ARM-software/acle>`_
+for more details about this extension, and for other related SME features.
+  }];
+}
+
 def AlwaysInlineDocs : Documentation {
   let Category = DocCatFunction;
   let Content = [{
 Inlining heuristics are disabled and inlining is always attempted regardless of
 optimization level.
 
 ``[[clang::always_inline]]`` spelling can be used as a statement attribute; other
 spellings of the attribute are not supported on statements. If a statement is

clang/include/clang/Basic/AttributeCommonInfo.h

private:
   SourceRange AttrRange;
   const SourceLocation ScopeLoc;
   // Corresponds to the Kind enum.
   unsigned AttrKind : 16;
   /// Corresponds to the Syntax enum.
   unsigned SyntaxUsed : 4;
   unsigned SpellingIndex : 4;
   unsigned IsAlignas : 1;
+  unsigned IsRegularKeywordAttribute : 1;
 
 protected:
   static constexpr unsigned SpellingNotCalculated = 0xf;
 
 public:
   /// Combines information about the source-code form of an attribute,
   /// including its syntax and spelling.
   class Form {
   public:
-    constexpr Form(Syntax SyntaxUsed, unsigned SpellingIndex, bool IsAlignas)
+    constexpr Form(Syntax SyntaxUsed, unsigned SpellingIndex, bool IsAlignas,
+                   bool IsRegularKeywordAttribute)
         : SyntaxUsed(SyntaxUsed), SpellingIndex(SpellingIndex),
-          IsAlignas(IsAlignas) {}
+          IsAlignas(IsAlignas),
+          IsRegularKeywordAttribute(IsRegularKeywordAttribute) {}
     constexpr Form(tok::TokenKind Tok)
         : SyntaxUsed(AS_Keyword), SpellingIndex(SpellingNotCalculated),
-          IsAlignas(Tok == tok::kw_alignas) {}
+          IsAlignas(Tok == tok::kw_alignas),
+          IsRegularKeywordAttribute(tok::isRegularKeywordAttribute(Tok)) {}
 
     Syntax getSyntax() const { return Syntax(SyntaxUsed); }
     unsigned getSpellingIndex() const { return SpellingIndex; }
     bool isAlignas() const { return IsAlignas; }
+    bool isRegularKeywordAttribute() const { return IsRegularKeywordAttribute; }
 
     static Form GNU() { return AS_GNU; }
     static Form CXX11() { return AS_CXX11; }
     static Form C2x() { return AS_C2x; }
     static Form Declspec() { return AS_Declspec; }
     static Form Microsoft() { return AS_Microsoft; }
-    static Form Keyword(bool IsAlignas) {
-      return Form(AS_Keyword, SpellingNotCalculated, IsAlignas);
+    static Form Keyword(bool IsAlignas, bool IsRegularKeywordAttribute) {
+      return Form(AS_Keyword, SpellingNotCalculated, IsAlignas,
+                  IsRegularKeywordAttribute);
     }
     static Form Pragma() { return AS_Pragma; }
     static Form ContextSensitiveKeyword() { return AS_ContextSensitiveKeyword; }
     static Form HLSLSemantic() { return AS_HLSLSemantic; }
     static Form Implicit() { return AS_Implicit; }
 
   private:
     constexpr Form(Syntax SyntaxUsed)
         : SyntaxUsed(SyntaxUsed), SpellingIndex(SpellingNotCalculated),
-          IsAlignas(0) {}
+          IsAlignas(0), IsRegularKeywordAttribute(0) {}
 
     unsigned SyntaxUsed : 4;
     unsigned SpellingIndex : 4;
     unsigned IsAlignas : 1;
+    unsigned IsRegularKeywordAttribute : 1;
   };
 
   AttributeCommonInfo(const IdentifierInfo *AttrName,
                       const IdentifierInfo *ScopeName, SourceRange AttrRange,
                       SourceLocation ScopeLoc, Kind AttrKind, Form FormUsed)
       : AttrName(AttrName), ScopeName(ScopeName), AttrRange(AttrRange),
         ScopeLoc(ScopeLoc), AttrKind(AttrKind),
         SyntaxUsed(FormUsed.getSyntax()),
         SpellingIndex(FormUsed.getSpellingIndex()),
-        IsAlignas(FormUsed.isAlignas()) {
+        IsAlignas(FormUsed.isAlignas()),
+        IsRegularKeywordAttribute(FormUsed.isRegularKeywordAttribute()) {
     assert(SyntaxUsed >= AS_GNU && SyntaxUsed <= AS_Implicit &&
            "Invalid syntax!");
   }
 
   AttributeCommonInfo(const IdentifierInfo *AttrName,
                       const IdentifierInfo *ScopeName, SourceRange AttrRange,
                       SourceLocation ScopeLoc, Form FormUsed)
       : AttributeCommonInfo(
AttributeCommonInfo(SourceRange AttrRange, Kind K, Form FormUsed)
       : AttributeCommonInfo(nullptr, nullptr, AttrRange, SourceLocation(), K,
                             FormUsed) {}
 
   AttributeCommonInfo(AttributeCommonInfo &&) = default;
   AttributeCommonInfo(const AttributeCommonInfo &) = default;
 
   Kind getParsedKind() const { return Kind(AttrKind); }
   Syntax getSyntax() const { return Syntax(SyntaxUsed); }
-  Form getForm() const { return Form(getSyntax(), SpellingIndex, IsAlignas); }
+  Form getForm() const {
+    return Form(getSyntax(), SpellingIndex, IsAlignas,
+                IsRegularKeywordAttribute);
+  }
   const IdentifierInfo *getAttrName() const { return AttrName; }
   SourceLocation getLoc() const { return AttrRange.getBegin(); }
   SourceRange getRange() const { return AttrRange; }
   void setRange(SourceRange R) { AttrRange = R; }
 
   bool hasScope() const { return ScopeName; }
   const IdentifierInfo *getScopeName() const { return ScopeName; }
   SourceLocation getScopeLoc() const { return ScopeLoc; }
public:
   }
 
   bool isGNUAttribute() const { return SyntaxUsed == AS_GNU; }
 
   bool isKeywordAttribute() const {
     return SyntaxUsed == AS_Keyword || SyntaxUsed == AS_ContextSensitiveKeyword;
   }
 
+  bool isRegularKeywordAttribute() const { return IsRegularKeywordAttribute; }
+
   bool isContextSensitiveKeywordAttribute() const {
     return SyntaxUsed == AS_ContextSensitiveKeyword;
   }
 
   unsigned getAttributeSpellingListIndex() const {
     assert((isAttributeSpellingListCalculated() || AttrName) &&
            "Spelling cannot be found");
     return isAttributeSpellingListCalculated()

clang/include/clang/Basic/CMakeLists.txt

clang_tablegen(AttrList.inc -gen-clang-attr-list
   SOURCE Attr.td
   TARGET ClangAttrList)
 
 clang_tablegen(AttrSubMatchRulesList.inc -gen-clang-attr-subject-match-rule-list
   -I ${CMAKE_CURRENT_SOURCE_DIR}/../../
   SOURCE Attr.td
   TARGET ClangAttrSubjectMatchRuleList)
 
+clang_tablegen(AttrTokenKinds.inc -gen-clang-attr-token-kinds
+  -I ${CMAKE_CURRENT_SOURCE_DIR}/../../
+  SOURCE Attr.td
+  TARGET ClangAttrTokenKinds
+  )
+
 clang_tablegen(AttrHasAttributeImpl.inc -gen-clang-attr-has-attribute-impl
   -I ${CMAKE_CURRENT_SOURCE_DIR}/../../
   SOURCE Attr.td
   TARGET ClangAttrHasAttributeImpl
   )
 
 # ARM NEON and MVE
 clang_tablegen(arm_neon.inc -gen-arm-neon-sema

clang/include/clang/Basic/TokenKinds.h

 }
 
 /// Return true if this is any of tok::annot_* kinds.
 bool isAnnotation(TokenKind K);
 
 /// Return true if this is an annotation token representing a pragma.
 bool isPragmaAnnotation(TokenKind K);
 
+inline constexpr bool isRegularKeywordAttribute(TokenKind K) {
+  return (false
+#define KEYWORD_ATTRIBUTE(X) || (K == tok::kw_##X)
+#include "clang/Basic/AttrTokenKinds.inc"
+  );
+}
+
 } // end namespace tok
 } // end namespace clang
 
 namespace llvm {
 template <> struct DenseMapInfo<clang::tok::PPKeywordKind> {
   static inline clang::tok::PPKeywordKind getEmptyKey() {
     return clang::tok::PPKeywordKind::pp_not_keyword;
   }

clang/include/clang/Basic/TokenKinds.def

 // Clang Extensions.
 KEYWORD(__builtin_convertvector          , KEYALL)
 ALIAS("__char16_t"   , char16_t          , KEYCXX)
 ALIAS("__char32_t"   , char32_t          , KEYCXX)
 KEYWORD(__builtin_bit_cast               , KEYALL)
 KEYWORD(__builtin_available              , KEYALL)
 KEYWORD(__builtin_sycl_unique_stable_name, KEYSYCL)
 
+// Keywords defined by Attr.td.
+#ifndef KEYWORD_ATTRIBUTE
+#define KEYWORD_ATTRIBUTE(X) KEYWORD(X, KEYALL)
+#endif
+#include "clang/Basic/AttrTokenKinds.inc"
+
 // Clang-specific keywords enabled only in testing.
 TESTING_KEYWORD(__unknown_anytype , KEYALL)
 
 
 //===----------------------------------------------------------------------===//
 // Objective-C @-preceded keywords.
 //===----------------------------------------------------------------------===//
 

clang/include/clang/Lex/Token.h

public:
 
   /// Return true if this is a "literal", like a numeric
   /// constant, string, etc.
   bool isLiteral() const {
     return tok::isLiteral(getKind());
   }
 
   /// Return true if this is any of tok::annot_* kind tokens.
-  bool isAnnotation() const {
-    return tok::isAnnotation(getKind());
+  bool isAnnotation() const { return tok::isAnnotation(getKind()); }
+
+  /// Return true if the token is a keyword that is parsed in the same
+  /// position as a standard attribute, but that has semantic meaning
+  /// and so cannot be a true attribute.
+  bool isRegularKeywordAttribute() const {
+    return tok::isRegularKeywordAttribute(getKind());
   }
 
   /// Return a source location identifier for the specified
   /// offset in the current file.
   SourceLocation getLocation() const {
     return SourceLocation::getFromRawEncoding(Loc);
   }
   unsigned getLength() const {

clang/lib/AST/TypePrinter.cpp

if (T->getAttrKind() == attr::AnnotateType) {
     // FIXME: Print the attribute arguments once we have a way to retrieve these
     // here. For the meantime, we just print `[[clang::annotate_type(...)]]`
     // without the arguments so that we know at least that we had _some_
     // annotation on the type.
     OS << " [[clang::annotate_type(...)]]";
     return;
   }
 
+  if (T->getAttrKind() == attr::ArmStreaming) {
+    OS << "__arm_streaming";
+    return;
+  }
+
   OS << " __attribute__((";
   switch (T->getAttrKind()) {
 #define TYPE_ATTR(NAME)
 #define DECL_OR_TYPE_ATTR(NAME)
 #define ATTR(NAME) case attr::NAME:
 #include "clang/Basic/AttrList.inc"
     llvm_unreachable("non-type attribute attached to type");
 
#include "clang/Basic/AttrList.inc"
   case attr::Ptr32:
   case attr::Ptr64:
   case attr::SPtr:
   case attr::UPtr:
   case attr::AddressSpace:
   case attr::CmseNSCall:
   case attr::AnnotateType:
   case attr::WebAssemblyFuncref:
+  case attr::ArmStreaming:
     llvm_unreachable("This attribute should have been handled already");
 
   case attr::NSReturnsRetained:
     OS << "ns_returns_retained";
     break;
 
   // FIXME: When Sema learns to form this AttributedType, avoid printing the
   // attribute again in printFunctionProtoAfter.

clang/lib/Sema/SemaDeclAttr.cpp

case ParsedAttr::AT_VectorCall:
     CC = CC_X86VectorCall;
     break;
   case ParsedAttr::AT_AArch64VectorPcs:
     CC = CC_AArch64VectorCall;
     break;
   case ParsedAttr::AT_AArch64SVEPcs:
     CC = CC_AArch64SVEPCS;
     break;
+  case ParsedAttr::AT_ArmStreaming:
+    CC = CC_C; // FIXME: placeholder until real SME support is added.
+    break;
   case ParsedAttr::AT_AMDGPUKernelCall:
     CC = CC_AMDGPUKernelCall;
     break;
   case ParsedAttr::AT_RegCall:
     CC = CC_X86RegCall;
     break;
   case ParsedAttr::AT_MSABI:
     CC = Context.getTargetInfo().getTriple().isOSWindows() ? CC_C :

clang/lib/Sema/SemaType.cpp

#define CALLING_CONV_ATTRS_CASELIST                                            \
   case ParsedAttr::AT_ThisCall:                                                \
   case ParsedAttr::AT_RegCall:                                                 \
   case ParsedAttr::AT_Pascal:                                                  \
   case ParsedAttr::AT_SwiftCall:                                               \
   case ParsedAttr::AT_SwiftAsyncCall:                                          \
   case ParsedAttr::AT_VectorCall:                                              \
   case ParsedAttr::AT_AArch64VectorPcs:                                        \
   case ParsedAttr::AT_AArch64SVEPcs:                                           \
+  case ParsedAttr::AT_ArmStreaming:                                            \
   case ParsedAttr::AT_AMDGPUKernelCall:                                        \
   case ParsedAttr::AT_MSABI:                                                   \
   case ParsedAttr::AT_SysVABI:                                                 \
   case ParsedAttr::AT_Pcs:                                                     \
   case ParsedAttr::AT_IntelOclBicc:                                            \
   case ParsedAttr::AT_PreserveMost:                                            \
   case ParsedAttr::AT_PreserveAll
 
auto inferPointerNullability =
 
     // If a nullability attribute is present, there's nothing to do.
     if (hasNullabilityAttr(attrs))
       return nullptr;
 
     // If we're supposed to infer nullability, do so now.
     if (inferNullability && !inferNullabilityInnerOnlyComplete) {
       ParsedAttr::Form form =
-          inferNullabilityCS ? ParsedAttr::Form::ContextSensitiveKeyword()
-                             : ParsedAttr::Form::Keyword(false /*IsAlignAs*/);
+          inferNullabilityCS
+              ? ParsedAttr::Form::ContextSensitiveKeyword()
+              : ParsedAttr::Form::Keyword(false /*IsAlignAs*/,
+                                          false /*IsRegularKeywordAttribute*/);
       ParsedAttr *nullabilityAttr = Pool.create(
           S.getNullabilityKeyword(*inferNullability), SourceRange(pointerLoc),
           nullptr, SourceLocation(), nullptr, 0, form);
 
       attrs.addAtEnd(nullabilityAttr);
 
       if (inferNullabilityCS) {
         state.getDeclarator().getMutableDeclSpec().getObjCQualifiers()
static Attr *getCCTypeAttr(ASTContext &Ctx, ParsedAttr &Attr) {
   case ParsedAttr::AT_SwiftAsyncCall:
     return createSimpleAttr<SwiftAsyncCallAttr>(Ctx, Attr);
   case ParsedAttr::AT_VectorCall:
     return createSimpleAttr<VectorCallAttr>(Ctx, Attr);
   case ParsedAttr::AT_AArch64VectorPcs:
     return createSimpleAttr<AArch64VectorPcsAttr>(Ctx, Attr);
   case ParsedAttr::AT_AArch64SVEPcs:
     return createSimpleAttr<AArch64SVEPcsAttr>(Ctx, Attr);
+  case ParsedAttr::AT_ArmStreaming:
+    return createSimpleAttr<ArmStreamingAttr>(Ctx, Attr);
   case ParsedAttr::AT_AMDGPUKernelCall:
     return createSimpleAttr<AMDGPUKernelCallAttr>(Ctx, Attr);
   case ParsedAttr::AT_Pcs: {
     // The attribute may have had a fixit applied where we treated an
     // identifier as a string literal.  The contents of the string are valid,
     // but the form may not be.
     StringRef Str;
     if (Attr.isArgExpr(0))

clang/lib/Serialization/ASTReaderDecl.cpp

Attr *ASTRecordReader::readAttr() {
   SourceRange AttrRange = Record.readSourceRange();
   SourceLocation ScopeLoc = Record.readSourceLocation();
   unsigned ParsedKind = Record.readInt();
   unsigned Syntax = Record.readInt();
   unsigned SpellingIndex = Record.readInt();
   bool IsAlignas = (ParsedKind == AttributeCommonInfo::AT_Aligned &&
                     Syntax == AttributeCommonInfo::AS_Keyword &&
                     SpellingIndex == AlignedAttr::Keyword_alignas);
+  bool IsRegularKeywordAttribute = Record.readBool();
 
-  AttributeCommonInfo Info(
-      AttrName, ScopeName, AttrRange, ScopeLoc,
+  AttributeCommonInfo Info(AttrName, ScopeName, AttrRange, ScopeLoc,
                            AttributeCommonInfo::Kind(ParsedKind),
-      {AttributeCommonInfo::Syntax(Syntax), SpellingIndex, IsAlignas});
+                           {AttributeCommonInfo::Syntax(Syntax), SpellingIndex,
+                            IsAlignas, IsRegularKeywordAttribute});
 
 #include "clang/Serialization/AttrPCHRead.inc"
 
   assert(New && "Unable to decode attribute?");
   return New;
 }
 
 /// Reads attributes from the current stream position.

clang/lib/Serialization/ASTWriter.cpp

void ASTRecordWriter::AddAttr(const Attr *A) {
 
   Record.AddIdentifierRef(A->getAttrName());
   Record.AddIdentifierRef(A->getScopeName());
   Record.AddSourceRange(A->getRange());
   Record.AddSourceLocation(A->getScopeLoc());
   Record.push_back(A->getParsedKind());
   Record.push_back(A->getSyntax());
   Record.push_back(A->getAttributeSpellingListIndexRaw());
+  Record.push_back(A->isRegularKeywordAttribute());
 
 #include "clang/Serialization/AttrPCHWrite.inc"
 }
 
 /// Emit the list of attributes to the specified record.
 void ASTRecordWriter::AddAttributes(ArrayRef<const Attr *> Attrs) {
   push_back(Attrs.size());
   for (const auto *A : Attrs)

clang/unittests/AST/AttrTest.cpp

TEST(Attr, AnnotateType) {
   {
     const VarDecl *Var = getVariableNode(AST.get(), "auto_var");
 
     AutoTypeLoc AutoTL;
     AssertAnnotatedAs(Var->getTypeSourceInfo()->getTypeLoc(), "auto", AutoTL);
   }
 }
 
+TEST(Attr, RegularKeywordAttribute) {
+  auto AST = clang::tooling::buildASTFromCode("");
+  auto &Ctx = AST->getASTContext();
+  auto Funcref = clang::WebAssemblyFuncrefAttr::CreateImplicit(Ctx);
+  EXPECT_EQ(Funcref->getSyntax(), clang::AttributeCommonInfo::AS_Keyword);
+  ASSERT_FALSE(Funcref->isRegularKeywordAttribute());
+
+  auto Streaming = clang::ArmStreamingAttr::CreateImplicit(Ctx);
+  EXPECT_EQ(Streaming->getSyntax(), clang::AttributeCommonInfo::AS_Keyword);
+  ASSERT_TRUE(Streaming->isRegularKeywordAttribute());
+}
+
 } // namespace

clang/utils/TableGen/ClangAttrEmitter.cpp

for (const auto &I : Attrs) {
 
     forEachUniqueSpelling(Attr, [&](const FlattenedSpelling &S) {
       OS << ".Case(\"" << S.name() << "\", true)\n";
     });
   }
   OS << "#endif // CLANG_ATTR_ACCEPTS_EXPR_PACK\n\n";
 }
 
+static bool isRegularKeywordAttribute(const FlattenedSpelling &S) {
+  return (S.variety() == "Keyword" &&
+          !S.getSpellingRecord().getValueAsBit("HasOwnParseRules"));
+}
+
 static void emitFormInitializer(raw_ostream &OS,
                                 const FlattenedSpelling &Spelling,
                                 StringRef SpellingIndex) {
   bool IsAlignas =
       (Spelling.variety() == "Keyword" && Spelling.name() == "alignas");
   OS << "{AttributeCommonInfo::AS_" << Spelling.variety() << ", "
      << SpellingIndex << ", " << (IsAlignas ? "true" : "false")
-     << " /*IsAlignas*/}";
+     << " /*IsAlignas*/, "
+     << (isRegularKeywordAttribute(Spelling) ? "true" : "false")
+     << " /*IsRegularKeywordAttribute*/}";
 }
 
 static void emitAttributes(RecordKeeper &Records, raw_ostream &OS,
                            bool Header) {
   std::vector<Record*> Attrs = Records.getAllDerivedDefinitions("Attr");
   ParsedAttrMap AttrMap = getParsedAttrList(Records);
 
   // Helper to print the starting character of an attribute argument. If there
for (const auto *Attr : Attrs) {
     for (const auto &S : Spellings)
       if (Variety.empty() || (Variety == S.variety() &&
                               (Scope.empty() || Scope == S.nameSpace())))
         OS << "    .Case(\"" << S.name() << "\", " << TestStr << ")\n";
   }
   OS << "    .Default(0);\n";
 }
 
+// Emits the list of tokens for regular keyword attributes.
+void EmitClangAttrTokenKinds(RecordKeeper &Records, raw_ostream &OS) {
+  emitSourceFileHeader("A list of tokens generated from the attribute"
+                       " definitions",
+                       OS);
+  // Assume for now that the same token is not used in multiple regular
+  // keyword attributes.
+  for (auto *R : Records.getAllDerivedDefinitions("Attr"))
+    for (const auto &S : GetFlattenedSpellings(*R))
+      if (isRegularKeywordAttribute(S)) {
+        if (!R->getValueAsListOfDefs("Args").empty())
+          PrintError(R->getLoc(),
+                     "RegularKeyword attributes with arguments are not "
+                     "yet supported");
+        OS << "KEYWORD_ATTRIBUTE("
+           << S.getSpellingRecord().getValueAsString("Name") << ")\n";
+      }
+  OS << "#undef KEYWORD_ATTRIBUTE\n";
+}
+
 // Emits the list of spellings for attributes.
 void EmitClangAttrHasAttrImpl(RecordKeeper &Records, raw_ostream &OS) {
   emitSourceFileHeader("Code to implement the __has_attribute logic", OS);
 
   // Separate all of the attributes out into four group: generic, C++11, GNU,
   // and declspecs. Then generate a big switch statement for each of them.
   std::vector<Record *> Attrs = Records.getAllDerivedDefinitions("Attr");
   std::vector<Record *> Declspec, Microsoft, GNU, Pragma, HLSLSemantic;

clang/utils/TableGen/TableGen.cpp

enum ActionType {
   GenClangAttrParserStringSwitches,
   GenClangAttrSubjectMatchRulesParserStringSwitches,
   GenClangAttrImpl,
   GenClangAttrList,
   GenClangAttrDocTable,
   GenClangAttrSubjectMatchRuleList,
   GenClangAttrPCHRead,
   GenClangAttrPCHWrite,
+  GenClangAttrTokenKinds,
   GenClangAttrHasAttributeImpl,
   GenClangAttrSpellingListIndex,
   GenClangAttrASTVisitor,
   GenClangAttrTemplateInstantiate,
   GenClangAttrParsedAttrList,
   GenClangAttrParsedAttrImpl,
   GenClangAttrParsedAttrKinds,
   GenClangAttrTextNodeDump,
cl::values(
                    "Generate a table of attribute documentation"),
         clEnumValN(GenClangAttrSubjectMatchRuleList,
                    "gen-clang-attr-subject-match-rule-list",
                    "Generate a clang attribute subject match rule list"),
         clEnumValN(GenClangAttrPCHRead, "gen-clang-attr-pch-read",
                    "Generate clang PCH attribute reader"),
         clEnumValN(GenClangAttrPCHWrite, "gen-clang-attr-pch-write",
                    "Generate clang PCH attribute writer"),
+        clEnumValN(GenClangAttrTokenKinds, "gen-clang-attr-token-kinds",
+                   "Generate a list of attribute-related clang tokens"),
         clEnumValN(GenClangAttrHasAttributeImpl,
                    "gen-clang-attr-has-attribute-impl",
                    "Generate a clang attribute spelling list"),
         clEnumValN(GenClangAttrSpellingListIndex,
                    "gen-clang-attr-spelling-index",
                    "Generate a clang attribute spelling index"),
         clEnumValN(GenClangAttrASTVisitor, "gen-clang-attr-ast-visitor",
                    "Generate a recursive AST visitor for clang attributes"),
case GenClangAttrSubjectMatchRuleList:
     EmitClangAttrSubjectMatchRuleList(Records, OS);
     break;
   case GenClangAttrPCHRead:
     EmitClangAttrPCHRead(Records, OS);
     break;
   case GenClangAttrPCHWrite:
     EmitClangAttrPCHWrite(Records, OS);
     break;
+  case GenClangAttrTokenKinds:
+    EmitClangAttrTokenKinds(Records, OS);
+    break;
   case GenClangAttrHasAttributeImpl:
     EmitClangAttrHasAttrImpl(Records, OS);
     break;
   case GenClangAttrSpellingListIndex:
     EmitClangAttrSpellingListIndex(Records, OS);
     break;
   case GenClangAttrASTVisitor:
     EmitClangAttrASTVisitor(Records, OS);

clang/utils/TableGen/TableGenBackends.h

void EmitClangAttrSubjectMatchRulesParserStringSwitches(
     llvm::RecordKeeper &Records, llvm::raw_ostream &OS);
 void EmitClangAttrClass(llvm::RecordKeeper &Records, llvm::raw_ostream &OS);
 void EmitClangAttrImpl(llvm::RecordKeeper &Records, llvm::raw_ostream &OS);
 void EmitClangAttrList(llvm::RecordKeeper &Records, llvm::raw_ostream &OS);
 void EmitClangAttrSubjectMatchRuleList(llvm::RecordKeeper &Records,
                                        llvm::raw_ostream &OS);
 void EmitClangAttrPCHRead(llvm::RecordKeeper &Records, llvm::raw_ostream &OS);
 void EmitClangAttrPCHWrite(llvm::RecordKeeper &Records, llvm::raw_ostream &OS);
+void EmitClangAttrTokenKinds(llvm::RecordKeeper &Records,
+                             llvm::raw_ostream &OS);
 void EmitClangAttrHasAttrImpl(llvm::RecordKeeper &Records,
                               llvm::raw_ostream &OS);
 void EmitClangAttrSpellingListIndex(llvm::RecordKeeper &Records,
                                     llvm::raw_ostream &OS);
 void EmitClangAttrASTVisitor(llvm::RecordKeeper &Records,
                              llvm::raw_ostream &OS);
 void EmitClangAttrTemplateInstantiate(llvm::RecordKeeper &Records,
                                       llvm::raw_ostream &OS);