[SVE] Auto-generate builtins and header for svld1.

[sdesmalen]

Summary

This is a first patch in a series for the SveEmitter to generate the arm_sve.h
header file and builtins.

I've tried my best to strip down this patch as best as I could, but there
are still a few changes that are not necessarily exercised by the load intrinsics
in this patch, mostly around the SVEType class which has some common logic to
represent types from a type and prototype string. I thought it didn't make
much sense to remove that from this patch and split it up.

[sdesmalen]

clang/include/clang/Basic/AArch64SVETypeFlags.h

2

I just see that this comment will need updating, as that line seems copied from SveEmitter.cpp.

[SjoerdMeijer]

Big patch, I only did a first scan, but looks very reasonable in general. Just a first round of nits and 2 questions.

clang/include/clang/Basic/arm_sve.td

121

This encoding, e.g, this is "csilUcUsUiUlhfd", is such a monstrosity. It's a very efficient encoding, but of course completely unreadable. I know there is prior art, and know that this is how it's been done, but just curious if you have given it thoughts how to do this in a normal way, a bit more c++y. I don't want to de-rail this work, but if we are adding a new emitter, perhaps now is the time to give it a thought, so was just curious.

clang/utils/TableGen/SveEmitter.cpp

1

I wanted to add the nit that SveEmiiter.cpp should perhaps be SVEEmitter.cpp, but then I saw at the bottom that MVE is spelled Mve, so perhaps this is fine then.

64

why a default of 128? Will this gives problems for SVE implementions with> 128 bits?

119

nit: for readability, perhaps don't abbreviate some of these member names?

R -> Record
BaseTS -> BaseTypeSpec
CK -> ClassKind

260

don't need this

264

just a return here

[sdesmalen]

• Renamed CK and BaseTS
• Refactored switch statementsd in SVEType::getTypeFlags()

clang/include/clang/Basic/arm_sve.td

121

Haha, its a bit of a monstrosity indeed. The only thing I can think of here would be having something like:

class TypeSpecs<list<string> val> {
  list<string> v = val;
}
def All_Int_Float_Ty : TypeSpecs<["c", "s", "i", "l", "Uc", "Us", "Ul", "h", "f", "d">;

def SVLD1 : Minst<"svld1[_{2}]", "dPc", All_Int_Float_Ty, [IsLoad]>;

But I suspect this gets a bit awkward because of the many permutations, I count more than 40. Not sure if that would really improve the readability.

clang/utils/TableGen/SveEmitter.cpp

64

SVE vectors are n x 128bits, so the 128 is scalable here.

119

Record and ClassKind are also the names of the enum though.
Perhaps I can rename CK to Class?

264

Good catch!

[SjoerdMeijer]

clang/include/clang/Basic/arm_sve.td

121

I would personally welcome any improvement here, even the smallest. But if you think it's tricky, then fair enough!

I've managed to completely ignore the MVE intrinsics work so far, but understood there were some innovations here and there (e.g. in tablegen). Probably because it is dealing with similar problems: a lot of intrinsics, some of them overloaded with different types. I'm going to have a little look now to see if there's anything we can borrow from that, or if that is unrelated....

clang/utils/TableGen/SveEmitter.cpp

64

ah, okay, fair enough, didn't realise that.

[SjoerdMeijer]

Adding @simon_tatham in case he feels wants to have a look too.

[simon_tatham]

clang/include/clang/Basic/arm_sve.td

121

In the MVE intrinsics implementation I completely avoided that entire system of string-based type specifications. There's another completely different way you can set up the types of builtins, and I used that instead.

You can declare the function for Builtins.def purposes with no type specification at all, and then you fill in its type signature using a declaration in the header file, with the unusual combination of __inline__ and no function body:

static __inline__ int32x4_t __builtin_arm_foo_bar(int16x8_t, float23x7t); // or whatever

In fact I went one step further: the user-facing names for the MVE intrinsics are declared in arm_mve.h with a special attribute indicating that they're aliases for clang builtins. And the MVE polymorphic intrinsics are done by adding __attribute__((overloadable)) to the declaration, which allows C++-style overloading based on parameter types even when compiling in C. So when the user invokes an MVE intrinsic by its polymorphic name, the compiler first does overload resolution to decide which declaration in the header file to select; then it looks at the builtin-alias attribute and discovers which internal builtin id it corresponds to; and then it can do codegen for that builtin directly, without a wrapper function in the header.

Pros of doing it this way:

• if the builtin requires some of its arguments to be compile-time constants, then you don't run into the problem that a wrapper function in the header fails to pass through the constantness. (In NEON this is worked around by making some wrapper functions be wrapper macros instead – but NEON doesn't have to deal with polymorphism.)
• declaring a builtin's type signature in the header file means that it can include definitions that the header file has created beforehand. For example, one of the arguments to the MVE vld2q family involves a small struct containing 2 or 4 vectors, and it would be tricky to get that struct type into the Builtins.def type specification before the header file can tell clang it exists.
• doing polymorphism like this, rather than making the polymorphic function be a macro expanding to something involving C11 _Generic, means the error messages are orders of magnitude more friendly when the user messes up a call. (Also it's remarkably fiddly to use _Generic in complicated cases, because of the requirement that even its untaken branches not provoke any type-checking or semantic errors.)
• I don't know of any way that the preprocessor + _Generic approach can avoid expanding its macro arguments at least twice. It can avoid evaluating twice, so that's safe in the side-effect sense, but you still have the problem that you get exponential inflation of the size of preprocessed output if calls to these macros are lexically nested too deeply.

Cons:

• you have to do all of your codegen inside the clang binary, starting from the function operands you're given, and ending up with LLVM IR. You don't get to do the tedious parts (like unpacking structs, or dereferencing pointer arguments for passed-by-reference parameters) in the wrapper function in the header, because there isn't one. I had to invent a whole system in MveEmitter to allow the IR generation to be specified in a not-too-verbose way.
• if the builtins don't have type declarations until the header is included, then users can't call them without the header file. Probably this is fine for SVE intrinsics the same way it is for MVE, where the builtins are a detail of that particular compiler's implementation and users are intended to use the compiler-independent public API. But in cases where the builtin itself was intended to be called directly by the end user (in the way that __builtin_clz is, for example), you'd probably want it to work everywhere.
• if you do polymorphism using __attribute__((overloadable)) then all the things you're overloading between have to be real functions. You can't make some of them be macros, with the extra flexibility a macro gives you. (But then, making them builtins rather than genuine functions restores some of that flexibility.)

Off the top of my head I don't know whether all these ideas can be separated from each other. It feels to me as if all the choices I made are leaning on each other and making a mutually supporting whole, and it's quite possible that if you tried to cherry-pick just one of these design decisions into an otherwise more conventional approach, it might all come crashing down. But I haven't tried it :-)

[sdesmalen]

In D75470#1907562, @SjoerdMeijer wrote:

Adding @simon_tatham in case he feels wants to have a look too.

Thanks Sjoerd! @simon_tatham and I had a chat about this offline today.

The SVE implementation now does more or less the same thing the MVE implementation; arm_sve.h also uses __attribute__((overloadable)) and __attribute__((arm_sve_alias("__builtin_..."))), the latter only to declare the overloaded intrinsics. That means we get the same benefits as Simon described.

There are a few details that are different:

• The MVE implementation *does not* use the type string in the actual Builtins.def. For example, the type string below is not "V16ScV16ScV16Sci", but rather "":


TARGET_HEADER_BUILTIN(__builtin_arm_mve_vcaddq_rot270_f16, "", "n", "arm_mve.h", ALL_LANGUAGES, "")

• This means that the implementation relies solely on the declaration in arm_mve.h to define the intrinsic prototype.
• If I understood this correctly, this is currently needed to represent the tuple types (which cannot yet be expressed in the bulitin type string format) returned from structured loads like ld2/ld3/ld4.

The SVE implementation *does* use the type string in Builtins.def.

• For SVE, the structured loads will just return a wider vector with 2, 3 or 4 times the number of elements, so the lack of support in the builtin type-string format is not an issue.
• An advantage of that is that we can use a #define svadd_u8(...) __builtin_svadd_u8(...) for the non-overloaded builtins which helps compile-time performance.

If all the intrinsics can be described *with* a builtin type string, we can actually work to define all builtins internally in Clang (as @efriedma suggested in D75298) rather than having to rely on the header file to define the prototypes, hopefully get rid of the expensive header file.
When we do this for SVE, MVE can probably follow the same approach.

clang/include/clang/Basic/arm_sve.td

121

Thanks for sharing some background here @simon_tatham!

[miyuki]

In D75470#1910071, @sdesmalen wrote:

The SVE implementation now does more or less the same thing the MVE implementation; arm_sve.h also uses __attribute__((overloadable)) and __attribute__((arm_sve_alias("__builtin_..."))), the latter only to declare the overloaded intrinsics. That means we get the same benefits as Simon described.

FYI: we are currently working on another M-profile extension, CDE. In my patches am reusing some of the MVE intrinsic machinery, and I decided to rename __clang_arm_mve_alias to __clang_arm_builtin_alias. You might want to reuse the same attribute name to reduce duplication, see https://reviews.llvm.org/D75850

[SjoerdMeijer]

clang/lib/CodeGen/CGBuiltin.cpp

5296

I am wondering if it is confusing/correct to use NeonInstrinsicInfo here?

7473

and the same here: findNeon...

[sdesmalen]

s/NeonIntrinsicInfo/ARMVectorIntrinsicInfo/
s/findNeonIntrinsicInMap/findARMVectorIntrinsicInMap/

[sdesmalen]

clang/lib/CodeGen/CGBuiltin.cpp

5296

We can reuse the same info-struct and find function, but the names are indeed misleading. I've renamed these.

[SjoerdMeijer]

Cheers, I think this looks very reasonable.

[thakis]

clang/include/clang/Basic/CMakeLists.txt

44

Update comment to also say "SVE" and "CDE" (or just say "# ARM builtin headers")

clang/utils/TableGen/TableGen.cpp

196

Any reason these aren't called -gen-arm-sve-builtin-def and -gen-arm-sve-builtin-codegen for consistency with CDE and MVE?

[thakis]

clang/utils/TableGen/SveEmitter.cpp

32

Including stuff from clang/Basic in clang/utils/TableGen is conceptually a layering violation: clang-tblgen is used to generate headers included in clang/Basic. In this case it happens to work, but it's because you're lucky, and it could break for subtle reasons if the TypeFlags header starts including some other header in Basic that happens to include something generated.

Please restructure this so that the TableGen code doesn't need an include from Basic.

[sdesmalen]

clang/utils/TableGen/SveEmitter.cpp

32

Thanks for pointing out! The only directory that seems to have common includes between Clang TableGen/CodeGen is the llvm/Support directory, any objection to me moving the header there?

clang/utils/TableGen/TableGen.cpp

196

Not really, I can change that.

[teemperor]

This patch broke the Clang build with enabled modules (which is used by the LLDB bot, but every other bot is also dead: http://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/ )

[thakis]

clang/utils/TableGen/SveEmitter.cpp

32

That seems like a strange place for this header.

Maybe you can rework things so that you don't have to share a header between clang's tablegen and clang's Basic? No other tablegen output so far has needed that. (see e.g. the /// These must be kept in sync with the flags in utils/TableGen/NeonEmitter.h. line in TargetBuiltins.h).

If that isn't possible at all, I suppose you could put the .h file in clang/utils/TableGen and also make clang-tblgen write the .h file and use the written .h file in Basic.

[thakis]

clang/utils/TableGen/TableGen.cpp

196

Looks like you renamed the files to be consistent (thanks!), but not the flag names. Can you make those consistent too?

clang/include/clang/Basic/AArch64SVETypeFlags.h

+//===- AArch64SVETypeFlags.h - Flags used to generate ACLE builtins- C++ -*-===//
+//
+//  Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+//  See https://llvm.org/LICENSE.txt for license information.
+//  SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_BASIC_AARCH64SVETYPEFLAGS_H
+#define LLVM_CLANG_BASIC_AARCH64SVETYPEFLAGS_H
+
+#include <stdint.h>
+
+namespace clang {
+
+/// Flags to identify the types for overloaded SVE builtins.
+class SVETypeFlags {
+  uint64_t Flags;
+
+public:
+  /// These must be kept in sync with the flags in
+  /// include/clang/Basic/arm_sve.td.
+  static const uint64_t MemEltTypeOffset = 4; // Bit offset of MemEltTypeMask
+  static const uint64_t EltTypeMask      = 0x00000000000f;
+  static const uint64_t MemEltTypeMask   = 0x000000000070;
+  static const uint64_t IsLoad           = 0x000000000080;
+
+  enum EltType {
+    Invalid,
+    Int8,
+    Int16,
+    Int32,
+    Int64,
+    Float16,
+    Float32,
+    Float64,
+    Bool8,
+    Bool16,
+    Bool32,
+    Bool64
+  };
+
+  enum MemEltTy {
+    MemEltTyDefault,
+    MemEltTyInt8,
+    MemEltTyInt16,
+    MemEltTyInt32,
+    MemEltTyInt64
+  };
+
+  SVETypeFlags(uint64_t F) : Flags(F) { }
+  SVETypeFlags(EltType ET, bool IsUnsigned) : Flags(ET) { }
+
+  EltType getEltType() const { return (EltType)(Flags & EltTypeMask); }
+  MemEltTy getMemEltType() const {
+    return (MemEltTy)((Flags & MemEltTypeMask) >> MemEltTypeOffset);
+  }
+
+  bool isLoad() const { return Flags & IsLoad; }
+
+  uint64_t getBits() const { return Flags; }
+  bool isFlagSet(uint64_t Flag) const { return Flags & Flag; }
+};
+
+} // end namespace clang
+
+#endif

clang/include/clang/Basic/BuiltinsAArch64.def

 BUILTIN(__builtin_sponentry, "v*", "c")
 
 // Transactional Memory Extension
 BUILTIN(__builtin_arm_tstart, "WUi", "nj")
 BUILTIN(__builtin_arm_tcommit, "v", "n")
 BUILTIN(__builtin_arm_tcancel, "vWUIi", "n")
 BUILTIN(__builtin_arm_ttest, "WUi", "nc")
 
-// SVE
-BUILTIN(__builtin_sve_svld1_s16, "q8sq16bSsC*", "n")
-BUILTIN(__builtin_sve_svld1_s32, "q4iq16bSiC*", "n")
-BUILTIN(__builtin_sve_svld1_s64, "q2Wiq16bSWiC*", "n")
-BUILTIN(__builtin_sve_svld1_s8, "q16Scq16bScC*", "n")
-BUILTIN(__builtin_sve_svld1_u16, "q8Usq16bUsC*", "n")
-BUILTIN(__builtin_sve_svld1_u32, "q4Uiq16bUiC*", "n")
-BUILTIN(__builtin_sve_svld1_u64, "q2UWiq16bUWiC*", "n")
-BUILTIN(__builtin_sve_svld1_u8, "q16Ucq16bUcC*", "n")
-BUILTIN(__builtin_sve_svld1_f64, "q2dq16bdC*", "n")
-BUILTIN(__builtin_sve_svld1_f32, "q4fq16bfC*", "n")
-BUILTIN(__builtin_sve_svld1_f16, "q8hq16bhC*", "n")
-
 TARGET_HEADER_BUILTIN(_BitScanForward, "UcUNi*UNi", "nh", "intrin.h", ALL_MS_LANGUAGES, "")
 TARGET_HEADER_BUILTIN(_BitScanReverse, "UcUNi*UNi", "nh", "intrin.h", ALL_MS_LANGUAGES, "")
 TARGET_HEADER_BUILTIN(_BitScanForward64, "UcUNi*ULLi", "nh", "intrin.h", ALL_MS_LANGUAGES, "")
 TARGET_HEADER_BUILTIN(_BitScanReverse64, "UcUNi*ULLi", "nh", "intrin.h", ALL_MS_LANGUAGES, "")
 
 TARGET_HEADER_BUILTIN(_InterlockedAdd,           "NiNiD*Ni",    "nh", "intrin.h", ALL_MS_LANGUAGES, "")
 TARGET_HEADER_BUILTIN(_InterlockedAnd64,         "LLiLLiD*LLi", "nh", "intrin.h", ALL_MS_LANGUAGES, "")
 TARGET_HEADER_BUILTIN(_InterlockedDecrement64,   "LLiLLiD*",    "nh", "intrin.h", ALL_MS_LANGUAGES, "")

clang/include/clang/Basic/BuiltinsSVE.def

+//===--- BuiltinsSVE.def - SVE Builtin function database --------*- C++ -*-===//
+//
+//  Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+//  See https://llvm.org/LICENSE.txt for license information.
+//  SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file defines the SVE-specific builtin function database.  Users of
+// this file must define the BUILTIN macro to make use of this information.
+//
+//===----------------------------------------------------------------------===//
+
+// The format of this database matches clang/Basic/Builtins.def.
+
+#define GET_SVE_BUILTINS
+#include "clang/Basic/arm_sve_builtins.inc"
+#undef GET_SVE_BUILTINS
+
+#undef BUILTIN

clang/include/clang/Basic/CMakeLists.txt

clang_tablegen(AttrSubMatchRulesList.inc -gen-clang-attr-subject-match-rule-list
   TARGET ClangAttrSubjectMatchRuleList)
 
 clang_tablegen(AttrHasAttributeImpl.inc -gen-clang-attr-has-attribute-impl
   -I ${CMAKE_CURRENT_SOURCE_DIR}/../../
   SOURCE Attr.td
   TARGET ClangAttrHasAttributeImpl
   )
 
 # ARM NEON and MVE
 clang_tablegen(arm_neon.inc -gen-arm-neon-sema
   SOURCE arm_neon.td
   TARGET ClangARMNeon)
 clang_tablegen(arm_fp16.inc -gen-arm-neon-sema
   SOURCE arm_fp16.td
   TARGET ClangARMFP16)
 clang_tablegen(arm_mve_builtins.inc -gen-arm-mve-builtin-def
   SOURCE arm_mve.td
   TARGET ClangARMMveBuiltinsDef)
 clang_tablegen(arm_mve_builtin_cg.inc -gen-arm-mve-builtin-codegen
   SOURCE arm_mve.td
   TARGET ClangARMMveBuiltinCG)
 clang_tablegen(arm_mve_builtin_sema.inc -gen-arm-mve-builtin-sema
   SOURCE arm_mve.td
   TARGET ClangARMMveBuiltinSema)
 clang_tablegen(arm_mve_builtin_aliases.inc -gen-arm-mve-builtin-aliases
   SOURCE arm_mve.td
   TARGET ClangARMMveBuiltinAliases)
-
+clang_tablegen(arm_sve_builtins.inc -gen-arm-sve-builtins
+  SOURCE arm_sve.td
+  TARGET ClangARMSveBuiltins)
+clang_tablegen(arm_sve_codegenmap.inc -gen-arm-sve-codegenmap
+  SOURCE arm_sve.td
+  TARGET ClangARMSveCodeGenMap)
 clang_tablegen(arm_cde_builtins.inc -gen-arm-cde-builtin-def
   SOURCE arm_cde.td
   TARGET ClangARMCdeBuiltinsDef)
 clang_tablegen(arm_cde_builtin_cg.inc -gen-arm-cde-builtin-codegen
   SOURCE arm_cde.td
   TARGET ClangARMCdeBuiltinCG)
 clang_tablegen(arm_cde_builtin_sema.inc -gen-arm-cde-builtin-sema
   SOURCE arm_cde.td
   TARGET ClangARMCdeBuiltinSema)
 clang_tablegen(arm_cde_builtin_aliases.inc -gen-arm-cde-builtin-aliases
   SOURCE arm_cde.td
   TARGET ClangARMCdeBuiltinAliases)

clang/include/clang/Basic/TargetBuiltins.h

enum {
       LastTIBuiltin = clang::Builtin::FirstTSBuiltin-1,
       LastNEONBuiltin = NEON::FirstTSBuiltin - 1,
 #define BUILTIN(ID, TYPE, ATTRS) BI##ID,
 #include "clang/Basic/BuiltinsARM.def"
       LastTSBuiltin
     };
   }
 
+  namespace SVE {
+  enum {
+    LastNEONBuiltin = NEON::FirstTSBuiltin - 1,
+#define BUILTIN(ID, TYPE, ATTRS) BI##ID,
+#include "clang/Basic/BuiltinsSVE.def"
+    FirstTSBuiltin,
+  };
+  }
+
   /// AArch64 builtins
   namespace AArch64 {
   enum {
     LastTIBuiltin = clang::Builtin::FirstTSBuiltin - 1,
     LastNEONBuiltin = NEON::FirstTSBuiltin - 1,
+    FirstSVEBuiltin = NEON::FirstTSBuiltin,
+    LastSVEBuiltin = SVE::FirstTSBuiltin - 1,
   #define BUILTIN(ID, TYPE, ATTRS) BI##ID,
   #include "clang/Basic/BuiltinsAArch64.def"
     LastTSBuiltin
   };
   }
 
   /// BPF builtins
   namespace BPF {

clang/include/clang/Basic/arm_sve.td

 //===--- arm_sve.td - ARM SVE compiler interface ------------------------===//
 //
 //  Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 //  See https://llvm.org/LICENSE.txt for license information.
 //  SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 //  This file defines the TableGen definitions from which the ARM SVE header
 //  file will be generated.  See:
 //
 //      https://developer.arm.com/architectures/system-architectures/software-standards/acle
 //
 //===----------------------------------------------------------------------===//
+
+//===----------------------------------------------------------------------===//
+// Instruction definitions
+//===----------------------------------------------------------------------===//
+// Every intrinsic subclasses "Inst". An intrinsic has a name, a prototype and
+// a sequence of typespecs.
+//
+// The name is the base name of the intrinsic, for example "svld1". This is
+// then mangled by the tblgen backend to add type information ("svld1_s16").
+//
+// A typespec is a sequence of uppercase characters (modifiers) followed by one
+// lowercase character. A typespec encodes a particular "base type" of the
+// intrinsic.
+//
+// An example typespec is "Us" - unsigned short - svuint16_t. The available
+// typespec codes are given below.
+//
+// The string given to an Inst class is a sequence of typespecs. The intrinsic
+// is instantiated for every typespec in the sequence. For example "sdUsUd".
+//
+// The prototype is a string that defines the return type of the intrinsic
+// and the type of each argument. The return type and every argument gets a
+// "modifier" that can change in some way the "base type" of the intrinsic.
+//
+// The modifier 'd' means "default" and does not modify the base type in any
+// way. The available modifiers are given below.
+//
+// Typespecs
+// ---------
+// c: char
+// s: short
+// i: int
+// l: long
+// f: float
+// h: half-float
+// d: double
+
+// Typespec modifiers
+// ------------------
+// P: boolean
+// U: unsigned
+
+// Prototype modifiers
+// -------------------
+// prototype: return (arg, arg, ...)
+//
+// d: default
+// c: const pointer type
+// P: predicate type
+
+class MergeType<int val> {
+  int Value = val;
+}
+def MergeNone    : MergeType<0>;
+def MergeAny     : MergeType<1>;
+def MergeOp1     : MergeType<2>;
+def MergeZero    : MergeType<3>;
+def MergeAnyExp  : MergeType<4>; // Use merged builtin with explicit
+def MergeZeroExp : MergeType<5>; // generation of its inactive argument.
+
+class MemEltTy<int val> {
+  int Value = val;
+}
+def MemEltTyDefault   : MemEltTy<0>;
+def MemEltTyInt8      : MemEltTy<1>;
+def MemEltTyInt16     : MemEltTy<2>;
+def MemEltTyInt32     : MemEltTy<3>;
+def MemEltTyInt64     : MemEltTy<4>;
+
+class FlagType<int val> {
+  int Value = val;
+}
+
+// These must be kept in sync with the flags in utils/TableGen/SveEmitter.h
+// and include/clang/Basic/TargetBuiltins.h
+def NoFlags                   : FlagType<0x00000000>;
+//                            0x00000001 => EltType
+//                            ...
+//                            0x0000000f => EltType
+//                            0x00000010 => MemEltType
+//                            ...
+//                            0x00000070 => MemEltType
+def IsLoad                    : FlagType<0x00000080>;
+
+// Every intrinsic subclasses Inst.
+class Inst<string n, string p, string t, MergeType mt, string i,
+           list<FlagType> ft, MemEltTy met> {
+  string Name = n;
+  string Prototype = p;
+  string Types = t;
+  string ArchGuard = "";
+  int Merge = mt.Value;
+  string LLVMIntrinsic = i;
+  list<FlagType> Flags = ft;
+  int MemEltType = met.Value;
+}
+
+// MInst: Instructions which access memory
+class MInst<string n, string p, string t, list<FlagType> f,
+            MemEltTy met=MemEltTyDefault, string i="">
+  : Inst<n, p, t, MergeNone, i, f, met> {}
+
+////////////////////////////////////////////////////////////////////////////////
+// Loads
+
+// Load one vector (scalar base)
+def SVLD1   : MInst<"svld1[_{2}]", "dPc", "csilUcUsUiUlhfd", [IsLoad]>;

clang/lib/Basic/Targets/AArch64.cpp

 
 const Builtin::Info AArch64TargetInfo::BuiltinInfo[] = {
 #define BUILTIN(ID, TYPE, ATTRS)                                               \
    {#ID, TYPE, ATTRS, nullptr, ALL_LANGUAGES, nullptr},
 #include "clang/Basic/BuiltinsNEON.def"
 
 #define BUILTIN(ID, TYPE, ATTRS)                                               \
    {#ID, TYPE, ATTRS, nullptr, ALL_LANGUAGES, nullptr},
+#include "clang/Basic/BuiltinsSVE.def"
+
+#define BUILTIN(ID, TYPE, ATTRS)                                               \
+   {#ID, TYPE, ATTRS, nullptr, ALL_LANGUAGES, nullptr},
 #define LANGBUILTIN(ID, TYPE, ATTRS, LANG)                                     \
   {#ID, TYPE, ATTRS, nullptr, LANG, nullptr},
 #define TARGET_HEADER_BUILTIN(ID, TYPE, ATTRS, HEADER, LANGS, FEATURE)         \
   {#ID, TYPE, ATTRS, HEADER, LANGS, FEATURE},
 #include "clang/Basic/BuiltinsAArch64.def"
 };
 
 AArch64TargetInfo::AArch64TargetInfo(const llvm::Triple &Triple,

clang/lib/CodeGen/CGBuiltin.cpp

 #include "CodeGenModule.h"
 #include "ConstantEmitter.h"
 #include "PatternInit.h"
 #include "TargetInfo.h"
 #include "clang/AST/ASTContext.h"
 #include "clang/AST/Attr.h"
 #include "clang/AST/Decl.h"
 #include "clang/AST/OSLog.h"
+#include "clang/Basic/AArch64SVETypeFlags.h"
 #include "clang/Basic/TargetBuiltins.h"
 #include "clang/Basic/TargetInfo.h"
 #include "clang/CodeGen/CGFunctionInfo.h"
 #include "llvm/ADT/SmallPtrSet.h"
 #include "llvm/ADT/StringExtras.h"
 #include "llvm/IR/DataLayout.h"
 #include "llvm/IR/InlineAsm.h"
 #include "llvm/IR/Intrinsics.h"
enum {
   VectorRet = AddRetType | VectorizeRetType,
   VectorRetGetArgs01 =
       AddRetType | Add2ArgTypes | VectorizeRetType | VectorizeArgTypes,
   FpCmpzModifiers =
       AddRetType | VectorizeRetType | Add1ArgType | InventFloatType
 };
 
 namespace {
-struct NeonIntrinsicInfo {
+struct ARMVectorIntrinsicInfo {
   const char *NameHint;
   unsigned BuiltinID;
   unsigned LLVMIntrinsic;
   unsigned AltLLVMIntrinsic;
   unsigned TypeModifier;
 
   bool operator<(unsigned RHSBuiltinID) const {
     return BuiltinID < RHSBuiltinID;
   }
-  bool operator<(const NeonIntrinsicInfo &TE) const {
+  bool operator<(const ARMVectorIntrinsicInfo &TE) const {
     return BuiltinID < TE.BuiltinID;
   }
 };
 } // end anonymous namespace
 
 #define NEONMAP0(NameBase) \
   { #NameBase, NEON::BI__builtin_neon_ ## NameBase, 0, 0, 0 }
 
 #define NEONMAP1(NameBase, LLVMIntrinsic, TypeModifier) \
   { #NameBase, NEON:: BI__builtin_neon_ ## NameBase, \
       Intrinsic::LLVMIntrinsic, 0, TypeModifier }
 
 #define NEONMAP2(NameBase, LLVMIntrinsic, AltLLVMIntrinsic, TypeModifier) \
   { #NameBase, NEON:: BI__builtin_neon_ ## NameBase, \
       Intrinsic::LLVMIntrinsic, Intrinsic::AltLLVMIntrinsic, \
       TypeModifier }
 
-static const NeonIntrinsicInfo ARMSIMDIntrinsicMap [] = {
+static const ARMVectorIntrinsicInfo ARMSIMDIntrinsicMap [] = {
   NEONMAP2(vabd_v, arm_neon_vabdu, arm_neon_vabds, Add1ArgType | UnsignedAlts),
   NEONMAP2(vabdq_v, arm_neon_vabdu, arm_neon_vabds, Add1ArgType | UnsignedAlts),
   NEONMAP1(vabs_v, arm_neon_vabs, 0),
   NEONMAP1(vabsq_v, arm_neon_vabs, 0),
   NEONMAP0(vaddhn_v),
   NEONMAP1(vaesdq_v, arm_neon_aesd, 0),
   NEONMAP1(vaeseq_v, arm_neon_aese, 0),
   NEONMAP1(vaesimcq_v, arm_neon_aesimc, 0),
static const ARMVectorIntrinsicInfo ARMSIMDIntrinsicMap [] = {
   NEONMAP0(vtst_v),
   NEONMAP0(vtstq_v),
   NEONMAP0(vuzp_v),
   NEONMAP0(vuzpq_v),
   NEONMAP0(vzip_v),
   NEONMAP0(vzipq_v)
 };
 
-static const NeonIntrinsicInfo AArch64SIMDIntrinsicMap[] = {
+static const ARMVectorIntrinsicInfo AArch64SIMDIntrinsicMap[] = {
   NEONMAP1(vabs_v, aarch64_neon_abs, 0),
   NEONMAP1(vabsq_v, aarch64_neon_abs, 0),
   NEONMAP0(vaddhn_v),
   NEONMAP1(vaesdq_v, aarch64_crypto_aesd, 0),
   NEONMAP1(vaeseq_v, aarch64_crypto_aese, 0),
   NEONMAP1(vaesimcq_v, aarch64_crypto_aesimc, 0),
   NEONMAP1(vaesmcq_v, aarch64_crypto_aesmc, 0),
   NEONMAP1(vcadd_rot270_v, aarch64_neon_vcadd_rot270, Add1ArgType),
static const ARMVectorIntrinsicInfo AArch64SIMDIntrinsicMap[] = {
   NEONMAP1(vst1q_x2_v, aarch64_neon_st1x2, 0),
   NEONMAP1(vst1q_x3_v, aarch64_neon_st1x3, 0),
   NEONMAP1(vst1q_x4_v, aarch64_neon_st1x4, 0),
   NEONMAP0(vsubhn_v),
   NEONMAP0(vtst_v),
   NEONMAP0(vtstq_v),
 };
 
-static const NeonIntrinsicInfo AArch64SISDIntrinsicMap[] = {
+static const ARMVectorIntrinsicInfo AArch64SISDIntrinsicMap[] = {
   NEONMAP1(vabdd_f64, aarch64_sisd_fabd, Add1ArgType),
   NEONMAP1(vabds_f32, aarch64_sisd_fabd, Add1ArgType),
   NEONMAP1(vabsd_s64, aarch64_neon_abs, Add1ArgType),
   NEONMAP1(vaddlv_s32, aarch64_neon_saddlv, AddRetType | Add1ArgType),
   NEONMAP1(vaddlv_u32, aarch64_neon_uaddlv, AddRetType | Add1ArgType),
   NEONMAP1(vaddlvq_s32, aarch64_neon_saddlv, AddRetType | Add1ArgType),
   NEONMAP1(vaddlvq_u32, aarch64_neon_uaddlv, AddRetType | Add1ArgType),
   NEONMAP1(vaddv_f32, aarch64_neon_faddv, AddRetType | Add1ArgType),
static const ARMVectorIntrinsicInfo AArch64SISDIntrinsicMap[] = {
   NEONMAP1(vrsqrteh_f16, aarch64_neon_frsqrte, Add1ArgType),
   NEONMAP1(vrsqrtsh_f16, aarch64_neon_frsqrts, Add1ArgType),
 };
 
 #undef NEONMAP0
 #undef NEONMAP1
 #undef NEONMAP2
 
+#define SVEMAP1(NameBase, LLVMIntrinsic, TypeModifier)                         \
+  {                                                                            \
+    #NameBase, SVE::BI__builtin_sve_##NameBase, Intrinsic::LLVMIntrinsic, 0,   \
+        TypeModifier                                                           \
+  }
+
+#define SVEMAP2(NameBase, TypeModifier)                                        \
+  { #NameBase, SVE::BI__builtin_sve_##NameBase, 0, 0, TypeModifier }
+static const ARMVectorIntrinsicInfo AArch64SVEIntrinsicMap[] = {
+#define GET_SVE_LLVM_INTRINSIC_MAP
+#include "clang/Basic/arm_sve_codegenmap.inc"
+#undef GET_SVE_LLVM_INTRINSIC_MAP
+};
+
+#undef SVEMAP1
+#undef SVEMAP2
+
 static bool NEONSIMDIntrinsicsProvenSorted = false;
 
 static bool AArch64SIMDIntrinsicsProvenSorted = false;
 static bool AArch64SISDIntrinsicsProvenSorted = false;
+static bool AArch64SVEIntrinsicsProvenSorted = false;
 
-
-static const NeonIntrinsicInfo *
-findNeonIntrinsicInMap(ArrayRef<NeonIntrinsicInfo> IntrinsicMap,
+static const ARMVectorIntrinsicInfo *
+findARMVectorIntrinsicInMap(ArrayRef<ARMVectorIntrinsicInfo> IntrinsicMap,
                             unsigned BuiltinID, bool &MapProvenSorted) {
 
 #ifndef NDEBUG
   if (!MapProvenSorted) {
     assert(std::is_sorted(std::begin(IntrinsicMap), std::end(IntrinsicMap)));
     MapProvenSorted = true;
   }
 #endif
 
-  const NeonIntrinsicInfo *Builtin = llvm::lower_bound(IntrinsicMap, BuiltinID);
+  const ARMVectorIntrinsicInfo *Builtin =
+      llvm::lower_bound(IntrinsicMap, BuiltinID);
 
   if (Builtin != IntrinsicMap.end() && Builtin->BuiltinID == BuiltinID)
     return Builtin;
 
   return nullptr;
 }
 
 Function *CodeGenFunction::LookupNeonLLVMIntrinsic(unsigned IntrinsicID,
if (Modifier & Add2ArgTypes)
     Tys.push_back(ArgType);
 
   if (Modifier & InventFloatType)
     Tys.push_back(FloatTy);
 
   return CGM.getIntrinsic(IntrinsicID, Tys);
 }
 
-static Value *EmitCommonNeonSISDBuiltinExpr(CodeGenFunction &CGF,
-                                            const NeonIntrinsicInfo &SISDInfo,
-                                            SmallVectorImpl<Value *> &Ops,
-                                            const CallExpr *E) {
+static Value *EmitCommonNeonSISDBuiltinExpr(
+    CodeGenFunction &CGF, const ARMVectorIntrinsicInfo &SISDInfo,
+    SmallVectorImpl<Value *> &Ops, const CallExpr *E) {
   unsigned BuiltinID = SISDInfo.BuiltinID;
   unsigned int Int = SISDInfo.LLVMIntrinsic;
   unsigned Modifier = SISDInfo.TypeModifier;
   const char *s = SISDInfo.NameHint;
 
   switch (BuiltinID) {
   case NEON::BI__builtin_neon_vcled_s64:
   case NEON::BI__builtin_neon_vcled_u64:
llvm::VectorType *VTy = GetNeonType(this, Type,
                                       getTarget().hasLegalHalfType());
   llvm::Type *Ty = VTy;
   if (!Ty)
     return nullptr;
 
   // Many NEON builtins have identical semantics and uses in ARM and
   // AArch64. Emit these in a single function.
   auto IntrinsicMap = makeArrayRef(ARMSIMDIntrinsicMap);
-  const NeonIntrinsicInfo *Builtin = findNeonIntrinsicInMap(
+  const ARMVectorIntrinsicInfo *Builtin = findARMVectorIntrinsicInMap(
       IntrinsicMap, BuiltinID, NEONSIMDIntrinsicsProvenSorted);
   if (Builtin)
     return EmitCommonNeonBuiltinExpr(
         Builtin->BuiltinID, Builtin->LLVMIntrinsic, Builtin->AltLLVMIntrinsic,
         Builtin->NameHint, Builtin->TypeModifier, E, Ops, PtrOp0, PtrOp1, Arch);
 
   unsigned Int;
   switch (BuiltinID) {
Value *CodeGenFunction::EmitSVEMaskedLoad(llvm::Type *ReturnTy,
   Value *Predicate = EmitSVEPredicateCast(Ops[0], MemoryTy);
   Value *BasePtr = Builder.CreateBitCast(Ops[1], MemoryTy->getPointerTo());
   BasePtr = Builder.CreateGEP(MemoryTy, BasePtr, Offset);
 
   Value *Splat0 = Constant::getNullValue(MemoryTy);
   return Builder.CreateMaskedLoad(BasePtr, Align(1), Predicate, Splat0);
 }
 
+Value *CodeGenFunction::EmitAArch64SVEBuiltinExpr(unsigned BuiltinID,
+                                                  const CallExpr *E) {
+  // Find out if any arguments are required to be integer constant expressions.
+  unsigned ICEArguments = 0;
+  ASTContext::GetBuiltinTypeError Error;
+  getContext().GetBuiltinType(BuiltinID, Error, &ICEArguments);
+  assert(Error == ASTContext::GE_None && "Should not codegen an error");
+
+  llvm::SmallVector<Value *, 4> Ops;
+  for (unsigned i = 0, e = E->getNumArgs(); i != e; i++) {
+    if ((ICEArguments & (1 << i)) == 0)
+      Ops.push_back(EmitScalarExpr(E->getArg(i)));
+    else
+      llvm_unreachable("Not yet implemented");
+  }
+
+  auto *Builtin = findARMVectorIntrinsicInMap(AArch64SVEIntrinsicMap, BuiltinID,
+                                              AArch64SVEIntrinsicsProvenSorted);
+  SVETypeFlags TypeFlags(Builtin->TypeModifier);
+  llvm::Type *Ty = ConvertType(E->getType());
+  if (TypeFlags.isLoad())
+    return EmitSVEMaskedLoad(Ty, Ops);
+
+  /// Should not happen
+  return nullptr;
+}
+
 Value *CodeGenFunction::EmitAArch64BuiltinExpr(unsigned BuiltinID,
                                                const CallExpr *E,
                                                llvm::Triple::ArchType Arch) {
+  if (BuiltinID >= AArch64::FirstSVEBuiltin &&
+      BuiltinID <= AArch64::LastSVEBuiltin)
+    return EmitAArch64SVEBuiltinExpr(BuiltinID, E);
+
   unsigned HintID = static_cast<unsigned>(-1);
   switch (BuiltinID) {
   default: break;
   case AArch64::BI__builtin_arm_nop:
     HintID = 0;
     break;
   case AArch64::BI__builtin_arm_yield:
   case AArch64::BI__yield:
case AArch64::BI__sevl:
     break;
   }
 
   if (HintID != static_cast<unsigned>(-1)) {
     Function *F = CGM.getIntrinsic(Intrinsic::aarch64_hint);
     return Builder.CreateCall(F, llvm::ConstantInt::get(Int32Ty, HintID));
   }
 
-  switch (BuiltinID) {
-  case AArch64::BI__builtin_sve_svld1_u8:
-  case AArch64::BI__builtin_sve_svld1_u16:
-  case AArch64::BI__builtin_sve_svld1_u32:
-  case AArch64::BI__builtin_sve_svld1_u64:
-  case AArch64::BI__builtin_sve_svld1_s8:
-  case AArch64::BI__builtin_sve_svld1_s16:
-  case AArch64::BI__builtin_sve_svld1_s32:
-  case AArch64::BI__builtin_sve_svld1_s64:
-  case AArch64::BI__builtin_sve_svld1_f16:
-  case AArch64::BI__builtin_sve_svld1_f32:
-  case AArch64::BI__builtin_sve_svld1_f64: {
-    llvm::SmallVector<Value *, 4> Ops = {EmitScalarExpr(E->getArg(0)),
-                                         EmitScalarExpr(E->getArg(1))};
-    llvm::Type *Ty = ConvertType(E->getType());
-    return EmitSVEMaskedLoad(Ty, Ops);
-  }
-  default:
-    break;
-  }
-
   if (BuiltinID == AArch64::BI__builtin_arm_prefetch) {
     Value *Address         = EmitScalarExpr(E->getArg(0));
     Value *RW              = EmitScalarExpr(E->getArg(1));
     Value *CacheLevel      = EmitScalarExpr(E->getArg(2));
     Value *RetentionPolicy = EmitScalarExpr(E->getArg(3));
     Value *IsData          = EmitScalarExpr(E->getArg(4));
 
     Value *Locality = nullptr;
if ((ICEArguments & (1 << i)) == 0) {
       bool IsConst = E->getArg(i)->isIntegerConstantExpr(Result, getContext());
       assert(IsConst && "Constant arg isn't actually constant?");
       (void)IsConst;
       Ops.push_back(llvm::ConstantInt::get(getLLVMContext(), Result));
     }
   }
 
   auto SISDMap = makeArrayRef(AArch64SISDIntrinsicMap);
-  const NeonIntrinsicInfo *Builtin = findNeonIntrinsicInMap(
+  const ARMVectorIntrinsicInfo *Builtin = findARMVectorIntrinsicInMap(
       SISDMap, BuiltinID, AArch64SISDIntrinsicsProvenSorted);
 
   if (Builtin) {
     Ops.push_back(EmitScalarExpr(E->getArg(E->getNumArgs() - 1)));
     Value *Result = EmitCommonNeonSISDBuiltinExpr(*this, *Builtin, Ops, E);
     assert(Result && "SISD intrinsic should have been handled");
     return Result;
   }

 
   llvm::VectorType *VTy = GetNeonType(this, Type);
   llvm::Type *Ty = VTy;
   if (!Ty)
     return nullptr;
 
   // Not all intrinsics handled by the common case work for AArch64 yet, so only
   // defer to common code if it's been added to our special map.
-  Builtin = findNeonIntrinsicInMap(AArch64SIMDIntrinsicMap, BuiltinID,
+  Builtin = findARMVectorIntrinsicInMap(AArch64SIMDIntrinsicMap, BuiltinID,
                                         AArch64SIMDIntrinsicsProvenSorted);
 
   if (Builtin)
     return EmitCommonNeonBuiltinExpr(
         Builtin->BuiltinID, Builtin->LLVMIntrinsic, Builtin->AltLLVMIntrinsic,
         Builtin->NameHint, Builtin->TypeModifier, E, Ops,
         /*never use addresses*/ Address::invalid(), Address::invalid(), Arch);
 
   if (Value *V = EmitAArch64TblBuiltinExpr(*this, BuiltinID, E, Ops, Arch))

clang/lib/CodeGen/CodeGenFunction.h

llvm::Value *EmitNeonShiftVector(llvm::Value *V, llvm::Type *Ty,
                                    bool negateForRightShift);
   llvm::Value *EmitNeonRShiftImm(llvm::Value *Vec, llvm::Value *Amt,
                                  llvm::Type *Ty, bool usgn, const char *name);
   llvm::Value *vectorWrapScalar16(llvm::Value *Op);
 
   llvm::Value *EmitSVEPredicateCast(llvm::Value *Pred, llvm::VectorType *VTy);
   llvm::Value *EmitSVEMaskedLoad(llvm::Type *ReturnTy,
                                  SmallVectorImpl<llvm::Value *> &Ops);
+  llvm::Value *EmitAArch64SVEBuiltinExpr(unsigned BuiltinID, const CallExpr *E);
 
   llvm::Value *EmitAArch64BuiltinExpr(unsigned BuiltinID, const CallExpr *E,
                                       llvm::Triple::ArchType Arch);
   llvm::Value *EmitBPFBuiltinExpr(unsigned BuiltinID, const CallExpr *E);
 
   llvm::Value *BuildVector(ArrayRef<llvm::Value*> Ops);
   llvm::Value *EmitX86BuiltinExpr(unsigned BuiltinID, const CallExpr *E);
   llvm::Value *EmitPPCBuiltinExpr(unsigned BuiltinID, const CallExpr *E);

clang/utils/TableGen/SveEmitter.cpp

 //===- SveEmitter.cpp - Generate arm_sve.h for use with clang -*- C++ -*-===//
 //
 //  Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 //  See https://llvm.org/LICENSE.txt for license information.
 //  SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This tablegen backend is responsible for emitting arm_sve.h, which includes

 //===----------------------------------------------------------------------===//
 
 #include "llvm/ADT/STLExtras.h"
 #include "llvm/ADT/DenseMap.h"
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/StringExtras.h"
 #include "llvm/TableGen/Record.h"
 #include "llvm/TableGen/Error.h"
+#include "clang/Basic/AArch64SVETypeFlags.h"
 #include <string>
 #include <sstream>
 #include <set>
 #include <cctype>
 
 using namespace llvm;
 
-//===----------------------------------------------------------------------===//
-// SVEEmitter
-//===----------------------------------------------------------------------===//
+enum ClassKind {
+  ClassNone,
+  ClassS,     // signed/unsigned, e.g., "_s8", "_u8" suffix
+  ClassG,     // Overloaded name without type suffix
+};
+
+using TypeSpec = std::string;
+using SVETypeFlags = clang::SVETypeFlags;
 
 namespace {
 
+class SVEType {
+  TypeSpec TS;
+  bool Float, Signed, Immediate, Void, Constant, Pointer;
+  bool DefaultType, IsScalable, Predicate, PredicatePattern, PrefetchOp;
+  unsigned Bitwidth, ElementBitwidth, NumVectors;
+
+public:
+  SVEType() : SVEType(TypeSpec(), 'v') {}
+
+  SVEType(TypeSpec TS, char CharMod)
+      : TS(TS), Float(false), Signed(true), Immediate(false), Void(false),
+        Constant(false), Pointer(false), DefaultType(false), IsScalable(true),
+        Predicate(false), PredicatePattern(false), PrefetchOp(false),
+        Bitwidth(128), ElementBitwidth(~0U), NumVectors(1) {
+    if (!TS.empty())
+      applyTypespec();
+    applyModifier(CharMod);
+  }
+
+  /// Return the value in SVETypeFlags for this type.
+  unsigned getTypeFlags() const;
+
+  bool isPointer() const { return Pointer; }
+  bool isVoidPointer() const { return Pointer && Void; }
+  bool isSigned() const { return Signed; }
+  bool isImmediate() const { return Immediate; }
+  bool isScalar() const { return NumVectors == 0; }
+  bool isVector() const { return NumVectors > 0; }
+  bool isScalableVector() const { return isVector() && IsScalable; }
+  bool isChar() const { return ElementBitwidth == 8; }
+  bool isVoid() const { return Void & !Pointer; }
+  bool isDefault() const { return DefaultType; }
+  bool isFloat() const { return Float; }
+  bool isInteger() const { return !Float && !Predicate; }
+  bool isScalarPredicate() const { return !Float && ElementBitwidth == 1; }
+  bool isPredicateVector() const { return Predicate; }
+  bool isPredicatePattern() const { return PredicatePattern; }
+  bool isPrefetchOp() const { return PrefetchOp; }
+  bool isConstant() const { return Constant; }
+  unsigned getElementSizeInBits() const { return ElementBitwidth; }
+  unsigned getNumVectors() const { return NumVectors; }
+
+  unsigned getNumElements() const {
+    assert(ElementBitwidth != ~0U);
+    return Bitwidth / ElementBitwidth;
+  }
+  unsigned getSizeInBits() const {
+    return Bitwidth;
+  }
+
+  /// Return the string representation of a type, which is an encoded
+  /// string for passing to the BUILTIN() macro in Builtins.def.
+  std::string builtin_str() const;
+
+private:
+  /// Creates the type based on the typespec string in TS.
+  void applyTypespec();
+
+  /// Applies a prototype modifier to the type.
+  void applyModifier(char Mod);
+};
+
+
+class SVEEmitter;
+
+/// The main grunt class. This represents an instantiation of an intrinsic with
+/// a particular typespec and prototype.
+class Intrinsic {
+  /// The unmangled name.
+  std::string Name;
+
+  /// The name of the corresponding LLVM IR intrinsic.
+  std::string LLVMName;
+
+  /// Intrinsic prototype.
+  std::string Proto;
+
+  /// The base type spec for this intrinsic.
+  TypeSpec BaseTypeSpec;
+
+  /// The base class kind. Most intrinsics use ClassS, which has full type
+  /// info for integers (_s32/_u32), or ClassG which is used for overloaded
+  /// intrinsics.
+  ClassKind Class;
+
+  /// The architectural #ifdef guard.
+  std::string Guard;
+
+  /// The types of return value [0] and parameters [1..].
+  std::vector<SVEType> Types;
+
+  /// The "base type", which is VarType('d', BaseTypeSpec).
+  SVEType BaseType;
+
+  /// The type of the memory element
+  enum MemEltType {
+    MemEltTypeDefault,
+    MemEltTypeInt8,
+    MemEltTypeInt16,
+    MemEltTypeInt32,
+    MemEltTypeInt64,
+    MemEltTypeInvalid
+  } MemEltTy;
+
+  SVETypeFlags Flags;
+
+public:
+  /// The type of predication.
+  enum MergeType {
+    MergeNone,
+    MergeAny,
+    MergeOp1,
+    MergeZero,
+    MergeAnyExp,
+    MergeZeroExp,
+    MergeInvalid
+  } Merge;
+
+  Intrinsic(StringRef Name, StringRef Proto, int64_t MT, int64_t MET,
+            StringRef LLVMName, SVETypeFlags Flags, TypeSpec BT, ClassKind Class,
+            SVEEmitter &Emitter, StringRef Guard)
+      : Name(Name.str()), LLVMName(LLVMName), Proto(Proto.str()),
+        BaseTypeSpec(BT), Class(Class), Guard(Guard.str()), BaseType(BT, 'd'),
+        MemEltTy(MemEltType(MET)), Flags(Flags), Merge(MergeType(MT)) {
+    // Types[0] is the return value.
+    for (unsigned I = 0; I < Proto.size(); ++I)
+      Types.emplace_back(BaseTypeSpec, Proto[I]);
+  }
+
+  ~Intrinsic()=default;
+
+  std::string getName() const { return Name; }
+  std::string getLLVMName() const { return LLVMName; }
+  std::string getProto() const { return Proto; }
+  TypeSpec getBaseTypeSpec() const { return BaseTypeSpec; }
+  SVEType getBaseType() const { return BaseType; }
+
+  StringRef getGuard() const { return Guard; }
+  ClassKind getClassKind() const { return Class; }
+  MergeType getMergeType() const { return Merge; }
+
+  SVEType getReturnType() const { return Types[0]; }
+  ArrayRef<SVEType> getTypes() const { return Types; }
+  SVEType getParamType(unsigned I) const { return Types[I + 1]; }
+  unsigned getNumParams() const { return Proto.size() - 1; }
+
+  SVETypeFlags getFlags() const { return Flags; }
+  bool isFlagSet(uint64_t Flag) const { return Flags.isFlagSet(Flag);}
+
+  int64_t getMemEltTypeEnum() const {
+    int64_t METEnum = (MemEltTy << SVETypeFlags::MemEltTypeOffset);
+    assert((METEnum &~ SVETypeFlags::MemEltTypeMask) == 0 && "Bad MemEltTy");
+    return METEnum;
+  }
+
+  /// Return the type string for a BUILTIN() macro in Builtins.def.
+  std::string getBuiltinTypeStr();
+
+  /// Return the name, mangled with type information. The name is mangled for
+  /// ClassS, so will add type suffixes such as _u32/_s32.
+  std::string getMangledName() const { return mangleName(ClassS); }
+
+  /// Returns true if the intrinsic is overloaded, in that it should also generate
+  /// a short form without the type-specifiers, e.g. 'svld1(..)' instead of
+  /// 'svld1_u32(..)'.
+  static bool isOverloadedIntrinsic(StringRef Name) {
+    auto BrOpen = Name.find("[");
+    auto BrClose = Name.find(']');
+    return BrOpen != std::string::npos && BrClose != std::string::npos;
+  }
+
+  /// Emits the intrinsic declaration to the ostream.
+  void emitIntrinsic(raw_ostream &OS) const;
+
+private:
+  std::string getMergeSuffix() const;
+  std::string mangleName(ClassKind LocalCK) const;
+  std::string replaceTemplatedArgs(std::string Name, TypeSpec TS,
+                                   std::string Proto) const;
+};
+
 class SVEEmitter {
+private:
+  RecordKeeper &Records;
+
 public:
-  // run - Emit arm_sve.h
-  void run(raw_ostream &o);
+  SVEEmitter(RecordKeeper &R) : Records(R) {}
+
+  /// Emit arm_sve.h.
+  void createHeader(raw_ostream &o);
+
+  /// Emit all the __builtin prototypes and code needed by Sema.
+  void createBuiltins(raw_ostream &o);
+
+  /// Emit all the information needed to map builtin -> LLVM IR intrinsic.
+  void createCodeGenMap(raw_ostream &o);
+
+  /// Create intrinsic and add it to \p Out
+  void createIntrinsic(Record *R, SmallVectorImpl<std::unique_ptr<Intrinsic>> &Out);
 };
 
 } // end anonymous namespace
 
 
 //===----------------------------------------------------------------------===//
+// Type implementation
+//===----------------------------------------------------------------------===//
+
+unsigned SVEType::getTypeFlags() const {
+  if (isFloat()) {
+    switch (ElementBitwidth) {
+    case 16: return SVETypeFlags::Float16;
+    case 32: return SVETypeFlags::Float32;
+    case 64: return SVETypeFlags::Float64;
+    default: llvm_unreachable("Unhandled float element bitwidth!");
+    }
+  }
+
+  if (isPredicateVector()) {
+    switch (ElementBitwidth) {
+    case 8:  return SVETypeFlags::Bool8;
+    case 16: return SVETypeFlags::Bool16;
+    case 32: return SVETypeFlags::Bool32;
+    case 64: return SVETypeFlags::Bool64;
+    default: llvm_unreachable("Unhandled predicate element bitwidth!");
+    }
+  }
+
+  switch (ElementBitwidth) {
+  case 8:  return SVETypeFlags::Int8;
+  case 16: return SVETypeFlags::Int16;
+  case 32: return SVETypeFlags::Int32;
+  case 64: return SVETypeFlags::Int64;
+  default: llvm_unreachable("Unhandled integer element bitwidth!");
+  }
+}
+
+std::string SVEType::builtin_str() const {
+  std::string S;
+  if (isVoid())
+    return "v";
+
+  if (isVoidPointer())
+    S += "v";
+  else if (!Float)
+    switch (ElementBitwidth) {
+    case 1: S += "b"; break;
+    case 8: S += "c"; break;
+    case 16: S += "s"; break;
+    case 32: S += "i"; break;
+    case 64: S += "Wi"; break;
+    case 128: S += "LLLi"; break;
+    default: llvm_unreachable("Unhandled case!");
+    }
+  else
+    switch (ElementBitwidth) {
+    case 16: S += "h"; break;
+    case 32: S += "f"; break;
+    case 64: S += "d"; break;
+    default: llvm_unreachable("Unhandled case!");
+    }
+
+  if (!isFloat()) {
+    if ((isChar() || isPointer()) && !isVoidPointer()) {
+      // Make chars and typed pointers explicitly signed.
+      if (Signed)
+        S = "S" + S;
+      else if (!Signed)
+        S = "U" + S;
+    } else if (!isVoidPointer() && !Signed) {
+      S = "U" + S;
+    }
+  }
+
+  // Constant indices are "int", but have the "constant expression" modifier.
+  if (isImmediate()) {
+    assert(!isFloat() && "fp immediates are not supported");
+    S = "I" + S;
+  }
+
+  if (isScalar()) {
+    if (Constant) S += "C";
+    if (Pointer) S += "*";
+    return S;
+  }
+
+  assert(isScalableVector() && "Unsupported type");
+  return "q" + utostr(getNumElements() * NumVectors) + S;
+}
+
+void SVEType::applyTypespec() {
+  for (char I : TS) {
+    switch (I) {
+    case 'P':
+      Predicate = true;
+      ElementBitwidth = 1;
+      break;
+    case 'U':
+      Signed = false;
+      break;
+    case 'c':
+      ElementBitwidth = 8;
+      break;
+    case 's':
+      ElementBitwidth = 16;
+      break;
+    case 'i':
+      ElementBitwidth = 32;
+      break;
+    case 'l':
+      ElementBitwidth = 64;
+      break;
+    case 'h':
+      Float = true;
+      ElementBitwidth = 16;
+      break;
+    case 'f':
+      Float = true;
+      ElementBitwidth = 32;
+      break;
+    case 'd':
+      Float = true;
+      ElementBitwidth = 64;
+      break;
+    default:
+      llvm_unreachable("Unhandled type code!");
+    }
+  }
+  assert(ElementBitwidth != ~0U && "Bad element bitwidth!");
+}
+
+void SVEType::applyModifier(char Mod) {
+  switch (Mod) {
+  case 'v':
+    Void = true;
+    break;
+  case 'd':
+    DefaultType = true;
+    break;
+  case 'c':
+    Constant = true;
+    LLVM_FALLTHROUGH;
+  case 'p':
+    Pointer = true;
+    Bitwidth = ElementBitwidth;
+    NumVectors = 0;
+    break;
+  case 'P':
+    Signed = true;
+    Float = false;
+    Predicate = true;
+    Bitwidth = 16;
+    ElementBitwidth = 1;
+    break;
+  default:
+    llvm_unreachable("Unhandled character!");
+  }
+}
+
+
+//===----------------------------------------------------------------------===//
+// Intrinsic implementation
+//===----------------------------------------------------------------------===//
+
+std::string Intrinsic::getBuiltinTypeStr() {
+  std::string S;
+
+  SVEType RetT = getReturnType();
+  // Since the return value must be one type, return a vector type of the
+  // appropriate width which we will bitcast.  An exception is made for
+  // returning structs of 2, 3, or 4 vectors which are returned in a sret-like
+  // fashion, storing them to a pointer arg.
+  if (RetT.getNumVectors() > 1) {
+    S += "vv*"; // void result with void* first argument
+  } else
+    S += RetT.builtin_str();
+
+  for (unsigned I = 0; I < getNumParams(); ++I)
+    S += getParamType(I).builtin_str();
+
+  return S;
+}
+
+std::string Intrinsic::replaceTemplatedArgs(std::string Name, TypeSpec TS,
+                                            std::string Proto) const {
+  std::string Ret = Name;
+  while (Ret.find('{') != std::string::npos) {
+    size_t Pos = Ret.find('{');
+    size_t End = Ret.find('}');
+    unsigned NumChars = End - Pos + 1;
+    assert(NumChars == 3 && "Unexpected template argument");
+
+    SVEType T;
+    char C = Ret[Pos+1];
+    switch(C) {
+    default:
+      llvm_unreachable("Unknown predication specifier");
+    case 'd':
+      T = SVEType(TS, 'd');
+      break;
+    case '0':
+    case '1':
+    case '2':
+    case '3':
+      T = SVEType(TS, Proto[C - '0']);
+      break;
+    }
+
+    // Replace templated arg with the right suffix (e.g. u32)
+    std::string TypeCode;
+    if (T.isInteger())
+      TypeCode = T.isSigned() ? 's' : 'u';
+    else if (T.isPredicateVector())
+      TypeCode = 'b';
+    else
+      TypeCode = 'f';
+    Ret.replace(Pos, NumChars, TypeCode + utostr(T.getElementSizeInBits()));
+  }
+
+  return Ret;
+}
+
+// ACLE function names have a merge style postfix.
+std::string Intrinsic::getMergeSuffix() const {
+  switch (getMergeType()) {
+    default:
+      llvm_unreachable("Unknown predication specifier");
+    case MergeNone:    return "";
+    case MergeAny:
+    case MergeAnyExp:  return "_x";
+    case MergeOp1:     return "_m";
+    case MergeZero:
+    case MergeZeroExp: return "_z";
+  }
+}
+
+std::string Intrinsic::mangleName(ClassKind LocalCK) const {
+  std::string S = getName();
+
+  if (LocalCK == ClassG) {
+    // Remove the square brackets and everything in between.
+    while (S.find("[") != std::string::npos) {
+      auto Start = S.find("[");
+      auto End = S.find(']');
+      S.erase(Start, (End-Start)+1);
+    }
+  } else {
+    // Remove the square brackets.
+    while (S.find("[") != std::string::npos) {
+      auto BrPos = S.find('[');
+      if (BrPos != std::string::npos)
+        S.erase(BrPos, 1);
+      BrPos = S.find(']');
+      if (BrPos != std::string::npos)
+        S.erase(BrPos, 1);
+    }
+  }
+
+  // Replace all {d} like expressions with e.g. 'u32'
+  return replaceTemplatedArgs(S, getBaseTypeSpec(), getProto()) +
+         getMergeSuffix();
+}
+
+void Intrinsic::emitIntrinsic(raw_ostream &OS) const {
+  // Use the preprocessor to enable the non-overloaded builtins.
+  if (getClassKind() != ClassG || getProto().size() <= 1) {
+    OS << "#define " << mangleName(getClassKind())
+       << "(...) __builtin_sve_" << mangleName(ClassS)
+       << "(__VA_ARGS__)\n";
+  } else {
+    llvm_unreachable("Not yet implemented. Overloaded intrinsics will follow "
+                     "in a future patch");
+  }
+}
+
+//===----------------------------------------------------------------------===//
 // SVEEmitter implementation
 //===----------------------------------------------------------------------===//
+void SVEEmitter::createIntrinsic(
+    Record *R, SmallVectorImpl<std::unique_ptr<Intrinsic>> &Out) {
+  StringRef Name = R->getValueAsString("Name");
+  StringRef Proto = R->getValueAsString("Prototype");
+  StringRef Types = R->getValueAsString("Types");
+  StringRef Guard = R->getValueAsString("ArchGuard");
+  StringRef LLVMName = R->getValueAsString("LLVMIntrinsic");
+  int64_t Merge = R->getValueAsInt("Merge");
+  std::vector<Record*> FlagsList = R->getValueAsListOfDefs("Flags");
+  int64_t MemEltType = R->getValueAsInt("MemEltType");
+
+  int64_t Flags = 0;
+  for (auto FlagRec : FlagsList)
+    Flags |= FlagRec->getValueAsInt("Value");
+
+  // Extract type specs from string
+  SmallVector<TypeSpec, 8> TypeSpecs;
+  TypeSpec Acc;
+  for (char I : Types) {
+    Acc.push_back(I);
+    if (islower(I)) {
+      TypeSpecs.push_back(TypeSpec(Acc));
+      Acc.clear();
+    }
+  }
+
+  // Remove duplicate type specs.
+  std::sort(TypeSpecs.begin(), TypeSpecs.end());
+  TypeSpecs.erase(std::unique(TypeSpecs.begin(), TypeSpecs.end()),
+                  TypeSpecs.end());
+
+  // Create an Intrinsic for each type spec.
+  for (auto TS : TypeSpecs) {
+    Out.push_back(std::make_unique<Intrinsic>(Name, Proto, Merge, MemEltType,
+                                              LLVMName, Flags, TS, ClassS,
+                                              *this, Guard));
+  }
+}
 
-void SVEEmitter::run(raw_ostream &OS) {
+void SVEEmitter::createHeader(raw_ostream &OS) {
   OS << "/*===---- arm_sve.h - ARM SVE intrinsics "
         "-----------------------------------===\n"
         " *\n"
         " *\n"
         " * Part of the LLVM Project, under the Apache License v2.0 with LLVM "
         "Exceptions.\n"
         " * See https://llvm.org/LICENSE.txt for license information.\n"
         " * SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception\n"
         " *\n"
         " *===-----------------------------------------------------------------"
         "------===\n"
         " */\n\n";
 
   OS << "#ifndef __ARM_SVE_H\n";
   OS << "#define __ARM_SVE_H\n\n";
 
   OS << "#if !defined(__ARM_FEATURE_SVE)\n";
   OS << "#error \"SVE support not enabled\"\n";
   OS << "#else\n\n";
 
   OS << "#include <stdint.h>\n\n";
-  OS << "#ifndef  __cplusplus\n";
+  OS << "#ifdef  __cplusplus\n";
+  OS << "extern \"C\" {\n";
+  OS << "#else\n";
   OS << "#include <stdbool.h>\n";
   OS << "#endif\n\n";
 
   OS << "typedef __fp16 float16_t;\n";
   OS << "typedef float float32_t;\n";
   OS << "typedef double float64_t;\n";
   OS << "typedef bool bool_t;\n\n";
 
   OS << "typedef __SVInt8_t svint8_t;\n";
   OS << "typedef __SVInt16_t svint16_t;\n";
   OS << "typedef __SVInt32_t svint32_t;\n";
   OS << "typedef __SVInt64_t svint64_t;\n";
   OS << "typedef __SVUint8_t svuint8_t;\n";
   OS << "typedef __SVUint16_t svuint16_t;\n";
   OS << "typedef __SVUint32_t svuint32_t;\n";
   OS << "typedef __SVUint64_t svuint64_t;\n";
   OS << "typedef __SVFloat16_t svfloat16_t;\n";
   OS << "typedef __SVFloat32_t svfloat32_t;\n";
   OS << "typedef __SVFloat64_t svfloat64_t;\n";
   OS << "typedef __SVBool_t  svbool_t;\n\n";
 
-  OS << "#define svld1_u8(...) __builtin_sve_svld1_u8(__VA_ARGS__)\n";
-  OS << "#define svld1_u16(...) __builtin_sve_svld1_u16(__VA_ARGS__)\n";
-  OS << "#define svld1_u32(...) __builtin_sve_svld1_u32(__VA_ARGS__)\n";
-  OS << "#define svld1_u64(...) __builtin_sve_svld1_u64(__VA_ARGS__)\n";
-  OS << "#define svld1_s8(...) __builtin_sve_svld1_s8(__VA_ARGS__)\n";
-  OS << "#define svld1_s16(...) __builtin_sve_svld1_s16(__VA_ARGS__)\n";
-  OS << "#define svld1_s32(...) __builtin_sve_svld1_s32(__VA_ARGS__)\n";
-  OS << "#define svld1_s64(...) __builtin_sve_svld1_s64(__VA_ARGS__)\n";
-  OS << "#define svld1_f16(...) __builtin_sve_svld1_f16(__VA_ARGS__)\n";
-  OS << "#define svld1_f32(...) __builtin_sve_svld1_f32(__VA_ARGS__)\n";
-  OS << "#define svld1_f64(...) __builtin_sve_svld1_f64(__VA_ARGS__)\n";
+  SmallVector<std::unique_ptr<Intrinsic>, 128> Defs;
+  std::vector<Record *> RV = Records.getAllDerivedDefinitions("Inst");
+  for (auto *R : RV)
+    createIntrinsic(R, Defs);
+
+  // Sort intrinsics in header file by following order/priority:
+  // - Architectural guard (i.e. does it require SVE2 or SVE2_AES)
+  // - Class (is intrinsic overloaded or not)
+  // - Intrinsic name
+  std::stable_sort(
+      Defs.begin(), Defs.end(), [](const std::unique_ptr<Intrinsic> &A,
+                                   const std::unique_ptr<Intrinsic> &B) {
+        return A->getGuard() < B->getGuard() ||
+               (unsigned)A->getClassKind() < (unsigned)B->getClassKind() ||
+               A->getName() < B->getName();
+      });
+
+  StringRef InGuard = "";
+  for (auto &I : Defs) {
+    // Emit #endif/#if pair if needed.
+    if (I->getGuard() != InGuard) {
+      if (!InGuard.empty())
+        OS << "#endif  //" << InGuard << "\n";
+      InGuard = I->getGuard();
+      if (!InGuard.empty())
+        OS << "\n#if " << InGuard << "\n";
+    }
+
+    // Actually emit the intrinsic declaration.
+    I->emitIntrinsic(OS);
+  }
 
-  OS << "#endif /*__ARM_FEATURE_SVE */\n";
+  if (!InGuard.empty())
+    OS << "#endif  //" << InGuard << "\n";
+
+  OS << "#ifdef __cplusplus\n";
+  OS << "} // extern \"C\"\n";
+  OS << "#endif\n\n";
+  OS << "#endif /*__ARM_FEATURE_SVE */\n\n";
   OS << "#endif /* __ARM_SVE_H */\n";
 }
 
+void SVEEmitter::createBuiltins(raw_ostream &OS) {
+  std::vector<Record *> RV = Records.getAllDerivedDefinitions("Inst");
+  SmallVector<std::unique_ptr<Intrinsic>, 128> Defs;
+  for (auto *R : RV)
+    createIntrinsic(R, Defs);
+
+  // The mappings must be sorted based on BuiltinID.
+  llvm::sort(Defs, [](const std::unique_ptr<Intrinsic> &A,
+                      const std::unique_ptr<Intrinsic> &B) {
+    return A->getMangledName() < B->getMangledName();
+  });
+
+  OS << "#ifdef GET_SVE_BUILTINS\n";
+  for (auto &Def : Defs) {
+    // Only create BUILTINs for non-overloaded intrinsics, as overloaded
+    // declarations only live in the header file.
+    if (Def->getClassKind() != ClassG)
+      OS << "BUILTIN(__builtin_sve_" << Def->getMangledName() << ", \""
+         << Def->getBuiltinTypeStr() << "\", \"n\")\n";
+  }
+  OS << "#endif\n\n";
+}
+
+void SVEEmitter::createCodeGenMap(raw_ostream &OS) {
+  std::vector<Record *> RV = Records.getAllDerivedDefinitions("Inst");
+  SmallVector<std::unique_ptr<Intrinsic>, 128> Defs;
+  for (auto *R : RV)
+    createIntrinsic(R, Defs);
+
+  // The mappings must be sorted based on BuiltinID.
+  llvm::sort(Defs, [](const std::unique_ptr<Intrinsic> &A,
+                      const std::unique_ptr<Intrinsic> &B) {
+    return A->getMangledName() < B->getMangledName();
+  });
+
+  OS << "#ifdef GET_SVE_LLVM_INTRINSIC_MAP\n";
+  for (auto &Def : Defs) {
+    // Builtins only exist for non-overloaded intrinsics, overloaded
+    // declarations only live in the header file.
+    if (Def->getClassKind() == ClassG)
+      continue;
+
+    assert(!Def->isFlagSet(SVETypeFlags::EltTypeMask) &&
+           !Def->isFlagSet(SVETypeFlags::MemEltTypeMask) &&
+           "Unexpected mask value");
+    uint64_t Flags = Def->getFlags().getBits() |
+                     Def->getBaseType().getTypeFlags() |
+                     Def->getMemEltTypeEnum();
+    auto FlagString = std::to_string(Flags);
+
+    std::string LLVMName = Def->getLLVMName();
+    std::string Builtin = Def->getMangledName();
+    if (!LLVMName.empty())
+      OS << "SVEMAP1(" << Builtin << ", " << LLVMName << ", " << FlagString
+         << "),\n";
+    else
+      OS << "SVEMAP2(" << Builtin << ", " << FlagString << "),\n";
+  }
+  OS << "#endif\n\n";
+}
+
 namespace clang {
 void EmitSveHeader(RecordKeeper &Records, raw_ostream &OS) {
-  SVEEmitter().run(OS);
+  SVEEmitter(Records).createHeader(OS);
+}
+
+void EmitSveBuiltins(RecordKeeper &Records, raw_ostream &OS) {
+  SVEEmitter(Records).createBuiltins(OS);
+}
+
+void EmitSveCodeGenMap(RecordKeeper &Records, raw_ostream &OS) {
+  SVEEmitter(Records).createCodeGenMap(OS);
 }
 
 } // End namespace clang

clang/utils/TableGen/TableGen.cpp

enum ActionType {
   GenArmNeonSema,
   GenArmNeonTest,
   GenArmMveHeader,
   GenArmMveBuiltinDef,
   GenArmMveBuiltinSema,
   GenArmMveBuiltinCG,
   GenArmMveBuiltinAliases,
   GenArmSveHeader,
+  GenArmSveBuiltins,
+  GenArmSveCodeGenMap,
   GenArmCdeHeader,
   GenArmCdeBuiltinDef,
   GenArmCdeBuiltinSema,
   GenArmCdeBuiltinCG,
   GenArmCdeBuiltinAliases,
   GenAttrDocs,
   GenDiagDocs,
   GenOptDocs,
cl::values(
         clEnumValN(GenArmNeon, "gen-arm-neon", "Generate arm_neon.h for clang"),
         clEnumValN(GenArmFP16, "gen-arm-fp16", "Generate arm_fp16.h for clang"),
         clEnumValN(GenArmNeonSema, "gen-arm-neon-sema",
                    "Generate ARM NEON sema support for clang"),
         clEnumValN(GenArmNeonTest, "gen-arm-neon-test",
                    "Generate ARM NEON tests for clang"),
         clEnumValN(GenArmSveHeader, "gen-arm-sve-header",
                    "Generate arm_sve.h for clang"),
+        clEnumValN(GenArmSveBuiltins, "gen-arm-sve-builtins",
+                   "Generate arm_sve_builtins.inc for clang"),
+        clEnumValN(GenArmSveCodeGenMap, "gen-arm-sve-codegenmap",
+                   "Generate arm_sve_codegenmap.inc for clang"),
         clEnumValN(GenArmMveHeader, "gen-arm-mve-header",
                    "Generate arm_mve.h for clang"),
         clEnumValN(GenArmMveBuiltinDef, "gen-arm-mve-builtin-def",
                    "Generate ARM MVE builtin definitions for clang"),
         clEnumValN(GenArmMveBuiltinSema, "gen-arm-mve-builtin-sema",
                    "Generate ARM MVE builtin sema checks for clang"),
         clEnumValN(GenArmMveBuiltinCG, "gen-arm-mve-builtin-codegen",
                    "Generate ARM MVE builtin code-generator for clang"),
case GenArmMveBuiltinCG:
     EmitMveBuiltinCG(Records, OS);
     break;
   case GenArmMveBuiltinAliases:
     EmitMveBuiltinAliases(Records, OS);
     break;
   case GenArmSveHeader:
     EmitSveHeader(Records, OS);
     break;
+  case GenArmSveBuiltins:
+    EmitSveBuiltins(Records, OS);
+    break;
+  case GenArmSveCodeGenMap:
+    EmitSveCodeGenMap(Records, OS);
+    break;
   case GenArmCdeHeader:
     EmitCdeHeader(Records, OS);
     break;
   case GenArmCdeBuiltinDef:
     EmitCdeBuiltinDef(Records, OS);
     break;
   case GenArmCdeBuiltinSema:
     EmitCdeBuiltinSema(Records, OS);

clang/utils/TableGen/TableGenBackends.h

 void EmitFP16(llvm::RecordKeeper &Records, llvm::raw_ostream &OS);
 void EmitNeonSema(llvm::RecordKeeper &Records, llvm::raw_ostream &OS);
 void EmitNeonTest(llvm::RecordKeeper &Records, llvm::raw_ostream &OS);
 void EmitNeon2(llvm::RecordKeeper &Records, llvm::raw_ostream &OS);
 void EmitNeonSema2(llvm::RecordKeeper &Records, llvm::raw_ostream &OS);
 void EmitNeonTest2(llvm::RecordKeeper &Records, llvm::raw_ostream &OS);
 
 void EmitSveHeader(llvm::RecordKeeper &Records, llvm::raw_ostream &OS);
+void EmitSveBuiltins(llvm::RecordKeeper &Records, llvm::raw_ostream &OS);
+void EmitSveCodeGenMap(llvm::RecordKeeper &Records, llvm::raw_ostream &OS);
 
 void EmitMveHeader(llvm::RecordKeeper &Records, llvm::raw_ostream &OS);
 void EmitMveBuiltinDef(llvm::RecordKeeper &Records, llvm::raw_ostream &OS);
 void EmitMveBuiltinSema(llvm::RecordKeeper &Records, llvm::raw_ostream &OS);
 void EmitMveBuiltinCG(llvm::RecordKeeper &Records, llvm::raw_ostream &OS);
 void EmitMveBuiltinAliases(llvm::RecordKeeper &Records, llvm::raw_ostream &OS);
 
 void EmitCdeHeader(llvm::RecordKeeper &Records, llvm::raw_ostream &OS);