[AArch64] Make ACLE intrinsics always available part1

[danielkiss]

Summary

A given arch feature might enabled by a pragma or a function attribute so in this cases would be nice to use intrinsics.
Today GCC offers the intrinsics without the march flag[1].
PR[2] for ACLE to clarify the intention and remove the need for -march flag for a given intrinsics.

This is going to be more useful when D127812 lands.

[1] https://godbolt.org/z/bxcMhav3z
[2] https://github.com/ARM-software/acle/pull/214

[samparker]

So, what kind of errors will the user get now if try to use an unsupported instruction? I'm assuming that it fails during isel and that doesn't sound like a good user experience.

[danielkiss]

In D133359#3776503, @samparker wrote:

So, what kind of errors will the user get now if try to use an unsupported instruction? I'm assuming that it fails during isel and that doesn't sound like a good user experience.

I'm adding target warnings. so the user will get "error: always_inline function 'foo' requires target feature 'x', but would be inlined into function 'bar' that is compiled without support for 'x'.

[danielkiss]

Armv8.3-A Javascript conversion and Armv8.5-A FP rounding don't have feature flag.
One option is to introduce feature flags or the the arch version could be used.
I'm looking comment for the arch version solution.

[dmgreen]

In D133359#3786469, @danielkiss wrote:

Armv8.3-A Javascript conversion and Armv8.5-A FP rounding don't have feature flag.
One option is to introduce feature flags or the the arch version could be used.
I'm looking comment for the arch version solution.

I'm looking into a similar patch for the parts of arm_neon.h that are not already handled, and was thinking along the same lines. The idea at current is to add support for arch= target attributes, as in https://gcc.gnu.org/onlinedocs/gcc/AArch64-Function-Attributes.html#AArch64-Function-Attributes. And then using "v8.1a" (rdma), "v8.3a" (complex) or "v8.5a" (frint) for the TARGET_BUILTIN, like you have here. I think that is fine for things internal to the compiler. The user can then use "arch=..." target attributes to specify different architecture versions. I'll try and put that part up for review.

What is you plan for AArch32? Keep it behind the same ifdef as before?

clang/lib/Basic/Targets/AArch64.cpp

529

I had a (completely untested) version of this function doing the same thing. I was basing it on string processing, though, so we didn't have to add new code for each new version of the architecture. I'm not sure if it was a good idea or not.

clang/test/Sema/aarch64-tme-errors.c

6

Does this get a "failure to inline" error now?

clang/test/Sema/builtins-arm64-mte.c

8

I think these should still be testing the ACLE intrinsics, not the builtins, if that is what we expect the users to use. Are the error's now different?

[dmgreen]

clang/lib/Basic/Targets/AArch64.cpp

529

It looked like this, by the way:

if (Enabled && Name == "v9.0a")                                              
  setFeatureEnabled(Features, "v8.5a", true);                                
else if (Enabled && Name != "v8.1a" &&                                       
         (Name.startswith("v8.") || Name.startswith("v9.")) &&               
         Name.endswith("a") &&                                               
         atoi(Name.substr(3, Name.size() - 1).data()) >= 1) {                
  SmallString<16> Storage;                                                   
  Twine NewName =                                                            
      Name.take_front(3) +                                                   
      std::to_string(atoi(Name.substr(3, Name.size() - 1).data()) - 1) + "a";
  setFeatureEnabled(Features, NewName.toStringRef(Storage), true);           
}

It is untested, apart from some very simple checks, doesn't handle errors and doesn't handle all the 9 feautres properly. It may be better not to be so recursive too if it can just enable multiple features at once.

[tschuett]

This looks like you want an enum over all versions with string2enum, enum2string, and comparison operations on the enum.

[compnerd]

clang/lib/Basic/Targets/AArch64.cpp

535–564

I think that we can compress some of the logic here. But I also wonder if we can write this in a different way: rather than creating a small vector and resizing it, can we actually avoid that all together and create a static constant data blob that we can iterate?

const char * const kv8Architectures[] = { "v8a", "v8.1a", "v8.2a", "v8.3a", "v8.4a", "v8.5a", "v8.6a", "v8.7a" };

unsigned major, minor;
std::tie(major, minor) = [parse_version](Name);
size_t v8Revisions = major == 8 ? minor : 4 + minor;
size_t v9Revisions = major == 9 : minor : 0;

The bit in the middle handwaves over the version parsing, but should hopefully convey the general idea.

[danielkiss]

Moved not ACLE related changes to D134349 and D134353.
This to be rebased on top of those and D134127.

clang/test/Sema/builtins-arm64-mte.c

8

These are custom errors see here e.g. that can't be triggered via the intrinsics anymore because we stop earlier.

[danielkiss]

Arm part will be a different patch since it is more complex to decide the support of a given feature. (e.g. ((CPUProfile != "M" && ArchVersion >= 6) || (CPUProfile == "M" && DSP))))

clang/lib/Basic/Targets/AArch64.cpp

535–564

solved in D134353.

clang/test/Sema/aarch64-tme-errors.c

6

Yes, but -verify does not go that far.

clang/test/Sema/builtins-arm64-mte.c

8

added test for intrinsics besides the builtins

[dmgreen]

Can you split the "mte" and "tme" parts away from the rest? I'm not sure if there are any problems with those part, but they are more involved and I am less familiar with it. It would be good to get a lot of the simpler parts in and out of the way first.

[danielkiss]

MTE/TME parts are removed, they will get their own patch.

[dmgreen]

Thanks. That's great. LGTM

clang/include/clang/Basic/BuiltinsAArch64.def

 BUILTIN(__builtin_arm_nop, "v", "")
 BUILTIN(__builtin_arm_yield, "v", "")
 BUILTIN(__builtin_arm_wfe, "v", "")
 BUILTIN(__builtin_arm_wfi, "v", "")
 BUILTIN(__builtin_arm_sev, "v", "")
 BUILTIN(__builtin_arm_sevl, "v", "")
 
 // CRC32
-BUILTIN(__builtin_arm_crc32b, "UiUiUc", "nc")
-BUILTIN(__builtin_arm_crc32cb, "UiUiUc", "nc")
-BUILTIN(__builtin_arm_crc32h, "UiUiUs", "nc")
-BUILTIN(__builtin_arm_crc32ch, "UiUiUs", "nc")
-BUILTIN(__builtin_arm_crc32w, "UiUiUi", "nc")
-BUILTIN(__builtin_arm_crc32cw, "UiUiUi", "nc")
-BUILTIN(__builtin_arm_crc32d, "UiUiWUi", "nc")
-BUILTIN(__builtin_arm_crc32cd, "UiUiWUi", "nc")
+TARGET_BUILTIN(__builtin_arm_crc32b, "UiUiUc", "nc", "crc")
+TARGET_BUILTIN(__builtin_arm_crc32cb, "UiUiUc", "nc", "crc")
+TARGET_BUILTIN(__builtin_arm_crc32h, "UiUiUs", "nc", "crc")
+TARGET_BUILTIN(__builtin_arm_crc32ch, "UiUiUs", "nc", "crc")
+TARGET_BUILTIN(__builtin_arm_crc32w, "UiUiUi", "nc", "crc")
+TARGET_BUILTIN(__builtin_arm_crc32cw, "UiUiUi", "nc", "crc")
+TARGET_BUILTIN(__builtin_arm_crc32d, "UiUiWUi", "nc", "crc")
+TARGET_BUILTIN(__builtin_arm_crc32cd, "UiUiWUi", "nc", "crc")
 
 // Memory Tagging Extensions (MTE)
-BUILTIN(__builtin_arm_irg, "v*v*Ui", "t")
-BUILTIN(__builtin_arm_addg, "v*v*Ui", "t")
-BUILTIN(__builtin_arm_gmi, "Uiv*Ui", "t")
-BUILTIN(__builtin_arm_ldg, "v*v*", "t")
-BUILTIN(__builtin_arm_stg, "vv*", "t")
-BUILTIN(__builtin_arm_subp, "Uiv*v*", "t")
+TARGET_BUILTIN(__builtin_arm_irg, "v*v*Ui", "t", "mte")
+TARGET_BUILTIN(__builtin_arm_addg, "v*v*Ui", "t", "mte")
+TARGET_BUILTIN(__builtin_arm_gmi, "Uiv*Ui", "t", "mte")
+TARGET_BUILTIN(__builtin_arm_ldg, "v*v*", "t", "mte")
+TARGET_BUILTIN(__builtin_arm_stg, "vv*", "t", "mte")
+TARGET_BUILTIN(__builtin_arm_subp, "Uiv*v*", "t", "mte")
 
 // Memory Operations
-BUILTIN(__builtin_arm_mops_memset_tag, "v*v*iz", "")
+TARGET_BUILTIN(__builtin_arm_mops_memset_tag, "v*v*iz", "", "mte,mops")
 
 // Memory barrier
 BUILTIN(__builtin_arm_dmb, "vUi", "nc")
 BUILTIN(__builtin_arm_dsb, "vUi", "nc")
 BUILTIN(__builtin_arm_isb, "vUi", "nc")
 
-BUILTIN(__builtin_arm_jcvt, "Zid", "nc")
+TARGET_BUILTIN(__builtin_arm_jcvt, "Zid", "nc", "v8.3a")
 
 // Prefetch
 BUILTIN(__builtin_arm_prefetch, "vvC*UiUiUiUi", "nc")
 
 // System Registers
 BUILTIN(__builtin_arm_rsr, "UicC*", "nc")
 BUILTIN(__builtin_arm_rsr64, "WUicC*", "nc")
 BUILTIN(__builtin_arm_rsrp, "v*cC*", "nc")

 LANGBUILTIN(__wfi,   "v", "",   ALL_MS_LANGUAGES)
 LANGBUILTIN(__sev,   "v", "",   ALL_MS_LANGUAGES)
 LANGBUILTIN(__sevl,  "v", "",   ALL_MS_LANGUAGES)
 
 // Misc
 BUILTIN(__builtin_sponentry, "v*", "c")
 
 // Transactional Memory Extension
-BUILTIN(__builtin_arm_tstart, "WUi", "nj")
-BUILTIN(__builtin_arm_tcommit, "v", "n")
-BUILTIN(__builtin_arm_tcancel, "vWUIi", "n")
-BUILTIN(__builtin_arm_ttest, "WUi", "nc")
+TARGET_BUILTIN(__builtin_arm_tstart, "WUi", "nj", "tme")
+TARGET_BUILTIN(__builtin_arm_tcommit, "v", "n", "tme")
+TARGET_BUILTIN(__builtin_arm_tcancel, "vWUIi", "n", "tme")
+TARGET_BUILTIN(__builtin_arm_ttest, "WUi", "nc", "tme")
 
 // Armv8.5-A FP rounding intrinsics
-BUILTIN(__builtin_arm_frint32zf, "ff", "")
-BUILTIN(__builtin_arm_frint32z, "dd", "")
-BUILTIN(__builtin_arm_frint64zf, "ff", "")
-BUILTIN(__builtin_arm_frint64z, "dd", "")
-BUILTIN(__builtin_arm_frint32xf, "ff", "")
-BUILTIN(__builtin_arm_frint32x, "dd", "")
-BUILTIN(__builtin_arm_frint64xf, "ff", "")
-BUILTIN(__builtin_arm_frint64x, "dd", "")
+TARGET_BUILTIN(__builtin_arm_frint32zf, "ff", "", "v8.5a")
+TARGET_BUILTIN(__builtin_arm_frint32z, "dd", "", "v8.5a")
+TARGET_BUILTIN(__builtin_arm_frint64zf, "ff", "", "v8.5a")
+TARGET_BUILTIN(__builtin_arm_frint64z, "dd", "", "v8.5a")
+TARGET_BUILTIN(__builtin_arm_frint32xf, "ff", "", "v8.5a")
+TARGET_BUILTIN(__builtin_arm_frint32x, "dd", "", "v8.5a")
+TARGET_BUILTIN(__builtin_arm_frint64xf, "ff", "", "v8.5a")
+TARGET_BUILTIN(__builtin_arm_frint64x, "dd", "", "v8.5a")
 
 // Armv8.5-A Random number generation intrinsics
-BUILTIN(__builtin_arm_rndr,   "iWUi*", "n")
-BUILTIN(__builtin_arm_rndrrs, "iWUi*", "n")
+TARGET_BUILTIN(__builtin_arm_rndr,   "iWUi*", "n", "rand")
+TARGET_BUILTIN(__builtin_arm_rndrrs, "iWUi*", "n", "rand")
 
 // Armv8.7-A load/store 64-byte intrinsics
-BUILTIN(__builtin_arm_ld64b, "vvC*WUi*", "n")
-BUILTIN(__builtin_arm_st64b, "vv*WUiC*", "n")
-BUILTIN(__builtin_arm_st64bv, "WUiv*WUiC*", "n")
-BUILTIN(__builtin_arm_st64bv0, "WUiv*WUiC*", "n")
+TARGET_BUILTIN(__builtin_arm_ld64b, "vvC*WUi*", "n", "ls64")
+TARGET_BUILTIN(__builtin_arm_st64b, "vv*WUiC*", "n", "ls64")
+TARGET_BUILTIN(__builtin_arm_st64bv, "WUiv*WUiC*", "n", "ls64")
+TARGET_BUILTIN(__builtin_arm_st64bv0, "WUiv*WUiC*", "n", "ls64")
 
 TARGET_HEADER_BUILTIN(_BitScanForward, "UcUNi*UNi", "nh", "intrin.h", ALL_MS_LANGUAGES, "")
 TARGET_HEADER_BUILTIN(_BitScanReverse, "UcUNi*UNi", "nh", "intrin.h", ALL_MS_LANGUAGES, "")
 TARGET_HEADER_BUILTIN(_BitScanForward64, "UcUNi*ULLi", "nh", "intrin.h", ALL_MS_LANGUAGES, "")
 TARGET_HEADER_BUILTIN(_BitScanReverse64, "UcUNi*ULLi", "nh", "intrin.h", ALL_MS_LANGUAGES, "")
 
 TARGET_HEADER_BUILTIN(_InterlockedAdd,           "NiNiD*Ni",    "nh", "intrin.h", ALL_MS_LANGUAGES, "")
 TARGET_HEADER_BUILTIN(_InterlockedAnd64,         "LLiLLiD*LLi", "nh", "intrin.h", ALL_MS_LANGUAGES, "")

clang/include/clang/Basic/TargetBuiltins.h

#include "clang/Basic/BuiltinsSVE.def"
   /// AArch64 builtins
   namespace AArch64 {
   enum {
     LastTIBuiltin = clang::Builtin::FirstTSBuiltin - 1,
     LastNEONBuiltin = NEON::FirstTSBuiltin - 1,
     FirstSVEBuiltin = NEON::FirstTSBuiltin,
     LastSVEBuiltin = SVE::FirstTSBuiltin - 1,
   #define BUILTIN(ID, TYPE, ATTRS) BI##ID,
+  #define TARGET_BUILTIN(ID, TYPE, ATTRS, FEATURE) BI##ID,
   #include "clang/Basic/BuiltinsAArch64.def"
     LastTSBuiltin
   };
   }
 
   /// BPF builtins
   namespace BPF {
   enum {

clang/lib/Basic/Targets/AArch64.h

void getTargetDefines(const LangOptions &Opts,
                         MacroBuilder &Builder) const override;
 
   ArrayRef<Builtin::Info> getTargetBuiltins() const override;
 
   Optional<std::pair<unsigned, unsigned>>
   getVScaleRange(const LangOptions &LangOpts) const override;
 
   bool hasFeature(StringRef Feature) const override;
+  void setFeatureEnabled(llvm::StringMap<bool> &Features, StringRef Name,
+                         bool Enabled) const override;
   bool handleTargetFeatures(std::vector<std::string> &Features,
                             DiagnosticsEngine &Diags) override;
 
   bool hasBFloat16Type() const override;
 
   CallingConvCheckResult checkCallingConvention(CallingConv CC) const override;
 
   bool isCLZForZeroUndef() const override;

clang/lib/Basic/Targets/AArch64.cpp

 #include "clang/Basic/BuiltinsNEON.def"
 
 #define BUILTIN(ID, TYPE, ATTRS)                                               \
    {#ID, TYPE, ATTRS, nullptr, ALL_LANGUAGES, nullptr},
 #include "clang/Basic/BuiltinsSVE.def"
 
 #define BUILTIN(ID, TYPE, ATTRS)                                               \
    {#ID, TYPE, ATTRS, nullptr, ALL_LANGUAGES, nullptr},
+#define TARGET_BUILTIN(ID, TYPE, ATTRS, FEATURE)                               \
+   {#ID, TYPE, ATTRS, nullptr, ALL_LANGUAGES, FEATURE},
 #define LANGBUILTIN(ID, TYPE, ATTRS, LANG)                                     \
   {#ID, TYPE, ATTRS, nullptr, LANG, nullptr},
 #define TARGET_HEADER_BUILTIN(ID, TYPE, ATTRS, HEADER, LANGS, FEATURE)         \
   {#ID, TYPE, ATTRS, HEADER, LANGS, FEATURE},
 #include "clang/Basic/BuiltinsAArch64.def"
 };
 
 static StringRef getArchVersionString(llvm::AArch64::ArchKind Kind) {
bool AArch64TargetInfo::hasFeature(StringRef Feature) const {
   return llvm::StringSwitch<bool>(Feature)
     .Cases("aarch64", "arm64", "arm", true)
     .Case("neon", FPU & NeonMode)
     .Cases("sve", "sve2", "sve2-bitperm", "sve2-aes", "sve2-sha3", "sve2-sm4", "f64mm", "f32mm", "i8mm", "bf16", FPU & SveMode)
     .Case("ls64", HasLS64)
     .Default(false);
 }
 
+void AArch64TargetInfo::setFeatureEnabled(llvm::StringMap<bool> &Features,
+                                          StringRef Name, bool Enabled) const {
+  Features[Name] = Enabled;
+  SmallVector<StringRef> PredecessorsArch8 = {
+      "v8a", "v8.1a", "v8.2a", "v8.3a", "v8.4a", "v8.5a", "v8.6a", "v8.7a"};
+  SmallVector<StringRef> PredecessorsArch9 = {"v9a", "v9.1a", "v9.2a", "v9.3a"};
+  if (Name == "v8.1a")
+    PredecessorsArch8.resize(1);
+  else if (Name == "v8.2a")
+    PredecessorsArch8.resize(2);
+  else if (Name == "v8.3a")
+    PredecessorsArch8.resize(3);
+  else if (Name == "v8.4a")
+    PredecessorsArch8.resize(4);
+  else if (Name == "v8.5a")
+    PredecessorsArch8.resize(5);
+  else if (Name == "v8.6a")
+    PredecessorsArch8.resize(6);
+  else if (Name == "v8.7a")
+    PredecessorsArch8.resize(7);
+  else if (Name == "v8.8a")
+    PredecessorsArch8.resize(8);
+  else
+    PredecessorsArch8.resize(0);
+
+  if (Name == "v9.1a") {
+    PredecessorsArch8.resize(5);
+    PredecessorsArch9.resize(1);
+  } else if (Name == "v9.2a") {
+    PredecessorsArch8.resize(6);
+    PredecessorsArch9.resize(2);
+  } else if (Name == "v9.3a") {
+    PredecessorsArch8.resize(7);
+    PredecessorsArch9.resize(3);
+  } else
+    PredecessorsArch9.resize(0);
+
+  for (auto n : PredecessorsArch8)
+    Features[n] = Enabled;
+  for (auto n : PredecessorsArch9)
+    Features[n] = Enabled;
+}
+
 bool AArch64TargetInfo::handleTargetFeatures(std::vector<std::string> &Features,
                                              DiagnosticsEngine &Diags) {
   FPU = FPUMode;
   HasCRC = false;
   HasAES = false;
   HasSHA2 = false;
   HasSHA3 = false;
   HasSM4 = false;
for (const auto &Feature : Features) {
     if (Feature == "+sha3") {
       HasSHA2 = true;
       HasSHA3 = true;
     }
     if (Feature == "+sm4")
       HasSM4 = true;
     if (Feature == "+strict-align")
       HasUnaligned = false;
-    if (Feature == "+v8a")
+    if (Feature == "+v8a" && ArchKind < llvm::AArch64::ArchKind::ARMV8A)
       ArchKind = llvm::AArch64::ArchKind::ARMV8A;
-    if (Feature == "+v8.1a")
+    if (Feature == "+v8.1a" && ArchKind < llvm::AArch64::ArchKind::ARMV8_1A)
       ArchKind = llvm::AArch64::ArchKind::ARMV8_1A;
-    if (Feature == "+v8.2a")
+    if (Feature == "+v8.2a" && ArchKind < llvm::AArch64::ArchKind::ARMV8_2A)
       ArchKind = llvm::AArch64::ArchKind::ARMV8_2A;
-    if (Feature == "+v8.3a")
+    if (Feature == "+v8.3a" && ArchKind < llvm::AArch64::ArchKind::ARMV8_3A)
       ArchKind = llvm::AArch64::ArchKind::ARMV8_3A;
-    if (Feature == "+v8.4a")
+    if (Feature == "+v8.4a" && ArchKind < llvm::AArch64::ArchKind::ARMV8_4A)
       ArchKind = llvm::AArch64::ArchKind::ARMV8_4A;
-    if (Feature == "+v8.5a")
+    if (Feature == "+v8.5a" && ArchKind < llvm::AArch64::ArchKind::ARMV8_5A)
       ArchKind = llvm::AArch64::ArchKind::ARMV8_5A;
-    if (Feature == "+v8.6a")
+    if (Feature == "+v8.6a" && ArchKind < llvm::AArch64::ArchKind::ARMV8_6A)
       ArchKind = llvm::AArch64::ArchKind::ARMV8_6A;
-    if (Feature == "+v8.7a")
+    if (Feature == "+v8.7a" && ArchKind < llvm::AArch64::ArchKind::ARMV8_7A)
       ArchKind = llvm::AArch64::ArchKind::ARMV8_7A;
-    if (Feature == "+v8.8a")
+    if (Feature == "+v8.8a" && ArchKind < llvm::AArch64::ArchKind::ARMV8_8A)
       ArchKind = llvm::AArch64::ArchKind::ARMV8_8A;
-    if (Feature == "+v9a")
+    if (Feature == "+v9a" && ArchKind < llvm::AArch64::ArchKind::ARMV9A)
       ArchKind = llvm::AArch64::ArchKind::ARMV9A;
-    if (Feature == "+v9.1a")
+    if (Feature == "+v9.1a" && ArchKind < llvm::AArch64::ArchKind::ARMV9_1A)
       ArchKind = llvm::AArch64::ArchKind::ARMV9_1A;
-    if (Feature == "+v9.2a")
+    if (Feature == "+v9.2a" && ArchKind < llvm::AArch64::ArchKind::ARMV9_2A)
       ArchKind = llvm::AArch64::ArchKind::ARMV9_2A;
-    if (Feature == "+v9.3a")
+    if (Feature == "+v9.3a" && ArchKind < llvm::AArch64::ArchKind::ARMV9_3A)
       ArchKind = llvm::AArch64::ArchKind::ARMV9_3A;
     if (Feature == "+v8r")
       ArchKind = llvm::AArch64::ArchKind::ARMV8R;
     if (Feature == "+fullfp16")
       HasFullFP16 = true;
     if (Feature == "+dotprod")
       HasDotProd = true;
     if (Feature == "+fp16fml")

clang/lib/Driver/ToolChains/Arch/AArch64.cpp

void aarch64::getAArch64TargetFeatures(const Driver &D,
   bool HasNoCrypto = false;
   int FullFP16Pos = -1;
   int NoFullFP16Pos = -1;
   int FP16FMLPos = -1;
   int NoFP16FMLPos = -1;
   int ArchFeatPos = -1;
 
   for (auto I = Features.begin(), E = Features.end(); I != E; I++) {
-    if (*I == "+v8a")   V8Version = 0;
-    else if (*I == "+v8.1a") V8Version = 1;
-    else if (*I == "+v8.2a") V8Version = 2;
-    else if (*I == "+v8.3a") V8Version = 3;
-    else if (*I == "+v8.4a") V8Version = 4;
-    else if (*I == "+v8.5a") V8Version = 5;
-    else if (*I == "+v8.6a") V8Version = 6;
-    else if (*I == "+v8.7a") V8Version = 7;
-    else if (*I == "+v8.8a") V8Version = 8;
-    else if (*I == "+v8.9a") V8Version = 9;
-    else if (*I == "+v9a")   V9Version = 0;
-    else if (*I == "+v9.1a") V9Version = 1;
-    else if (*I == "+v9.2a") V9Version = 2;
-    else if (*I == "+v9.3a") V9Version = 3;
-    else if (*I == "+v9.4a") V9Version = 4;
+    if (*I == "+v8a" && V8Version < 0)
+      V8Version = 0;
+    else if (*I == "+v8.1a" && V8Version < 1)
+      V8Version = 1;
+    else if (*I == "+v8.2a" && V8Version < 2)
+      V8Version = 2;
+    else if (*I == "+v8.3a" && V8Version < 3)
+      V8Version = 3;
+    else if (*I == "+v8.4a" && V8Version < 4)
+      V8Version = 4;
+    else if (*I == "+v8.5a" && V8Version < 5)
+      V8Version = 5;
+    else if (*I == "+v8.6a" && V8Version < 6)
+      V8Version = 6;
+    else if (*I == "+v8.7a" && V8Version < 7)
+      V8Version = 7;
+    else if (*I == "+v8.8a" && V8Version < 8)
+      V8Version = 8;
+    else if (*I == "+v8.9a" && V8Version < 9)
+      V8Version = 9;
+    else if (*I == "+v9a" && V9Version < 0)
+      V9Version = 0;
+    else if (*I == "+v9.1a" && V9Version < 1)
+      V9Version = 1;
+    else if (*I == "+v9.2a" && V9Version < 2)
+      V9Version = 2;
+    else if (*I == "+v9.3a" && V9Version < 3)
+      V9Version = 3;
+    else if (*I == "+v9.4a" && V9Version < 4)
+      V9Version = 4;
     else if (*I == "+sm4")  HasSM4 = true;
     else if (*I == "+sha3") HasSHA3 = true;
     else if (*I == "+sha2") HasSHA2 = true;
     else if (*I == "+aes")  HasAES = true;
     else if (*I == "-sm4")  HasNoSM4 = true;
     else if (*I == "-sha3") HasNoSHA3 = true;
     else if (*I == "-sha2") HasNoSHA2 = true;
     else if (*I == "-aes")  HasNoAES = true;

clang/lib/Headers/arm_acle.h

 /*===---- arm_acle.h - ARM Non-Neon intrinsics -----------------------------===
  *
  * Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
  * See https://llvm.org/LICENSE.txt for license information.
  * SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
  *
  *===-----------------------------------------------------------------------===
  */
 
 #ifndef __ARM_ACLE_H
 #define __ARM_ACLE_H
 
 #ifndef __ARM_ACLE
 #error "ACLE intrinsics support not enabled."
 #endif
 
+#if defined(__ARM_64BIT_STATE) && __ARM_64BIT_STATE
+#include <stddef.h>
+#endif
 #include <stdint.h>
 
 #if defined(__cplusplus)
 extern "C" {
 #endif
 
 /* 8 SYNCHRONIZATION, BARRIER AND HINT INTRINSICS */
 /* 8.3 Memory barriers */

 }
 static __inline__ int32_t __attribute__((__always_inline__, __nodebug__))
 __smusdx(int16x2_t __a, int16x2_t __b) {
   return __builtin_arm_smusdx(__a, __b);
 }
 #endif
 
 /* 9.7 CRC32 intrinsics */
-#if defined(__ARM_FEATURE_CRC32) && __ARM_FEATURE_CRC32
-static __inline__ uint32_t __attribute__((__always_inline__, __nodebug__))
+#if (defined(__ARM_32BIT_STATE) && __ARM_32BIT_STATE && defined(__ARM_FEATURE_CRC32) && __ARM_FEATURE_CRC32) || \
+     defined(__ARM_64BIT_STATE) && __ARM_64BIT_STATE
+static __inline__ uint32_t __attribute__((__always_inline__, __nodebug__, target("crc")))
 __crc32b(uint32_t __a, uint8_t __b) {
   return __builtin_arm_crc32b(__a, __b);
 }
 
-static __inline__ uint32_t __attribute__((__always_inline__, __nodebug__))
+static __inline__ uint32_t __attribute__((__always_inline__, __nodebug__, target("crc")))
 __crc32h(uint32_t __a, uint16_t __b) {
   return __builtin_arm_crc32h(__a, __b);
 }
 
-static __inline__ uint32_t __attribute__((__always_inline__, __nodebug__))
+static __inline__ uint32_t __attribute__((__always_inline__, __nodebug__, target("crc")))
 __crc32w(uint32_t __a, uint32_t __b) {
   return __builtin_arm_crc32w(__a, __b);
 }
 
-static __inline__ uint32_t __attribute__((__always_inline__, __nodebug__))
+static __inline__ uint32_t __attribute__((__always_inline__, __nodebug__, target("crc")))
 __crc32d(uint32_t __a, uint64_t __b) {
   return __builtin_arm_crc32d(__a, __b);
 }
 
-static __inline__ uint32_t __attribute__((__always_inline__, __nodebug__))
+static __inline__ uint32_t __attribute__((__always_inline__, __nodebug__, target("crc")))
 __crc32cb(uint32_t __a, uint8_t __b) {
   return __builtin_arm_crc32cb(__a, __b);
 }
 
-static __inline__ uint32_t __attribute__((__always_inline__, __nodebug__))
+static __inline__ uint32_t __attribute__((__always_inline__, __nodebug__, target("crc")))
 __crc32ch(uint32_t __a, uint16_t __b) {
   return __builtin_arm_crc32ch(__a, __b);
 }
 
-static __inline__ uint32_t __attribute__((__always_inline__, __nodebug__))
+static __inline__ uint32_t __attribute__((__always_inline__, __nodebug__, target("crc")))
 __crc32cw(uint32_t __a, uint32_t __b) {
   return __builtin_arm_crc32cw(__a, __b);
 }
 
-static __inline__ uint32_t __attribute__((__always_inline__, __nodebug__))
+static __inline__ uint32_t __attribute__((__always_inline__, __nodebug__, target("crc")))
 __crc32cd(uint32_t __a, uint64_t __b) {
   return __builtin_arm_crc32cd(__a, __b);
 }
 #endif
 
 /* Armv8.3-A Javascript conversion intrinsic */
-#if defined(__ARM_64BIT_STATE) && __ARM_64BIT_STATE && defined(__ARM_FEATURE_JCVT)
-static __inline__ int32_t __attribute__((__always_inline__, __nodebug__))
+#if defined(__ARM_64BIT_STATE) && __ARM_64BIT_STATE
+static __inline__ int32_t __attribute__((__always_inline__, __nodebug__, target("v8.3a")))
 __jcvt(double __a) {
   return __builtin_arm_jcvt(__a);
 }
 #endif
 
 /* Armv8.5-A FP rounding intrinsics */
-#if defined(__ARM_64BIT_STATE) && __ARM_64BIT_STATE && defined(__ARM_FEATURE_FRINT)
-static __inline__ float __attribute__((__always_inline__, __nodebug__))
+#if defined(__ARM_64BIT_STATE) && __ARM_64BIT_STATE
+static __inline__ float __attribute__((__always_inline__, __nodebug__, target("v8.5a")))
 __frint32zf(float __a) {
   return __builtin_arm_frint32zf(__a);
 }
 
-static __inline__ double __attribute__((__always_inline__, __nodebug__))
+static __inline__ double __attribute__((__always_inline__, __nodebug__, target("v8.5a")))
 __frint32z(double __a) {
   return __builtin_arm_frint32z(__a);
 }
 
-static __inline__ float __attribute__((__always_inline__, __nodebug__))
+static __inline__ float __attribute__((__always_inline__, __nodebug__, target("v8.5a")))
 __frint64zf(float __a) {
   return __builtin_arm_frint64zf(__a);
 }
 
-static __inline__ double __attribute__((__always_inline__, __nodebug__))
+static __inline__ double __attribute__((__always_inline__, __nodebug__, target("v8.5a")))
 __frint64z(double __a) {
   return __builtin_arm_frint64z(__a);
 }
 
-static __inline__ float __attribute__((__always_inline__, __nodebug__))
+static __inline__ float __attribute__((__always_inline__, __nodebug__, target("v8.5a")))
 __frint32xf(float __a) {
   return __builtin_arm_frint32xf(__a);
 }
 
-static __inline__ double __attribute__((__always_inline__, __nodebug__))
+static __inline__ double __attribute__((__always_inline__, __nodebug__, target("v8.5a")))
 __frint32x(double __a) {
   return __builtin_arm_frint32x(__a);
 }
 
-static __inline__ float __attribute__((__always_inline__, __nodebug__))
+static __inline__ float __attribute__((__always_inline__, __nodebug__, target("v8.5a")))
 __frint64xf(float __a) {
   return __builtin_arm_frint64xf(__a);
 }
 
-static __inline__ double __attribute__((__always_inline__, __nodebug__))
+static __inline__ double __attribute__((__always_inline__, __nodebug__, target("v8.5a")))
 __frint64x(double __a) {
   return __builtin_arm_frint64x(__a);
 }
 #endif
 
 /* Armv8.7-A load/store 64-byte intrinsics */
-#if defined(__ARM_64BIT_STATE) && __ARM_64BIT_STATE && defined(__ARM_FEATURE_LS64)
+#if defined(__ARM_64BIT_STATE) && __ARM_64BIT_STATE
 typedef struct {
     uint64_t val[8];
 } data512_t;
 
-static __inline__ data512_t __attribute__((__always_inline__, __nodebug__))
+static __inline__ data512_t __attribute__((__always_inline__, __nodebug__, target("ls64")))
 __arm_ld64b(const void *__addr) {
     data512_t __value;
     __builtin_arm_ld64b(__addr, __value.val);
     return __value;
 }
-static __inline__ void __attribute__((__always_inline__, __nodebug__))
+static __inline__ void __attribute__((__always_inline__, __nodebug__, target("ls64")))
 __arm_st64b(void *__addr, data512_t __value) {
     __builtin_arm_st64b(__addr, __value.val);
 }
-static __inline__ uint64_t __attribute__((__always_inline__, __nodebug__))
+static __inline__ uint64_t __attribute__((__always_inline__, __nodebug__, target("ls64")))
 __arm_st64bv(void *__addr, data512_t __value) {
     return __builtin_arm_st64bv(__addr, __value.val);
 }
-static __inline__ uint64_t __attribute__((__always_inline__, __nodebug__))
+static __inline__ uint64_t __attribute__((__always_inline__, __nodebug__, target("ls64")))
 __arm_st64bv0(void *__addr, data512_t __value) {
     return __builtin_arm_st64bv0(__addr, __value.val);
 }
 #endif
 
 /* 10.1 Special register intrinsics */
 #define __arm_rsr(sysreg) __builtin_arm_rsr(sysreg)
 #define __arm_rsr64(sysreg) __builtin_arm_rsr64(sysreg)
 #define __arm_rsrp(sysreg) __builtin_arm_rsrp(sysreg)
 #define __arm_rsrf(sysreg) __builtin_bit_cast(float, __arm_rsr(sysreg))
 #define __arm_rsrf64(sysreg) __builtin_bit_cast(double, __arm_rsr64(sysreg))
 #define __arm_wsr(sysreg, v) __builtin_arm_wsr(sysreg, v)
 #define __arm_wsr64(sysreg, v) __builtin_arm_wsr64(sysreg, v)
 #define __arm_wsrp(sysreg, v) __builtin_arm_wsrp(sysreg, v)
 #define __arm_wsrf(sysreg, v) __arm_wsr(sysreg, __builtin_bit_cast(uint32_t, v))
 #define __arm_wsrf64(sysreg, v) __arm_wsr64(sysreg, __builtin_bit_cast(uint64_t, v))
 
 /* Memory Tagging Extensions (MTE) Intrinsics */
-#if defined(__ARM_FEATURE_MEMORY_TAGGING) && __ARM_FEATURE_MEMORY_TAGGING
-#define __arm_mte_create_random_tag(__ptr, __mask)  __builtin_arm_irg(__ptr, __mask)
-#define __arm_mte_increment_tag(__ptr, __tag_offset)  __builtin_arm_addg(__ptr, __tag_offset)
-#define __arm_mte_exclude_tag(__ptr, __excluded)  __builtin_arm_gmi(__ptr, __excluded)
-#define __arm_mte_get_tag(__ptr) __builtin_arm_ldg(__ptr)
-#define __arm_mte_set_tag(__ptr) __builtin_arm_stg(__ptr)
-#define __arm_mte_ptrdiff(__ptra, __ptrb) __builtin_arm_subp(__ptra, __ptrb)
-#endif
+#if defined(__ARM_64BIT_STATE) && __ARM_64BIT_STATE
+static __inline__ void* __attribute__((__always_inline__, __nodebug__, target("mte")))
+__arm_mte_create_random_tag(void *__ptr, uint64_t __mask) {
+    return __builtin_arm_irg(__ptr, __mask);
+}
+static __inline__ void*  __attribute__((__always_inline__, __nodebug__, target("mte")))
+ __arm_mte_increment_tag(void* __ptr, unsigned __tag_offset) {
+    switch(__tag_offset % 16) {
+    case 0: return __builtin_arm_addg(__ptr, 0);
+    case 1: return __builtin_arm_addg(__ptr, 1);
+    case 2: return __builtin_arm_addg(__ptr, 2);
+    case 3: return __builtin_arm_addg(__ptr, 3);
+    case 4: return __builtin_arm_addg(__ptr, 4);
+    case 5: return __builtin_arm_addg(__ptr, 5);
+    case 6: return __builtin_arm_addg(__ptr, 6);
+    case 7: return __builtin_arm_addg(__ptr, 7);
+    case 8: return __builtin_arm_addg(__ptr, 8);
+    case 9: return __builtin_arm_addg(__ptr, 9);
+    case 10: return __builtin_arm_addg(__ptr, 10);
+    case 11: return __builtin_arm_addg(__ptr, 11);
+    case 12: return __builtin_arm_addg(__ptr, 12);
+    case 13: return __builtin_arm_addg(__ptr, 13);
+    case 14: return __builtin_arm_addg(__ptr, 14);
+    case 15: return __builtin_arm_addg(__ptr, 15);
+    }
+}
+static __inline__ uint64_t __attribute__((__always_inline__, __nodebug__, target("mte")))
+ __arm_mte_exclude_tag(void *__ptr, uint64_t __excluded) {
+    return __builtin_arm_gmi(__ptr, __excluded);
+}
+static __inline__ void* __attribute__((__always_inline__, __nodebug__, target("mte")))
+ __arm_mte_get_tag(void *__ptr) {
+  return __builtin_arm_ldg(__ptr);
+}
+static __inline__ void __attribute__((__always_inline__, __nodebug__, target("mte")))
+ __arm_mte_set_tag(void *__ptr) {
+  __builtin_arm_stg(__ptr);
+}
+static __inline__ ptrdiff_t __attribute__((__always_inline__, __nodebug__, target("mte")))
+ __arm_mte_ptrdiff(void *__ptra,void *__ptrb) {
+  return __builtin_arm_subp(__ptra, __ptrb);
+}
 
 /* Memory Operations Intrinsics */
-#if defined(__ARM_FEATURE_MOPS) && __ARM_FEATURE_MOPS && defined(__ARM_FEATURE_MEMORY_TAGGING) && __ARM_FEATURE_MEMORY_TAGGING
-#define __arm_mops_memset_tag(__tagged_address, __value, __size)    \
-  __builtin_arm_mops_memset_tag(__tagged_address, __value, __size)
+static __inline__ void* __attribute__((__always_inline__, __nodebug__, target("mte,mops")))
+__arm_mops_memset_tag(void *__tagged_address, int __value, size_t __size) {
+  return __builtin_arm_mops_memset_tag(__tagged_address, __value, __size);
+}
 #endif
-
 /* Transactional Memory Extension (TME) Intrinsics */
-#if defined(__ARM_FEATURE_TME) && __ARM_FEATURE_TME
+#if defined(__ARM_64BIT_STATE) && __ARM_64BIT_STATE
 
 #define _TMFAILURE_REASON  0x00007fffu
 #define _TMFAILURE_RTRY    0x00008000u
 #define _TMFAILURE_CNCL    0x00010000u
 #define _TMFAILURE_MEM     0x00020000u
 #define _TMFAILURE_IMP     0x00040000u
 #define _TMFAILURE_ERR     0x00080000u
 #define _TMFAILURE_SIZE    0x00100000u
 #define _TMFAILURE_NEST    0x00200000u
 #define _TMFAILURE_DBG     0x00400000u
 #define _TMFAILURE_INT     0x00800000u
 #define _TMFAILURE_TRIVIAL 0x01000000u
 
-#define __tstart()        __builtin_arm_tstart()
-#define __tcommit()       __builtin_arm_tcommit()
-#define __tcancel(__arg)  __builtin_arm_tcancel(__arg)
-#define __ttest()         __builtin_arm_ttest()
-
-#endif /* __ARM_FEATURE_TME */
+static __inline__ uint64_t __attribute__((__always_inline__, __nodebug__, target("tme")))
+__tstart(void) {
+  __builtin_arm_tstart();
+}
+static __inline__ void __attribute__((__always_inline__, __nodebug__, target("tme")))
+__tcommit(void) {
+  __builtin_arm_tcommit();
+}
+static __inline__ void __attribute__((__always_inline__, __nodebug__, target("tme")))
+__tcancel(void) {
+}
+#define __tcancel(__arg)  __tcancel();__builtin_arm_tcancel(__arg)
+
+static __inline__ uint64_t __attribute__((__always_inline__, __nodebug__, target("tme")))
+__ttest(void) {
+  return __builtin_arm_ttest();
+}
+#endif
 
 /* Armv8.5-A Random number generation intrinsics */
-#if defined(__ARM_64BIT_STATE) && __ARM_64BIT_STATE && defined(__ARM_FEATURE_RNG)
-static __inline__ int __attribute__((__always_inline__, __nodebug__))
+#if defined(__ARM_64BIT_STATE) && __ARM_64BIT_STATE
+static __inline__ int __attribute__((__always_inline__, __nodebug__, target("rand")))
 __rndr(uint64_t *__p) {
   return __builtin_arm_rndr(__p);
 }
-static __inline__ int __attribute__((__always_inline__, __nodebug__))
+static __inline__ int __attribute__((__always_inline__, __nodebug__, target("rand")))
 __rndrrs(uint64_t *__p) {
   return __builtin_arm_rndrrs(__p);
 }
 #endif
 
 #if defined(__cplusplus)
 }
 #endif
 
 #endif /* __ARM_ACLE_H */

clang/test/CodeGen/aarch64-mops.c

-// RUN: %clang_cc1 -triple aarch64-arm-unknown-eabi -Wno-int-conversion -target-feature +mops -target-feature +mte -w -S -emit-llvm -o - %s  | FileCheck --check-prefix=CHECK-MOPS   %s
-// RUN: %clang_cc1 -triple aarch64-arm-unknown-eabi -Wno-int-conversion -target-feature +mops -Wno-implicit-function-declaration -w -S -emit-llvm -o - %s  | FileCheck --check-prefix=CHECK-NOMOPS %s
-// RUN: %clang_cc1 -triple aarch64-arm-unknown-eabi -Wno-int-conversion -Wno-implicit-function-declaration -target-feature +mte -w -S -emit-llvm -o - %s  | FileCheck --check-prefix=CHECK-NOMOPS %s
-// RUN: %clang_cc1 -triple aarch64-arm-unknown-eabi -Wno-int-conversion -Wno-implicit-function-declaration -w -S -emit-llvm -o - %s  | FileCheck --check-prefix=CHECK-NOMOPS %s
+// RUN: %clang_cc1 -triple aarch64-arm-unknown-eabi -Wno-int-conversion -target-feature +mops -target-feature +mte -w -S -emit-llvm -o - %s 2>&1 | FileCheck --check-prefix=CHECK-MOPS %s
+// RUN: not %clang_cc1 -triple aarch64-arm-unknown-eabi -Wno-int-conversion -Wno-implicit-function-declaration -target-feature +mte -w -S -emit-llvm -o - %s 2>&1 | FileCheck --check-prefix=CHECK-NOMOPS %s
+// RUN: not %clang_cc1 -triple aarch64-arm-unknown-eabi -Wno-int-conversion -target-feature +mops -Wno-implicit-function-declaration -w -S -emit-llvm -o - %s 2>&1 | FileCheck --check-prefix=CHECK-NOMTE %s
+// RUN: not %clang_cc1 -triple aarch64-arm-unknown-eabi -Wno-int-conversion -Wno-implicit-function-declaration -w -S -emit-llvm  %s -o - 2>&1 | FileCheck --check-prefix=CHECK-NOMTE %s
 
 #include <arm_acle.h>
 #include <stddef.h>
 
 // CHECK-LABEL:       @bzero_0(
 // CHECK-MOPS:        @llvm.aarch64.mops.memset.tag
-// CHECK-NOMOPS-NOT:  @llvm.aarch64.mops.memset.tag
+// CHECK-NOMTE:       error: always_inline function '__arm_mops_memset_tag' requires target feature 'mte', but would be inlined into function 'bzero_0' that is compiled without support for 'mte'
+// CHECK-NOMOPS:      error: always_inline function '__arm_mops_memset_tag' requires target feature 'mops', but would be inlined into function 'bzero_0' that is compiled without support for 'mops'
 void *bzero_0(void *dst) {
   return __arm_mops_memset_tag(dst, 0, 0);
 }
 
 // CHECK-LABEL:       @bzero_1(
 // CHECK-MOPS:        @llvm.aarch64.mops.memset.tag
-// CHECK-NOMOPS-NOT:  @llvm.aarch64.mops.memset.tag
 void *bzero_1(void *dst) {
   return __arm_mops_memset_tag(dst, 0, 1);
 }
 
 // CHECK-LABEL:       @bzero_10(
 // CHECK-MOPS:        @llvm.aarch64.mops.memset.tag
-// CHECK-NOMOPS-NOT:  @llvm.aarch64.mops.memset.tag
 void *bzero_10(void *dst) {
   return __arm_mops_memset_tag(dst, 0, 10);
 }
 
 // CHECK-LABEL:       @bzero_10000(
 // CHECK-MOPS:        @llvm.aarch64.mops.memset.tag
-// CHECK-NOMOPS-NOT:  @llvm.aarch64.mops.memset.tag
 void *bzero_10000(void *dst) {
   return __arm_mops_memset_tag(dst, 0, 10000);
 }
 
 // CHECK-LABEL:       @bzero_n(
 // CHECK-MOPS:        @llvm.aarch64.mops.memset.tag
-// CHECK-NOMOPS-NOT:  @llvm.aarch64.mops.memset.tag
 void *bzero_n(void *dst, size_t size) {
   return __arm_mops_memset_tag(dst, 0, size);
 }
 
 // CHECK-LABEL:       @memset_0(
 // CHECK-MOPS:        @llvm.aarch64.mops.memset.tag
-// CHECK-NOMOPS-NOT:  @llvm.aarch64.mops.memset.tag
 void *memset_0(void *dst, int value) {
   return __arm_mops_memset_tag(dst, value, 0);
 }
 
 // CHECK-LABEL:       @memset_1(
 // CHECK-MOPS:        @llvm.aarch64.mops.memset.tag
-// CHECK-NOMOPS-NOT:  @llvm.aarch64.mops.memset.tag
 void *memset_1(void *dst, int value) {
   return __arm_mops_memset_tag(dst, value, 1);
 }
 
 // CHECK-LABEL:       @memset_10(
 // CHECK-MOPS:        @llvm.aarch64.mops.memset.tag
-// CHECK-NOMOPS-NOT:  @llvm.aarch64.mops.memset.tag
 void *memset_10(void *dst, int value) {
   return __arm_mops_memset_tag(dst, value, 10);
 }
 
 // CHECK-LABEL:       @memset_10000(
 // CHECK-MOPS:        @llvm.aarch64.mops.memset.tag
-// CHECK-NOMOPS-NOT:  @llvm.aarch64.mops.memset.tag
 void *memset_10000(void *dst, int value) {
   return __arm_mops_memset_tag(dst, value, 10000);
 }
 
 // CHECK-LABEL:       @memset_n(
 // CHECK-MOPS:        @llvm.aarch64.mops.memset.tag
-// CHECK-NOMOPS-NOT:  @llvm.aarch64.mops.memset.tag
 void *memset_n(void *dst, int value, size_t size) {
   return __arm_mops_memset_tag(dst, value, size);
 }

clang/test/CodeGen/aarch64-tme.cpp

 // RUN: %clang_cc1             -triple aarch64-eabi -target-feature +tme -S -emit-llvm %s -o - | FileCheck %s
-// RUN: %clang_cc1 -DUSE_ACLE  -triple aarch64-eabi -target-feature +tme -S -emit-llvm %s -o - | FileCheck %s
+// RUN: %clang_cc1 -DUSE_ACLE  -triple aarch64-eabi -target-feature +tme -S -emit-llvm %s -o - | FileCheck %s -check-prefix=CHECK_ACLE
+// RUN: not %clang_cc1 -DUSE_ACLE  -triple aarch64-eabi -S -emit-llvm %s -o - 2>&1 | FileCheck %s -check-prefix=CHECK_NO_TME
 
 #define A -1
 constexpr int f() { return 65536; }
 
 #ifdef USE_ACLE
 #include "arm_acle.h"
 void test_tme_funcs() {
   __tstart();

 }
 #endif
 // CHECK: call i64 @llvm.aarch64.tstart()
 // CHECK: call i64 @llvm.aarch64.ttest()
 // CHECK: call void @llvm.aarch64.tcommit()
 // CHECK: call void @llvm.aarch64.tcancel(i64 30874)
 // CHECK: call void @llvm.aarch64.tcancel(i64 65535)
 
+// CHECK_ACLE: call i64 @llvm.aarch64.tstart()
+// CHECK_ACLE: call i64 @llvm.aarch64.ttest()
+// CHECK_ACLE: call void @llvm.aarch64.tcommit()
+// CHECK_ACLE: call void @llvm.aarch64.tcancel(i64 30874)
+// CHECK_ACLE: call void @llvm.aarch64.tcancel(i64 65535)
+
 // CHECK: declare i64 @llvm.aarch64.tstart() #1
 // CHECK: declare i64 @llvm.aarch64.ttest() #1
 // CHECK: declare void @llvm.aarch64.tcommit() #1
 // CHECK: declare void @llvm.aarch64.tcancel(i64 immarg) #1
 
+// CHECK_ACLE: declare void @llvm.aarch64.tcancel(i64 immarg) #1
+// CHECK_ACLE: define{{.*}} void @arm_feature_tme_defined()
+// CHECK_ACLE: declare i64 @llvm.aarch64.tstart() #1
+// CHECK_ACLE: declare i64 @llvm.aarch64.ttest() #1
+// CHECK_ACLE: declare void @llvm.aarch64.tcommit() #1
+
+// CHECK_NO_TME: error: always_inline function '__tstart' requires target feature 'tme', but would be inlined into function 'test_tme_funcs' that is compiled without support for 'tme'
+// CHECK_NO_TME: error: always_inline function '__ttest' requires target feature 'tme', but would be inlined into function 'test_tme_funcs' that is compiled without support for 'tme'
+// CHECK_NO_TME: error: always_inline function '__tcommit' requires target feature 'tme', but would be inlined into function 'test_tme_funcs' that is compiled without support for 'tme'
+// CHECK_NO_TME: error: always_inline function '__tcancel' requires target feature 'tme', but would be inlined into function 'test_tme_funcs' that is compiled without support for 'tme'
+// CHECK_NO_TME: error: always_inline function '__tcancel' requires target feature 'tme', but would be inlined into function 'test_tme_funcs' that is compiled without support for 'tme'
+
 #ifdef __ARM_FEATURE_TME
 extern "C" void arm_feature_tme_defined() {}
 #endif
-// CHECK: define{{.*}} void @arm_feature_tme_defined()
 
 // CHECK: attributes #1 = { nounwind willreturn }
 

clang/test/CodeGen/arm64-crc32.c

 // REQUIRES: aarch64-registered-target
-// RUN: %clang_cc1 -triple arm64-none-linux-gnu \
+// RUN: %clang_cc1 -triple arm64-none-linux-gnu -target-feature +crc \
 // RUN:  -disable-O0-optnone -S -emit-llvm -o - %s | opt -S -mem2reg | FileCheck %s
-// RUN: %clang_cc1 -triple aarch64-windows \
+// RUN: %clang_cc1 -triple aarch64-windows  -target-feature +crc \
 // RUN:  -disable-O0-optnone -S -emit-llvm -o - %s | opt -S -mem2reg | FileCheck %s
 #include <stdint.h>
 
 uint32_t crc32b(uint32_t a, uint8_t b)
 {
         return __builtin_arm_crc32b(a,b);
 // CHECK: [[T0:%[0-9]+]] = zext i8 %b to i32
 // CHECK: call i32 @llvm.aarch64.crc32b(i32 %a, i32 [[T0]])

clang/test/CodeGen/arm64-mte.c

 // Test memory tagging extension intrinsics
 // RUN: %clang_cc1 -no-opaque-pointers -triple aarch64-none-linux-eabi -target-feature +mte -O3 -S -emit-llvm -o - %s  | FileCheck %s
 #include <stddef.h>
 #include <arm_acle.h>
 
 // CHECK-LABEL: define{{.*}} i32* @create_tag1
 int *create_tag1(int *a, unsigned b) {
 // CHECK: [[T0:%[0-9]+]] = bitcast i32* %a to i8*
-// CHECK: [[T1:%[0-9]+]] = zext i32 %b to i64
+// CHECK: [[T1:%[0-9a-z]+]] = zext i32 %b to i64
 // CHECK: [[T2:%[0-9]+]] = tail call i8* @llvm.aarch64.irg(i8* [[T0]], i64 [[T1]])
 // CHECK: bitcast i8* [[T2]] to i32*
         return __arm_mte_create_random_tag(a,b);
 }
 
 // CHECK-LABEL: define{{.*}} i16* @create_tag2
 short *create_tag2(short *a, unsigned b) {
 // CHECK: [[T0:%[0-9]+]] = bitcast i16* %a to i8*
-// CHECK: [[T1:%[0-9]+]] = zext i32 %b to i64
+// CHECK: [[T1:%[0-9a-z]+]] = zext i32 %b to i64
 // CHECK: [[T2:%[0-9]+]] = tail call i8* @llvm.aarch64.irg(i8* [[T0]], i64 [[T1]])
 // CHECK: bitcast i8* [[T2]] to i16*
         return __arm_mte_create_random_tag(a,b);
 }
 
 // CHECK-LABEL: define{{.*}} i8* @create_tag3
 char *create_tag3(char *a, unsigned b) {
-// CHECK: [[T1:%[0-9]+]] = zext i32 %b to i64
+// CHECK: [[T1:%[0-9a-z]+]] = zext i32 %b to i64
 // CHECK: [[T2:%[0-9]+]] = tail call i8* @llvm.aarch64.irg(i8* %a, i64 [[T1]])
 // CHECK: ret i8* [[T2:%[0-9]+]]
         return __arm_mte_create_random_tag(a,b);
 }
 
 // CHECK-LABEL: define{{.*}} i8* @increment_tag1
 char *increment_tag1(char *a) {
 // CHECK: call i8* @llvm.aarch64.addg(i8* %a, i64 3)
         return __arm_mte_increment_tag(a,3);
 }
 
 // CHECK-LABEL: define{{.*}} i16* @increment_tag2
 short *increment_tag2(short *a) {
 // CHECK: [[T0:%[0-9]+]] = bitcast i16* %a to i8*
 // CHECK: [[T1:%[0-9]+]] = tail call i8* @llvm.aarch64.addg(i8* [[T0]], i64 3)
 // CHECK: [[T2:%[0-9]+]]  = bitcast i8* [[T1]] to i16*
         return __arm_mte_increment_tag(a,3);
 }
 
 // CHECK-LABEL: define{{.*}} i32 @exclude_tag
 unsigned exclude_tag(int *a, unsigned m) {
-// CHECK: [[T0:%[0-9]+]] = zext i32 %m to i64
 // CHECK: [[T1:%[0-9]+]] = bitcast i32* %a to i8*
+// CHECK: [[T0:%[0-9a-z]+]] = zext i32 %m to i64
 // CHECK: [[T2:%[0-9]+]] = tail call i64 @llvm.aarch64.gmi(i8* [[T1]], i64 [[T0]])
 // CHECK: trunc i64 [[T2]] to i32
   return __arm_mte_exclude_tag(a, m);
 }
 
 // CHECK-LABEL: define{{.*}} i32* @get_tag1
 int *get_tag1(int *a) {
 // CHECK: [[T0:%[0-9]+]] = bitcast i32* %a to i8*

 // CHECK: [[T1:%[0-9]+]] = bitcast i32* %b to i8*
 // CHECK: [[T2:%[0-9]+]] = tail call i64 @llvm.aarch64.subp(i8* [[T0]], i8* [[T1]])
 // CHECK: ret i64 [[T2]]
    return __arm_mte_ptrdiff(a, b);
 }
 
 // CHECK-LABEL: define{{.*}} i64 @subtract_pointers_null_1
 ptrdiff_t subtract_pointers_null_1(int *a) {
-// CHECK: [[T0:%[0-9]+]] = bitcast i32* %a to i8*
+// CHECK: [[T0:%[0-9a-z]+]] = bitcast i32* %a to i8*
 // CHECK: [[T1:%[0-9]+]] = tail call i64 @llvm.aarch64.subp(i8* [[T0]], i8* null)
 // CHECK: ret i64 [[T1]]
    return __arm_mte_ptrdiff(a, NULL);
 }
 
 // CHECK-LABEL: define{{.*}} i64 @subtract_pointers_null_2
 ptrdiff_t subtract_pointers_null_2(int *a) {
-// CHECK: [[T0:%[0-9]+]] = bitcast i32* %a to i8*
-// CHECK: [[T1:%[0-9]+]] = tail call i64 @llvm.aarch64.subp(i8* null, i8* [[T0]])
+// CHECK: [[T0:%[0-9a-z]+]] = bitcast i32* %a to i8*
+// CHECK: [[T1:%[0-9a-z]+]] = tail call i64 @llvm.aarch64.subp(i8* null, i8* [[T0]])
 // CHECK: ret i64 [[T1]]
    return __arm_mte_ptrdiff(NULL, a);
 }
 
 // Check arithmetic promotion on return type
 // CHECK-LABEL: define{{.*}} i32 @subtract_pointers4
 int subtract_pointers4(void* a, void *b) {
-// CHECK: [[T0:%[0-9]+]] = tail call i64 @llvm.aarch64.subp(i8* %a, i8* %b)
+// CHECK: [[T0:%[0-9a-z]+]] = tail call i64 @llvm.aarch64.subp(i8* %a, i8* %b)
 // CHECK-NEXT: %cmp = icmp slt i64 [[T0]], 1
 // CHECK-NEXT:  = zext i1 %cmp to i32
   return __arm_mte_ptrdiff(a,b) <= 0;
 }

clang/test/CodeGen/arm_acle.c

 // NOTE: Assertions have been autogenerated by utils/update_cc_test_checks.py
 // RUN: %clang_cc1 -no-opaque-pointers -ffreestanding -triple armv8a-none-eabi -target-feature +crc -target-feature +dsp -O0 -disable-O0-optnone -S -emit-llvm -o - %s | opt -S -mem2reg | FileCheck %s -check-prefixes=ARM,AArch32
 // RUN: %clang_cc1 -no-opaque-pointers -ffreestanding -Wno-error=implicit-function-declaration -triple aarch64-none-eabi -target-feature +neon -target-feature +crc -target-feature +crypto -O0 -disable-O0-optnone -S -emit-llvm -o - %s | opt -S -mem2reg | FileCheck %s -check-prefixes=ARM,AArch64
-// RUN: %clang_cc1 -no-opaque-pointers -ffreestanding -triple aarch64-none-eabi -target-feature +v8.3a -O0 -disable-O0-optnone -S -emit-llvm -o - %s | opt -S -mem2reg | FileCheck %s -check-prefixes=ARM,AArch64,AArch6483
-// RUN: %clang_cc1 -no-opaque-pointers -ffreestanding -triple aarch64-none-eabi -target-feature +v8.5a -target-feature +rand -O0 -disable-O0-optnone -S -emit-llvm -o - %s | opt -S -mem2reg | FileCheck %s -check-prefixes=ARM,AArch64,AArch6483,AArch6485
+// RUN: %clang_cc1 -no-opaque-pointers -ffreestanding -triple aarch64-none-eabi -target-feature +crc -target-feature +v8.3a -O0 -disable-O0-optnone -S -emit-llvm -o - %s | opt -S -mem2reg | FileCheck %s -check-prefixes=ARM,AArch64,AArch6483
+// RUN: %clang_cc1 -no-opaque-pointers -ffreestanding -triple aarch64-none-eabi -target-feature +crc -target-feature +v8.5a -target-feature +rand -O0 -disable-O0-optnone -S -emit-llvm -o - %s | opt -S -mem2reg | FileCheck %s -check-prefixes=ARM,AArch64,AArch6483,AArch6485
 
 #include <arm_acle.h>
 
 // REQUIRES: arm-registered-target,aarch64-registered-target
 
 /* 8 SYNCHRONIZATION, BARRIER AND HINT INTRINSICS */
 /* 8.3 Memory Barriers */
 

 // AArch64-NEXT:  entry:
 // AArch64-NEXT:    call void @llvm.aarch64.hint(i32 5)
 // AArch64-NEXT:    ret void
 //
 void test_sevl(void) {
   __sevl();
 }
 
-#if __ARM_32BIT_STATE
+#ifdef __ARM_32BIT_STATE
 // AArch32-LABEL: @test_dbg(
 // AArch32-NEXT:  entry:
 // AArch32-NEXT:    call void @llvm.arm.dbg(i32 0)
 // AArch32-NEXT:    ret void
 //
 void test_dbg(void) {
   __dbg(0);
 }

 void test_wsrf64(double v) {
 #ifdef __ARM_32BIT_STATE
   __arm_wsrf64("cp1:2:c3", v);
 #else
   __arm_wsrf64("1:2:3:4:5", v);
 #endif
 }
 
-#ifdef __ARM_64BIT_STATE
+#if defined(__ARM_64BIT_STATE) && defined(__ARM_FEATURE_JCVT)
 // AArch6483-LABEL: @test_jcvt(
 // AArch6483-NEXT:  entry:
 // AArch6483-NEXT:    [[TMP0:%.*]] = call i32 @llvm.aarch64.fjcvtzs(double [[V:%.*]])
 // AArch6483-NEXT:    ret i32 [[TMP0]]
 //
 int32_t test_jcvt(double v) {
   return __jcvt(v);
 }
 #endif
 
 
-#if __ARM_64BIT_STATE && defined(__ARM_FEATURE_RNG)
+#if defined(__ARM_64BIT_STATE) && defined(__ARM_FEATURE_RNG)
 
 // AArch6485-LABEL: @test_rndr(
 // AArch6485-NEXT:  entry:
 // AArch6485-NEXT:    [[TMP0:%.*]] = call { i64, i1 } @llvm.aarch64.rndr()
 // AArch6485-NEXT:    [[TMP1:%.*]] = extractvalue { i64, i1 } [[TMP0]], 0
 // AArch6485-NEXT:    [[TMP2:%.*]] = extractvalue { i64, i1 } [[TMP0]], 1
 // AArch6485-NEXT:    store i64 [[TMP1]], i64* [[__ADDR:%.*]], align 8
 // AArch6485-NEXT:    [[TMP3:%.*]] = zext i1 [[TMP2]] to i32

clang/test/CodeGen/builtins-arm64.c

void prefetch(void) {
 
   __builtin_arm_prefetch(0, 0, 0, 1, 1); // pldl1strm
   // CHECK: call {{.*}} @llvm.prefetch.p0i8(i8* null, i32 0, i32 0, i32 1)
 
   __builtin_arm_prefetch(0, 0, 0, 0, 0); // plil1keep
   // CHECK: call {{.*}} @llvm.prefetch.p0i8(i8* null, i32 0, i32 3, i32 0)
 }
 
+__attribute__((target("v8.5a")))
 int32_t jcvt(double v) {
   //CHECK-LABEL: @jcvt(
   //CHECK: call i32 @llvm.aarch64.fjcvtzs
   return __builtin_arm_jcvt(v);
 }
 
 __typeof__(__builtin_arm_rsr("1:2:3:4:5")) rsr(void);
 

 // CHECK-NEXT:  entry:
 // CHECK-NEXT:    [[TMP0:%.*]] = call { i64, i1 } @llvm.aarch64.rndr()
 // CHECK-NEXT:    [[TMP1:%.*]] = extractvalue { i64, i1 } [[TMP0]], 0
 // CHECK-NEXT:    [[TMP2:%.*]] = extractvalue { i64, i1 } [[TMP0]], 1
 // CHECK-NEXT:    store i64 [[TMP1]], i64* [[__ADDR:%.*]], align 8
 // CHECK-NEXT:    [[TMP3:%.*]] = zext i1 [[TMP2]] to i32
 // CHECK-NEXT:    ret i32 [[TMP3]]
 //
+__attribute__((target("rand")))
 int rndr(uint64_t *__addr) {
   return __builtin_arm_rndr(__addr);
 }
 
 // CHECK-LABEL: @rndrrs(
 // CHECK-NEXT:  entry:
 // CHECK-NEXT:    [[TMP0:%.*]] = call { i64, i1 } @llvm.aarch64.rndrrs()
 // CHECK-NEXT:    [[TMP1:%.*]] = extractvalue { i64, i1 } [[TMP0]], 0
 // CHECK-NEXT:    [[TMP2:%.*]] = extractvalue { i64, i1 } [[TMP0]], 1
 // CHECK-NEXT:    store i64 [[TMP1]], i64* [[__ADDR:%.*]], align 8
 // CHECK-NEXT:    [[TMP3:%.*]] = zext i1 [[TMP2]] to i32
 // CHECK-NEXT:    ret i32 [[TMP3]]
 //
+__attribute__((target("rand")))
 int rndrrs(uint64_t *__addr) {
   return __builtin_arm_rndrrs(__addr);
 }
 
 // CHECK: ![[M0]] = !{!"1:2:3:4:5"}

clang/test/Sema/aarch64-tme-errors.c

 // RUN: %clang_cc1 -triple aarch64-eabi -verify %s
 
 #include "arm_acle.h"
 
 void test_no_tme_funcs(void) {
-  __tstart();         // expected-error{{call to undeclared function '__tstart'; ISO C99 and later do not support implicit function declarations}}
+  __tstart();         // ACLE always provides intrinsics.
   __builtin_tstart(); // expected-error{{use of unknown builtin '__builtin_tstart'}}
 }

clang/test/Sema/builtins-arm64-mte.c

 // RUN: %clang_cc1 -triple arm64-arm-eabi %s -target-feature +mte -fsyntax-only -verify
 // RUN: %clang_cc1 -triple arm64-arm-eabi %s -target-feature +mte -x c++ -fsyntax-only -verify
 #include <stddef.h>
 #include <arm_acle.h>
 
 int  *create_tag1(int a, unsigned b) {
   // expected-error@+1 {{first argument of MTE builtin function must be a pointer ('int' invalid)}}
-  return __arm_mte_create_random_tag(a,b);
+  return __builtin_arm_irg(a,b);
 }
 
 int  *create_tag2(int *a, unsigned *b) {
   // expected-error@+1 {{second argument of MTE builtin function must be an integer type ('unsigned int *' invalid)}}
-  return __arm_mte_create_random_tag(a,b);
+  return __builtin_arm_irg(a,b);
 }
 
 int  *create_tag3(const int *a, unsigned b) {
 #ifdef __cplusplus
   // expected-error@+1 {{cannot initialize return object of type 'int *' with an rvalue of type 'const int *'}}
-  return __arm_mte_create_random_tag(a,b);
+  return __builtin_arm_irg(a,b);
 #else
   // expected-warning@+1 {{returning 'const int *' from a function with result type 'int *' discards qualifiers}}
-  return __arm_mte_create_random_tag(a,b);
+  return __builtin_arm_irg(a,b);
 #endif
 }
 
 int  *create_tag4(volatile int *a, unsigned b) {
 #ifdef __cplusplus
   // expected-error@+1 {{cannot initialize return object of type 'int *' with an rvalue of type 'volatile int *'}}
-  return __arm_mte_create_random_tag(a,b);
+  return __builtin_arm_irg(a,b);
 #else
   // expected-warning@+1 {{returning 'volatile int *' from a function with result type 'int *' discards qualifiers}}
-  return __arm_mte_create_random_tag(a,b);
+  return __builtin_arm_irg(a,b);
 #endif
 }
 
 int  *increment_tag1(int *a, unsigned b) {
   // expected-error@+1 {{argument to '__builtin_arm_addg' must be a constant integer}}
-  return __arm_mte_increment_tag(a,b);
+  return __builtin_arm_addg(a,b);
 }
 
 int  *increment_tag2(int *a) {
   // expected-error@+1 {{argument value 16 is outside the valid range [0, 15]}}
-  return __arm_mte_increment_tag(a,16);
+  return __builtin_arm_addg(a,16);
 }
 
 int  *increment_tag3(int *a) {
   // expected-error@+1 {{argument value -1 is outside the valid range [0, 15]}}
-  return __arm_mte_increment_tag(a,-1);
+  return __builtin_arm_addg(a,-1);
 }
 
 int  *increment_tag4(const int *a) {
 #ifdef __cplusplus
   // expected-error@+1 {{cannot initialize return object of type 'int *' with an rvalue of type 'const int *'}}
-  return __arm_mte_increment_tag(a,5);
+  return __builtin_arm_addg(a,5);
 #else
   // expected-warning@+1 {{returning 'const int *' from a function with result type 'int *' discards qualifiers}}
-  return __arm_mte_increment_tag(a,5);
+  return __builtin_arm_addg(a,5);
 #endif
 }
 
 int *increment_tag5(const volatile int *a) {
 #ifdef __cplusplus
   // expected-error@+1 {{cannot initialize return object of type 'int *' with an rvalue of type 'const volatile int *'}}
-  return __arm_mte_increment_tag(a,5);
+  return __builtin_arm_addg(a,5);
 #else
   // expected-warning@+1 {{returning 'const volatile int *' from a function with result type 'int *' discards qualifiers}}
-  return __arm_mte_increment_tag(a,5);
+  return __builtin_arm_addg(a,5);
 #endif
 }
 
 unsigned exclude_tag1(int *ptr, unsigned m) {
    // expected-error@+1 {{first argument of MTE builtin function must be a pointer ('int' invalid)}}
-   return  __arm_mte_exclude_tag(*ptr, m);
+   return  __builtin_arm_gmi(*ptr, m);
 }
 
 unsigned exclude_tag2(int *ptr, int *m) {
    // expected-error@+1 {{second argument of MTE builtin function must be an integer type ('int *' invalid)}}
-   return  __arm_mte_exclude_tag(ptr, m);
+   return  __builtin_arm_gmi(ptr, m);
 }
 
 void get_tag1(void) {
    // expected-error@+1 {{too few arguments to function call, expected 1, have 0}}
-   __arm_mte_get_tag();
+   __builtin_arm_ldg();
 }
 
 int *get_tag2(int ptr) {
    // expected-error@+1 {{first argument of MTE builtin function must be a pointer ('int' invalid)}}
-   return __arm_mte_get_tag(ptr);
+   return __builtin_arm_ldg(ptr);
 }
 
 int *get_tag3(const volatile int *ptr) {
 #ifdef __cplusplus
   // expected-error@+1 {{cannot initialize return object of type 'int *' with an rvalue of type 'const volatile int *'}}
-  return __arm_mte_get_tag(ptr);
+  return __builtin_arm_ldg(ptr);
 #else
   // expected-warning@+1 {{returning 'const volatile int *' from a function with result type 'int *' discards qualifiers}}
-  return __arm_mte_get_tag(ptr);
+  return __builtin_arm_ldg(ptr);
 #endif
 }
 
 void set_tag1(void) {
    // expected-error@+1 {{too few arguments to function call, expected 1, have 0}}
-   __arm_mte_set_tag();
+   __builtin_arm_stg();
 }
 
 void set_tag2(int ptr) {
    // expected-error@+1 {{first argument of MTE builtin function must be a pointer ('int' invalid)}}
-   __arm_mte_set_tag(ptr);
+   __builtin_arm_stg(ptr);
 }
 
 ptrdiff_t subtract_pointers1(int a, int *b) {
   // expected-error@+1 {{first argument of MTE builtin function must be a null or a pointer ('int' invalid)}}
-  return __arm_mte_ptrdiff(a, b);
+  return __builtin_arm_subp(a, b);
 }
 
 ptrdiff_t subtract_pointers2(int *a, int b) {
   // expected-error@+1 {{second argument of MTE builtin function must be a null or a pointer ('int' invalid)}}
-  return __arm_mte_ptrdiff(a, b);
+  return __builtin_arm_subp(a, b);
 }
 
 ptrdiff_t subtract_pointers3(char *a, int *b) {
   // expected-error@+1 {{'char *' and 'int *' are not pointers to compatible types}}
-  return __arm_mte_ptrdiff(a, b);
+  return __builtin_arm_subp(a, b);
 }
 
 ptrdiff_t subtract_pointers4(int *a, char *b) {
   // expected-error@+1 {{'int *' and 'char *' are not pointers to compatible types}}
-  return __arm_mte_ptrdiff(a, b);
+  return __builtin_arm_subp(a, b);
 }
 
 #ifdef __cplusplus
 ptrdiff_t subtract_pointers5() {
   // expected-error@+1 {{at least one argument of MTE builtin function must be a pointer ('std::nullptr_t', 'std::nullptr_t' invalid)}}
-  return __arm_mte_ptrdiff(nullptr, nullptr);
+  return __builtin_arm_subp(nullptr, nullptr);
 }
 #endif