[SveEmitter] Add builtins for svreinterpret

[sdesmalen]

Summary

The reinterpret builtins are generated separately because they
need the cross product of all types, 121 functions in total,
which is inconvenient to specify in the arm_sve.td file.

[efriedma]

clang/lib/CodeGen/CGBuiltin.cpp

7880

I'm vaguely suspicious this might be wrong for big-endian targets. I mean, this isn't unreasonable, but users might be surprised if svreinterpret isn't a no-op.

[sdesmalen]

clang/lib/CodeGen/CGBuiltin.cpp

7880

For SVE the loads and stores (svld1 and svst1) are all endian safe, so no special consideration needs to be taken for big endian targets.

The ACLE specifies that:

The svreinterpret functions simply reinterpret a vector of one type as a vector of another type, without changing any of the bits.

[efriedma]

clang/lib/CodeGen/CGBuiltin.cpp

7880

"bitcast" is specified to mean "reinterpret the bits like a store+load". On big-endian NEON (and, I assume, SVE), that isn't a no-op. See http://llvm.org/docs/BigEndianNEON.html .

I mean, if the answer here is "yes, svreinterpret is supposed to lower to a REV", then that's fine. But I'd like to see some explciit acknowledgement that that's intentional.

[sdesmalen]

clang/lib/CodeGen/CGBuiltin.cpp

7880

Thanks for pointing out that page, but for SVE I don't think the svreinterpret should lower to a REV.

This is probably where things are different from Neon. The ACLE SVE vectors such as svint32_t are opaque vector types and the only way to load/store them from/to memory is through the use of the svld1 and svst1 intrinsics which are endian safe (in that they use the ld1/st1 instructions that do endianess conversion on big endian targets). The ACLE does not expose any full-vector load/store (ldr/str) operations.

[efriedma]

clang/lib/CodeGen/CGBuiltin.cpp

7880

Like that page describes, we use ld1/st1 for big-endian NEON, to match the LLVM IR rules for laying out a vector. If you use ld1/st1 to load/store vectors on big-endian NEON, a bitcast is not a no-op. As far as I know, SVE ld1/st1 is equivalent to NEON ld1/st1 in the case where vscale=1. Therefore, on big-endian SVE, a bitcast is not a no-op.

That leaves the following options:

1. svreinterpret is not a no-op.
2. svreinterpret is not equivalent to an LLVM IR bitcast, so this patch needs to be changed.

[efriedma]

clang/lib/CodeGen/CGBuiltin.cpp

7880

(If you don't care about big-endian SVE right now, that's fine, but please at least leave a FIXME.)

[sdesmalen]

• Added FIXME in CGBuiltins for big-endian svreinterpret.
• Added diagnostic in arm_sve.h that big-endian is not yet supported.

[efriedma]

LGTM

clang/lib/CodeGen/CGBuiltin.cpp

 Value *CodeGenFunction::EmitAArch64SVEBuiltinExpr(unsigned BuiltinID,
                                                   const CallExpr *E) {
   // Find out if any arguments are required to be integer constant expressions.
   unsigned ICEArguments = 0;
   ASTContext::GetBuiltinTypeError Error;
   getContext().GetBuiltinType(BuiltinID, Error, &ICEArguments);
   assert(Error == ASTContext::GE_None && "Should not codegen an error");
 
+  llvm::Type *Ty = ConvertType(E->getType());
+  if (BuiltinID >= SVE::BI__builtin_sve_reinterpret_s8_s8 &&
+      BuiltinID <= SVE::BI__builtin_sve_reinterpret_f64_f64) {
+    Value *Val = EmitScalarExpr(E->getArg(0));
+    return Builder.CreateBitCast(Val, Ty);
+  }
+
   llvm::SmallVector<Value *, 4> Ops;
   for (unsigned i = 0, e = E->getNumArgs(); i != e; i++) {
     if ((ICEArguments & (1 << i)) == 0)
       Ops.push_back(EmitScalarExpr(E->getArg(i)));
     else {
       // If this is required to be a constant, constant fold it so that we know
       // that the generated intrinsic gets a ConstantInt.
       llvm::APSInt Result;
       if (!E->getArg(i)->isIntegerConstantExpr(Result, getContext()))
         llvm_unreachable("Expected argument to be a constant");
 
       // Immediates for SVE llvm intrinsics are always 32bit.  We can safely
       // truncate because the immediate has been range checked and no valid
       // immediate requires more than a handful of bits.
       Result = Result.extOrTrunc(32);
       Ops.push_back(llvm::ConstantInt::get(getLLVMContext(), Result));
     }
   }
 
   auto *Builtin = findARMVectorIntrinsicInMap(AArch64SVEIntrinsicMap, BuiltinID,
                                               AArch64SVEIntrinsicsProvenSorted);
   SVETypeFlags TypeFlags(Builtin->TypeModifier);
-  llvm::Type *Ty = ConvertType(E->getType());
   if (TypeFlags.isLoad())
     return EmitSVEMaskedLoad(E, Ty, Ops, Builtin->LLVMIntrinsic,
                              TypeFlags.isZExtReturn());
   else if (TypeFlags.isStore())
     return EmitSVEMaskedStore(E, Ops, Builtin->LLVMIntrinsic);
   else if (TypeFlags.isGatherLoad())
     return EmitSVEGatherLoad(TypeFlags, Ops, Builtin->LLVMIntrinsic);
   else if (TypeFlags.isScatterStore())

clang/test/CodeGen/aarch64-sve-intrinsics/acle_sve_reinterpret.c

+// RUN: %clang_cc1 -D__ARM_FEATURE_SVE -triple aarch64-none-linux-gnu -target-feature +sve -fallow-half-arguments-and-returns -S -O1 -Werror -Wall -emit-llvm -o - %s | FileCheck %s
+// RUN: %clang_cc1 -D__ARM_FEATURE_SVE -DSVE_OVERLOADED_FORMS -triple aarch64-none-linux-gnu -target-feature +sve -fallow-half-arguments-and-returns -S -O1 -Werror -Wall -emit-llvm -o - %s | FileCheck %s
+
+#include <arm_sve.h>
+
+#ifdef SVE_OVERLOADED_FORMS
+// A simple used,unused... macro, long enough to represent any SVE builtin.
+#define SVE_ACLE_FUNC(A1,A2_UNUSED,A3,A4_UNUSED) A1##A3
+#else
+#define SVE_ACLE_FUNC(A1,A2,A3,A4) A1##A2##A3##A4
+#endif
+
+svint8_t test_svreinterpret_s8_s8(svint8_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s8_s8
+  // CHECK: ret <vscale x 16 x i8> %op
+  return SVE_ACLE_FUNC(svreinterpret_s8,_s8,,)(op);
+}
+
+svint8_t test_svreinterpret_s8_s16(svint16_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s8_s16
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 8 x i16> %op to <vscale x 16 x i8>
+  // CHECK: ret <vscale x 16 x i8> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_s8,_s16,,)(op);
+}
+
+svint8_t test_svreinterpret_s8_s32(svint32_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s8_s32
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 4 x i32> %op to <vscale x 16 x i8>
+  // CHECK: ret <vscale x 16 x i8> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_s8,_s32,,)(op);
+}
+
+svint8_t test_svreinterpret_s8_s64(svint64_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s8_s64
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 2 x i64> %op to <vscale x 16 x i8>
+  // CHECK: ret <vscale x 16 x i8> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_s8,_s64,,)(op);
+}
+
+svint8_t test_svreinterpret_s8_u8(svuint8_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s8_u8
+  // CHECK: ret <vscale x 16 x i8> %op
+  return SVE_ACLE_FUNC(svreinterpret_s8,_u8,,)(op);
+}
+
+svint8_t test_svreinterpret_s8_u16(svuint16_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s8_u16
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 8 x i16> %op to <vscale x 16 x i8>
+  // CHECK: ret <vscale x 16 x i8> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_s8,_u16,,)(op);
+}
+
+svint8_t test_svreinterpret_s8_u32(svuint32_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s8_u32
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 4 x i32> %op to <vscale x 16 x i8>
+  // CHECK: ret <vscale x 16 x i8> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_s8,_u32,,)(op);
+}
+
+svint8_t test_svreinterpret_s8_u64(svuint64_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s8_u64
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 2 x i64> %op to <vscale x 16 x i8>
+  // CHECK: ret <vscale x 16 x i8> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_s8,_u64,,)(op);
+}
+
+svint8_t test_svreinterpret_s8_f16(svfloat16_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s8_f16
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 8 x half> %op to <vscale x 16 x i8>
+  // CHECK: ret <vscale x 16 x i8> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_s8,_f16,,)(op);
+}
+
+svint8_t test_svreinterpret_s8_f32(svfloat32_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s8_f32
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 4 x float> %op to <vscale x 16 x i8>
+  // CHECK: ret <vscale x 16 x i8> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_s8,_f32,,)(op);
+}
+
+svint8_t test_svreinterpret_s8_f64(svfloat64_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s8_f64
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 2 x double> %op to <vscale x 16 x i8>
+  // CHECK: ret <vscale x 16 x i8> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_s8,_f64,,)(op);
+}
+
+svint16_t test_svreinterpret_s16_s8(svint8_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s16_s8
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 16 x i8> %op to <vscale x 8 x i16>
+  // CHECK: ret <vscale x 8 x i16> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_s16,_s8,,)(op);
+}
+
+svint16_t test_svreinterpret_s16_s16(svint16_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s16_s16
+  // CHECK: ret <vscale x 8 x i16> %op
+  return SVE_ACLE_FUNC(svreinterpret_s16,_s16,,)(op);
+}
+
+svint16_t test_svreinterpret_s16_s32(svint32_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s16_s32
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 4 x i32> %op to <vscale x 8 x i16>
+  // CHECK: ret <vscale x 8 x i16> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_s16,_s32,,)(op);
+}
+
+svint16_t test_svreinterpret_s16_s64(svint64_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s16_s64
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 2 x i64> %op to <vscale x 8 x i16>
+  // CHECK: ret <vscale x 8 x i16> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_s16,_s64,,)(op);
+}
+
+svint16_t test_svreinterpret_s16_u8(svuint8_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s16_u8
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 16 x i8> %op to <vscale x 8 x i16>
+  // CHECK: ret <vscale x 8 x i16> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_s16,_u8,,)(op);
+}
+
+svint16_t test_svreinterpret_s16_u16(svuint16_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s16_u16
+  // CHECK: ret <vscale x 8 x i16> %op
+  return SVE_ACLE_FUNC(svreinterpret_s16,_u16,,)(op);
+}
+
+svint16_t test_svreinterpret_s16_u32(svuint32_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s16_u32
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 4 x i32> %op to <vscale x 8 x i16>
+  // CHECK: ret <vscale x 8 x i16> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_s16,_u32,,)(op);
+}
+
+svint16_t test_svreinterpret_s16_u64(svuint64_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s16_u64
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 2 x i64> %op to <vscale x 8 x i16>
+  // CHECK: ret <vscale x 8 x i16> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_s16,_u64,,)(op);
+}
+
+svint16_t test_svreinterpret_s16_f16(svfloat16_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s16_f16
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 8 x half> %op to <vscale x 8 x i16>
+  // CHECK: ret <vscale x 8 x i16> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_s16,_f16,,)(op);
+}
+
+svint16_t test_svreinterpret_s16_f32(svfloat32_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s16_f32
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 4 x float> %op to <vscale x 8 x i16>
+  // CHECK: ret <vscale x 8 x i16> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_s16,_f32,,)(op);
+}
+
+svint16_t test_svreinterpret_s16_f64(svfloat64_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s16_f64
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 2 x double> %op to <vscale x 8 x i16>
+  // CHECK: ret <vscale x 8 x i16> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_s16,_f64,,)(op);
+}
+
+svint32_t test_svreinterpret_s32_s8(svint8_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s32_s8
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 16 x i8> %op to <vscale x 4 x i32>
+  // CHECK: ret <vscale x 4 x i32> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_s32,_s8,,)(op);
+}
+
+svint32_t test_svreinterpret_s32_s16(svint16_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s32_s16
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 8 x i16> %op to <vscale x 4 x i32>
+  // CHECK: ret <vscale x 4 x i32> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_s32,_s16,,)(op);
+}
+
+svint32_t test_svreinterpret_s32_s32(svint32_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s32_s32
+  // CHECK: ret <vscale x 4 x i32> %op
+  return SVE_ACLE_FUNC(svreinterpret_s32,_s32,,)(op);
+}
+
+svint32_t test_svreinterpret_s32_s64(svint64_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s32_s64
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 2 x i64> %op to <vscale x 4 x i32>
+  // CHECK: ret <vscale x 4 x i32> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_s32,_s64,,)(op);
+}
+
+svint32_t test_svreinterpret_s32_u8(svuint8_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s32_u8
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 16 x i8> %op to <vscale x 4 x i32>
+  // CHECK: ret <vscale x 4 x i32> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_s32,_u8,,)(op);
+}
+
+svint32_t test_svreinterpret_s32_u16(svuint16_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s32_u16
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 8 x i16> %op to <vscale x 4 x i32>
+  // CHECK: ret <vscale x 4 x i32> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_s32,_u16,,)(op);
+}
+
+svint32_t test_svreinterpret_s32_u32(svuint32_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s32_u32
+  // CHECK: ret <vscale x 4 x i32> %op
+  return SVE_ACLE_FUNC(svreinterpret_s32,_u32,,)(op);
+}
+
+svint32_t test_svreinterpret_s32_u64(svuint64_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s32_u64
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 2 x i64> %op to <vscale x 4 x i32>
+  // CHECK: ret <vscale x 4 x i32> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_s32,_u64,,)(op);
+}
+
+svint32_t test_svreinterpret_s32_f16(svfloat16_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s32_f16
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 8 x half> %op to <vscale x 4 x i32>
+  // CHECK: ret <vscale x 4 x i32> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_s32,_f16,,)(op);
+}
+
+svint32_t test_svreinterpret_s32_f32(svfloat32_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s32_f32
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 4 x float> %op to <vscale x 4 x i32>
+  // CHECK: ret <vscale x 4 x i32> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_s32,_f32,,)(op);
+}
+
+svint32_t test_svreinterpret_s32_f64(svfloat64_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s32_f64
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 2 x double> %op to <vscale x 4 x i32>
+  // CHECK: ret <vscale x 4 x i32> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_s32,_f64,,)(op);
+}
+
+svint64_t test_svreinterpret_s64_s8(svint8_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s64_s8
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 16 x i8> %op to <vscale x 2 x i64>
+  // CHECK: ret <vscale x 2 x i64> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_s64,_s8,,)(op);
+}
+
+svint64_t test_svreinterpret_s64_s16(svint16_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s64_s16
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 8 x i16> %op to <vscale x 2 x i64>
+  // CHECK: ret <vscale x 2 x i64> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_s64,_s16,,)(op);
+}
+
+svint64_t test_svreinterpret_s64_s32(svint32_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s64_s32
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 4 x i32> %op to <vscale x 2 x i64>
+  // CHECK: ret <vscale x 2 x i64> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_s64,_s32,,)(op);
+}
+
+svint64_t test_svreinterpret_s64_s64(svint64_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s64_s64
+  // CHECK: ret <vscale x 2 x i64> %op
+  return SVE_ACLE_FUNC(svreinterpret_s64,_s64,,)(op);
+}
+
+svint64_t test_svreinterpret_s64_u8(svuint8_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s64_u8
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 16 x i8> %op to <vscale x 2 x i64>
+  // CHECK: ret <vscale x 2 x i64> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_s64,_u8,,)(op);
+}
+
+svint64_t test_svreinterpret_s64_u16(svuint16_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s64_u16
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 8 x i16> %op to <vscale x 2 x i64>
+  // CHECK: ret <vscale x 2 x i64> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_s64,_u16,,)(op);
+}
+
+svint64_t test_svreinterpret_s64_u32(svuint32_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s64_u32
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 4 x i32> %op to <vscale x 2 x i64>
+  // CHECK: ret <vscale x 2 x i64> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_s64,_u32,,)(op);
+}
+
+svint64_t test_svreinterpret_s64_u64(svuint64_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s64_u64
+  // CHECK: ret <vscale x 2 x i64> %op
+  return SVE_ACLE_FUNC(svreinterpret_s64,_u64,,)(op);
+}
+
+svint64_t test_svreinterpret_s64_f16(svfloat16_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s64_f16
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 8 x half> %op to <vscale x 2 x i64>
+  // CHECK: ret <vscale x 2 x i64> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_s64,_f16,,)(op);
+}
+
+svint64_t test_svreinterpret_s64_f32(svfloat32_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s64_f32
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 4 x float> %op to <vscale x 2 x i64>
+  // CHECK: ret <vscale x 2 x i64> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_s64,_f32,,)(op);
+}
+
+svint64_t test_svreinterpret_s64_f64(svfloat64_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_s64_f64
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 2 x double> %op to <vscale x 2 x i64>
+  // CHECK: ret <vscale x 2 x i64> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_s64,_f64,,)(op);
+}
+
+svuint8_t test_svreinterpret_u8_s8(svint8_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u8_s8
+  // CHECK: ret <vscale x 16 x i8> %op
+  return SVE_ACLE_FUNC(svreinterpret_u8,_s8,,)(op);
+}
+
+svuint8_t test_svreinterpret_u8_s16(svint16_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u8_s16
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 8 x i16> %op to <vscale x 16 x i8>
+  // CHECK: ret <vscale x 16 x i8> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_u8,_s16,,)(op);
+}
+
+svuint8_t test_svreinterpret_u8_s32(svint32_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u8_s32
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 4 x i32> %op to <vscale x 16 x i8>
+  // CHECK: ret <vscale x 16 x i8> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_u8,_s32,,)(op);
+}
+
+svuint8_t test_svreinterpret_u8_s64(svint64_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u8_s64
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 2 x i64> %op to <vscale x 16 x i8>
+  // CHECK: ret <vscale x 16 x i8> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_u8,_s64,,)(op);
+}
+
+svuint8_t test_svreinterpret_u8_u8(svuint8_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u8_u8
+  // CHECK: ret <vscale x 16 x i8> %op
+  return SVE_ACLE_FUNC(svreinterpret_u8,_u8,,)(op);
+}
+
+svuint8_t test_svreinterpret_u8_u16(svuint16_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u8_u16
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 8 x i16> %op to <vscale x 16 x i8>
+  // CHECK: ret <vscale x 16 x i8> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_u8,_u16,,)(op);
+}
+
+svuint8_t test_svreinterpret_u8_u32(svuint32_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u8_u32
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 4 x i32> %op to <vscale x 16 x i8>
+  // CHECK: ret <vscale x 16 x i8> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_u8,_u32,,)(op);
+}
+
+svuint8_t test_svreinterpret_u8_u64(svuint64_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u8_u64
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 2 x i64> %op to <vscale x 16 x i8>
+  // CHECK: ret <vscale x 16 x i8> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_u8,_u64,,)(op);
+}
+
+svuint8_t test_svreinterpret_u8_f16(svfloat16_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u8_f16
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 8 x half> %op to <vscale x 16 x i8>
+  // CHECK: ret <vscale x 16 x i8> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_u8,_f16,,)(op);
+}
+
+svuint8_t test_svreinterpret_u8_f32(svfloat32_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u8_f32
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 4 x float> %op to <vscale x 16 x i8>
+  // CHECK: ret <vscale x 16 x i8> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_u8,_f32,,)(op);
+}
+
+svuint8_t test_svreinterpret_u8_f64(svfloat64_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u8_f64
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 2 x double> %op to <vscale x 16 x i8>
+  // CHECK: ret <vscale x 16 x i8> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_u8,_f64,,)(op);
+}
+
+svuint16_t test_svreinterpret_u16_s8(svint8_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u16_s8
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 16 x i8> %op to <vscale x 8 x i16>
+  // CHECK: ret <vscale x 8 x i16> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_u16,_s8,,)(op);
+}
+
+svuint16_t test_svreinterpret_u16_s16(svint16_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u16_s16
+  // CHECK: ret <vscale x 8 x i16> %op
+  return SVE_ACLE_FUNC(svreinterpret_u16,_s16,,)(op);
+}
+
+svuint16_t test_svreinterpret_u16_s32(svint32_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u16_s32
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 4 x i32> %op to <vscale x 8 x i16>
+  // CHECK: ret <vscale x 8 x i16> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_u16,_s32,,)(op);
+}
+
+svuint16_t test_svreinterpret_u16_s64(svint64_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u16_s64
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 2 x i64> %op to <vscale x 8 x i16>
+  // CHECK: ret <vscale x 8 x i16> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_u16,_s64,,)(op);
+}
+
+svuint16_t test_svreinterpret_u16_u8(svuint8_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u16_u8
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 16 x i8> %op to <vscale x 8 x i16>
+  // CHECK: ret <vscale x 8 x i16> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_u16,_u8,,)(op);
+}
+
+svuint16_t test_svreinterpret_u16_u16(svuint16_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u16_u16
+  // CHECK: ret <vscale x 8 x i16> %op
+  return SVE_ACLE_FUNC(svreinterpret_u16,_u16,,)(op);
+}
+
+svuint16_t test_svreinterpret_u16_u32(svuint32_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u16_u32
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 4 x i32> %op to <vscale x 8 x i16>
+  // CHECK: ret <vscale x 8 x i16> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_u16,_u32,,)(op);
+}
+
+svuint16_t test_svreinterpret_u16_u64(svuint64_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u16_u64
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 2 x i64> %op to <vscale x 8 x i16>
+  // CHECK: ret <vscale x 8 x i16> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_u16,_u64,,)(op);
+}
+
+svuint16_t test_svreinterpret_u16_f16(svfloat16_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u16_f16
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 8 x half> %op to <vscale x 8 x i16>
+  // CHECK: ret <vscale x 8 x i16> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_u16,_f16,,)(op);
+}
+
+svuint16_t test_svreinterpret_u16_f32(svfloat32_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u16_f32
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 4 x float> %op to <vscale x 8 x i16>
+  // CHECK: ret <vscale x 8 x i16> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_u16,_f32,,)(op);
+}
+
+svuint16_t test_svreinterpret_u16_f64(svfloat64_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u16_f64
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 2 x double> %op to <vscale x 8 x i16>
+  // CHECK: ret <vscale x 8 x i16> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_u16,_f64,,)(op);
+}
+
+svuint32_t test_svreinterpret_u32_s8(svint8_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u32_s8
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 16 x i8> %op to <vscale x 4 x i32>
+  // CHECK: ret <vscale x 4 x i32> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_u32,_s8,,)(op);
+}
+
+svuint32_t test_svreinterpret_u32_s16(svint16_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u32_s16
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 8 x i16> %op to <vscale x 4 x i32>
+  // CHECK: ret <vscale x 4 x i32> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_u32,_s16,,)(op);
+}
+
+svuint32_t test_svreinterpret_u32_s32(svint32_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u32_s32
+  // CHECK: ret <vscale x 4 x i32> %op
+  return SVE_ACLE_FUNC(svreinterpret_u32,_s32,,)(op);
+}
+
+svuint32_t test_svreinterpret_u32_s64(svint64_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u32_s64
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 2 x i64> %op to <vscale x 4 x i32>
+  // CHECK: ret <vscale x 4 x i32> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_u32,_s64,,)(op);
+}
+
+svuint32_t test_svreinterpret_u32_u8(svuint8_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u32_u8
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 16 x i8> %op to <vscale x 4 x i32>
+  // CHECK: ret <vscale x 4 x i32> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_u32,_u8,,)(op);
+}
+
+svuint32_t test_svreinterpret_u32_u16(svuint16_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u32_u16
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 8 x i16> %op to <vscale x 4 x i32>
+  // CHECK: ret <vscale x 4 x i32> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_u32,_u16,,)(op);
+}
+
+svuint32_t test_svreinterpret_u32_u32(svuint32_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u32_u32
+  // CHECK: ret <vscale x 4 x i32> %op
+  return SVE_ACLE_FUNC(svreinterpret_u32,_u32,,)(op);
+}
+
+svuint32_t test_svreinterpret_u32_u64(svuint64_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u32_u64
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 2 x i64> %op to <vscale x 4 x i32>
+  // CHECK: ret <vscale x 4 x i32> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_u32,_u64,,)(op);
+}
+
+svuint32_t test_svreinterpret_u32_f16(svfloat16_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u32_f16
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 8 x half> %op to <vscale x 4 x i32>
+  // CHECK: ret <vscale x 4 x i32> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_u32,_f16,,)(op);
+}
+
+svuint32_t test_svreinterpret_u32_f32(svfloat32_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u32_f32
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 4 x float> %op to <vscale x 4 x i32>
+  // CHECK: ret <vscale x 4 x i32> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_u32,_f32,,)(op);
+}
+
+svuint32_t test_svreinterpret_u32_f64(svfloat64_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u32_f64
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 2 x double> %op to <vscale x 4 x i32>
+  // CHECK: ret <vscale x 4 x i32> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_u32,_f64,,)(op);
+}
+
+svuint64_t test_svreinterpret_u64_s8(svint8_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u64_s8
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 16 x i8> %op to <vscale x 2 x i64>
+  // CHECK: ret <vscale x 2 x i64> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_u64,_s8,,)(op);
+}
+
+svuint64_t test_svreinterpret_u64_s16(svint16_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u64_s16
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 8 x i16> %op to <vscale x 2 x i64>
+  // CHECK: ret <vscale x 2 x i64> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_u64,_s16,,)(op);
+}
+
+svuint64_t test_svreinterpret_u64_s32(svint32_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u64_s32
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 4 x i32> %op to <vscale x 2 x i64>
+  // CHECK: ret <vscale x 2 x i64> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_u64,_s32,,)(op);
+}
+
+svuint64_t test_svreinterpret_u64_s64(svint64_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u64_s64
+  // CHECK: ret <vscale x 2 x i64> %op
+  return SVE_ACLE_FUNC(svreinterpret_u64,_s64,,)(op);
+}
+
+svuint64_t test_svreinterpret_u64_u8(svuint8_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u64_u8
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 16 x i8> %op to <vscale x 2 x i64>
+  // CHECK: ret <vscale x 2 x i64> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_u64,_u8,,)(op);
+}
+
+svuint64_t test_svreinterpret_u64_u16(svuint16_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u64_u16
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 8 x i16> %op to <vscale x 2 x i64>
+  // CHECK: ret <vscale x 2 x i64> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_u64,_u16,,)(op);
+}
+
+svuint64_t test_svreinterpret_u64_u32(svuint32_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u64_u32
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 4 x i32> %op to <vscale x 2 x i64>
+  // CHECK: ret <vscale x 2 x i64> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_u64,_u32,,)(op);
+}
+
+svuint64_t test_svreinterpret_u64_u64(svuint64_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u64_u64
+  // CHECK: ret <vscale x 2 x i64> %op
+  return SVE_ACLE_FUNC(svreinterpret_u64,_u64,,)(op);
+}
+
+svuint64_t test_svreinterpret_u64_f16(svfloat16_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u64_f16
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 8 x half> %op to <vscale x 2 x i64>
+  // CHECK: ret <vscale x 2 x i64> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_u64,_f16,,)(op);
+}
+
+svuint64_t test_svreinterpret_u64_f32(svfloat32_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u64_f32
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 4 x float> %op to <vscale x 2 x i64>
+  // CHECK: ret <vscale x 2 x i64> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_u64,_f32,,)(op);
+}
+
+svuint64_t test_svreinterpret_u64_f64(svfloat64_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_u64_f64
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 2 x double> %op to <vscale x 2 x i64>
+  // CHECK: ret <vscale x 2 x i64> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_u64,_f64,,)(op);
+}
+
+svfloat16_t test_svreinterpret_f16_s8(svint8_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_f16_s8
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 16 x i8> %op to <vscale x 8 x half>
+  // CHECK: ret <vscale x 8 x half> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_f16,_s8,,)(op);
+}
+
+svfloat16_t test_svreinterpret_f16_s16(svint16_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_f16_s16
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 8 x i16> %op to <vscale x 8 x half>
+  // CHECK: ret <vscale x 8 x half> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_f16,_s16,,)(op);
+}
+
+svfloat16_t test_svreinterpret_f16_s32(svint32_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_f16_s32
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 4 x i32> %op to <vscale x 8 x half>
+  // CHECK: ret <vscale x 8 x half> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_f16,_s32,,)(op);
+}
+
+svfloat16_t test_svreinterpret_f16_s64(svint64_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_f16_s64
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 2 x i64> %op to <vscale x 8 x half>
+  // CHECK: ret <vscale x 8 x half> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_f16,_s64,,)(op);
+}
+
+svfloat16_t test_svreinterpret_f16_u8(svuint8_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_f16_u8
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 16 x i8> %op to <vscale x 8 x half>
+  // CHECK: ret <vscale x 8 x half> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_f16,_u8,,)(op);
+}
+
+svfloat16_t test_svreinterpret_f16_u16(svuint16_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_f16_u16
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 8 x i16> %op to <vscale x 8 x half>
+  // CHECK: ret <vscale x 8 x half> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_f16,_u16,,)(op);
+}
+
+svfloat16_t test_svreinterpret_f16_u32(svuint32_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_f16_u32
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 4 x i32> %op to <vscale x 8 x half>
+  // CHECK: ret <vscale x 8 x half> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_f16,_u32,,)(op);
+}
+
+svfloat16_t test_svreinterpret_f16_u64(svuint64_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_f16_u64
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 2 x i64> %op to <vscale x 8 x half>
+  // CHECK: ret <vscale x 8 x half> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_f16,_u64,,)(op);
+}
+
+svfloat16_t test_svreinterpret_f16_f16(svfloat16_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_f16_f16
+  // CHECK: ret <vscale x 8 x half> %op
+  return SVE_ACLE_FUNC(svreinterpret_f16,_f16,,)(op);
+}
+
+svfloat16_t test_svreinterpret_f16_f32(svfloat32_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_f16_f32
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 4 x float> %op to <vscale x 8 x half>
+  // CHECK: ret <vscale x 8 x half> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_f16,_f32,,)(op);
+}
+
+svfloat16_t test_svreinterpret_f16_f64(svfloat64_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_f16_f64
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 2 x double> %op to <vscale x 8 x half>
+  // CHECK: ret <vscale x 8 x half> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_f16,_f64,,)(op);
+}
+
+svfloat32_t test_svreinterpret_f32_s8(svint8_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_f32_s8
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 16 x i8> %op to <vscale x 4 x float>
+  // CHECK: ret <vscale x 4 x float> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_f32,_s8,,)(op);
+}
+
+svfloat32_t test_svreinterpret_f32_s16(svint16_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_f32_s16
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 8 x i16> %op to <vscale x 4 x float>
+  // CHECK: ret <vscale x 4 x float> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_f32,_s16,,)(op);
+}
+
+svfloat32_t test_svreinterpret_f32_s32(svint32_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_f32_s32
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 4 x i32> %op to <vscale x 4 x float>
+  // CHECK: ret <vscale x 4 x float> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_f32,_s32,,)(op);
+}
+
+svfloat32_t test_svreinterpret_f32_s64(svint64_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_f32_s64
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 2 x i64> %op to <vscale x 4 x float>
+  // CHECK: ret <vscale x 4 x float> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_f32,_s64,,)(op);
+}
+
+svfloat32_t test_svreinterpret_f32_u8(svuint8_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_f32_u8
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 16 x i8> %op to <vscale x 4 x float>
+  // CHECK: ret <vscale x 4 x float> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_f32,_u8,,)(op);
+}
+
+svfloat32_t test_svreinterpret_f32_u16(svuint16_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_f32_u16
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 8 x i16> %op to <vscale x 4 x float>
+  // CHECK: ret <vscale x 4 x float> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_f32,_u16,,)(op);
+}
+
+svfloat32_t test_svreinterpret_f32_u32(svuint32_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_f32_u32
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 4 x i32> %op to <vscale x 4 x float>
+  // CHECK: ret <vscale x 4 x float> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_f32,_u32,,)(op);
+}
+
+svfloat32_t test_svreinterpret_f32_u64(svuint64_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_f32_u64
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 2 x i64> %op to <vscale x 4 x float>
+  // CHECK: ret <vscale x 4 x float> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_f32,_u64,,)(op);
+}
+
+svfloat32_t test_svreinterpret_f32_f16(svfloat16_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_f32_f16
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 8 x half> %op to <vscale x 4 x float>
+  // CHECK: ret <vscale x 4 x float> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_f32,_f16,,)(op);
+}
+
+svfloat32_t test_svreinterpret_f32_f32(svfloat32_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_f32_f32
+  // CHECK: ret <vscale x 4 x float> %op
+  return SVE_ACLE_FUNC(svreinterpret_f32,_f32,,)(op);
+}
+
+svfloat32_t test_svreinterpret_f32_f64(svfloat64_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_f32_f64
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 2 x double> %op to <vscale x 4 x float>
+  // CHECK: ret <vscale x 4 x float> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_f32,_f64,,)(op);
+}
+
+svfloat64_t test_svreinterpret_f64_s8(svint8_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_f64_s8
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 16 x i8> %op to <vscale x 2 x double>
+  // CHECK: ret <vscale x 2 x double> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_f64,_s8,,)(op);
+}
+
+svfloat64_t test_svreinterpret_f64_s16(svint16_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_f64_s16
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 8 x i16> %op to <vscale x 2 x double>
+  // CHECK: ret <vscale x 2 x double> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_f64,_s16,,)(op);
+}
+
+svfloat64_t test_svreinterpret_f64_s32(svint32_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_f64_s32
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 4 x i32> %op to <vscale x 2 x double>
+  // CHECK: ret <vscale x 2 x double> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_f64,_s32,,)(op);
+}
+
+svfloat64_t test_svreinterpret_f64_s64(svint64_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_f64_s64
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 2 x i64> %op to <vscale x 2 x double>
+  // CHECK: ret <vscale x 2 x double> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_f64,_s64,,)(op);
+}
+
+svfloat64_t test_svreinterpret_f64_u8(svuint8_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_f64_u8
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 16 x i8> %op to <vscale x 2 x double>
+  // CHECK: ret <vscale x 2 x double> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_f64,_u8,,)(op);
+}
+
+svfloat64_t test_svreinterpret_f64_u16(svuint16_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_f64_u16
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 8 x i16> %op to <vscale x 2 x double>
+  // CHECK: ret <vscale x 2 x double> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_f64,_u16,,)(op);
+}
+
+svfloat64_t test_svreinterpret_f64_u32(svuint32_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_f64_u32
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 4 x i32> %op to <vscale x 2 x double>
+  // CHECK: ret <vscale x 2 x double> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_f64,_u32,,)(op);
+}
+
+svfloat64_t test_svreinterpret_f64_u64(svuint64_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_f64_u64
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 2 x i64> %op to <vscale x 2 x double>
+  // CHECK: ret <vscale x 2 x double> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_f64,_u64,,)(op);
+}
+
+svfloat64_t test_svreinterpret_f64_f16(svfloat16_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_f64_f16
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 8 x half> %op to <vscale x 2 x double>
+  // CHECK: ret <vscale x 2 x double> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_f64,_f16,,)(op);
+}
+
+svfloat64_t test_svreinterpret_f64_f32(svfloat32_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_f64_f32
+  // CHECK: %[[CAST:.*]] = bitcast <vscale x 4 x float> %op to <vscale x 2 x double>
+  // CHECK: ret <vscale x 2 x double> %[[CAST]]
+  return SVE_ACLE_FUNC(svreinterpret_f64,_f32,,)(op);
+}
+
+svfloat64_t test_svreinterpret_f64_f64(svfloat64_t op)
+{
+  // CHECK-LABEL: test_svreinterpret_f64_f64
+  // CHECK: ret <vscale x 2 x double> %op
+  return SVE_ACLE_FUNC(svreinterpret_f64,_f64,,)(op);
+}

clang/utils/TableGen/SveEmitter.cpp

private:
   std::string getMergeSuffix() const { return MergeSuffix; }
   std::string mangleName(ClassKind LocalCK) const;
   std::string replaceTemplatedArgs(std::string Name, TypeSpec TS,
                                    std::string Proto) const;
 };
 
 class SVEEmitter {
 private:
+  // The reinterpret builtins are generated separately because they
+  // need the cross product of all types (121 functions in total),
+  // which is inconvenient to specify in the arm_sve.td file or
+  // generate in CGBuiltin.cpp.
+  struct ReinterpretTypeInfo {
+    const char *Suffix;
+    const char *Type;
+    const char *BuiltinType;
+  };
+  SmallVector<ReinterpretTypeInfo, 11> Reinterprets = {
+      {"s8", "svint8_t", "q16Sc"},   {"s16", "svint16_t", "q8Ss"},
+      {"s32", "svint32_t", "q4Si"},  {"s64", "svint64_t", "q2SWi"},
+      {"u8", "svuint8_t", "q16Uc"},  {"u16", "svuint16_t", "q8Us"},
+      {"u32", "svuint32_t", "q4Ui"}, {"u64", "svuint64_t", "q2UWi"},
+      {"f16", "svfloat16_t", "q8h"}, {"f32", "svfloat32_t", "q4f"},
+      {"f64", "svfloat64_t", "q2d"}};
+
   RecordKeeper &Records;
   llvm::StringMap<uint64_t> EltTypes;
   llvm::StringMap<uint64_t> MemEltTypes;
   llvm::StringMap<uint64_t> FlagTypes;
   llvm::StringMap<uint64_t> MergeTypes;
   llvm::StringMap<uint64_t> ImmCheckTypes;
 
 public:
void SVEEmitter::createHeader(raw_ostream &OS) {
   OS << "typedef __SVUint16_t svuint16_t;\n";
   OS << "typedef __SVUint32_t svuint32_t;\n";
   OS << "typedef __SVUint64_t svuint64_t;\n";
   OS << "typedef __SVFloat16_t svfloat16_t;\n";
   OS << "typedef __SVFloat32_t svfloat32_t;\n";
   OS << "typedef __SVFloat64_t svfloat64_t;\n";
   OS << "typedef __SVBool_t  svbool_t;\n\n";
 
   OS << "typedef enum\n";
   OS << "{\n";
   OS << "  SV_POW2 = 0,\n";
   OS << "  SV_VL1 = 1,\n";
   OS << "  SV_VL2 = 2,\n";
   OS << "  SV_VL3 = 3,\n";
   OS << "  SV_VL4 = 4,\n";
   OS << "  SV_VL5 = 5,\n";
   OS << "  SV_VL6 = 6,\n";
void SVEEmitter::createHeader(raw_ostream &OS) {
   OS << "} sv_prfop;\n\n";
 
   OS << "/* Function attributes */\n";
   OS << "#define __ai static inline __attribute__((__always_inline__, "
         "__nodebug__))\n";
   OS << "#define __aio static inline __attribute__((__always_inline__, "
         "__nodebug__, __overloadable__))\n\n";
 
+  // Add reinterpret functions.
+  for (auto ShortForm : { false, true } )
+    for (const ReinterpretTypeInfo &From : Reinterprets)
+      for (const ReinterpretTypeInfo &To : Reinterprets) {
+        if (ShortForm) {
+          OS << "__aio " << From.Type << " svreinterpret_" << From.Suffix;
+          OS << "(" << To.Type << " op) {\n";
+          OS << "  return __builtin_sve_reinterpret_" << From.Suffix << "_"
+             << To.Suffix << "(op);\n";
+          OS << "}\n\n";
+        } else
+          OS << "#define svreinterpret_" << From.Suffix << "_" << To.Suffix
+             << "(...) __builtin_sve_reinterpret_" << From.Suffix << "_"
+             << To.Suffix << "(__VA_ARGS__)\n";
+      }
+
   SmallVector<std::unique_ptr<Intrinsic>, 128> Defs;
   std::vector<Record *> RV = Records.getAllDerivedDefinitions("Inst");
   for (auto *R : RV)
     createIntrinsic(R, Defs);
 
   // Sort intrinsics in header file by following order/priority:
   // - Architectural guard (i.e. does it require SVE2 or SVE2_AES)
   // - Class (is intrinsic overloaded or not)
void SVEEmitter::createHeader(raw_ostream &OS) {
   }
 
   if (!InGuard.empty())
     OS << "#endif  //" << InGuard << "\n";
 
 OS << "#define MAKE_DUPQ_16(TYPE,SUFFIX,PTRUE) \\\n";
 OS << " __ai sv##TYPE svdupq_##SUFFIX(TYPE x0, TYPE x1,\\\n";
 OS << "                               TYPE x2, TYPE x3,\\\n";
 OS << "                               TYPE x4, TYPE x5,\\\n";
 OS << "                               TYPE x6, TYPE x7,\\\n";
 OS << "                               TYPE x8, TYPE x9,\\\n";
 OS << "                               TYPE x10, TYPE x11,\\\n";
 OS << "                               TYPE x12, TYPE x13,\\\n";
 OS << "                               TYPE x14, TYPE x15) {\\\n";
 OS << "  TYPE x[] = { x0, x1, x2, x3, x4, x5, x6, x7,\\\n";
 OS << "               x8, x9, x10, x11, x12, x13, x14, x15 };\\\n";
 OS << "  return svld1rq_##SUFFIX(PTRUE, x);\\\n";
void SVEEmitter::createBuiltins(raw_ostream &OS) {
   OS << "#ifdef GET_SVE_BUILTINS\n";
   for (auto &Def : Defs) {
     // Only create BUILTINs for non-overloaded intrinsics, as overloaded
     // declarations only live in the header file.
     if (Def->getClassKind() != ClassG)
       OS << "BUILTIN(__builtin_sve_" << Def->getMangledName() << ", \""
          << Def->getBuiltinTypeStr() << "\", \"n\")\n";
   }
+
+  // Add reinterpret builtins
+  for (const ReinterpretTypeInfo &From : Reinterprets)
+    for (const ReinterpretTypeInfo &To : Reinterprets)
+      OS << "BUILTIN(__builtin_sve_reinterpret_" << From.Suffix << "_"
+         << To.Suffix << +", \"" << From.BuiltinType << To.BuiltinType
+         << "\", \"n\")\n";
+
   OS << "#endif\n\n";
   }
 
 void SVEEmitter::createCodeGenMap(raw_ostream &OS) {
   std::vector<Record *> RV = Records.getAllDerivedDefinitions("Inst");
   SmallVector<std::unique_ptr<Intrinsic>, 128> Defs;
   for (auto *R : RV)
     createIntrinsic(R, Defs);
 
   // The mappings must be sorted based on BuiltinID.