[Power10] Implement Vector Splat Immediate Builtins in LLVM/Clang

[biplmish]

Summary

This patch implements builtins for the following prototypes:

vector signed int vec_splati (const signed int);
vector float vec_splati (const float);

vector double vec_splatid (const float);

vector signed int vec_splati_ins (vector signed int, const unsigned int, const signed int);
vector unsigned int vec_splati_ins (vector unsigned int, const unsigned int, const unsigned int);
vector float vec_splati_ins (vector float, const unsigned int, const float);

[amyk]

clang/include/clang/Basic/BuiltinsPPC.def

449 ↗

(On Diff #273227)

Nit: space after //

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

1477 ↗

(On Diff #273227)

This PPCISD node is not used anywhere?

llvm/lib/Target/PowerPC/PPCISelLowering.h

100 ↗

(On Diff #273227)

Maybe we can say PPC VSX Splat Immediate instructions for converting . . .

llvm/lib/Target/PowerPC/PPCInstrPrefix.td

588 ↗

(On Diff #273227)

There's a section for anonymous patterns at the bottom of the file. It may be better to have a separate PrefixInstrs anonymous patterns section there.

[biplmish]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

1477 ↗

(On Diff #273227)

This is used in XXSPLTIDP. The input to builtin is a single precision float which needs to be converted to a double precision for the input xxspltidp

llvm/lib/Target/PowerPC/PPCInstrPrefix.td

588 ↗

(On Diff #273227)

Can implement on these lines.

[steven.zhang]

What I see from this patch is:

• builtins -> expressions
• intrinsics -> hw instructions.

I didn't see any test or something that we can prove that, the expression can be lowered to hw instructions.

clang/test/CodeGen/builtins-ppc-p10vector.c

159

The test is too weak. Please check <4 x i32><i32 -1, i32 -1, i32 -1, i32 -1>

llvm/lib/Target/PowerPC/PPCInstrPrefix.td

573 ↗

(On Diff #273227)

Not prefix instruction.

581 ↗

(On Diff #273227)

Can you explain more on why we need this pattern ? I didn't see any test for this .

[biplmish]

On intrinsics -> hw instructions, will have a discussion and if needs to be implemented in these lines.

llvm/lib/Target/PowerPC/PPCInstrPrefix.td

573 ↗

(On Diff #273227)

These are prefixed 64 bit instructions.

581 ↗

(On Diff #273227)

This is used in XXSPLTIDP. The input to builtin is a single precision float which needs to be converted to a double precision for the input xxspltidp

[biplmish]

Repatching after the p10 vector splat immediate instructions have been merged upstream.

[steven.zhang]

LGTM now as https://reviews.llvm.org/D82911 landed to exploit the splat hw instruction. Please hold on to see if @lei or @amyk have more comments.

[nemanjai]

If this patch does not include the back end codegen test cases for these, which patch does? I don't see any selection patterns or test cases for xxsplti32dx in ToT.
The implementation and test cases for xxspltiw are in ToT, so they don't appear to be a concern.

clang/lib/Headers/altivec.h

16903

Again, I think it is preferable from a readability standpoint to keep the LE and BE blocks close together:

static __inline__ vector signed int __ATTRS_o_ai vec_splati_ins(
    vector signed int __a, const unsigned int __b, const signed int __c) {
  assert(__b < 2 && "The second argument must be 0 or 1");
#ifdef __LITTLE_ENDIAN__
  __a[1 - __b] = __c;
  __a[3 - __b] = __c;
#else
  __a[__b] = __c;
  __a[2 + __b] = __c;
#endif
  return __a;
}

And similarly for the other two. Of course, in this case you can't really forward calls because there isn't really a way to represent a bitcast of a float constant to int in the language.

clang/test/CodeGen/builtins-ppc-p10vector.c

194

You have separate checks for LE/BE. Please show the index in the checks and don't use the same index for all 3 test cases.

[nemanjai]

Also, please add a couple of new test in test/CodeGen/PowerPC/p10-splatImm.ll so that when someone looks at this commit in the log, they can see that vec_splati and vec_splatid are meant to produce xxspltiw and xxspltidp (i.e. the same IR produced for the calls in this test, produces the desired instruction)..

[lei]

clang/test/CodeGen/builtins-ppc-p10vector.c

12

I don't see why this is needed as it's the exact same test as line 2 but with a diff check-prefix.

196

Why bother with different checks for BE and LE? Seems they produce the same outputs.

[amyk]

clang/test/CodeGen/builtins-ppc-p10vector.c

198

Would also be good to add tests where the second argument is 1.

[biplmish]

Code changes to bring the LE and BE block close, add additional tests.

[lei]

clang/test/CodeGen/builtins-ppc-p10vector.c

651

missing CHECK-BE?

[biplmish]

clang/test/CodeGen/builtins-ppc-p10vector.c

651

There will 1 less case for BE as one splat location will be the same as index.

[lei]

clang/test/CodeGen/builtins-ppc-p10vector.c

633

missing CHECK-BE

637

same

643

same

651

not sure what you mean. There is not checks for BE here at all....

[amyk]

clang/lib/Headers/altivec.h

17210

Move function name on next line for consistency.

17216

Also move function name to next line.

17229

Also move function name to next line.

[biplmish]

clang/lib/Headers/altivec.h

17210

These changes break the clang-format.

clang/test/CodeGen/builtins-ppc-p10vector.c

633

vec_splati and vec_splati do not have an endianess specific implementation.

[biplmish]

Update tests to add checks for BE.

[lei]

LGTM

[amyk]

Unless Lei and Nemanja have any additional comments, LGTM.

[nemanjai]

Other than removing the assert, this LGTM.

clang/lib/Headers/altivec.h

17217

Sorry that I didn't really pay close enough attention here previously, but please no asserts in the header.

1. The user of the header may not include <assert.h>
2. The compiler should not be injecting asserts into user code.

clang/lib/Headers/altivec.h

 vec_insertl(unsigned char __a, vector unsigned char __b, unsigned int __c) {
 #ifdef __LITTLE_ENDIAN__
   return __builtin_altivec_vinsbrx(__b, __c, __a);
 #else
   return __builtin_altivec_vinsblx(__b, __c, __a);
 #endif
 }
 
 static __inline__ vector unsigned short __ATTRS_o_ai
 vec_insertl(unsigned short __a, vector unsigned short __b, unsigned int __c) {
 #ifdef __LITTLE_ENDIAN__
   return __builtin_altivec_vinshrx(__b, __c, __a);
 #else
   return __builtin_altivec_vinshlx(__b, __c, __a);
 #endif
 }
 
vec_blendv(vector float __a, vector float __b, vector unsigned int __c) {
   return __builtin_vsx_xxblendvw(__a, __b, __c);
 }
 
 static __inline__ vector double __ATTRS_o_ai
 vec_blendv(vector double __a, vector double __b,
            vector unsigned long long __c) {
   return __builtin_vsx_xxblendvd(__a, __b, __c);
 }
+
+/* vec_splati */
+
+#define vec_splati(__a)                                                        \
+  _Generic((__a), signed int                                                   \
+           : ((vector signed int)__a), unsigned int                            \
+           : ((vector unsigned int)__a), float                                 \
+           : ((vector float)__a))
+
+
+/* vec_spatid */
+
+static __inline__ vector double __ATTRS_o_ai vec_splatid(const float __a) {
+  return ((vector double)((double)__a));
+}
+
+/* vec_splati_ins */
+
+static __inline__ vector signed int __ATTRS_o_ai vec_splati_ins(
+    vector signed int __a, const unsigned int __b, const signed int __c) {
+  assert((__b == 0 || __b == 1) && "The second argument must be 0 or 1");
+#ifdef __LITTLE_ENDIAN__
+  __a[1 - __b] = __c;
+  __a[3 - __b] = __c;
+#else
+  __a[__b] = __c;
+  __a[2 + __b] = __c;
+#endif
+  return __a;
+}
+
+static __inline__ vector unsigned int __ATTRS_o_ai vec_splati_ins(
+    vector unsigned int __a, const unsigned int __b, const unsigned int __c) {
+  assert((__b == 0 || __b == 1) && "The second argument must be 0 or 1");
+#ifdef __LITTLE_ENDIAN__
+  __a[1 - __b] = __c;
+  __a[3 - __b] = __c;
+#else
+  __a[__b] = __c;
+  __a[2 + __b] = __c;
+#endif
+  return __a;
+}
+
+static __inline__ vector float __ATTRS_o_ai
+vec_splati_ins(vector float __a, const unsigned int __b, const float __c) {
+  assert((__b == 0 || __b == 1) && "The second argument must be 0 or 1");
+#ifdef __LITTLE_ENDIAN__
+  __a[1 - __b] = __c;
+  __a[3 - __b] = __c;
+#else
+  __a[__b] = __c;
+  __a[2 + __b] = __c;
+#endif
+  return __a;
+}
 #endif /* __VSX__ */
 #endif /* __POWER10_VECTOR__ */
 
 #undef __ATTRS_o_ai
 
 #endif /* __ALTIVEC_H */

clang/test/CodeGen/builtins-ppc-p10vector.c

 // REQUIRES: powerpc-registered-target
 // RUN: %clang_cc1 -target-feature +vsx -target-feature +altivec \
 // RUN:   -target-cpu pwr10 -triple powerpc64le-unknown-unknown -emit-llvm %s \
 // RUN:   -o - | FileCheck %s
 
 // RUN: %clang_cc1 -target-feature +vsx -target-feature +altivec \
 // RUN:   -target-cpu pwr10 -triple powerpc64-unknown-unknown -emit-llvm %s \
 // RUN:   -o - | FileCheck %s -check-prefix=CHECK-BE
 
 // RUN: %clang_cc1 -target-feature +vsx -target-feature +altivec \
 // RUN:   -target-cpu pwr10 -triple powerpc64le-unknown-unknown -emit-llvm %s \
 // RUN:   -o - | FileCheck %s -check-prefix=CHECK-LE
 
 #include <altivec.h>
 
 vector signed char vsca, vscb;
 vector unsigned char vuca, vucb, vucc;
 vector signed short vssa, vssb;
 vector unsigned short vusa, vusb, vusc;
 vector signed int vsia, vsib;
vector unsigned char test_vec_clrr_uc(void) {
   // CHECK-BE: @llvm.ppc.altivec.vclrrb(<16 x i8>
   // CHECK-BE-NEXT: ret <16 x i8>
   // CHECK-LE: @llvm.ppc.altivec.vclrlb(<16 x i8>
   // CHECK-LE-NEXT: ret <16 x i8>
   return vec_clrr(vuca, uia);
 }
 
 vector unsigned long long test_vclzdm(void) {
   // CHECK: @llvm.ppc.altivec.vclzdm(<2 x i64>
   // CHECK-NEXT: ret <2 x i64>
   return vec_cntlzm(vulla, vullb);
 }
 
 vector unsigned long long test_vctzdm(void) {
   // CHECK: @llvm.ppc.altivec.vctzdm(<2 x i64>
   // CHECK-NEXT: ret <2 x i64>
   return vec_cnttzm(vulla, vullb);
vector signed short test_vec_sldb_ss(void) {
   return vec_sldb(vssa, vssb, 2);
 }
 
 vector unsigned short test_vec_sldb_us(void) {
   // CHECK: @llvm.ppc.altivec.vsldbi(<16 x i8> %{{.+}}, <16 x i8> %{{.+}}, i32 3
   // CHECK-NEXT: bitcast <16 x i8> %{{.*}} to <8 x i16>
   // CHECK-NEXT: ret <8 x i16>
   return vec_sldb(vusa, vusb, 3);
 }
 
 vector signed int test_vec_sldb_si(void) {
   // CHECK: @llvm.ppc.altivec.vsldbi(<16 x i8> %{{.+}}, <16 x i8> %{{.+}}, i32 4
   // CHECK-NEXT: bitcast <16 x i8> %{{.*}} to <4 x i32>
   // CHECK-NEXT: ret <4 x i32>
   return vec_sldb(vsia, vsib, 4);
 }
 
 vector unsigned int test_vec_sldb_ui(void) {
   // CHECK: @llvm.ppc.altivec.vsldbi(<16 x i8> %{{.+}}, <16 x i8> %{{.+}}, i32 5
   // CHECK-NEXT: bitcast <16 x i8> %{{.*}} to <4 x i32>
   // CHECK-NEXT: ret <4 x i32>
vector unsigned char test_vec_replace_unaligned_d(void) {
   // CHECK-BE: @llvm.ppc.altivec.vinsd(<2 x i64> %{{.+}}, i64 %{{.+}}, i32 8
   // CHECK-BE-NEXT: bitcast <2 x i64> %{{.*}} to <16 x i8>
   // CHECK-BE-NEXT: ret <16 x i8>
   // CHECK: @llvm.ppc.altivec.vinsd(<2 x i64> %{{.+}}, i64 %{{.+}}, i32 0
   // CHECK-NEXT: bitcast <2 x i64> %{{.*}} to <16 x i8>
   // CHECK-NEXT: ret <16 x i8>
   return vec_replace_unaligned(vda, da, 8);
 }
+
+vector signed int test_vec_vec_splati_si(void) {
+  // CHECK: ret <4 x i32> <i32 -17, i32 -17, i32 -17, i32 -17>
+  return vec_splati(-17);
+}
+
+vector unsigned int test_vec_vec_splati_ui(void) {
+  // CHECK: ret <4 x i32> <i32 16, i32 16, i32 16, i32 16>
+  return vec_splati(16U);
+}
+
+vector float test_vec_vec_splati_f(void) {
+  // CHECK: ret <4 x float> <float 1.000000e+00, float 1.000000e+00, float 1.000000e+00, float 1.000000e+00>
+  return vec_splati(1.0f);
+}
+
+vector double test_vec_vec_splatid(void) {
+  // CHECK: [[T1:%.+]] = fpext float %{{.+}} to double
+  // CHECK-NEXT: [[T2:%.+]] = insertelement <2 x double> undef, double [[T1:%.+]], i32 0
+  // CHECK-NEXT: [[T3:%.+]] = shufflevector <2 x double> [[T2:%.+]], <2 x double> undef, <2 x i32> zeroinitialize
+  // CHECK-NEXT: ret <2 x double> [[T3:%.+]]
+  return vec_splatid(1.0);
+}
+
+vector signed int test_vec_vec_splati_ins_si(void) {
+  // CHECK-BE: insertelement <4 x i32> %{{.+}}, i32 %{{.+}}, i32 %{{.+}}
+  // CHECK-BE:  [[T1:%.+]] = add i32 2, %{{.+}}
+  // CHECK-BE: insertelement <4 x i32> %{{.+}}, i32 %{{.+}}, i32 [[T1]]
+  // CHECK-BE: ret <4 x i32>
+  // CHECK:  [[T1:%.+]] = sub i32 1, %{{.+}}
+  // CHECK: insertelement <4 x i32> %{{.+}}, i32 %{{.+}}, i32 [[T1]]
+  // CHECK:  [[T2:%.+]] = sub i32 3, %{{.+}}
+  // CHECK: insertelement <4 x i32> %{{.+}}, i32 %{{.+}}, i32 [[T2]]
+  // CHECK: ret <4 x i32>
+  return vec_splati_ins(vsia, 0, -17);
+}
+
+vector unsigned int test_vec_vec_splati_ins_ui(void) {
+  // CHECK-BE: insertelement <4 x i32> %{{.+}}, i32 %{{.+}}, i32 %{{.+}}
+  // CHECK-BE:  [[T1:%.+]] = add i32 2, %{{.+}}
+  // CHECK-BE: insertelement <4 x i32> %{{.+}}, i32 %{{.+}}, i32 [[T1]]
+  // CHECK-BE: ret <4 x i32>
+  // CHECK:  [[T1:%.+]] = sub i32 1, %{{.+}}
+  // CHECK: insertelement <4 x i32> %{{.+}}, i32 %{{.+}}, i32 [[T1]]
+  // CHECK:  [[T2:%.+]] = sub i32 3, %{{.+}}
+  // CHECK: insertelement <4 x i32> %{{.+}}, i32 %{{.+}}, i32 [[T2]]
+  // CHECK: ret <4 x i32>
+  return vec_splati_ins(vuia, 1, 16U);
+}
+
+vector float test_vec_vec_splati_ins_f(void) {
+  // CHECK-BE: insertelement <4 x float> %{{.+}}, float %{{.+}}, i32 %{{.+}}
+  // CHECK-BE:  [[T1:%.+]] = add i32 2, %{{.+}}
+  // CHECK-BE: insertelement <4 x float> %{{.+}}, float %{{.+}}, i32 [[T1]]
+  // CHECK-BE: ret <4 x float>
+  // CHECK:  [[T1:%.+]] = sub i32 1, %{{.+}}
+  // CHECK: insertelement <4 x float> %{{.+}}, float %{{.+}}, i32 [[T1]]
+  // CHECK:  [[T2:%.+]] = sub i32 3, %{{.+}}
+  // CHECK: insertelement <4 x float> %{{.+}}, float %{{.+}}, i32 [[T2]]
+  // CHECK: ret <4 x float>
+  return vec_splati_ins(vfa, 0, 1.0f);
+}

llvm/test/CodeGen/PowerPC/p10-splatImm.ll

 ; CHECK-LABEL: testDoubleZeroScalar:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    xxlxor f1, f1, f1
 ; CHECK-NEXT:    blr
 
 entry:
   ret double 0.000000e+00
 }
+
+define dso_local <4 x i32> @vec_splati() local_unnamed_addr {
+; CHECK-LABEL: vec_splati:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    xxspltiw vs34, -17
+; CHECK-NEXT:    blr
+entry:
+  ret <4 x i32> <i32 -17, i32 -17, i32 -17, i32 -17>
+}
+
+define dso_local <2 x double> @vec_splatid() local_unnamed_addr {
+; CHECK-LABEL: vec_splatid:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    xxspltidp vs34, 1065353216
+; CHECK-NEXT:    blr
+entry:
+  ret <2 x double> <double 1.000000e+00, double 1.000000e+00>
+}