[PowerPC][Power10] Implemented Vector Shift Builtins

[Conanap]

Summary

Implemented Builtins for vector right and left shift along with appropriate test cases for ISA3.1. Depends on D83516.

[bsaleil]

Shouldn't we have test cases to test vec_sl, vec_sr and vec_sra ?

llvm/include/llvm/IR/IntrinsicsPowerPC.td

809

nit: indentation issue

llvm/lib/Target/PowerPC/PPCInstrPrefix.td

963

nit: extra spaces before : here and in the next two lines

[amyk]

This will need to be rebased against your 2608 instruction definitions patch.
But yes, I believe you are missing the clang and llc test case for this patch. Requesting changes due to missing tests.

clang/lib/Headers/altivec.h

17151

/* vs[l | r | raq] */ (with a new line after the comment)

[Conanap]

Added tests, added extra comments.

[Conanap]

Also removed instr def as it will be part of D83516.

[amyk]

clang/lib/Headers/altivec.h

17151

I think we can remove /* vector shifts for quadwords */.
Then, we can add a new line after /* vs[l | r | raq] */ for consistency of the comments.

clang/test/CodeGen/builtins-ppc-p10vector.c

593

vector unsigned you mean? For the return of this test and every other unsigned test case.

llvm/lib/Target/PowerPC/PPCInstrPrefix.td

1030

nit: remove extra space after <

1032

nit: remove after space before ,

1039

Unrelated space change?

llvm/test/CodeGen/PowerPC/p10-vector-shift.ll

10

Can probably remove these globals as you are not using them in your test.

[amyk]

clang/lib/Headers/altivec.h

17153

I believe there are supposed to be signed and unsigned versions of the functions? It looks like you currently have all of them as unsigned.

[Conanap]

Formatting fixes, fixed test case return type, updated builtins' signatures to correct signatures.

[stefanp]

Just a few nits for this patch.

Edit: Please wait for Amy to give the approval.

clang/lib/Headers/altivec.h

17158

nit:
Is this supposed to be vec_slq?

llvm/test/CodeGen/PowerPC/p10-vector-shift.ll

11

nit:
If you are going to use #0 you can probably define attributes #0 = { nounwind } at the bottom of this file.

42

nit:
You probably don't need the #1 as it is not defined anyway.

[NeHuang]

Overall seems fine to me, but of course, please wait to hear from Amy.
Just some nits for the test case.

llvm/test/CodeGen/PowerPC/p10-vector-shift.ll

10

nit: please add a description for test purpose.

11

nit: The test name and CHECK-LABEL should use test_vec_vslq

11

+1, if no specific attributes needed. please remove all the #0 and #1

[amyk]

Please also change the function names.

clang/lib/Headers/altivec.h

17151

Actually, sorry, I think this comment should be the following instead:
/* vec_s[l | r | ra] */ since these functions are actually supposed to be vec_sl, vec_sr, vec_sra.

17158

Actually vec_sl seems to be correct. However, that would mean the other functions need to be renamed as the functions are supposed to be: vec_sl, vec_sr, vec_sra. Albion, could you please rename these functions and also ensure your test uses the correct naming.

[Conanap]

Fixed function names, test case clean up

[amyk]

I realize it may be possible to open code these, as these functions already exist in altivec.h. Could you look into if this is the case?

[Conanap]

Converted the impelmentation to an open coded implementation and updated the test cases as appropriate.

[lei]

Please address the auto generated clang-format issues for the added code in this patch.

[Conanap]

Fixed formatting based on bot feedback.

[Conanap]

Replaced a forgotten test file.

[amyk]

clang/lib/Headers/altivec.h

17151

Add a space after this comment.

17161

Please remove any commented code.

17177

Could you please fix the indentation of the returns to make them all consistent?

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

1100 ↗

(On Diff #281783)

No brackets are needed here.

Also, I think it might make sense to move this block into the previous hasP9Altivec condition since in there it has:

setOperationAction(ISD::SHL, MVT::v1i128, Legal);
setOperationAction(ISD::SRL, MVT::v1i128, Legal);

llvm/lib/Target/PowerPC/PPCInstrPrefix.td

1053

There is no need to make a new predicate block, you can put these anonymous patterns in the block above.

1056

I noticed that we have patterns for the PPCISD nodes, but I think no tests to show these patterns?

[amyk]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

1100 ↗

(On Diff #281783)

Oops, I made that comment above since I thought you were putting the block inside a hasP9Altivec block. Sorry.

Maybe you can move this condition right underneath the hasP9Altivec block that contains the SHL/SRL to be closer to those instructions.

[amyk]

llvm/lib/Target/PowerPC/PPCInstrPrefix.td

1056

My apologies, upon closer inspection I noticed that your tests are in fact the ones with the PPCISD nodes but you are missing the regular shl/srl/sra. Please add the tests for these, as well.

[Conanap]

Added shl tests, formatting fixes

clang/lib/Headers/altivec.h

17177

I ran clang-format this time, lmk if we should change this as it doesn't look like clang-format was too consistent with itself

[amyk]

Thanks for addressing the comments. LGTM.

llvm/test/CodeGen/PowerPC/p10-vector-shift.ll

11

nit:

These test cases demonstrate that the vector shift quadword instructions introduced 
within Power10 are correctly exploited.

clang/include/clang/Basic/BuiltinsPPC.def

 BUILTIN(__builtin_altivec_vrlh, "V8sV8sV8Us", "")
 BUILTIN(__builtin_altivec_vrlw, "V4iV4iV4Ui", "")
 BUILTIN(__builtin_altivec_vrld, "V2LLiV2LLiV2ULLi", "")
 
 BUILTIN(__builtin_altivec_vsel_4si, "V4iV4iV4iV4Ui", "")
 
 BUILTIN(__builtin_altivec_vsl, "V4iV4iV4i", "")
 BUILTIN(__builtin_altivec_vslo, "V4iV4iV4i", "")
+BUILTIN(__builtin_altivec_vslq, "V1ULLLiV1ULLLiV1ULLLi", "")
 
 BUILTIN(__builtin_altivec_vsrab, "V16cV16cV16Uc", "")
 BUILTIN(__builtin_altivec_vsrah, "V8sV8sV8Us", "")
 BUILTIN(__builtin_altivec_vsraw, "V4iV4iV4Ui", "")
+BUILTIN(__builtin_altivec_vsraq, "V1ULLLiV1ULLLiV1ULLLi", "")
 
 BUILTIN(__builtin_altivec_vsr, "V4iV4iV4i", "")
 BUILTIN(__builtin_altivec_vsro, "V4iV4iV4i", "")
+BUILTIN(__builtin_altivec_vsrq, "V1ULLLiV1ULLLiV1ULLLi", "")
 
 BUILTIN(__builtin_altivec_vrfin, "V4fV4f", "")
 
 BUILTIN(__builtin_altivec_vrsqrtefp, "V4fV4f", "")
 
 BUILTIN(__builtin_altivec_vsubcuw, "V4UiV4UiV4Ui", "")
 
 BUILTIN(__builtin_altivec_vsum4sbs, "V4SiV16ScV4Si", "")

clang/lib/Headers/altivec.h

#ifdef __LITTLE_ENDIAN__
   __a[3 - __b] = __c;
 #else
   __a[__b] = __c;
   __a[2 + __b] = __c;
 #endif
   return __a;
 }
 #endif /* __VSX__ */
+
+/* vector shifts for quadwords */
+/* vs[l | r | raq] */
+static __inline__ vector unsigned __int128 __ATTRS_o_ai
+vec_slq(vector unsigned __int128 __a, vector unsigned __int128 __b) {
+  return __builtin_altivec_vslq(__a, __b);
+}
+
+static __inline__ vector unsigned __int128 __ATTRS_o_ai
+vec_sl(vector unsigned __int128 __a, vector signed __int128 __b) {
+  return __builtin_altivec_vslq(__a, (vector unsigned __int128) __b);
+}
+
+static __inline__ vector unsigned __int128 __ATTRS_o_ai
+vec_srq(vector unsigned __int128 __a, vector unsigned __int128 __b) {
+  return __builtin_altivec_vsrq(__a, __b);
+}
+
+static __inline__ vector unsigned __int128 __ATTRS_o_ai
+vec_srq(vector unsigned __int128 __a, vector signed __int128 __b) {
+  return __builtin_altivec_vsrq(__a, (vector unsigned __int128) __b);
+}
+
+static __inline__ vector unsigned __int128 __ATTRS_o_ai
+vec_sraq(vector unsigned __int128 __a, vector unsigned __int128 __b) {
+  return __builtin_altivec_vsraq(__a, __b);
+}
+
+static __inline__ vector unsigned __int128 __ATTRS_o_ai
+vec_sraq(vector unsigned __int128 __a, vector signed __int128 __b) {
+  return __builtin_altivec_vsraq(__a, (vector unsigned __int128) __b);
+}
 #endif /* __POWER10_VECTOR__ */
 
 #undef __ATTRS_o_ai
 
 #endif /* __ALTIVEC_H */

clang/test/CodeGen/builtins-ppc-p10vector.c

 vector signed char vsca, vscb;
 vector unsigned char vuca, vucb, vucc;
 vector signed short vssa, vssb;
 vector unsigned short vusa, vusb, vusc;
 vector signed int vsia, vsib;
 vector unsigned int vuia, vuib, vuic;
 vector signed long long vslla, vsllb;
 vector unsigned long long vulla, vullb, vullc;
+vector signed __int128 vi128a;
 vector unsigned __int128 vui128a, vui128b, vui128c;
 vector float vfa, vfb;
 vector double vda, vdb;
 unsigned int uia, uib;
 unsigned char uca;
 unsigned short usa;
 unsigned long long ulla;
 
vector float test_vec_vec_splati_ins_f(void) {
   // CHECK-BE: ret <4 x float>
   // CHECK:  [[T1:%.+]] = sub i32 1, %{{.+}}
   // CHECK: insertelement <4 x float> %{{.+}}, float %{{.+}}, i32 [[T1]]
   // CHECK:  [[T2:%.+]] = sub i32 3, %{{.+}}
   // CHECK: insertelement <4 x float> %{{.+}}, float %{{.+}}, i32 [[T2]]
   // CHECK: ret <4 x float>
   return vec_splati_ins(vfa, 0, 1.0f);
 }
+
+vector signed __int128 test_vec_slq_unsigned (void) {
+  // CHECK-LABEL: test_vec_slq_unsigned
+  // CHECK: call <1 x i128> @llvm.ppc.altivec.vslq(<1 x i128> %{{.+}}, <1 x i128> %{{.+}})
+  // CHECK: ret <1 x i128> %{{.+}}
+  return vec_slq(vui128a, vui128b);
+}
+
+vector signed __int128 test_vec_slq_signed (void) {
+  // CHECK-LABEL: test_vec_slq_signed
+  // CHECK: call <1 x i128> @llvm.ppc.altivec.vslq(<1 x i128> %{{.+}}, <1 x i128> %{{.+}})
+  // CHECK: ret <1 x i128>
+  return vec_slq(vui128a, vi128a);
+}
+
+vector signed __int128 test_vec_srq_unsigned (void) {
+  // CHECK-LABEL: test_vec_srq_unsigned
+  // CHECK: call <1 x i128> @llvm.ppc.altivec.vsrq(<1 x i128> %{{.+}}, <1 x i128> %{{.+}})
+  // CHECK: ret <1 x i128>
+  return vec_srq(vui128a, vui128b);
+}
+
+vector signed __int128 test_vec_srq_signed (void) {
+  // CHECK-LABEL: test_vec_srq_signed
+  // CHECK: call <1 x i128> @llvm.ppc.altivec.vsrq(<1 x i128> %{{.+}}, <1 x i128> %{{.+}})
+  // CHECK: ret <1 x i128>
+  return vec_srq(vui128a, vi128a);
+}
+
+vector signed __int128 test_vec_sraq_unsigned (void) {
+  // CHECK-LABEL: test_vec_sraq_unsigned
+  // CHECK: call <1 x i128> @llvm.ppc.altivec.vsraq(<1 x i128> %{{.+}}, <1 x i128> %{{.+}})
+  // CHECK: ret <1 x i128>
+  return vec_sraq(vui128a, vui128b);
+}
+
+vector signed __int128 test_vec_sraq_signed (void) {
+  // CHECK-LABEL: test_vec_sraq_signed
+  // CHECK: call <1 x i128> @llvm.ppc.altivec.vsraq(<1 x i128> %{{.+}}, <1 x i128> %{{.+}})
+  // CHECK: ret <1 x i128>
+  return vec_sraq(vui128a, vi128a);
+}

llvm/include/llvm/IR/IntrinsicsPowerPC.td

 def int_ppc_altivec_vsl   : PowerPC_Vec_WWW_Intrinsic<"vsl">;
 def int_ppc_altivec_vslo  : PowerPC_Vec_WWW_Intrinsic<"vslo">;
 
 def int_ppc_altivec_vslb  : PowerPC_Vec_BBB_Intrinsic<"vslb">;
 def int_ppc_altivec_vslv  : PowerPC_Vec_BBB_Intrinsic<"vslv">;
 def int_ppc_altivec_vsrv  : PowerPC_Vec_BBB_Intrinsic<"vsrv">;
 def int_ppc_altivec_vslh  : PowerPC_Vec_HHH_Intrinsic<"vslh">;
 def int_ppc_altivec_vslw  : PowerPC_Vec_WWW_Intrinsic<"vslw">;
+def int_ppc_altivec_vslq  : PowerPC_Vec_QQQ_Intrinsic<"vslq">;
 
 // Right Shifts.
 def int_ppc_altivec_vsr   : PowerPC_Vec_WWW_Intrinsic<"vsr">;
 def int_ppc_altivec_vsro  : PowerPC_Vec_WWW_Intrinsic<"vsro">;
 
 def int_ppc_altivec_vsrb  : PowerPC_Vec_BBB_Intrinsic<"vsrb">;
 def int_ppc_altivec_vsrh  : PowerPC_Vec_HHH_Intrinsic<"vsrh">;
 def int_ppc_altivec_vsrw  : PowerPC_Vec_WWW_Intrinsic<"vsrw">;
+def int_ppc_altivec_vsrq  : PowerPC_Vec_QQQ_Intrinsic<"vsrq">;
 def int_ppc_altivec_vsrab : PowerPC_Vec_BBB_Intrinsic<"vsrab">;
 def int_ppc_altivec_vsrah : PowerPC_Vec_HHH_Intrinsic<"vsrah">;
 def int_ppc_altivec_vsraw : PowerPC_Vec_WWW_Intrinsic<"vsraw">;
+def int_ppc_altivec_vsraq : PowerPC_Vec_QQQ_Intrinsic<"vsraq">;
 
 // Rotates.
 def int_ppc_altivec_vrlb  : PowerPC_Vec_BBB_Intrinsic<"vrlb">;
 def int_ppc_altivec_vrlh  : PowerPC_Vec_HHH_Intrinsic<"vrlh">;
 def int_ppc_altivec_vrlw  : PowerPC_Vec_WWW_Intrinsic<"vrlw">;
 def int_ppc_altivec_vrld  : PowerPC_Vec_DDD_Intrinsic<"vrld">;
 
 let TargetPrefix = "ppc" in {  // All PPC intrinsics start with "llvm.ppc.".

llvm/lib/Target/PowerPC/PPCInstrPrefix.td

def VCLRLB : VXForm_1<397, (outs vrrc:$vD), (ins vrrc:$vA, gprc:$rB),
                          "vclrlb $vD, $vA, $rB", IIC_VecGeneral,
                          [(set v16i8:$vD,
                                (int_ppc_altivec_vclrlb v16i8:$vA, i32:$rB))]>;
    def VCLRRB : VXForm_1<461, (outs vrrc:$vD), (ins vrrc:$vA, gprc:$rB),
                          "vclrrb $vD, $vA, $rB", IIC_VecGeneral,
                          [(set v16i8:$vD,
                                (int_ppc_altivec_vclrrb v16i8:$vA, i32:$rB))]>;
 
   // The XFormMemOp flag for the following 8 instructions is set on
   // the instruction format.
   let mayLoad = 1, mayStore = 0 in {
     def LXVRBX : X_XT6_RA5_RB5<31, 13, "lxvrbx", vsrc, []>;
     def LXVRHX : X_XT6_RA5_RB5<31, 45, "lxvrhx", vsrc, []>;
     def LXVRWX : X_XT6_RA5_RB5<31, 77, "lxvrwx", vsrc, []>;
     def LXVRDX : X_XT6_RA5_RB5<31, 109, "lxvrdx", vsrc, []>;
   }
 
let Predicates = [IsISA3_1] in {
   def VCMPSQ : VXForm_BF3_VAB5<321, (outs crrc:$BF), (ins vrrc:$vA, vrrc:$vB),
                                "vcmpsq $BF, $vA, $vB", IIC_VecGeneral, []>;
 
   def VRLQNM : VX1_VT5_VA5_VB5<325, "vrlqnm", []>;
   def VRLQMI : VXForm_1<69, (outs vrrc:$vD),
                         (ins vrrc:$vA, vrrc:$vB, vrrc:$vDi),
                         "vrlqmi $vD, $vA, $vB", IIC_VecFP, []>,
                         RegConstraint<"$vDi = $vD">, NoEncode<"$vDi">;
-  def VSLQ   : VX1_VT5_VA5_VB5<261, "vslq", []>;
-  def VSRAQ  : VX1_VT5_VA5_VB5<773, "vsraq", []>;
-  def VSRQ   : VX1_VT5_VA5_VB5<517, "vsrq", []>;
+  def VSLQ  : VX1_Int_Ty< 261, "vslq", int_ppc_altivec_vslq, v1i128>;
+  def VSRAQ : VX1_Int_Ty< 773, "vsraq", int_ppc_altivec_vsraq, v1i128>;
+  def VSRQ  : VX1_Int_Ty< 517, "vsrq" , int_ppc_altivec_vsrq, v1i128>;
   def VRLQ   : VX1_VT5_VA5_VB5<5, "vrlq", []>;
 
   def XSCVQPUQZ : X_VT5_XO5_VB5<63, 0, 836, "xscvqpuqz", []>;
   def XSCVQPSQZ : X_VT5_XO5_VB5<63, 8, 836, "xscvqpsqz", []>;
   def XSCVUQQP  : X_VT5_XO5_VB5<63, 3, 836, "xscvuqqp", []>;
   def XSCVSQQP  : X_VT5_XO5_VB5<63, 11, 836, "xscvsqqp", []>;
+
 }
 
 //---------------------------- Anonymous Patterns ----------------------------//
 let Predicates = [IsISA3_1] in {
   def : Pat<(v16i8 (int_ppc_vsx_xxgenpcvbm v16i8:$VRB, imm:$IMM)),
             (v16i8 (COPY_TO_REGCLASS (XXGENPCVBM $VRB, imm:$IMM), VRRC))>;
   def : Pat<(v8i16 (int_ppc_vsx_xxgenpcvhm v8i16:$VRB, imm:$IMM)),
             (v8i16 (COPY_TO_REGCLASS (XXGENPCVHM $VRB, imm:$IMM), VRRC))>;
   def : Pat<(v4i32 (int_ppc_vsx_xxgenpcvwm v4i32:$VRB, imm:$IMM)),
             (v4i32 (COPY_TO_REGCLASS (XXGENPCVWM $VRB, imm:$IMM), VRRC))>;
   def : Pat<(v2i64 (int_ppc_vsx_xxgenpcvdm v2i64:$VRB, imm:$IMM)),
             (v2i64 (COPY_TO_REGCLASS (XXGENPCVDM $VRB, imm:$IMM), VRRC))>;
 }
 
 let AddedComplexity = 400, Predicates = [PrefixInstrs] in {
  def : Pat<(v4i32 (build_vector i32immNonAllOneNonZero:$A,
                                 i32immNonAllOneNonZero:$A,
                                 i32immNonAllOneNonZero:$A,
                                 i32immNonAllOneNonZero:$A)),
            (v4i32 (XXSPLTIW imm:$A))>;
  def : Pat<(f32 nzFPImmAsi32:$A),
            (COPY_TO_REGCLASS (XXSPLTIDP (getFPAs32BitInt fpimm:$A)),
                              VSFRC)>;
  def : Pat<(f64 nzFPImmAsi32:$A),
            (COPY_TO_REGCLASS (XXSPLTIDP (getFPAs32BitInt fpimm:$A)),

llvm/test/CodeGen/PowerPC/p10-vector-shift.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -verify-machineinstrs -mtriple=powerpc64le-unknown-linux-gnu \
+; RUN:   -mcpu=pwr10 -ppc-asm-full-reg-names -ppc-vsr-nums-as-vr < %s | \
+; RUN:   FileCheck %s
+
+; RUN: llc -verify-machineinstrs -mtriple=powerpc64-unknown-linux-gnu \
+; RUN:   -mcpu=pwr10 -ppc-asm-full-reg-names -ppc-vsr-nums-as-vr < %s | \
+; RUN:   FileCheck %s
+
+@vui128a = dso_local global <1 x i128> zeroinitializer, align 16
+@vui128b = dso_local global <1 x i128> zeroinitializer, align 16
+
+define dso_local <1 x i128> @test_vec_slq(<1 x i128> %a, <1 x i128> %b) #0 {
+; CHECK-LABEL: test_vec_slq:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    vslq v2, v2, v3
+; CHECK-NEXT:    blr
+entry:
+  %slq = call <1 x i128> @llvm.ppc.altivec.vslq(<1 x i128> %a, <1 x i128> %b)
+  ret <1 x i128> %slq
+}
+
+define dso_local <1 x i128> @test_vec_vsrq(<1 x i128> %a, <1 x i128> %b) #0 {
+; CHECK-LABEL: test_vec_vsrq:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    vsrq v2, v2, v3
+; CHECK-NEXT:    blr
+entry:
+  %slq = call <1 x i128> @llvm.ppc.altivec.vsrq(<1 x i128> %a, <1 x i128> %b)
+  ret <1 x i128> %slq
+}
+
+define dso_local <1 x i128> @test_vec_vsraq(<1 x i128> %a, <1 x i128> %b) #0 {
+; CHECK-LABEL: test_vec_vsraq:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    vsraq v2, v2, v3
+; CHECK-NEXT:    blr
+entry:
+  %slq = call <1 x i128> @llvm.ppc.altivec.vsraq(<1 x i128> %a, <1 x i128> %b)
+  ret <1 x i128> %slq
+}
+
+; Function Attrs: nounwind readnone
+declare <1 x i128> @llvm.ppc.altivec.vslq(<1 x i128>, <1 x i128>) #1
+
+; Function Attrs: nounwind readnone
+declare <1 x i128> @llvm.ppc.altivec.vsrq(<1 x i128>, <1 x i128>) #1
+
+; Function Attrs: nounwind readnone
+declare <1 x i128> @llvm.ppc.altivec.vsraq(<1 x i128>, <1 x i128>) #1