[PowerPC] Implement Load VSX Vector and Sign Extend and Zero Extend

[Conanap]

Summary

Implemented Builtins for Load VSX Vector and sign extend and zero extend along with appropriate test cases. The new instructions are used for ZEXT but SEXT will continue to use the previously generated ASM.

[amyk]

I have a few comments from the last time we looked at this together.
Just also FYI that the backend tests will be apart of the p10-vsx-builtins.ll file introduced in https://reviews.llvm.org/D82431.

clang/lib/Headers/altivec.h

16514

Nit: space after this comment.

16517

I realized that I think we are supposed to use unaligned_vec_si128 and unaligned_vec_ui128 instead of vector signed __int128 and vector unsigned __int128 when we return (for here, and other places).

16535

Nit: space after this comment.

16917

Remove unrelated whitespace change.

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

14136

Move this comment above the combine function since this comment was intended to describe the function.

14143

Can omit the { } since we just have a single return underneath.

14150

Can omit the { } since we just have a single return underneath.

llvm/lib/Target/PowerPC/PPCInstrPrefix.td

976

Can remove this comment.

llvm/test/CodeGen/PowerPC/p10-vsx-builtins.ll

6 ↗

(On Diff #273159)

Please update this comment to be more descriptive. Maybe something like,

These test cases tests that zero extending loads utilize the Load VSX Vector Rightmost (lxvr[b|h|w|d]) instructions in Power10.

8 ↗

(On Diff #273159)

Please remove the comments/checks above the functions.

[Conanap]

Addressed Amy's comments regarding documentation of the changes.

[Conanap]

Fixed return signature for the open coded functions in altivec.h

[anil9]

clang/test/CodeGen/builtins-ppc-p10vector.c

17

nit: It seems that most pull requests follow an ordering like first signed declaration and then unsigned, declaration, this one follows too , except the above two lines. And should the above declarations of chars, be along with these lines ?

llvm/test/CodeGen/PowerPC/p10-vsx-builtins.ll

56 ↗

(On Diff #273421)

I am not too familiar with the builtins but I never saw a check outside of the two braces in the test cases before, is it not posible to include it inside the test cases ?

[Conanap]

Reordered some declarations in test cases and removed unecessary extra CHECKs.

[lei]

Please move encoding tests to ppc64-encoding-ISA31.[txt|s].

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

14161

nit: check ValidLDType first:

if (! ValidLDType || (LD->getValueType(0) != MVT::i128) ||
    (LD->getExtensionType() != ISD::ZEXTLOAD) ||)

llvm/lib/Target/PowerPC/PPCISelLowering.h

547

nit: This shouldn't matter to code gen, but it would be nice to have this defined closer to other similar instructions. Maybe after LXVD2X?

llvm/lib/Target/PowerPC/PPCInstrPrefix.td

453

nit: indentation

[Conanap]

Addressed Anil's comments with regards to the test cases

[Conanap]

llvm/test/CodeGen/PowerPC/p10-vsx-builtins.ll

56 ↗

(On Diff #273421)

these were extra checks that I should've removed; thanks for catching it

[Conanap]

Fixed some formatting stuff.

[Conanap]

Addressed Lei's comments

[lei]

llvm/lib/Target/PowerPC/PPCISelLowering.h

500

missing ,?

llvm/lib/Target/PowerPC/PPCInstrPrefix.td

448

We should put all such def at the top of this file under:

// PowerPC specific type constraints.

452

Put these under the type constraints section and doc with:

// PowerPC specific DAG Nodes.

llvm/test/CodeGen/PowerPC/p10-vsx-builtins.ll

3 ↗

(On Diff #275124)

remve extra line. Makes the comment harder to read.

10 ↗

(On Diff #275124)

You don't need the local_unnamed_addr #0

[Conanap]

Fixed a missing comma; ensured it builds.

[Conanap]

fixed a missing comma and ensure that it compiles

[Conanap]

Moved some code to the top of the file as per Lei's request

[Conanap]

Addressed Lei's comments

[lei]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

14160

nit: don't need () aroud !ValidLDType

14166

I don't think we need to explicitly say this sine everything we do here is for pattern matching to appropriate instructions in the backend...

14167

Can just merge this into the next line and remove this tmp value.

llvm/test/CodeGen/PowerPC/p10-vsx-builtins.ll

6 ↗

(On Diff #275423)

Please add tests for BE.

[Conanap]

Now depends on D83364.

[Conanap]

Also removed unnecessary brackets and comments.

[amyk]

Please update this patch to remove the instruction defs and MC tests. Also, you can update the patch to put your backend llc tests in the file I've introduced in: https://reviews.llvm.org/D82467

clang/test/CodeGen/builtins-ppc-p10vector.c

14

unintended change?

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

14165

You did not assign anything here?

[Conanap]

Removed unecessary comments and unintended changes.

[Conanap]

Removed duplicate test code found in the instructions and MC Test implementation of VSX Vector store and load with sign extend or zero extend; re-added a test file that was omitted in the last diff update.

[amyk]

I think the patterns for the load instructions may have accidentally been deleted. Please add them back to the patch. Thank you.

[Conanap]

Restored accidentally deleted pattern, removed duplicate tests, moved new tests to another pre-existing file instead.

[stefanp]

The title of the patch mentions both zero extend and sign extend.
However, it seems that we only have instructions for the zero extend case. Is that right?
I see both types of tests in:
test/CodeGen/builtins-ppc-p10vector.c
But I only see codegen tests for the zreo extend version.
test/CodeGen/PowerPC/builtins-ppc-p10vsx.ll

I bring it up because I want to make sure that this is intentional.

clang/lib/Headers/altivec.h

16516

I'm a little confused about this.
Your function signature says we return vector unsigned __int128 but the return statement casts to unaligned_vec_si128. Is that how this is supposed to look?

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

14161

It looks like you are doing this for zero extend only.
What about ISD::EXTLOAD? In that case the upper bits are undefined anyway so we can just assume a zero extend right?

llvm/test/CodeGen/PowerPC/builtins-ppc-p10vsx.ll

41 ↗

(On Diff #277952)

nit:
Was this line auto-added?
Usually this comment is added at the top of the file by the script. I don't think it is required here.

[NeHuang]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

14154

nit: if we are loading either a byte....

llvm/lib/Target/PowerPC/PPCInstrPrefix.td

29

nit: one space is enough.

[Conanap]

Return signature fix, added recognition for ISD:EXTLOAD, some code clean up.

[lei]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

14164

nit: indentation.

14167

nit: indentation

[Conanap]

Addressed formatting comments

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

14154

I'm not too sure what you mean, would you be able to elaborate?
The comment is:

This transformation is only valid if the we are loading either a byte,
halfword, word, or doubleword.

Thanks!

[lei]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

14154

Extra the in your sentence:
This transformation is only valid if the we are loading -> This transformation is only valid if we are loading

[Conanap]

Included sext test case.

[amyk]

Could we also elaborate in the description on how we are utilizing the new load instructions for zero extend case but not the sign extend case?

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

14164

Over 80 characters?

llvm/test/CodeGen/PowerPC/builtins-ppc-p10vsx.ll

5 ↗

(On Diff #281035)

This line actually is for -O0 and not a BE RUN line. I believe the most up to date version of the test includes the LE, BE and O0 line.

[Conanap]

Some updates on formatting and updated to match an updated test file.

[NeHuang]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

14168

please clang-format the changed files before updating the patch.

llvm/test/CodeGen/PowerPC/builtins-ppc-p10vsx.ll

4 ↗

(On Diff #281974)

Why do we need different check prefixes for LE and BE? They produce the same asm. It seems we only need CHECK-O0.

[Conanap]

Clang formatted relevant lines, combined LE and BE tests as they produced the same ASM.

[amyk]

clang/test/CodeGen/builtins-ppc-p10vector.c

590

It would be good to be a little more specific with the CHECK lines.

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

14236

This line here and below looks like unintended changes from clang-format I am guessing? I am OK with them being removed during commit.

[amyk]

I realized I didn't put a comment on this earlier but this overall LGTM, but I think it would be good to see if @nemanjai has any additional comments on this patch.

[nemanjai]

LGTM aside from a couple of minor nits.

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

14141

This line is redundant. Please remove it.

14160

The second condition was already checked on line 13604. You can omit it here.

clang/lib/Headers/altivec.h

 vec_xl_be(signed long long  __offset, unsigned __int128 *__ptr) {
   return vec_xl(__offset, __ptr);
 }
 #endif
 #else
   #define vec_xl_be vec_xl
 #endif
 
+#if defined(__POWER10_VECTOR__) && defined(__VSX__)
+
+/* vect_xl_sext */
+
+static __inline__ vector unsigned __int128 __ATTRS_o_ai
+vec_xl_sext(signed long long __offset, signed char *__pointer) {
+  return (unaligned_vec_si128)*(__pointer + __offset);
+}
+
+static __inline__ vector unsigned __int128 __ATTRS_o_ai
+vec_xl_sext(signed long long __offset, signed short *__pointer) {
+  return (unaligned_vec_si128)*(__pointer + __offset);
+}
+
+static __inline__ vector unsigned __int128 __ATTRS_o_ai
+vec_xl_sext(signed long long __offset, signed int *__pointer) {
+  return (unaligned_vec_si128)*(__pointer + __offset);
+}
+
+static __inline__ vector unsigned __int128 __ATTRS_o_ai
+vec_xl_sext(signed long long __offset, signed long long *__pointer) {
+  return (unaligned_vec_si128)*(__pointer + __offset);
+}
+
+/* vec_xl_zext */
+
+static __inline__ vector unsigned __int128 __ATTRS_o_ai
+vec_xl_zext(signed long long __offset, unsigned char *__pointer) {
+  return (unaligned_vec_ui128)*(__pointer + __offset);
+}
+
+static __inline__ vector unsigned __int128 __ATTRS_o_ai
+vec_xl_zext(signed long long __offset, unsigned short *__pointer) {
+  return (unaligned_vec_ui128)*(__pointer + __offset);
+}
+
+static __inline__ vector unsigned __int128 __ATTRS_o_ai
+vec_xl_zext(signed long long __offset, unsigned int *__pointer) {
+  return (unaligned_vec_ui128)*(__pointer + __offset);
+}
+
+static __inline__ vector unsigned __int128 __ATTRS_o_ai
+vec_xl_zext(signed long long __offset, unsigned long long *__pointer) {
+  return (unaligned_vec_ui128)*(__pointer + __offset);
+}
+
+#endif
+
 /* vec_xst */
 
 static inline __ATTRS_o_ai void vec_xst(vector signed char __vec,
                                         signed long long __offset,
                                         signed char *__ptr) {
   *(unaligned_vec_schar *)(__ptr + __offset) = __vec;
 }
 

 static __inline__ vector unsigned char __ATTRS_o_ai
 vec_clrr(vector unsigned char __a, unsigned int __n) {
 #ifdef __LITTLE_ENDIAN__
   return __builtin_altivec_vclrlb((vector signed char)__a, __n);
 #else
   return __builtin_altivec_vclrrb((vector signed char)__a, __n);
 #endif
 }
 
 /* vec_cntlzm */
 
 static __inline__ vector unsigned long long __ATTRS_o_ai
 vec_cntlzm(vector unsigned long long __a, vector unsigned long long __b) {
   return __builtin_altivec_vclzdm(__a, __b);
 }
 
 /* vec_cnttzm */

clang/test/CodeGen/builtins-ppc-p10vector.c

 // REQUIRES: powerpc-registered-target
 // RUN: %clang_cc1 -target-feature +vsx -target-feature +altivec \
 // RUN:   -target-cpu pwr10 -triple powerpc64le-unknown-unknown -emit-llvm %s \
 // RUN:   -o - | FileCheck %s
 
 // RUN: %clang_cc1 -target-feature +vsx -target-feature +altivec \
 // RUN:   -target-cpu pwr10 -triple powerpc64-unknown-unknown -emit-llvm %s \
 // RUN:   -o - | FileCheck %s -check-prefix=CHECK-BE
 
 // RUN: %clang_cc1 -target-feature +vsx -target-feature +altivec \
 // RUN:   -target-cpu pwr10 -triple powerpc64le-unknown-unknown -emit-llvm %s \
 // RUN:   -o - | FileCheck %s -check-prefix=CHECK-LE
 
-#include <altivec.h>
+#include "altivec.h"
 
 vector signed char vsca, vscb;
 vector unsigned char vuca, vucb, vucc;
 vector signed short vssa, vssb;
 vector unsigned short vusa, vusb, vusc;
 vector signed int vsia, vsib;
 vector unsigned int vuia, vuib, vuic;
 vector signed long long vslla, vsllb;
 vector unsigned long long vulla, vullb, vullc;
 vector unsigned __int128 vui128a, vui128b, vui128c;
 vector float vfa, vfb;
 vector double vda, vdb;
-unsigned int uia, uib;
-unsigned char uca;
-unsigned short usa;
-unsigned long long ulla;
+signed int *iap;
+unsigned int uia, uib, *uiap;
+signed char *cap;
+unsigned char uca, *ucap;
+signed short *sap;
+unsigned short usa, *usap;
+signed long long *llap, llb;
+unsigned long long ulla, *ullap;
 
 vector unsigned long long test_vpdepd(void) {
   // CHECK: @llvm.ppc.altivec.vpdepd(<2 x i64>
   // CHECK-NEXT: ret <2 x i64>
   return vec_pdep(vulla, vullb);
 }
 
 vector unsigned long long test_vpextd(void) {
vector float test_vec_vec_splati_ins_f(void) {
   // CHECK-BE: ret <4 x float>
   // CHECK:  [[T1:%.+]] = sub i32 1, %{{.+}}
   // CHECK: insertelement <4 x float> %{{.+}}, float %{{.+}}, i32 [[T1]]
   // CHECK:  [[T2:%.+]] = sub i32 3, %{{.+}}
   // CHECK: insertelement <4 x float> %{{.+}}, float %{{.+}}, i32 [[T2]]
   // CHECK: ret <4 x float>
   return vec_splati_ins(vfa, 0, 1.0f);
 }
+
+vector signed __int128 test_vec_xl_sext_i8(void) {
+  // CHECK: load i8
+  // CHECK: sext i8
+  // CHECK: ret <1 x i128>
+  return vec_xl_sext(llb, cap);
+}
+
+vector signed __int128 test_vec_xl_sext_i16(void) {
+  // CHECK: load i16
+  // CHECK: sext i16
+  // CHECK: ret <1 x i128>
+  return vec_xl_sext(llb, sap);
+}
+
+vector signed __int128 test_vec_xl_sext_i32(void) {
+  // CHECK: load i32
+  // CHECK: sext i32
+  // CHECK: ret <1 x i128>
+  return vec_xl_sext(llb, iap);
+}
+
+vector signed __int128 test_vec_xl_sext_i64(void) {
+  // CHECK: load i64
+  // CHECK: sext i64
+  // CHECK: ret <1 x i128>
+  return vec_xl_sext(llb, llap);
+}
+
+vector unsigned __int128 test_vec_xl_zext_i8(void) {
+  // CHECK: load i8
+  // CHECK: zext i8
+  // CHECK: ret <1 x i128>
+  return vec_xl_zext(llb, ucap);
+}
+
+vector unsigned __int128 test_vec_xl_zext_i16(void) {
+  // CHECK: load i16
+  // CHECK: zext i16
+  // CHECK: ret <1 x i128>
+  return vec_xl_zext(llb, usap);
+}
+
+vector unsigned __int128 test_vec_xl_zext_i32(void) {
+  // CHECK: load i32
+  // CHECK: zext i32
+  // CHECK: ret <1 x i128>
+  return vec_xl_zext(llb, uiap);
+}
+
+vector unsigned __int128 test_vec_xl_zext_i64(void) {
+  // CHECK: load i64
+  // CHECK: zext i64
+  // CHECK: ret <1 x i128>
+  return vec_xl_zext(llb, ullap);
+}

llvm/lib/Target/PowerPC/PPCISelLowering.h

enum NodeType : unsigned {
     /// followed by a byte-width for the store.
     STXSIX,
 
     /// VSRC, CHAIN = LXVD2X_LE CHAIN, Ptr - Occurs only for little endian.
     /// Maps directly to an lxvd2x instruction that will be followed by
     /// an xxswapd.
     LXVD2X,
 
+    /// LXVRZX - Load VSX Vector Rightmost and Zero Extend
+    /// This node represents v1i128 BUILD_VECTOR of a zero extending load
+    /// instruction from <byte, halfword, word, or doubleword> to i128.
+    /// Allows utilization of the Load VSX Vector Rightmost Instructions.
+    LXVRZX,
+
     /// VSRC, CHAIN = LOAD_VEC_BE CHAIN, Ptr - Occurs only for little endian.
     /// Maps directly to one of lxvd2x/lxvw4x/lxvh8x/lxvb16x depending on
     /// the vector type to load vector in big-endian element order.
     LOAD_VEC_BE,
 
     /// VSRC, CHAIN = LD_VSX_LH CHAIN, Ptr - This is a floating-point load of a
     /// v2f32 value into the lower half of a VSR register.
     LD_VSX_LH,
enum NodeType : unsigned {
     /// Loads the entry for GA from the TOC, where the TOC base is given by
     /// the last operand.
     TOC_ENTRY
   };
 
   } // end namespace PPCISD
 
   /// Define some predicates that are used for node matching.
   namespace PPC {
 
     /// isVPKUHUMShuffleMask - Return true if this is the shuffle mask for a
     /// VPKUHUM instruction.
     bool isVPKUHUMShuffleMask(ShuffleVectorSDNode *N, unsigned ShuffleKind,
                               SelectionDAG &DAG);
 
     /// isVPKUWUMShuffleMask - Return true if this is the shuffle mask for a
     /// VPKUWUM instruction.

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

const char *PPCTargetLowering::getTargetNodeName(unsigned Opcode) const {
   case PPCISD::BUILD_SPE64:     return "PPCISD::BUILD_SPE64";
   case PPCISD::EXTRACT_SPE:     return "PPCISD::EXTRACT_SPE";
   case PPCISD::EXTSWSLI:        return "PPCISD::EXTSWSLI";
   case PPCISD::LD_VSX_LH:       return "PPCISD::LD_VSX_LH";
   case PPCISD::FP_EXTEND_HALF:  return "PPCISD::FP_EXTEND_HALF";
   case PPCISD::MAT_PCREL_ADDR:  return "PPCISD::MAT_PCREL_ADDR";
   case PPCISD::LD_SPLAT:        return "PPCISD::LD_SPLAT";
   case PPCISD::FNMSUB:          return "PPCISD::FNMSUB";
+  case PPCISD::LXVRZX:          return "PPCISD::LXVRZX";
   }
   return nullptr;
 }
 
 EVT PPCTargetLowering::getSetCCResultType(const DataLayout &DL, LLVMContext &C,
                                           EVT VT) const {
   if (!VT.isVector())
     return Subtarget.useCRBits() ? MVT::i1 : MVT::i32;
static SDValue combineBVOfVecSExt(SDNode *N, SelectionDAG &DAG) {
   if (Elems != CorrectElems) {
     return addShuffleForVecExtend(N, DAG, Input, Elems, CorrectElems);
   }
 
   // Regular lowering will catch cases where a shuffle is not needed.
   return SDValue();
 }
 
+// Look for the pattern of a load from a narrow width to i128, feeding
+// into a BUILD_VECTOR of v1i128. Replace this sequence with a PPCISD node
+// (LXVRZX). This node represents a zero extending load that will be matched
+// to the Load VSX Vector Rightmost instructions.
+static SDValue combineBVZEXTLOAD(SDNode *N, SelectionDAG &DAG) {
+  SDLoc dl(N);
+  
+  // This combine is only eligible for a BUILD_VECTOR of v1i128.
+  // Other return types are not valid for the LXVRZX replacement.
+  if (N->getValueType(0) != MVT::v1i128)
+    return SDValue();
+
+  SDValue Operand = N->getOperand(0);
+  // Proceed with the transformation if the operand to the BUILD_VECTOR
+  // is a load instruction.
+  if (Operand.getOpcode() != ISD::LOAD)
+    return SDValue();
+
+  LoadSDNode *LD = dyn_cast<LoadSDNode>(Operand);
+  EVT MemoryType = LD->getMemoryVT();
+
+  // This transformation is only valid if the we are loading either a byte,
+  // halfword, word, or doubleword.
+  bool ValidLDType = MemoryType == MVT::i8 || MemoryType == MVT::i16 ||
+                     MemoryType == MVT::i32 || MemoryType == MVT::i64;
+
+  // Ensure that the load from the narrow width is being zero extended to i128.
+  if (!ValidLDType || (LD->getValueType(0) != MVT::i128) ||
+      (LD->getExtensionType() != ISD::ZEXTLOAD))
+    return SDValue();
+
+  // The width of the narrow type becomes an operand of the LXVRZX node
+  SDValue Width = ;
+  SDValue LoadOps[] = {LD->getChain(), LD->getBasePtr(), DAG.getIntPtrConstant(MemoryType.getScalarSizeInBits(), dl)};
+
+  return DAG.getMemIntrinsicNode(PPCISD::LXVRZX, dl,
+                                 DAG.getVTList(MVT::v1i128, MVT::Other),
+                                 LoadOps, MemoryType, LD->getMemOperand());
+}
+
 SDValue PPCTargetLowering::DAGCombineBuildVector(SDNode *N,
                                                  DAGCombinerInfo &DCI) const {
   assert(N->getOpcode() == ISD::BUILD_VECTOR &&
          "Should be called with a BUILD_VECTOR node");
 
   SelectionDAG &DAG = DCI.DAG;
   SDLoc dl(N);
 
SDValue PPCTargetLowering::DAGCombineBuildVector(SDNode *N,
   // input types (i.e. it can't handle things like v4i16) so do not run before
   // legalization.
   if (Subtarget.hasP9Altivec() && !DCI.isBeforeLegalize()) {
     Reduced = combineBVOfVecSExt(N, DAG);
     if (Reduced)
       return Reduced;
   }
 
+  // On Power10, the Load VSX Vector Rightmost instructions can be utilized
+  // if this is a BUILD_VECTOR of v1i128, and if the operand to the BUILD_VECTOR
+  // is a load from <valid narrow width> to i128.
+  if (Subtarget.isISA3_1()) {
+    SDValue BVOfZLoad = combineBVZEXTLOAD(N, DAG);
+    if (BVOfZLoad)
+      return BVOfZLoad;
+  }
 
   if (N->getValueType(0) != MVT::v2f64)
     return SDValue();
 
   // Looking for:
   // (build_vector ([su]int_to_fp (extractelt 0)), [su]int_to_fp (extractelt 1))
   if (FirstInput.getOpcode() != ISD::SINT_TO_FP &&
       FirstInput.getOpcode() != ISD::UINT_TO_FP)
     return SDValue();
   if (N->getOperand(1).getOpcode() != ISD::SINT_TO_FP &&
       N->getOperand(1).getOpcode() != ISD::UINT_TO_FP)
     return SDValue();
   if (FirstInput.getOpcode() != N->getOperand(1).getOpcode())
     return SDValue();
 
   SDValue Ext1 = FirstInput.getOperand(0);
   SDValue Ext2 = N->getOperand(1).getOperand(0);
   if(Ext1.getOpcode() != ISD::EXTRACT_VECTOR_ELT ||
      Ext2.getOpcode() != ISD::EXTRACT_VECTOR_ELT)
     return SDValue();
 
   ConstantSDNode *Ext1Op = dyn_cast<ConstantSDNode>(Ext1.getOperand(1));
   ConstantSDNode *Ext2Op = dyn_cast<ConstantSDNode>(Ext2.getOperand(1));
   if (!Ext1Op || !Ext2Op)
     return SDValue();
   if (Ext1.getOperand(0).getValueType() != MVT::v4i32 ||
       Ext1.getOperand(0) != Ext2.getOperand(0))

llvm/lib/Target/PowerPC/PPCInstrPrefix.td

 def PPCxxsplti32dx : SDNode<"PPCISD::XXSPLTI32DX", SDT_PPCSplat32, []>;
 
 //===----------------------------------------------------------------------===//
 
 // PC Relative flag (for instructions that use the address of the prefix for
 // address computations).
 class isPCRel { bit PCRel = 1; }
 
+// PowerPC specific type constraints.
+def SDT_PPCLXVRZX : SDTypeProfile<1, 2, [
+  SDTCisVT<0, v1i128>, SDTCisPtrTy<1>, SDTCisPtrTy<2>
+]>;
+
+// PPC Specific DAG Nodes.
+def PPClxvrzx : SDNode<"PPCISD::LXVRZX", SDT_PPCLXVRZX,
+                       [SDNPHasChain, SDNPMayLoad]>;
+
 // Top-level class for prefixed instructions.
 class PI<bits<6> pref, bits<6> opcode, dag OOL, dag IOL, string asmstr,
          InstrItinClass itin> : Instruction {
   field bits<64> Inst;
   field bits<64> SoftFail = 0;
   bit PCRel = 0; // Default value, set by isPCRel.
   let Size = 8;
 

 let Predicates = [PrefixInstrs] in {
   let Interpretation64Bit = 1, isCodeGenOnly = 1 in {
     defm PADDI8 :
       MLS_DForm_R_SI34_RTA5_p<14, (outs g8rc:$RT), (ins g8rc:$RA, s34imm:$SI),
                               (ins immZero:$RA, s34imm:$SI),
                               "paddi $RT, $RA, $SI", IIC_LdStLFD>;
     let isReMaterializable = 1, isAsCheapAsAMove = 1, isMoveImm = 1 in {
       def PLI8 : MLS_DForm_SI34_RT5<14, (outs g8rc:$RT),
                                     (ins s34imm:$SI),
                                     "pli $RT, $SI", IIC_IntSimple, []>;
     }
   }
   defm PADDI :
     MLS_DForm_R_SI34_RTA5_p<14, (outs gprc:$RT), (ins gprc:$RA, s34imm:$SI),
                             (ins immZero:$RA, s34imm:$SI),
                             "paddi $RT, $RA, $SI", IIC_LdStLFD>;
   let isReMaterializable = 1, isAsCheapAsAMove = 1, isMoveImm = 1 in {
     def PLI : MLS_DForm_SI34_RT5<14, (outs gprc:$RT),
                                  (ins s34imm:$SI),
                                  "pli $RT, $SI", IIC_IntSimple, []>;
   }
 
def : Pat<(v16i8 (int_ppc_vsx_xxgenpcvbm v16i8:$VRB, imm:$IMM)),
             (v16i8 (COPY_TO_REGCLASS (XXGENPCVBM $VRB, imm:$IMM), VRRC))>;
   def : Pat<(v8i16 (int_ppc_vsx_xxgenpcvhm v8i16:$VRB, imm:$IMM)),
             (v8i16 (COPY_TO_REGCLASS (XXGENPCVHM $VRB, imm:$IMM), VRRC))>;
   def : Pat<(v4i32 (int_ppc_vsx_xxgenpcvwm v4i32:$VRB, imm:$IMM)),
             (v4i32 (COPY_TO_REGCLASS (XXGENPCVWM $VRB, imm:$IMM), VRRC))>;
   def : Pat<(v2i64 (int_ppc_vsx_xxgenpcvdm v2i64:$VRB, imm:$IMM)),
             (v2i64 (COPY_TO_REGCLASS (XXGENPCVDM $VRB, imm:$IMM), VRRC))>;
 }
 
 let AddedComplexity = 400, Predicates = [PrefixInstrs] in {
  def : Pat<(v4i32 (build_vector i32immNonAllOneNonZero:$A,
                                 i32immNonAllOneNonZero:$A,
                                 i32immNonAllOneNonZero:$A,
                                 i32immNonAllOneNonZero:$A)),
            (v4i32 (XXSPLTIW imm:$A))>;
  def : Pat<(f32 nzFPImmAsi32:$A),
            (COPY_TO_REGCLASS (XXSPLTIDP (getFPAs32BitInt fpimm:$A)),

llvm/test/CodeGen/PowerPC/ISA31-vsx-builtins.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; These test cases tests that zero extending loads utilize the Load VSX Vector Rightmost
+; (lxvr[b|h|w|d]x) instructions in Power10.
+; RUN: llc -verify-machineinstrs -mtriple=powerpc64le-unknown-linux-gnu \
+; RUN:   -mcpu=pwr10 -ppc-asm-full-reg-names -ppc-vsr-nums-as-vr < %s | \
+; RUN:   FileCheck %s
+; RUN: llc -verify-machineinstrs -mtriple=powerpc64-unknown-linux-gnu \
+; RUN:   -mcpu=pwr10 -ppc-asm-full-reg-names -ppc-vsr-nums-as-vr < %s | \
+; RUN:   FileCheck %s
+
+define dso_local <1 x i128> @vec_xl_zext(i64 %__offset, i8* nocapture readonly %__pointer) {
+; CHECK-LABEL: vec_xl_zext:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    lxvrbx v2, r4, r3
+; CHECK-NEXT:    blr
+entry:
+  %add.ptr = getelementptr inbounds i8, i8* %__pointer, i64 %__offset
+  %0 = load i8, i8* %add.ptr, align 1
+  %conv = zext i8 %0 to i128
+  %splat.splatinsert = insertelement <1 x i128> undef, i128 %conv, i32 0
+  ret <1 x i128> %splat.splatinsert
+}
+
+define dso_local <1 x i128> @vec_xl_zext_short(i64 %__offset, i16* nocapture readonly %__pointer) {
+; CHECK-LABEL: vec_xl_zext_short:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    sldi r3, r3, 1
+; CHECK-NEXT:    lxvrhx v2, r4, r3
+; CHECK-NEXT:    blr
+entry:
+  %add.ptr = getelementptr inbounds i16, i16* %__pointer, i64 %__offset
+  %0 = load i16, i16* %add.ptr, align 2
+  %conv = zext i16 %0 to i128
+  %splat.splatinsert = insertelement <1 x i128> undef, i128 %conv, i32 0
+  ret <1 x i128> %splat.splatinsert
+}
+
+define dso_local <1 x i128> @vec_xl_zext_word(i64 %__offset, i32* nocapture readonly %__pointer) {
+; CHECK-LABEL: vec_xl_zext_word:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    sldi r3, r3, 2
+; CHECK-NEXT:    lxvrwx v2, r4, r3
+; CHECK-NEXT:    blr
+entry:
+  %add.ptr = getelementptr inbounds i32, i32* %__pointer, i64 %__offset
+  %0 = load i32, i32* %add.ptr, align 4
+  %conv = zext i32 %0 to i128
+  %splat.splatinsert = insertelement <1 x i128> undef, i128 %conv, i32 0
+  ret <1 x i128> %splat.splatinsert
+}
+
+define dso_local <1 x i128> @vec_xl_zext_dw(i64 %__offset, i64* nocapture readonly %__pointer) {
+; CHECK-LABEL: vec_xl_zext_dw:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    sldi r3, r3, 3
+; CHECK-NEXT:    lxvrdx v2, r4, r3
+; CHECK-NEXT:    blr
+entry:
+  %add.ptr = getelementptr inbounds i64, i64* %__pointer, i64 %__offset
+  %0 = load i64, i64* %add.ptr, align 8
+  %conv = zext i64 %0 to i128
+  %splat.splatinsert = insertelement <1 x i128> undef, i128 %conv, i32 0
+  ret <1 x i128> %splat.splatinsert
+}
+

llvm/test/MC/Disassembler/PowerPC/ppc64-encoding-ISA31.txt

 # CHECK: stxvrhx 33, 3, 1
 0x7c 0x23 0x09 0x5b
 
 # CHECK: stxvrwx 34, 3, 1
 0x7c 0x43 0x09 0x9b
 
 # CHECK: stxvrdx 35, 3, 1
 0x7c 0x63 0x09 0xdb
+
+# CHECK: lxvrbx 32, 1, 2
+0x7c 0x01 0x10 0x1b
+
+# CHECK: lxvrhx 33, 1, 2
+0x7c 0x21 0x10 0x5b
+
+# CHECK: lxvrdx 34, 1, 2
+0x7c 0x41 0x10 0xdb
+
+# CHECK: lxvrwx 35, 1, 2
+0x7c 0x61 0x10 0x9b

llvm/test/MC/PowerPC/ppc64-encoding-ISA31.s

 # CHECK-LE: stxvrhx 33, 3, 1                      # encoding: [0x5b,0x09,0x23,0x7c]
             stxvrhx 33, 3, 1
 # CHECK-BE: stxvrwx 34, 3, 1                      # encoding: [0x7c,0x43,0x09,0x9b]
 # CHECK-LE: stxvrwx 34, 3, 1                      # encoding: [0x9b,0x09,0x43,0x7c]
             stxvrwx 34, 3, 1
 # CHECK-BE: stxvrdx 35, 3, 1                      # encoding: [0x7c,0x63,0x09,0xdb]
 # CHECK-LE: stxvrdx 35, 3, 1                      # encoding: [0xdb,0x09,0x63,0x7c]
             stxvrdx 35, 3, 1
+# CHECK-BE: lxvrbx 32, 1, 2                       # encoding: [0x7c,0x01,0x10,0x1b]
+# CHECK-LE: lxvrbx 32, 1, 2                       # encoding: [0x1b,0x10,0x01,0x7c]
+            lxvrbx 32, 1, 2
+# CHECK-BE: lxvrhx 33, 1, 2                       # encoding: [0x7c,0x21,0x10,0x5b]
+# CHECK-LE: lxvrhx 33, 1, 2                       # encoding: [0x5b,0x10,0x21,0x7c]
+            lxvrhx 33, 1, 2
+# CHECK-BE: lxvrdx 34, 1, 2                       # encoding: [0x7c,0x41,0x10,0xdb]
+# CHECK-LE: lxvrdx 34, 1, 2                       # encoding: [0xdb,0x10,0x41,0x7c]
+            lxvrdx 34, 1, 2
+# CHECK-BE: lxvrwx 35, 1, 2                       # encoding: [0x7c,0x61,0x10,0x9b]
+# CHECK-LE: lxvrwx 35, 1, 2                       # encoding: [0x9b,0x10,0x61,0x7c]
+            lxvrwx 35, 1, 2