[PowerPC] Exploit xxsplti32dx (constant materialization) for scalars

[Conanap]

Summary

Previously related differential (exploit xxsplti32dx for vectors) here: https://reviews.llvm.org/D90173

This patch exploits the xxsplti32dx instruction available on Power10 in place of constant pool loads where xxspltidp would not be able to, usually because the immediate cannot fit into 32 bits.

[Conanap]

Updated to ensure the shortcircuit protects against the destructive function.

[stefanp]

llvm/lib/Target/PowerPC/PPCISelLowering.h

1324

Is the APInt version of this function used anywere?

llvm/lib/Target/PowerPC/PPCInstrInfo.td

412

Why are we running this here?
We don't check the return of the function so we must assume that it returns true.
In that case the value of APFloatOfN won't change because convertToNonDenormSingle will only change the value of the parameter if it returns true. But checkNonDenormCannotConvertToSingle only returns true if convertToNonDenormSingle return false.

llvm/test/CodeGen/PowerPC/constant-pool.ll

48

I'm looking to understand this test case.
We are trying to materialize a special PowerPC long double (double-double). It seems that we have materialized one half of it and not the other half.

Is it because the first half is a denormal?
Why are we avoiding denormals anyway? It seems like we can completely specify a 64 bit double with two xxsplti32dx instructions.

369

What is going on here?
It almost looks like we are spilling vs3 half way through materializing a constant.

[amyk]

In addition to the nit comments, I also have the same question as Stefan for getFPAs64BitIntHi/getFPAs64BitIntLo.

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

8591

nit: end with period.

llvm/lib/Target/PowerPC/PPCInstrPrefix.td

2612

2614

Have this : $A on the line above? Same as the one below.
Also a minor nit, but add a space to separate the (.

[Conanap]

Addressed some nits and a problem where sometimes the compiler would spill half way through materialization.

llvm/lib/Target/PowerPC/PPCISelLowering.h

1324

Hm I don't think so, although I implemented it for consistency with XXSPLTIDP (convertToNonDenormSingle). I'll remove this if that is preferred.

llvm/test/CodeGen/PowerPC/constant-pool.ll

369

This is fixed now, thank you for spotting this.

llvm/test/CodeGen/PowerPC/pcrel-linkeropt.ll

147

Just a heads up - the tests in this file are autogenerated, hence some unrelated changes.

[stefanp]

Comments relate to just cleaning up the patch a little.

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

8593

I'm wondering if it would not be better to just inline this. It's just "not" of another call. That would simplify the patch a little.

15874

Isn't this just :

return convertToNonDenormSingle(APFloatOfImm) ||
       !convertToNonDenormSingle(APFloatOfImm);

Which is always true?

Basically the logic is that we can now materialize without a load any f32 or any f64.

llvm/lib/Target/PowerPC/PPCISelLowering.h

1324

nit:
Ok, unless other reviewers disagree, just remove it.

[Conanap]

Addressed Stefan's comments, converted the check to a mirror of the original function for XXSPLTIDP except non-destructive.

[Conanap]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

8593

so in the process of removing this, I thought I might as well just write a non-destructive test in its place. The tests didn't have any problems with just checkConvertToNonDenormSingle but might as well be on the safer side of things.

[amyk]

llvm/lib/Target/PowerPC/PPCInstrPrefix.td

1881

I think it might be good to add a comment of why the XXSPLTI32DX instruction needs to be split out like this from the other instructions.

llvm/test/CodeGen/PowerPC/pcrel-linkeropt.ll

41

nit: Remove duplicate comment.

60

nit: Remove duplicate comment.

[Conanap]

Updated some comments.

[amyk]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

15873–15874

Minor nit on comment.

[stefanp]

Thank you for adding this!
Other than one minor nit I think this LGTM.

Feel free to address nits on commits.

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

8594

nit:
You can just inline this. It is only used in one place.

llvm/lib/Target/PowerPC/PPCISelLowering.h

 
   bool isIntS16Immediate(SDNode *N, int16_t &Imm);
   bool isIntS16Immediate(SDValue Op, int16_t &Imm);
   bool isIntS34Immediate(SDNode *N, int64_t &Imm);
   bool isIntS34Immediate(SDValue Op, int64_t &Imm);
 
   bool convertToNonDenormSingle(APInt &ArgAPInt);
   bool convertToNonDenormSingle(APFloat &ArgAPFloat);
+  bool checkNonDenormCannotConvertToSingle(APInt &ArgAPInt);
+  bool checkNonDenormCannotConvertToSingle(APFloat &ArgAPFloat);
 
 } // end namespace llvm
 
 #endif // LLVM_TARGET_POWERPC_PPC32ISELLOWERING_H

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

bool llvm::convertToNonDenormSingle(APInt &ArgAPInt) {
   double DpValue = ArgAPInt.bitsToDouble();
   APFloat APFloatDp(DpValue);
   bool Success = convertToNonDenormSingle(APFloatDp);
   if (Success)
     ArgAPInt = APFloatDp.bitcastToAPInt();
   return Success;
 }
 
+// Check that the APFloat cannot be exploited with XXSPLTIDP
+// which can only exploit values that can fit in 32 bits. This
+// allows us to exploit with XXSPLTI32DX instead.
+bool llvm::checkNonDenormCannotConvertToSingle(APFloat &ArgAPFloat) {
+  // Only convert if it loses info, since XXSPLTIDP should
+  // handle the other case.
+  return !convertToNonDenormSingle(ArgAPFloat);
+}
+
+// Check that the APInt cannot be exploited with XXSPLTIDP
+// which can only exploit values that can fit in 32 bits. This
+// allows us to exploit with XXSPLTI32DX instead.
+bool llvm::checkNonDenormCannotConvertToSingle(APInt &ArgAPInt) {
+  double DpValue = ArgAPInt.bitsToDouble();
+  APFloat APFloatDp(DpValue);
+  return checkNonDenormCannotConvertToSingle(APFloatDp);
+}
+
 // If this is a case we can't handle, return null and let the default
 // expansion code take care of it.  If we CAN select this case, and if it
 // selects to a single instruction, return Op.  Otherwise, if we can codegen
 // this case more efficiently than a constant pool load, lower it to the
 // sequence of ops that should be used.
 SDValue PPCTargetLowering::LowerBUILD_VECTOR(SDValue Op,
                                              SelectionDAG &DAG) const {
   SDLoc dl(Op);
default:
     // false. Examples: f16, f80.
     return false;
   case MVT::f32:
   case MVT::f64:
     if (Subtarget.hasPrefixInstrs()) {
       // With prefixed instructions, we can materialize anything that can be
       // represented with a 32-bit immediate, not just positive zero.
       APFloat APFloatOfImm = Imm;
-      return convertToNonDenormSingle(APFloatOfImm);
+      return convertToNonDenormSingle(APFloatOfImm) ||
+             checkNonDenormCannotConvertToSingle(APFloatOfImm);
     }
     LLVM_FALLTHROUGH;
   case MVT::ppcf128:
     return Imm.isPosZero();
   }
 }
 
 // For vector shift operation op, fold

llvm/lib/Target/PowerPC/PPCInstrInfo.cpp

bool PPCInstrInfo::isReallyTriviallyReMaterializable(const MachineInstr &MI,
   case PPC::ADDIStocHA:
   case PPC::ADDIStocHA8:
   case PPC::ADDItocL:
   case PPC::LOAD_STACK_GUARD:
   case PPC::XXLXORz:
   case PPC::XXLXORspz:
   case PPC::XXLXORdpz:
   case PPC::XXLEQVOnes:
+  case PPC::XXSPLTI32DX:
   case PPC::V_SET0B:
   case PPC::V_SET0H:
   case PPC::V_SET0:
   case PPC::V_SETALLONESB:
   case PPC::V_SETALLONESH:
   case PPC::V_SETALLONES:
   case PPC::CRSET:
   case PPC::CRUNSET:

llvm/lib/Target/PowerPC/PPCInstrInfo.td

 // and get a i32 with the resulting bits.
 def getFPAs32BitInt : SDNodeXForm<fpimm, [{
   APFloat APFloatOfN = N->getValueAPF();
   convertToNonDenormSingle(APFloatOfN);
   return CurDAG->getTargetConstant(APFloatOfN.bitcastToAPInt().getZExtValue(),
                                    SDLoc(N), MVT::i32);
 }]>;
 
+// Check if the value can be converted to be single precision immediate, which
+// can be exploited by XXSPLTIDP. Ensure that it cannot be converted to single
+// precision before exploiting with XXSPLTI32DX.
+def nzFPImmAsi64 : PatLeaf<(fpimm), [{
+  APFloat APFloatOfN = N->getValueAPF();
+  return !N->isExactlyValue(+0.0) && checkNonDenormCannotConvertToSingle(APFloatOfN);
+}]>;
+
+// Get the Hi bits of a 64 bit immediate.
+def getFPAs64BitIntHi : SDNodeXForm<fpimm, [{
+  APFloat APFloatOfN = N->getValueAPF();
+  uint32_t Hi = (uint32_t)((APFloatOfN.bitcastToAPInt().getZExtValue() &
+                            0xFFFFFFFF00000000LL) >> 32);
+  return CurDAG->getTargetConstant(Hi, SDLoc(N), MVT::i32);
+}]>;
+
+// Get the Lo bits of a 64 bit immediate.
+def getFPAs64BitIntLo : SDNodeXForm<fpimm, [{
+  APFloat APFloatOfN = N->getValueAPF();
+  uint32_t Lo = (uint32_t)(APFloatOfN.bitcastToAPInt().getZExtValue() &
+                           0xFFFFFFFF);
+  return CurDAG->getTargetConstant(Lo, SDLoc(N), MVT::i32);
+}]>;
+
 def imm34 : PatLeaf<(imm), [{
   return isInt<34>(N->getSExtValue());
 }]>;
 
 def getImmAs64BitInt : SDNodeXForm<imm, [{
   return getI64Imm(N->getSExtValue(), SDLoc(N));
 }]>;
 

llvm/lib/Target/PowerPC/PPCInstrPrefix.td

def XXSPLTIW : 8RR_DForm_IMM32_XT6<32, 3, (outs vsrc:$XT),
                                      (ins i32imm:$IMM32),
                                      "xxspltiw $XT, $IMM32", IIC_VecGeneral,
                                      []>;
   def XXSPLTIDP : 8RR_DForm_IMM32_XT6<32, 2, (outs vsrc:$XT),
                                       (ins i32imm:$IMM32),
                                       "xxspltidp $XT, $IMM32", IIC_VecGeneral,
                                       [(set v2f64:$XT,
                                             (PPCxxspltidp i32:$IMM32))]>;
-  def XXSPLTI32DX :
-      8RR_DForm_IMM32_XT6_IX<32, 0, (outs vsrc:$XT),
-                             (ins vsrc:$XTi, u1imm:$IX, i32imm:$IMM32),
-                             "xxsplti32dx $XT, $IX, $IMM32", IIC_VecGeneral,
-                             [(set v2i64:$XT,
-                                   (PPCxxsplti32dx v2i64:$XTi, i32:$IX,
-                                                   i32:$IMM32))]>,
-                             RegConstraint<"$XTi = $XT">, NoEncode<"$XTi">;
   def XXPERMX :
     8RR_XX4Form_IMM3_XTABC6<34, 0, (outs vsrc:$XT), (ins vsrc:$XA, vsrc:$XB,
                             vsrc:$XC, u3imm:$UIM),
                             "xxpermx $XT, $XA, $XB, $XC, $UIM",
                             IIC_VecPerm, []>;
   def XXBLENDVB :
     8RR_XX4Form_XTABC6<33, 0, (outs vsrc:$XT), (ins vsrc:$XA, vsrc:$XB,
                        vsrc:$XC), "xxblendvb $XT, $XA, $XB, $XC",
                        IIC_VecGeneral, []>;
   def XXBLENDVH :
     8RR_XX4Form_XTABC6<33, 1, (outs vsrc:$XT), (ins vsrc:$XA, vsrc:$XB,
                        vsrc:$XC), "xxblendvh $XT, $XA, $XB, $XC",
                        IIC_VecGeneral, []>;
   def XXBLENDVW :
     8RR_XX4Form_XTABC6<33, 2, (outs vsrc:$XT), (ins vsrc:$XA, vsrc:$XB,
                        vsrc:$XC), "xxblendvw $XT, $XA, $XB, $XC",
                        IIC_VecGeneral, []>;
   def XXBLENDVD :
     8RR_XX4Form_XTABC6<33, 3, (outs vsrc:$XT), (ins vsrc:$XA, vsrc:$XB,
                        vsrc:$XC), "xxblendvd $XT, $XA, $XB, $XC",
                        IIC_VecGeneral, []>;
 }
 
+let isReMaterializable = 1, isMoveImm = 1, Predicates = [PrefixInstrs] in {
+  def XXSPLTI32DX :
+      8RR_DForm_IMM32_XT6_IX<32, 0, (outs vsrc:$XT),
+                             (ins vsrc:$XTi, u1imm:$IX, i32imm:$IMM32),
+                             "xxsplti32dx $XT, $IX, $IMM32", IIC_VecGeneral,
+                             [(set v2i64:$XT,
+                                   (PPCxxsplti32dx v2i64:$XTi, i32:$IX,
+                                                   i32:$IMM32))]>,
+                             RegConstraint<"$XTi = $XT">, NoEncode<"$XTi">;
+}
+
 let Predicates = [IsISA3_1] in {
   def SETBC : XForm_XT5_BI5<31, 384, (outs gprc:$RT), (ins crbitrc:$BI),
                             "setbc $RT, $BI", IIC_IntCompare, []>;
   def SETBCR : XForm_XT5_BI5<31, 416, (outs gprc:$RT), (ins crbitrc:$BI),
                              "setbcr $RT, $BI", IIC_IntCompare, []>;
   def SETNBC : XForm_XT5_BI5<31, 448, (outs gprc:$RT), (ins crbitrc:$BI),
                              "setnbc $RT, $BI", IIC_IntCompare, []>;
   def SETNBCR : XForm_XT5_BI5<31, 480, (outs gprc:$RT), (ins crbitrc:$BI),
def : Pat<(v4i32 (build_vector i32immNonAllOneNonZero:$A,
            (v4i32 (XXSPLTIW imm:$A))>;
  def : Pat<(f32 nzFPImmAsi32:$A),
            (COPY_TO_REGCLASS (XXSPLTIDP (getFPAs32BitInt fpimm:$A)),
                              VSFRC)>;
  def : Pat<(f64 nzFPImmAsi32:$A),
            (COPY_TO_REGCLASS (XXSPLTIDP (getFPAs32BitInt fpimm:$A)),
                              VSFRC)>;
 
+// To replace constant pool with XXSPLTI32DX for scalars.
+def : Pat<(f32 nzFPImmAsi64:$A),
+          (COPY_TO_REGCLASS (XXSPLTI32DX (XXSPLTI32DX(IMPLICIT_DEF), 0,
+                                        (getFPAs64BitIntHi $A)),
+                                        1, (getFPAs64BitIntLo $A)),
+                            VSRC)>;
+
+def : Pat<(f64 nzFPImmAsi64:$A),
+          (COPY_TO_REGCLASS (XXSPLTI32DX (XXSPLTI32DX (IMPLICIT_DEF), 0,
+                                        (getFPAs64BitIntHi $A)),
+                                        1, (getFPAs64BitIntLo $A)),
+                            VSRC)>;
+
   // Anonymous patterns for XXEVAL
   // AND
   // and(A, B, C)
   def : xxevalPattern<(and v4i32:$vA, (and v4i32:$vB, v4i32:$vC)), 1>;
   // and(A, xor(B, C))
   def : xxevalPattern<(and v4i32:$vA, (xor v4i32:$vB, v4i32:$vC)), 6>;
   // and(A, or(B, C))
   def : xxevalPattern<(and v4i32:$vA, (or v4i32:$vB, v4i32:$vC)), 7>;

llvm/test/CodeGen/PowerPC/constant-pool.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -verify-machineinstrs -mtriple=powerpc64le-- \
 ; RUN:   -mcpu=pwr10 -ppc-asm-full-reg-names < %s | FileCheck %s
 ; RUN: llc -verify-machineinstrs -target-abi=elfv2 -mtriple=powerpc64-- \
 ; RUN:   -mcpu=pwr10 -ppc-asm-full-reg-names < %s | FileCheck %s
 ; RUN: llc -verify-machineinstrs -target-abi=elfv2 -mtriple=powerpc64-- \
 ; RUN:   -mcpu=pwr9 -ppc-asm-full-reg-names < %s | FileCheck %s --check-prefix=CHECK-P9
 
  define float @FloatConstantPool() {
 ; CHECK-LABEL: FloatConstantPool:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    plfs f1, .LCPI0_0@PCREL(0), 1
+; CHECK-NEXT:    xxsplti32dx vs1, 0, 0
+; CHECK-NEXT:    xxsplti32dx vs1, 1, 8388577
+; CHECK-NEXT:    # kill: def $f1 killed $f1 killed $vsl1
 ; CHECK-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: FloatConstantPool:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    addis r3, r2, .LCPI0_0@toc@ha
 ; CHECK-P9-NEXT:    lfs f1, .LCPI0_0@toc@l(r3)
 ; CHECK-P9-NEXT:    blr
 entry:
   ret float 0x380FFFF840000000
 }
 
  define double @DoubleConstantPool() {
 ; CHECK-LABEL: DoubleConstantPool:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    plfd f1, .LCPI1_0@PCREL(0), 1
+; CHECK-NEXT:    xxsplti32dx vs1, 0, 1048574
+; CHECK-NEXT:    xxsplti32dx vs1, 1, 780229072
+; CHECK-NEXT:    # kill: def $f1 killed $f1 killed $vsl1
 ; CHECK-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: DoubleConstantPool:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    addis r3, r2, .LCPI1_0@toc@ha
 ; CHECK-P9-NEXT:    lfd f1, .LCPI1_0@toc@l(r3)
 ; CHECK-P9-NEXT:    blr
 entry:
   ret double 2.225070e-308
 }
 
  define ppc_fp128 @LongDoubleConstantPool() {
 ; CHECK-LABEL: LongDoubleConstantPool:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    plfd f1, .LCPI2_0@PCREL(0), 1
-; CHECK-NEXT:    plfd f2, .LCPI2_1@PCREL(0), 1
+; CHECK-NEXT:    xxsplti32dx vs1, 0, 56623104
+; CHECK-NEXT:    xxsplti32dx vs2, 0, -2146625897
+; CHECK-NEXT:    xxsplti32dx vs1, 1, -609716532
+; CHECK-NEXT:    xxsplti32dx vs2, 1, 1339675259
+; CHECK-NEXT:    # kill: def $f1 killed $f1 killed $vsl1
+; CHECK-NEXT:    # kill: def $f2 killed $f2 killed $vsl2
 ; CHECK-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: LongDoubleConstantPool:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    addis r3, r2, .LCPI2_0@toc@ha
 ; CHECK-P9-NEXT:    lfd f1, .LCPI2_0@toc@l(r3)
 ; CHECK-P9-NEXT:    addis r3, r2, .LCPI2_1@toc@ha
 ; CHECK-P9-NEXT:    lfd f2, .LCPI2_1@toc@l(r3)

 ; CHECK-P9-NEXT:    blr
 entry:
   ret <2 x double> <double 2.225070e-308, double 2.225000e-308>
 }
 
 define double @two_constants(double %a) {
 ; CHECK-LABEL: two_constants:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    plfd f0, .LCPI11_0@PCREL(0), 1
+; CHECK-NEXT:    xxsplti32dx vs0, 0, 1074446467
+; CHECK-NEXT:    xxsplti32dx vs0, 1, 309237645
 ; CHECK-NEXT:    xsadddp f0, f1, f0
-; CHECK-NEXT:    plfd f1, .LCPI11_1@PCREL(0), 1
+; CHECK-NEXT:    xxsplti32dx vs1, 0, 1073922179
+; CHECK-NEXT:    xxsplti32dx vs1, 1, 309237645
 ; CHECK-NEXT:    xsadddp f1, f0, f1
 ; CHECK-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: two_constants:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    addis r3, r2, .LCPI11_0@toc@ha
 ; CHECK-P9-NEXT:    lfd f0, .LCPI11_0@toc@l(r3)
 ; CHECK-P9-NEXT:    addis r3, r2, .LCPI11_1@toc@ha
 ; CHECK-P9-NEXT:    xsadddp f0, f1, f0
 ; CHECK-P9-NEXT:    lfd f1, .LCPI11_1@toc@l(r3)
 ; CHECK-P9-NEXT:    xsadddp f1, f0, f1
 ; CHECK-P9-NEXT:    blr
 entry:
   %0 = fadd double %a, 3.344000e+00
   %1 = fadd double %0, 2.344000e+00
   ret double %1
 }
 
 define double @two_constants_two_bb(i32 %m, double %a) {
 ; CHECK-LABEL: two_constants_two_bb:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    cmplwi r3, 0
 ; CHECK-NEXT:    beq cr0, .LBB12_2
 ; CHECK-NEXT:  # %bb.1:
-; CHECK-NEXT:    plfd f1, .LCPI12_0@PCREL(0), 1
+; CHECK-NEXT:    xxsplti32dx vs1, 0, 1074935889
+; CHECK-NEXT:    xxsplti32dx vs1, 1, -343597384
+; CHECK-NEXT:    # kill: def $f1 killed $f1 killed $vsl1
 ; CHECK-NEXT:    blr
 ; CHECK-NEXT:  .LBB12_2: # %if.end
-; CHECK-NEXT:    plfd f0, .LCPI12_1@PCREL(0), 1
+; CHECK-NEXT:    xxsplti32dx vs0, 0, 1076085391
+; CHECK-NEXT:    xxsplti32dx vs0, 1, 1546188227
 ; CHECK-NEXT:    xsadddp f1, f1, f0
+; CHECK-NEXT:    # kill: def $f1 killed $f1 killed $vsl1
 ; CHECK-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: two_constants_two_bb:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    cmplwi r3, 0
 ; CHECK-P9-NEXT:    beq cr0, .LBB12_2
 ; CHECK-P9-NEXT:  # %bb.1:
 ; CHECK-P9-NEXT:    addis r3, r2, .LCPI12_0@toc@ha

 return:
   %retval.0 = phi double [ %add, %if.end ], [ 4.555000e+00, %entry ]
   ret double %retval.0
 }
 
 define double @three_constants_f64(double %a, double %c) {
 ; CHECK-LABEL: three_constants_f64:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    plfd f0, .LCPI13_0@PCREL(0), 1
+; CHECK-NEXT:    xxsplti32dx vs0, 0, 1074446467
+; CHECK-NEXT:    xxsplti32dx vs0, 1, 309237645
 ; CHECK-NEXT:    xsadddp f0, f1, f0
-; CHECK-NEXT:    plfd f1, .LCPI13_1@PCREL(0), 1
+; CHECK-NEXT:    xxsplti32dx vs1, 0, 1073922179
+; CHECK-NEXT:    xxsplti32dx vs1, 1, 309237645
 ; CHECK-NEXT:    xsadddp f0, f0, f1
-; CHECK-NEXT:    plfd f1, .LCPI13_2@PCREL(0), 1
+; CHECK-NEXT:    xxsplti32dx vs1, 0, 1073948393
+; CHECK-NEXT:    xxsplti32dx vs1, 1, 2027224564
 ; CHECK-NEXT:    xsadddp f1, f0, f1
 ; CHECK-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: three_constants_f64:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    addis r3, r2, .LCPI13_0@toc@ha
 ; CHECK-P9-NEXT:    lfd f0, .LCPI13_0@toc@l(r3)
 ; CHECK-P9-NEXT:    addis r3, r2, .LCPI13_1@toc@ha
entry:
   %0 = fadd fp128 %a, 0xL8000000000000000400123851EB851EB
   %1 = fadd fp128 %0, 0xL8000000000000000400123851EB991EB
   %2 = fadd fp128 %1, 0xL8000000000000000400123851EB771EB
   ret fp128 %2
 }
 
 define ppc_fp128 @three_constants_ppcf128(ppc_fp128 %a, ppc_fp128 %c) {
 ; CHECK-LABEL: three_constants_ppcf128:
-; CHECK:         .localentry three_constants_ppcf128, 1
-; CHECK-NEXT:  # %bb.0: # %entry
+; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    mflr r0
 ; CHECK-NEXT:    std r0, 16(r1)
 ; CHECK-NEXT:    stdu r1, -32(r1)
 ; CHECK-NEXT:    .cfi_def_cfa_offset 32
 ; CHECK-NEXT:    .cfi_offset lr, 16
-; CHECK-NEXT:    plfd f3, .LCPI16_0@PCREL(0), 1
 ; CHECK-NEXT:    xxlxor f4, f4, f4
+; CHECK-NEXT:    xxsplti32dx vs3, 0, 1074935889
+; CHECK-NEXT:    xxsplti32dx vs3, 1, -343597384
+; CHECK-NEXT:    # kill: def $f3 killed $f3 killed $vsl3
 ; CHECK-NEXT:    bl __gcc_qadd@notoc
-; CHECK-NEXT:    plfd f3, .LCPI16_1@PCREL(0), 1
 ; CHECK-NEXT:    xxlxor f4, f4, f4
+; CHECK-NEXT:    xxsplti32dx vs3, 0, 1074935889
+; CHECK-NEXT:    xxsplti32dx vs3, 1, -1719329096
+; CHECK-NEXT:    # kill: def $f3 killed $f3 killed $vsl3
 ; CHECK-NEXT:    bl __gcc_qadd@notoc
-; CHECK-NEXT:    plfd f3, .LCPI16_2@PCREL(0), 1
 ; CHECK-NEXT:    xxlxor f4, f4, f4
+; CHECK-NEXT:    xxsplti32dx vs3, 0, 1074935889
+; CHECK-NEXT:    xxsplti32dx vs3, 1, 8724152
+; CHECK-NEXT:    # kill: def $f3 killed $f3 killed $vsl3
 ; CHECK-NEXT:    bl __gcc_qadd@notoc
 ; CHECK-NEXT:    addi r1, r1, 32
 ; CHECK-NEXT:    ld r0, 16(r1)
 ; CHECK-NEXT:    mtlr r0
 ; CHECK-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: three_constants_ppcf128:
 ; CHECK-P9:       # %bb.0: # %entry

llvm/test/CodeGen/PowerPC/p10-splatImm-CPload-pcrel.ll

 ; CHECK-BE-NEXT:    blr
 entry:
   ret <2 x double> <double 0xFFFFFFFFFFFFFFF0, double 0xFFFFFFFFFFFFFFF0>
 }
 
 define dso_local double @testDoubleNonRepresentableScalar() local_unnamed_addr {
 ; CHECK-LE-LABEL: testDoubleNonRepresentableScalar:
 ; CHECK-LE:       # %bb.0: # %entry
-; CHECK-LE-NEXT:    plfd f1, .LCPI3_0@PCREL(0), 1
+; CHECK-LE-NEXT:    xxsplti32dx vs1, 0, 1081435463
+; CHECK-LE-NEXT:    xxsplti32dx vs1, 1, -1374389535
+; CHECK-LE-NEXT:    # kill: def $f1 killed $f1 killed $vsl1
 ; CHECK-LE-NEXT:    blr
 ;
 ; CHECK-NOPCREL-BE-LABEL: testDoubleNonRepresentableScalar:
 ; CHECK-NOPCREL-BE:       # %bb.0: # %entry
-; CHECK-NOPCREL-BE-NEXT:    addis r3, r2, .LCPI3_0@toc@ha
-; CHECK-NOPCREL-BE-NEXT:    lfd f1, .LCPI3_0@toc@l(r3)
+; CHECK-NOPCREL-BE-NEXT:    xxsplti32dx vs1, 0, 1081435463
+; CHECK-NOPCREL-BE-NEXT:    xxsplti32dx vs1, 1, -1374389535
+; CHECK-NOPCREL-BE-NEXT:    # kill: def $f1 killed $f1 killed $vsl1
 ; CHECK-NOPCREL-BE-NEXT:    blr
 ;
 ; CHECK-NOPCREL-LE-LABEL: testDoubleNonRepresentableScalar:
 ; CHECK-NOPCREL-LE:       # %bb.0: # %entry
-; CHECK-NOPCREL-LE-NEXT:    addis r3, r2, .LCPI3_0@toc@ha
-; CHECK-NOPCREL-LE-NEXT:    lfd f1, .LCPI3_0@toc@l(r3)
+; CHECK-NOPCREL-LE-NEXT:    xxsplti32dx vs1, 0, 1081435463
+; CHECK-NOPCREL-LE-NEXT:    xxsplti32dx vs1, 1, -1374389535
+; CHECK-NOPCREL-LE-NEXT:    # kill: def $f1 killed $f1 killed $vsl1
 ; CHECK-NOPCREL-LE-NEXT:    blr
 ;
 ; CHECK-NOPREFIX-LABEL: testDoubleNonRepresentableScalar:
 ; CHECK-NOPREFIX:       # %bb.0: # %entry
 ; CHECK-NOPREFIX-NEXT:    addis r3, r2, .LCPI3_0@toc@ha
 ; CHECK-NOPREFIX-NEXT:    lfd f1, .LCPI3_0@toc@l(r3)
 ; CHECK-NOPREFIX-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: testDoubleNonRepresentableScalar:
 ; CHECK-BE:       # %bb.0: # %entry
-; CHECK-BE-NEXT:    plfd f1, .LCPI3_0@PCREL(0), 1
+; CHECK-BE-NEXT:    xxsplti32dx vs1, 0, 1081435463
+; CHECK-BE-NEXT:    xxsplti32dx vs1, 1, -1374389535
+; CHECK-BE-NEXT:    # kill: def $f1 killed $f1 killed $vsl1
 ; CHECK-BE-NEXT:    blr
 entry:
   ret double 3.423300e+02
 }
 
 define dso_local float @testFloatDenormScalar() local_unnamed_addr {
 ; CHECK-LE-LABEL: testFloatDenormScalar:
 ; CHECK-LE:       # %bb.0: # %entry

 ; CHECK-BE-NEXT:    blr
 entry:
   ret float 0x380B38FB80000000
 }
 
 define dso_local double @testFloatDenormToDoubleScalar() local_unnamed_addr {
 ; CHECK-LE-LABEL: testFloatDenormToDoubleScalar:
 ; CHECK-LE:       # %bb.0: # %entry
-; CHECK-LE-NEXT:    plfs f1, .LCPI5_0@PCREL(0), 1
+; CHECK-LE-NEXT:    xxsplti32dx vs1, 0, 940259579
+; CHECK-LE-NEXT:    xxsplti32dx vs1, 1, -2147483648
+; CHECK-LE-NEXT:    # kill: def $f1 killed $f1 killed $vsl1
 ; CHECK-LE-NEXT:    blr
 ;
 ; CHECK-NOPCREL-BE-LABEL: testFloatDenormToDoubleScalar:
 ; CHECK-NOPCREL-BE:       # %bb.0: # %entry
-; CHECK-NOPCREL-BE-NEXT:    addis r3, r2, .LCPI5_0@toc@ha
-; CHECK-NOPCREL-BE-NEXT:    lfs f1, .LCPI5_0@toc@l(r3)
+; CHECK-NOPCREL-BE-NEXT:    xxsplti32dx vs1, 0, 940259579
+; CHECK-NOPCREL-BE-NEXT:    xxsplti32dx vs1, 1, -2147483648
+; CHECK-NOPCREL-BE-NEXT:    # kill: def $f1 killed $f1 killed $vsl1
 ; CHECK-NOPCREL-BE-NEXT:    blr
 ;
 ; CHECK-NOPCREL-LE-LABEL: testFloatDenormToDoubleScalar:
 ; CHECK-NOPCREL-LE:       # %bb.0: # %entry
-; CHECK-NOPCREL-LE-NEXT:    addis r3, r2, .LCPI5_0@toc@ha
-; CHECK-NOPCREL-LE-NEXT:    lfs f1, .LCPI5_0@toc@l(r3)
+; CHECK-NOPCREL-LE-NEXT:    xxsplti32dx vs1, 0, 940259579
+; CHECK-NOPCREL-LE-NEXT:    xxsplti32dx vs1, 1, -2147483648
+; CHECK-NOPCREL-LE-NEXT:    # kill: def $f1 killed $f1 killed $vsl1
 ; CHECK-NOPCREL-LE-NEXT:    blr
 ;
 ; CHECK-NOPREFIX-LABEL: testFloatDenormToDoubleScalar:
 ; CHECK-NOPREFIX:       # %bb.0: # %entry
 ; CHECK-NOPREFIX-NEXT:    addis r3, r2, .LCPI5_0@toc@ha
 ; CHECK-NOPREFIX-NEXT:    lfs f1, .LCPI5_0@toc@l(r3)
 ; CHECK-NOPREFIX-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: testFloatDenormToDoubleScalar:
 ; CHECK-BE:       # %bb.0: # %entry
-; CHECK-BE-NEXT:    plfs f1, .LCPI5_0@PCREL(0), 1
+; CHECK-BE-NEXT:    xxsplti32dx vs1, 0, 940259579
+; CHECK-BE-NEXT:    xxsplti32dx vs1, 1, -2147483648
+; CHECK-BE-NEXT:    # kill: def $f1 killed $f1 killed $vsl1
 ; CHECK-BE-NEXT:    blr
 entry:
   ret double 0x380B38FB80000000
 }

llvm/test/CodeGen/PowerPC/pcrel-call-linkage-leaf.ll

 
 
 define dso_local double @UsesX2AsConstPoolTOC() local_unnamed_addr {
 ; CHECK-ALL-LABEL: UsesX2AsConstPoolTOC:
 ; CHECK-LARGE:     ld r2, .Lfunc_toc7-.Lfunc_gep7(r12)
 ; CHECK-LARGE:     add r2, r2, r12
 ; CHECK-S-NOT:       .localentry
 ; CHECK-ALL:       # %bb.0: # %entry
-; CHECK-S-NEXT:    plfd f1, .LCPI7_0@PCREL(0), 1
+; CHECK-S-NEXT:    xxsplti32dx vs1, 0, 1078011044
+; CHECK-S-NEXT:    xxsplti32dx vs1, 1, -337824948
+; CHECK-S-NEXT:    # kill: def $f1 killed $f1 killed $vsl1
 ; CHECK-S-NEXT:    blr
 entry:
   ret double 0x404124A4EBDD334C
 }
 
 

llvm/test/CodeGen/PowerPC/pcrel-linkeropt.ll

 @ArrayIn = external global [10 x i32], align 4
 @ArrayOut = external local_unnamed_addr global [10 x i32], align 4
 @IntPtrIn = external local_unnamed_addr global i32*, align 8
 @IntPtrOut = external local_unnamed_addr global i32*, align 8
 @FuncPtrIn = external local_unnamed_addr global void (...)*, align 8
 @FuncPtrOut = external local_unnamed_addr global void (...)*, align 8
 
 define dso_local void @ReadWrite8() local_unnamed_addr #0 {
+; In this test the stb r3, 0(r4) cannot be optimized because it
+; uses the register r3 and that register is defined by lbz r3, 0(r3)
+; which is defined between the pld and the stb.
 ; CHECK-LABEL: ReadWrite8:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    pld r3, input8@got@pcrel(0), 1
 ; CHECK-NEXT:  .Lpcrel0:
 ; CHECK-NEXT:    pld r4, output8@got@pcrel(0), 1
 ; CHECK-NEXT:    .reloc .Lpcrel0-8,R_PPC64_PCREL_OPT,.-(.Lpcrel0-8)
 ; CHECK-NEXT:    lbz r3, 0(r3)
-; In this test the stb r3, 0(r4) cannot be optimized because it
-; uses the register r3 and that register is defined by lbz r3, 0(r3)
-; which is defined between the pld and the stb.
 ; CHECK-NEXT:    stb r3, 0(r4)
 ; CHECK-NEXT:    blr
 entry:
   %0 = load i8, i8* @input8, align 1
   store i8 %0, i8* @output8, align 1
   ret void
 }
 
 define dso_local void @ReadWrite16() local_unnamed_addr #0 {
+; In this test the sth r3, 0(r4) cannot be optimized because it
+; uses the register r3 and that register is defined by lhz r3, 0(r3)
+; which is defined between the pld and the sth.
 ; CHECK-LABEL: ReadWrite16:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    pld r3, input16@got@pcrel(0), 1
 ; CHECK-NEXT:  .Lpcrel1:
 ; CHECK-NEXT:    pld r4, output16@got@pcrel(0), 1
 ; CHECK-NEXT:    .reloc .Lpcrel1-8,R_PPC64_PCREL_OPT,.-(.Lpcrel1-8)
 ; CHECK-NEXT:    lhz r3, 0(r3)
-; In this test the sth r3, 0(r4) cannot be optimized because it
-; uses the register r3 and that register is defined by lhz r3, 0(r3)
-; which is defined between the pld and the sth.
 ; CHECK-NEXT:    sth r3, 0(r4)
 ; CHECK-NEXT:    blr
 entry:
   %0 = load i16, i16* @input16, align 2
   store i16 %0, i16* @output16, align 2
   ret void
 }
 
entry:
   ret void
 }
 
 define dso_local void @ReadWritef64() local_unnamed_addr #0 {
 ; CHECK-LABEL: ReadWritef64:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    pld r3, inputf64@got@pcrel(0), 1
 ; CHECK-NEXT:  .Lpcrel5:
-; CHECK-NEXT:    plfd f1, .LCPI6_0@PCREL(0), 1
+; CHECK-NEXT:    xxsplti32dx vs1, 0, 1075524403
+; CHECK-NEXT:    xxsplti32dx vs1, 1, 858993459
 ; CHECK-NEXT:    .reloc .Lpcrel5-8,R_PPC64_PCREL_OPT,.-(.Lpcrel5-8)
 ; CHECK-NEXT:    lfd f0, 0(r3)
 ; CHECK-NEXT:    pld r3, outputf64@got@pcrel(0), 1
 ; CHECK-NEXT:    xsadddp f0, f0, f1
 ; CHECK-NEXT:    stfd f0, 0(r3)
 ; CHECK-NEXT:    blr
 entry:
   %0 = load double, double* @inputf64, align 8
entry:
   store void (...)* @Callee, void (...)** @FuncPtrOut, align 8
   ret void
 }
 
 declare void @Callee(...)
 
 define dso_local void @FuncPtrCall() local_unnamed_addr #0 {
 ; CHECK-LABEL: FuncPtrCall:
-; CHECK:         .localentry FuncPtrCall, 1
-; CHECK-NEXT:  # %bb.0: # %entry
+; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    pld r3, FuncPtrIn@got@pcrel(0), 1
 ; CHECK-NEXT:  .Lpcrel10:
 ; CHECK-NEXT:    .reloc .Lpcrel10-8,R_PPC64_PCREL_OPT,.-(.Lpcrel10-8)
 ; CHECK-NEXT:    ld r12, 0(r3)
 ; CHECK-NEXT:    mtctr r12
 ; CHECK-NEXT:    bctr
 ; CHECK-NEXT:    #TC_RETURNr8 ctr 0
 entry:

 entry:
   %0 = load <4 x i32>, <4 x i32>* @inputVi32, align 16
   %vecext = extractelement <4 x i32> %0, i32 1
   ret i32 %vecext
 }
 
 define dso_local signext i32 @VecMultiUse() local_unnamed_addr #0 {
 ; CHECK-LABEL: VecMultiUse:
-; CHECK:         .localentry VecMultiUse, 1
-; CHECK-NEXT:  # %bb.0: # %entry
+; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    mflr r0
 ; CHECK-NEXT:    std r29, -24(r1) # 8-byte Folded Spill
 ; CHECK-NEXT:    std r30, -16(r1) # 8-byte Folded Spill
 ; CHECK-NEXT:    std r0, 16(r1)
 ; CHECK-NEXT:    stdu r1, -64(r1)
 ; CHECK-NEXT:    pld r30, inputVi32@got@pcrel(0), 1
 ; CHECK-NEXT:    lwz r29, 4(r30)
 ; CHECK-NEXT:    bl Callee@notoc
entry:
   %5 = load <4 x i32>, <4 x i32>* @inputVi32, align 16
   %vecext2 = extractelement <4 x i32> %5, i32 0
   %add3 = add nsw i32 %4, %vecext2
   ret i32 %add3
 }
 
 define dso_local signext i32 @UseAddr(i32 signext %a) local_unnamed_addr #0 {
 ; CHECK-LABEL: UseAddr:
-; CHECK:         .localentry UseAddr, 1
-; CHECK-NEXT:  # %bb.0: # %entry
+; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    mflr r0
 ; CHECK-NEXT:    std r30, -16(r1) # 8-byte Folded Spill
 ; CHECK-NEXT:    std r0, 16(r1)
 ; CHECK-NEXT:    stdu r1, -48(r1)
 ; CHECK-NEXT:    pld r4, ArrayIn@got@pcrel(0), 1
 ; CHECK-NEXT:    lwz r5, 16(r4)
 ; CHECK-NEXT:    add r30, r5, r3
 ; CHECK-NEXT:    mr r3, r4

llvm/test/CodeGen/PowerPC/pcrel.ll

 ; RUN: llc -verify-machineinstrs -mtriple=powerpc64le-unknown-linux-gnu \
 ; RUN:   -mcpu=pwr10 -ppc-asm-full-reg-names -ppc-vsr-nums-as-vr < %s | \
 ; RUN:   FileCheck %s --check-prefix=CHECK-S
 ; RUN: llc -verify-machineinstrs -target-abi=elfv2 -mtriple=powerpc64-- \
 ; RUN:   -mcpu=pwr10 -ppc-asm-full-reg-names -ppc-vsr-nums-as-vr \
 ; RUN:   --filetype=obj < %s | \
 ; RUN:   llvm-objdump --mcpu=pwr10 -dr - | FileCheck %s --check-prefix=CHECK-O
 
 ; Constant Pool Index.
 ; CHECK-S-LABEL: ConstPool
-; CHECK-S:       plfd f1, .LCPI0_0@PCREL(0), 1
+; CHECK-S:       xxsplti32dx vs1, 0, 1081002676
+; CHECK-S-NEXT:       xxsplti32dx vs1, 1, 962072674
 ; CHECK-S:       blr
 
 ; CHECK-O-LABEL: ConstPool
-; CHECK-O:       plfd 1, 0(0), 1
-; CHECK-O-NEXT:  R_PPC64_PCREL34  .rodata.cst8
-; CHECK-O:       blr
+; CHECK-O:       xxsplti32dx 1, 0, 1081002676
+; CHECK-O-NEXT:  xxsplti32dx 1, 1, 962072674
+; CHECK-O-NEXT:  blr
 define dso_local double @ConstPool() local_unnamed_addr {
   entry:
     ret double 0x406ECAB439581062
 }
 
 @valIntLoc = common dso_local local_unnamed_addr global i32 0, align 4
 define dso_local signext i32 @ReadLocalVarInt() local_unnamed_addr  {
 ; CHECK-S-LABEL: ReadLocalVarInt