[PowerPC] Custom lowering for funnel shifts

[foad]

Summary

The custom lowering saves an instruction over the generic expansion, by
taking advantage of the fact that PowerPC shift instructions are well
defined in the shift-by-bitwidth case.

[foad]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

616–624

This is just guesswork. I'm really not sure which types we should do this for, under what conditions.

llvm/test/CodeGen/PowerPC/funnel-shift.ll

31–43

Can I pre-commit this new test case, and fshr_i64?

llvm/test/CodeGen/PowerPC/pr44183.ll

16–22

This regressions seems to be caused by not constant folding based on the fact that r3 is known to be 4. Can anyone suggest how to fix it? Do I have to spot known constant shift amounts in PPCTargetLowering::LowerFunnelShift?

[spatel]

llvm/test/CodeGen/PowerPC/funnel-shift.ll

31–43

Yes, please pre-commit those, so we see the diffs.

[foad]

Rebase after precommitting new test cases.

[spatel]

llvm/test/CodeGen/PowerPC/pr44183.ll

16–22

This is an unusual test because it contains "i216" types, but that's apparently the way the code was written in:
http://bugs.llvm.org/PR44183

Maybe because of that, the constants are marked opaque from DAG creation time:
Creating constant: t6: i216 = OpaqueConstant<4>
...so default constant folding is bypassed. Someone with more PPC knowledge probably needs to decide what -- if anything -- needs to be done here.

[RKSimon]

Have you triaged whats going on in the mulfixsat test regressions?

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

8998

Maybe also show the pseudocode as used by PPC for comparison?

[foad]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

8998

This is pseudocode for the PPC expansion. Did you mean, show the more complicated expansion that TargetLowering would have used?

llvm/test/CodeGen/PowerPC/umulfixsat.ll

12

I think the extra instruction here is just due to bad luck in the register allocation or scheduling.

29

It's hard to see because of the register allocation and scheduling differences but there is a slight regression here, from:

rotlwi  3, 3, 31
rlwimi 3, 5, 31, 0, 0

to:

srwi 5, 5, 1
slwi 4, 3, 31
or 4, 4, 5

I suppose I have regressed funnel shifts by a constant amount. I'll see if I can fix it.

[RKSimon]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

8998

Doh! Sorry about that, I misread it this morning - no need to add anything here.

[foad]

Enable combining of PPC-specific shift opcodes.

[foad]

In D83948#2161728, @foad wrote:

Enable combining of PPC-specific shift opcodes.

My rationale is that in PPCTargetLowering::LowerFunnelShift it's correct to use PPCISD::SHL and PPCISD::SRL (instead of the ISD:: versions) precisely because they are defined in the shift-by-bitwidth case. But I think it's safe for BitPermutationSelector to treat them the same as the ISD:: versions. @hfinkel @inouehrs does this seem reasonable?

[RKSimon]

This seems reasonable to me @hfinkel @nemanjai any more comments?

[nemanjai]

LGTM.

llvm/test/CodeGen/PowerPC/pr44183.ll

16–22

I agree with Sanjay, this is due to the weird size. However, it is not a concern. I'll post a very small tweak to our Reg+Reg -> Reg+Imm transformation that recovers this.

[nemanjai]

The fix to recover the regression: https://reviews.llvm.org/D84659

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

case Intrinsic::fshr: {
     SDValue X = getValue(I.getArgOperand(0));
     SDValue Y = getValue(I.getArgOperand(1));
     SDValue Z = getValue(I.getArgOperand(2));
     EVT VT = X.getValueType();
     SDValue BitWidthC = DAG.getConstant(VT.getScalarSizeInBits(), sdl, VT);
     SDValue Zero = DAG.getConstant(0, sdl, VT);
     SDValue ShAmt = DAG.getNode(ISD::UREM, sdl, VT, Z, BitWidthC);
 
-    auto FunnelOpcode = IsFSHL ? ISD::FSHL : ISD::FSHR;
-    if (TLI.isOperationLegalOrCustom(FunnelOpcode, VT)) {
-      setValue(&I, DAG.getNode(FunnelOpcode, sdl, VT, X, Y, Z));
-      return;
-    }
-
     // When X == Y, this is rotate. If the data type has a power-of-2 size, we
     // avoid the select that is necessary in the general case to filter out
     // the 0-shift possibility that leads to UB.
     if (X == Y && isPowerOf2_32(VT.getScalarSizeInBits())) {
       auto RotateOpcode = IsFSHL ? ISD::ROTL : ISD::ROTR;
       if (TLI.isOperationLegalOrCustom(RotateOpcode, VT)) {
         setValue(&I, DAG.getNode(RotateOpcode, sdl, VT, X, Z));
         return;
if (X == Y && isPowerOf2_32(VT.getScalarSizeInBits())) {
       SDValue NegZ = DAG.getNode(ISD::SUB, sdl, VT, Zero, Z);
       SDValue NShAmt = DAG.getNode(ISD::UREM, sdl, VT, NegZ, BitWidthC);
       SDValue ShX = DAG.getNode(ISD::SHL, sdl, VT, X, IsFSHL ? ShAmt : NShAmt);
       SDValue ShY = DAG.getNode(ISD::SRL, sdl, VT, X, IsFSHL ? NShAmt : ShAmt);
       setValue(&I, DAG.getNode(ISD::OR, sdl, VT, ShX, ShY));
       return;
     }
 
+    auto FunnelOpcode = IsFSHL ? ISD::FSHL : ISD::FSHR;
+    if (TLI.isOperationLegalOrCustom(FunnelOpcode, VT)) {
+      setValue(&I, DAG.getNode(FunnelOpcode, sdl, VT, X, Y, Z));
+      return;
+    }
+
     // fshl: (X << (Z % BW)) | (Y >> (BW - (Z % BW)))
     // fshr: (X << (BW - (Z % BW))) | (Y >> (Z % BW))
     SDValue InvShAmt = DAG.getNode(ISD::SUB, sdl, VT, BitWidthC, ShAmt);
     SDValue ShX = DAG.getNode(ISD::SHL, sdl, VT, X, IsFSHL ? ShAmt : InvShAmt);
     SDValue ShY = DAG.getNode(ISD::SRL, sdl, VT, Y, IsFSHL ? InvShAmt : ShAmt);
     SDValue Or = DAG.getNode(ISD::OR, sdl, VT, ShX, ShY);
 
     // If (Z % BW == 0), then the opposite direction shift is shift-by-bitwidth,

llvm/lib/Target/PowerPC/PPCISelLowering.h

private:
     SDValue LowerSELECT_CC(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerFP_TO_INT(SDValue Op, SelectionDAG &DAG,
                            const SDLoc &dl) const;
     SDValue LowerINT_TO_FP(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerFLT_ROUNDS_(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerSHL_PARTS(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerSRL_PARTS(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerSRA_PARTS(SDValue Op, SelectionDAG &DAG) const;
+    SDValue LowerFunnelShift(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerBUILD_VECTOR(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerVECTOR_SHUFFLE(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerINSERT_VECTOR_ELT(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerEXTRACT_VECTOR_ELT(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerINTRINSIC_WO_CHAIN(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerINTRINSIC_VOID(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerBSWAP(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerATOMIC_CMP_SWAP(SDValue Op, SelectionDAG &DAG) const;

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

 //===-- PPCISelLowering.cpp - PPC DAG Lowering Implementation -------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file implements the PPCISelLowering class.
if (Subtarget.use64BitRegs()) {
     setOperationAction(ISD::SRL_PARTS, MVT::i64, Custom);
   } else {
     // 32-bit PowerPC wants to expand i64 shifts itself.
     setOperationAction(ISD::SHL_PARTS, MVT::i32, Custom);
     setOperationAction(ISD::SRA_PARTS, MVT::i32, Custom);
     setOperationAction(ISD::SRL_PARTS, MVT::i32, Custom);
   }
 
+  // PowerPC has better expansions for funnel shifts than the generic
+  // TargetLowering::expandFunnelShift.
+  if (Subtarget.has64BitSupport()) {
+    setOperationAction(ISD::FSHL, MVT::i64, Custom);
+    setOperationAction(ISD::FSHR, MVT::i64, Custom);
+  }
+  setOperationAction(ISD::FSHL, MVT::i32, Custom);
+  setOperationAction(ISD::FSHR, MVT::i32, Custom);
+
   if (Subtarget.hasVSX()) {
     setOperationAction(ISD::FMAXNUM_IEEE, MVT::f64, Legal);
     setOperationAction(ISD::FMAXNUM_IEEE, MVT::f32, Legal);
     setOperationAction(ISD::FMINNUM_IEEE, MVT::f64, Legal);
     setOperationAction(ISD::FMINNUM_IEEE, MVT::f32, Legal);
   }
 
   if (Subtarget.hasAltivec()) {
SDValue PPCTargetLowering::LowerSRA_PARTS(SDValue Op, SelectionDAG &DAG) const {
   SDValue Tmp6 = DAG.getNode(PPCISD::SRA, dl, VT, Hi, Tmp5);
   SDValue OutHi = DAG.getNode(PPCISD::SRA, dl, VT, Hi, Amt);
   SDValue OutLo = DAG.getSelectCC(dl, Tmp5, DAG.getConstant(0, dl, AmtVT),
                                   Tmp4, Tmp6, ISD::SETLE);
   SDValue OutOps[] = { OutLo, OutHi };
   return DAG.getMergeValues(OutOps, dl);
 }
 
+SDValue PPCTargetLowering::LowerFunnelShift(SDValue Op,
+                                            SelectionDAG &DAG) const {
+  SDLoc dl(Op);
+  EVT VT = Op.getValueType();
+  unsigned BitWidth = VT.getSizeInBits();
+
+  bool IsFSHL = Op.getOpcode() == ISD::FSHL;
+  SDValue X = Op.getOperand(0);
+  SDValue Y = Op.getOperand(1);
+  SDValue Z = Op.getOperand(2);
+  EVT AmtVT = Z.getValueType();
+
+  // fshl: (X << (Z % BW)) | (Y >> (BW - (Z % BW)))
+  // fshr: (X << (BW - (Z % BW))) | (Y >> (Z % BW))
+  // This is simpler than TargetLowering::expandFunnelShift because we can rely
+  // on PowerPC shift by BW being well defined.
+  Z = DAG.getNode(ISD::AND, dl, AmtVT, Z,
+                  DAG.getConstant(BitWidth - 1, dl, AmtVT));
+  SDValue SubZ =
+      DAG.getNode(ISD::SUB, dl, AmtVT, DAG.getConstant(BitWidth, dl, AmtVT), Z);
+  X = DAG.getNode(PPCISD::SHL, dl, VT, X, IsFSHL ? Z : SubZ);
+  Y = DAG.getNode(PPCISD::SRL, dl, VT, Y, IsFSHL ? SubZ : Z);
+  return DAG.getNode(ISD::OR, dl, VT, X, Y);
+}
+
 //===----------------------------------------------------------------------===//
 // Vector related lowering.
 //
 
 /// getCanonicalConstSplat - Build a canonical splat immediate of Val with an
 /// element size of SplatSize. Cast the result to VT.
 static SDValue getCanonicalConstSplat(uint64_t Val, unsigned SplatSize, EVT VT,
                                       SelectionDAG &DAG, const SDLoc &dl) {
SDValue PPCTargetLowering::LowerOperation(SDValue Op, SelectionDAG &DAG) const {
   case ISD::SINT_TO_FP:         return LowerINT_TO_FP(Op, DAG);
   case ISD::FLT_ROUNDS_:        return LowerFLT_ROUNDS_(Op, DAG);
 
   // Lower 64-bit shifts.
   case ISD::SHL_PARTS:          return LowerSHL_PARTS(Op, DAG);
   case ISD::SRL_PARTS:          return LowerSRL_PARTS(Op, DAG);
   case ISD::SRA_PARTS:          return LowerSRA_PARTS(Op, DAG);
 
+  case ISD::FSHL:               return LowerFunnelShift(Op, DAG);
+  case ISD::FSHR:               return LowerFunnelShift(Op, DAG);
+
   // Vector-related lowering.
   case ISD::BUILD_VECTOR:       return LowerBUILD_VECTOR(Op, DAG);
   case ISD::VECTOR_SHUFFLE:     return LowerVECTOR_SHUFFLE(Op, DAG);
   case ISD::INTRINSIC_WO_CHAIN: return LowerINTRINSIC_WO_CHAIN(Op, DAG);
   case ISD::SCALAR_TO_VECTOR:   return LowerSCALAR_TO_VECTOR(Op, DAG);
   case ISD::EXTRACT_VECTOR_ELT: return LowerEXTRACT_VECTOR_ELT(Op, DAG);
   case ISD::INSERT_VECTOR_ELT:  return LowerINSERT_VECTOR_ELT(Op, DAG);
   case ISD::MUL:                return LowerMUL(Op, DAG);

llvm/test/CodeGen/PowerPC/funnel-shift.ll

 declare i64 @llvm.fshr.i64(i64, i64, i64)
 declare <4 x i32> @llvm.fshr.v4i32(<4 x i32>, <4 x i32>, <4 x i32>)
 
 ; General case - all operands can be variables.
 
 define i32 @fshl_i32(i32 %x, i32 %y, i32 %z) {
 ; CHECK-LABEL: fshl_i32:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    andi. 5, 5, 31
+; CHECK-NEXT:    clrlwi 5, 5, 27
 ; CHECK-NEXT:    subfic 6, 5, 32
-; CHECK-NEXT:    slw 5, 3, 5
+; CHECK-NEXT:    slw 3, 3, 5
 ; CHECK-NEXT:    srw 4, 4, 6
-; CHECK-NEXT:    or 4, 5, 4
-; CHECK-NEXT:    iseleq 3, 3, 4
+; CHECK-NEXT:    or 3, 3, 4
 ; CHECK-NEXT:    blr
   %f = call i32 @llvm.fshl.i32(i32 %x, i32 %y, i32 %z)
   ret i32 %f
 }
 
 define i64 @fshl_i64(i64 %x, i64 %y, i64 %z) {
 ; CHECK-LABEL: fshl_i64:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    andi. 5, 5, 63
+; CHECK-NEXT:    clrlwi 5, 5, 26
 ; CHECK-NEXT:    subfic 6, 5, 64
-; CHECK-NEXT:    sld 5, 3, 5
+; CHECK-NEXT:    sld 3, 3, 5
 ; CHECK-NEXT:    srd 4, 4, 6
-; CHECK-NEXT:    or 4, 5, 4
-; CHECK-NEXT:    iseleq 3, 3, 4
+; CHECK-NEXT:    or 3, 3, 4
 ; CHECK-NEXT:    blr
   %f = call i64 @llvm.fshl.i64(i64 %x, i64 %y, i64 %z)
   ret i64 %f
 }
 
 ; Verify that weird types are minimally supported.
 declare i37 @llvm.fshl.i37(i37, i37, i37)
 define i37 @fshl_i37(i37 %x, i37 %y, i37 %z) {
 ; CHECK-LABEL: fshl_i37:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    lis 6, -8857
 ; CHECK-NEXT:    clrldi 5, 5, 27
 ; CHECK-NEXT:    ori 6, 6, 51366
; CHECK-NEXT:    blr
   ret i7 %f
 }
 
 ; With constant shift amount, this is rotate + insert (missing extended mnemonics).
 
 define i32 @fshl_i32_const_shift(i32 %x, i32 %y) {
 ; CHECK-LABEL: fshl_i32_const_shift:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    rotlwi 4, 4, 9
-; CHECK-NEXT:    rlwimi 4, 3, 9, 0, 22
-; CHECK-NEXT:    mr 3, 4
+; CHECK-NEXT:    srwi 4, 4, 23
+; CHECK-NEXT:    slwi 3, 3, 9
+; CHECK-NEXT:    or 3, 3, 4
 ; CHECK-NEXT:    blr
   %f = call i32 @llvm.fshl.i32(i32 %x, i32 %y, i32 9)
   ret i32 %f
 }
 
 ; Check modulo math on shift amount.
 
 define i32 @fshl_i32_const_overshift(i32 %x, i32 %y) {
 ; CHECK-LABEL: fshl_i32_const_overshift:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    rotlwi 4, 4, 9
-; CHECK-NEXT:    rlwimi 4, 3, 9, 0, 22
-; CHECK-NEXT:    mr 3, 4
+; CHECK-NEXT:    srwi 4, 4, 23
+; CHECK-NEXT:    slwi 3, 3, 9
+; CHECK-NEXT:    or 3, 3, 4
 ; CHECK-NEXT:    blr
   %f = call i32 @llvm.fshl.i32(i32 %x, i32 %y, i32 41)
   ret i32 %f
 }
 
 ; 64-bit should also work.
 
 define i64 @fshl_i64_const_overshift(i64 %x, i64 %y) {
 ; CHECK-LABEL: fshl_i64_const_overshift:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    rotldi 4, 4, 41
-; CHECK-NEXT:    rldimi 4, 3, 41, 0
-; CHECK-NEXT:    mr 3, 4
+; CHECK-NEXT:    rldicl 4, 4, 41, 23
+; CHECK-NEXT:    sldi 3, 3, 41
+; CHECK-NEXT:    or 3, 3, 4
 ; CHECK-NEXT:    blr
   %f = call i64 @llvm.fshl.i64(i64 %x, i64 %y, i64 105)
   ret i64 %f
 }
 
 ; This should work without any node-specific logic.
 
 define i8 @fshl_i8_const_fold() {
 ; CHECK-LABEL: fshl_i8_const_fold:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    li 3, 128
 ; CHECK-NEXT:    blr
   %f = call i8 @llvm.fshl.i8(i8 255, i8 0, i8 7)
   ret i8 %f
 }
 
 ; Repeat everything for funnel shift right.
 
 ; General case - all operands can be variables.
 
 define i32 @fshr_i32(i32 %x, i32 %y, i32 %z) {
 ; CHECK-LABEL: fshr_i32:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    andi. 5, 5, 31
+; CHECK-NEXT:    clrlwi 5, 5, 27
 ; CHECK-NEXT:    subfic 6, 5, 32
-; CHECK-NEXT:    srw 5, 4, 5
+; CHECK-NEXT:    srw 4, 4, 5
 ; CHECK-NEXT:    slw 3, 3, 6
-; CHECK-NEXT:    or 3, 3, 5
-; CHECK-NEXT:    iseleq 3, 4, 3
+; CHECK-NEXT:    or 3, 3, 4
 ; CHECK-NEXT:    blr
   %f = call i32 @llvm.fshr.i32(i32 %x, i32 %y, i32 %z)
   ret i32 %f
 }
 
 define i64 @fshr_i64(i64 %x, i64 %y, i64 %z) {
 ; CHECK-LABEL: fshr_i64:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    andi. 5, 5, 63
+; CHECK-NEXT:    clrlwi 5, 5, 26
 ; CHECK-NEXT:    subfic 6, 5, 64
-; CHECK-NEXT:    srd 5, 4, 5
+; CHECK-NEXT:    srd 4, 4, 5
 ; CHECK-NEXT:    sld 3, 3, 6
-; CHECK-NEXT:    or 3, 3, 5
-; CHECK-NEXT:    iseleq 3, 4, 3
+; CHECK-NEXT:    or 3, 3, 4
 ; CHECK-NEXT:    blr
   %f = call i64 @llvm.fshr.i64(i64 %x, i64 %y, i64 %z)
   ret i64 %f
 }
 
 ; Verify that weird types are minimally supported.
 declare i37 @llvm.fshr.i37(i37, i37, i37)
 define i37 @fshr_i37(i37 %x, i37 %y, i37 %z) {
; CHECK-NEXT:    blr
   ret i7 %f
 }
 
 ; With constant shift amount, this is rotate + insert (missing extended mnemonics).
 
 define i32 @fshr_i32_const_shift(i32 %x, i32 %y) {
 ; CHECK-LABEL: fshr_i32_const_shift:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    rotlwi 4, 4, 23
-; CHECK-NEXT:    rlwimi 4, 3, 23, 0, 8
-; CHECK-NEXT:    mr 3, 4
+; CHECK-NEXT:    srwi 4, 4, 9
+; CHECK-NEXT:    slwi 3, 3, 23
+; CHECK-NEXT:    or 3, 3, 4
 ; CHECK-NEXT:    blr
   %f = call i32 @llvm.fshr.i32(i32 %x, i32 %y, i32 9)
   ret i32 %f
 }
 
 ; Check modulo math on shift amount. 41-32=9.
 
 define i32 @fshr_i32_const_overshift(i32 %x, i32 %y) {
 ; CHECK-LABEL: fshr_i32_const_overshift:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    rotlwi 4, 4, 23
-; CHECK-NEXT:    rlwimi 4, 3, 23, 0, 8
-; CHECK-NEXT:    mr 3, 4
+; CHECK-NEXT:    srwi 4, 4, 9
+; CHECK-NEXT:    slwi 3, 3, 23
+; CHECK-NEXT:    or 3, 3, 4
 ; CHECK-NEXT:    blr
   %f = call i32 @llvm.fshr.i32(i32 %x, i32 %y, i32 41)
   ret i32 %f
 }
 
 ; 64-bit should also work. 105-64 = 41.
 
 define i64 @fshr_i64_const_overshift(i64 %x, i64 %y) {
 ; CHECK-LABEL: fshr_i64_const_overshift:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    rotldi 4, 4, 23
-; CHECK-NEXT:    rldimi 4, 3, 23, 0
-; CHECK-NEXT:    mr 3, 4
+; CHECK-NEXT:    rldicl 4, 4, 23, 41
+; CHECK-NEXT:    sldi 3, 3, 23
+; CHECK-NEXT:    or 3, 3, 4
 ; CHECK-NEXT:    blr
   %f = call i64 @llvm.fshr.i64(i64 %x, i64 %y, i64 105)
   ret i64 %f
 }
 
 ; This should work without any node-specific logic.
 
 define i8 @fshr_i8_const_fold() {

llvm/test/CodeGen/PowerPC/ppcf128-endian.ll

 entry:
   %shl = shl i128 %x, 1
   %0 = bitcast i128 %shl to ppc_fp128
   ret ppc_fp128 %0
 }
 
 ; CHECK-LABEL: convert_to2:
 ; CHECK: std 3, [[OFF1:.*]](1)
-; CHECK: std 5, [[OFF2:.*]](1)
+; CHECK: std 4, [[OFF2:.*]](1)
 ; CHECK: lfd 1, [[OFF1]](1)
 ; CHECK: lfd 2, [[OFF2]](1)
 ; CHECK: blr
 
 define double @convert_vector(<4 x i32> %x) {
 entry:
   %cast = bitcast <4 x i32> %x to ppc_fp128
   %conv = fptrunc ppc_fp128 %cast to double

llvm/test/CodeGen/PowerPC/pr44183.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -verify-machineinstrs -mtriple=powerpc64le-unknown-linux-gnu \
 ; RUN:     -ppc-asm-full-reg-names -mcpu=pwr8 < %s | FileCheck %s
 %struct.m.2.5.8.11 = type { %struct.l.0.3.6.9, [7 x i8], %struct.a.1.4.7.10 }
 %struct.l.0.3.6.9 = type { i8 }
 %struct.a.1.4.7.10 = type { [27 x i8], [0 x i32], [4 x i8] }
 define void @_ZN1m1nEv(%struct.m.2.5.8.11* %this) local_unnamed_addr nounwind align 2 {
 ; CHECK-LABEL: _ZN1m1nEv:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    mflr r0
+; CHECK-NEXT:    std r29, -24(r1) # 8-byte Folded Spill
 ; CHECK-NEXT:    std r30, -16(r1) # 8-byte Folded Spill
 ; CHECK-NEXT:    std r0, 16(r1)
-; CHECK-NEXT:    stdu r1, -48(r1)
+; CHECK-NEXT:    stdu r1, -64(r1)
 ; CHECK-NEXT:    mr r30, r3
-; CHECK-NEXT:    ld r4, 8(r30)
+; CHECK-NEXT:    li r3, 4
+; CHECK-NEXT:    ld r4, 16(r30)
+; CHECK-NEXT:    ld r5, 8(r30)
+; CHECK-NEXT:    subfic r29, r3, 64
+; CHECK-NEXT:    rldicl r3, r5, 60, 4
+; CHECK-NEXT:    sld r4, r4, r29
 ; CHECK-NEXT:    lwz r5, 36(r30)
-; CHECK-NEXT:    rldicl r4, r4, 60, 4
-; CHECK-NEXT:    rlwinm r3, r4, 31, 0, 0
+; CHECK-NEXT:    or r3, r4, r3
+; CHECK-NEXT:    rlwinm r3, r3, 31, 0, 0
 ; CHECK-NEXT:    clrlwi r4, r5, 31
 ; CHECK-NEXT:    or r4, r4, r3
 ; CHECK-NEXT:    bl _ZN1llsE1d
 ; CHECK-NEXT:    nop
 ; CHECK-NEXT:    ld r3, 16(r30)
 ; CHECK-NEXT:    ld r4, 8(r30)
 ; CHECK-NEXT:    rldicl r4, r4, 60, 4
-; CHECK-NEXT:    sldi r3, r3, 60
-; CHECK-NEXT:    or r3, r4, r3
+; CHECK-NEXT:    sld r3, r3, r29
+; CHECK-NEXT:    or r3, r3, r4
 ; CHECK-NEXT:    sldi r3, r3, 31
 ; CHECK-NEXT:    clrldi r4, r3, 32
 ; CHECK-NEXT:    bl _ZN1llsE1d
 ; CHECK-NEXT:    nop
-; CHECK-NEXT:    addi r1, r1, 48
+; CHECK-NEXT:    addi r1, r1, 64
 ; CHECK-NEXT:    ld r0, 16(r1)
 ; CHECK-NEXT:    ld r30, -16(r1) # 8-byte Folded Reload
+; CHECK-NEXT:    ld r29, -24(r1) # 8-byte Folded Reload
 ; CHECK-NEXT:    mtlr r0
 ; CHECK-NEXT:    blr
 entry:
   %bc = getelementptr inbounds %struct.m.2.5.8.11, %struct.m.2.5.8.11* %this, i64 0, i32 2
   %0 = bitcast %struct.a.1.4.7.10* %bc to i216*
   %bf.load = load i216, i216* %0, align 8
   %bf.lshr = lshr i216 %bf.load, 4
   %shl.i23 = shl i216 %bf.lshr, 31

llvm/test/CodeGen/PowerPC/smulfixsat.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=ppc32 | FileCheck %s
 
 declare  i32 @llvm.smul.fix.sat.i32  (i32, i32, i32)
 
 define i32 @func1(i32 %x, i32 %y) nounwind {
 ; CHECK-LABEL: func1:
 ; CHECK:       # %bb.0:
+; CHECK-NEXT:    mullw 6, 3, 4
 ; CHECK-NEXT:    lis 5, 32767
-; CHECK-NEXT:    mulhw. 6, 3, 4
-; CHECK-NEXT:    lis 7, -32768
-; CHECK-NEXT:    mullw 3, 3, 4
+; CHECK-NEXT:    mulhw. 3, 3, 4
+; CHECK-NEXT:    srawi 4, 6, 31
+; CHECK-NEXT:    cmplw 1, 3, 4
+; CHECK-NEXT:    lis 3, -32768
 ; CHECK-NEXT:    ori 4, 5, 65535
-; CHECK-NEXT:    srawi 5, 3, 31
-; CHECK-NEXT:    cmplw 1, 6, 5
-; CHECK-NEXT:    bc 12, 0, .LBB0_1
+; CHECK-NEXT:    ori 5, 6, 0
+; CHECK-NEXT:    bc 12, 0, .LBB0_2
+; CHECK-NEXT:  # %bb.1:
+; CHECK-NEXT:    ori 3, 4, 0
 ; CHECK-NEXT:    b .LBB0_2
-; CHECK-NEXT:  .LBB0_1:
-; CHECK-NEXT:    addi 4, 7, 0
 ; CHECK-NEXT:  .LBB0_2:
-; CHECK-NEXT:    bclr 12, 6, 0
-; CHECK-NEXT:  # %bb.3:
-; CHECK-NEXT:    ori 3, 4, 0
+; CHECK-NEXT:    bc 12, 6, .LBB0_3
+; CHECK-NEXT:    blr
+; CHECK-NEXT:  .LBB0_3:
+; CHECK-NEXT:    addi 3, 5, 0
 ; CHECK-NEXT:    blr
   %tmp = call i32 @llvm.smul.fix.sat.i32(i32 %x, i32 %y, i32 0)
   ret i32 %tmp
 }
 
 define i32 @func2(i32 %x, i32 %y) nounwind {
 ; CHECK-LABEL: func2:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    mulhw. 6, 3, 4
+; CHECK-NEXT:    mullw 6, 3, 4
 ; CHECK-NEXT:    lis 5, 32767
-; CHECK-NEXT:    mullw 3, 3, 4
-; CHECK-NEXT:    rotlwi 3, 3, 31
-; CHECK-NEXT:    ori 4, 5, 65535
-; CHECK-NEXT:    rlwimi 3, 6, 31, 0, 0
+; CHECK-NEXT:    ori 5, 5, 65535
+; CHECK-NEXT:    mulhw. 3, 3, 4
+; CHECK-NEXT:    srwi 4, 6, 1
+; CHECK-NEXT:    slwi 6, 3, 31
+; CHECK-NEXT:    or 4, 6, 4
 ; CHECK-NEXT:    bc 12, 1, .LBB1_1
 ; CHECK-NEXT:    b .LBB1_2
 ; CHECK-NEXT:  .LBB1_1:
-; CHECK-NEXT:    addi 3, 4, 0
+; CHECK-NEXT:    addi 4, 5, 0
 ; CHECK-NEXT:  .LBB1_2:
-; CHECK-NEXT:    cmpwi 6, -1
-; CHECK-NEXT:    lis 4, -32768
-; CHECK-NEXT:    bc 12, 0, .LBB1_3
-; CHECK-NEXT:    blr
-; CHECK-NEXT:  .LBB1_3:
-; CHECK-NEXT:    addi 3, 4, 0
+; CHECK-NEXT:    cmpwi 3, -1
+; CHECK-NEXT:    lis 3, -32768
+; CHECK-NEXT:    bclr 12, 0, 0
+; CHECK-NEXT:  # %bb.3:
+; CHECK-NEXT:    ori 3, 4, 0
 ; CHECK-NEXT:    blr
   %tmp = call i32 @llvm.smul.fix.sat.i32(i32 %x, i32 %y, i32 1)
   ret i32 %tmp
 }

llvm/test/CodeGen/PowerPC/umulfixsat.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=ppc32 | FileCheck %s
 
 declare  i32 @llvm.umul.fix.sat.i32(i32, i32, i32)
 
 define i32 @func1(i32 %x, i32 %y) nounwind {
 ; CHECK-LABEL: func1:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    li 5, -1
 ; CHECK-NEXT:    mulhwu. 6, 3, 4
+; CHECK-NEXT:    li 5, -1
 ; CHECK-NEXT:    mullw 3, 3, 4
+; CHECK-NEXT:    ori 3, 3, 0
 ; CHECK-NEXT:    bclr 12, 2, 0
 ; CHECK-NEXT:  # %bb.1:
 ; CHECK-NEXT:    ori 3, 5, 0
 ; CHECK-NEXT:    blr
   %tmp = call i32 @llvm.umul.fix.sat.i32(i32 %x, i32 %y, i32 0)
   ret i32 %tmp
 }
 
 define i32 @func2(i32 %x, i32 %y) nounwind {
 ; CHECK-LABEL: func2:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    mulhwu 6, 3, 4
+; CHECK-NEXT:    mullw 6, 3, 4
 ; CHECK-NEXT:    li 5, -1
-; CHECK-NEXT:    cmplwi 6, 1
-; CHECK-NEXT:    mullw 3, 3, 4
-; CHECK-NEXT:    rotlwi 3, 3, 31
-; CHECK-NEXT:    rlwimi 3, 6, 31, 0, 0
-; CHECK-NEXT:    bc 12, 1, .LBB1_1
+; CHECK-NEXT:    mulhwu 3, 3, 4
+; CHECK-NEXT:    srwi 4, 6, 1
+; CHECK-NEXT:    slwi 6, 3, 31
+; CHECK-NEXT:    or 4, 6, 4
+; CHECK-NEXT:    cmplwi 3, 1
+; CHECK-NEXT:    bc 12, 1, .LBB1_2
+; CHECK-NEXT:  # %bb.1:
+; CHECK-NEXT:    ori 3, 4, 0
 ; CHECK-NEXT:    blr
-; CHECK-NEXT:  .LBB1_1:
+; CHECK-NEXT:  .LBB1_2:
 ; CHECK-NEXT:    addi 3, 5, 0
 ; CHECK-NEXT:    blr
   %tmp = call i32 @llvm.umul.fix.sat.i32(i32 %x, i32 %y, i32 1)
   ret i32 %tmp
 }